Add UnreferencedAftFibAckUnsupported deviation

[mudhireddy]

Add a new deviation to skip fiback verification for unreferenced NH, NHG aft entries as part of testing

[OpenConfigBot]

Pull Request Functional Test Report for #4197 / acee215

Virtual Devices

Device	Test	Test Documentation	Job	Raw Log
Arista cEOS		TE-15.1: gRIBI Compliance
Cisco 8000E		TE-15.1: gRIBI Compliance
Cisco XRd		TE-15.1: gRIBI Compliance
Juniper ncPTX		TE-15.1: gRIBI Compliance
Nokia SR Linux		TE-15.1: gRIBI Compliance
Openconfig Lemming		TE-15.1: gRIBI Compliance

Hardware Devices

Device	Test	Test Documentation	Raw Log
Arista 7808		TE-15.1: gRIBI Compliance
Cisco 8808		TE-15.1: gRIBI Compliance
Juniper PTX10008		TE-15.1: gRIBI Compliance
Nokia 7250 IXR-10e		TE-15.1: gRIBI Compliance

Help

[coveralls]

Pull Request Test Coverage Report for Build 15087674661

Details

• 0 of 8 (0.0%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.01%) to 18.03%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
internal/deviations/deviations.go	0	3	0.0%
proto/metadata_go_proto/metadata.pb.go	0	5	0.0%

Totals
Change from base Build 15079550658:	-0.01%
Covered Lines:	2478
Relevant Lines:	13744

---

💛 - Coveralls

[mudhireddy]

@nflath, fyi, This one replaces #4189

[mudhireddy]

@nflath , Can you please provide your comments on this pull request

[mudhireddy]

Can someone review this please?

[nflath]

What is the rationale for this?

[mudhireddy]

What is the rationale for this?

Arista's EOS will not install entries into hardware if there are no references to the afts by design and will not send FIB_ACK for those.

NextHopGroup gets installed only when there is an IP aft that uses it.
NextHop gets installed only when there is at least one installed NextHopGroup that uses it.

This test expects to receive FIB_ACK for NH which are referenced by NHG but there is no IP aft that references NHG. So EOS will not support it and hence this deviation.

[mudhireddy]

@nflath , hope that answers your question. let me know if you need any further details and if not can you please approve this request

[gggsmith]

@nflath , hope that answers your question. let me know if you need any further details and if not can you please approve this request

@mudhireddy as the behaviour is inconsistent with gribi specification - it would also be great to know if there are any existing discussions or tracking items (e.g., feature requests or bug reports) regarding Arista's plans for support for this specific part of the specification.

[gggsmith]

Instead of using a deviation for this, we'd prefer a different approach. It's important that we have clear visibility into which tests are affected by this non-compliance, and we can't let a "green" status mask these issues.

With that in mind, please create a separate test for this and move all the failing test cases into it.

Thank you!

[mudhireddy]

Thanks for the review. sure I can create a separate test-suite for failing cases.

Test cases for this feature profile test is coming from a different repo as per README.md file which is https://github.com/openconfig/gribigo/tree/main/compliance

I need to separate the failed cases in that repo first and create a PR for it and let it merge. After that I can work on featureprofiles repo to split compliance test as suggested in the review.

Please let me know if this aligns with your expectation and if there are any pitfalls of this approach.

P.S: Is there any documentation available to use to create new test-suites like TE-15.2 in featureprofiles repo ? Thanks.

[mudhireddy]

created a PR for splitting gribigo test cases: openconfig/gribigo#266

[gggsmith]

Thanks for the review. sure I can create a separate test-suite for failing cases.

Test cases for this feature profile test is coming from a different repo as per README.md file which is https://github.com/openconfig/gribigo/tree/main/compliance

I need to separate the failed cases in that repo first and create a PR for it and let it merge. After that I can work on featureprofiles repo to split compliance test as suggested in the review.

Please let me know if this aligns with your expectation and if there are any pitfalls of this approach.

P.S: Is there any documentation available to use to create new test-suites like TE-15.2 in featureprofiles repo ? Thanks.

Apologies for the late reply. Rather than making the gribigo library aware of different gRIBI support levels, I think we should create a new test (e.g., gribigo_fiback_compliance, TE-15.2) that uses Reference to generate the required test cases.

[mudhireddy]

Rather than making the gribigo library aware of different gRIBI support levels, I think we should create a new test (e.g., gribigo_fiback_compliance, TE-15.2) that uses Reference to generate the required test cases.

No worries and Thank you for your reply. I understood it as, use the Reference field for these non-compliant test cases ("TE-3.1.2", "TE-3.1.3") to skip running them as part of TE-15.1 and add a new test suite TE-15.2, gribigo_fiback_compliance that only runs those skipped in TE-15.1. am I interpreting it correct ?

Also can you please point me to a doc(if it exists) that describes how to add new test suite ?

[mudhireddy]

Rather than making the gribigo library aware of different gRIBI support levels, I think we should create a new test (e.g., gribigo_fiback_compliance, TE-15.2) that uses Reference to generate the required test cases.

No worries and Thank you for your reply. I understood it as, use the Reference field for these non-compliant test cases ("TE-3.1.2", "TE-3.1.3") to skip running them as part of TE-15.1 and add a new test suite TE-15.2, gribigo_fiback_compliance that only runs those skipped in TE-15.1. am I interpreting it correct ?

Also can you please point me to a doc(if it exists) that describes how to add new test suite ?

@gggsmith ?

Dismissing the review for now to unblock the test fixes activity. We can discuss more on the internal bug for changes.