NVIDIA switching: uN and uA on Spectrum-4
S2·E3The slide that says SRv6 replaces our MPLS WAN · Hotel lobby, 07:20 on day three, ninety minutes before the bake-off review
Builds on: FRR: locators, the SID Manager, IS-IS SRv6 and TI-LFA
Before you read: what do you already know?
3 quick questions. Wrong answers are fine and expected; trying first makes the lesson stick.
After this lesson you can
- State the four constraints on Cumulus Linux SRv6 support and cite the page they come from.
- Write the full NVUE locator and static-SID sequence and its vtysh equivalent for a given block, node and function split.
- Analyse why a uA SID needs router advertisements on a non-Spectrum neighbour and predict the failure when they are absent.
- Distinguish what Cumulus Linux does not implement from what the Spectrum silicon cannot do.
Episode 3 — The slide that says SRv6 replaces our MPLS WAN
Two coffees, one already cold, and the deck the Dell account team presents at nine. Slide four promises the Spectrum-4 leaves in the quote will replace the customer’s MPLS WAN with SRv6 L3VPN; the SE wrote it on day one, your name in the owner column of his promise spreadsheet. The architect has already asked, in writing, about Spectrum-3. Procurement asks her only question — does this change the BOM or the lead time — and this time it does.
Both claims need work before nine. Cumulus Linux supports uN and uA — End and End.X with NEXT-CSID, defined in RFC 9800 — and nothing else: no service behaviors, no VPN, no IGP or BGP SID distribution, every SID static.[1][2][5] The scope is Spectrum-4 only, default VRF only, a maximum of 256 locators.[1] The answer you can defend on Spectrum-3 is the software one: Cumulus Linux does not support segment routing there, because nothing you have read says anything about that silicon’s datapath.[1]
This is not an unfinished router. NVIDIA built it for a different job: the NICs connected to the fabric perform SRv6 origination and termination, and the switches act as SRv6-aware nodes executing a path the host encoded.[1] That job needs exactly two primitives — advance the list, or leave by this link — which is what uN and uA are.[5]
“Cumulus Linux does not support it” and “the ASIC cannot do it” are different sentences, and only one is yours to say. Rewrite slide four from what the vendor page actually says.
1What NVIDIA actually built, in its own words
The temptation with a vendor SRv6 page is to read it as a subset of the RFCs and grade it. That misreads the product. NVIDIA states the architecture first: “The NICs connected to the switch fabric perform SRv6 origination and termination, and the switches act as SRv6-aware nodes. SRv6 allows NICs to directly control the path that traffic takes throughout the fabric by encoding an ordered list of SRv6 segment identifiers (uSIDs) in the packet header.”[1] The switch is not a router doing traffic engineering. It is a fabric executing a path a host chose.
From that, the feature set follows. “Cumulus Linux supports uN (End with NEXT-CSID) and uA (End.X with NEXT-CSID) endpoint behaviors, defined in RFC9800.”[1][2] uN advances a compressed segment list — shift the container, forward on the new destination address. uA cross-connects out one named adjacency. Those are the two primitives a host-driven path needs: “keep going along the list” and “leave by this specific link”.[5]
The scope is four sentences long and every one of them is a design constraint: “Cumulus Linux supports segment routing: On the Spectrum-4 switch only. In the default VRF only.”[1][7] “You can configure a maximum of 256 locators.”[1] And every SID is static — there is no IGP or BGP SID distribution on the switch at all.[1]
Note which document NVIDIA anchors on. The page’s own Related Information links are RFC 9800 and the IANA Segment Routing registry, not RFC 8986 alone.[1] That is a deliberate signal: this is a compression-era implementation, and every conversation about it is a uSID conversation.[5]
uA + next-csid
IANA codepoint 52Cumulus on Spectrum-4: supportedA local CSID (from the LIB) bound to End.X: it identifies one adjacency, so "the tuple (Locator-Block, CSID) identifies a different segment on each node". Requires an outgoing interface, and optionally the L3 adjacency address.
RFC 9800 §4.1: NEXT-CSID flavor of End.X. A local CSID from the LIB.
next-csid: RFC 9800 §4.1, Standards Track, June 2025 — and it Updates RFC 8754.
`action End.X flavors next-csid nh6 <addr>`. This combination works — `flavors psp` on End.X does not.
next-csid: `flavors next-csid [lblen N] [nflen N]`, defaults 32 / 16. Supported on End (= uN) and End.X (= uA) only. Lengths must be > 0, ≤ 120 and byte-aligned; `lblen 33` is rejected.
staticd: `behavior uA interface IFNAME [nexthop X:X::X:X]`. "The uA behavior requires the outgoing interface." Added in FRR 10.4.
next-csid: Locator `behavior usid` + `format usid-f3216`; staticd exposes it as the uN / uA / uDT* behavior names.
`nv set router segment-routing static srv6-sid fcbb:fe8::/32 behavior uA` + `interface swp1` + `locator-name LEAF`. ⚠ Needs router advertisements on the peer when the peer is not a Spectrum switch.
next-csid: The only thing Cumulus Linux implements: uN and uA, "defined in RFC9800".
Static-config HLD rev 0.4 (2025-11-12) adds uA: `interface` mandatory, `adj` optional ("if omitted, the next hop is automatically resolved from the interface").
next-csid: Exposed as the uN / uA / uDT* actions in SRV6_MY_SIDS.
No adjacency-bound endpoint primitive.
next-csid: DOCA Flow has no CSID container logic; it copies the whole 128-bit active SID into the DA.
FAE angle: the interop footgun in a mixed Dell/NVIDIA rack — "if the adjacent device is a non-Spectrum switch, you need to enable router advertisements on the adjacent device on the connected interface to ensure proper next hop discovery." Also: `nv show router segment-routing srv6 sid <sid>` prints End.X for a SID configured as uA — NVUE shows the base behavior, not the flavored name.
2The command surface, both of them
The full NVUE sequence, verbatim from the documentation:
nv set router segment-routing srv6 state enabled
nv set router segment-routing srv6 locator LEAF prefix fcbb::/16
nv set router segment-routing srv6 locator LEAF block-length 16
nv set router segment-routing srv6 locator LEAF node-length 0
nv set router segment-routing srv6 locator LEAF func-length 16
nv set router segment-routing static srv6-sid fcbb:fe8::/32 behavior uA
nv set router segment-routing static srv6-sid fcbb:fe8::/32 interface swp1
nv set router segment-routing static srv6-sid fcbb:fe8::/32 locator-name LEAF
nv config apply[1] The locator parameter ranges: block-length 16-64 with a default of 16, node-length 0-64 with a default of 16, func-length 0-64 with a default of 0, and “the prefix length must match the sum of block length and the node length”.[1][2] The example above is a uA only locator: 16/0/16, prefix /16, all function bits.
NVIDIA publishes the legal combinations as a table, and it is worth understanding rather than memorising:
| Format | Block Length | Node Length | Function Length |
|---|---|---|---|
| uN | 32 | 16 | 0 |
| uA + uN | 16 | 16 | 16 |
| uN only | 16 | 16 | 0 |
| uA only | 16 | 0 | 16 |
[1][2] A pure uN locator needs no function bits because the node identifier is the instruction — “advance the list at this node”. uA needs function bits because it names one adjacency among several on the same node. The first row, 32/16/0, is RFC 9800’s mandatory-to-implement F3216 profile; the rows below it trade block bits for function bits inside a 32-bit locator.[5]
Underneath, this is FRR. The vtysh equivalent is exactly the static-sids node from the previous lesson: segment-routing → srv6 → static-sids → sid fcbb:fe8::/32 locator LEAF behavior uA interface swp1, with the locator configured under locators → locator LEAF → prefix fcbb::/16 block-len 16 node-len 0 func-bits 16.[1][6] That equivalence is useful twice: it tells you what to expect from vtysh on a real switch, and it tells you that a Linux or FRR node can stand in for the configuration half of this lesson when you have no Spectrum-4.
Note the range mismatch between the two documents: upstream FRR documents node-len as (16-64), while NVIDIA’s own vtysh example uses node-len 0 for a uA-only locator and NVUE gives node-length a range of 0-64. Check which grammar the build in front of you accepts before you blame the configuration — if a stock FRR build refuses node-len 0, the documented upstream range is why, not a Cumulus bug.[1][2][11]
To remove: nv set router segment-routing srv6 state disabled, nv unset router segment-routing srv6 locator, nv unset router segment-routing static srv6-sid [<prefix>].[1]
3uA, router advertisements, and a Dell rack
uA is the behavior with an interop dependency, and the documentation states it precisely: “For uA segment identifiers, next hop (peer link-local) learning occurs with router advertisements. Spectrum switches enable router advertisements on the interface automatically when you configure a uA segment identifier; however, if the adjacent device is a non-Spectrum switch, you need to enable router advertisements on the adjacent device on the connected interface to ensure proper next hop discovery.”[1][2]
Unpack what that means in a mixed rack. uA is a cross-connect: it must resolve to one specific L3 adjacency, and on an IPv6 link that adjacency is a link-local address.[8] The switch learns that address the ordinary IPv6 way, from the neighbour’s router advertisements. Spectrum handles its own side automatically. A Dell PowerSwitch on the other end of the link does not know a uA SID exists, and if its interface is not sending RAs the Spectrum side has nothing to resolve. The symptom is a configured SID that never forwards, on a link that passes every other test — ping works, the IGP is up, the neighbour table has entries for everything except the one thing uA needs.
This is the interop footgun to raise before the bake-off rather than during it, and it is entirely fixable: enable RAs on the connected interface of the neighbouring device.
A second documented gotcha lives nearby: “Avoid reusing IPv6 prefixes for both static routes and static segment identifiers… If you configure a static route that overrides a segment identifier or a segment identifier that overrides a static route, unset, then reset the static segment identifier.”[1] SIDs are routes, so a static route and a SID for the same prefix are two claims on one FIB entry, and the recovery is an explicit unset-then-reset rather than a reload.
| Feature | Linux seg6 | FRR | Cumulus / Spectrum-4 | Community SONiC | DOCA Flow 3.5.0 | Dell |
|---|---|---|---|---|---|---|
uN End with NEXT-CSID | ||||||
uA End.X with NEXT-CSID | ||||||
End / End.X classic RFC 8986, uncompressed | ||||||
End.DT46 / uDT46 service SID, decap into a VRF | ||||||
SRH push H.Encaps / H.Encaps.Red | ||||||
SRH removal POP in hardware | ||||||
REPLACE-C-SID RFC 9800 second flavor | ||||||
PSP / USP / USD RFC 8986 flavors | ||||||
IGP SID distribution IS-IS per RFC 9352 | ||||||
BGP service SIDs L3VPN per RFC 9252 | ||||||
TI-LFA over SRv6 RFC 9855, October 2025 | ||||||
HMAC TLV RFC 8754 TLV type 5 |
✓ yes · ✗ no · ◐ conditional · ? unknown · — n/a. Click a cell for the evidence.
⚠ = not confirmed on a fetched primary source (hover for why). Facts as of DOCA 3.5.0 (Sep 2026). Selections are saved.
4Verifying: where the SIDs show up and what the counters mean
Segment routing endpoints are not a special object class. “Segment routing endpoints are installed as IPv6 routes into the RIB and FIB. To show segment routing endpoints, view the IPv6 RIB with the nv show vrf <vrf-id> router rib ipv6 route command.”[1] That single sentence explains both the static-route collision above and why an ordinary routing check is a valid SRv6 check.
The dedicated show commands are nv show router segment-routing, nv show router segment-routing srv6 locator [<locator-id>] and nv show router segment-routing srv6 sid <sid>, with the vtysh equivalents show segment-routing srv6 locator [<locator> detail] and show segment-routing srv6 sid <sid>.[1]
Expect one display asymmetry that reliably confuses people: nv show router segment-routing srv6 sid fcbb:fe8::/32 prints behavior End.X for a SID configured as uA.[1] NVUE shows the RFC 8986 base behavior rather than the flavored name.[8] Nothing is wrong; do not “fix” it.
Statistics are where the silicon becomes visible: nv show router segment-routing srv6 stats gives per-SID Hit Counters (SID and Packets) plus a Drop Counters section with “Total no-sid-dropped packets”; nv show router segment-routing srv6 stats sid <sid> and ... stats no-sid-drops narrow it, and nv action clear router segment-routing srv6 stats [sid <sid>|no-sid-drops] resets it.[1] The no-SID drop counter is the one to watch in a bring-up: it counts packets that arrived carrying a SID this switch does not have, which is the signature of a locator or length mismatch between the NIC that built the list and the switch executing it.
One caveat, verbatim: “When you enable packet trimming with segment routing, Cumulus Linux counts the trimmed packet twice in the SRv6 statistics.”[1] On an AI fabric where trimming is likely to be on, treat hit counters as a relative signal rather than an exact packet count.
5Reading the negative space
The list of what Cumulus Linux SRv6 does not document is longer than what it does, and stating it accurately is most of the FAE value here: no End, End.T, End.DT4/DT6/DT46, End.DX4/DX6/DX2 or End.B6 behaviors; no PSP, USP or USD as configurable flavors; no REPLACE-C-SID; no IS-IS or BGP SRv6 SID signalling; no SRv6 L3VPN; no SRv6 EVPN; no TI-LFA over SRv6; no HMAC. All SIDs static.[1][2]
The version history matters as much as the feature list. Every nv set router segment-routing … command is stamped “Introduced in Cumulus Linux 5.14.0”; the 5.14 What’s New page lists “Segment routing and Clear SRv6 statistics” as a new feature next to an FRR upgrade to 10.0.3; and the Segment Routing page does not exist in 5.13, which confirms 5.14 as the introducing release.[2][3][4] More importantly, the scope text is byte-for-byte identical in 5.14, 5.15, 5.16 and 5.18 — no SRv6 functional expansion has landed through 5.18.[1][7] A customer asking “will the next release add End.DT46?” deserves to know that four releases changed nothing.
Now the sentence to get right. The only NVIDIA statement available about which ASIC does SRv6 is the software one: Cumulus Linux supports segment routing on the Spectrum-4 switch only.[1] No per-ASIC SRv6 capability table was found on any NVIDIA product or datasheet page, so “Spectrum-2 and Spectrum-3 cannot do SRv6” is [UNVERIFIED] and should not be said. The defensible version is: “Cumulus Linux does not support SRv6 on those platforms.” Those two sentences sound the same in a meeting and are very different in a bake-off report — one is a citation, the other is an inference about silicon you have no source for.
Scenario: Spectrum-4 leaf on Cumulus 5.16, locator LEAF, one uA SID pointing out swp1 toward a Dell PowerSwitch.
Step 1 — choose the split before typing. uA needs function bits, and you also want uN on the same locator, so the published combination is 16/16/16 and the prefix must be block plus node — a /32.[1] If you only need uA, use 16/0/16 with a /16 prefix as in NVIDIA’s own example.
Step 2 — enable and define the locator.
nv set router segment-routing srv6 state enabled
nv set router segment-routing srv6 locator LEAF prefix fcbb:0001::/32
nv set router segment-routing srv6 locator LEAF block-length 16
nv set router segment-routing srv6 locator LEAF node-length 16
nv set router segment-routing srv6 locator LEAF func-length 16Step 3 — the SID, in three lines that belong together.
nv set router segment-routing static srv6-sid fcbb:1:e001::/48 behavior uA
nv set router segment-routing static srv6-sid fcbb:1:e001::/48 interface swp1
nv set router segment-routing static srv6-sid fcbb:1:e001::/48 locator-name LEAF
nv config diff
nv config apply[1] Read the diff before applying; a SID that collides with an existing static route is the documented failure this catches.[1]
Step 4 — the neighbour’s side. On the Dell PowerSwitch facing swp1, enable router advertisements on that interface. Without it, next-hop discovery for the uA SID never completes and the SID silently forwards nothing.[1]
Step 5 — verify in three places. nv show router segment-routing srv6 locator LEAF (lengths as configured); nv show router segment-routing srv6 sid fcbb:1:e001::/48 (expect behavior End.X, which is correct); nv show vrf default router rib ipv6 route (the SID present as an IPv6 route).[1]
Step 6 — traffic and counters. Source uSID traffic from a ConnectX host, then nv show router segment-routing srv6 stats. Expect the hit counter on this SID to rise; check nv show router segment-routing srv6 stats no-sid-drops too — a rising no-SID drop count with a flat hit count means the NIC built a list with a different block or length split.[1]
Rollback, in reverse order: nv unset router segment-routing static srv6-sid fcbb:1:e001::/48, nv unset router segment-routing srv6 locator, nv set router segment-routing srv6 state disabled, nv config apply. Disable RAs on the Dell interface only if they were off before you started.[1]
Same leaf, now add a uN SID on the same switch using the F3216 profile, plus the Linux stand-in you would build in the Dell lab to demonstrate the equivalent behavior without a Spectrum-4.
- Which published combination is the F3216 profile, and what prefix length does it force?
____ nv set router segment-routing srv6 locator SPINE prefix ____ block-length ____ node-length ____ func-length ____nv set router segment-routing static srv6-sid ____ behavior ____— and which ofinterfaceandvrfmust you not set here?____- Write the vtysh equivalent of the two locator lines.
____ - On a Linux node, write the
seg6localroute that behaves like this uN SID, including the flavor and the lengths that are already the kernel default.____ - Which counter would you read on each side to compare them, and why is the Linux one more trustworthy under packet trimming?
____
A Dell OEM customer has an AI cluster: BlueField-3 SuperNICs, Spectrum-4 leaves on Cumulus 5.16, Dell PowerSwitch aggregation running Enterprise SONiC. They send a one-line requirement: “we want SRv6 with per-tenant isolation, TI-LFA protection, and the tenant VRFs terminated on the leaf.”
Write the response. It must: (a) separate the parts of that request Cumulus Linux implements from the parts it does not, each with a citation; (b) name where each unimplemented part could live instead, if anywhere; (c) state the Dell aggregation question honestly rather than asserting support or non-support; (d) contain no sentence about Spectrum silicon capability that your sources do not support. Acceptance: a customer could forward your answer to a competitor without you having to retract anything.
Slide four, rewritten
Slide four now says host-driven path selection across an AI fabric, with three constraints beside it: Spectrum-4 only, default VRF only, static uN and uA.[1] You raise the uA dependency unprompted: in a mixed rack the non-Spectrum switch on the far end must send router advertisements, or the SID never resolves a next hop and the failure looks like an NVIDIA defect.[1][2] The architect asks when service SIDs arrive on the switch; the NVIDIA PM puts up a roadmap slide and says nothing further is announced.
What you say to the architect: I will tell you exactly what Cumulus Linux supports, and not what the ASIC cannot do. He writes both sentences down, then asks the obvious next one: if the leaves only execute a path, what builds it?
Lab
Optional, customer lab only — there is no NVIDIA switch in the Dell lab. Run this when you have access to a Spectrum-4 on Cumulus 5.14 or newer. Mutating.
- Pre-flight inventory:
nv show systemfor the release,nv config diff(expect empty),nv show vrf default router rib ipv6 routesaved to a file, and the current interface configuration of the port you will use. Save the running configuration before touching anything. - Configure one uN and one uA SID using the Worked example. Rollback for every step: the matching
nv unsetline plusnv config apply; the saved configuration file is the backstop.[1] - Source uSID traffic from a ConnectX host and read
nv show router segment-routing srv6 statsandnv show router segment-routing srv6 stats no-sid-drops. Expected: hit counters rising on the SIDs you configured. If trimming is enabled on this switch, remember the trimmed packet is counted twice.[1] - Prove the interop footgun on purpose. Connect the uA interface to a Dell PowerSwitch with router advertisements disabled on the Dell side. Expected: next-hop discovery fails and the uA SID forwards nothing while every other test on the link passes. Then enable RAs on the Dell interface and watch it recover. Rollback: return the Dell interface to its previous RA setting. Capture both states — this transcript is worth more in a customer meeting than the documentation sentence.[1]
- Confirm the static-route collision behavior: add a static route for a prefix that overlaps a configured SID, observe, then recover with unset-then-reset of the segment identifier as documented. Rollback: remove the static route.[1]
- Restore to the step 0 inventory and diff the RIB dump against the file you saved.
In the Dell lab without a switch, substitute step 1 and 2 with the Linux stand-in — a node running End.X flavors next-csid for uA and End flavors next-csid for uN — and diff the two configurations line by line. Say clearly in your notes which findings came from silicon and which from a software stand-in.[10]
The deliverable is the answer to an open question: does the simulator expose the SRv6 command set at all? Use the NVIDIA Air free tier or a local Cumulus VX. Everything here is on a throwaway simulated switch; the rollback for the whole lab is deleting the simulation.
- Enter the documented NVUE sequence exactly as published — all nine lines, ending in
nv config apply.[1] Expected: acceptance. If a line is rejected as unknown, record the exact error and thenv show systemversion: that is the lab’s primary finding. - Run
nv show router segment-routing,nv show router segment-routing srv6 locator LEAFandnv show router segment-routing srv6 sid fcbb:fe8::/32. Record whether each command exists. Whether the simulator exposes this command set is an open question in the course research, and documenting the answer is the deliverable.[1] - Confirm the display asymmetry:
nv show router segment-routing srv6 sid <sid>should printbehavior End.Xfor the uA SID. Expected:End.X. If it printsuA, note it — the documentation says otherwise and the version you are on matters.[1] - Drop to
vtyshand confirm the same objects undershow segment-routing srv6 locator detailandshow segment-routing srv6 sid. Expected: the same locator and SID, in FRR’s format. This is the check that tells you which half of the feature is FRR and which half is NVUE.[1][6] - Send traffic through the simulated switch and read
nv show router segment-routing srv6 stats. Expected: no hit counters — with no Spectrum-4 ASIC underneath there is nothing to count. Write down explicitly which half of the feature you just proved (configuration, FRR objects, RIB installation) and which half you did not (forwarding, counters, trimming interaction).[1] - Build the Linux stand-in: on a containerlab node,
ip -6 route add fcbb:bbbb:1:e000::/64 encap seg6local action End.X flavors next-csid nh6 fe80::2 count dev eth2, and diff it line by line against the NVUE uA configuration. Expected: the same three facts on both sides — locator block and node lengths, the adjacency, the behavior.[10]
Deliverable: a one-page note stating which SRv6 commands exist in the simulator you used, what version it reported, and a two-column diff of the NVUE and Linux configurations of the same uA behavior.
Retrieval check
10 questions from memory. Answer before looking anything up; misses become flashcards.
Explain it to a Dell SE
Explain to a Dell account team, in five sentences, exactly what SRv6 NVIDIA switching does today, what it does not do, and which customer request it is the right answer to.
Sources
Facts in this lesson were checked against Cumulus Linux 5.18 Segment Routing page and NVUE Segment Routing reference re-fetched 2026-09-07; text identical in 5.14, 5.15, 5.16 and 5.18. Dates are when each page was fetched.
- Segment Routing | Cumulus Linux 5.18 · fetched 2026-09-07
- Segment Routing | NVUE Command Reference (Set and Unset) · fetched 2026-09-07
- What's New | Cumulus Linux 5.14 · fetched 2026-09-07
- Segment Routing | Cumulus Linux 5.13 (page does not exist) · fetched 2026-09-07
- RFC 9800 - Compressed SRv6 Segment List Encoding · fetched 2026-09-07
- FRR documentation - Static (latest) · fetched 2026-09-07
- Segment Routing | Cumulus Linux 5.16 · fetched 2026-09-07
- RFC 8986 - SRv6 Network Programming · fetched 2026-09-07
- NVIDIA Certified Professional: AI Networking (NCP-AIN) · fetched 2026-09-07
- iproute2 man source: ip-route.8.in (main) · fetched 2026-09-07
- FRR documentation - Zebra (latest) · fetched 2026-09-07
The same idea elsewhere
Other lessons that cover this ground, sometimes from another course's angle.
- Design and defend an SRv6 PoC for a Dell OEM accountElsewhere in this course · Same ground: interop, frr and limits
- PFC, lossless and the case for lossyRoCE course · Same ground: NVUE, hw_counters and cumulus
- L3VPN and EVPN services over SRv6Elsewhere in this course · Same ground: frr, limits and verify