Glossary

Every term the courses define, in the words of the lesson that defines it.

408 terms

A

Active Segment: The instruction currently being executed. RFC 8402: in the MPLS data plane it is the top label, in the IPv6 data plane it is the destination address. Every SRv6 endpoint behavior is a variation on what a node does when its own SID shows up as the destination address.; Defined in SRv6 1.1 · Source routing, segments, and the two data planes
Adaptive routing (ar_mode, ar_sl_mask): Moving traffic to less congested paths according to the current temporal state of the network, chosen from the alternatives the routing engine programmed. ar_mode defaults to 3 (auto: the routing engine decides); ar_sl_mask defaults to 0xFFFF and gates which SLs are eligible; ar_transport_mask defaults to 0x000A, which covers RC and excludes UD. It needs both ends: the MLNX-OS ib-single-switch system profile on the switch and the option enabled in the SM.; Defined in InfiniBand 3.2 · Adaptive routing, SHIELD and hash-based forwarding
Adaptive routing (Cumulus Linux): Selecting forwarding paths dynamically from switch state such as queue occupancy and port utilization, on a per-packet basis. Documented on Spectrum-4 at 400G and 200G for RoCEv2 unicast and VXLAN-encapsulated RoCE on layer-3 next-hop interfaces in the default VRF; disabled by default both globally and per interface.; Defined in Spectrum-X 4.1 · Adaptive routing: per-packet spraying and its eligibility rules
adaptive routing (JSQ): Per-packet forwarding decision inside the switch, described by NVIDIA as a quantized hardware approximation of Join-Shortest-Queue: sample egress queue depths and pick the least-congested port within hundreds of nanoseconds. Disabled by default in Cumulus Linux.; Defined in Spectrum-X 1.1 · What Spectrum-X is, and what it is not
Adaptive routing eligibility: The published envelope in which adaptive routing operates: RoCEv2 unicast or VXLAN-encapsulated RoCE, on layer-3 default-VRF next-hop router interfaces, on Spectrum-4 at 400G or 200G (not 800G), with no support on subinterfaces, SVIs, bonds or bond members and no integration with resilient hashing. Most 'AR does nothing' reports are eligibility, not defect.; Defined in Spectrum-X 6.2 · Scenario: 'the fabric is slow and adaptive routing does nothing'
Adaptive routing speed exclusion: Cumulus Linux states that adaptive routing on Spectrum-4 is supported at 400G and 200G but not at 800G. It is the reason reference architectures break 800G ports out to 2x400G rather than running them native, and it is a design-review item because enabling or disabling AR reloads switchd and interrupts traffic.; Defined in RA 4.1 · The Ethernet vs InfiniBand decision framework
adaptive_routing_forced_en: Field of the host NIC's ROCE_ACCL register discussed in NVIDIA's developer forum, where an mlxreg --set returns success while the field stays zero against a non-Spectrum-X switch. NVIDIA's answer in that thread is that adaptive routing is supported with the Spectrum-X solution only.; Defined in Spectrum-X 6.2 · Scenario: 'the fabric is slow and adaptive routing does nothing'
ai-ethernet-stats: The Cumulus telemetry stat group carrying adaptive routing, SRv6 and packet-trimming data, with a default sample interval of 1 second. It emits nothing unless adaptive routing, packet trimming or SRv6 is enabled separately — the most common cause of an empty Spectrum-X telemetry dashboard.; Defined in Spectrum-X 5.3 · Streaming telemetry: OTLP, high-frequency counters, latency measurement
algbw: Algorithm bandwidth, S/t: the operation size divided by the time it took. It answers how long an operation of a given size will take and says nothing directly about the hardware.; Defined in RoCE 3.5 · algbw, busbw, and reading a nccl-tests result
algbw and busbw: algbw = S/t, the raw operation size over time; busbw = algbw times a per-collective correction factor (2(n-1)/n for AllReduce, (n-1)/n for AllGather, ReduceScatter and AllToAll, 1 for Broadcast and Reduce) so the figure reflects the hardware bottleneck.; Defined in RoCE 5.2 · Scenario: "the 400G NIC only does 180 Gb/s"
Argus: Agentless DOCA security service (Beta) that uses hardware-level live machine introspection and DOCA DMA to read host memory from the DPU and emit process, module, container and threat events to a SIEM; needs DPU mode, firmware 24.35.0388+, BFB 4.11.0+, PF_BAR2_SIZE=2 PF_BAR2_ENABLE=1 and IOMMU passthrough.; Defined in DOCA 5.5 · Security: Argus, App Shield, and zero trust
asic-monitor: The switch service NetQ's Adaptive Routing dashboard depends on for ASIC monitoring. Disabling it makes the dashboard's histogram metrics report zero rather than unavailable — a silent failure that looks like a quiet fabric.; Defined in Spectrum-X 5.2 · NetQ 5.1: validation instead of inspection
auto-negotiation (PAM4 rule): On by default. Spectrum-4 to Spectrum-4 links using PAM4 must have it enabled, while SN4700 and SN4410 QSFP-DD 400G ports do not support it and need forced speed - the two rules point in opposite directions and are the classic cause of a link that will not come up.; Defined in Spectrum-X 1.3 · Links that come up: breakout, FEC and auto-negotiation

B

Behavior versus function: Not synonyms. The function is the bit string inside the SID; the behavior is what the node does when that SID is the active segment. The IANA registry is called SRv6 Endpoint Behaviors and lists behaviors with codepoints - 1 End, 5 End.X, 20 End.DT46, 27 End.B6.Encaps.Red - with gaps, so codepoints are looked up rather than computed.; Defined in SRv6 1.3 · SIDs as instructions: LOC:FUNCT:ARG and the behavior zoo
BER view: nv show interface --view ber, a tabular PHY health summary across all interfaces with the columns Eff-BER, Raw-BER, Sym-BER, Sym-Errs, Eff-Errs and Last-Clr(min). It localises a marginal link that is still reporting up; per-lane SNR then comes from nv show interface <id> link phy health.; Defined in Spectrum-X 6.1 · The triage toolkit: resources, optics and what tcpdump will not tell you
BF-Bundle / bf-fwbundle: The BlueField Software Bundle installed on the Arm cores: DOCA runtime, the Arm OS (Ubuntu 24.04 64k in 3.5.0) and platform software (firmware, UEFI). The bf-fwbundle is the firmware-only BFB that upgrades ATF-UEFI, NIC and BMC firmware without overwriting the OS.; Defined in DOCA 1.5 · The DOCA framework map, versions and LTS
bf-bundle vs bf-fwbundle: bf-bundle is the full BFB (Arm OS, DOCA runtime, platform firmware and UEFI); bf-fwbundle is the firmware-only BFB that updates ATF/UEFI, NIC firmware and BMC firmware without overwriting the OS.; Defined in DOCA 2.3 · rshim and the BFB path
bf.cfg: A shell-style configuration appended to a BFB (no spaces around =) that sets passwords and firmware-update options and provides hooks bfb_pre_install, bfb_modify_os (target mounted under /mnt) and bfb_post_install.; Defined in DOCA 2.3 · rshim and the BFB path
BFB: BlueField boot image bundle (OS + firmware) pushed over rshim with bfb-install to provision the Arm side.; Defined in DOCA 2.3 · rshim and the BFB path
BGP unnumbered (RFC 8950 / ENHE): eBGP peering over IPv6 link-local addresses with no IPv4 on the fabric link, carrying IPv4 prefixes with an IPv6 next hop. Extended Next Hop Encoding is capability code 5; receivers decide the next-hop family from its length field, not the NLRI AFI.; Defined in Spectrum-X 2.4 · The underlay: eBGP unnumbered, ECMP and VRFs
BlueField-4 BMC-driven bring-up: Per NVIDIA/skills, BlueField-4 day-one provisioning runs from the card's BMC: bundle ISO via UEFI HTTP Boot (recommended), PXE or Redfish Virtual Media onto the Grace Arm complex, and firmware via PLDM over Redfish UpdateService; unlike the BlueField-3 rshim and bfb-install path. DOCA 3.5.0 does not list BlueField-4.; Defined in DOCA 7.4 · Roadmap: BlueField-4, ConnectX-9, DOCA Memos
BTH (Base Transport Header): The InfiniBand transport header that RoCE carries unchanged, immediately after the UDP header in RoCEv2. Its fields include the opcode, partition key, destination and source QP numbers and PSN (24 effective bits each), and flag groups carrying FECN and BECN - congestion bits that are separate from the two-bit ECN field in the IP header.; Defined in RoCE 1.3 · RoCEv1 vs RoCEv2: the encapsulation
busbw: Bus bandwidth: algbw multiplied by a per-collective correction factor (AllReduce 2(n-1)/n; ReduceScatter, AllGather and AllToAll (n-1)/n; Broadcast and Reduce 1). It should reflect the speed of the hardware bottleneck — NVLink, PCI, QPI or network — and is uninterpretable without the rank count and the collective.; Defined in RoCE 3.5 · algbw, busbw, and reading a nccl-tests result

C

C2P / P2C airflow suffix: The airflow direction committed by an SN5000 ordering part number - connector-to-power or power-to-connector. It is a build option, not a field setting, so a mis-ordered chassis cannot be corrected on site.; Defined in Spectrum-X 1.2 · Choosing the switch: SN2201 through SN6800-LD
C2P and P2C: Airflow conventions in the SN5000 ordering table: C2P is connector-to-power (reverse, back-to-front) and P2C is power-to-connector. Airflow is a property of the ordering part number, so a chassis ordered against the wrong hot-aisle direction is an RMA rather than a configuration change.; Defined in Spectrum-X 6.3 · Scenario: sizing, quoting and defending a Dell AI pod fabric
Cages versus logical ports: Transceivers go in cages, not in logical ports. An SN5600 or SN5610 has 64 OSFP cages presenting 128x400 GbE; a QM9700 has 32 cages carrying 64 ports of NDR400. Quoting a transceiver count that matches the logical port figure orders twice what fits.; Defined in RA 2.3 · The sizing chain: GPUs, NICs, planes, leaves, spines
CCMAD / IFA1 / IFA2: The three DOCA PCC probe mechanisms. CCMAD carries round-trip-time information for proactive congestion detection; IFA1 is In-band Flow Analyzer 1 in-band feedback; IFA2 is an optimized alternative.; Defined in Spectrum-X 4.4 · Programmable congestion control with DOCA PCC
Cell-based VOQ fabric: Arista's own vocabulary for the 7800R4 — a fabric that sprays uniform cells across all fabric links with virtual output queues and distributed credit scheduling. A different answer to the congestion problem than per-packet adaptive routing plus SuperNIC reordering, not a worse one. Use the competitor's published terms; paraphrasing their architecture is how you get corrected by an engineer who runs it.; Defined in RA 4.3 · Competitors, stated neutrally
Changeable field: A pipe-level match or action field set to all ones (0xffff, 0xffffffff) with a NULL mask, meaning each entry supplies its own value. Contrast constant (fixed value) and ignored (zero).; Defined in DOCA 4.2 · The DOCA Flow model
check_sl: An ibdiagnet routing sub-option (ibdiagnet -r --r_opt check_sl) that validates that no SL2VL mapping routes to VL15. VL15 in that table means drop this service level, so the check finds silently discarded traffic.; Defined in InfiniBand 5.3 · Escalation bridge call: five InfiniBand tickets
cl-resource-query: Linux command on Cumulus Linux that reports software-maintained forwarding resource utilization (routes, hosts, MAC entries, ACL rules, counters). Named verbatim in NCP-AIN objective 5.1; its NVUE counterpart, nv show platform asic <id> resource, reports from hardware when available, so small differences between the two are documented and expected.; Defined in Spectrum-X 6.1 · The triage toolkit: resources, optics and what tcpdump will not tell you
cma_roce_mode: The tool that reads and writes the RoCE version used by the RDMA connection manager, which is a global value of the RDMA-CM module rather than a per-connection option. Read with cma_roce_mode -d <dev> -p <port>, set with -m 1 (RoCEv1) or -m 2 (RoCEv2). It is why a perftest run pinned with -x and an application using RDMA-CM can use different versions on the same host.; Defined in RoCE 1.4 · GIDs: how a connection picks its RoCE version
CNI (Container Network Interface): The API through which Kubernetes network implementations plug in. An implementation is a network plugin that attaches pods plus an IPAM plugin that assigns their addresses. Kubernetes itself implements none of the four requirements of its own network model - it only defines them and expects a CNI implementation to satisfy them.; Defined in K8sNet 1.1 · The Kubernetes network model and where a second CNI fits
CNP (congestion notification packet): The RoCEv2 packet a receiving NIC sends back after seeing CE-marked traffic, so the sender can throttle. It needs its own high-priority class on the return path - NVIDIA switch defaults put it on DSCP 48, switch priority 6, traffic class 6, scheduled strict.; Defined in RoCE 2.4 · ECN, CNP and DCQCN: closing the loop
Connection tracking (CT) pipe: DOCA_FLOW_PIPE_CT: a bidirectional connection table in hardware. Entries are installed with doca_flow_ct_entry_prepare, doca_flow_ct_add_entry (origin and reply) and doca_flow_ct_entries_process on a dedicated CT queue.; Defined in DOCA 4.3 · Domains, connection tracking, aging, meters
Control-plane-only capture: The documented limit of tcpdump on a Cumulus switch: it monitors traffic sent to and coming from the switch CPUs and does not monitor data plane traffic (the page points to cl-acltool). Forwarded packets are handled by the ASIC, so a capture on a leaf cannot contain a customer's GPU-to-GPU flow.; Defined in Spectrum-X 6.1 · The triage toolkit: resources, optics and what tcpdump will not tell you
Converged network: The north-south fabric: two 400 GbE ports per compute and management node to two separate switches, reaching up to 40 GB/s per node on B300 and 200 GbE per port on the H100/H200/B200 generation. Storage and in-band management ride it under VLAN isolation.; Defined in RA 2.4 · The other three networks: converged, storage, out-of-band
cpu_manager_state: The kubelet checkpoint at /var/lib/kubelet/cpu_manager_state that records exclusive CPU assignments across restarts. Switching CPU Manager from none to static means drain, stop kubelet, remove this file, update the config, start kubelet, uncordon - skip the removal and the kubelet refuses to start on the policy mismatch.; Defined in K8sNet 3.5 · NUMA alignment with Topology Manager and CPU Manager
Credit loop: A cyclic dependency between link-level flow-control credits that deadlocks a fabric. It is what Up/Down, fat-tree, LASH, DOR, torus-2QoS and Nue exist to prevent, and what options like --max_reverse_hops and -z/--connect_roots can reintroduce. Prove its absence with ibdiagnet -r, whose success string is -I- no credit loops found.; Defined in InfiniBand 3.1 · Routing engines and how OpenSM picks a path
CSID container: RFC 9800 term for a 128-bit IPv6 address that holds a list of one or more Compressed SIDs plus the argument of the last one. Capacity is K = floor((128 - LBL) / LNFL), which gives 6 at a 32-bit Locator-Block and a 16-bit CSID length. NEXT-C-SID keeps the sequence contiguous in the argument; REPLACE-C-SID packs later containers in reverse position order.; Defined in SRv6 1.4 · Compression: RFC 9800, NEXT-C-SID, and what "uSID" really means

D

--data_validation: perftest mode that checks every byte against an expected pattern to detect corruption, DMA races and stale data on RDMA WRITE or READ. RC only, bandwidth tests only, tx_depth at least 32 identical on both sides. Only the errors counter is a defect; races, retries and skips are normal at speed.; Defined in RoCE 3.2 · NUMA, data validation, and doca_perftest
Data-path accelerator (DPA): 16 hyperthreaded cores on the SuperNIC, programmable through DOCA for congestion control and traffic management. It is where a DOCA PCC user algorithm actually executes.; Defined in Spectrum-X 4.2 · Why spraying needs a SuperNIC: reordering and direct data placement
DCQCN roles (CP, NP, RP): The three parties in the ECN loop: the Congestion Point, a switch, detects congestion and marks CE; the Notification Point, the receiver, answers marked packets with CNPs; the Reaction Point, the sender, reduces its rate on the CNPs it receives. The switch never generates a CNP.; Defined in RoCE 2.4 · ECN, CNP and DCQCN: closing the loop
Decompress-only (BlueField-3 Compress): In DOCA 3.5.0 the Compress library on BlueField-3 supports deflate decompress and LZ4 decompress (stream and block tasks) but no hardware compression; deflate compress is a BlueField-2 capability. Checksums: Adler from deflate tasks, CRC from all tasks, xxHash from LZ4 decompress.; Defined in DOCA 6.2 · Compress, SHA, Erasure Coding, AES-GCM
Default partition asymmetry: With no partitions.conf at all every end port is a full member of 0x7fff; with a file present that has no rule for Default every end port is demoted to limited. Creating a partition file is therefore what breaks a working cluster.; Defined in InfiniBand 2.3 · Partitions and PKeys: the most common mistake in IB
Dell Technologies ETC program: The route through which an SN5600 is supported as a PowerScale back-end switch. The published footnote directs you to the Dell account team, states that the Spectrum-4 must be configured manually for a PowerScale cluster, and pins Cumulus Linux 5.9.1. It is a third support path, separate from ProSupport and from an NVIDIA TAM.; Defined in RA 3.5 · Who do I call: three support paths, one asset tag
Derived versus published number: A figure you computed (leaf counts, optics totals, per-domain question counts) versus one printed on a page you fetched. Every derived number must be labelled derived wherever it is written down, because an unlabelled derivation is quoted back later as if a vendor published it.; Defined in RA 4.4 · Scenario: a 256-GPU Dell AI Factory design review
devel-3.5.0-host: NGC tag of the host-side DOCA 3.5.0 development container (nvcr.io/nvidia/doca/doca). The un-suffixed devel-3.5.0 is the BlueField-side image; base-rt and full-rt are the runtime-only flavors.; Defined in DOCA 3.1 · Development environments
Device plugin: A vendor DaemonSet that advertises hardware to kubelet without changing Kubernetes core code. It registers on the hardcoded socket /var/lib/kubelet/device-plugins/kubelet.sock, serves its own gRPC socket in the same directory, and implements GetDevicePluginOptions, ListAndWatch, Allocate, GetPreferredAllocation and PreStartContainer.; Defined in K8sNet 2.1 · How kubelet learns a NIC exists
deviceID (SR-IOV CNI): The required field of the sriov CNI delegate, a VF PCI address such as 0000:03:02.3. It is not written by hand in a NAD: Multus receives the allocated VF's PCI address from the device plugin and passes it to the sriov plugin at runtime.; Defined in K8sNet 2.2 · SR-IOV plumbing: device plugin, CNI and operator
deviceType vfio-pci: The SriovNetworkNodePolicy setting that passes a VF's PCI device directly into a guest VM through the VFIO userspace interface. The host keeps no netdevice, the policy sets isRdma false because RDMA is incompatible with passthrough, and the guest must carry the mlx5_core driver itself.; Defined in K8sNet 2.5 · vfio-pci for VMs, and the DRA SR-IOV driver
DGX SuperPOD RA: The reference-architecture family for NVIDIA-branded DGX systems with NVIDIA-led installation and support through an NVIDIA Technical Account Manager. It covers InfiniBand, the NVLink network, Ethernet fabric topologies, storage specifications, rack layouts and wiring guides.; Defined in RA 1.1 · Which reference architecture governs this deal
Direct data placement: The ConnectX-8 generation phrasing of the endpoint repair: SuperNICs place packets directly in order into the buffer as they arrive at the receiving end, rather than reassembling after the fact.; Defined in Spectrum-X 4.2 · Why spraying needs a SuperNIC: reordering and direct data placement
Direct Data Placement (DDP): SuperNIC mechanism that writes out-of-order sprayed packets to their correct offset in host or GPU memory at the RoCE transport layer, so the application sees in-order data without switch-side reordering.; Defined in DOCA 7.1 · Spectrum-X architecture
Disqualifying question: A gate asked before a PoC is designed because it changes the proposal rather than adding a risk. For SRv6 on NVIDIA the two are whether the customer needs REPLACE-C-SID, which NVIDIA switching does not document, and whether they need EVPN over SRv6, which neither Cumulus Linux nor FRR's BGP guide documents.; Defined in SRv6 3.4 · Design and defend an SRv6 PoC for a Dell OEM account
Distributed Services Engine (DSE): VMware's DPU offload that runs ESXio on the BlueField Arm cores. Factory default is DPU mode with DSE disabled; switching between NIC mode and DSE-enabled DPU mode requires re-installing ESXi (Broadcom KB 379391); Dell supports it only on Dell-sourced DPUs.; Defined in DOCA 8.2 · iDRAC, BIOS, CPLD and the DPU
DMA-BUF path: The NVIDIA-recommended GPUDirect RDMA mechanism, using the Linux kernel DMA-BUF framework. Requires the Open Kernel module driver, CUDA 11.7 or higher and Linux kernel 5.12 or higher; MLNX_OFED or DOCA-OFED is optional. It loads no extra kernel module, so a clean lsmod is the expected state.; Defined in K8sNet 3.4 · GPUDirect RDMA: dma-buf, nvidia-peermem and GDS
DMA-BUF vs nvidia-peermem: The two GPUDirect peer-memory paths. DMA-BUF needs open kernel modules, CUDA 11.7 or later and kernel 5.12 or later and makes OFED optional; legacy nvidia-peermem requires MLNX_OFED or DOCA-OFED and must be built after it. NVIDIA recommends DMA-BUF.; Defined in RoCE 3.3 · GPUDirect RDMA: proving it is actually on
DMS (DOCA Management Service): Centralized configuration and provisioning service for BlueField and ConnectX; in DOCA 3.5.0 exposed through dms-cli (alias dmsc) and libdms with NVIDIA-native /nvidia/* YANG paths, replacing the 3.4.0 dmsd gNMI/gNOI server on port 9339. Alpha in 3.5.0.; Defined in DOCA 5.3 · DMS: one interface for every device
DOCA: NVIDIA's SDK and runtime framework for BlueField and ConnectX: drivers, libraries (Flow, DMA, RDMA, Comch, DPA…), services (HBN, DTS, DPF…) and tools, released quarterly with LTS lines.; Defined in DOCA 1.5 · The DOCA framework map, versions and LTS
DOCA Bench modes: doca_bench --mode throughput (default, maximum volume), bulk-latency (batched jobs with 100 latency buckets set by --latency-bucket-range) and precision-latency (one job at a time, job-limited only); results to console and, with --csv-output-file, to CSV including configuration values.; Defined in DOCA 3.5 · Debugging and measuring with DOCA Bench
DOCA Ethernet (RXQ/TXQ): Beta library giving applications raw receive and transmit queues on a device without DPDK; RXQs need a DOCA Flow pipe to receive traffic. 3.5.0 adds a shared mempool RXQ type, CQE compression and WQE inline data.; Defined in DOCA 4.4 · DOCA Ethernet and OVS-DOCA
DOCA Flow pipe: A match/action table in the e-switch created with doca_flow_pipe_create. Seven types: BASIC, CONTROL, LPM, ACL, HASH, ORDERED_LIST, CT. Entries are added asynchronously and completed with doca_flow_entries_process.; Defined in DOCA 4.2 · The DOCA Flow model
DOCA Memos: DOCA framework announced at GTC March 2026 for dedicated KV-cache storage processing, tied to the BlueField-4 AI-native storage platform, with an up-to-5x inference throughput claim; no API, quality level or DOCA release number is published yet.; Defined in DOCA 7.4 · Roadmap: BlueField-4, ConnectX-9, DOCA Memos
DOCA PCC: The Programmable Congestion Control framework that hosts ZTR-RTT and customer algorithms, with reaction-point and notification-point device libraries built by DPACC and run on the DPA. Verified floor: BlueField-3 or later, ETHERNET link type only, firmware 32.38.1000 and higher.; Defined in RoCE 2.5 · Zero Touch RoCE, ZTR-RTT CC and DOCA PCC
DOCA PCC floor: The only verified hardware and firmware requirement in this area: NVIDIA BlueField-3 Platform or later, Ethernet link type only, firmware 32.38.1000 or higher, with mlxconfig USER_PROGRAMMABLE_CC=1 and PCC_INT_EN=0. It applies to DOCA PCC and must not be quoted as a ZTR-RTT requirement.; Defined in RoCE 5.3 · Scenario: "we do not want to configure PFC"
DOCA Verbs Multi-Rail: New in DOCA 3.5.0: a doca_verbs_multi_rail_cfg and doca_verbs_multi_rail object pair that splits and rebalances traffic across several RC QPs and rails, with probe QPs and a recovery path for NIC resiliency. Shipped as the verbs_multi_rail_server_client reference application (disabled unless --use-multi-rail); the public Verbs page did not detail the API when fetched.; Defined in DOCA 6.1 · DMA, RDMA, and Verbs
DOCA_ARCH_DPU: Preprocessor macro defined in doca_build_config.h when compiling for the BlueField Arm side; samples use #ifdef DOCA_ARCH_DPU to select DPU-only roles and applications/meson.build probes it to set is_dpu.; Defined in DOCA 3.1 · Development environments
doca_buf: View onto mapped memory with headroom, dataroom and tailroom, allocated from a doca_buf_inventory (buf_get_by_addr or buf_get_by_data) and released with doca_buf_dec_refcount. It never owns the underlying memory; ownership passes to the library while a task is in flight.; Defined in DOCA 3.2 · DOCA Core objects and lifecycle
doca_ctx state machine: Idle (configuration allowed, no tasks), Starting (waiting on an asynchronous event such as a Comch connect), Running (task allocation and submission allowed, configuration disabled), Stopping (in-flight tasks flushed). doca_ctx_stop may return DOCA_ERROR_IN_PROGRESS; progress the PE until Idle.; Defined in DOCA 3.2 · DOCA Core objects and lifecycle
doca_dev_rep: Representor handle opened on the DPU side for a host PCIe function, via doca_devinfo_rep_create_list and doca_dev_rep_open. A Comch server needs one (the -r flag); a host client does not.; Defined in DOCA 3.2 · DOCA Core objects and lifecycle
doca_mmap: Memory map that registers an address range with one or more devices. Configured with doca_mmap_add_dev, doca_mmap_set_permissions and doca_mmap_set_memrange, then doca_mmap_start; exported across PCIe with doca_mmap_export_pci and imported with doca_mmap_create_from_export.; Defined in DOCA 3.2 · DOCA Core objects and lifecycle
doca_perftest: DOCA SDK 3.5.0's cluster-scoped RDMA benchmark, positioned as the replacement for legacy point-to-point tools. Adds built-in orchestration from one host, native multi-process concurrency, traffic patterns such as all-to-all and bisection, and hardware-aligned synchronized start and stop.; Defined in RoCE 3.2 · NUMA, data validation, and doca_perftest
DOCA-Host: The DOCA software package installed on the host server, delivered as one of five installation profiles: doca-all, doca-networking, doca-ofed, doca-roce, doca-host-basic.; Defined in DOCA 1.5 · The DOCA framework map, versions and LTS
DOCA-Host profile: A meta-package name (doca-all, doca-networking, doca-ofed, doca-roce, doca-host-basic) given to apt or yum that selects which DOCA drivers, libraries and tools install on the host; each pulls doca-runtime and doca-devel by default.; Defined in DOCA 2.1 · DOCA-Host install and profiles
Downlink split: The assumed division of a leaf's logical ports between server-facing downlinks and spine-facing uplinks - commonly half and half, so 64 and 64 on a 128-port SN5600 leaf. No reference architecture publishes it, so it is the one number in the chain you must always label as your own.; Defined in RA 2.3 · The sizing chain: GPUs, NICs, planes, leaves, spines
DPA (Data Path Accelerator): The programmable many-threaded processor inside BlueField-3 (16 cores, 256 threads per the datasheet) and ConnectX-8, sitting next to the NIC engines. Programmed through DOCA DPA (Beta) or Flex IO (GA); threads are a hardware resource capped at 256, execution units live in partitions managed by dpaeumgmt, and DPA-to-NIC memory is not coherent without an explicit writeback or fence.; Defined in DOCA 6.3 · DPA, Flex IO, and PCC
DPA (data-path accelerator): BlueField-3's embedded subsystem for workloads that need high-performance access to the NIC engines: 16 cores, 256 threads, per-EU L1 data cache, shared L2, backed by NIC private memory in DDR. Programmed via DOCA DPA (Beta) or FlexIO SDK (GA); also present on ConnectX-7/8/9.; Defined in DOCA 1.3 · BlueField-3 under the hood
DPACC: The DPA compiler driver (/opt/mellanox/doca/tools/dpacc) that compiles __dpa_global__ and __dpa_rpc__ device code for a target (-mcpu=nv-dpa-bf3 or nv-dpa-cx8) into a host archive embedding the device ELF; its --app-name token must equal the struct doca_dpa_app symbol passed to doca_dpa_set_app. dpacc-extract recovers the device ELF for gdb-multiarch.; Defined in DOCA 6.3 · DPA, Flex IO, and PCC
DPE (DOCA Privileged Executer): Daemon on the DPU (systemctl start dpe) that runs telemetry providers the DTS container cannot run itself (enable-provider=grpc.<name>) and is required by the BlueMan dashboard.; Defined in DOCA 5.2 · Observability: DTS, BlueMan, Firefly
DPF (DOCA Platform Framework): Kubernetes operator (latest v26.4.1; DOCA 3.5.0 pairs with v26.4.0) that provisions DPUs and DOCA services through CRDs such as DPFOperatorConfig, BFB, DPUCluster, DPUSet, DPU, DPUFlavor, DPUService and DPUServiceChain, in host-trusted (DMS in-band) or zero-trust (BMC Redfish) mode.; Defined in DOCA 5.6 · DPF: DPUs at Kubernetes scale
DPL (DOCA Pipeline Language): A P4-16-derived language for the e-switch's dRMT run-to-completion pipeline, Beta in 3.5.0. Compiled with nvp4c into a .dplconfig and p4info, loaded by dpl_rtd and controlled over P4Runtime (port 9559).; Defined in DOCA 4.5 · DPL: P4 on BlueField
DPU (Data Processing Unit): A NIC with its own Arm CPU complex, memory and hardware accelerators that runs infrastructure software (networking, storage, security) independently of the host CPU. BlueField is NVIDIA's DPU line.; Defined in DOCA 1.1 · Why infrastructure moved onto the NIC
DPU mode / NIC mode: DPU mode: the Arm cores are active and the embedded Arm system runs services that manage the NIC resources and data path (default for DPU SKUs; INTERNAL_CPU_OFFLOAD_ENGINE reads ENABLED(0)). NIC mode: the Arm cores are inactive on BlueField-3 and the device functions as a ConnectX adapter (default for SuperNIC SKUs; reads DISABLED(1)).; Defined in DOCA 1.4 · Modes of operation: DPU, NIC, Zero-Trust
DPU mode / NIC mode / Zero-Trust mode: BlueField operating modes. DPU mode: Arm owns the e-switch and the host sees a NIC through it. NIC mode: Arm is bypassed and the card behaves as a ConnectX. Zero-Trust: DPU mode with host privileges restricted via mlxprivhost.; Defined in DOCA 1.4 · Modes of operation: DPU, NIC, Zero-Trust
dscp2prio map: The NIC's DSCP-to-priority table, printed by mlnx_qos and edited with --dscp2prio set,<dscp>,<prio>. Its default groups DSCP in eights, so 24-31 already land on priority 3 and 48-55 on priority 6 with no configuration.; Defined in RoCE 2.2 · Host QoS with mlnx_qos: trust state, prio2tc, ETS
DTS (DOCA Telemetry Service): Container that collects counters from providers (sysfs, ethtool, nvidia-smi, DCGM, bfperf and others) and exports them via a Prometheus endpoint on port 9100, Fluent Bit, OTLP, Prometheus Remote Write or binary files; configured in dts_config.ini.; Defined in DOCA 5.2 · Observability: DTS, BlueMan, Firefly
Dual plane: Breaking each 800 Gb/s SuperNIC port into 2 x 400 Gb/s interfaces that land on two different leaf switches in two independent fabrics, each scaling to 1,024 interfaces of 400 Gb/s. It is not link bonding for redundancy, and it doubles leaf count against a single-plane design at the same per-GPU bandwidth. Dual plane needs twin-port transceivers; single plane uses single-port OSFP.; Defined in RA 1.4 · The networks in an AI factory, and what the RA refuses to specify
DUP (Dell Update Package): Dell's packaged firmware update. For BlueField, Dell publishes a BlueField-2 NIC-firmware DUP (4RR0M) applicable only via iDRAC/Lifecycle Controller; BlueField-3 firmware is obtained from NVIDIA DOCA downloads (BF-FW-Bundle/BFB) and must be followed by a full power cycle.; Defined in DOCA 8.2 · iDRAC, BIOS, CPLD and the DPU
dynamic ECN: Percentage rather than byte ECN thresholds (min-threshold-percent / max-threshold-percent), requiring Spectrum-4 or later, introduced in Cumulus Linux 5.17.0 and GA in 5.18.0. The NVUE reference contradicts itself on the mode keyword - the syntax says remote or absolute while its own example uses relative - so verify against the running switch before writing either into a runbook.; Defined in Spectrum-X 3.3 · Tuning the lossless edge: PFC headroom, ECN thresholds, watchdog

E

E-switch (embedded switch): The hardware switch inside BlueField/ConnectX that connects uplinks, PFs, VFs and SFs. In switchdev mode the PF acts as the e-switch and manages all ports; in DPU mode the Arm owns it and the configuration must not be changed.; Defined in DOCA 4.1 · E-switch, representors, and switchdev
East-west vs north-south (HGX B300 RA): East-west is GPU-to-GPU traffic between nodes over eight baseboard ConnectX-8 SuperNICs (1:1 with GPUs, up to 800 Gb/s each); north-south is the converged customer and storage network through one BlueField-3 DPU per server.; Defined in DOCA 7.2 · HGX reference architecture roles
ECMP polarization: Every switch computing the same hash over the same fields, so diversity created at one tier is undone at the next and flows pile onto one path. The documented fix is a per-switch hash seed, which requires a switchd restart that interrupts all network services.; Defined in Spectrum-X 2.4 · The underlay: eBGP unnumbered, ECMP and VRFs
ECPF (Embedded CPU Function ownership): Another name for DPU mode: the BlueField Arm subsystem owns the NIC resources and data path. The HGX B300 RA runs its single north-south BlueField-3 in ECPF/DPU mode while the eight ConnectX-8 adapters carry GPU traffic.; Defined in DOCA 7.2 · HGX reference architecture roles
End.DT2M argument transposition: The EVPN multi-homing case where only the argument of the service SID is signalled, transposed into the ESI Label field with the TLV SID set to zero, and reassembled at the receiver from the Route Type 3 SID. RFC 9819 exists because that reassembly was underspecified in RFC 9252.; Defined in SRv6 3.1 · L3VPN and EVPN services over SRv6
Enterprise Reference Architecture: NVIDIA's family of partner-built AI factory design documents (Program Whitepaper, HGX AI Factory, NVL72 AI Factory, and the H100/H200/B200 edition). The program whitepaper scopes itself to deployments of 32 to 256 GPUs; individual platform RAs publish wider ranges of their own.; Defined in RA 1.1 · Which reference architecture governs this deal
Enterprise Support entitlement: The boundary where community and FAE debugging ends: NVIDIA's published answer to a bricked BlueField-3 (PSC BL1 loop, stalled BFB transfer, DRAM not available) was to open an Enterprise Support case handled per entitlement.; Defined in DOCA 9.2 · Scenario: escalations
Escalation runbook (ask / run / do not first): A per-symptom triage sheet with three columns: what to ask the customer, which read-only commands to run, and which destructive actions (reflash, RMA, mode flip, warm reboot) to refuse until the evidence bundle exists, plus an escalate-when line.; Defined in DOCA 9.2 · Scenario: escalations
ETS (enhanced transmission selection): The scheduling algorithm that gives each traffic class a minimum guaranteed share of bandwidth rather than a cap: unused bandwidth from a class below its allocation is available to others. Configured with mlnx_qos --tsa ets and --tcbw percentages summing to 100.; Defined in RoCE 2.2 · Host QoS with mlnx_qos: trust state, prio2tc, ETS
Evidence bundle: The minimum artefacts requested before diagnosis: LC log excerpt, dmesg for mlx5_core, cat /dev/rshim0/misc, an rshim console capture, mlxconfig q, flint q or mlxfwmanager --query, lspci -d 15b3: -nn, and exact BFB and DOCA-Host versions.; Defined in DOCA 9.2 · Scenario: escalations
Evidence pack: The artifacts attached to each qualification row: DOCA Bench CSV, DOCA Telemetry Service metrics export, mlxconfig q output before and after, iDRAC Lifecycle Log export, dmesg mlx5_core lines and the rshim console log.; Defined in DOCA 8.3 · Writing a DOCA validation plan for an OEM server
Extended resource: The countable name a device plugin publishes, shaped as vendor-domain/resourcetype. It is integer only, not overcommittable, and its devices cannot be shared between containers - which is why a pod asking for an unadvertised name stays Pending with no CNI error at all.; Defined in K8sNet 2.1 · How kubelet learns a NIC exists
Extended resource (scheduling contract): A device-plugin-advertised name of the form vendor-domain/resourcetype, such as nvidia.com/rail0 or rdma/rdma_shared_device_a. Extended resources are integer-only, not overcommittable, and devices cannot be shared between containers. Every scheduler in this module — MPI Operator, Run:ai, Slurm on Kubernetes — places work by counting them, so an unadvertised rail is invisible to all of them.; Defined in K8sNet 4.3 · Schedulers on top: MPI Operator, Run:ai and Slurm on Kubernetes
Extended traceroute: Cumulus Linux implementation of the RFC 5837 ICMP interface extensions, enabled with nv set system global icmp ipv4 errors-extension ingress-interface and used with nv action traceroute system <destination> errors-extension. On a BGP-unnumbered fabric it is the only way to name the ingress interface of each hop.; Defined in Spectrum-X 6.1 · The triage toolkit: resources, optics and what tcpdump will not tell you
external action (DOCA Flow): A plugin that extends the standard DOCA Flow action set. SRv6 is implemented as one, registered with doca_flow_external_action_srv6_register(), which MUST be called before doca_flow_init() - calling it afterwards results in undefined behavior rather than an error. It requires vnf or switch mode with hardware steering enabled, and each distinct SRH configuration consumes DOCA_FLOW_SRV6_ACTION_MEM_SIZE (128) bytes of actions memory. Only PUSH is supported in hardware; SRH removal is software.; Defined in SRv6 2.4 · NVIDIA NIC and DPU: SRH push with DOCA Flow

F

FEC (RS / Base-R): Forward error correction modes on Spectrum ports: auto (default), rs (IEEE 802.3 Clause 108 or 91), baser (Clause 74) and off. Explicitly configuring fec auto causes a link flap on every commit, so leave it unset unless the media dictates a mode.; Defined in Spectrum-X 1.3 · Links that come up: breakout, FEC and auto-negotiation
Felix / BIRD / confd / Typha: Calico's four components. Felix programs routes and ACLs on the host for local endpoints. BIRD is the BGP client that distributes Felix's routes to peers. confd watches the datastore for BGP configuration and regenerates BIRD's config. Typha sits between the datastore and Felix to cut per-node datastore load; Calico states one Typha instance can support hundreds of Felix instances and publishes no node-count threshold.; Defined in K8sNet 1.2 · Primary CNI choices: Calico and Cilium
Finding (FAE deliverable): A written escalation result under 300 words with four sections: what was measured with the exact commands, what was inferred and from which documented behaviour, the single change to make, and how the customer verifies it themselves.; Defined in RoCE 5.2 · Scenario: "the 400G NIC only does 180 Gb/s"
Firefly: DOCA PTP/SyncE service packaging ptp4l (linuxptp v4.2), phc2sys, synced and a servo, with default, media (SMPTE 2059-2), l2-telco (G.8275.1) and custom profiles; needs firmware 24.34.1002+ and REAL_TIME_CLOCK_ENABLE=1.; Defined in DOCA 5.2 · Observability: DTS, BlueMan, Firefly
Fixed ToS to sk_prio table: The kernel's four-entry mapping ToS 0 to sk_prio 0, ToS 8 to 2, ToS 24 to 4 and ToS 16 to 6. Because it is fixed, only four ToS values are usable for RoCE QoS mapping; setsockopt with SK_PRIO is the documented bypass.; Defined in RoCE 2.1 · ToS, DSCP, PCP, SL: where marking gets lost
FNM port: The dedicated OSFP in-band fabric-management port on Quantum-X800 switches, separated on the front panel so UFM does not consume data ports. The B300 reference architecture connects UFM 3.5 to four FNM ports on the Q3400.; Defined in InfiniBand 5.2 · Design review: an eight-rail IB pod on Dell XE9680
Force DSCP: A driver-level override that re-marks RoCE traffic without changing the application, written to /sys/class/infiniband/<dev>/tc/<port>/traffic_class either as a global 0-255 value or as a src_ip/dst_ip rule. A global value beats all rules, tclass=-1 deletes a rule, and negative values disable the feature.; Defined in RoCE 2.1 · ToS, DSCP, PCP, SL: where marking gets lost
Four-way version match: The rule that DOCA-Host packages, NIC firmware, the BF-Bundle and the Arm OS release must agree within one DOCA release; read each from its own source (pkg-config, applications/VERSION, doca_caps, flint q, /etc/mlnx-release).; Defined in DOCA 2.1 · DOCA-Host install and profiles
Full versus limited membership: A P_Key match occurs only if at least one of the two nodes is a full member. Full members reach both full and limited members; two limited members of the same partition can never exchange traffic.; Defined in InfiniBand 2.3 · Partitions and PKeys: the most common mistake in IB
Full vs limited membership: Bit 15 of a P_Key is the membership type - 1 means full member, 0 means limited. A P_Key match occurs only if at least one of the sending or receiving nodes is a full member, so full members reach both kinds while limited members reach only full members. The partition-file default for defmember is limited, and RFC 4391 requires a Full Membership P_Key for IPoIB plus a FullMember join of the broadcast group - which is why a partition written without =full produces an interface that comes up cleanly and never resolves ARP.; Defined in InfiniBand 1.4 · IPoIB: the management plane on an IB fabric
FW_RESET_AFTER_CONFIG_UPDATE: The flag set to true on DGX and HGX platforms with an external BMC so the firmware is explicitly reset before the reboot. Without it a configuration change may not persist and the node can reboot in a loop trying.; Defined in K8sNet 3.3 · NIC firmware config in Kubernetes: RoCE, PFC and reboots

G

Generation map: The one-row-per-platform table of adapters: H100/H200/B200 uses B3140H SuperNICs east-west plus B3220 DPUs at 2x200 GbE; HGX B300 uses eight ConnectX-8 SuperNICs at 800 Gb/s per GPU plus a B3240 at 2x400 GbE; GB200 NVL72 uses four ConnectX-7 NDR400 plus two BlueField-3 per tray; GB300 NVL72 uses four ConnectX-8 plus one B3240 per tray.; Defined in RA 1.5 · The generation map: which SuperNIC is on the baseboard
GID (Global Identifier): The RDMA address of a port. On RoCE a GID entry is created automatically for every IP address configured on an Ethernet device of that port, and carries three things: a GID value, a GID type (IB/RoCE v1 or RoCE v2) and the associated netdev. IPv4 addresses appear as IPv4-mapped IPv6 addresses. Read them with show_gids or from /sys/class/infiniband/<dev>/ports//gids/ plus gid_attrs/types/ and gid_attrs/ndevs/.; Defined in RoCE 1.4 · GIDs: how a connection picks its RoCE version
GID index: The position of an entry in a port's GID table. It is the only mechanism a connected queue pair has for choosing a RoCE version: the address vector supplied at INIT to RTR names an index and the GID type at that index sets the RoCE type of the QP. perftest exposes it as -x/--gid-index.; Defined in RoCE 1.4 · GIDs: how a connection picks its RoCE version
GID table: The per-port table exposed under /sys/class/infiniband/{dev}/ports/{port}/ that holds a GID value, a GID type and an associated net device per index. Because the driver associates every index to both RoCEv1 and RoCEv2, each configured IP address produces two entries with identical values and different types.; Defined in K8sNet 2.4 · RoCE, GID tables and the NCCL knobs that read them
Global pause versus PFC: rx_pause_ctrl_phy and tx_pause_ctrl_phy count link-layer 802.3x pause, which is on only when PFC is not. Their presence, together with missing per-priority pause counters, proves a fabric believed to be PFC-based is running global pause.; Defined in RoCE 4.2 · Triage in order: symptom to counter to cause
GPUDirect RDMA: A direct PCI Express data path between GPU memory and a third-party peer device such as a NIC, removing the bounce through host memory. Requires both devices to share the same upstream PCIe root complex and an IOMMU that performs no translation other than 1:1.; Defined in RoCE 3.3 · GPUDirect RDMA: proving it is actually on
GPUNetIO: DOCA library (Beta in 3.5.0) that lets CUDA kernels control NIC receive and send queues directly, built on GPUDirect RDMA, GDRCopy and BlueField DMA. Needs nvidia-open drivers for dmabuf (or nvidia-peermem), a PIX or PXB PCIe topology between GPU and NIC, adequate BAR1, and an active CUDA context; without GPUDirect RDMA it falls back to DOCA_GPU_MEM_TYPE_CPU_GPU with a CPU proxy.; Defined in DOCA 6.4 · GPUNetIO and UROM
GUID (Global Unique Identifier): The 64-bit EUI-64 value maintained by hardware. It is the only persistent identifier of a port, and it can never be used as an address in a packet. Because it is persistent, every fabric configuration file keys on GUIDs - root-GUID files, compute-node GUID files and the port lists in partitions.conf - so inventory and alerting should too. The SM detects duplicate GUIDs and isolates rather than configures the devices carrying them; OpenSM exits on that condition unless -y/--stay_on_fatal is set.; Defined in InfiniBand 1.2 · LIDs, GIDs, GUIDs and LMC

H

hardware track (GB300 / B300 / H200): The three columns of the validated stack. H200 is the BlueField-3 SuperNIC track; GB300 and B300 are ConnectX-8 tracks. Not every point release exists on every track, so choose the track before reading a row.; Defined in Spectrum-X 1.4 · The RA row: version pinning across switch, NIC and host
Hash-Based Forwarding (HBF): Deterministic spreading of flows across an adaptive-routing group by hashing header fields, rather than reacting to congestion. Controlled by hbf_hash_type (0 CRC, 1 XOR), hbf_seed_typ, hbf_seed (0xFFFFFFFF means take the 32 LSB of the node GUID), hbf_hash_fields and hbf_sl_mask. Weighted HBF adds hbf_weights, a three-value ratio across the AR group's subgroups.; Defined in InfiniBand 3.2 · Adaptive routing, SHIELD and hash-based forwarding
HBN (Host-Based Networking): DOCA service that runs FRR-based BGP/EVPN routing, ECMP, L4 firewall and NAT in a container on the BlueField Arm cores, turning the server into its own L3 leaf; requires DPU or zero-trust mode and SFC.; Defined in DOCA 5.1 · HBN: BGP/EVPN on the server
Heavy sweep: A full re-discovery of the fabric, triggered by a trap, by a detected topology change, or manually with SIGHUP. The unicast routing cache (-A) suppresses the routing recalculation when nothing actually changed.; Defined in InfiniBand 2.1 · Bringing up OpenSM
High-frequency telemetry (HFT): Counter collection at single-digit-millisecond to microsecond intervals for catching microbursts: sample-interval-usec 100-12,750 in multiples of 50, default 5000. Spectrum-4 and later, not on 8-lane ports, one job at a time on the ASIC, a maximum of 25 scheduled sessions, and a single gRPC or IPFIX destination.; Defined in Spectrum-X 5.3 · Streaming telemetry: OTLP, high-frequency counters, latency measurement
HLD versus shipping feature: A community SONiC high-level design is upstream design work, not evidence that a distribution such as Dell Enterprise SONiC builds, tests or supports the feature. The two questions that decide it are the ASIC's SAI my_sid_entry support and whether the distribution ships FRR 10.3 or later.; Defined in SRv6 3.3 · Who actually ships SRv6: the Dell and NVIDIA support matrix
host-device: The secondary CNI that moves an existing device out of the host network namespace into the container, selected by device, hwaddr, kernelpath or pciBusID, or by the deviceID runtime capability. One device means one pod. CNI_IFNAME is disregarded so the interface keeps its own name, IPAM is bypassed for DPDK-bound devices, and the device returns to the host namespace on CNI DEL.; Defined in K8sNet 1.5 · macvlan, ipvlan, host-device and cluster-wide IPAM
hw_counters: The optional sysfs directory under an RDMA device or port (/sys/class/infiniband/<dev>/ports//hw_counters/) holding the driver's real RoCE counters: transport errors, RNR, retries, ICRC and the DCQCN four. Names come from the in-tree mlx5_ib driver, not from a vendor page.; Defined in RoCE 4.1 · The counter map: sysfs, ethtool and rdma statistic

I

ibdiagnet: One sweep of switches, HCAs, routers, aggregation nodes and gateways that checks duplicate GUIDs and node descriptions, LID assignment, links in INIT, unresponsive devices, routing-table correctness and credit-loop freedom, width and speed, high-BER links and counter thresholds. It ships in the ibutils2 package.; Defined in InfiniBand 4.3 · ibdiagnet: the command you run before escalating
ibdiagnet2.nodes_info: The dump file carrying nodes information including FW version for every node the sweep saw. It is the fastest fleet-wide firmware inventory available and it includes switches.; Defined in InfiniBand 4.3 · ibdiagnet: the command you run before escalating
ibdiagnet2.sm: The ibdiagnet dump file listing every subnet manager seen in the fabric. It is the proof of a two-master condition, where sminfo only reports the SM your local port is pointing at.; Defined in InfiniBand 5.3 · Escalation bridge call: five InfiniBand tickets
iblinkinfo: Reports link info for each port in the fabric node by node, with partial scans from a start node. -d prints only nodes with a port in the Down state, -l prints one grep-friendly line per link, and its exit status is 0 success, -1 scan failure, 1 inconsistencies found in check mode.; Defined in InfiniBand 4.1 · Discovering a fabric: ibnetdiscover and iblinkinfo
ibnetdiscover: Performs IB subnet discovery and outputs a human-readable topology file: GUIDs, node types, port numbers, port LIDs and NodeDescriptions for all nodes and links. Output goes to standard output unless a topology file is named, and --cache / --load-cache / --diff make before-and-after comparison possible.; Defined in InfiniBand 4.1 · Discovering a fabric: ibnetdiscover and iblinkinfo
ibportstate: Queries or changes port state, physical state, speed and width. Its man page carries the safety fact that an InfiniBand HCA port state may be changed locally without the knowledge of the Subnet Manager, and speed or width changes take effect only after link renegotiation.; Defined in InfiniBand 4.4 · Link surgery and PHY triage
IBV_WC_RETRY_EXC_ERR (12): Completion status meaning the requester retransmitted retry_cnt times within its local ACK timeout and the responder never acknowledged. It is a fabric symptom - loss, blackholing, an MTU or PFC mismatch - and is corroborated by local_ack_timeout_err and req_transport_retries_exceeded on the requester.; Defined in RoCE 1.5 · QP state machine, retries, and connection setup
IBV_WC_RNR_RETRY_EXC_ERR (13): Completion status meaning the responder was reachable but had no receive work request posted, and the requester exhausted rnr_retry after waiting min_rnr_timer each time. It is an application flow-control symptom, corroborated by out_of_buffer on the receiver and rnr_nak_retry_err on the requester.; Defined in RoCE 1.5 · QP state machine, retries, and connection setup
ICMP Parameter Problem Code 4: SR Upper-layer Header Error from RFC 8986 section 4.1.1. Returned when a SID is programmed but the upper-layer header is not permitted on it, so a Code 4 reply to a pinged SID is a success signal, not a failure.; Defined in SRv6 3.2 · Ping the SID: OAM and the five failures that actually happen
IGP MSD types 41 42 44 45: RFC 9352 Maximum SID Depth advertisements: 41 SRH Max Segments Left, 42 SRH Max End Pop, 44 SRH Max H.encaps, 45 SRH Max End D. If no value is advertised the supported value is 0, so silence and zero are indistinguishable to a peer - the classic interop bug where a headend's deep segment list is dropped at the endpoint with nothing logged. uSID largely neutralises it by keeping Segments Left at 0 or 1.; Defined in SRv6 2.2 · FRR: locators, the SID Manager, IS-IS SRv6 and TI-LFA
INTERNAL_CPU_OFFLOAD_ENGINE: The single mlxconfig parameter that sets BlueField-3 mode: 0 (ENABLED) is DPU mode, 1 (DISABLED) is NIC mode; an RO flag means Zero Trust has locked it from the host. Applied by power cycle.; Defined in DOCA 2.5 · Switching modes safely
IPC_LOCK: The Linux capability an RDMA pod must add in securityContext.capabilities so memory registration can pin pages; it is the in-container equivalent of unlimited memlock and its absence produces a registration failure before any data moves.; Defined in RoCE 4.5 · RDMA in Kubernetes: the NVIDIA Network Operator
IPoIB (IP over InfiniBand): A native IP implementation on the InfiniBand link, not a tunnel: the driver sets the interface type to ARPHRD_INFINIBAND with a 20-octet hardware address of 8 reserved bits, a 24-bit QPN and a 128-bit GID, and every IP or ARP datagram is prefixed by a 4-octet encapsulation header. ARP uses hardware type 32. The driver creates one interface per port using the P_Key at index 0; other partitions need a child interface, created with echo <pkey> > /sys/class/net/ib0/create_child and removed with delete_child. It is what carries ssh, schedulers, monitoring and SHARP's UD-multicast result distribution on an InfiniBand-only pod.; Defined in InfiniBand 1.4 · IPoIB: the management plane on an IB fabric
IPoIB datagram vs connected mode: Set and read through /sys/class/net/<intf>/mode. Datagram mode uses the IB UD transport and its MTU is the IB L2 MTU minus the 4-octet encapsulation header, giving the mandatory 2044 bytes on a 2K fabric; stateless offloads are supported only in this mode. Connected mode uses RC and allows an MTU up to the maximal IP packet size of 64K, but keeps the UD queue pair for multicast and for peers that do not support connected mode. NVIDIA notes the physical port MTU defaults to 4k while the IPoIB logical MTU defaults to 2k - mtu=5 versus mtu=4 in an OpenSM partition file.; Defined in InfiniBand 1.4 · IPoIB: the management plane on an IB fabric
ipvlan: macvlan's cousin in which all interfaces share the host's MAC. Modes are l2 (default), l3 and l3s. Two consequences follow from the shared MAC: DHCP needs ClientID which the DHCP plugin does not support, and a container cannot reach its own host over the ipvlan interface - host communication needs a second attachment such as ptp.; Defined in K8sNet 1.5 · macvlan, ipvlan, host-device and cluster-wide IPAM
ISSU / half resource mode: The Cumulus warm-boot upgrade, hitless with sub-second data plane disruption, requiring BGP graceful restart mode full and nv set system forwarding resource-mode half. Half-resource mode reduces forwarding resources by 50 percent and changing it needs a disruptive switchd restart, so it belongs in an earlier window. Spectrum-6 switches do not support ISSU.; Defined in Spectrum-X 5.5 · Upgrades that do not lose the fabric

J

Join-Shortest-Queue (JSQ) approximation: NVIDIA's architectural description of in-switch adaptive routing: a quantized hardware approximation of JSQ that samples egress queue depths and steers each packet to the least-congested port at sub-microsecond intervals.; Defined in Spectrum-X 4.1 · Adaptive routing: per-packet spraying and its eligibility rules

K

kubectl netop-sosreport: The Network Operator's official collector (kubectl-netop_sosreport, or ./network-operator-sosreport.sh standalone). It gathers every Network Operator CR and CRD, DaemonSet and Deployment specs, current and previous container logs, node labels and allocatable resources, and runs lsmod, ibstat, ibv_devinfo, mst status and dmesg on the node. Run it before changing anything, because it captures previous-container logs you are about to destroy.; Defined in K8sNet 4.4 · The triage ladder: sos-report, error strings and the checklists
kubeProxyReplacement: Cilium's eBPF replacement for kube-proxy, enabled with kubeProxyReplacement=true plus k8sServiceHost and k8sServicePort - the last two are required because kube-proxy is normally what provisions the API-server Service. It exits rather than degrading when kernel support is missing, so check uname -r before blaming anything else. The same tuning page carries the kernel cliffs that matter on a PowerEdge: netkit needs 6.8 or newer and BIG TCP needs 6.3 (IPv4) or 5.19 (IPv6) with an mlx4, mlx5 or ice NIC.; Defined in K8sNet 1.2 · Primary CNI choices: Calico and Cilium

L

L3 VNI: The VXLAN network identifier bound to a tenant VRF with nv set vrf RED evpn vni 4001, carried in Type-5 routes together with the router MAC. It cannot share an id with any L2 VNI - if it does, the L2 VNI is silently not created.; Defined in Spectrum-X 2.5 · BGP-EVPN symmetric IRB for tenant isolation
LAG hash randomizer: Enabled together with adaptive routing on Spectrum-4 and later: sprays adaptive-routing-eligible packets across the members of a static layer-3 bond. It coexists with the rule that adaptive routing itself is unsupported on bonds and bond members. Named as a feature in Cumulus Linux 5.16.0.; Defined in Spectrum-X 4.1 · Adaptive routing: per-packet spraying and its eligibility rules
Latency measurement: The Cumulus 5.18 per-link latency feature on Spectrum-4 and later, restricted to layer 3 and untagged physical links, with minimum probe intervals that scale with traffic classes per port (1-4 TCs at least 1 s, up to 13-16 TCs at least 4 s). It cannot measure leaf-to-leaf, spine-to-spine, GPU or server-facing links, link-local IPv6, or one direction alone.; Defined in Spectrum-X 5.3 · Streaming telemetry: OTLP, high-frequency counters, latency measurement
Level 1 / Level 2 dependency compatibility: Level 1 (perpetual): either layer may be upgraded independently within the three-year LTS window; applies to the DOCA-OFED profile versus firmware or BF-Bundle. Level 2 (limited): compatible within one October-to-July cycle; breaking changes may arrive with the October GA; applies to all other profiles, DOCA-DPU and DOCA Services.; Defined in DOCA 1.5 · The DOCA framework map, versions and LTS
LID (Local Identifier): A 16-bit value assigned to a port when it becomes active; it is what steers a packet inside one subnet, since the destination within a subnet is determined by it. LIDs are not guaranteed stable across reboots - the SM may attempt to preserve them but is not required to - and OpenSM's -r/--reassign_lids forces reassignment, which may disrupt a running subnet. Only switch port 0 carries a LID; a switch's external ports are transparent to end nodes and are reached as a port number behind that port-0 LID.; Defined in InfiniBand 1.2 · LIDs, GIDs, GUIDs and LMC
Lifecycle Log (iDRAC): iDRAC's event log; the evidence source for every Dell BlueField-3 KB ('Device not detected: Nvidia Network Adapter', 'A fatal error was detected on a component at bus [n], device 0, function 0') and a required artifact in the qualification evidence pack.; Defined in DOCA 8.2 · iDRAC, BIOS, CPLD and the DPU
lifespan: A file inside hw_counters that caches counter values. It is in milliseconds and defaults to 10; a value between 0 and 10000 may be written to it. Polling faster than the lifespan re-reads the cache and produces an apparently frozen counter.; Defined in RoCE 4.1 · The counter map: sysfs, ethtool and rdma statistic
link breakout: Splitting one cage into several logical ports with nv set interface <id> link breakout 1x|2x|4x|8x (syntax since Cumulus Linux 5.4.0). Sub-ports are named swp1s0, swp1s1 and so on; the Linux equivalent is a line such as 1=4x in /etc/cumulus/ports.conf.; Defined in Spectrum-X 1.3 · Links that come up: breakout, FEC and auto-negotiation
Link utilization threshold: Optional adaptive-routing input, off by default. Turned on globally with nv set router adaptive-routing link-utilization-threshold enabled, after which an adaptive-routing interface defaults to 70 percent and can be overridden per interface in the range 1 to 100.; Defined in Spectrum-X 4.1 · Adaptive routing: per-packet spraying and its eligibility rules
LINK_TYPE_P1: The mlxconfig parameter that sets a port's personality, with IB as 1 and ETH as 2. A change is written to non-volatile configuration and takes effect only after a reboot or flint swreset; the rollback is the value recorded by the preceding query.; Defined in InfiniBand 4.5 · Drivers and firmware: DOCA-OFED, mlxfwmanager, mlxconfig, NVOS
link-down reason code: NVOS per-port diagnostics in three families: PHY Firmware Indication (0-1023), Firmware Management Issues (1024-2047) and Link Down Reason Indication (0-43), with reasons such as Hi_SER_or_Hi_BER, Block_Lock_loss and FEC_sync_loss. The family separates the medium from the cage and the environment.; Defined in InfiniBand 4.4 · Link surgery and PHY triage
LinkX prefixes: The part-number categories that let you read a cable line without a catalogue: MCP passive copper DAC, MCA active copper, MFS fibre AOC, MMA and MMS transceivers, MFP fibre accessories.; Defined in RA 2.5 · Optics, cables and the power nobody budgeted
lkey and rkey: The two keys returned by ibv_reg_mr. The lkey is used by the local queue pair in its scatter-gather entries; the rkey is handed to a peer so it can perform one-sided reads, writes and atomics against that memory region. A local lkey fault shows up as LOC_PROT_ERR, a bad rkey or missing access flag on the peer as REM_ACCESS_ERR.; Defined in RoCE 1.1 · The verbs object model: device, PD, QP, CQ, MR
LMC (LID Mask Control): Sets how many LIDs each port receives: 2^LMC, with a legal range of 0-7 and a default of 0. Extra LIDs let the SM express several distinct routes to the same port, so values above zero should only be used where the topology really provides multiple paths between ports. Trap: LMC greater than zero is not supported by fat-tree routing - specifying it silently causes the default routing algorithm to be invoked instead - and LASH does not support it either. Out-of-range values in a config file are clamped back to the default and logged rather than rejected.; Defined in InfiniBand 1.2 · LIDs, GIDs, GUIDs and LMC
LOC:FUNCT:ARG: The RFC 8986 SID structure: L locator bits, then F function bits, then A argument bits, with L+F+A no greater than 128 and any remaining bits required to be zero. The locator is written B:N where B is the operator's SID block and N identifies the node - the block-len and node-len knobs in FRR, NVUE and SONiC.; Defined in SRv6 1.3 · SIDs as instructions: LOC:FUNCT:ARG and the behavior zoo
locator (FRR): prefix X:X::X:X/M [block-len] [node-len] [func-bits] under segment-routing / srv6 / locators, owned by the zebra SRv6 SID Manager (FRR 10.2) which hands chunks of the block to isisd, bgpd and staticd. FRR requires block-len plus node-len to equal the locator length and their sum with func-bits to be at most 128; defaults are block-len 24, node-len prefix-len minus 24 and 16 function bits - not the F3216 uSID profile, which must be typed as 32 and 16.; Defined in SRv6 2.2 · FRR: locators, the SID Manager, IS-IS SRv6 and TI-LFA
Lossless versus lossy RoCE mode: On NVIDIA switches the two modes differ by exactly two things: lossless adds PFC on the RoCE switch priority and puts that priority in the lossless buffer pool. DSCP and priority mappings, ETS and ECN marking on the RoCE traffic class are programmed in both.; Defined in RoCE 2.3 · PFC, lossless and the case for lossy
lossless-single-ipool (NetQ gap): The single shared buffer RoCE mode that NetQ does not support. NetQ validates lossless and lossy RoCE only, so a fabric running lossless-single-ipool gets no netq check roce result at all and must be verified with per-switch nv show qos roce reads.; Defined in Spectrum-X 5.2 · NetQ 5.1: validation instead of inspection
Lossy (ECN-only) RoCE mode: nv set qos roce mode lossy - keeps ECN marking on TC3, the DSCP mappings and the strict-priority CNP queue while dropping PFC and the lossless buffer pool; it removes the pause mechanism, not congestion control.; Defined in RoCE 5.3 · Scenario: "we do not want to configure PFC"
lossy-multi-tc remark profiles: The two predefined per-interface profiles that mark trimmed packets by port role: lossy-multi-tc-host-group sets DSCP 21 and lossy-multi-tc-network-group sets DSCP 11, both at switch priority 4. They take effect only after nv set system forwarding packet-trim remark dscp port-level.; Defined in Spectrum-X 3.4 · Packet trimming and lossy-multi-tc

M

M_Key protection level: 0 to 3, controlling how much an un-keyed management request may do. Level 0 permits queries and blocks changes; level 2 blocks queries as well, which is the documented reason diagnostics go blind after a key rollout. Level 3 is identical to 2.; Defined in InfiniBand 2.5 · Keys, and locking a fabric down without locking yourself out
macvlan: A secondary CNI that functions like a switch already connected to the host interface: each virtual interface shares the physical device but gets a distinct MAC, so existing DHCP servers work. Modes are bridge (default), private, vepa and passthru. Its mtu field is capped at the master's MTU, and one master cannot be enslaved by macvlan and ipvlan at the same time.; Defined in K8sNet 1.5 · macvlan, ipvlan, host-device and cluster-wide IPAM
Marking chain: The four documented conversions that decide a RoCE flow's queue: application ToS via rdma_set_option, the fixed ToS-to-sk_prio table, sk_prio to user priority via tc or the VLAN egress map, and user priority to traffic class via mlnx_qos or lldpad. Any link can be missing without an error.; Defined in RoCE 2.1 · ToS, DSCP, PCP, SL: where marking gets lost
max_op_vls: The cap on operational virtual lanes. UFM ships it at 2, so a multi-class SL2VL design collapses onto two lanes; the parameter table warns that changing it flaps all fabric links.; Defined in InfiniBand 2.4 · QoS: service levels, virtual lanes, SL2VL and VL arbitration
MFT (Mellanox Firmware Tools): The low-level toolbox (mst, mlxconfig, flint, mlxfwmanager, mlxfwreset, mlxprivhost, mlxlink, mlxdump, mlxreg) that talks to ConnectX and BlueField below the OS driver. Never run two tools on one device at the same time.; Defined in DOCA 2.2 · The MFT toolbox: read-only vs mutating
MLAG: Multi-chassis link aggregation: two Cumulus switches present one LACP identity to a dual-attached host using a shared system MAC from 44:38:39:ff:00:00-44:38:39:ff:ff:ff. It synchronizes dynamic state between peers but never configuration.; Defined in Spectrum-X 2.3 · Layer 2 that survives a reload: bridges, bonds, MLAG and VRR
MLX5_SCATTER_TO_CQE=0: Environment variable that disables the Scatter to CQE feature. NVIDIA's documented workaround when GPUDirect fails in perftest with a 'Couldn't allocate MR' error.; Defined in InfiniBand 5.1 · Proving the fabric: perftest and nccl-tests
mlxlink: MFT tool for checking and debugging link status on passive, active, transceiver and backplane links. -m shows module info, -c shows physical counters and BER, --rx_fec_histogram bins FEC errors; errors, warnings and notes are printed on stderr.; Defined in InfiniBand 4.4 · Link surgery and PHY triage
MMA4Z00-NS: The workhorse 800 Gb/s twin-port OSFP transceiver: 2x400 Gb/s multimode 2xSR4 on 100G-PAM4, two MPO-12/APC connectors, 50 m maximum reach, 15 W, and usable only in Quantum-2 and Spectrum-4 OSFP air-cooled switches. An older PDF snippet quotes 8 W - flag the conflict when the figure enters a power budget.; Defined in RA 2.5 · Optics, cables and the power nobody budgeted
mmap export descriptor: The opaque blob produced by doca_mmap_export_pci (PCIe peer) or doca_mmap_export_rdma (network peer) after doca_mmap_start; the peer rebuilds the memory map with doca_mmap_create_from_export and can then address that memory in DMA or RDMA tasks. It grants access and must travel over a trusted channel.; Defined in DOCA 6.1 · DMA, RDMA, and Verbs
MPIJob (v2beta1): The Kubeflow MPI Operator's CRD for allreduce-style distributed training, carrying mpiReplicaSpecs with Launcher and Worker pod templates, slotsPerWorker and runPolicy.cleanPodPolicy. Its README documents no SSH setup, hostNetwork, IPC_LOCK or RDMA specifics, so the RDMA half of an MPIJob must be sourced from the Network Operator's pod examples instead.; Defined in K8sNet 4.3 · Schedulers on top: MPI Operator, Run:ai and Slurm on Kubernetes
MSD types 41 42 44 45: The IS-IS Maximum SID Depth types for SRv6: SRH Max Segments Left, SRH Max End Pop, SRH Max H.encaps and SRH Max End D. If no value is advertised the supported value is zero, which makes an MSD mismatch a silent packet-dropping interop bug.; Defined in SRv6 3.2 · Ping the SID: OAM and the five failures that actually happen
MTU budget: The PoC plan line written as a subtraction: smallest path MTU minus encapsulation overhead equals the largest inner packet. Overhead is 40 bytes of outer IPv6 plus an 8-byte SRH base plus 16 bytes per segment, or 40 bytes with no SRH for a single uSID carrier.; Defined in SRv6 3.4 · Design and defend an SRv6 PoC for a Dell OEM account
MTU budget (switch / host PF / pod): The rule that three MTUs must agree. Calico subtracts per encapsulation - 20 for IP-in-IP, 50 for VXLAN over IPv4, 70 for VXLAN over IPv6, 60 and 80 for WireGuard over IPv4 and IPv6 - and requires workload and tunnel MTU to be identical. Cilium auto-detects the device MTU instead. A mismatch on the secondary interface produces RDMA that connects and then stalls on large messages. On a Calico manifest install the pod-side value is the veth_mtu key in the calico-config ConfigMap, patched then followed by a DaemonSet restart.; Defined in K8sNet 1.3 · MTU budgeting across switch, host PF and pod
Multiplane: Splitting each SuperNIC into two or more ports on separate network planes, load balanced packet by packet in hardware. Dual- and quad-plane are ConnectX-8 only, with swplb GA and hwplb tech preview; BlueField-3 and ConnectX-7 are single-plane. NVIDIA claims up to 128,000 GPUs in two tiers.; Defined in Spectrum-X 4.5 · Multiplane fabrics and multi-tenant performance isolation
multiplaneMode: The Spectrum-X Operator field that selects how many physical planes a rail spans: none for single-plane deployments on BlueField-3 or ConnectX-7/8 with H100 through B200 and GB200, hwplb for Hardware Multiplane on ConnectX-8 with B300/GB300 (the recommended mode, requiring DOCA xPlane), and swplb for Software Multiplane. The deployed shape is decided by the rail topology entries, not by the declared mode.; Defined in K8sNet 4.1 · Spectrum-X rails inside Kubernetes
Multus (thick plugin / thin plugin): The meta-plugin that attaches additional interfaces to pods by delegating to other CNI plugins. The thick plugin arrived in version 4.0 as two binaries - multus-daemon plus the multus-shim CNI plugin - supports metrics, costs more resources, and is recommended for most environments. The thin plugin is the older single-binary model for constrained nodes.; Defined in K8sNet 1.4 · Multus and the NetworkAttachmentDefinition

N

NCA-AIIO: NVIDIA-Certified Associate: AI Infrastructure and Operations - one hour, 50 questions, $125, valid two years; an entry-level credential for roles from technical pre-sales to data center operations in which the words RDMA, RoCE and InfiniBand appear in no objective.; Defined in RoCE 5.4 · NCP-AIN: what this course covers and what it does not
NCA-AIIO calibration: The honest statement of this course's relationship to the NVIDIA-Certified Associate: AI Infrastructure and Operations exam — 50 questions, one hour, $125, valid two years — whose 22 objectives across three domains weighted 38/40/22 never name CNI, Multus, SR-IOV, RDMA or the Network Operator. The course over-serves objectives such as 2.5, 2.7, 2.8, 2.9, 2.10, 3.1, 3.2 and 3.4 rather than covering the exam.; Defined in K8sNet 4.5 · Dell AI Factory scenarios and the NCA-AIIO positioning
NCCL_IB_ADAPTIVE_ROUTING: NCCL environment variable enabling adaptive-routing-capable transfers over IB Verbs. Enabled (1) by default on InfiniBand networks and disabled (0) by default on RoCE.; Defined in InfiniBand 5.1 · Proving the fabric: perftest and nccl-tests
NCCL_IB_GID_INDEX: The NCCL variable that defines the Global ID index used in RoCE mode, default -1 meaning automatic selection. Which index a container sees is not documented anywhere, so a hardcoded value copied from an old runbook is a measurement claim that was never re-taken.; Defined in K8sNet 2.4 · RoCE, GID tables and the NCCL knobs that read them
NCCL_IB_HCA: The NCCL environment variable that selects which HCAs a job uses, in the format hca with optional port, rail and plane, with ^ to exclude and = to force an exact match. Its default of all available IB interfaces is what lets a job drift off its rail.; Defined in RA 2.1 · Rail-optimized design and what breaks it
NCCL_IB_HCA vs NCCL_SOCKET_IFNAME: NCCL_IB_HCA selects the RDMA data path (max 32 HCAs, caret excludes, equals matches exactly); NCCL_SOCKET_IFNAME selects the IP interfaces used for bootstrap and socket transport. Setting only the latter does not move RDMA traffic.; Defined in RoCE 3.4 · NCCL environment variables on a RoCE fabric
NCCL_IB_TC: NCCL's InfiniBand traffic class, default 0. On RoCE it is the ToS byte of the outer IP header, so ToS = DSCP x 4 and a DSCP-26 fabric needs 104. Left at the default, training traffic rides the default queue no matter how the fabric is configured.; Defined in RoCE 3.4 · NCCL environment variables on a RoCE fabric
NCP-AIN: NVIDIA-Certified Professional: AI Networking - 120 minutes, 70 to 75 questions, $400, valid two years, prerequisite two to three years of NVIDIA data-center operational experience; six domains weighted Design 5, Spectrum 30, InfiniBand 30, Kubernetes 5, Troubleshooting 20, Automation 10.; Defined in RoCE 5.4 · NCP-AIN: what this course covers and what it does not
NCP-AIN domain weights: The six published NCP-AIN domains: AI Data Center Design and Optimization 5%, NVIDIA Spectrum Networking 30%, NVIDIA InfiniBand Networking 30%, Kubernetes Integration 5%, Troubleshooting Tools 20%, Automation and Configuration 10%. The exam is 70–75 questions in 120 minutes for $400, valid two years, with a published prerequisite of two to three years of operational data-center experience with NVIDIA hardware.; Defined in RA 4.5 · NCP-AIN: mapping the blueprint and closing the gaps
NCP-IB retirement: The InfiniBand professional exam is retired and its page redirects candidates to the Professional AI Networking Exam. The skills did not lose value — InfiniBand became a 30% domain inside NCP-AIN — but an IB-only certification path is a path to a dead exam.; Defined in RA 4.5 · NCP-AIN: mapping the blueprint and closing the gaps
NDR / XDR: The current InfiniBand generations, read off NVIDIA's own force_link_speed value tables rather than a slide: force_link_speed_ext maps 4 to 53.125 Gbps per lane (HDR) and 8 to 106.25 (NDR), and force_link_speed_ext2 value 1 is XDR at 212.50. Those are per-lane signalling rates; NVIDIA states the marketed peak port rates as 400 Gbps per direction for NDR and 800 for XDR. All three force_link_speed parameters default to 'use what the port reports as supported' and are marked Do NOT change - forcing a speed is how you lose a rate, not gain one.; Defined in InfiniBand 1.3 · Speeds, widths, and the link that says SDR
netq check: The NetQ command family that validates a protocol or service across every device that reports in, rather than reporting one switch's state. Objects include addresses, agents, bgp, clag, cl-version, evpn, interfaces, mlag, mtu, ntp, roce, sensors, topology, vlan and vxlan, each running numbered tests.; Defined in Spectrum-X 5.2 · NetQ 5.1: validation instead of inspection
netq check roce: Six cross-switch consistency tests - 0 RoCE mode, 1 classification, 2 congestion control (ECN and RED thresholds), 3 flow control (PFC), 4 ETS mode, 5 miscellaneous. It is the only practical way to find one misconfigured leaf, and it does not support lossless-single-ipool, so fabrics in that mode get no RoCE validation at all.; Defined in Spectrum-X 3.5 · Playbook: 'RoCE is slow'
NetworkAttachmentDefinition (NAD): The custom resource that names a secondary network and carries its CNI JSON, normally inline in spec.config. A NAD with no spec is instead backed by a matching config file in /etc/cni/multus/net.d/ on the node. Pods reference it through the k8s.v1.cni.cncf.io/networks annotation, which supports a comma list, name@ifname interface pinning, namespace/name qualification, and a JSON list carrying name, namespace, interface and default-route.; Defined in K8sNet 1.4 · Multus and the NetworkAttachmentDefinition
NIC Configuration Operator: Component of the NVIDIA Network Operator that applies NIC firmware and nvconfig settings through Kubernetes CRDs (NicConfigurationTemplate, NicDevice, NicFirmwareTemplate, NicFirmwareSource, NicInterfaceNameTemplate); enabled via NicClusterPolicy.; Defined in DOCA 7.3 · The validated stack and version pinning
NIC mode only (platform matrix line): The Network Operator platform-support statement that BlueField-3 and BlueField-3 SuperNIC are supported in NIC mode only. It is the scoping rule that separates the two operator stories: a customer who wants a fast NIC for host pods is in Network Operator territory, and one who wants services running on the card is in DPF territory.; Defined in K8sNet 4.2 · BlueField in Kubernetes: DPF versus Network Operator
NIC-to-GPU distance (PIX / PXB / PHB / NODE / SYS): The nvidia-smi topo -m legend, best to worst for GPUDirect: a single PCIe switch, multiple switches without the host bridge, across a PCIe host bridge, across host bridges within a NUMA node, and across the SMP interconnect between NUMA nodes. NCCL_NET_GDR_LEVEL uses the same vocabulary.; Defined in RoCE 3.3 · GPUDirect RDMA: proving it is actually on
NicClusterPolicy: The Network Operator custom resource describing cluster-wide NIC configuration, including the rdmaSharedDevicePlugin block whose rdmaHcaMax sets how many pods share an HCA and whose selectors.ifNames names the netdev rather than the RDMA device.; Defined in RoCE 4.5 · RDMA in Kubernetes: the NVIDIA Network Operator
NicConfigurationTemplate: The CR (configuration.net.nvidia.com/v1alpha1) that pushes firmware settings to selected NICs. It carries nodeSelector, nicSelector (nicType device id or pciAddresses), resetToDefault, and a template with numVfs, linkType, pciPerformanceOptimized, roceOptimized, gpuDirectOptimized, rawNvConfig and force.; Defined in K8sNet 3.3 · NIC firmware config in Kubernetes: RoCE, PFC and reboots
NicNodePolicy: Per-node-group policies for heterogeneous clusters, each selecting nodes by label. Names are at most 30 characters, two instances must not select overlapping nodes (validated at admission and continuously at runtime), and a given section lives in either NicClusterPolicy or NicNodePolicy but never both.; Defined in K8sNet 3.2 · Version matrices, upgrades and heterogeneous clusters
nl2docad: Netlink-to-DOCA daemon inside HBN that watches kernel routing and EVPN state and programs matching packet-processing rules into BlueField hardware through DOCA APIs.; Defined in DOCA 5.1 · HBN: BGP/EVPN on the server
No Memory Found (Dell KB 000227031): A POST event on R660/R760/R760XA/XE9680 with BlueField-3 in DPU mode caused by unsynchronized BlueField-3, iDRAC and CPLD state machines after a graceful DPU shutdown; the server reboots itself after about a minute. Fix is in a CPLD release.; Defined in DOCA 2.6 · Dell-specific bring-up: iDRAC, BIOS, CPLD, aux power, DSE
No Memory Found (KB 000227031): POST event on BlueField-3 enabled PowerEdge R660/R760/R760XA/XE9680 when the BlueField-3, iDRAC and CPLD state machines are not synchronized after the DPU graceful shutdown; the server reboots itself after about a minute. Impacted: CPLD 1.1.5/1.1.7, iDRAC 7.10.50.00, firmware 32.40.1000; fix is a CPLD release.; Defined in DOCA 8.2 · iDRAC, BIOS, CPLD and the DPU
No SHA engine (BlueField-3): The DOCA SHA library (SHA1, SHA2-256, SHA2-512; GA) does not run on BlueField-3 because the silicon has no SHA acceleration engine; hashing stays on a CPU or moves to BlueField-2. No firmware setting changes this.; Defined in DOCA 6.2 · Compress, SHA, Erasure Coding, AES-GCM
no-sid-drops: The Cumulus Linux SRv6 drop counter (nv show router segment-routing srv6 stats no-sid-drops) counting packets that arrived carrying a SID the switch does not have - the signature of a locator or length mismatch between the NIC that built the list and the switch executing it. More diagnostic than a zero hit counter, which cannot distinguish no traffic from wrong SID. With packet trimming enabled, Cumulus counts a trimmed packet twice in SRv6 statistics.; Defined in SRv6 2.3 · NVIDIA switching: uN and uA on Spectrum-4
Node nomenclature (CPUs-GPUs-NICs-speed): The Enterprise RA naming convention for a node configuration, for example HGX 2-8-9-400 = 2 CPUs, 8 GPUs, 9 NICs, 400 GbE east-west per GPU. Other published strings are PCIe-Optimized 2-4-3-200, PCIe-Optimized 2-8-5-200, Scale-out Grace 2-2-3-400 and the GB300 tray's 2-4-5-800 (dual plane).; Defined in RA 1.2 · Reading node nomenclature: 2-8-9-400
Non-commitment notice: The statement carried in NVIDIA documents that they are not a commitment to develop, release or deliver any material and that specifications may change without notice. Quoted whenever a customer asks for roadmap guarantees (for example BlueField-4).; Defined in DOCA 9.3 · Scenario: sizing and positioning
nv set qos roce: The single NVUE command that configures RoCE on Cumulus Linux, equivalent to mode lossless. It programs trust mode, RoCE data on switch priority 3 with DSCP 26, CNP on priority 6 with DSCP 48, an ETS scheme, an ECN threshold on TC3, and in lossless mode PFC plus the lossless buffer pool.; Defined in RoCE 4.4 · The switch half: Cumulus NVUE and Dell Enterprise SONiC
nv-ipam (IPPool / CIDRPool): NVIDIA's cluster-wide IPAM, made of ipam-controller (assigns each node a unique range from the pool CRs), ipam-node (allocates and persists over gRPC) and the nv-ipam CNI binary. IPPool hands each node a count of addresses via perNodeBlockSize; CIDRPool hands each node a subnet via perNodeNetworkPrefix. poolType selects which, ippool or cidrpool. Network Operator deploys it through NicClusterPolicy.spec.nvIpam.; Defined in K8sNet 1.5 · macvlan, ipvlan, host-device and cluster-wide IPAM
nv-wjh-cli: The 5.18 local WJH command (nv-wjh-cli poll --data drops|aggregates|l1 --output-type table|json|pcap), running under the systemd unit nv-wjh.service. Runbooks written for earlier releases call it what-just-happened poll, which does not exist on 5.18.; Defined in Spectrum-X 5.1 · What Just Happened: naming the drop
nvconfig: NIC firmware configuration - where numVfs, linkType, PFC and trust actually live. It is not CNI state and not driver state, which is why applying a NicConfigurationTemplate means a node reboot rather than a reload.; Defined in K8sNet 3.3 · NIC firmware config in Kubernetes: RoCE, PFC and reboots
NVIDIA Channel card: Dell's term (KB 000225111) for a BlueField DPU not sourced through a Dell SKU; Dell has not validated VMware DSE on such cards and cannot support it if enabled.; Defined in DOCA 8.1 · PowerEdge support matrix
NVIDIA-Certified Storage levels: Two tiers in the Enterprise RA program: Foundation certifies the 2-4-3-200 and 2-8-5-200 node types, Enterprise certifies 2-8-9-400. The level a storage vendor holds therefore constrains which node configuration they can be sold against.; Defined in RA 1.2 · Reading node nomenclature: 2-8-9-400
nvidia-peermem: The legacy GPUDirect RDMA kernel module from the GPU driver. Works with any supported driver and has no CUDA or kernel minimum, but requires MLNX_OFED or DOCA-OFED to be installed. Enabled with --set driver.rdma.enabled=true and visible as an nvidia-peermem-ctr container in the driver daemonset.; Defined in K8sNet 3.4 · GPUDirect RDMA: dma-buf, nvidia-peermem and GDS
nvidia.nvue collection: NVIDIA's Ansible collection for NVUE (ansible-galaxy collection install nvidia.nvue), providing nvidia.nvue.command (a wrapper around the NVUE CLI with templating) and nvidia.nvue.api (a wrapper around the REST API) plus object modules. NVIDIA describes the modules as community supported and not NVIDIA validated, best-effort.; Defined in Spectrum-X 5.4 · Automating NVUE: REST API, config templates and Ansible
NVLink Switch domain: The set of GPUs one job can address coherently over NVLink. The NVLink 4 Switch builds 8-GPU domains at 7.2 TB/s aggregate; the NVLink 5 Switch builds 8 or 72 GPU domains reaching 130 TB/s aggregate in an NVL72 rack. Above the domain size the fabric choice becomes the performance decision.; Defined in RA 1.3 · Scale-up is not scale-out: NVLink vs the fabric
NVUE: NVIDIA User Experience: an object-oriented, schema-driven model of a complete Cumulus Linux system, defined with OpenAPI 3.0.2 and exposed identically through the nv CLI and a REST API on port 8765.; Defined in Spectrum-X 2.1 · The NVUE object model, revisions and snippets
NVUE REST API (/nvue_v1): The HTTP interface to the same OpenAPI 3.0.2 object model the CLI drives, enabled by default on port 8765 with HTTP basic auth against switch credentials. Paths are /nvue_v1/<resource>/<path> and revisions are selected with ?rev=; after upgrading to 5.6 or later an unchanged cumulus password returns 403 on every call.; Defined in Spectrum-X 5.4 · Automating NVUE: REST API, config templates and Ansible
NVUE snippet: Raw text NVUE writes verbatim into a Linux configuration file it does not model, such as /etc/frr/frr.conf. Set or unset only, one snippet per file, 1 MB cap on flexible snippets, and blocked from /bin /sbin /boot /root /sys /proc /dev.; Defined in Spectrum-X 2.1 · The NVUE object model, revisions and snippets

O

O-flag: Bit 2 of the SRH Flags octet, defined by RFC 9259. A supporting endpoint copies the packet with a timestamp to an OAM process and forwards the original unchanged. Implementation is optional; a node that does not support it ignores it, and support may be advertised in the IS-IS SRv6 Capabilities sub-TLV type 25.; Defined in SRv6 3.2 · Ping the SID: OAM and the five failures that actually happen
octets divided by 4: The unit of the Data components of PortCounters (PortXmitData, PortRcvData and the SL variants). Multiply the counter by four before comparing it with bytes reported by a benchmark.; Defined in InfiniBand 4.2 · Counters and errors that mean something
OEM support boundary: Hardware, RMA and entitlement follow the sales channel to Dell for PowerSwitch SN-series; Cumulus Linux behaviour and its release lifetime belong to NVIDIA - Dell's own install KB 000376999 reproduces the NVIDIA ONIE procedure.; Defined in Spectrum-X 1.5 · Dell PowerSwitch SN-series: who supports what
Offload break-even: The point where a DPU's annual value (cores freed times per-core value, plus net watts saved, plus isolation value) exceeds its premium and its own power draw (150 W for B3220/B3240, 75 W for the B3140H SuperNIC per Dell). With no Arm-side services the cores-freed term is zero and NIC mode or a ConnectX wins; every vendor or partner number in the case must carry its label, hardware generation and workload.; Defined in DOCA 6.5 · Offload economics
One-way latency (perftest): Half of a measured round trip. perftest times the round trip and reports half of it, so the figure is only meaningful when both directions of the path are symmetric.; Defined in RoCE 3.1 · What ib_write_bw actually measures
ONIE: The installer environment on the switch that finds and installs a network operating system, via DHCP option 114, onie-nos-install, onie-install or a USB stick holding an image renamed onie-installer-x86_64. An ONIE install does not retain configuration files.; Defined in Spectrum-X 2.2 · Day-0: ONIE install, first boot and ZTP
Operator-owned network CR: MacvlanNetwork, HostDeviceNetwork and IPoIBNetwork (apiVersion mellanox.com/v1alpha1). Each generates the NetworkAttachmentDefinition for you, so in the operator flow you hand-write a NAD only for the IB SR-IOV case.; Defined in K8sNet 3.1 · Network Operator and the NicClusterPolicy
OPN (ordering part number): The 900-9D3xx-00xx-xxx string that identifies a BlueField-3 board, its crypto-enabled or crypto-disabled variant and form factor; the specifications page maps each OPN to a SKU.; Defined in DOCA 1.3 · BlueField-3 under the hood
/opt/mellanox/doca: Install root for DOCA on both host and BlueField: lib/<arch>-linux-gnu for libraries (trace/ subdirectory for the trace flavor), infrastructure/include for headers, infrastructure/lib/pkgconfig for .pc files, samples/, applications/ and tools/ (doca_caps, doca_bench).; Defined in DOCA 3.1 · Development environments
Out-of-band network: The physically isolated 1 GbE RJ45 management network into SN2201 48-port switches, implemented as a dedicated VXLAN at the aggregation layer. Counted in jacks: at minimum the system BMC and the BlueField-3 BMC per node, plus every switch, PDU, storage controller and management server.; Defined in RA 2.4 · The other three networks: converged, storage, out-of-band
Out-of-band vs RDMA-CM connection (DOCA RDMA): Two ways to connect DOCA RDMA contexts: out-of-band exports a sensitive connection blob with doca_rdma_export that the application transports and the peer feeds to doca_rdma_connect; RDMA-CM uses doca_rdma_start_listen_to_port on a server and doca_rdma_addr_create plus doca_rdma_connect_to_addr on clients, with connection-state callbacks and doca_rdma_connection_accept.; Defined in DOCA 6.1 · DMA, RDMA, and Verbs
OVS-DOCA: The OVS datapath offloading layer (DPIF) built on DOCA Flow; GA in 3.5.0. Enabled with other_config:doca-init=true and hw-offload=true plus a restart; verified with ovs-appctl dpctl/offload-stats-show.; Defined in DOCA 4.4 · DOCA Ethernet and OVS-DOCA

P

P_Key (partition key): A 16-bit key carried in the packet BTH and enforced by both HCA and switch, where bits 14:0 are the partition value and bit 15 is the membership type. Enforced on all IB packets except SMP MADs.; Defined in InfiniBand 2.3 · Partitions and PKeys: the most common mistake in IB
packet trimming: Spectrum-4-and-later mechanism that truncates a packet about to be discarded for want of buffer and forwards the header so the receiver retransmits fast instead of waiting for a timeout. Known unicast IPv4/IPv6 only; excludes ISSU, VXLAN, flooding and multicast. Default profile: DSCP 11, 256 bytes, switch priority 4, traffic classes 1-3. Consumes one service port.; Defined in Spectrum-X 3.4 · Packet trimming and lossy-multi-tc
Partner DPU (consignment): Dell quote line for an NVIDIA-channel BlueField-3 DPU (e.g., 352-BBFH / XWFTP) consigned into the Dell factory build; ships in DPU mode; Dell states it is validated and supported with a Dell system, but features Dell did not validate (such as VMware DSE) stay unsupported on it.; Defined in DOCA 8.1 · PowerEdge support matrix
Partner DPU / consignment card: Dell's term for an NVIDIA BlueField-3 sold through Dell's configurator (for example 352-BBFH, dual-port 400GbE) and supported by Dell Technical Support when used with a Dell system.; Defined in DOCA 9.1 · Scenario: a qualification request
Pause direction rule: rx_prio[p]_pause rising means the network is pausing this adapter; tx_prio[p]_pause rising means this adapter is pausing the network. The direction, not the magnitude, identifies which side of a link is the victim.; Defined in RoCE 4.2 · Triage in order: symptom to counter to cause
Pause storm: A sustained PFC condition in which a device transmits pause frames long enough to hit a timeout that disables pause transmission, during which drops can occur. Counted by tx_pause_storm_warning_events and tx_pause_storm_error_events - a lossless fabric failing over into loss.; Defined in RoCE 2.3 · PFC, lossless and the case for lossy
PCC (Programmable Congestion Control): A DOCA DPA application (Beta in 3.5.0) implementing congestion control on the NIC: the reaction point (RP) adjusts transmit rate, the notification point (NP) answers congestion notifications. Requires BlueField-3 or later, Ethernet link, firmware 32.38.1000 or later, and mlxconfig USER_PROGRAMMABLE_CC=1 (RP) or PCC_INT_EN=0 (NP) followed by a power cycle.; Defined in DOCA 6.3 · DPA, Flex IO, and PCC
PCIe ACS: Access Control Services on PCIe bridges, which forces peer transactions up to the root complex and therefore breaks GPUDirect. Detected with lspci -vvv | grep ACSCtl and disabled at runtime with setpci ECAP_ACS+0x6.w=0000; inside a VM it cannot be disabled and ATS is enabled instead.; Defined in RoCE 3.3 · GPUDirect RDMA: proving it is actually on
peerlink.4094: The mandatory subinterface of the MLAG peer-link bond named peerlink. VLAN 4094 must not appear in the bridge VLAN list and cannot be configured as a bridged VLAN.; Defined in Spectrum-X 2.3 · Layer 2 that survives a reload: bridges, bonds, MLAG and VRR
perfquery reset mask: The third positional argument to a perfquery reset. 0x0fff resets only the error counters of the named port and 0xf000 resets only the non-error counters; a clear is irreversible, so the pre-values must be recorded first.; Defined in InfiniBand 4.2 · Counters and errors that mean something
perftest: The linux-rdma collection of uverbs micro-benchmarks (ib_write_bw, ib_send_lat and friends) used for hardware and software tuning and for functional testing. Its README states the benchmarks generate a synthetic stream and are not designed to emulate any real application traffic.; Defined in RoCE 3.1 · What ib_write_bw actually measures
perftest NUMA auto-binding: Current perftest detects the IB device's NUMA node, binds the benchmark thread and its memory allocations to it, and reports the node in the output header. It skips binding when the node is undeterminable or when taskset or numactl already set an affinity.; Defined in RoCE 3.2 · NUMA, data validation, and doca_perftest
perftest six-check list: The order in which an FAE rules out a bandwidth complaint: units (--report_gbits versus MiB/s), queue-pair count (-q defaults to 1), message size (-s or -a), the NUMA line in the output header, GID index and traffic class (-x and --tclass), and matching perftest versions on both sides.; Defined in RoCE 5.2 · Scenario: "the 400G NIC only does 180 Gb/s"
PFC (IEEE 802.1Qbb): Priority-based Flow Control, approved 2011-06-16: a pause mechanism similar to 802.3x PAUSE but acting on individual priorities. It is hop-by-hop, so a path is lossless only if every endpoint and every switch in it has PFC enabled on that priority.; Defined in RoCE 2.3 · PFC, lossless and the case for lossy
PFC headroom: Buffer reserved to absorb the frames already in flight between sending an XOFF and the upstream port actually stopping. Sized from cable length (nv set qos roce cable-length, default 100 m) plus the xoff/xon thresholds, and since 5.17 also from small-packet-probability, which reached GA in 5.18.; Defined in Spectrum-X 3.3 · Tuning the lossless edge: PFC headroom, ECN thresholds, watchdog
PFC watchdog: Detection and mitigation for a pause deadlock. Polls every 100 ms with a robustness of 3 intervals, works only on lossless traffic queues and only on physical swp interfaces, and defaults to disabled per interface. Read deadlock occurrence counts with nv show interface <if> qos pfc-watchdog.; Defined in Spectrum-X 3.3 · Tuning the lossless edge: PFC headroom, ECN thresholds, watchdog
Pipe domain: Where a pipe executes: DEFAULT (ingress), SECURE_INGRESS (decrypt), EGRESS (encap), SECURE_EGRESS (encrypt). Ingress and egress each have one root slot, and no egress-to-ingress transition is allowed.; Defined in DOCA 4.3 · Domains, connection tracking, aging, meters
Plane load balancing (PLB): The third Spectrum-X loop: a dedicated hardware engine in the SuperNIC that distributes traffic across network planes using local queue feedback and per-plane congestion telemetry, and temporarily filters a congested or failed plane out of the eligible set.; Defined in Spectrum-X 4.3 · Telemetry-based congestion control: three loops, not one
Port state vs physical state: Two independent readings of the same port. Physical state (Sleep, Polling, LinkUp) answers whether the two ends trained; port state (DOWN, INIT, ARMED, ACTIVE, ACTIVE_DEFER) answers whether anything has configured the port. LinkUp with Initializing therefore means a healthy cable and no Subnet Manager - the single most misread pair on a bring-up, and an SM question rather than a cable question. Both are readable from sysfs at ports/<n>/phys_state and ports/<n>/state, alongside rate, lid and sm_lid, with no diagnostic package installed.; Defined in InfiniBand 1.5 · Reading a host's IB state in 60 seconds
Positioning memo: A one-page recommendation for a Dell SE choosing among NIC mode, DPU mode and ConnectX-8: recommendation, why, pinned versions, objection answers, Dell support statement, and a section of items not confirmed by a citable page.; Defined in DOCA 9.3 · Scenario: sizing and positioning
PowerEdge XE9680: Dell's 6U AI server for the Hopper generation. Published accelerator options are 8x HGX H100 80GB or 8x HGX H200 141GB 700W SXM5, 8x AMD MI300X or 8x Intel Gaudi3, with up to 10 Gen5 x16 slots (two of them called out for SmartNIC/DPU) and 2800W Titanium PSUs. It is the compute half of Dell AI Factory Solution 19845005.1.; Defined in RA 3.1 · The PowerEdge AI server line
PowerEdge XE9780 / XE9785: Dell's 10U air-cooled B300-generation AI servers. The XE9780 takes eight HGX B300 NVL8 270 GB 1100W GPUs or eight B200, with 8x ConnectX-8 OSFP ports for east-west and a slot priority of 6, 9, 3, 12 (max 4) for BlueField-3 DPUs and OSFP NICs. The XE9785 is the AMD MI355X or B300 variant with 8x CX-8 OSFP by default.; Defined in RA 3.1 · The PowerEdge AI server line
PowerScale back-end network: The private network that acts as the backplane of a PowerScale cluster, built on redundant switches with int-a, optional int-b and failover networks on separate IPv4 subnets, carrying RBM over TCP on Ethernet. Dell does not support connecting any other devices to the back-end switches and states that altering its configuration can cause a cluster-wide outage.; Defined in RA 3.4 · PowerScale back-end networking and the version-pin trap
PowerScale NOS-pin conflict: Dell qualifies Cumulus Linux 5.9.1 on SN5600 for the PowerScale back-end network while the AI compute fabric follows the current Spectrum-X validated-stack row. The two pins differ, so one switch cannot serve both roles — a design-review catch, not an install-day discovery.; Defined in RA 4.4 · Scenario: a 256-GPU Dell AI Factory design review
PPCC register: The firmware register through which ZTR-RTT parameters and counters are read and written with mlxreg: cmd_type 8 sets a parameter, 1 with counter_en=1 enables debug mode and counters, 13 resets them, 12 queries them, and 2 on algo_slot 0 disables deployment mode.; Defined in RoCE 2.5 · Zero Touch RoCE, ZTR-RTT CC and DOCA PCC
primary CNI vs secondary network: The two-layer split that defines this course. The primary CNI owns eth0, Services, DNS and NetworkPolicy for every pod. RDMA and GPU east-west traffic never goes through it: it rides a secondary interface attached by Multus onto an SR-IOV VF, a host device, or a macvlan/IPoIB child link. NVIDIA does not replace the primary CNI - it adds the second path.; Defined in K8sNet 1.1 · The Kubernetes network model and where a second CNI fits
Priority trust state: The NIC setting that decides whether packets are prioritized on the VLAN PCP bits or on the IP DSCP bits. PCP is the default, DSCP is always explicit, and it must be set before SR-IOV is enabled so virtual functions inherit it.; Defined in RoCE 2.2 · Host QoS with mlnx_qos: trust state, prio2tc, ETS
progress engine (doca_pe): Object that drives task execution and delivers every completion and event callback strictly inside doca_pe_progress. Contexts attach with doca_pe_connect_ctx before start; event-driven mode uses doca_pe_get_notification_handle, doca_pe_request_notification and doca_pe_clear_notification.; Defined in DOCA 3.3 · Tasks, the progress engine, and completion
Progress engine (PE): DOCA Core object that drives task completion; the application polls it (or waits on its event handle) to run completion callbacks.; Defined in DOCA 3.3 · Tasks, the progress engine, and completion
proprietary packages (DOCA-Host): Packages the DOCA-Host installer does not install and that are installed on request, including opensm, ibutils2, hcoll and sharp. Because ibdiagnet lives in ibutils2, a correct doca-all install can still leave a host with no subnet manager and no fabric diagnostics.; Defined in InfiniBand 4.5 · Drivers and firmware: DOCA-OFED, mlxfwmanager, mlxconfig, NVOS
Protection domain (PD): The scope that binds memory regions, queue pairs, shared receive queues and address handles together. Keys are validated against the PD, so an lkey or rkey from one domain is meaningless to a queue pair in another. ibv_advise_mr takes the PD rather than the MR as its first argument.; Defined in RoCE 1.1 · The verbs object model: device, PD, QP, CQ, MR
PSID: A 16-ASCII-character string embedded in a firmware image that uniquely identifies the configuration of that firmware for a specific card and OEM variant. It appears in every mlxfwmanager query and is the identity to quote in a support case.; Defined in InfiniBand 4.5 · Drivers and firmware: DOCA-OFED, mlxfwmanager, mlxconfig, NVOS
PSP (Penultimate Segment Pop): A flavor of End, End.X and End.T in which the penultimate endpoint removes the SRH once Segments Left reaches zero. Deterministically controlled by the SR source node, which inserts a PSP-flavored SID; a node offering both advertises them as two different SIDs. In Linux only PSP is implemented of the three flavors, and only on End - End.X flavors psp returns Unsupported Flavor operation(s).; Defined in SRv6 1.3 · SIDs as instructions: LOC:FUNCT:ARG and the behavior zoo
Pure SONiC vs Dell Enterprise SONiC: NVIDIA's SN6000 datasheet offers Cumulus Linux or its community-based Pure SONiC with A2P support; Dell's sheet substitutes Enterprise SONiC Distribution by Dell Technologies, whose published 4.4.0 platform matrix lists only Broadcom-NPU switches. No SONiC appears in any Spectrum-X RA row.; Defined in Spectrum-X 1.5 · Dell PowerSwitch SN-series: who supports what
PUSH / NEXT / CONTINUE: The three SR primitives. PUSH imposes a segment (outermost label, or the first segment in the SRH). NEXT is a POP of the top label in SR-MPLS and a copy of the next segment from the SRH into the IPv6 destination address in SRv6. CONTINUE is a SWAP in SR-MPLS and plain IPv6 forwarding on the destination address in SRv6.; Defined in SRv6 1.1 · Source routing, segments, and the two data planes
PXN (PCI x NVLink): NCCL technique that reaches a NIC through NVLink and then PCI rather than through the CPU, moving data to a GPU on the destination's rail before it leaves the node; it removes cross-rail spine traffic and aggregates messages for more than 2X better all2all latency.; Defined in RoCE 5.1 · Rail-optimized fabrics on Dell hardware

Q

QP state machine: RESET to INIT to RTR to RTS, plus SQD, SQE and ERR. Each transition is an ibv_modify_qp call with a transport-specific attribute mask: RC needs AV, PATH_MTU, DEST_QPN, RQ_PSN, MAX_DEST_RD_ATOMIC and MIN_RNR_TIMER at INIT to RTR and the four retry knobs at RTR to RTS, while UD needs only state and SQ_PSN.; Defined in RoCE 1.5 · QP state machine, retries, and connection setup
Qualification (design-in): The FAE-owned process of proving a specific BlueField SKU works in a specific OEM server: scoping questions, a test matrix (mode, profile, firmware/BFB, server firmware, OS, slot/power), pass/fail evidence, KB-mapped risks, and deliverables.; Defined in DOCA 9.1 · Scenario: a qualification request

R

r_roce_ecn_rp and r_roce_ecn_np: The two host-side DCQCN algorithms exposed under /sys/class/net/<if>/ecn/, enabled per priority 0-7 with echo 1 into enable/<X> and tuned under params/. NVIDIA's current Linux pages publish no parameter names or defaults, so the list must be read off the host.; Defined in RoCE 2.4 · ECN, CNP and DCQCN: closing the loop
RA row: One line of the Spectrum-X Validated Solution Stack pinning Cumulus Linux, NIC firmware, DOCA-Host, NetQ, NCCL, HPC-X, Network Operator and DTS together for one release on one hardware track. The RA number is a bundle identifier, not a switch-OS version.; Defined in Spectrum-X 1.4 · The RA row: version pinning across switch, NIC and host
Rail (rail-optimized topology): A fabric lane in which NIC-0 of every node connects to the same leaf switch L0, NIC-1 of every node to L1, and so on, so that same-rail node-to-node traffic is one hop and flows on different rails do not interfere.; Defined in RoCE 5.1 · Rail-optimized fabrics on Dell hardware
Rail-optimized: A cabling contract, not a switch feature: every node's rail-N port lands on leaf switch N, so same-rail traffic inside a scalable unit is one hop. Traffic between nodes or between rails traverses the spine layer.; Defined in InfiniBand 5.2 · Design review: an eight-rail IB pod on Dell XE9680
Rail-optimized topology: A cabling contract, not a switch feature: each GPU's rail-N port lands on leaf N, so traffic per rail is always one hop away from the other nodes in the scalable unit while traffic between nodes or rails traverses the spine. No SM parameter reports it; confirm it from the cabling map or with ibdiagnet --rail_validation, which writes ibdiagnet2.rails.; Defined in InfiniBand 3.4 · Topologies: fat-tree, rail-optimized, twin-plane
RDMA netns design fork: The mutually exclusive choice between rdma system set netns shared (required by the RDMA shared device plugin, which over-advertises one HCA as a scheduling counter) and netns exclusive (required for per-pod isolation with SR-IOV plus a chained rdma CNI). Exclusive mode makes rdma-shared-dp-ds crash-loop with Exiting.. can not change : incorrect RDMA subsystem network namespace, and switching to exclusive while namespaces and RDMA devices already exist returns EBUSY.; Defined in K8sNet 4.4 · The triage ladder: sos-report, error strings and the checklists
RDMA netns mode: The RDMA subsystem's namespace setting, read with rdma system show and set with rdma system set netns shared or exclusive. Exclusive makes RDMA devices visible in a single network namespace and is what per-pod isolation requires; changing shared to exclusive returns EBUSY once active network namespaces exist with RDMA devices present.; Defined in K8sNet 2.3 · RDMA in pods: shared plugin vs exclusive netns
rdma statistic: The iproute2 rdma tool subcommand that reads RDMA counters, enables optional congestion counters (set link <dev>/<port> optional-counters cc_rx_ce_pkts) and binds a counter to a single queue pair. It is the only way to attribute errors to one QP, because a sysfs counter set can be shared by many.; Defined in RoCE 4.1 · The counter map: sysfs, ethtool and rdma statistic
rdma-cni: A chained CNI plugin that moves the RDMA interfaces associated with a network interface into the container's network namespace. It appears after sriov or ib-sriov in the NAD plugins array, needs kernel 5.3+, iproute2 5.3+ and Multus 3.4.1+, and is meaningless without exclusive netns mode.; Defined in K8sNet 2.3 · RDMA in pods: shared plugin vs exclusive netns
rdmaHcaMax: The required field of the RDMA shared device plugin with no documented default. It is the number of times one HCA is over-advertised as a scheduling counter, not a partition: nothing is isolated and every pod holding a unit is using the same card.; Defined in K8sNet 2.3 · RDMA in pods: shared plugin vs exclusive netns
Reading a BOM backwards: Identifying the reference-architecture node type from a quote's adapter line. Nine NICs per two-CPU eight-GPU node is HGX 2-8-9-400; the specific parts (B3140H and B3220 versus ConnectX-8 and B3240) also fix the GPU generation, the fabric speed and the storage certification level implied.; Defined in RA 3.3 · Reading a real Dell AI Factory BOM
Reading gap versus hands-on gap: A reading gap closes with a lesson; a hands-on gap closes only with a simulator, bench time or a customer lab. NCP-AIN is written for an operator, so a gap plan needs both columns — DSX Air and the Dell lab close most of the Spectrum-side rows, while InfiniBand fabric bring-up, PKeys, UFM system health, optics faults and PFC storms on real buffers need borrowed hardware.; Defined in RA 4.5 · NCP-AIN: mapping the blueprint and closing the gaps
Reduced SRH: An SRH that omits the first segment of the policy because that segment is already in the IPv6 destination address, with Last Entry set to n-2. Saves exactly 16 bytes. Produced by the H.Encaps.Red headend behavior and by Linux mode encap.red, and it forces the HMAC TLV D-bit because destination address verification is no longer possible.; Defined in SRv6 1.2 · The SRH on the wire
Regression trigger: A change that reopens a signed-off qualification row: BIOS, iDRAC or CPLD update; BlueField firmware update; BFB/DOCA version change (October GA = Level-2 boundary); OVS-DOCA or HBN service version; riser or power configuration change. Each maps to the subset of tests it re-runs.; Defined in DOCA 8.3 · Writing a DOCA validation plan for an OEM server
--report_gbits: The perftest flag that reports Max/Average bandwidth in Gbit/sec instead of the default MiB/sec. The missing flag is the single most common cause of a false 'the link is half speed' escalation.; Defined in RoCE 3.1 · What ib_write_bw actually measures
Representor: A netdev on the Arm side that stands for a host PF/VF/SF port in switchdev mode; traffic to and from the host port is steered through it.; Defined in DOCA 4.1 · E-switch, representors, and switchdev
resourcePrefix: The domain half of an extended resource name, and the single most expensive default in this stack. The sriov-network-device-plugin defaults to intel.com, the RDMA shared plugin defaults to rdma, the NVIDIA Network Operator writes nvidia.com and OpenShift writes openshift.io. The same string must appear in three places: advertised by the plugin, referenced in the NAD, requested by the pod.; Defined in K8sNet 2.2 · SR-IOV plumbing: device plugin, CNI and operator
resourcePrefix mismatch: The most common cause of a permanently Pending pod with no CNI error at all. The SR-IOV network device plugin defaults to intel.com, the RDMA shared device plugin defaults to rdma, the Network Operator writes nvidia.com, and OpenShift writes openshift.io. The advertised name, the NAD's resourceName and the pod request must all agree including the prefix.; Defined in K8sNet 4.4 · The triage ladder: sos-report, error strings and the checklists
ResourceSlice: The object the DRA SR-IOV driver publishes for each VF, carrying attributes such as NUMA node, PCI bus ID, PCIe root, vendor, PCI address, VF id, PF name and eswitch mode. A ResourceClaimTemplate selects against those attributes with a CEL expression, which is how a request can state the topology it needs instead of asking for a bare count.; Defined in K8sNet 2.5 · vfio-pci for VMs, and the DRA SR-IOV driver
revision (pending / applied / startup): The three states an NVUE configuration passes through: pending after nv set or nv unset, applied after nv config apply, and startup persisted in /etc/nvue.d/startup.yaml. From Cumulus Linux 5.9 an apply saves startup automatically.; Defined in Spectrum-X 2.1 · The NVUE object model, revisions and snippets
Riser configuration (RC): Dell's named PCIe riser layout for a PowerEdge; the R770 and R7725 manuals allow BlueField-3 only in specific RCs and slot pairs (R770: RC 1/2 slots 31,36 and RC 6-2/11-2 slots 7,2; R7725: RC 2/3/4/5/8 slots 2,7).; Defined in DOCA 8.1 · PowerEdge support matrix
RoCE Adaptive Routing (packet spraying): Spectrum-X switch behavior that forwards every packet of an eligible RoCE flow over the ECMP member with the least-loaded egress queue, also influenced by neighbor-switch status notifications, instead of pinning the whole flow to one link by hash.; Defined in DOCA 7.1 · Spectrum-X architecture
roce enable pfc-priority: The Dell Enterprise SONiC form (roce enable pfc-priority 3,4). roce_enable activates the RoCEv2 default buffer settings and must be enabled before PFC priorities are configured; the Ansible module dellemc.enterprise_sonic.sonic_roce supports only state merged.; Defined in RoCE 4.4 · The switch half: Cumulus NVUE and Dell Enterprise SONiC
RoCE LAG: Hardware bonding for dual-port NVIDIA cards on kernel 4.9 and above. It engages under bond modes 1, 2 or 4 and collapses mlx5_0 and mlx5_1 into a single device named mlx5_bond_0. Any bond change that breaks those rules exits LAG silently.; Defined in RoCE 4.3 · MTU, RoCE LAG, and what has to be installed
RoCE lossy mode: nv set qos roce mode lossy — the documented ECN-only RoCE path on Cumulus Linux with no PFC, offered to a customer who refuses PFC outright. The trap: once in lossy mode, nv set qos roce alone does not restore the default, so lossless must be set explicitly.; Defined in RA 4.2 · Seven objections and the published answers
RoCE modes: lossless (default, PFC plus ECN on switch priority 3), lossy (ECN only, no pause), lossless-single-ipool (single shared ingress pool, unsupported by NetQ validation), lossy-multi-tc (the packet-trimming partner, added in 5.14) and dci-1 (Beta on Spectrum-4+, TC3 with PFC intra-DC and TC4 with ECN DC-to-DC).; Defined in Spectrum-X 3.2 · nv set qos roce: the one command and its five modes
RoCEv1: RDMA over Converged Ethernet carried in a dedicated ethertype 0x8915, with or without a VLAN tag, using the InfiniBand GRH instead of an IP header. It is a layer-2 protocol and is therefore confined to one Ethernet broadcast domain; the regular Ethernet MTU applies to the frame.; Defined in RoCE 1.3 · RoCEv1 vs RoCEv2: the encapsulation
RoCEv2: RDMA over Converged Ethernet encapsulated in IP plus UDP with well-known destination port 4791 (IANA: roce, registered 2014-10-17), which makes it routable across L3. The UDP source port carries an opaque flow identifier for ECMP hashing, so an ACL must match the destination port and wildcard the source port.; Defined in RoCE 1.3 · RoCEv1 vs RoCEv2: the encapsulation
Routing engine (MinHop, Up/Down, fat-tree): The unicast routing algorithm OpenSM runs, selected with -R as an ordered comma list tried in turn; Min Hop is always attempted last unless no_fallback is in the list. MinHop balances by LID count and optimises nothing; Up/Down suits impure trees but autodetects roots statistically and bans running the SM inside a spine switch system; fat-tree needs rank 2 to 8, equal port groups per rank and all CAs at one rank, and does not support LMC greater than 0. Upstream defaults to MinHop, UFM ships ar_updn.; Defined in InfiniBand 3.1 · Routing engines and how OpenSM picks a path
rshim: The host-side management interface to the BlueField Arm system: boot stream, console, network over tmfifo (192.168.100.x), and misc control.; Defined in DOCA 2.3 · rshim and the BFB path
RShim ownership: Only one RShim back end (host or the card's BMC) can be attached at a time; integrated-BMC cards default to BMC ownership. Hand over with systemctl stop/disable on the owner and enable/start on the other side; rshim -F sends a one-time ownership request.; Defined in DOCA 2.4 · Out-of-band: BMC, Redfish, and the BlueField-4 path

S

Scalable Function (SF): A lightweight function that shares PCIe resources with its parent PF and needs no SR-IOV; created on the Arm with mlxdevm port add ... flavour pcisf and bound over the auxiliary bus. SF numbers 1000+ are reserved for virtio-net.; Defined in DOCA 4.1 · E-switch, representors, and switchdev
Scalable Unit (SU): The reference architectures' unit of purchase and the source of every switch and cable count. H200 NDR: 31 nodes and 248 GPUs on 8 leaf and 4 spine - 32 by design, minus one DGX for UFM connectivity. B300 XDR: 72 nodes and 576 GPUs on 8 leaf and 4 spine, scaling to 18 SU with 144 leaf, 72 spine and 10,368 cables of each class. GB200 uses four spine-leaf-groups per SU to match its four IB rails. When a customer cannot fill an SU, the RA says to cable the full SU anyway and leave the positions empty.; Defined in InfiniBand 3.4 · Topologies: fat-tree, rail-optimized, twin-plane
Scale-out: GPU-to-GPU communication between nodes across the leaf-spine fabric - message-semantic RDMA over InfiniBand or Spectrum-X Ethernet. On B300 and GB300 this is 800 Gb/s per GPU through ConnectX-8 SuperNICs.; Defined in RA 1.3 · Scale-up is not scale-out: NVLink vs the fabric
Scale-up: GPU-to-GPU communication inside the accelerator over NVLink and NVSwitch - memory-semantic, load/store, no IP header. NVLink 4 gives 900 GB/s per GPU, NVLink 5 gives 1,800 GB/s, and NVLink 6 is listed at 3,600 GB/s as preliminary.; Defined in RA 1.3 · Scale-up is not scale-out: NVLink vs the fabric
Scoping email: The first written reply to a qualification request: four to ten questions (role, sourcing, mode, platform levels), a proposed validated-stack baseline, a phased plan, risks with KB numbers, and deliverables. Sent before any test runs.; Defined in DOCA 9.1 · Scenario: a qualification request
seg6_action_table[]: The table in net/ipv6/seg6_local.c that decides which behaviors Linux actually implements: End, End.X, End.T, End.DX2, End.DX6, End.DX4, End.DT4, End.DT6, End.DT46, End.B6, End.B6.Encaps and End.BPF. The uAPI header declares four more (End.BM, End.S, End.AS, End.AM) that are not implemented and ip-route(8) documents three fewer than the kernel supports, so only the table is authoritative. It also carries the per-behavior flavor masks - End supports psp and next-csid while End.X supports next-csid only, which is why End.X flavors psp returns "Unsupported Flavor operation(s)".; Defined in SRv6 2.1 · SRv6 in the Linux kernel: seg6 and seg6local
seg6_enabled: Per-interface Linux sysctl (net.ipv6.conf.<iface>.seg6_enabled) that accepts or drops SR-enabled IPv6 packets on that interface - packets with an SRH present and DA equal to a local address. Default 0, so a stock host silently drops every SRv6 packet addressed to it. Set all, default and the specific data interface; whether all alone suffices is not stated in the kernel documentation.; Defined in SRv6 2.1 · SRv6 in the Linux kernel: seg6 and seg6local
seg6local: The Linux lightweight tunnel that binds an SRv6 behavior to a local prefix: ip -6 route add PREFIX encap seg6local action <behavior> [params] [count] dev DEV. Its sibling encap seg6 is the headend (modes inline, encap, encap.red, l2encap). The count keyword enables the SEG6_LOCAL_CNT_PACKETS, _BYTES and _ERRORS counters, read with ip -s -6 route show. End.DT4, End.DT46 and End.DT6 with vrftable additionally require a real VRF device plus net.vrf.strict_mode=1.; Defined in SRv6 2.1 · SRv6 in the Linux kernel: seg6 and seg6local
Segments Left and Last Entry: The two counters that locate a packet in its journey. Last Entry is the zero-based index of the last element of the Segment List and does not change; Segments Left counts down at each endpoint and doubles as the index of the next segment. The list is stored in reverse, so Segment List[0] holds the LAST segment of the policy.; Defined in SRv6 1.2 · The SRH on the wire
SFC / br-hbn: Service function chaining for HBN: an OVS bridge (br-hbn) plus SFs (pf0dpu1, pf0dpu3) created at BFB install (ENABLE_BR_HBN=yes) that force uplink and host traffic through the HBN pod; in-pod names carry the _if suffix.; Defined in DOCA 5.1 · HBN: BGP/EVPN on the server
Shared device plugin versus SR-IOV: Two ways to give pods RDMA. The shared plugin places many pods on one physical function and its single QoS context, which suits storage clients and inference; SR-IOV gives each pod a virtual function, which is where isolation and per-tenant QoS actually exist.; Defined in RoCE 4.5 · RDMA in Kubernetes: the NVIDIA Network Operator
SHARP: Scalable Hierarchical Aggregation and Reduction Protocol — NVIDIA's in-switch collective reduction for InfiniBand. sharp_am (the aggregation manager) runs in UFM and libsharp links into the application on the compute nodes; the default transport is IPoIB on TCP port 6126. No equivalent in-network reduction is named on any fetched Spectrum-X page, so concede this criterion plainly and move to whether the customer's collectives are reduction-bound.; Defined in RA 4.1 · The Ethernet vs InfiniBand decision framework
SHARP aggregation tree (LLT and SAT): A set of switches and links used by one or more SHARP jobs; one tree can serve several jobs in different areas, and one multi-rail job can use several trees. LLT is a low-latency tree and SAT a streaming aggregation tree, printed by sharp_hello -v 3 as type:LLT and type:SAT. Quantum supports up to 126 trees in the subnet (63 and 63, one active streaming tree per switch); Quantum-2 up to 1023 (511 and 511, one active tree per port). ConnectX-5 does low-latency only.; Defined in InfiniBand 3.3 · SHARP: in-network reduction
SHARP routing-engine requirement: SHARP builds trees only under specific engines: updn, ar_updn, ftree or ar_ftree for tree topologies, dfp for DragonFly+, and dor with dor_hyper_cube_mode for hypercubes. It is the first triage question, and the reason a deadlock-driven choice such as lash or nue removes SHARP from a design. The switch-embedded MLNX-OS SM supports SHARP not at all.; Defined in InfiniBand 3.3 · SHARP: in-network reduction
sharp_am and libsharp: SHARP's two components. The aggregation manager runs in UFM on a dedicated server with a master subnet manager, and that server cannot serve as a compute node; libsharp links into client applications and reaches compute nodes through HPC-X. They talk over IPoIB on TCP port 6126 by default, or over native IB with smx_protocol = 1.; Defined in InfiniBand 3.3 · SHARP: in-network reduction
SHIELD and pFRN: NVIDIA's fast link fault recovery. shield_mode defaults to 3 (auto: the enhanced form under ar_updn, ar_ftree and dfp2, otherwise local). Proactive Fault Routing Notification carries the switch-to-switch messages on pfrn_sl = 4, which must be mapped in the SL2VL tables - VL15 there means drop this SL, so a mis-mapped pfrn_sl discards the notifications silently while every parameter still reads correct.; Defined in InfiniBand 3.2 · Adaptive routing, SHIELD and hash-based forwarding
SID Structure Sub-Sub-TLV: Service Data Sub-Sub-TLV Type 1, which advertises the Locator Block, Locator Node, Function and Argument lengths of a service SID. FRR prints it as a bracketed vector such as [32 16 32 0 0 0] in show bgp ipv6 vpn detail.; Defined in SRv6 3.1 · L3VPN and EVPN services over SRv6
Simulation fidelity limit: The set of features unsupported in a virtual Cumulus Linux switch — ACLs, ISSU, PTP, SPAN and ERSPAN, QoS shaping and buffer management, WJH, NAT, adaptive routing, storm control, OpenTelemetry and ASIC monitoring, with artificial sensor output. DSX Air rehearses the command surface and control plane; it cannot confirm ASIC data-plane behaviour.; Defined in Spectrum-X 6.4 · NCP-AIN: blueprint map, gaps and a study plan
Single plane: The documented cost option: one 400 Gb/s connection per GPU into a single compute fabric, which the HGX RA states reduces total GPU bandwidth by 50 percent and calls well suited to workloads that do not warrant maximum throughput. The RA expresses the choice in the transceiver - single-port OSFP for single plane, twin-port for dual - so it is a BOM fork rather than a field option. Dual plane is recommended at 64 nodes and above.; Defined in RA 2.2 · Dual plane: two fabrics, not a bond
Six logical networks: The GB300 Enterprise RA's named separation - GPU Compute (east-west), CPU Converged (north-south), Storage, Customer, Support Server Networking and Out-of-Band - carried over one physical fabric with VLAN isolation providing the logical separation.; Defined in RA 1.4 · The networks in an AI factory, and what the RA refuses to specify
Sizing chain: The four multiplications said out loud in front of a customer: GPUs times NICs per GPU gives NIC count; NIC ports times planes gives server-facing interfaces; interfaces per plane divided by leaf downlink radix gives leaves per plane; leaf uplinks divided by spine radix gives spines.; Defined in RA 2.3 · The sizing chain: GPUs, NICs, planes, leaves, spines
SL2VL mapping: The per-port table that turns a packet's service level into a virtual lane, configured through the SM's qos_ca_ qos_swe_ qos_sw0_ and qos_rtr_ parameters. VL15 used in that table means drop this SL rather than a real lane.; Defined in InfiniBand 2.4 · QoS: service levels, virtual lanes, SL2VL and VL arbitration
Slot-priority matrix: The table in a Dell technical guide that says which physical PCIe slots a given card type may occupy and how many are allowed, per GPU configuration. It is the document that decides whether an RA's adapter count physically fits, and it is paired with a thermal restriction matrix that can disallow a card the slot table permits.; Defined in RA 3.1 · The PowerEdge AI server line
SM priority: The 0 to 15 value that decides mastership, where 0 is the lowest and is upstream OpenSM's default while UFM ships 15 read-only. A tie is broken by the lower port GUID; an out-of-range value is reset to the built-in default rather than clamped.; Defined in InfiniBand 2.2 · Where the SM runs: switch, host, or UFM
SMP / MAD (QP0, VL15): Subnet Management Packets are the management datagrams the SM sends to the Subnet Manager Agent (SMA) that every HCA and switch runs. They travel on queue pair 0 over Virtual Lane 15, the lossy management lane that every port must support and that is not credit-flow-controlled. SMPs are either LID-routed (LIDR) or direct-routed (DR); direct routing carries an explicit port-by-port path and therefore works before any LID exists, which is why the base infiniband-diags tools function on an unconfigured subnet.; Defined in InfiniBand 1.1 · Who's who in an InfiniBand subnet
SMP blocker (SM_DISABLE): An HCA firmware setting applied with mlxconfig -d <device> s SM_DISABLE=1 that stops a host answering subnet management packets. NVIDIA recommends it on every non-SM host; it needs a reboot or flint swreset to take effect.; Defined in InfiniBand 2.5 · Keys, and locking a fabric down without locking yourself out
SN5600D: The DC-busbar variant of the Spectrum-4 SN5600, listed by Dell as 64x 800 GbE OSFP plus 1x 25 GbE SFP28 at 51.2 Tbps and specified for 40-60 VDC input. The AC siblings SN5610 and SN5600 take 200-240 VAC, and SN5610 alone is rated to 40 degrees C rather than 35.; Defined in RA 3.2 · Dell's switch catalog: SN, Q and Z in one price list
SNAP-4: BlueField-3 storage emulation service that presents NVMe and virtio-blk PCIe devices to the host backed by SPDK block devices (NVMe-oF, malloc, AIO); enabled with NVME_EMULATION_ENABLE / VIRTIO_BLK_EMULATION_ENABLE and driven by snap_rpc.py and spdk_rpc.py.; Defined in DOCA 5.4 · Storage offload: SNAP, Virtio, DevEmu, STA
Solution ID 19845005.1: A published Dell AI Factory with NVIDIA component list: 3x PowerEdge R660 and 2x XE9680 with 16x H200 SXM, 'NVIDIA Bluefield-3 (1x3220/8x3140H)', 3x SN5600 as 400G back end plus 200G front end, 2x PowerSwitch S5232F-ON for the storage cluster at 100G, 1x SN2201 out-of-band, 3x PowerScale F710, an APC 42U rack noted at approximately 48/2kWs, 19x NVAIE and ProSupport+/ProDeploy/ProConsult.; Defined in RA 3.3 · Reading a real Dell AI Factory BOM
Spectrum-X: NVIDIA's end-to-end Ethernet platform for AI: Spectrum switches plus SuperNICs plus software (Cumulus Linux, SONiC, NetQ, DOCA), co-designed so the switch sprays RoCE packets per packet and the SuperNIC restores order and meters congestion.; Defined in DOCA 7.1 · Spectrum-X architecture
Spectrum-X profile ConfigMap: From Network Operator 26.7.0, NIC tuning ships as a versioned ConfigMap carrying the watch label network.nvidia.com/operator.nic-configuration.spectrum-x-profile, and spectrumXOptimized.version names that ConfigMap by name, character for character. A stale value from 26.4 produces the runtime error spectrum-x config version <name> not found.; Defined in K8sNet 4.1 · Spectrum-X rails inside Kubernetes
Spectrum-X Validated Solution Stack: NVIDIA's page of tested version combinations per platform (GB300, B300, H200). v2.3.1 (September 2026) pins Cumulus 5.18.1, BF FW Bundle 3.5.0, BF-3 FW 32.50.1002, CX-8 FW 40.50.1002, DOCA-Host 3.5.0-082, NetQ 5.1.0, NCCL 2.30.7, HPC-X 2.51, Network Operator 26.7.0, DTS 1.26.5. It does not track SONiC, GPU driver, CUDA or part numbers.; Defined in DOCA 7.3 · The validated stack and version pinning
sr0: The dummy interface FRR uses to install SRv6 SIDs into the Linux data plane. FRR does not create it: ip link add sr0 type dummy && ip link set sr0 up. Without it IS-IS still allocates and advertises SIDs, so every control-plane show command looks healthy while ip -6 route show | grep seg6local is empty.; Defined in SRv6 2.2 · FRR: locators, the SID Manager, IS-IS SRv6 and TI-LFA
SRH (Segment Routing Header, Routing Type 4): The IPv6 routing header that carries the segment list. Eight fixed bytes - Next Header, Hdr Ext Len, Routing Type 4, Segments Left, Last Entry, Flags, Tag - followed by 128-bit segment list entries and optional TLVs. Next Header, Hdr Ext Len and Routing Type are not mutable; Segments Left is.; Defined in SRv6 1.2 · The SRH on the wire
srh_size: The fixed total SRH size in bytes declared on a DOCA Flow SRv6 pipe action - 8 for the base header plus 16 per segment - and every entry on that pipe action must supply an SRH of exactly that size, so each segment count needs its own pipe action. The per-entry fields the application must set itself are hdr_ext_len (2n and even), routing_type 4, segments_left and last_entry; DOCA Flow validates none of them. Creation also depends on an internal scratch register whose availability varies with hardware model and firmware configuration, so identical code can fail on one card and succeed on another.; Defined in SRv6 2.4 · NVIDIA NIC and DPU: SRH push with DOCA Flow
SriovNetworkNodeState: The read-only CRD of the SR-IOV Network Operator that reports the SR-IOV hardware actually discovered on a node. It is the first object to read when a customer says numVfs did not apply, because it separates what was requested from what the card exposes.; Defined in K8sNet 2.2 · SR-IOV plumbing: device plugin, CNI and operator
SRv6 external action (DOCA Flow): The DOCA Flow plugin that extends the standard action set with SRv6, registered with doca_flow_external_action_srv6_register() before doca_flow_init(). Hardware performs PUSH only, removal is done in software, and PUSH requires a port scratch register whose availability depends on hardware model and firmware.; Defined in SRv6 3.3 · Who actually ships SRv6: the Dell and NVIDIA support matrix
SRv6 Services TLV: The container RFC 9252 defines inside the existing BGP Prefix-SID attribute for carrying SRv6 service SIDs. Service Sub-TLV Type 1 holds a 16-octet SID value plus a 2-octet SRv6 Endpoint Behavior; the opaque behavior 0xFFFF abstracts the real one.; Defined in SRv6 3.1 · L3VPN and EVPN services over SRv6
srv6-only: An FRR BGP setting that is on by default the moment any SRv6 locator is configured. Every BGP update without an SRv6 prefix-SID attribute becomes invalid until no srv6-only is entered, which is why enabling SRv6 appears to break an existing MPLS L3VPN.; Defined in SRv6 3.1 · L3VPN and EVPN services over SRv6
startup.yaml and config-backup snapshots: /etc/nvue.d/startup.yaml is the saved NVUE configuration read at boot; Cumulus Linux 5.18.0 adds automatic snapshots under /var/lib/config-backup/auto-snapshots/{applied,weekly}/ with a 512 MB cap, restored by nv action restore system config backup <path>. Both live on the switch, so the off-box copy plus nv config show -o commands is the real backup.; Defined in Spectrum-X 5.5 · Upgrades that do not lose the fabric
Storage floor: The Enterprise RA planning rule of approximately 12.5 Gb/s per GPU scaling linearly, with 16 GPUs needing around 200 Gb/s. Quoted in bits per GPU while the converged capability is quoted in bytes per node, so convert before comparing. The DGX SuperPOD RA states a separate hard minimum above 40 GBps per node.; Defined in RA 2.4 · The other three networks: converged, storage, out-of-band
Subnet Administrator (SA): Runs on the same node as the Subnet Manager and complements it by responding to information and registration requests - events, service attributes, topology and switch forwarding tables. Short version: the SM writes configuration downward to the agents, the SA answers queries upward from tools and applications. An application that cannot resolve a path is an SA problem; a fabric that forwards nothing at all is an SM problem.; Defined in InfiniBand 1.1 · Who's who in an InfiniBand subnet
Subnet Manager (SM): The central authority of an InfiniBand subnet. It discovers the physical topology, assigns LIDs to end nodes switches and routers, establishes paths among end nodes, and sweeps the subnet to react to topology changes such as node reboots and link flaps. Switches forward on tables the SM wrote, so a fabric with no SM trains its links and carries no traffic. Mastership goes to the highest priority and on a tie to the lower GUID; NVIDIA recommends no more than two SMs per fabric, and UFM in Management or HA mode requires every other SM to be disabled first.; Defined in InfiniBand 1.1 · Who's who in an InfiniBand subnet
SUBNET UP: The success marker OpenSM writes to both /var/log/messages and /var/log/opensm.log when it has set the subnet up correctly. One occurrence is healthy; a line that repeats every few minutes means the SM keeps re-initializing.; Defined in InfiniBand 2.1 · Bringing up OpenSM
SuperNIC: A BlueField-3 or ConnectX-8 variant positioned for east-west AI traffic: high bandwidth, RoCE and Spectrum-X features, fewer Arm cores, NIC mode by default.; Defined in DOCA 1.2 · The NVIDIA networking portfolio
SuperNIC versus DPU: A role split by traffic direction. SuperNICs (BlueField-3 B3140H, ConnectX-8) carry east-west GPU traffic at or near 1:1 with GPUs; DPUs (BlueField-3 B3220, B3240) carry north-south storage, in-band management and tenant services on their own Arm cores, with their own BMC on the out-of-band network.; Defined in RA 1.5 · The generation map: which SuperNIC is on the baseboard
Support boundary: The Dell OEM split an FAE must be able to state: hardware, RMA and entitlement to Dell as PowerSwitch SN-series; Cumulus Linux software and release lifetime to NVIDIA (Dell's own install KB reproduces the NVIDIA procedure); Dell Enterprise SONiC to Dell, on Dell Broadcom-based platforms only.; Defined in Spectrum-X 6.3 · Scenario: sizing, quoting and defending a Dell AI pod fabric
switch priority: The internal 0-7 class a Spectrum switch assigns to a frame after trust-mode classification. With RoCE enabled the defaults are DSCP 24-31 to switch priority 3 (RoCE data) and DSCP 48-55 to switch priority 6 (CNP); everything else falls through to traffic class 0.; Defined in Spectrum-X 3.1 · How a RoCE packet gets classified
Switch-embedded SM: The subnet manager built into a managed switch, enabled in MLNX-OS with ib smnode <name> enable. Documented to 2048 nodes on x86 systems and explicitly without adaptive routing, fault routing (SHIELD or FRN), congestion control, SHARP or DragonFly+ support.; Defined in InfiniBand 2.2 · Where the SM runs: switch, host, or UFM
symmetric IRB (RFC 9135): Integrated routing and bridging where both ingress and egress VTEPs perform MAC and IP lookups, using Type-2 routes with the Label2 field populated plus Type-5 for prefixes. Asymmetric IRB instead does three lookups at ingress and one MAC lookup at egress.; Defined in Spectrum-X 2.5 · BGP-EVPN symmetric IRB for tenant isolation

T

Targeted congestion control: The second Spectrum-X loop: switches generate ECN marks only when adaptive routing capacity is exhausted, and senders adjust rate at RTT timescales using precise RTT probes combined with ECN marks plus in-band switch telemetry including timestamps and buffer utilization.; Defined in Spectrum-X 4.3 · Telemetry-based congestion control: three loops, not one
task ownership: A task belongs to the application after alloc or alloc_init, to the context after doca_task_submit, and to the application again when its completion or error callback runs, after which it is freed with doca_task_free or resubmitted. Buffers follow the same rule.; Defined in DOCA 3.3 · Tasks, the progress engine, and completion
Telemetry-based congestion control: Spectrum-X closed loop for incast: the switch emits in-band telemetry (queue depth, port utilization) and the sending SuperNIC meters its injection rate at microsecond scale; adaptive routing cannot solve incast because a single receiver port cannot be routed around.; Defined in DOCA 7.1 · Spectrum-X architecture
the four RoCE counters: pause-packets and pause-duration (is PFC firing), ecn-marked-packets (is ECN signalling), no-buffer-discard (is the lossless pool or headroom too small) and pg-max-usage (peak priority-group occupancy). Read them with nv show interface <if> qos roce counters; high pause with zero marks means thresholds are too high, and zero pause on a congested lossless fabric means classification is wrong.; Defined in Spectrum-X 3.5 · Playbook: 'RoCE is slow'
Three front doors: The three published support routes in Dell AI deployments: a Dell-sold PowerEdge with Dell PowerSwitch SN-series goes to Dell ProSupport (Dell announced SN5600, SN2201 and Quantum-X800 as backed by ProSupport and Deployment Services); a DGX SuperPOD is NVIDIA-led with support through NVIDIA Technical Account Managers; a PowerScale back-end switch runs through the Dell ETC program.; Defined in RA 3.5 · Who do I call: three support paths, one asset tag
Three-state support answer: The discipline of answering every capability question as published, not published or unverified, each with a URL and a fetch date. Not published is a statement about a document you read; unverified is a statement about a document you could not read.; Defined in SRv6 3.3 · Who actually ships SRv6: the Dell and NVIDIA support matrix
TI-LFA: Topology Independent Fast Reroute, published as RFC 9855 (Proposed Standard, October 2025) after years as a draft. In FRR it is the per-interface command isis fast-reroute ti-lfa [level] [node-protection [link-fallback]], verified with show isis fast-reroute summary. Zero protected prefixes usually means it was configured at router level; incomplete protection usually means node-protection without link-fallback.; Defined in SRv6 2.2 · FRR: locators, the SID Manager, IS-IS SRv6 and TI-LFA
tmfifo_net0: The virtual network interface over RShim; host 192.168.100.1, Arm 192.168.100.2. Verify with ip route get 192.168.100.2 (dev tmfifo_net0 or tm-br) before trusting a ping.; Defined in DOCA 2.3 · rshim and the BFB path
TopologyAffinityError: The exact pod rejection reason produced under restricted and single-numa-node when alignment cannot be satisfied. It reports alignment, not capacity, and it is the system working - the remedies are prefer-closest-numa-nodes, a co-request in one container, or better physical NIC placement, never setting the policy to none.; Defined in K8sNet 3.5 · NUMA alignment with Topology Manager and CPU Manager
TopologyInfo: The field in a device plugin's response that carries the NUMA node of a device. It is the hint Topology Manager reads, so a plugin that omits it makes the NIC look topology-free and NIC-to-GPU alignment silently impossible.; Defined in K8sNet 2.1 · How kubelet learns a NIC exists
TOS octet: The single IPv4 header byte (IPv6 Traffic Class) that carries six bits of DSCP followed by two bits of ECN. TOS = DSCP << 2, which is why doRoCE.sh's default of 106 is DSCP 26 and not DSCP 106. The four RFC 3168 ECN codepoints are 00 Not-ECT, 01 ECT(1), 10 ECT(0) and 11 CE, and a router sets CE only where it would otherwise have dropped the packet.; Defined in Spectrum-X 3.1 · How a RoCE packet gets classified
Transit node: One of the three RFC 8754 node roles, and the one that matters commercially: it does nothing SR-specific and simply forwards an IPv6 packet on its destination address. Only nodes whose SIDs appear in the segment list must be segment endpoint nodes, so a switch with no SRv6 support is a legal hop.; Defined in SRv6 1.1 · Source routing, segments, and the two data planes
Transport service type (RC, UC, UD): The queue pair's transport. RC is reliable and connected and is the only standard type (with XRC) that supports RDMA READ and the atomics. UC is connected but unacknowledged: one-sided WRITE but no READ and no atomics. UD is a datagram and can only SEND, has no path_mtu, and carries its address per work request in an address handle.; Defined in RoCE 1.2 · Transports and opcodes: what RC, UC and UD can actually do
Transposition Scheme: Carrying only the common part of a service SID in the TLV and moving the variable Function or Argument bits into the service encoding's existing label field. Transposition Offset gives the bit position and Transposition Length the number of bits moved; for IPv4 VPN the length must be at most 20 bits.; Defined in SRv6 3.1 · L3VPN and EVPN services over SRv6
trust mode: Which marking the switch believes, set with nv set qos mapping <profile> trust l2|l3|port|both. l2 maps 802.1p PCP straight to switch priority, l3 groups the 64 DSCP values into eight ranges, port ignores markings and uses the port default priority, and both prefers DSCP for IP traffic while applying 802.1p to non-IP frames.; Defined in Spectrum-X 3.1 · How a RoCE packet gets classified
Twin-plane fat tree: The XDR-era shape named by the DGX B300 reference architecture: rail-optimized, non-blocking, twin-plane fat tree on Quantum-X800 Q3400-RA at 800 Gbps. NVIDIA names the shape but the fetched pages do not publish how the planes are wired, so any two-plane drawing is a reading to present as such, not a quotation.; Defined in InfiniBand 3.4 · Topologies: fat-tree, rail-optimized, twin-plane
Twin-port OSFP (finned vs flat top): The Quantum-2 cage format: two transceiver engines in one OSFP plug, giving 800 Gb/s electrical to the switch and 2x400G optics over two MPO connectors - so a QM9700's 64 ports occupy 32 cages, and a BOM counts cages, not ports. The top is thermal: finned at the air-cooled switch, flat (-FLT) at DGX and liquid-cooled ends, with -FTF for an asymmetric run. NDR copper reaches 2 m passive (MCP4Y10) and 5 m active (MCA4J80); beyond that it is optics plus fibre and MPO patch panels.; Defined in InfiniBand 3.5 · Quantum switches and the LinkX BOM

U

uA router-advertisement dependency: A uA SID resolves its peer link-local next hop from router advertisements. Spectrum switches enable RAs automatically on an interface carrying a uA SID, but a non-Spectrum neighbour - a Dell PowerSwitch, for example - must have RAs enabled by hand on the connected interface or next-hop discovery never completes and the SID forwards nothing while every other test on the link passes.; Defined in SRv6 2.3 · NVIDIA switching: uN and uA on Spectrum-4
UFM System Health report: A UFM report that runs checks on the UFM server itself: UFM Configuration, UFM Processing (OpenSM, ibpm, ModelMain, httpd, MySql), Memory, CPU, Disk and Fabric Interface. Server health, not fabric telemetry.; Defined in InfiniBand 5.4 · NCP-AIN prep: blueprint, drills and honest gaps
Ultra Ethernet Consortium (UEC): A Linux Foundation Joint Development Foundation consortium whose released specification is v1.0.3 and whose General Members include both NVIDIA and Dell. In a customer conversation it is context, not a competitor: state the membership and the specification version and refuse to speculate on product timing.; Defined in RA 4.2 · Seven objections and the published answers
uN and uA: The only two endpoint behaviors Cumulus Linux implements: uN is End with NEXT-CSID (advance the compressed segment list) and uA is End.X with NEXT-CSID (cross-connect out one named adjacency), both per RFC 9800. Supported on the Spectrum-4 switch only, in the default VRF only, as static SIDs only, with a maximum of 256 locators - unchanged from Cumulus Linux 5.14.0 through 5.18. NVUE prints a uA SID's behavior as End.X.; Defined in SRv6 2.3 · NVIDIA switching: uN and uA on Spectrum-4
Unverifiable competitor claim: A specification circulating in search summaries that cannot be sourced on the vendor's own page — every Tomahawk 6 figure is in this category because the Broadcom product page fetches as a content-free shell. The correct response is to say you could not verify it on a primary page and offer to re-check, never to counter it with another unverified number.; Defined in RA 4.3 · Competitors, stated neutrally
UQD8 v2: The quick-disconnect coupling used by the liquid-cooled SN6000 -LD switches. Those models need facility water at 17-45 C and a 48-60 VDC busbar, which makes the switch choice a mechanical and electrical decision before it is a network one.; Defined in Spectrum-X 1.2 · Choosing the switch: SN2201 through SN6800-LD
UROM: Unified Resources and Offload Manager: a DOCA framework for offloading parts of HPC and AI parallel workloads to the DPU through plugins (UCC, RDMO, Graph) executed by a UROM daemon and workers on the Arm cores, with a host library and UCX over TCP as the control channel. Library Alpha, service GA in 3.5.0.; Defined in DOCA 6.4 · GPUNetIO and UROM
useHostMofed: The GPU Operator helm setting that states who owns the OFED driver on a node - true when the host already has MOFED or DOCA installed, false when the Network Operator supplies it. A mismatch produces a driver container that builds and loads and never binds to the NIC.; Defined in K8sNet 3.4 · GPUDirect RDMA: dma-buf, nvidia-peermem and GDS
uSID: Vendor and industry vocabulary for the NEXT-C-SID flavor at a 32-bit Locator-Block and 16-bit CSID length - the mandatory-to-implement profile. RFC 9800 never uses the word, nor micro-segment. uN is End with NEXT-CSID and uA is End.X with NEXT-CSID; End-of-Carrier 0000 is industry carrier vocabulary, not RFC text. Cumulus Linux implements uN and uA on Spectrum-4 in the default VRF only.; Defined in SRv6 1.4 · Compression: RFC 9800, NEXT-C-SID, and what "uSID" really means

V

Validated solution stack: The published Spectrum-X table of matched component versions per GPU platform — switch NOS, ConnectX-8 firmware, BlueField-3 firmware, DOCA-Host, NetQ, NCCL, HPC-X, Network Operator and DTS — shipped as versioned rows from v1.0.1 in April 2024 to v2.3.1 in September 2026. It is the answer to "800G is unproven" and the one competitive differentiator that is both published and checkable, because no competitor publishes an equivalent table.; Defined in RA 4.2 · Seven objections and the published answers
Validated-stack row: One release row of the NVIDIA Spectrum-X Validated Solution Stack (for example v2.3.1, September 2026: Cumulus 5.18.1, BF-3 FW 32.50.1002, DOCA-Host 3.5.0-082, NetQ 5.1.0). Every version in a qualification or memo is pinned to a row.; Defined in DOCA 9.1 · Scenario: a qualification request
Version intersection: The Kubernetes band where every operator in the stack is inside its own published window. For Network Operator 26.7.0 (1.32-1.36) and GPU Operator 26.7.x (1.33-1.37) the intersection is 1.33-1.36; a cluster on 1.32 or 1.37 has one operator outside support.; Defined in K8sNet 3.2 · Version matrices, upgrades and heterogeneous clusters
Version-pin conflict: The collision between two published software pins on one switch model: Dell qualifies the SN5600 at Cumulus Linux 5.9.1 for a PowerScale back end, while the NVIDIA Spectrum-X validated stack v2.3.1 pins 5.18.1 for the AI fabric. A single switch runs one version, so a shared switch is outside one of the two support statements.; Defined in RA 3.4 · PowerScale back-end networking and the version-pin trap
VL arbitration credit: The 64-byte unit in which VLArb weights are expressed. A 4 KB MTU packet costs 64 credits, so weights below 64 give a lane less than one packet per arbitration turn and useful weights are multiples of 64.; Defined in InfiniBand 2.4 · QoS: service levels, virtual lanes, SL2VL and VL arbitration
VRF meta-plugin (rail isolation): The Spectrum-X component that puts each rail in its own VRF named after the rail topology entry. It applies only to rails whose cidrPoolRef resolves to an IPv6 CIDRPool; rails using an inline railTopology[].ipam get no VRF isolation. Because RDMA-CM is not VRF-aware, tests such as ib_write_bw -R must be launched with ip vrf exec rail0.; Defined in K8sNet 4.1 · Spectrum-X rails inside Kubernetes
VRR: Virtual Router Redundancy: the MLAG anycast gateway, which activates the virtual MAC on both peers so either can route frames it receives. Default virtual MAC 00:00:5E:00:01:01; VRR and VRRP cannot coexist on one switch.; Defined in Spectrum-X 2.3 · Layer 2 that survives a reload: bridges, bonds, MLAG and VRR

W

-W / --report-counters: The perftest flag that diffs named sysfs counters across a run, for example counters/port_xmit_data,hw_counters/out_of_buffer. It provides evidence about a measurement that is independent of the benchmark's own arithmetic.; Defined in RoCE 3.1 · What ib_write_bw actually measures
What Just Happened (WJH): NVIDIA's drop-reason telemetry on Spectrum switches, delivered as two components: an agent that streams contextual telemetry off-box to NetQ, and a local service that diagnoses dropped packets across L1, L2, L3/router, tunnel, buffer and ACL categories. It names why a packet was discarded rather than counting discards.; Defined in Spectrum-X 5.1 · What Just Happened: naming the drop
What the RA does not publish: The Enterprise RAs specify topology, adapter counts and port speeds but no BGP or EVPN configuration, no IP addressing, no MTU, no DSCP/PFC/ECN values and no adaptive-routing or congestion-control parameters. Those live in the Cumulus and Spectrum-X software documentation.; Defined in RA 1.4 · The networks in an AI factory, and what the RA refuses to specify
Whereabouts: An IPAM CNI plugin that assigns IP addresses cluster-wide. Required fields are type and range; optional ones include range_start, range_end, exclude, network_name, gateway, routes, enable_overlapping_ranges (default true) and node_slice_size for the experimental Fast IPAM mode. State lives in IPPool and OverlappingRangeIPReservation CRs. Documented issue: system crashes may leave stranded allocations.; Defined in K8sNet 1.5 · macvlan, ipvlan, host-device and cluster-wide IPAM
WJH channel: The unit of WJH configuration on Cumulus Linux 5.18: a 4-16 character name bound to exactly one trigger type (l1, l2, l3, tunnel, buffer or acl), optionally carrying drop filters, a polling-interval of 5-300 s, an aggregate-cache-size of 500-5000 and buffer latency or congestion thresholds.; Defined in Spectrum-X 5.1 · What Just Happened: naming the drop
WJH drop-reason id range: The numeric ranges WJH uses when streamed over gNMI, which let you classify an event before reading its headers: L1 10021-10031, L2 201-210, Router 301-327, Tunnel 402-404, ACL 601-604, Buffer 503-506, each with a severity of Notice, Warning or Error.; Defined in Spectrum-X 5.1 · What Just Happened: naming the drop

X

-x / --gid-index: The perftest flag that pins which GID index a run uses. Because the GID type at that index sets the RoCE version of a connected QP, a run without -x cannot be reported as RoCEv1 or RoCEv2.; Defined in RoCE 3.1 · What ib_write_bw actually measures
XE9680 mixed NIC/DPU rule: When two different high-speed NIC/DPU card types are configured, the type with the smaller quantity (two cards or fewer) must go in slots 31 and 40 first; the other type then follows the published slot priority 33, 37, 35, 39, 32, 36, 34, 38, 31, 40.; Defined in RoCE 5.1 · Rail-optimized fabrics on Dell hardware
XE9680 slot priority: Dell's published population order for single-port NDR400 InfiniBand cards in a PowerEdge XE9680 with NVIDIA GPUs: 33, 37, 35, 39, 32, 36, 34, 38, then 31 and 40, with odd-numbered slots filled before even.; Defined in InfiniBand 5.2 · Design review: an eight-rail IB pod on Dell XE9680
XE9680 slot-to-socket map: Dell PowerEdge XE9680 PCIe layout: slots 32 and 33 (Riser 4) and 34 and 35 (Riser 3) on Processor 2, slots 36 and 37 (Riser 2) and 38 and 39 (Riser 1) on Processor 1, plus base-board slot 31 on Processor 2 and slot 40 on Processor 1.; Defined in RoCE 5.1 · Rail-optimized fabrics on Dell hardware

Z

Zero Trust (Restricted) mode: A variation of DPU mode in which the Arm side enforces restrictions on the external host (port ownership, hardware counters, tracer, RShim, firmware flash). Set and cleared with mlxprivhost on the Arm OS; shows as an RO flag on the host's mlxconfig query.; Defined in DOCA 1.4 · Modes of operation: DPU, NIC, Zero-Trust
ZTP (DHCP option 239): Zero touch provisioning: the running switch fetches the script named by DHCP option 239 and runs it. The script must carry the CUMULUS-AUTOPROVISIONING flag and exit 0; state is read with ztp -s and logged to /var/log/autoprovision.; Defined in Spectrum-X 2.2 · Day-0: ONIE install, first boot and ZTP
ZTR-RTT CC: NVIDIA's Zero Touch RoCE congestion control inside the DOCA PCC framework: it reacts to hardware-timestamped round-trip time plus CNPs with additive increase and multiplicative decrease, so it needs no special switch configuration. NVIDIA publishes no supported device list or firmware floor for it.; Defined in RoCE 2.5 · Zero Touch RoCE, ZTR-RTT CC and DOCA PCC

0–9 and symbols

2-8-9-400: The NVIDIA Enterprise Reference Architecture pattern name that the published Dell AI Factory XE9680 solution brief aligns to. NVIDIA's Enterprise RA index lists it alongside patterns such as 2-8-10-400, 2-8-9-800 and 2-8-5-200 and decodes none of the digits, so it must be quoted as an undecoded label rather than expanded into a specification.; Defined in K8sNet 4.5 · Dell AI Factory scenarios and the NCA-AIIO positioning
5f00::/16: The IPv6 block IANA assigned for SRv6 SIDs under RFC 9602, allocation date 2024-04, so that a domain can identify and filter SID traffic at its edge and fail closed. Available rather than mandatory: FRR examples use fcbb:bbbb style locators, Cumulus Linux uses fcbb::/16 and SONiC uses FCBB:BBBB:: space, so most edge filters are hand-maintained.; Defined in SRv6 1.4 · Compression: RFC 9800, NEXT-C-SID, and what "uSID" really means
8-pin ATX 12V aux power: The additional PCIe power connector some BlueField-3 B3240 configurations require because they can draw more than 75 W, which is all a PCIe slot supplies. It is a build-sheet line item and the most common rack-and-stack surprise on a B300 node.; Defined in RA 1.5 · The generation map: which SuperNIC is on the baseboard
8-pin auxiliary power (ATX 12V PCIe): Supplementary power connector required by the 150 W BlueField-3 DPU SKUs (B3240, B3220, B3210, B3210E); without it a DPU-mode card halts with 'ATX power not detected! Halting system!!'. The 75 W B3140H SuperNIC runs on slot power.; Defined in DOCA 8.1 · PowerEdge support matrix