Skip to content
nodeza.learn
DOCA course/1. Foundations/Lesson 1.2

The NVIDIA networking portfolio

S1·E2Sixty-four lines, one part number · Hotel lobby, 7:40 a.m., an hour before the design review

S1·E2Understand~25 minsources checked todayverified against BlueField-2/-3 HW user guides, NVIDIA Spectrum-X and Quantum-X800 pages, ConnectX-8 manual, DOCA 3.5.0 release notes / general support / changes pages, 2026-09-06

Builds on: Why infrastructure moved onto the NIC

Before you read: what do you already know?

3 quick questions. Wrong answers are fine and expected; trying first makes the lesson stick.

After this lesson you can

  • Place ConnectX, BlueField, Spectrum and Quantum products on one map with the generation and link speed of each.
  • Compare BlueField-2, BlueField-3 and BlueField-4 by cores, memory, PCIe and DOCA 3.5.0 support status.
  • State which SuperNIC generation NVIDIA pairs with Hopper, Blackwell and Vera Rubin systems.
  • Read a firmware version string and identify the adapter family and whether it matches the DOCA 3.5.0 bundle.

Episode 2 — Sixty-four lines, one part number

The situation · Hotel lobby, 7:40 a.m., an hour before the design review

The BOM is still folded where the Dell SE folded it last night. Sixty-four nodes: forty with Blackwell GPUs, twenty-four multi-tenant bare metal, and a BlueField-3 SuperNIC on every line of both classes. He is on his second coffee and will not drink this one either; his spreadsheet says the fabric was signed off on Tuesday. The customer’s architect approved this BOM, the review starts in an hour, and your job in the lobby is to decide whether it is a mistake or a decision.

The portfolio exists because one fabric cannot be one part. NVIDIA pairs each endpoint with a GPU generation on the Spectrum-X page: BlueField-3 for Hopper-era systems, optimized for RDMA/RoCE and multi-tenant isolation; ConnectX-8 at 800 Gb/s total throughput via 2x400G, PCIe Gen6, for Blackwell systems; ConnectX-9 at 1,600 Gb/s per GPU for Vera Rubin NVL72 systems[9]. Read the parentheses and the map explains itself: one endpoint is sold on isolation, the other two on bandwidth.

So forty of those lines are a question, not yet an error. Before you can ask it you need the whole map in your head — which family, which generation, which firmware line, and what DOCA 3.5.0 actually lists: BlueField-3 at 32.50.1002, ConnectX-8 at 40.50.1002[11]. Match the endpoint to the GPU generation first and to the tenancy model second; a part number that ignores either one is a guess with a lead time. You open your notes at the top of the map.

1Three families, one map

NVIDIA networking as it reaches a Dell customer is three product families and two platforms. The families: adapters (ConnectX), DPUs and SuperNICs (BlueField), and switches (Spectrum for Ethernet, Quantum for InfiniBand). The platforms tie them together: Spectrum-X, “the world’s first end-to-end, AI-optimized Ethernet platform”, and the Quantum-X800 InfiniBand platform, which NVIDIA defines as the Quantum-X800 switch plus ConnectX-8 and ConnectX-9 SuperNICs and LinkX cables.[9][10]

Adapters first. The ConnectX-8 SuperNIC runs at “up to 800 gigabits per second (Gb/s)” over “PCIe Gen6 @ 64GT/s through x16 edge connector” and supports InfiniBand and Ethernet in one device.[8] ConnectX-9 is “1,600 Gb/s per GPU via 4×200 G SerDes, PCIe Gen6”.[9] ConnectX-7 is the 400 Gb/s generation; the official pages fetched for this lesson identify it only by its firmware line, 28.50.1002 in DOCA 3.5.0, so treat the 400G figure as widely reported rather than quoted.[11]

DPUs: BlueField-2 reaches “up to 200 Gigabit Ethernet”, BlueField-3 is NVIDIA’s “400 Gb/s infrastructure compute platform”, and BlueField-4 “an 800 Gb/s infrastructure platform for gigascale AI factories”.[1][6]

Switches: the Spectrum-6 ASIC offers “102.4 Tb/s per switch chip with 200 G SerDes—double the bandwidth per lane versus the prior generation”; the SN6600 “offers 128 ports of 800 G OSFP connectivity” and the SN6800 “packs 512 ports of 800 G in a 5U form factor”.[9] Spectrum-4 and the SN5600 are the prior generation, but their figures are not on any page fetched here; quote them only after checking. On the InfiniBand side the Quantum-X800 switch has “144 ports of 800 Gb/s connectivity per port”; Quantum-2 is the NDR generation, whose speed the fetched page does not state.[10]

ProductSpeedPCIeRoleGPU generation
NIC
SuperNIC (no Arm)
SuperNIC (no Arm)
DPU
DPU
SuperNIC (Arm inactive)
DPU / storage processor
Ethernet switch
Ethernet switch
InfiniBand switch
InfiniBand switch

⚠ = not confirmed on a fetched primary source (hover for why). Facts as of DOCA 3.5.0 (Sep 2026). Selections are saved.

Products tab: each row is one part with its generation, speed, firmware line and DOCA 3.5.0 status. Hover a row to see which source states each cell.

2BlueField, generation by generation

BlueField-2 BlueField-3 BlueField-4
Arm complex “Eight 64-bit Armv8 A72 cores interconnected by a coherent mesh network”[1] “x8 / x16 Armv8.2+ A78 Hercules cores (64-bit)”[3] 64 cores, Grace / Neoverse V2 — press only[7]
Core clocks P-Series 500/2500 MHz (550/2750 on advanced models); E-Series 430/2000 MHz[2] 16-core P-Series 560/2133 MHz; E-Series 505/2000 MHz[14] not published
NIC core integrates ConnectX-6 Dx[1] 400 Gb/s class, 1 to 4 ports[4] ConnectX-9 — press only[7]
Memory “8GB/16GB/32GB @3200MT/s single-channel DDR4”[1] “16GB/32GB @5200MT/s or @5600MT/s single/dual-channel DDR5”[3] press figures conflict — unverified
Boot storage eMMC, 64GB or 128GB by model[2] 40GB eMMC plus 128GB SSD[3] press only — unverified
Host bus “PCIe Gen 4.0 (16GT/s) through an x8/x16 edge connector”[1] “up to 32 lanes of PCIe Gen 5.0”, x16 edge connector[3] not published
Power 75W via slot; P-Series FHHL adds 75W through a 6-pin ATX connector[2] minimum 75W; DPU SKUs add an 8-pin ATX connector[3] not published
DOCA 3.5.0 supported, firmware 24.50.1002[11] supported, firmware 32.50.1002[11] not listed[12][13]

Two rows deserve emphasis. The BlueField-2 to BlueField-3 step doubled the core count, moved from A72 to A78, from DDR4 to DDR5, and from Gen4 to Gen5, while the boot medium became an eMMC plus a discrete 128GB SSD.[1][3] Lesson 1.3 opens each of those up.

The BlueField-4 row is deliberately thin. NVIDIA’s product page gives one sentence and one number, 800 Gb/s.[6] ServeTheHome’s coverage of the October 2025 announcement adds 64 Arm cores, a ConnectX-9 network core and early availability “as part of NVIDIA Vera Rubin platforms in 2026”.[7] Nothing about BlueField-4 appears in the DOCA 3.5.0 release notes, the 3.5.0 changes page or the 3.5.0 firmware table.[12][13][11] That is the entire verified record as of this lesson’s fetch date.

3Which NIC rides with which GPU

The Spectrum-X page states the pairing rule in one sentence per generation: “the BlueField-3 (for NVIDIA Hopper-era systems, optimized for RDMA/RoCE and multi-tenant isolation)”, “the ConnectX-8 (800 Gb/s total throughput via 2×400 G, PCIe Gen6, for NVIDIA Blackwell systems)”, and “the ConnectX-9 (1,600 Gb/s per GPU via 4×200 G SerDes, PCIe Gen6, for NVIDIA Vera Rubin NVL72 systems)”.[9] Read the parentheses carefully: BlueField-3 is differentiated by isolation, the two ConnectX parts by bandwidth. That is the SmartNIC-versus-DPU split from lesson 1.1, applied to AI factories.

ConnectX-8 adds a mechanical detail that matters in a server: its “32-lane PCIe bus is split into two 16-lane buses”, and with “Socket-Direct or the Multi-Host capability, the PCIe extension card is connected to the SuperNIC and is used as an end-point extension” — the second x16 arrives over a Gen5 auxiliary connection card or SFF-TA-1016 MCIO cable.[8] The same split exists on BlueField-3 DPU SKUs via the PCIe Auxiliary Card Kit (lesson 1.3).

At fabric scale, Spectrum-X “networks can scale up to 128,000 GPUs in two tiers”, and the Spectrum-X Photonics switches claim “10x greater mean-time-between-interruptions (MTBI), 5x longer uptime, and 5x better power efficiency compared to traditional pluggable transceiver-based switches”.[9] On the InfiniBand side the Quantum-X800 platform adds SHARP v4 in-network computing, adaptive routing and telemetry-based congestion control across the same ConnectX-8 and ConnectX-9 endpoints.[10]

4Firmware lines and the DOCA support boundary

Every adapter family has its own firmware major line, and the DOCA 3.5.0 general-support table lists one bundled build per family: BlueField-3 32.50.1002, BlueField-2 24.50.1002, ConnectX-9 82.50.1002, ConnectX-8 40.50.1002, ConnectX-7 28.50.1002, ConnectX-6 Lx 26.50.1002, ConnectX-6 Dx 22.50.1002, ConnectX-6 20.43.8004, ConnectX-5 and -5 Ex 16.35.8008, ConnectX-4 Lx 14.32.1912.[11] Two things follow. The leading number identifies the silicon (32 is BlueField-3, 40 is ConnectX-8) regardless of what the sticker says. And the current generations share the .50.1002 tail, so a mlxfwmanager --query that shows 32.50.1002 tells you both “BlueField-3” and “on the 3.5.0 pairing” in one line.[11][16]

The release notes list the adapters DOCA 3.5.0 supports: BlueField-3, BlueField-2, BlueField, ConnectX-9, -8, -7, -6 Lx, -6 Dx, -6, -5 and -5 Ex, -4 Lx and -4.[12] The changes page sets the exit: “DOCA 3.6.0 (October 2026) will be the final DOCA release to support ConnectX-4 Lx and ConnectX-5 devices” and “Starting with DOCA 4.0, scheduled for release in January 2027, these devices will no longer be supported”.[13] The older three-digit firmware tails on those rows (20.43.8004, 16.35.8008, 14.32.1912) are the visible sign of families no longer receiving the current build.[11]

The upper boundary is the MLNX_OFED line: “NVIDIA BlueField-2 and NVIDIA ConnectX-7 are the last generation of BlueField and ConnectX families that are supported by MLNX_OFED”, so BlueField-3, ConnectX-8 and ConnectX-9 hosts run DOCA-Host.[11] And for the AI-factory context, the release notes state that “DOCA v3.5.0 is tested as part of Spectrum-X reference architecture release version 2.3”.[12]

Episode 2 — Forty minutes later, in the room

How it ended

The network lead answers before you finish asking: isolation, two tenants per node. That makes the SuperNIC defensible on the twenty-four bare-metal nodes, and it caps the forty GPU nodes at NVIDIA’s 400 Gb/s BlueField-3 class[6] rather than the 800 Gb/s ConnectX-8 the Spectrum-X page pairs with Blackwell[9], which nobody had priced. Procurement now wants lead times on two part numbers instead of one. The PM on the bridge says “not announced” about BlueField-4, and he is right to: it has a product-page sentence and appears in none of the DOCA 3.5.0 pages[12][13]. Split the fabric. The first two cards reach the lab at four, and one of them will not boot.

Lab

Read-only inventory. Do not update firmware or change any parameter.

  1. Start MFT and list devices:[16] 
    sudo mst start
sudo mst status -v
    Expected: /dev/mst/mt41692_pciconf0 — 41692 is the device ID shared by all BlueField-3 DPUs and SuperNICs.[5]
  2. Query firmware and decode it:[16] 
    sudo mlxfwmanager --query
    Expected: FW 32.xx.xxxx on a BlueField-3. Record the full version, PSID and part number. Compare with 32.50.1002: an exact match means the card is on the 3.5.0 pairing; anything else, note the gap for lesson 2.x.[11]
  3. Count PCIe functions and record them; a BlueField-3 exposes several functions to the host, and their number will matter when you read modes in lesson 1.4: 
    lspci | grep -i -E 'mellanox|bluefield'
  4. Record the mode parameters without changing them:[15] 
    sudo mlxconfig -d /dev/mst/mt41692_pciconf0 q INTERNAL_CPU_MODEL INTERNAL_CPU_OFFLOAD_ENGINE
    Expected: ENABLED(0) on a DPU SKU (DPU mode), DISABLED(1) on a SuperNIC SKU (NIC mode). Write the SKU you infer next to the part number from step 2 and check it against the SKU tables in lesson 1.3.[14]

Retrieval check

9 questions from memory. Answer before looking anything up; misses become flashcards.

Explain it to a Dell SE

Explain to a Dell SE, in four sentences, how to pick the NVIDIA NIC for an HGX server from the GPU generation, and how to sanity-check the choice from a firmware query.

12 flashcards for this lesson — 0 in deck. Spaced review lives at /review.

Sources

Facts in this lesson were checked against BlueField-2/-3 HW user guides, NVIDIA Spectrum-X and Quantum-X800 pages, ConnectX-8 manual, DOCA 3.5.0 release notes / general support / changes pages, 2026-09-06. Dates are when each page was fetched.

  1. BlueField-2 DPU User Guide: Introduction · fetched 2026-09-06
  2. BlueField-2 DPU User Guide: Specifications · fetched 2026-09-06
  3. BlueField-3 User Guide: Introduction · fetched 2026-09-06
  4. NVIDIA BlueField-3 DPU Datasheet (PDF, APR21 revision) · fetched 2026-09-06
  5. NVIDIA BlueField-3 Networking Platform User Guide (index) · fetched 2026-09-06
  6. NVIDIA BlueField DPU product page · fetched 2026-09-06
  7. ServeTheHome: NVIDIA BlueField-4 with 64 Arm Cores and 800G Networking Announced for 2026 · fetched 2026-09-06
  8. ConnectX-8 SuperNIC User Manual: Introduction · fetched 2026-09-06
  9. NVIDIA Spectrum-X Ethernet platform · fetched 2026-09-06
  10. NVIDIA Quantum-X800 InfiniBand Platform · fetched 2026-09-06
  11. DOCA General Support (OS matrix, firmware table) · fetched 2026-09-06 · DOCA 3.5.0
  12. DOCA Release Notes v3.5.0 · fetched 2026-09-06 · DOCA 3.5.0
  13. DOCA 3.5.0 Changes and New Features · fetched 2026-09-06 · DOCA 3.5.0
  14. BlueField-3 User Guide: Specifications · fetched 2026-09-06
  15. BlueField Modes of Operation · fetched 2026-09-06 · DOCA 3.5.0
  16. MFT 4.34.1-18 LTS: General Information (tool list) · fetched 2026-09-06
  17. NGC catalog: nvidia/doca/doca container · fetched 2026-09-06 · DOCA 3.5.0

The same idea elsewhere

Other lessons that cover this ground, sometimes from another course's angle.