The networks in an AI factory, and what the RA refuses to specify

1Three fabrics you can touch

Start with the physical layer, because it is the part a Dell install team wires.

The HGX AI Factory RA defines three physical fabrics: the Compute (node east/west) network, built with switches using NVIDIA Spectrum technology in a full non-blocking fat tree topology; the CPU Converged (node north/south) network carrying storage and management; and the Out-of-Band Management network for infrastructure management traffic.^[3]

The compute fabric is rail-optimized through each GPU’s own ConnectX-8 SuperNIC.^[3] The recommended wiring is dual plane: each GPU’s 800 Gb/s splits into 2x400 Gb/s interfaces landing on different leaf switches, each plane an independent fabric that scales to 1024 interfaces of 400 Gb/s.^[3] Single plane is the documented cost option using a single 400 Gb/s connection per GPU and accepting a 50% reduction in total GPU bandwidth.^[3]

The converged fabric is deliberately unglamorous: each compute and management node is connected with two 400 GbE ports to two separate switches, reaching up to 40 GB/s per node.^[3] On GB300 NVL72 the same idea has two shapes - each compute tray connects to two separate switches using dual 400 Gb/s ports, while each management node connects to the same switches with four 200 Gb/s ports.^[4]

The OOB network provides bulk management 1 Gb RJ45 connectivity for all the nodes and uses low-cost bulk management switches; the NVIDIA SN2201 switch connects to the BMC and OOB 1 Gbps ports of those components.^[2]

2Six logical networks over one physical fabric

Above the cable the RA gets more granular. The NVL72 AI Factory logical architecture names six logical networks: GPU Compute (East/West), CPU Converged (North/South), Storage connectivity, Customer Network Connectivity, Support Server Networking, and Out-of-band Management Networking.^[1]

They are not six sets of switches. The separation is logical: VLAN isolation would be used to provide logical separation of the networks over the single physical fabric.^[1] The HGX RA uses the same sentence about the networks above the single physical fabric.^[2]

So the mapping is many-to-few. Compute has its own physical fabric. OOB has its own physical fabric. Everything else - storage, customer access, support servers, in-band management - rides the converged fabric separated by VLAN.

flowchart TB
  subgraph P1["Physical: compute fabric"]
    EW["GPU Compute east-west"]
  end
  subgraph P2["Physical: converged fabric, VLAN separated"]
    NS["CPU Converged north-south"]
    ST["Storage"]
    CU["Customer network"]
    SS["Support server networking"]
  end
  subgraph P3["Physical: OOB, 1 Gb RJ45"]
    OOB["BMCs, switch mgmt, DPU mgmt"]
  end
  EW --- NODE["Compute node"]
  NS --- NODE
  OOB --- NODE

The SuperPOD editions arrange the same traffic slightly differently but with the same idea. In-band management on the DC-busbar B300 RA carries cluster service management, NFS storage access, uplink connectivity and user access to Slurm or Kubernetes.^[5] The out-of-band management network is physically rolled up into the aggregation layer of each SU as a dedicated VXLAN.^[5] The customer edge is specified as at least 2x 100GbE links with DR1 single-mode connectivity, peered with BGP.^[5]

3Where the RA stops, on purpose

This is the part that saves you a quarter of an escalation.

Neither Enterprise RA logical-architecture page publishes BGP or EVPN configuration, an IP addressing plan, an MTU, DSCP, PFC or ECN values, adaptive-routing parameters or congestion-control settings.^[1][2] That was re-checked against both pages on 2026-09-07 and both answered no on every item.^[1][2]

That is not an omission, it is a boundary. Topology and hardware are stable for the life of a platform; configuration moves with every software release. So the RA describes what to buy and how to cable it, and the software documentation describes how to configure it:

Those documents are taught in the RoCE and Spectrum-X tracks rather than here. The skill this lesson wants is narrower and more useful in a meeting: name the document, do not improvise the value.

4Counting the ports nobody counts

Two counting habits separate a design review that works from one that produces a change order.

Out-of-band ports per node. The Quantum-X800 SuperPOD edition spells out that the OOB network reaches DGX B300 compute trays including system BMCs and BlueField-3 BMCs.^[6] That is two RJ45 ports per node before you count switch management ports, and it surprises people who budgeted one. An SN2201 provides 48 ports of 1GbE plus uplinks, so the node-to-OOB-switch ratio is roughly half of what a naive count suggests.^[2][8]

Storage bandwidth as a floor, not a target. The SuperPOD RA requires that I/O per node must exceed 40 GBps, and offers the storage fabric either as InfiniBand with MQM9700-NS2R for AC power or MQM9701-NS2R for DC, or as Ethernet with SN5600 for AC or SN5600D for DC.^[5] The GB200 SuperPOD publishes the blocking directly: a slightly blocking fabric with a blocking factor of 5:3 on the DGX node side, with 16x 800 Gbps of non-blocking bandwidth to the storage appliances.^[13]

Both habits share a shape. The RA gives ratios and floors; the port count comes from multiplying them by the customer’s node count, and the errors come from the ports nobody wrote down - the DPU’s BMC, the switch management port, the second converged port.