RoCE and RDMA for AI fabrics

Explain the verbs object model, pick the right transport, and read a RoCEv2 packet and a GID table well enough to say which version and which priority a connection is actually using.

1. 1.1 The verbs object model: device, PD, QP, CQ, MRThe card that says InfiniBand on an Ethernet cableU
2. 1.2 Transports and opcodes: what RC, UC and UD can actually doThe one requirement that changed the transportAn
3. 1.3 RoCEv1 vs RoCEv2: the encapsulationWorks in the rack, dies between racksAn
4. 1.4 GIDs: how a connection picks its RoCE versionThe benchmark crosses the subnet and the application does notAp
5. 1.5 QP state machine, retries, and connection setupForty minutes, every timeAn
6. 1.6 Multipath RC: a new transport in the annex, not in the APIThe annex he cannot readAn
7. Checkpoint

Build and defend the whole marking-and-congestion chain — application ToS through host trust state to switch queue, PFC vs ECN vs ZTR-RTT — and say which one a customer's symptom actually points at.

1. 2.1 ToS, DSCP, PCP, SL: where marking gets lostThe capture that ends the argumentAn
2. 2.2 Host QoS with mlnx_qos: trust state, prio2tc, ETSTwo identical nodes that are not identicalAp
3. 2.3 PFC, lossless and the case for lossyThe architect who wants more pauseE
4. 2.4 ECN, CNP and DCQCN: closing the loop02:10, and nothing is slowing downAn
5. 2.5 Zero Touch RoCE, ZTR-RTT CC and DOCA PCCThe line item they refuse to buyE
6. 2.6 Packet spraying and the reordering it hands to the NICThe sixth word on the labelE
7. Checkpoint

Produce and defend a number: run perftest and nccl-tests correctly, prove GPUDirect is actually on, and translate busbw into a statement about which piece of hardware is the bottleneck.

1. 3.1 What ib_write_bw actually measuresThe 180 that was never 180Ap
2. 3.2 NUMA, data validation, and doca_perftestThe first word on the lineAn
3. 3.3 GPUDirect RDMA: proving it is actually onWhichever slot was easiest to reachAn
4. 3.4 NCCL environment variables on a RoCE fabricThe design was correct; the traffic never entered itAp
5. 3.5 algbw, busbw, and reading a nccl-tests resultThe 1.75x he called marketingAn
6. Checkpoint

Run a RoCE fabric in production: know where every counter lives, work a symptom to a cause in a fixed order, configure the switch half on Cumulus and Dell SONiC, and plumb RDMA into Kubernetes.

1. 4.1 The counter map: sysfs, ethtool and rdma statisticThe dashboard that swears the rail is deadAp
2. 4.2 Triage in order: symptom to counter to cause01:40, and the bridge wants to change a DCQCN parameterE
3. 4.3 MTU, RoCE LAG, and what has to be installedThe bond that took the RDMA device with itAp
4. 4.4 The switch half: Cumulus NVUE and Dell Enterprise SONiCDoes this switch do RoCE, or only PFC?Ap
5. 4.5 RDMA in Kubernetes: the NVIDIA Network OperatorNobody proved the fabric, and acceptance is at nineAn
6. Checkpoint

Turn the technical material into what the job pays for: rail-aware designs on Dell hardware, defensible recommendations under customer pressure, and a mapped study plan for NVIDIA-Certified Professional: AI Networking.

1. 5.1 Rail-optimized fabrics on Dell hardwareThe two cards in the labelled slotsAn
2. 5.2 Scenario: "the 400G NIC only does 180 Gb/s"Two numbers, six people, and no command lineE
3. 5.3 Scenario: "we do not want to configure PFC"The slide that promises no switch configurationC
4. 5.4 NCP-AIN: what this course covers and what it does notEight exam seats in a draft emailE
5. 5.5 Positioning multipath: what ships, what is a specificationA quarter later, the RFP paragraphC
6. Checkpoint