InfiniBand with Quantum
0 of 24 cases closed
Foundations
Subnet management
Routing & design
Diagnostics
FAE & cert
InfiniBand with Quantum
OpenSM, UFM, partitions, adaptive routing and SHARP on NVIDIA Quantum fabrics.
InfiniBand from the subnet manager up: LIDs, GIDs and GUIDs; OpenSM, the switch-embedded SM and UFM; PKey partitions and SL/VL QoS; routing engines, adaptive routing, SHIELD and SHARP; rail-optimized fat trees on Quantum-2 (QM9700) and Quantum-X800 (Q3400); the infiniband-diags / ibdiagnet / mlxlink toolbox; and the Dell XE9680 side of an eight-rail pod. Every lesson ships two lab variants (no IB hardware, or the Dell-lab ConnectX / BlueField-3) and the course ends with FAE scenarios plus the InfiniBand and Troubleshooting domains of NCP-AIN.
Who it is for: FAEs and cluster engineers bringing up and operating Quantum InfiniBand fabrics.
- Not started
- In progress
- Complete, case open
- Case closed
- Locked
- Checkpoint
- A case closes when the lesson’s retrieval check reaches 80%.
Describe an InfiniBand subnet correctly — entities, addresses, link rates and IPoIB — and read a host's IB state without guessing.
- 1.1Who's who in an InfiniBand subnetNobody on the call knows who the master isUnderstand25 min
- 1.2LIDs, GIDs, GUIDs and LMCThe node that vanished at 03:12Understand25 min
- 1.3Speeds, widths, and the link that says SDRA photograph of an iDRAC screenAnalyze25 min
- 1.4IPoIB: the management plane on an IB fabricThe tenant network that comes up and answers nothingApply30 min
- 1.5Reading a host's IB state in 60 secondsThe card the technician is already holdingApply20 min
- Module 1 checkpoint14 questions · pass mark 80% · 12 min
Stand up and secure the control plane of an InfiniBand fabric: OpenSM or UFM, PKey partitions, SL/VL QoS, and the MAD keys — without locking yourself out.
- 2.1Bringing up OpenSMBoth ports are Active and nobody started a subnet managerApply30 min
- 2.2Where the SM runs: switch, host, or UFMThe slide says embedded SM, and underneath it says SHARPAnalyze30 min
- 2.3Partitions and PKeys: the most common mistake in IBThe tenant partition went in and half the cluster stopped talkingAnalyze35 min
- 2.4QoS: service levels, virtual lanes, SL2VL and VL arbitrationThe printed QoS policy and the storage team that starves anywayAnalyze30 min
- 2.5Keys, and locking a fabric down without locking yourself outThe hardening plan that would blind the fabric it hardensEvaluate30 min
- Module 2 checkpoint14 questions · pass mark 80% · 12 min
Choose a routing engine, a topology and a Quantum platform for a customer's fabric, and state exactly what adaptive routing, SHIELD and SHARP each require before they will work.
- 3.1Routing engines and how OpenSM picks a pathSUBNET UP, and the tree is not a treeAnalyze30 min
- 3.2Adaptive routing, SHIELD and hash-based forwardingThe call at 02:10, the bridge at 02:40Analyze30 min
- 3.3SHARP: in-network reductionThe server they want to deleteAnalyze30 min
- 3.4Topologies: fat-tree, rail-optimized, twin-planeThe budget cut, read in a hotel lobbyAnalyze30 min
- 3.5Quantum switches and the LinkX BOMThe cable that fits and then cooksEvaluate35 min
- Module 3 checkpoint14 questions · pass mark 80% · 12 min
Diagnose an InfiniBand fabric with infiniband-diags, ibdiagnet, mlxlink and the firmware tools, in the order that finds the fault fastest and with evidence a customer can send you.
- 4.1Discovering a fabric: ibnetdiscover and iblinkinfoEverything came backApply25 min
- 4.2Counters and errors that mean somethingSeven digits and a purchase orderAnalyze30 min
- 4.3ibdiagnet: the command you run before escalatingThree theories and eleven hoursApply35 min
- 4.4Link surgery and PHY triageSend us a cableAnalyze30 min
- 4.5Drivers and firmware: DOCA-OFED, mlxfwmanager, mlxconfig, NVOSThe migration plan with one line missingApply30 min
- Module 4 checkpoint14 questions · pass mark 80% · 12 min
Handle a Dell OEM InfiniBand engagement end to end — design review, escalation bridge call, performance proof — and pass the InfiniBand and Troubleshooting domains of NCP-AIN.
- 5.1Proving the fabric: perftest and nccl-testsThe number nobody can defendApply35 min
- 5.2Design review: an eight-rail IB pod on Dell XE9680Sixty-four nodes and a quote nobody derivedCreate35 min
- 5.3Escalation bridge call: five InfiniBand ticketsTen minutes on a bridge with forty people listeningEvaluate30 min
- 5.4NCP-AIN prep: blueprint, drills and honest gapsThe question before the account reviewEvaluate30 min
- Module 5 checkpoint14 questions · pass mark 80% · 12 min