InfiniBand with Quantum

Describe an InfiniBand subnet correctly — entities, addresses, link rates and IPoIB — and read a host's IB state without guessing.

1. 1.1 Who's who in an InfiniBand subnetNobody on the call knows who the master isU
2. 1.2 LIDs, GIDs, GUIDs and LMCThe node that vanished at 03:12U
3. 1.3 Speeds, widths, and the link that says SDRA photograph of an iDRAC screenAn
4. 1.4 IPoIB: the management plane on an IB fabricThe tenant network that comes up and answers nothingAp
5. 1.5 Reading a host's IB state in 60 secondsThe card the technician is already holdingAp
6. Checkpoint

Stand up and secure the control plane of an InfiniBand fabric: OpenSM or UFM, PKey partitions, SL/VL QoS, and the MAD keys — without locking yourself out.

1. 2.1 Bringing up OpenSMBoth ports are Active and nobody started a subnet managerAp
2. 2.2 Where the SM runs: switch, host, or UFMThe slide says embedded SM, and underneath it says SHARPAn
3. 2.3 Partitions and PKeys: the most common mistake in IBThe tenant partition went in and half the cluster stopped talkingAn
4. 2.4 QoS: service levels, virtual lanes, SL2VL and VL arbitrationThe printed QoS policy and the storage team that starves anywayAn
5. 2.5 Keys, and locking a fabric down without locking yourself outThe hardening plan that would blind the fabric it hardensE
6. Checkpoint

Choose a routing engine, a topology and a Quantum platform for a customer's fabric, and state exactly what adaptive routing, SHIELD and SHARP each require before they will work.

1. 3.1 Routing engines and how OpenSM picks a pathSUBNET UP, and the tree is not a treeAn
2. 3.2 Adaptive routing, SHIELD and hash-based forwardingThe call at 02:10, the bridge at 02:40An
3. 3.3 SHARP: in-network reductionThe server they want to deleteAn
4. 3.4 Topologies: fat-tree, rail-optimized, twin-planeThe budget cut, read in a hotel lobbyAn
5. 3.5 Quantum switches and the LinkX BOMThe cable that fits and then cooksE
6. Checkpoint

Diagnose an InfiniBand fabric with infiniband-diags, ibdiagnet, mlxlink and the firmware tools, in the order that finds the fault fastest and with evidence a customer can send you.

1. 4.1 Discovering a fabric: ibnetdiscover and iblinkinfoEverything came backAp
2. 4.2 Counters and errors that mean somethingSeven digits and a purchase orderAn
3. 4.3 ibdiagnet: the command you run before escalatingThree theories and eleven hoursAp
4. 4.4 Link surgery and PHY triageSend us a cableAn
5. 4.5 Drivers and firmware: DOCA-OFED, mlxfwmanager, mlxconfig, NVOSThe migration plan with one line missingAp
6. Checkpoint

Handle a Dell OEM InfiniBand engagement end to end — design review, escalation bridge call, performance proof — and pass the InfiniBand and Troubleshooting domains of NCP-AIN.

1. 5.1 Proving the fabric: perftest and nccl-testsThe number nobody can defendAp
2. 5.2 Design review: an eight-rail IB pod on Dell XE9680Sixty-four nodes and a quote nobody derivedC
3. 5.3 Escalation bridge call: five InfiniBand ticketsTen minutes on a bridge with forty people listeningE
4. 5.4 NCP-AIN prep: blueprint, drills and honest gapsThe question before the account reviewE
5. Checkpoint