Skip to content
nodeza.learn
RoCE course/5. FAE and cert/Lesson 5.4

NCP-AIN: what this course covers and what it does not

S5·E4Eight exam seats in a draft email · NVIDIA briefing room, Friday, the last ten minutes of a Dell account review

S5·E4Evaluate~35 minsources checked todaylab mutates hardwareverified against NCP-AIN certification page and Exam Study Guide document 4417000. MAY26, both re-fetched 2026-09-07

Builds on: Rail-optimized fabrics on Dell hardware, Scenario: "the 400G NIC only does 180 Gb/s"

Before you read: what do you already know?

3 quick questions. Wrong answers are fine and expected; trying first makes the lesson stick.

After this lesson you can

  • Recall the NCP-AIN logistics and the six weighted domains from the official certification page.
  • Map each exam objective onto a lesson in this course or onto a named gap.
  • Judge which parts of the blueprint you cannot rehearse on a Dell lab pair with no InfiniBand hardware.
  • Produce a dated study plan with one gap per week and a self-assessment against the study guide's job responsibilities.

Episode 4 — Eight exam seats in a draft email

The situation · NVIDIA briefing room, Friday, the last ten minutes of a Dell account review

The acceptance review passed at 09:40. Two nodes at rail speed, one switch-side change, a written recommendation the network lead has copied into a notebook that no longer closes flat, and a purchase order moving through procurement, who wants to know the lead time on eight-way nodes and is, this once, asking the second-best question in the room. Ten minutes are left in the account review. The account manager waits until it empties and asks the one a logo cannot answer: which customer conversations route to your team, and which route to an NVIDIA specialist? Someone in her org has already decided the answer is a certification. A purchase order for eight exam seats is sitting in a draft email, and she would like your blessing before she sends it.

Certifications exist to make that routing decision legible to people who cannot audit your skills directly. The NCP-AIN page states the terms plainly - 120 minutes, 70 to 75 questions, $400, valid for two years - and the entry condition is not a course but “Two to three years of operational experience working in a data center with NVIDIA hardware solutions”.[1] The useful part is the blueprint: six weighted domains that sum to 100, with InfiniBand at 30% and Spectrum networking at another 30%.[1][2] The PM’s roadmap slide is still on the screen behind her, and everything on it is not announced.

Eight seats is a purchase. A mapped blueprint is an answer.

Start with the logistics and the six domains.

1The logistics and the six domains

The official page states the exam facts plainly: code NCP-AIN, “120 minutes”, “70 to 75 questions”, “$400”, and “This certification is valid for two years from issuance. Recertification may be achieved by retaking the exam.” The prerequisite is “Two to three years of operational experience working in a data center with NVIDIA hardware solutions. The candidate should be able to deploy and manage NVIDIA AI networking infrastructure in support of AI workloads.”[1] Registration is through Certiverse, and the official study guide carries the document code 4417000. MAY26.[2]

The six domains and their weights, which sum to 100:[1][2]

Domain Weight
AI Data Center Design and Optimization 5%
NVIDIA Spectrum Networking 30%
NVIDIA InfiniBand Networking 30%
Kubernetes Integration 5%
Troubleshooting Tools 20%
Automation and Configuration 10%

The guide also lists eight job responsibilities, and they read like an FAE job description rather than an operator one: configure high-performance networking for AI workloads, optimize for throughput and latency, fine-tune protocols, integrate NVIDIA networking with servers, GPUs and storage, deploy in data centers, “Diagnose and resolve networking issues impacting AI workloads”, “Provide technical support and guidance to teams managing AI infrastructure”, and collaborate with data scientists and IT professionals to understand requirements.[2] Two of the guide’s suggested readings are Dell documents - “AI Overview and Fabric Requirements | Dell Technologies” and the Dell Info Hub Spectrum-X overview - which is direct evidence that the Dell OEM account sits inside this exam’s frame of reference.[2]

2What this course covers, objective by objective

Roughly 35% of the blueprint is carried by lessons you have already done or will do in this course. The method: weight each domain by the share of its objectives this course maps - Design 5%, four of eight Spectrum objectives (~15%), Kubernetes 5%, two of five Troubleshooting objectives (~8%), and a sliver of Automation.

Objective Where it lives here
2.1 Configure Spectrum-X switches for RoCE roce-m4-04-switch-side-roce, supported by roce-m2-03-pfc-and-lossless
2.2 Enable and verify QoS ECN and PFC roce-m2-01 through roce-m2-04, host and switch halves
2.7 Install NVIDIA DOCA roce-m4-03-mtu-lag-packaging
2.8 Configure SuperNIC functionality for packet processing and congestion control roce-m2-05-ztr-rtt-and-pcc, roce-m2-04-ecn-cnp-dcqcn
4.1 and 4.2 Deploy and verify the Network Operator roce-m4-05-roce-in-kubernetes
5.3 Verify low-latency interconnects between GPUs CPUs and storage roce-m3-01, roce-m3-03, roce-m3-05
5.5 Use ib_write_lat and ib_write_bw to diagnose connectivity roce-m3-01, roce-m3-02, roce-m4-02
1.1 to 1.3 AI factory architecture, rail topologies, GPU-to-GPU roce-m5-01-rail-design-dell, roce-m3-03, roce-m3-05

Three of those deserve a note. Objective 2.2 names “advanced features like adaptive routing, and telemetry” alongside QoS, ECN and PFC; this course touches adaptive routing only as NCCL_IB_ADAPTIVE_ROUTING and does not cover telemetry at all.[2] Objective 2.1’s practical surface is the single NVUE command and its modes - nv set qos roce, mode lossy, and the defaults each programs.[9] Objective 2.7 reduces to the DOCA-Host profiles and their contents, where doca-ofed is “Driver-only installation equivalent to MLNX_OFED” and the lighter doca-roce profile carries “rdma-core, ofed-scripts, mlnx-tools, mlnx-ofa_kernel, perftest”.[10] Domain 5’s own reading list names “linux-rdma/perftest” and the NCCL troubleshooting page, which is exactly the material behind module 3 and module 4 here.[2][8][7]

Rendering diagram…
Diagram source (Mermaid)
flowchart TD
  A[NCP-AIN blueprint] --> B[Covered here about 35 percent]
  A --> C[Gap about 65 percent]
  B --> B1[Spectrum RoCE QoS ECN PFC]
  B --> B2[DOCA install]
  B --> B3[Perftest and counters]
  B --> B4[Network Operator]
  C --> C1[InfiniBand domain 30 percent]
  C --> C2[BGP-EVPN Air WJH NetQ]
  C --> C3[cl-resource-query and UFM]
  C --> C4[NVUE templates and Ansible]
  C1 --> D[One gap per week]
  C2 --> D
  C3 --> D
  C4 --> D
The blueprint split into what this course carries and what it does not, with every gap feeding one week of the study plan.

3The 65% this course does not teach

Being explicit about the gap is the point of this lesson. No lesson was invented for these because the research notes contain no material for them, and inventing coverage is worse than naming a gap.

All of Domain 3, InfiniBand, 30% on its own. The objectives are initial configuration and provisioning including high availability, partition keys for secure multi-tenancy, InfiniBand QoS and adaptive routing, and using UFM to monitor link status and bandwidth utilization.[2] This course meets InfiniBand only as the contrast commands in the triage playbook.

Four Spectrum objectives. 2.3 BGP-EVPN multi-tenancy, 2.4 NVIDIA Air as a simulation and design tool, 2.5 in-band telemetry and What Just Happened, and 2.6 NetQ for real-time monitoring including congestion detection and latency measurement.[2]

Two Troubleshooting objectives. 5.1 “Use tools like cl-resource-query to check resource allocation in Spectrum-X environments” and 5.4 “Use UFM system health to diagnose InfiniBand issues”; 5.2 WJH again.[2]

Most of Automation. 6.1 NVUE templates entirely, and 6.2 Ansible beyond the single dellemc.enterprise_sonic.sonic_roce task that appears in the switch-side lesson.[11]

There is a second gap that is about your bench, not the syllabus. Objective 5.5 names “ib_write_lat, ib_write_bw, ibping, ibstat, ibdiagnet, ibnodes, and iblinkinfo”.[2] On a Dell lab pair with ConnectX and BlueField-3 over Ethernet, the first two run natively; the rest need an InfiniBand fabric with a subnet manager running. Mark them as read-only study and get partner-lab or customer-lab time if you want them in your hands.

Finally, treat the guide itself as a moving target. Its Kubernetes domain suggests “NVIDIA Network Operator v25.7.0” while the shipping documentation index lists v26.7.0 as the current release, so the reading is already one line behind.[2][5][6] Re-fetch the certification page and the guide at study time and diff the weights and objective wording against the table above.

4The associate exam, and who it is for

The other certification a Dell account team will ask about is NCA-AIIO: code NCA-AIIO, one hour, 50 questions, $125, valid two years, study guide document 4694224. Jan26.[3][4] It is positioned as “an entry-level credential that validates the foundational concepts of AI computing related to infrastructure and operations… appropriate for a wide variety of job roles, from technical pre-sales to data center operations.”[4]

Its weights are Essential AI knowledge 38%, AI Infrastructure 40% and AI Operations 22%. The networking content lives entirely in four Domain 2 objectives - “Determine networking requirements for AI workloads”, “Identify and describe data center networking protocols and key concepts”, “Identify high-speed data center network options and their use cases”, and “Explain the purpose and benefits of a DPU in a data center” - plus one recommended unit of the fundamentals course.[4] The words RDMA, RoCE and InfiniBand appear in no NCA-AIIO objective at all.[4]

The practical consequence: this course massively over-serves NCA-AIIO. If you have worked through modules 1 to 4, the associate exam’s networking content is beneath the level you are already operating at, and its other 60% is AI infrastructure and operations material this course never touches. Recommend NCA-AIIO to the Dell pre-sales team and to operations generalists; recommend NCP-AIN to the people who will own the fabric.

Qualification request · decision 1/6A 0 · P 0 · S 0

Brief — Email, Tuesday: "We want to offer the BlueField-3 B3140H SuperNIC in the XE9780 for Spectrum-X customers. Can NVIDIA qualify it? I need a plan by Friday."

Dell platform PM (17G AI servers): So — can you get it qualified? What do you need from us to start?

Scoping drill: the exam's job responsibilities seven and eight are this conversation - guiding a team and understanding their requirement before committing. Score yourself on positioning and safety.

5Building the plan, and grading yourself honestly

A study plan for this exam has three parts, and only the third is unusual.

The re-fetch. Before booking, pull the certification page and the study guide again and diff them against the table in segment 1. You are checking three things: the six weights, the objective wording, and the suggested readings.[1][2] Record the guide’s document code with the date you fetched it, so the next diff has a baseline.

One gap per week. Take the four gap groups from segment 3 and give each one a week with a named deliverable, not a reading list. InfiniBand fabric management is the largest and deserves more than a week on its own; the Spectrum gaps (Air, WJH, NetQ, BGP-EVPN) are the ones a Dell FAE is most likely to meet in a customer conversation next quarter.

The self-assessment against the eight job responsibilities. Score yourself one to five on each of the guide’s responsibilities, and for every score below four write the single piece of evidence that would raise it - a lab you ran, a finding you wrote, a customer conversation you handled.[2] This is the part that transfers to the day job whether or not you sit the exam, because it is the same list the hiring conversation used.

What you can rehearse without leaving the Dell lab is a slice of Domain 2 and a slice of Domain 5 - install and verify DOCA-Host (2.7), configure host QoS and ECN (2.2), run the perftest pair (5.5), diagnose an injected fault from counters (5.3) - two of the eight Spectrum objectives and two of the five Troubleshooting objectives, about 15% of the weighted exam. It is still the highest-return drill available to you and it needs no NVIDIA switch; the rest of Domains 2 and 5 is in the gap list above.[2][8]

Case closed — one page instead of a purchase order

How it ended

One page goes back, not a blessing. About 35% of the blueprint is already carried by modules the team has run, mapped to lesson ids they can re-run tonight. The rest is named, not glossed - InfiniBand at 30%, plus BGP-EVPN, Air, WJH, NetQ and NVUE templates - each with one week and one deliverable.[2] The associate exam goes to the pre-sales generalists, whose networking content is four Domain 2 objectives.[4]

The cluster order clears Monday. The night-shift operator sends a photo: eight reprinted labels on the slots the priority list chose, and a ninth on the study plan: RE-FETCH ME.

What you say: “Route the fabric conversations to us and the rest to the specialists. Ask me again in a quarter, after I re-fetch the guide and diff it.”

Turning the blueprint into eight dated weeks

Starting point: modules 1 to 4 of this course completed, a Dell lab pair with ConnectX and BlueField-3 over Ethernet, containerlab with Dell Enterprise SONiC, no NVIDIA switch and no InfiniBand.

Week 0 - re-fetch. Pull the certification page and the study guide, diff the six weights and the objective wording against the table in segment 1, and record the guide’s document code with today’s date. Deliverable: a one-page diff, even if it is empty.

Week 1 - Domain 5 drill on the lab pair, timed at 45 minutes: install and verify DOCA-Host (ofed_info -s), configure host QoS and ECN and read the trust state back, run ib_write_lat and ib_write_bw with identical flags on both sides, and diagnose one fault a colleague injected using counters only. Deliverable: the score per objective in the guide’s own wording.

Week 2 - Domain 2 switch half in simulation. Build the NVUE lossless and lossy captures, plus the Dell Enterprise SONiC roce enable path in containerlab. Deliverable: four config artifacts and a note on which of them containerlab actually enforces without a real ASIC.

Week 3 - the first gap: NVIDIA Air as a design tool, objective 2.4. Deliverable: one simulated topology you built yourself, not a tutorial you followed.

Week 4 - telemetry gap: WJH and NetQ, objectives 2.5 and 2.6. Deliverable: written descriptions of what each tool answers that counters do not.

Weeks 5 and 6 - InfiniBand, the 30% domain. Split it: initial configuration, HA and PKeys in week 5; IB QoS, adaptive routing and UFM in week 6. Deliverable each week: a written comparison against the RoCE equivalent you already know, because contrast is how this material sticks when you cannot touch it.

Week 7 - Automation, objectives 6.1 and 6.2. Deliverable: one NVUE template and one Ansible playbook beyond sonic_roce, both run against something.

Week 8 - self-assessment against the eight job responsibilities, then book. Deliverable: the score sheet with one evidence item per responsibility scored below four.

Lab

Mutating steps ahead. Needs a maintenance window, out-of-band access (BMC/iDRAC/rshim console) and a rollback path. Record the pre-flight inventory before changing anything. Never on a production host.

Timed practical, 45 minutes on the Dell-lab pair, covering the Spectrum and Troubleshooting objectives you can actually reach. Firmware, card mode and switch configuration are untouched, but steps 2 and 3 change host software and host QoS state - do not run them on a host carrying other work, and capture step 1’s five outputs first as the rollback baseline.

  1. Pre-flight inventory: ibv_devinfo, ibdev2netdev, ofed_info -s, show_gids and mlnx_qos -i <interface> on both hosts. Record all five outputs - this is your rollback baseline for step 2 and step 3.
  2. Objective 2.7, DOCA install and verify. Confirm the installed profile and version with ofed_info -s, and if you install or change a profile, follow it with sudo /etc/init.d/openibd restart and sudo mst restart. Rollback: reinstall the profile recorded in step 1.
  3. Objective 2.2, host QoS and ECN. Set the trust state and PFC bitmap with mlnx_qos, and enable the ECN reaction and notification points for your RoCE priority under /sys/class/net/<interface>/ecn/. Read everything back with mlnx_qos -i <interface>. Rollback: restore the exact trust state, PFC list and ECN enable bits recorded in step 1; the ECN sysfs bits do not survive a reboot, the mlnx_qos settings must be restored by hand.
  4. Objective 5.5, the measurement. Run ib_write_lat and then ib_write_bw with identical flags on both sides, with --report_gbits and an explicit -q and -x. Expected: a latency figure you can defend as half a round trip, and a bandwidth figure near line rate. If either is unstable, fix that before step 5.
  5. Objective 5.3 and 5.5, the diagnosis. Have a colleague inject one fault - a wrong GID index, a missing traffic class, or trust set to pcp - and diagnose it from counters alone using ethtool -S per-priority counters and the hw_counters directory. Expected: a named cause and the counter that proved it, inside ten minutes.
  6. Score each step against the objective’s own wording from the guide and write the sentence you would say to a customer for each. Optional, in a customer or partner lab: repeat the InfiniBand-only commands - ibping, ibstat, ibdiagnet, ibnodes, iblinkinfo - on real IB gear, and note which of them have no RoCE equivalent at all.

Retrieval check

10 questions from memory. Answer before looking anything up; misses become flashcards.

Explain it to a Dell SE

Explain to your manager, in four sentences, what this course qualifies you to sit and what it does not.

14 flashcards for this lesson — 0 in deck. Spaced review lives at /review.

Sources

Facts in this lesson were checked against NCP-AIN certification page and Exam Study Guide document 4417000. MAY26, both re-fetched 2026-09-07. Dates are when each page was fetched.

  1. NVIDIA-Certified Professional: AI Networking (NCP-AIN) certification page · fetched 2026-09-07
  2. NVIDIA-Certified Professional: AI Networking Exam Study Guide (doc 4417000. MAY26) · fetched 2026-09-07
  3. NVIDIA-Certified Associate: AI Infrastructure and Operations (NCA-AIIO) certification page · fetched 2026-09-07
  4. NVIDIA-Certified Associate: AI Infrastructure and Operations Exam Study Guide (doc 4694224. Jan26) · fetched 2026-09-07
  5. NVIDIA Cloud Orchestration documentation index (Network Operator versions) · fetched 2026-09-07
  6. NVIDIA Network Operator v26.7.0 · fetched 2026-09-07
  7. Networking Troubleshooting - NCCL 2.31.2 documentation · fetched 2026-09-07
  8. OFED Performance Tests README (linux-rdma/perftest master) · fetched 2026-09-07
  9. RDMA over Converged Ethernet - RoCE | Cumulus Linux 5.18 · fetched 2026-09-07
  10. DOCA Profiles - DOCA-Host · fetched 2026-09-07
  11. dellemc.enterprise_sonic.sonic_roce module - Manage RoCE QoS configuration on SONiC · fetched 2026-09-07
  12. Doubling all2all Performance with NVIDIA Collective Communication Library 2.12 (NVIDIA Technical Blog) · fetched 2026-09-07

The same idea elsewhere

Other lessons that cover this ground, sometimes from another course's angle.