Dell AI Factory scenarios and the NCA-AIIO positioning

1The bill of materials you are allowed to quote

There is exactly one Dell AI Factory configuration in this course that can be quoted line by line, and it is worth memorising because it ends arguments. Dell publishes a solution brief for the PowerEdge XE9680 based Dell AI Factory with NVIDIA, “endorsed by NVIDIA Enterprise Reference Architectures (Enterprise RAs) aligned with 2-8-9-400 configuration”.^[1]

Each XE9680 carries “two high-performance processors and eight NVIDIA H200 SXM GPUs”, and the four R670 servers “provide dedicated resources for orchestration and system administration”.^[1] The table carries one footnote that matters more than its size suggests: “Larger scale clusters supported and require additional network switching.”^[1] Two switches for the converged network is a starting point, not a ceiling.

The label itself is the trap. NVIDIA’s Enterprise Reference Architecture index lists patterns including 2-8-9-400, 2-8-10-400, 2-8-9-800 and 2-8-5-200, and explains none of the digits anywhere on the page.^[3] The obvious reading is not stated by either vendor, so treat 2-8-9-400 as an undecoded pattern name. What the index does state is scale: Enterprise RAs cover AI Factories “ranging from 32 to 256 GPUs”.^[3]

2Two fabrics, two design points, and the question about which NOS they bought

The brief separates two networks with two different design points, and conflating them is a common source of confused escalations. The GPU network: “Spectrum-X provides a non-blocking, rail-optimized spine-leaf fabric that ensures deterministic, low-diameter GPU paths. With RDMA/RoCEv2, PFC, ECN, adaptive routing, and congestion control, the fabric delivers low-latency, predictable, and highly scalable performance for LLM training workloads.”^[1] The enterprise network: “The platform uses an EVPN-VXLAN fabric that supports scalable north-south traffic, workload mobility, and tenant segmentation through a modern BGP-based control plane. EVPN-Multihoming (EVPN-MH) provides active/active server connectivity without MLAG.”^[1]

Those are different problems solved by different mechanisms on the same purchase order. Rail optimisation and congestion control decide whether a collective finishes on time; EVPN-MH decides whether a server keeps both uplinks live without a chassis pairing protocol. When a customer says “the fabric”, ask which one.

The deployment layer is also published: “Using Kubernetes operators—including the NVIDIA GPU Operator, NVIDIA Network Operator, NVIDIA NIM Operator and Dell CSI Operator—the environment can be provisioned, configured, and managed using declarative, cloud-native patterns.”^[1] Dell also claims the design “aligns directly with the NVIDIA AI Enterprise infrastructure support matrix and the Spectrum-X validated solution stack”.^[1] That claim is the reason the version arithmetic in this course belongs in a Dell conversation at all: alignment to a support matrix is only meaningful if the four versions on the cluster are inside their windows.^[6][7]

Then there is the network operating system. Dell announced that it is “delivering NVIDIA Spectrum-X Ethernet with Enterprise SONiC Distribution by Dell Technologies—a validated, open networking solution engineered for AI”, naming RoCEv2 for direct GPU memory access over Ethernet, adaptive routing, congestion management using ECN marking and PFC, and enhanced telemetry.^[2] The announcement names no switch model numbers and no SONiC release numbers.^[2] That absence is load-bearing for anyone practising on Dell Enterprise SONiC 4.5.1 in containerlab: the lab image can teach the PFC, ECN and DSCP-trust half of the RoCE story, and it cannot be claimed to be the validated release.

3Deliverable one and two: the qualification answer and the escalation intake

An FAE produces a small number of repeatable artifacts. The first is the qualification answer to “which CNI do you support?” — a question that assumes one layer where there are two. The answer has a fixed shape: NVIDIA does not replace the primary CNI, whatever the platform ships; it adds Multus plus a device plugin for the fast path, and what it does constrain is the secondary path and the driver.^[8] Then the one check that earns the answer: confirm the primary CNI has not claimed the RDMA-capable PFs as pod uplinks, because a node where the same interface is both the Calico or Cilium uplink and the device plugin’s selector gives a working cluster that silently loses RDMA whenever the CNI reconfigures the interface.^[8]

The second is the escalation intake form, and its whole purpose is to make you collect evidence before offering theory. Four versions, in this order, then two commands:

# the four versions, in dependency order
kubectl version                                   # 1. Kubernetes
helm list -A                                      # 2+3. GPU Operator, Network Operator
kubectl get nicclusterpolicy nic-cluster-policy -o yaml \
  | grep -A3 ofedDriver                           # 4. DOCA-OFED driver tag

# the two pieces of evidence no theory survives
kubectl describe node <node> | grep -A20 Allocatable
kubectl netop-sosreport --verbose --log-lines 10000

Kubernetes comes first because it bounds both operators: Network Operator 26.7.0 states its Kubernetes range as >=1.32 and <=1.36 while GPU Operator 26.7.x states 1.33—1.37, so only 1.33 through 1.36 satisfies both published windows.^[6][7] The driver tag comes last because it is what decides whether the RDMA and GPUDirect path can bind at all; 26.7.0 ships GA doca3.5.0-26.07-0.7.7.0-0 and LTS doca3.2.2-25.10-2.4.1.0-4.^[6] Collect the sos-report before changing anything, because it captures previous-container logs you are about to destroy.^[10]

4Deliverable three: the positioning answer, and the escalation that is really a NUMA problem

The positioning answer has three forks, and each one is settled by a published line rather than an opinion.

Network Operator versus DPF is settled by the platform matrix: the Network Operator supports BlueField-3 and BlueField-3 SuperNIC in NIC mode only.^[6] A customer who wants DPU-mode offload with services managed on the card is asking for DPF, which treats the card as a second computer with its own cluster. A customer who wants pods on the host to reach the fabric fast is asking for the Network Operator. That single line resolves most of the scoping conversation without any architecture debate.

Converged versus split fabric is settled by a question, not by an assertion. The published 2-8-9-400 brief is converged: two SN5610 carry the converged network with one SN2201 for out-of-band.^[1] Older Dell XE9680 material described a three-network split of frontend, backend and out-of-band, but that material could not be fetched for this course, so it is not cited here and must not be presented as a second design you have read. Ask which generation the customer is quoting.

Which NOS they bought is settled by segment 2. Ask it early.

The third fork is where positioning turns into an escalation. “Training is slow” on a dual-socket XE9680 with eight GPUs and multiple NICs is very often a NIC on one socket feeding a GPU on another. Topology Manager’s default policy is none, which attempts no alignment at all; single-numa-node requires every requested resource to come from one NUMA node or the pod is rejected, and restricted admits only when all resources align to a common set.^[9] The Kubernetes-side fix is single-numa-node with pod scope plus CPU Manager on static, and the pod requesting the GPU and the NIC resource in the same container so both hint providers vote.^[9][8] Then warn the customer about the inversion before it happens: pods start getting rejected with TopologyAffinityError because no single NUMA node has both a free GPU and a free NIC.^[9] That rejection is the system working. The answers are restricted with prefer-closest-numa-nodes, or fixing the physical NIC placement — not switching Topology Manager back off.^[9]

# /var/lib/kubelet/config.yaml
topologyManagerPolicy: "single-numa-node"
topologyManagerScope: "pod"
cpuManagerPolicy: "static"

spec:
  containers:
    - name: trainer
      securityContext:
        capabilities:
          add: ["IPC_LOCK"]
      resources:
        limits:      # requests must match limits → Guaranteed QoS
          nvidia.com/gpu: "1"
          nvidia.com/hostdev: "1"
          cpu: "8"

5Calibrating the certification claim

The NCA-AIIO is worth holding and it is not what this course is. The published facts: 50 questions, “One hour”, $125, online and remotely proctored, and “This certification is valid for two years from issuance. Recertification may be achieved by retaking the exam.”^[4] The recommended preparation is the self-paced “AI Infrastructure and Operations Fundamentals” course, typical completion time seven hours.^[4]

The blueprint is an associate-level document. It defines three domains — Essential AI knowledge at 38%, AI Infrastructure at 40%, AI Operations at 22% — across 22 numbered objectives, and describes the audience as “IT professionals new to AI operations and infrastructure” across roles “from technical pre-sales to data center operations”.^[5] Nothing in those 22 objectives names CNI, Multus, SR-IOV, RDMA or the Network Operator.^[5]

So the honest sentence is: this course over-serves a subset of objectives rather than covering the exam. It contributes to 2.5 “Identify key components and considerations of a cluster of an accelerated infrastructure”, 2.7 “Determine networking requirements for AI workloads”, 2.8 “Identify and describe data center networking protocols and key concepts”, 2.9 “Identify high-speed data center network options and their use cases”, 2.10 “Explain the purpose and benefits of a DPU in a data center”, 3.1, 3.2 and 3.4.^[5] It contributes almost nothing to Domain 1 and to the monitoring objective 3.3, and Domains 1 and 3 together carry 60% of the exam.^[5] Someone who studies only this course will be over-prepared on rails and under-prepared on training-versus-inference architecture, power and cooling, and DCGM.

Say that plainly in an interview. “This material goes deeper than the exam on Kubernetes networking and does not cover most of Domain 1” is a stronger statement about your judgement than any claim of exam readiness.