Writing a DOCA validation plan for an OEM server

1Scope: what the plan promises and to whom

A qualification plan is a promise with a boundary. It says: this PowerEdge model, in this riser configuration, with this BlueField-3 SKU at these firmware versions, in these modes, on this host OS, passed these checks, and here is the evidence. Dell’s manuals define the physical part of that boundary: the R770 allows DPUs in RC 1 and RC 2 (slots 31, 36) and RC 6-2 and RC 11-2 (slots 7, 2), and the R7725 allows the B3220 and B3240 in RC 2, 3, 4, 5 and 8, slots 2 and 7.^[1][2] A plan that says “R770” without a riser configuration has not defined its scope.

The SKU is the second boundary. Write the OPN, not the marketing name: the B3220 is 900-9D3B6-00CV-AA0 or -00SV-AA0, the B3140H is 900-9D3D4-00EN-HA0 or -00NN-HA0, and the two have different memory, core counts and default modes.^[16][3] The sourcing line matters too, because Dell’s support statement follows it: DSE rows exist only for Dell-sourced SKUs, since Dell “has not validated this feature and is unable to support it” on NVIDIA Channel cards.^[14]

The version set is the third boundary, and it is the one customers forget. Dell KB 000227031’s boot issue is tied to CPLD 1.1.5/1.1.7, iDRAC 7.10.50.00 and BlueField firmware 32.40.1000; KB 000379421’s PCIe training failure is fixed from firmware 32.46.3048.^[7][8] A sign-off therefore names every version it was earned at, and the plan names the changes that void it.

The lifecycle below is the shape every plan in this module follows. Read-only cases run before mutating ones, the evidence pack is assembled before sign-off, and a regression trigger re-enters at the inventory step, not at scope.^[5][7]

flowchart TD; S[Scope and support boundary] --> M[Test matrix]; M --> I[Pre-flight inventory]; I --> R[Read-only cases]; R --> X[Mutating cases]; X --> E[Evidence pack]; E --> P([Pass criteria met?]); P -- yes --> O[Sign-off row]; P -- no --> D["Defect - match to a KB or file new"]; D --> M; T["Regression trigger (BIOS, iDRAC, CPLD, firmware, BFB, riser)"] --> I

2The matrix: modes × profiles × firmware × OS × slot and power

Five axes, each with a reason from the sources.

Modes. DPU mode (ECPF), NIC mode and zero-trust are separate rows because each changes what the host can see and do, and each change requires an Arm and NIC reset: “Power cycle is recommended.”^[3] On ESXi the split is sharper: DSE-enabled DPU mode and NIC mode require re-installing ESXi when you switch between them, so they are separate host builds, not a toggle.^[4]

DOCA-Host profiles. Test the DOCA-OFED profile and the full profile separately, because their guarantees differ: DOCA-OFED is Level 1 (compatible with older and newer BlueField firmware) while DOCA services and the full profile are Level 2 (same October-to-July cycle).^[5] A pass on one says nothing about the other.

Firmware and BFB. Anchor on the current Spectrum-X validated stack row — v2.3.1 pairs BlueField-3 firmware 32.50.1002 with DOCA-Host 3.5.0-082, BFB 3.5.0 and DTS 1.26.5 — and add an N-1 row to exercise the Level-1 forward/backward claim.^[6][5] KB 000379421 requires the BFB to stay aligned with the firmware, so firmware and BFB are one column, not two.^[8]

Server firmware. BIOS, iDRAC and CPLD are axes because the Dell issues were version-specific: CPLD 1.1.5/1.1.7 with iDRAC 7.10.50.00 for the boot-sync event, and firmware 32.46.3048 as the PCIe-training fix.^[7][8]

Host OS. Use the DOCA 3.5.0 release’s supported host list (verify it in the release notes; it is not reproduced here) and ESXi 8.0 U3b or later for any DSE row.^[4]

Slot, riser and power. Every riser configuration Dell allows, with and without the 8-pin auxiliary cable on the 150 W SKUs; the expected result without it is the “ATX power not detected! Halting system!!” halt, which the plan records as a known negative, not a defect.^[1][2][10]

A full cross-product is too large; prune by writing the customer’s target row first, then one row per axis that varies only that axis. The evidence still has to exist for every row you sign.^[6]

3Pass/fail criteria and the evidence that proves them

Every criterion has a pass condition, a fail signature and an artifact.

Enumeration. Pass: lspci shows the BlueField PCI functions; in DPU mode the rshim driver attaches (/dev/rshim0) and tmfifo_net0 is up; mlxconfig q INTERNAL_CPU_OFFLOAD_ENGINE reports the intended mode.^[11][3] Fail: repeated “Device not detected: Nvidia Network Adapter” in the Lifecycle Log, or a card visible in lspci with no /dev/rshim — the forums show the latter when the BMC owns the RShim.^[8][11] Artifacts: lspci -nn output, ls /dev/rshim*, mlxconfig q output.

Boot synchronization. Pass: twenty or more mixed cold, warm and graceful cycles with no “No Memory Found” at POST and no fatal PCIe entries in the Lifecycle Log.^[7][9] Artifact: the Lifecycle Log export with cycle timestamps.

Provisioning. Pass: bfb-install completes, the rshim console shows BL2, BL31 and UEFI, and ssh to 192.168.100.2 over tmfifo works. Fail: the install “completes” but the console is silent and ssh never answers — the forum signature of a firmware/BFB mismatch.^[12] Artifact: the console log.

Data path. Pass: OVS-DOCA flows are hardware-offloaded; a deliberately unsupported rule (mirror or multi-output) falls back to software without dropping traffic, because OVS-DOCA cannot offload multi-output flows.^[13] Artifact: flow dumps before and after the negative case.

Performance. Record DOCA Bench output as CSV and perftest/NCCL numbers per mode and profile; export DOCA Telemetry Service metrics (validated-stack version 1.26.5) to Prometheus or NetQ as the evidence store.^[6] DOCA Bench usage details are not in these notes; the plan requires the CSV, not a specific invocation.

Mode flip. Pass: mlxconfig q before shows 0, after the power cycle shows 1 (or the reverse), and the host view changes accordingly.^[3] Firmware update. Pass: the update is followed by a full system power cycle and the Lifecycle Log has no fatal PCIe entry; a warm reboot is a fail by construction.^[9]

VMware. DSE enable and disable with the ESXi reinstall between them, for Dell-sourced SKUs only.^[4][14]

4Regression triggers and sign-off

A signed row stays signed until one of its axes moves. The triggers, each with the subset it re-runs:

The reasons are the Dell KBs and the dependency policy.^{[7][8][9][5][1]}

Sign-off is one row per (server, riser, SKU, mode, profile, firmware, BFB, OS) with pass/fail, the log excerpts that prove it (Lifecycle Log, dmesg mlx5_core lines, rshim console), and the validated-stack row it maps to.^[6] The row’s owner signs, the date is the date of the last mutating step, and the trigger list is attached so the next change request can find it.

Two cautions when a service is in scope. HBN 3.5.0 ships with DOCA 3.5.0 (BSP 4.16.0) on Ubuntu 22.04 according to its release notes, while the DOCA general-support page names Ubuntu 24.04 64k as the bundle default; the plan must quote both and record which BFB variant NVIDIA confirms for HBN before an image is pushed. HBN runs on the Arm side and “does not support BlueField platforms with 8GB on-board DDR memory”; check the SKU’s memory before promising HBN, and remember it needs DPU mode.^[15][3] And any DSE row is out of scope for an NVIDIA-channel card, whatever the customer’s licence says.^[14]