Out-of-band: BMC, Redfish, and the BlueField-4 path
S2·E4The channel already has an owner · Hotel lobby, Thursday 06:50, ninety minutes before the design workshop
Builds on: rshim and the BFB path
Before you read: what do you already know?
3 quick questions. Wrong answers are fine and expected; trying first makes the lesson stick.
After this lesson you can
- Describe the BlueField-3 BMC's interfaces (IPMI, Redfish, serial) and the exact commands that reach each.
- Read and change the operating mode and host privilege through Redfish and state what reset each change needs.
- Explain rshim ownership between the BMC and the host, the handoff procedure, and what force mode does.
- Contrast the BlueField-4 provisioning path (ISO, PLDM, Redfish) with the BlueField-3 rshim/BFB path and say why it differs.
Episode 4 — The channel already has an owner
Three nodes with no /dev/rshim0, and at half past eight an NVIDIA PM is booked into the customer’s workshop with a roadmap slide. The architect sends a screenshot from the floor: lspci shows the BlueField-3, systemctl status rshim is green, and the device file is not there. He wants to know whether all three cards are faulty before he walks into that room and asks procurement for replacements on demo eve.
They are almost certainly not faulty. Only one rshim back end can be attached to a card at a time; when a second one tries, the daemon reports another backend already attached, and the troubleshooting guide calls that correct behavior.[4] On a card with an integrated BMC, and every BlueField-3 model has one, DPU and SuperNIC alike, the BMC holds the channel by default, so the host is simply second in line.[11][4]
That BMC exists because the card is a computer in its own right. It can be reached when the host OS is down or deliberately untrusted, it speaks IPMI and a standard DMTF Redfish interface over HTTPS, and through Redfish it reads and sets the card’s operating mode and the host’s privilege without the host cooperating at all.[2][1]
Nothing is broken; the channel already has an owner, and your job is to choose which one.
You type back four commands and one warning about which side to disable rather than merely stop. Segment 1 is where those interfaces live.
1The BlueField BMC and how you reach it
Every BlueField-3 model, DPU and SuperNIC alike, carries an integrated BMC.[11] Its documentation (release 26.04) covers Connecting to BMC Interfaces, Platform Management Interface, First-time Installation, BlueField Management, BMC Management, NIC Subsystem Management, a Table of Common Redfish Commands, and Unsupported BMC Functionalities in NIC Mode.[3] That last title is worth noting: some BMC functions depend on the Arm being active.
Three interfaces. The BMC “based on the Intelligent Platform Management Interface (IPMI) standard, supports both out-of-band (OOB) dedicated interfaces, and a serial port to access the CLI of the BMC”.[2] Over the LAN the documented form is ipmitool -C 17 -I lanplus -H <bmc_ip> -U ADMIN -P ADMIN <ipmitool_arguments>; over the UART it is plain ipmitool <ipmitool_arguments>.[2] And “The BlueField’s BMC provides a standard DMTF Redfish management interface, which is accessible via an HTTPS RESTful interface”, entered with:[2]
curl -k -u root:'<password>' -H 'Content-Type: application/json' -X GET https://<bmc_ip>/redfish/v1The ADMIN/ADMIN and root credentials in those examples are documentation defaults; the bf.cfg keys BMC_PASSWORD and NEW_BMC_PASSWORD exist so a BFB push can rotate them.[6] A Dell PowerEdge adds a second controller to this picture: iDRAC manages the server, the card’s BMC manages the card. Which iDRAC features touch the BlueField BMC is covered in lesson 2.6, with its evidence limits.
1. Reach the BMC over the 1GbE OOB port
ssh root@<bmc_ip> # first login forces a change of the default password 0penBmc (12–20 chars) curl -k -u root:'<password>' https://<bmc_ip>/redfish/v1/Systems/Bluefield/Oem/Nvidia
{ … "Mode": "DpuMode" … }If it does not match · The OOB eth0 takes DHCP by default (RJ45 "MGMT" on the bracket). Integrated-BMC cards default to BMC ownership of rshim; the guide requires "Ensure that the RShim interface is disabled on the host side": systemctl stop rshim; systemctl disable rshim on the host before a BMC push.
sourceSources: BF-Bundle Installation and Upgrade, BlueField Platform Software Troubleshooting (RShim), BSP 4.15.0 bf.cfg guide, rshim-user-space (rshim.8, rshim.conf, bfb-install), BlueField BMC 26.04 (Deploying Software Using BFB, Modes), NVIDIA/skills doca-bf4-deployment.
2Mode and privilege over Redfish
The BMC exposes the same mode switch you know from mlxconfig, without host involvement. Read the mode with GET /redfish/v1/Systems/Bluefield/Oem/Nvidia; the response carries "Mode": "DpuMode" or "NicMode".[1] Set it with a POST to /redfish/v1/Systems/Bluefield/Oem/Nvidia/Actions/Mode.Set and a body of {"Mode": "NicMode"} or {"Mode": "DpuMode"}; “Two consecutive Arm reboots are required to apply configuration”.[1]
Zero Trust is a different resource: PATCH /redfish/v1/Chassis/Card1/NetworkAdapters/NvidiaNetworkAdapter/Oem/Nvidia/HostPrivilegeConfig/Settings with {"PrivilegeMode": "Restricted"} to lock the host out or "Privileged" to let it back in; “Power cycle is required to apply configuration”.[1] This is the BMC equivalent of mlxprivhost r and mlxprivhost p on the Arm, and it is the path you use when the Arm OS is not reachable.[12]
BMC=https://<bmc_ip>; AUTH="-k -u root:<password> -H Content-Type:application/json"
# read mode (read-only)
curl $AUTH -X GET $BMC/redfish/v1/Systems/Bluefield/Oem/Nvidia
# set mode (mutating: two consecutive Arm reboots to apply)
curl $AUTH -X POST $BMC/redfish/v1/Systems/Bluefield/Oem/Nvidia/Actions/Mode.Set -d '{"Mode": "NicMode"}'
# host privilege (mutating: power cycle to apply)
curl $AUTH -X PATCH $BMC/redfish/v1/Chassis/Card1/NetworkAdapters/NvidiaNetworkAdapter/Oem/Nvidia/HostPrivilegeConfig/Settings -d '{"PrivilegeMode": "Restricted"}'The reset rules line up with the modes page: configuration takes effect only after the Arm and NIC components reset, and the order rule (Zero Trust to DPU first, then DPU to NIC) applies whichever interface you use.[12] GET is read-only; POST and PATCH are mutating and belong inside a window with a rollback, exactly like mlxconfig s.
3Who owns rshim: the BMC-host handoff
Only one rshim back end can be attached to a card at a time. When a second one tries, the daemon reports another backend already attached; the troubleshooting guide calls this “correct behavior as there can only be one RShim back end active at any given time”.[4] On cards with an integrated BMC the BMC owns rshim by default, so a host that runs rshim and sees no /dev/rshim0 is usually not broken; it is second in line.[4]
The documented handoff is symmetrical:[4]
# BMC -> host
bmc# systemctl stop rshim
bmc# systemctl disable rshim
host# systemctl enable rshim
host# systemctl start rshim
# host -> BMC: the same four commands with the roles swappedStop, then disable, on the side that gives up ownership; enable, then start, on the side that takes it. Disabling matters: a stopped-but-enabled service comes back at the next boot and takes the channel again.
The daemon also has a force mode. rshim -h lists -F, --force run in force mode, and the same option can be set as FORCE_MODE in the daemon’s configuration file; in force mode “we will send a one-time ownership request command for each rshim backend if they are found to be detached (aka. in drop mode)”.[5] Use it when the other side is unreachable and you have decided the host must own the channel. How the flag reaches the daemon depends on your distribution’s unit file; check with systemctl cat rshim and put -F where that unit passes options. The pages fetched for this lesson do not document a specific OPTIONS= variable, so read the unit rather than assuming one.
The same BMC can be driven from a BFB push. bf.cfg carries BMC_USER (default root), BMC_PASSWORD / NEW_BMC_PASSWORD, UPDATE_BMC_FW (yes), BMC_REBOOT (no), UPDATE_CEC_FW (yes), BMC_IP_TIMEOUT (600 s) and BMC_TASK_TIMEOUT (1800 s), so one install can update the Arm OS, the NIC firmware and the BMC firmware together.[6]
4BlueField-4: ISO and PLDM, not a BFB over rshim
BlueField-4 is not in the DOCA 3.5.0 supported-hardware tables; the release notes list BlueField-3, BlueField-2 and ConnectX adapters only.[10] What exists today is NVIDIA’s own deployment skill, and it is explicit that “BF4 is not BF3”: the methods, the Grace naming and the dpu-bmc/Redfish surface are BF-4-specific, and the BF-3 rshim/BFB path is a different skill.[9][7]
Day-1 bring-up of a BF-4 is “driven from its BMC”. Grace is “the Arm CPU complex on BlueField-4”, and the standards in play are Redfish (DMTF), PLDM for Firmware Update (DMTF) and UEFI (HTTP Boot, PXE, Boot Manager).[7] Three OS install methods:[7]
- A. UEFI HTTP Boot (recommended). An operator HTTP server hosts the bundle ISO; through BMC SSH and
obmc-console-clientyou set, in UEFI, Device Manager, Network Device List, the OOB MAC, HTTP Boot Configuration, Boot URI, then Boot Manager, UEFI HTTP. - B. PXE Boot. DHCP plus TFTP; the method to reach for “when you … need a custom
bf.cfg”. - C. Redfish Virtual Media. Upload the ISO to the BMC eMMC via Redfish SimpleUpdate (a “5 GB local-payload limit”), attach it via Redfish VirtualMedia, set
BootSourceOverrideto USB (Once, UEFI), reset Grace; it “requires a recent dpu-bmc version”, and you must “detach all virtual media afterward to avoid boot loops”.
Firmware (BMC, NIC firmware, SBIOS, ERoT) moves by PLDM: a multipart POST of the .fwpkg to the Redfish UpdateService update-multipart endpoint, then monitor the Redfish Task (PercentComplete, Messages), verify with pldmtool fw_update GetFwParams -m <eid>, activate with a power cycle; Redfish FirmwareInventory is the cross-check, and BMC factory reset is recovery-only.[7][8] Cloud-init works through a seed ISO labelled CIDATA, with BMC-facing URIs fixed to image.iso and config.iso.[8] The post-install anchor is still cat /etc/mlnx-release on Grace.[7]
Why the change: BF-3 provisioning depends on a host-visible rshim function and a bootstream format; BF-4 standardises on what every server BMC already speaks (Redfish, PLDM, UEFI boot), so the same tooling that provisions the server can provision the card, from the BMC, with no host cooperation. Treat everything in this segment as NVIDIA guidance from the skills repository, not as DOCA product documentation, until BF-4 appears in a DOCA release.[9]
End of Episode 4 — Ownership, decided on purpose
The architect stops the BMC’s rshim service and disables it, then enables and starts the host’s; /dev/rshim0 appears on all three before the workshop.[4] The operator labels the bezels “RSHIM — ONE OWNER”. In the room the question changes shape: not “are the cards broken” but “who owns this channel in your design”, with the Redfish mode read as the answer either side can check.[1] The PM, asked about the next generation, says “not announced”; you add the checkable half: BlueField-4 is not in the DOCA 3.5.0 tables.[10]
What you say: “Disable it, do not just stop it. A stopped service still enabled takes the channel back at the next boot.”[4]
Then a mode read returns DpuMode on the four nodes whose only job tomorrow is a throughput number.
Lab
Read-only: GET, query and status commands only. Do not run the POST, PATCH, handoff or force-mode steps here; they belong in a window.
- From a management host,
curl -k -u root:'<password>' -H 'Content-Type: application/json' -X GET https://<bmc_ip>/redfish/v1. Expected: a JSON service root. If not: confirm the OOB port has a link and an address; the Dell-lab BMC address is in the lab sheet.[2] curl ... -X GET https://<bmc_ip>/redfish/v1/Systems/Bluefield/Oem/Nvidia. Expected: aModefield readingDpuModeorNicMode. Record it and compare withmlxconfig q INTERNAL_CPU_OFFLOAD_ENGINEon the host from lesson 2.2; they must agree.[1]ipmitool -C 17 -I lanplus -H <bmc_ip> -U <user> -P <password> mc info. Expected: the BMC’s firmware revision and manufacturer. If it fails while Redfish works, note it; IPMI over LAN may be disabled by policy on the lab BMC.[2]- On the host:
systemctl status rshim; ls /dev/rshim0/ 2>/dev/null; journalctl -u rshim -n 30 --no-pager | grep -i 'already attached'. Expected on a host-owned card: the four device files and no match; on a BMC-owned card: no/dev/rshim0and theanother backend already attachedline.[4][5] Record which case you have; do not change it. systemctl cat rshim | grep -iE 'ExecStart|Environment'. Record how options reach the daemon on this distribution; that is where-Fwould go if a window ever required force mode.[5]- If a BFB was pushed in lesson 2.3, check on the Arm whether the BMC firmware step ran:
ssh ubuntu@192.168.100.2 sudo bf-infoand look for the BMC component. Compare with theUPDATE_BMC_FWvalue in thebf.cfgyou used.[6]
- Write the three Redfish calls from Segment 2 into a script with
BMCand the password as variables, and mark each one# reador# mutating. Expected: one GET marked read, one POST and one PATCH marked mutating, each with the reset it needs in a comment.[1] If your PATCH comment says “two Arm reboots”, fix it: HostPrivilegeConfig needs a power cycle. - Write the two
ipmitoolforms from the BMC page (UART and LAN) with placeholders. Expected: the LAN form includes-C 17 -I lanplus -H <bmc_ip> -U <user> -P <password>.[2] - Write the four-command handoff in both directions (BMC to host, host to BMC) and, under each, the one-line reason for
disable. Expected: stop, disable on the giving side; enable, start on the taking side.[4] - Read the
rshim -husage lines quoted in Segment 3 and write what-Fdoes in one sentence in your own words, then compare with the quote.[5] - Build a two-column table “BF-3 vs BF-4” with rows for OS image format, transport, firmware update mechanism, who drives day-1, and the post-install anchor. Expected: BFB over rshim vs ISO over HTTP boot/PXE/virtual media; bfb-install or bf-fwbundle vs PLDM through Redfish UpdateService; host or BMC vs BMC;
/etc/mlnx-releasein both rows.[7]
Retrieval check
10 questions from memory. Answer before looking anything up; misses become flashcards.
Explain it to a Dell SE
Explain to a Dell SE, in four sentences, what the BlueField's own BMC is for when the server already has an iDRAC, and why BlueField-4 will not be installed the way BlueField-3 is.
Sources
Facts in this lesson were checked against BlueField BMC 26.04 docs (Modes of Operation Configuration, Platform Management Interface), BlueField Platform Software Troubleshooting Guide, rshim daemon source, NVIDIA/skills doca-bf4-deployment, 2026-09-06. Dates are when each page was fetched.
- BlueField BMC 26.04: Modes of Operation Configuration (Redfish) · fetched 2026-09-06
- BlueField BMC 26.04: Platform Management Interface (IPMI, Redfish) · fetched 2026-09-06
- BlueField BMC 26.04 documentation index · fetched 2026-09-06
- BlueField Platform Software Troubleshooting Guide: SoC Management Interface (RShim ownership) · fetched 2026-09-06
- rshim daemon source (rshim.c: -F force mode, FORCE_MODE, another backend already attached) · fetched 2026-09-06
- BlueField BSP 4.15.0: Customizing BlueField Software Deployment (BMC keys in bf.cfg) · fetched 2026-09-06
- NVIDIA/skills: doca-bf4-deployment CAPABILITIES (BMC-driven bring-up, ISO methods, PLDM) · fetched 2026-09-06
- NVIDIA/skills: doca-bf4-deployment TASKS (PLDM update, cloud-init) · fetched 2026-09-06
- NVIDIA/skills: doca-bf4-deployment SKILL (BF4 is not BF3) · fetched 2026-09-06
- DOCA Release Notes v3.5.0 (supported adapters, no BlueField-4) · fetched 2026-09-06 · DOCA 3.5.0
- NVIDIA BlueField-3 Networking Platform User Guide (integrated BMC) · fetched 2026-09-06
- BlueField Modes of Operation (reset rules) · fetched 2026-09-06 · DOCA 3.5.0
The same idea elsewhere
Other lessons that cover this ground, sometimes from another course's angle.