Skip to content
nodeza.learn
RoCE course/1. Foundations/Lesson 1.5

QP state machine, retries, and connection setup

S1·E5Forty minutes, every time · Customer lab, Monday, first week after handover

S1·E5Analyze~30 minsources checked todaylab mutates hardwareverified against rdma-core and librdmacm man pages (master) plus NVIDIA DOCA-Host RoCE and DOCA RDMA documentation, 2026-09-07

Builds on: Transports and opcodes: what RC, UC and UD can actually do, GIDs: how a connection picks its RoCE version

Before you read: what do you already know?

3 quick questions. Wrong answers are fine and expected; trying first makes the lesson stick.

After this lesson you can

  • Reproduce the RESET to INIT to RTR to RTS sequence and name the attributes each transport requires at each transition.
  • Explain how `timeout`, `retry_cnt`, `rnr_retry` and `min_rnr_timer` produce completion status 12 and status 13.
  • Trace the RDMA-CM active and passive sequences and identify which event reports each failure mode.
  • Contrast connection setup through RDMA-CM with an out-of-band exchange and justify which one a given application should use.

Episode 5 — Forty minutes, every time

The situation · Customer lab, Monday, first week after handover

The cluster was handed over Friday; the training job dies after roughly forty minutes, every attempt. The platform engineer has a screenshot that says only “RDMA errors”, the switch team is on the call, tired of being blamed, and nobody wants to burn forty minutes per experiment finding out who owns it. The network lead says what he has said once a week since Round Rock: show me the counter.

Two completion statuses decide the meeting. If the requester transmits and no acknowledgement arrives inside timeout, it retransmits up to retry_cnt times, and exhausting that budget completes the work request as IBV_WC_RETRY_EXC_ERR — status 12, rendered as “transport retry counter exceeded”.[1][8] If instead the responder answers with a receiver-not-ready NAK because it has no receive buffer posted, the requester waits min_rnr_timer and retries up to rnr_retry times, and exhaustion gives IBV_WC_RNR_RETRY_EXC_ERR, status 13.[8] The switch team stays or leaves on that single answer.

Those knobs exist because a reliable connection has to decide how long to keep believing in a packet that has not been acknowledged. They are installed as the queue pair walks INIT to RTR to RTS, and they are RC-only, because a transport that never acknowledges has nothing to retry.[1] You do not have to wait forty minutes either: perftest exposes the local ACK timeout as -u, four microseconds times two to the power of the value.[5]

Status 12 means nobody answered; status 13 means somebody answered with nowhere to put it.

Walk the states first: every knob in this argument is installed on a transition.

1Four states and the masks that separate the transports

A queue pair walks IBV_QPS_RESET to INIT to RTR to RTS, with SQD, SQE and ERR off the happy path.[1] Each transition is one ibv_modify_qp() call, and what makes the transports differ is not the states but the attribute masks each transition requires.

RESET to INIT is local-only. UD requires IBV_QP_STATE, IBV_QP_PKEY_INDEX, IBV_QP_PORT, IBV_QP_QKEY; RC and UC require the same three plus IBV_QP_ACCESS_FLAGS instead of the q_key.[1] That single difference is the whole security model appearing early: a datagram QP is authorised by a q_key, a connected QP declares up front what the peer will be allowed to do one-sided.

INIT to RTR is the connection step. UD requires only IBV_QP_STATE.[1] UC requires IBV_QP_STATE, IBV_QP_AV, IBV_QP_PATH_MTU, IBV_QP_DEST_QPN, IBV_QP_RQ_PSN.[1] RC requires all of those plus IBV_QP_MAX_DEST_RD_ATOMIC and IBV_QP_MIN_RNR_TIMER — the two attributes only a reliable responder needs.[1] Note what IBV_QP_AV implies after the previous lesson: this is where the GID index, and therefore the RoCE version, is fixed.[6]

RTR to RTS arms the requester. UC and UD need IBV_QP_STATE and IBV_QP_SQ_PSN only; RC adds IBV_QP_MAX_QP_RD_ATOMIC, IBV_QP_RETRY_CNT, IBV_QP_RNR_RETRY, IBV_QP_TIMEOUT.[1] And path_mtu, whose enum is IBV_MTU_256 through IBV_MTU_4096, is “valid for RC/UC QPs only” — which is why UD traffic behaves differently from RC under an MTU mismatch.[1]

RC — Reliable Connected: everything: SEND, WRITE, READ, atomics
RESETINITRTRRTS4 attrs7 attrs6 attrsplus SQD · SQE · ERR — a QP in ERR flushes every posted WR as IBV_WC_WR_FLUSH_ERR (5)
state 2 / 4
RC · transition 1 of 3

RESET → INIT

ibv_modify_qp(qp, &attr, IBV_QP_STATE | …)
IBV_QP_STATEIBV_QP_PKEY_INDEXIBV_QP_PORTIBV_QP_ACCESS_FLAGS

Local-only step: which physical port and which P_Key index this QP lives on. The single difference between the transports here is the last attribute — UD needs IBV_QP_QKEY (datagrams are authorised by a q_key), the connected transports need IBV_QP_ACCESS_FLAGS (what the peer will be allowed to do one-sided).

FAE angle: IBV_QP_ACCESS_FLAGS set here is what makes a later RDMA_WRITE from the peer legal. A REM_ACCESS_ERR (10) on the peer often traces back to this line, not to the MR.

Sources: ibv_devinfo(1) · ibv_create_qp(3) · ibv_create_cq_ex(3) · ibv_create_srq(3) · ibv_reg_mr(3) · ibv_poll_cq(3) · ibv_query_device_ex(3) · NVIDIA Optimized Memory Access · perftest

Step the machine with Prev and Next and commit a prediction for the required mask before revealing it. Switch transport between RC, UC and UD on the same transition to see the masks shrink.

2The four knobs that manufacture errors 12 and 13

The RC-only attributes are four 8-bit values: min_rnr_timer (“Minimum RNR NAK timer (valid only for RC QPs)”), timeout (“Local ack timeout for primary path”), retry_cnt (“Retry count”) and rnr_retry (“RNR retry”).[1] They are not tuning trivia; they are the mechanism that turns a network condition into a specific completion status.

If the requester sends and no acknowledgement arrives inside timeout, it retransmits, up to retry_cnt times. When that budget is exhausted the work request completes as IBV_WC_RETRY_EXC_ERR, which is position 12 in the status enum.[8] If instead the responder answers with a receiver-not-ready NAK because it has no receive buffer posted, the requester waits min_rnr_timer and retries up to rnr_retry times, and exhaustion produces IBV_WC_RNR_RETRY_EXC_ERR, position 13.[8] ibv_wc_status_str() renders these as “transport retry counter exceeded” and “RNR retry counter exceeded”.[8]

The host counters corroborate the split. The in-tree mlx5_ib driver defines local_ack_timeout_err, req_transport_retries_exceeded, rnr_nak_retry_err, req_rnr_retries_exceeded and out_of_buffer, all surfacing under /sys/class/infiniband/<dev>/ports/<p>/hw_counters/.[9] The names carry the mapping and the lab below confirms it on real hardware: the req_ counters and local_ack_timeout_err move on the requester behind a status 12, the RNR pair behind a status 13, and out_of_buffer on the responder is why the RNRs were sent. NVIDIA publishes no fetchable page defining these counters, so verify the deltas on the card rather than quoting a definition.

perftest lets you make the failure happen on demand: -u, --qp-timeout=<timeout> is documented as “QP timeout, timeout value is 4 usec * 2^(timeout), default 14”.[5] Lowering it shortens the wait before a retry-exceeded appears, which is how you provoke a clean demonstration instead of waiting.

3Two ways to exchange the connection

Something must carry the queue pair number, the starting PSN and the address between the two sides. There are exactly two families of answer.

The RDMA connection manager is the first. “The RDMA CM is a communication manager used to setup reliable, connected and unreliable datagram data transfers”, giving “an RDMA transport neutral interface for establishing connections”, and it “can control both the QP and communication management portions of an RDMA API, or only the communication management piece”.[2] The active sequence is rdma_getaddrinfo, rdma_create_event_channel, rdma_create_id, rdma_resolve_addr (wait for RDMA_CM_EVENT_ADDR_RESOLVED), rdma_create_qp, rdma_resolve_route (wait for ROUTE_RESOLVED), rdma_connect (wait for ESTABLISHED), then data, rdma_disconnect, rdma_destroy_qp, rdma_destroy_id, rdma_destroy_event_channel.[2] The passive side is rdma_create_event_channel, rdma_create_id, rdma_bind_addr, rdma_listen, wait for RDMA_CM_EVENT_CONNECT_REQUEST — which arrives with a new id — then rdma_create_qp and rdma_accept.[2]

Rendering diagram…
Diagram source (Mermaid)
flowchart LR; A[rdma_resolve_addr] --> B[ADDR_RESOLVED] --> C[rdma_create_qp] --> D[rdma_resolve_route] --> E[ROUTE_RESOLVED] --> F[rdma_connect] --> G[ESTABLISHED] --> H[data transfer] --> I[rdma_disconnect]
The active-side RDMA-CM sequence. Every event box must also be acknowledged with rdma_ack_cm_event before the id can be destroyed.

Events are how failures announce themselves. RDMA_CM_EVENT_UNREACHABLE is “generated on the active side to notify the user that the remote server is not reachable”; REJECTED means “a connection request or response was rejected by the remote end point”; ADDR_CHANGE fires when “the network device associated with this ID through address resolution changed its HW address”; TIMEWAIT_EXIT means “the QP associated with a connection has exited its timewait state”.[3] The discipline that beginners skip: “All events that are reported must be acknowledged by calling rdma_ack_cm_event”, and “Destruction of an rdma_cm_id will block until related events have been acknowledged” — the usual cause of a clean-shutdown hang.[3]

rdma_conn_param carries the negotiated depths and a small payload: private_data_len is capped at 56 bytes for RDMA_PS_TCP and 180 bytes for RDMA_PS_UDP; responder_resources is “the maximum number of outstanding RDMA read and atomic operations that the local side will accept from the remote side” and initiator_depth the same for operations it will issue; retry_count and rnr_retry_count are 3-bit values and RDMA_PS_TCP only.[4] Behaviour is port-space dependent: “for RDMA_PS_TCP, it initiates connection; for RDMA_PS_UDP, it initiates remote QP lookup”.[4]

The second family is out-of-band exchange: the application moves the same values over any channel it already trusts. perftest does this by default and only uses the CM when given -R, --rdma_cm.[5] DOCA offers both: doca_rdma_export() produces a connection blob that the application transports and the peer consumes with doca_rdma_connect(), with the warning that the exported data “contains sensitive information” and must be transmitted securely; or the built-in CM path, doca_rdma_start_listen_to_port() on the server and doca_rdma_addr_create() plus doca_rdma_connect_to_addr() on the client, addressed by IPv4, IPv6 or GID.[7] For an application that already runs its own rdma_cm event loop, the bridge functions doca_rdma_bridge_prepare_connection(), doca_rdma_bridge_accept() and doca_rdma_bridge_established() splice the two together.[7]

On RoCE specifically there is a reason to prefer the CM that has nothing to do with convenience: with no Subnet Manager “querying a path is impossible. Therefore, the path record structure must be filled with relevant values before establishing a connection. Hence, it is recommended working with RDMA-CM to establish a connection as it takes care of filling the path record structure.”[6]

Worked → faded → problem: make error 12 happen, then read it

Goal: provoke a transport-retry-exceeded completion deliberately, and confirm the diagnosis from the counters rather than from the log line.

  1. Pre-flight: record the baseline counters on the client. for c in local_ack_timeout_err req_transport_retries_exceeded rnr_nak_retry_err out_of_buffer packet_seq_err; do echo "$c $(cat /sys/class/infiniband/mlx5_0/ports/1/hw_counters/$c)"; done.[9]
  2. Establish a normal run first so you know the path works: ib_write_bw -d mlx5_0 -x <v2 index> -s 4096 -D 20 on both sides. Expected: a result table and no counter movement beyond noise.[5]
  3. Shorten the retry budget so the failure is quick, and start a long run: add -u 8 to both sides. The local ACK timeout becomes 4 usec times 2 to the power 8 instead of 2 to the power 14.[5]
  4. Interrupt the path mid-run: unplug the DAC on the server side, or administratively down the switch port. Expected: the client run terminates with a completion failure whose text is “transport retry counter exceeded”.[8]
  5. Re-read the counters. Expected: local_ack_timeout_err and req_transport_retries_exceeded have moved; out_of_buffer and rnr_nak_retry_err have not.[9] That asymmetry is the evidence that this is a fabric event and not an application one.
  6. Restore the link and re-run step 2 to confirm the path is healthy again.
  7. Now do the contrast case if you can arrange it: a responder that stops posting receives produces status 13 with out_of_buffer and rnr_nak_retry_err moving instead, and local_ack_timeout_err flat.[9][8]

Case closed

How it ended

It is status 13, and the counters agree: out_of_buffer and rnr_nak_retry_err move while local_ack_timeout_err and req_transport_retries_exceeded stay flat.[9] The switch team drops off, the application team goes looking at the loop that posts receives, and you leave a -u 8 reproduction so the next experiment costs minutes, not forty of them.[5] The archive is live before the freeze. The SE closes the last row of the spreadsheet and finally drinks the coffee, cold; the operator prints one more label, STATUS 13, ASK THE RECEIVER; the network lead’s notebook now carries a counter list in his own handwriting, the real deliverable. What you say: “Twelve means nobody answered; that one is ours. Thirteen means they answered and had nowhere to put it — that one lives in your receiver.”

Lab

Mutating steps ahead. Needs a maintenance window, out-of-band access (BMC/iDRAC/rshim console) and a rollback path. Record the pre-flight inventory before changing anything. Never on a production host.

Dell-lab ConnectX or BlueField-3 pair. Steps 1 to 4 are read-only. Step 5 deliberately flaps a link and names its rollback.

  1. Pre-flight inventory: ibv_devinfo -l, ibdev2netdev, show_gids, cma_roce_mode -d mlx5_0 -p 1, and a baseline read of local_ack_timeout_err, req_transport_retries_exceeded, rnr_nak_retry_err, out_of_buffer and packet_seq_err under /sys/class/infiniband/mlx5_0/ports/1/hw_counters/. Save all of it.
  2. Healthy baseline: ib_write_bw -d mlx5_0 -x <v2 index> -s 4096 -D 20 on both sides. Expected: a result table; counters essentially unchanged.
  3. Contrast the connection families as in the no-hardware lab: -x <index> versus -R, and record which GID each run used by capturing with tcpdump -i <netdev> -nn -c 10 "udp port 4791 or ether proto 0x8915".
  4. Shorten the retry budget and observe the difference in time-to-failure without breaking anything yet: ib_write_bw -d mlx5_0 -u 8 -s 4096 -D 20 versus the default -u 14. Expected: identical results on a healthy path; the setting only changes how long the requester waits before retransmitting.
  5. Mutating step — a deliberate link flap on a lab host only. With a -u 8 run in progress, down the port on the far side: ip link set <netdev> down on the server. Expected: the client completes with “transport retry counter exceeded” and local_ack_timeout_err plus req_transport_retries_exceeded move while out_of_buffer stays flat. Rollback, mandatory: ip link set <netdev> up, wait for ibv_devinfo -d mlx5_0 | grep state to report PORT_ACTIVE (4), re-run step 2 and confirm the baseline result returns. If the port does not come back, check the switch side before reloading any driver.
  6. Record the counter deltas next to the completion status. That pairing - status code plus corroborating counter, on the correct host - is the artifact worth keeping for module 4.

Retrieval check

10 questions from memory. Answer before looking anything up; misses become flashcards.

Explain it to a Dell SE

Explain to a Dell SE, in four sentences, what has to be exchanged between two machines before an RDMA connection can move data, and why an RDMA job can fail with a retry error even though both machines are healthy.

14 flashcards for this lesson — 0 in deck. Spaced review lives at /review.

Sources

Facts in this lesson were checked against rdma-core and librdmacm man pages (master) plus NVIDIA DOCA-Host RoCE and DOCA RDMA documentation, 2026-09-07. Dates are when each page was fetched.

  1. ibv_modify_qp(3) - rdma-core man page · fetched 2026-09-07
  2. rdma_cm(7) - librdmacm man page · fetched 2026-09-07
  3. rdma_get_cm_event(3) - librdmacm man page · fetched 2026-09-07
  4. rdma_connect(3) - librdmacm man page · fetched 2026-09-07
  5. perftest(1) man page (linux-rdma/perftest master) · fetched 2026-09-07
  6. RDMA over Converged Ethernet (DOCA-Host) - generated PDF · fetched 2026-09-07
  7. DOCA RDMA - DOCA-Host · fetched 2026-09-07
  8. rdma-core libibverbs/verbs.h (master) · fetched 2026-09-07
  9. Linux kernel - drivers/infiniband/hw/mlx5/counters.c · fetched 2026-09-07
  10. Environment Variables - NCCL 2.31.2 documentation · fetched 2026-09-07

The same idea elsewhere

Other lessons that cover this ground, sometimes from another course's angle.