InfiniBand vs Ethernet for AI clusters: choosing the right fabric

The fabric is one of the few parts of an AI cluster where a decision made early is expensive to reverse. Both InfiniBand and high-speed Ethernet can build very large, very fast GPU clusters today. The interesting question is not which one is “better” in the abstract, but which one fits the workload, the operational model and the team that has to run it.

What actually drives the choice

In practice the decision comes down to a handful of engineering realities:

• Collective performance and tail latency. Training jobs live and die by the slowest flow. The fabric’s behaviour under incast and its congestion control matter more than peak link bandwidth.
• Scale and topology. Fat-tree, rail-optimized, multi-rail and DragonFly++ each trade cost, cabling and failure domains differently.
• Operational maturity. Day-2 operations — telemetry, fabric management, firmware, upgrade paths — are where a design either survives or becomes a liability.
• Ecosystem and software. NCCL, the collective libraries, and the scheduler all have assumptions about the transport.

InfiniBand

InfiniBand remains the reference point for tightly coupled training. Credit-based flow control, a lossless fabric by design, and mature collective offload make it predictable under the bursty, all-to-all traffic that training generates.

Its strengths are well understood: deterministic latency, hardware-based congestion control, and a management model (subnet manager, fabric telemetry) built specifically for this problem. The trade-offs are equally real: a distinct operational skillset, a separate driver stack, and a narrower supplier ecosystem.

Ethernet with RoCEv2 and Spectrum-X

Ethernet with RDMA (RoCEv2) has closed much of the gap. Modern approaches add adaptive routing, congestion control tuned for AI traffic, and telemetry that understands collective patterns. The appeal is obvious: one fabric, one operational toolchain, and alignment with how the rest of the data center is built and automated.

The catch is that lossless Ethernet is not a checkbox. RoCEv2 depends on correct PFC configuration, ECN thresholds that match the workload, and disciplined QoS end to end. Get those wrong and the fabric will happily deliver inconsistent collective performance that is painful to diagnose.

The practical decision criteria

When I work through this with teams, the criteria that carry the most weight are usually:

1. Workload mix. Pure, large-scale training versus a heterogeneous mix of inference, storage and general traffic.
2. Team and toolchain. Which fabric can your engineers operate at 3 a.m. without specialist escalation?
3. Ecosystem fit. What does your reference architecture, vendor support and GPU platform assume?
4. Growth model. How does the fabric grow from hundreds to thousands of endpoints without a redesign?
5. Observability. Can you see per-flow, per-hop behaviour well enough to prove where a slowdown comes from?

Where the two converge

The interesting trend is convergence. Ethernet is borrowing the ideas that made InfiniBand predictable — adaptive routing, congestion control, telemetry — while InfiniBand is opening up its management and integration story. The question of “which fabric” is becoming less about capability and more about which operational model you want to own for the next five years.

My bias is to choose the fabric your team can operate, instrument and automate — then validate it against the actual collectives you run, not a benchmark you found online.