NVIDIA Networking

Why a cluster needs a network of its own

While there is one server, the network is just how you reach it. As soon as there are several servers computing one shared job, the network becomes part of the computer. Nodes exchange intermediate results constantly, and the latency of those exchanges adds directly to training time.

Hence a separate line of hardware. Adapters that write into a neighbouring server's memory without involving the CPU, switches with sub-microsecond latency, and cables rated for 400 and 800 gigabits.

What this section contains

• Adapters and DPUs. ConnectX-7 and ConnectX-8 for fast fabric, BlueField-3 when you want encryption, storage and policy taken off the host CPU.
• Switches. Quantum for InfiniBand, Spectrum for Ethernet with RoCE.
• Cables and transceivers. DAC for short runs inside a rack, AOC and optics with transceivers for longer spans.
• NVLink bridges. A direct channel between cards inside a single server.

InfiniBand or Ethernet

The short answer. InfiniBand gives lower latency and in-network reductions, but demands specific operational knowledge. Ethernet with RoCE is simpler, because it is the same technology as the rest of your infrastructure and the same team runs it.

A fabric is bought as a set: adapter, a cable of the right type and length, a switch port at the same speed. A mistake in any link means the connection either never comes up or drops to a lower rate.

Tell us the node count and the workload, and we will put together the whole list, cables for your distances included.