NVIDIA DGX Spark: What It Actually Handles

A box the size of a chunky router, 128 GB of memory, a price tag in decent used-car territory. NVIDIA sells DGX Spark as a personal supercomputer for working with large models. We looked into what it actually handles, and where the slide-deck numbers part ways with what people see on their own desks.

What is inside

At the core sits the GB10 Grace Blackwell chip: twenty Arm cores (ten Cortex-X925 plus ten Cortex-A725) and a Blackwell GPU with 6144 CUDA cores and fifth-generation Tensor Cores. CPU and GPU are linked over NVLink-C2C and share coherent memory, so data never has to be copied between system RAM and video memory.

The first surprise for owners: instead of 128 GB the system reports about 119 GiB, or 122,570 MB. Nothing was taken by the OS, this is the gap between the vendor's gigabytes and the gibibytes Linux counts in. NVIDIA moved the explanation into a separate knowledge base article because the question kept coming up.

The one limit the presentations leave out

A language model works in two phases. First it reads your whole prompt (prefill), then it emits the answer one token at a time (decode). Phase one is bound by compute, phase two by memory speed: producing a single token means reading every weight in the model.

Spark has compute to spare, and memory that moves at 273 GB/s. That gap is behind almost every complaint owners have. Here is how it looks in numbers on gpt-oss 120B:

Reading a prompt at 1725 tokens per second is genuinely quick. Generating at 55 tokens is roughly the pace of a person reading aloud. Fine for chat, thin for pushing documents through a pipeline.

The number to know before you buy

People buy 128 GB mainly for 70-billion-parameter models, which will not fit into an ordinary graphics card. The LMSYS team benchmarked Llama 3.1 70B in FP8 with one user and got 2.7 tokens per second. At that rate a three-hundred-word answer takes a minute and a half to type itself out.

The model fits in memory and it runs. The only question is whether you are willing to sit and watch it. Acceptable for overnight batch work, not for anything interactive.

Bandwidth again. A dense 70-billion-parameter model in FP8 occupies about 70 GB, and every single token requires reading all 70 GB. At 273 GB/s the theoretical ceiling comes out below four tokens, and the measured 2.7 lands exactly inside that arithmetic.

Now the part that changes the picture

Everything above assumes a single user. Once requests pile up, the weights get read once per batch, and the machine shows a completely different face. Dendro Logic's measurements on vLLM:

A hundred-and-twenty-fold gain on Nemotron Super 49B is not a typo. This is where the machine's real purpose lives: it was built to serve a workload, not to sit in a chat window with one person. If you have a department of fifteen developers who need an in-house coding assistant, Spark will cope. If you are alone and want the largest model you could find, the money went the wrong way.

Power: where the “half the performance” reports came from

NVIDIA's forum has a telling thread titled “Only getting half the advertised performance and capping at 100W”. The owner concluded the unit was defective.

NVIDIA's answer is straightforward: 240 W is the peak rating of the external power brick. The GPU section is specified to draw 120 W at most, and the rest goes to the CPU, the ConnectX-7 network chip and peripherals. So 100 W on the GPU under load is expected behavior, not a defect. The advertised petaflop needs a footnote as well: that figure is FP4 with sparsity, and Wendell at Level1Techs points out that such numbers arrive in half-second bursts.

Idle draw is a separate story. NVIDIA does not specify it at all, even though buyers treat the device as an always-on workstation. The main idle consumer turned out to be the network chip, and a firmware update closed part of that.

Two nodes: what that buys and what it does not

Two Sparks connect directly with a QSFP cable through ConnectX-7 and act as a single 256 GB resource. NVIDIA claims support for models up to 405 billion parameters on such a pair, and the documentation adds that direct cabling covers up to three systems, or four through a switch.

What you should not expect is double the speed. Measurements show prompt reading gaining between 6 and 25 percent depending on the model, while generation gains nothing or even slips. Pairing exists to fit a larger model, not to get an answer sooner.

Who it suits and who it does not

A good fit if you build for CUDA and need an environment identical to production without a cloud bill every month. If the data cannot leave company premises. If you have a team to serve rather than one person. If you fine-tune models up to 70 billion parameters with QLoRA and can afford to wait.

A poor fit if you want snappy interactive chat with the biggest model you could find. If you work in Windows: the OS is Linux only, on aarch64, and some of the software you rely on was never built for it. If you are counting on games or classic rendering.

What we have in stock

• PNY NVIDIA DGX Spark AI Server: the Founders Edition build with 4 TB of storage
• NVIDIA AI Enterprise for DGX Spark: one-year subscription, three and five-year terms also available
• NVIDIA Mission Control: cluster management once you run more than one node
• Nutanix servers: for when the prototype outgrows the desk and you need a production platform

FAQ