DGX Spark
A DGX Spark runs 40 of the 40 open models tracked here at Q4_K_M with 8k context, computed on 3 November 2026. The largest is Mistral Small 4 119B (MoE), needing 72.5 GB and generating about 51 tokens per second.
- Memory
- 128 GB unified
- For the GPU
- 126 GB
- Bandwidth
- 273 GB/s
- Board power
- 140 W
- Released
- 2025
- Availability
- Sold new
Open the DGX Spark in the LLM checker
What a DGX Spark runs, model by model
DGX Spark (128 GB) addresses 126 GB of its 128 GB of unified memory, worked out from what NVIDIA documents (see the sources below), at 273 GB/s. Every model below is at Q4_K_M with 8k of context, the setting most people actually use. Once the 5% safety margin comes off, the card has 119.7 GB to spend: memory decides what loads, bandwidth how fast it answers. Both are estimates from stated formulas, and the VRAM calculator shows the working.
On this card a model fits at or under 119.7 GB, and above 107.1 GB it is tight: it loads, but there is little left for the conversation, and a desktop on the same 128 GB can push it over. Tokens per second are single-stream estimates from memory bandwidth; ≤ marks a ceiling real runtimes fall well below.
Computed 3 November 2026The model to actually run on it
Nemotron 3.5 Bolt 74B-A9B (MoE) is the best use of this card: 45.9 GB of the 126 GB available, an estimated 37 tokens per second, comfortable for chat, and 131,072 tokens of context still available. It is a mixture-of-experts model, so all 74.9B parameters sit in memory but only 9B are read per token, which is why it is quick for its size.
How this is chosen, since no benchmark is quoted: models are compared by size, a mixture-of-experts model counting at the geometric mean of its total and active parameters (a rule of thumb, not a measurement). The pick is the biggest class that fits with memory to spare and answers at 25 tokens per second or better, and within that class the most recent general-purpose release; coding and reasoning specialists are listed above but not recommended by default.
Mistral Small 4 119B (MoE) is the largest model the card will hold, at 72.5 GB, with room for 131,072 tokens of context. It is a mixture-of-experts model reading only 6.5B parameters per token, so it is the faster of the two at an estimated 51 tokens per second; what it gives up is the depth of a dense model that reads all of its weights for every token.
If quality matters more than parameter count, Llama 3.3 70B fits at Q8_0 in about 81 GB. Q8 is near-lossless; a smaller model at Q8 often beats a larger one squeezed into Q3.
How much context actually fits
Model size is the question people ask; context is the one that bites. The KV cache grows linearly with the conversation, so a model that loads comfortably can still run out of memory halfway through a long document. On 126 GB, the context you get is whatever the weights leave behind, which is why two cards that run the same model can feel completely different in use. These are the largest contexts this card holds, at Q4_K_M weights.
| Model | Max context, FP16 KV | With Q8 KV cache |
|---|---|---|
| Nemotron 3.5 Bolt 74B-A9B (MoE) | 131,072 tokens | 131,072 tokens |
| Llama 3.3 70B | 131,072 tokens | 131,072 tokens |
| Nanbeige4.3 33B | 131,072 tokens | 131,072 tokens |
| Qwen3 32B | 40,960 tokens · model limit | 40,960 tokens · model limit |
Every model in this table already reaches its limit at FP16, so quantising the KV cache buys nothing here. Capped at 128k tokens, or at the model's own native window where that is smaller, marked "model limit": free memory beyond that point buys nothing. Rows that hold far more context than their size suggests are hybrid, sliding-window or latent-attention models, which cache only a fraction of what a standard transformer does; each model page shows the working.
Where this card stops
Nothing in this list is out of reach. Every model tracked here fits at Q4_K_M with 8k of context, so the ceiling you will meet is context length and batch size, not parameter count.
Buy or rent
Buy it for CUDA and capacity in a small box, not for speed: a used 3090 generates faster on anything that fits in 24 GB. If the large models are an occasional experiment, renting a large card for those hours costs less than a machine that runs them slowly every day.
Where to buy: NVIDIA · Dell · Lenovo · Acer.
To weigh it against every other machine for your own job (chat, agents, images or video), use the AI computer picker.
Retail prices drift, and a figure written today would be wrong within a month, so this page does not carry one. Put the figure you are actually looking at, with your electricity rate and your real monthly hours, into the build vs rent calculator, which already knows this card draws 140 W under load. It returns the month owning becomes cheaper, or tells you it never does. Hours per month decides it far more often than the price does.
The honest take
NVIDIA's small Linux box with 128 GB the GPU can share: large models fit, but they are read at about a quarter of an RTX 4090's speed.
It holds models no consumer card can, and it runs CUDA, which a Mac and an AMD box do not. The catch is 273 GB/s. A dense 70B model fits comfortably and crawls; mixture-of-experts models, which keep many parameters but read only a few billion per token, are what this machine is for. Think of it as a CUDA development box with a lot of memory, not a fast one.
What it is good at
- CUDA on 128 GB of unified memory: the NVIDIA software stack, from vLLM to TensorRT-LLM, runs as it does on a datacenter GPU.
- Blackwell, so FP4 weight formats run natively.
- A 150 mm box on a 240 W power supply, quiet enough for a desk.
What it is not
- 273 GB/s: dense models above 30B generate slowly however well they fit.
- An Arm Linux machine running DGX OS. Tools need ARM64 builds; Windows software does not run.
- nvidia-smi reports memory use as "Not Supported" and cudaMemGetInfo under-reports, which makes the memory look missing when it is not.
The thing people get wrong: NVIDIA's "up to 200 billion parameters" assumes 4-bit weights, as the datasheet's own footnote says. At Q8 the same memory holds roughly half that.
Compared with the alternatives
- RTX Spark: 102 GB · 300 GB/s · best fit Nemotron 3.5 Bolt 74B-A9B (MoE) at ~40 tok/s. NVIDIA's chip for Windows laptops and mini PCs: up to 128 GB shared with the GPU, read at 300 GB/s, with CUDA.
- Ryzen AI Max+ 395: 96 GB · 256 GB/s · best fit Nemotron 3.5 Bolt 74B-A9B (MoE) at ~34 tok/s. AMD's 128 GB laptop and mini-PC chip: up to 96 GB for the GPU on Windows, and an x86 route to large mixture-of-experts models.
- MacBook Pro M5 Max: 96 GB · 614 GB/s · best fit Nemotron 3.5 Bolt 74B-A9B (MoE) at ~82 tok/s. Apple's 128 GB laptop: large models fit and are read at 614 GB/s, if you buy the 40-core GPU.
The unified-memory machines are compared head to head, on the same models, in DGX Spark vs Strix Halo vs Mac Studio.
How it stacks up against the other personal AI computers, from the Ghost Core to an RTX PRO 6000 workstation: personal AI computers compared.
All 40 cards are compared side by side on the GPU index, and the same numbers from the model's point of view are on each model fit table. To check one specific pairing rather than read a table, use the can-I-run-it checker.
Questions
- Can a DGX Spark run a 70B model?
- Yes. Llama 3.3 70B at Q4_K_M with 8k of context needs about 45.8 GB, inside the 126 GB this card can address, and it generates an estimated 5 tokens per second, which is too slow for interactive use.
- What is the best model to run on a DGX Spark?
- Nemotron 3.5 Bolt 74B-A9B (MoE). At Q4_K_M and 8k context it needs about 45.9 GB of the 126 GB available and generates an estimated 37 tokens per second, which is comfortable for chat. If quality matters more than size, Llama 3.3 70B fits at Q8 in about 81 GB.
- How many tokens per second does a DGX Spark generate?
- It depends entirely on the size of the model, because generating a token means reading every active weight out of memory. At 273 GB/s and 70% efficiency, this card produces an estimated 41 tokens per second on an 8B model at Q4, and about 51 on the largest model it holds, Mistral Small 4 119B (MoE). Reading speed is roughly 5 to 8 words per second, so anything above 25 feels immediate. Treat 41 as a ceiling on 273 GB/s rather than a measurement; the higher a figure, the further real runtimes fall below it.
- Is 126 GB enough for running LLMs locally?
- It runs 40 of the 40 models tracked here at Q4_K_M with 8k of context, up to 119B parameters. The honest test is not the model list but the context: Nemotron 3.5 Bolt 74B-A9B (MoE) on this card holds about 131,072 tokens before memory runs out.
- DGX Spark or RTX Spark for local models?
- This card holds more: 126 GB against 102 GB, but both run all 40 of these models at Q4_K_M with 8k of context, so the extra memory goes to longer context and higher precision. On speed, the RTX Spark is faster: 300 GB/s against 273 GB/s, and bandwidth is what sets chat speed.
- How much of the DGX Spark's 128 GB can the GPU use?
- NVIDIA does not publish a figure. The GPU shares the whole pool with the Arm CPU and DGX OS, less a 2 GB display reserve (4 GB is an option in the BIOS), so what a model can use is whatever the system leaves free. This site plans around 126 GB and keeps a 5% margin for the operating system. Do not judge it by nvidia-smi, which shows memory use as "Not Supported" on this machine.
- DGX Spark or Mac Studio for local LLMs?
- The Spark runs CUDA, so the NVIDIA-only half of the ecosystem, including most fine-tuning and many image and video tools, works on it. A Mac Studio M5 Ultra reads memory at 1,200 GB/s against 273 GB/s and goes up to 512 GB, so it generates faster and holds more, without CUDA. For chatting with big models, the Mac; for building on NVIDIA hardware or fine-tuning, the Spark.
- Memory: weights (parameters × bytes per parameter) + KV cache (2 × layers × KV heads × head dim × context × bytes for a standard transformer; sliding-window, hybrid and latent-attention models counted as they actually cache) + 0.5 GB runtime + 4% of weights. Full derivation in the VRAM calculator.
- Speed: 70% × bandwidth ÷ active weight bytes, single stream, no batching, prompt processing excluded. A ceiling, not a measurement: the 9 of 40 figures marked ≤ are where real runtimes fall furthest below the number, because so few weights are read per token. See the tokens-per-second estimator.
- Card specification (manufacturer figures): 128 GB, 273 GB/s, 140 W; see NVIDIA's own page for this machine. Unified memory: NVIDIA publishes no GPU share: the GPU can use whatever of the 128 GB the operating system leaves free, less a 2 GB display reserve. Figures here use 126 GB; the operating system comes out of the 5% margin every fit keeps, so run the largest models without a desktop session. Source: NVIDIA's DGX Spark release notes (the 2 GB display reserve).
- Model architecture values come from each model's published config; the self-hosted LLM guide explains what each one changes.