Nvidia serving recipe speeds a 124B model 42.5% on one DGX Spark
single source· 1 articles · confidence: low · first seen 2026-11-03 20:28 UTC
What this means for you
Nothing to act on. This is a demonstration post, not a release — no configuration, patch or supported-model list is given, so the 42.5% cannot be reproduced from what was published. Watch for the method, not the number.
Nvidia has posted three stacked serving optimisations that speed up a 124B-parameter mixture-of-experts model (only part of the network runs on each request, so it is cheaper to serve than its size suggests) on a single DGX Spark. On Ling-3.0-flash, from Ant Group's inclusionAI, decode throughput (the rate at which the model generates output tokens) on GSM8K rose 42.5%, with no detectable change in accuracy, according to the 3 November write-up. The write-up gives neither the test setup behind the throughput figure nor the accuracy numbers themselves, so the 42.5% cannot be checked from what was published.
Key facts
- ·Nvidia demonstrated three stacked optimisations for mixture-of-experts inference on a single DGX Spark, published 3 November 2026. source
- ·The model used was Ling-3.0-flash, a 124B-parameter model from Ant Group's inclusionAI. source
- ·GSM8K decode throughput rose 42.5% with the three optimisations applied. source
- ·No detectable accuracy change is reported; no accuracy figures or evaluation date are given. source
What the sources say
- MarkTechPost — Short write-up of Nvidia's demo; gives the model, the machine and the throughput gain but no method detail.
Sources
The original reporting. Follow these — they did the work.