Local LLM benchmark finds memory budgets do not prevent slowdowns

single source· 1 articles · confidence: medium · first seen 2026-09-11 20:00 UTC

What this means for you

If you serve models on Apple Silicon, the finding worth keeping is the failure mode, not the ranking: two engines completed every request while memory ran near the machine's limit and throughput fell, so a memory budget is not headroom. Nothing to switch to yet — this is one paper, unreplicated.

Researchers released SiliconBench, a benchmark for running language models locally on machines where the CPU and GPU share one pool of RAM, as on Apple Silicon. It scores nine serving engines on speed, memory and output fidelity, on chat and agent workloads, checking quality against an NVIDIA reference. On Qwen3-0.6B, vllm-metal more than doubled throughput between concurrency 1 and 16; CUDA vLLM and SGLang scaled better. Only three of the nine stacks cleared every gate — completion, fidelity and model coverage. Two completed every request while memory approached physical capacity and throughput fell. Code and per-run results are released; no independent replication yet.

Key facts

  • ·SiliconBench evaluates nine Apple Silicon LLM serving engines on three axes: speed, memory and output fidelity. source
  • ·On Qwen3-0.6B, vllm-metal more than doubled throughput on both chat and agent workloads between concurrency 1 and 16. source
  • ·Only three of the nine stacks satisfied all three gates: request completion, fidelity against an NVIDIA reference, and model coverage. source
  • ·Two stacks completed every request while memory use approached physical capacity, with throughput declining despite explicit memory budgets. source
  • ·In two-machine tests, tensor parallelism over Thunderbolt RDMA scaled, while pipeline parallelism over TCP regressed. source
  • ·The benchmark code, per-run results and maintenance journals were released alongside the paper on 11 September 2026. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire