Prism ML packs a 27B model into a 5.95 GB ternary file

single source· 1 articles · confidence: medium · first seen 2026-09-17 22:13 UTC

What this means for you

If you want to try these weights, stock llama.cpp will not load them; you need Prism's fork. Budget about 6 GB of disk, and treat the reported throughput as unverified — one M5 Pro ranged from 20 to 44 tokens per second across runs, with a warning that Metal's tensor API was unavailable.

Prism ML has published ternary-quantised weights for Bonsai 2 27B: a GGUF file of about 5.95 GB, in which each of the 27 billion parameters is stored as one of three values rather than a full-precision number. Running it needs Prism's own llama.cpp fork, not a stock build; Simon Willison posted a working download-and-serve recipe for macOS. He measured about 20 tokens per second on an M5 Pro, then 44 after restarting the server, and the runtime reported Metal's tensor API as unsupported. No evaluation results, benchmark dates or licence terms appear in the write-up.

Key facts

  • ·The file Ternary-Bonsai-2-27B-PTQ1_0.gguf is about 5.95 GB. source
  • ·The weights require Prism ML's fork of llama.cpp (release prism-b10685-7dffb15) rather than a stock build. source
  • ·Willison measured roughly 20 tokens per second on an M5 Pro, and 44 tokens per second after a server restart. source
  • ·On startup the server reported "the tensor API is not supported in this environment - disabling". source
  • ·The example server command sets a context length of 32768 tokens. source
  • ·The weights are hosted in the Hugging Face repository prism-ml/Ternary-Bonsai-2-27B-gguf. source

What the sources say

  • Simon Willison (press) — Runbook for serving the ternary weights locally, with download commands and a note that throughput was inconsistent.

Sources

The original reporting. Follow these — they did the work.

← the wire