Zyphra reports a sparse attention retrofit that matches dense at 256K
single source· 1 articles · confidence: low · first seen 2026-11-03 01:09 UTC
What this means for you
Nothing to act on yet: no weights, no serving code and no evaluation date, so the claim cannot be checked. If you pay for 256K-token inference, the shape of the method is the interesting part — attention alone is retrained, not the whole model.
Zyphra has published a way to cut long-context inference costs in models it did not train. Zyphra Hierarchical Attention keeps a tree index over past tokens and computes attention only over selected pairs — sparse attention, so cost grows more slowly with sequence length than dense attention, which compares every token with every other. The retrofit freezes the model's weights and trains the new pattern on 4B tokens of distillation (learning to reproduce the original's outputs). Zyphra says it matches dense attention on reasoning and 256K-token retrieval benchmarks; no evaluation date, harness or base models are given.
Key facts
- ·Zyphra Hierarchical Attention is a tree-indexed sparse attention method, meaning only a subset of token pairs is computed. source
- ·The retrofit leaves the base model's weights frozen and uses 4B tokens of distillation to install the new attention pattern. source
- ·Zyphra reports the method matches dense attention on reasoning and on 256K-token retrieval benchmarks. source
- ·No evaluation date, harness, base model list, code release or pricing is given in the report. source
- ·The report was published on 3 November 2026 at 01:09 UTC. source
What the sources say
- MarkTechPost — Short announcement of the method; gives the token budget and benchmark claim but no paper detail.
Sources
The original reporting. Follow these — they did the work.