A closed-form capacity score picks architectures without training a model

single source· 1 articles · confidence: medium · first seen 2026-09-18 20:00 UTC

What this means for you

If you pick architectures by parameter count or by hand search, this is a claim to test, not a tool to adopt. The paper gives no implementation, no weights and no harness details, so the pruning result cannot be reproduced from the text alone.

A preprint proposes Neural Spectral Capacity, a closed-form score that ranks how much a network can represent from its architecture alone, with no model built, no data and no gradients. The score comes from the singular-value spectrum of each weight matrix — how strongly each layer stretches its inputs. Its authors report it ranks designs better than parameter count or FLOPs across seven Transformer and CNN families, and that a dynamic-programming solver using it trimmed LLaMA-7B to what the paper calls the best 5.7B model on eight commonsense reasoning tasks, about 5,900 times faster than the strongest training-free alternative. It is not peer-reviewed, and no evaluation date is given.

Key facts

  • ·Neural Spectral Capacity is computed from the architectural specification alone, with no model instantiation, data or gradients, using the Marchenko-Pastur law under standard random initialisation. source
  • ·On FlexiBERT, NSC scores 0.505 (Kendall tau) on architecture pairs whose parameter counts differ by less than 10%, where parameter count alone scores 0.082. source
  • ·NSC-DP, a dynamic-programming solver using the score, found a Transformer-XL architecture on WikiText-103 that beats the human-designed baseline, in 2 seconds on a CPU. source
  • ·The method prunes LLaMA-7B to a 5.7B model reported as best across eight commonsense reasoning tasks with no calibration data, about 5,900x faster than the strongest training-free proxy baseline. source
  • ·NSC outperforms parameter count, FLOPs and representative training-free proxies at ranking architectures across seven Transformer and CNN families. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire