New confidence method reads a model's graded past attempts

single source· 1 articles · confidence: medium · first seen 2026-09-14 20:00 UTC

What this means for you

Nothing to act on today: this is a preprint, with no code, weights or API. The claim worth tracking is that confidence can be read off a stored record of past outcomes without logit access, which would make it portable to any hosted model — and could cut ten samples per query to one.

A preprint proposes estimating a language model's confidence from its own graded history rather than from the answer it is producing now. XConf stores past episodes — the task, the stated confidence, the outcome, a lesson — and for a new task retrieves similar cases where the model claimed similar confidence, reads off how often they were right, and has the model restate its confidence. Across nine benchmarks and four models from three families it beat or matched ten-sample self-consistency on 23 of 24 comparisons, with lower calibration error, at one generation instead of ten. No independent evaluation is reported.

Key facts

  • ·XConf estimates a model's confidence from a record of its own graded past episodes, not from the current inference run. source
  • ·It beats or matches ten-sample self-consistency on discrimination (AUROC) in 23 of 24 comparisons, across nine benchmarks and four models from three families. source
  • ·The method requires no logit access and no weight updates, and costs one answer generation against ten for self-consistency. source
  • ·Abstaining on the 10% least-confident episodes raised the delivered success rate by up to 8.7 points on agent tasks. source
  • ·The Recall stage retrieves past episodes on similar tasks met with a similar stated confidence; the Reflect stage has the model name a recurring failure mode and restate its confidence. source
  • ·Posted as arXiv preprint 2609.17708 on 14 September 2026; no evaluation by authors outside the paper is reported. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire