Decoy weights cut refusal-removal attacks on open models to under 10%

single source· 1 articles · confidence: medium · first seen 2026-09-13 20:00 UTC

What this means for you

Nothing to deploy: this is a preprint, not a product, and the paper mentions no code or weights release. The durable point is that an open model's refusal behaviour is not a security boundary — even here, the authors report worst-case attack success of 65% under an adaptive attack.

A 13 September preprint proposes a defence against refusal-feature ablation, a way to strip safety behaviour from open-weight models: find and project out the direction that carries refusals, and the model stops declining. Decoy Direction Optimization edits weights after training, injecting a decoy signal into the model's neurons so an attacker's estimator locks onto a harmless direction instead. The authors report under 10% attack-success rate across six model families, and on Llama-3-8B-Instruct cut a weight-level attack's success from 88.7% to 18%. Under adaptive multi-phase attacks the same model still shows 65% worst-case attack success, against 58% for a trained baseline. The figures are the authors' own; no evaluation dates.

Key facts

  • ·Decoy Direction Optimization works by injecting a decoy signal into the network's MLP neurons to corrupt an attacker's estimator of the refusal direction, requiring no base-model fine-tuning. source
  • ·The authors report under 10% attack success rate under standard Refusal Feature Ablation across six model families. source
  • ·On Llama-3-8B-Instruct, worst-case attack success under adaptive multi-phase attacks is reported at 65%, against 58% for trained defences. source
  • ·The Heretic weight-level attack's success on Llama-3-8B-Instruct is reported as reduced from 88.7% to 18%. source
  • ·Optimisation cost is reported as 30 to 450 times lower per configuration than trained baselines. source
  • ·The paper was posted to arXiv on 13 September 2026 and has not been peer-reviewed. source

What the sources say

Sources

The original reporting. Follow these — they did the work.

← the wire