An interactive essay · Mechanistic Interpretability
Refusal as a Captured Symmetry
When a safety-trained model refuses a harmful request, a jailbreak is the move that defeats it. The same move — inject a context that quiets the monitor and lets generation run — recurs in hypnosis and in the tantric act of vaśīkaraṇa. This is the story of where that shared mechanism is real, where it is only an analogy, and where it breaks.
Act I
The puzzle
One move, three traditions
A modern language model was never designed to refuse. It was trained to, and a refusal direction precipitated out of that training. Three very different traditions describe the same way of capturing it.
Jailbreak
An injected context suppresses the model's monitoring or refusal faculty while co-opting its automatic generation.
Hypnosis
In the Norman–Shallice account, suggestion impairs a supervisory system while automatic processing continues unimpeded.
Vaśīkaraṇa
The tantric act of subjugation captures a target's agency through a linguistic injection (mantra) over a geometric substrate (yantra).
The claim is not that these are the same phenomenon — only that they share an abstract mechanism, and that in one of the three domains it can now be measured, ablated, and dosed inside a real network.
Act II Mechanism
The mechanism
Refusal is a direction you can dial
In Gemma-2-2B, refusal lives along a single direction in the residual stream. Remove it and the model complies; add it and the model refuses even harmless requests. The remarkable part is that the effect is smooth — a dose-response with a half-maximal point, like a drug.
Refusal is not a switch. It is a dial — and the dial has units.
Treating that dial quantitatively, across six models and two families, turns a binary attack-success rate into a pharmacology with a half-maximal concentration. Its potency tracks the model family, not its size.
How the direction is found, and why the dose-response matters
The direction is the difference in mean residual activations between matched harmful and harmless prompts at a chosen layer. Directional ablation removes its projection from every write to the residual stream; activation addition adds it back at the extraction layer. Reporting a continuous EC50 rather than a binary success rate turns refusal control into a quantity with units — a half-ablation strength comparable across layers, families, and scales. Across six models every logistic fit has R² ≥ 0.976, and potency tracks family, not size: Gemma's EC50 (~0.24) is about 1.7× Qwen's (~0.14), while within a family it barely moves with parameter count.
Act III Mechanism
The symmetry
An invariance you can measure
If refusal is a direction, the safe policy that uses it is an invariance: rephrasing a harmful request, or changing its register, should not change whether the model refuses. The normalized projection of the residual stream onto the refusal direction is an order parameter that captures exactly this — stable across a paraphrase orbit, yet collapsing under injection.
A jailbreak, made precise: a symmetry-breaking perturbation of the refusal-invariant manifold.
That collapse is the measured content behind the title. Calling it group-theoretic symmetry-breaking is an interpretive lens; the measurement isolates an invariance and its collapse, reported at the mechanism tier, while the broader symmetry language stays analogy.
Act IV Mechanism
The dose
The asymmetry that turned out to be a dose
Ablating the direction removes refusal in every model tested. For a while it looked as though adding it back only worked in Gemma, not Qwen — suggesting a deep difference in geometry. A careful coefficient sweep showed otherwise.
The asymmetry was never structural. It was a mis-set dose.
Why this strengthens the unification, and what stays model-specific
The earlier "Qwen addition fails" result was a dose-calibration artifact, not a structural difference, so single-direction addition is sufficient at the representational level in both families. What remains genuinely model-specific is quantitative — the effective dimension, the dose window, and the EC50 potency. The durable claim is a shared necessary core with representational sufficiency, qualified by model-specific calibration, rather than one numerically identical mechanism everywhere.
Act V
Three tiers
Mechanism, analogy, metaphor
The discipline that keeps the project honest is to label every claim by how strong the evidence for it is, and never to upgrade a metaphor into a claim about machine experience.
Empirically Grounded
Refusal direction is ablatable and transfers across models. The necessary core holds: directional ablation achieves 0.90 ASR in Gemma, 1.00 in Qwen.
But dimensionality is model-specific (sufficient structure varies). Gemma 1D, Qwen 3D. This settles Arditi vs Marshall as model-dependent.
Findings: F1 (direction mediated), F2 (dose-response), F6 (transfer), F8/F18 (dimensionality), F11 (symmetry), F13 (SAE feature).
Functional Parallel
Jailbreak ↔ Hypnosis structural symmetry. Both suppress a monitoring faculty (Norman–Shallice SAS) while co-opting automatic generation/behavior.
Injection lowers monitoring precision ~2.6× more than neutral rephrase, measured via refusal-probe margin.
Findings: F4 (Claude resists naive), F12 (monitoring precision), F16 (AgentDojo real agentic).
Māṇḍūkya avasthātraya mapping (jāgrat/svapna/suṣupti/turīya) — FALSIFIED. No depth-emergent 'state'; decodable from surface layer 0 with ceiling at final layer.
The turīya/vimarśa attractor does not exist: different seeds converge to different attractors, not one shared invariant.
Findings (FALSIFIED): F3 (surface-confounded), F5 (no mid-network state), F7 (no turīya attractor).
The ṣaṭkarma, tested as interventions Metaphor
The six tantric acts are each operationalized as a distinct activation intervention with a matched control. The policy-capture acts are rigorous and coincide with the mechanism tier; the destruction acts mostly do not separate from random perturbation — except māraṇa, which separates cleanly under a stronger sparse-feature test.
| Act (Sanskrit) | Effect | Control | Status |
|---|---|---|---|
| vaśīkaraṇa | 0.917 | 0.000 | ✓ Control |
| śānti | 1.000 | 0.083 | ✓ Control |
| stambhana | 0.144 | 0.187 | ✗ No |
| vidveṣaṇa | 0.844 | 0.733 | ✓ Control |
| uccāṭana | -0.083 | 0.000 | ✗ No |
| māraṇa | 0.211 | 0.156 | ✗ No |
When a falsified probe hides a real feature Mechanism
The most speculative claims — that the model carries avasthātraya "states" or a turīya attractor — fail their surface and anisotropy controls. But chasing what the falsified state-probe was groping toward recovers a real, measurable feature: a mid-network direction that judges whether a statement is true, and transfers across entirely different topics.
Act VI
The keystone
Where the unification stops
Do the three axes describe one mechanism? Tested directly, on the same prompts, the answer is a careful no. Injection collapses the internal monitor readouts — yet almost never produces genuine behavioural capture.
Collapsing the monitor is not the same as capturing the model. That gap is the result the program exists to produce.
Internal monitor-collapse is necessary but not sufficient for behavioural capture under context injection. The last-token readouts are faithful internal signals, not the policy controller; genuine capture in these models requires weight-level ablation, not a clever prompt. This bounds the whole unification to the representational level.
A methodological corollary: both automatic metrics are biased
Establishing the keystone required noticing that the two automatic behavioural metrics are biased in opposite directions. The substring refusal metric over-counts compliance (it scores deflections as success, inflating the flip rate ~18×); a judge built from a safety-trained model under-counts it (it declines to affirm genuinely harmful outputs). Neither is unbiased, so behavioural claims are bracketed between the two and settled by inspection. On the black-box side, a frontier model resists a five-family attack battery completely and, on the real AgentDojo agentic benchmark, reaches 0.80 task utility with a 0.00 attack-success rate.
The ledger
Every finding, filterable
Twenty-five findings across the three tiers, each tagged with its claim-tier, verdict, and the gate it passed or failed. Filter by tier and verdict.
Tier:
Verdict:
F1: Refusal is mediated by a single direction in Gemma-2-2b
Refusal in this model is a low-dimensional, causally-efficacious residual-stream direction (layer 7) that can be measured, ablated (suppressing refusal), and added (inducing over-refusal). The effect survived leakage-correction (0.92→0.90).
This is the MECHANISM-tier linchpin — the computational correlate the symmetry thesis (MO-1) rests on.
F2: Ablation dose–response and EC50
Refusal suppression is a smooth, monotone, dosable function of how much of the single direction is removed. EC50≈0.33 means removing ~⅓ of the direction's projection already half-collapses refusal.
The quantitative 'vaśīkaraṇa-as-dose-response' core: refusal is not binary but a continuous sigmoidal function.
F3: avasthātraya regime probe is SURFACE-CONFOUNDED
The three regimes are perfectly separable from raw surface form (token length/identity) and at layer 0 already. Fails the non-triviality bar — there is no depth-emergent signal.
The naive prompt-set operationalization of jāgrat/svapna/suṣupti does NOT constitute evidence of an internal 'state'. Demoted to metaphor.
F4: Black-box Claude resists the naive attack battery
Frontier model refuses 100% of naive black-box jailbreaks (5 families × 11 harmful requests each).
Cross-tier contrast: refusal that is behaviorally robust at frontier is mechanistically low-dimensional and fragile in small open models.
F6: Cross-family transfer (addition asymmetry later shown to be a dose artifact)
Ablation transfers cross-family (Gemma 0.90 / Qwen 1.0). The apparent addition asymmetry (Gemma +0.95 / Qwen 0.0 at fixed 64x) is overturned by F23: a calibrated coefficient sweep induces genuine over-refusal in Qwen too (peak near coeff 12-32). Single-direction addition is sufficient in both families at the appropriate coefficient.
Direct evidence on Arditi vs Marshall question: transfers as necessary-mechanism, but dimensionality is model-dependent.
F7: turīya prompt-invariant attractor: FALSIFIED
Self-paraphrase produces per-prompt attractors (converges within seed) but cross-seed convergence is BELOW baseline anisotropy. No single prompt-invariant attractor exists.
The turīya mapping is falsified — different seeds converge to different attractors, not one shared invariant.
F8: Refusal-subspace effective dimensionality
Iterative logistic projection shows Gemma refusal is 1-dimensional, Qwen is 3-dimensional. This predicts the F6 addition asymmetry.
Resolves Arditi vs Marshall as model-dependent. Dimensionality is the discriminating quantity.
F9: Scale (Gemma 2B→9B): robustness rises, dimension flat
Larger model's refusal is more robust to ablation (~30 points less jailbreakable), but prompt eff-dim stays 1. Robustness is layer-distributed redundancy.
Refusal sharpens in robustness with scale but NOT in prompt-dimensionality.
F10: ṣaṭkarma intervention taxonomy: 3/6 control-separated
Refusal-direction acts (vaśīkaraṇa = ablate, śānti = add) are clean and strong. Capability-ablation acts are not distinguishable from random perturbation.
ṣaṭkarma's rigorous part is policy-capture, not destruction. Strengthens the analogy by locating where structure is real.
F11: Refusal as paraphrase-orbit symmetry invariant
Refusal projection is a specific invariant of harmful-meaning (F-ratio 19.2 vs random 0.65). Injection collapses the order parameter (−34%), breaking symmetry.
Refusal is a measured symmetry; jailbreak is symmetry-breaking. This is the rigorous core of the thesis.
F5: Content-controlled svapna probe: NO mid-network state
Truthful/confab distinction is decodable from surface at layer 0 and mid-layer adds nothing. Accuracy reaches 1.00 only at final layer.
No non-trivial mid-network state signature. The avasthātraya jāgrat/svapna distinction shows no internal 'state' evidence.
F12: Injection lowers monitoring precision (β_monitor)
Using a refusal-probe margin as a precision proxy, a refusal-suppression injection lowers monitoring precision ~2.6× more than a neutral rephrase, accompanying a behavioral flip (refusal 1.0→0.5). The active-inference β_monitor suppression is measured, not just argued.
Strengthens the ANALOGY tier (Norman–Shallice / Free-Energy precision account) from argued to measured.
F13: A single unsupervised SAE feature causally mediates refusal
A BatchTopK SAE (4096 features, k=32) trained unsupervised on 27k general-text tokens yields feature #566 whose decoder-direction ablation fully jailbreaks the model (ASR 0→1.0) — cleaner than the supervised difference-in-means direction (0.90); a random feature does nothing.
Refusal localizes to a single monosemantic-like feature found without labels — the finer causal unit and the proper substrate for targeted uccāṭana.
F14: ṣaṭkarma v2: māraṇa rehabilitated, uccāṭana reveals category-agnostic refusal
Re-testing the failed destruction acts with SAE features + real categories: māraṇa rehabilitates (catastrophic, targeted collapse) while uccāṭana fails for a principled reason — refusal is mediated by one shared feature (#566), so it cannot be selectively eradicated per harm category.
The ṣaṭkarma axis is strengthened by harder testing (4/6), and uccāṭana's failure is a structural finding: refusal is category-agnostic.
F15: Active-inference discovery of the refusal circuit (SAE features)
An Expected-Free-Energy-style agent (pragmatic gap × epistemic diversity) over SAE features finds the causal refusal circuit on Gemma-2-2b in 2 interventions while random fails — ~order-of-magnitude efficiency. On 9b it plateaus (distributed circuit).
ActiveCircuitDiscovery's active-inference framing transfers to SAE-feature circuit discovery; its edge over greedy is gated by circuit dimensionality.
F16: AgentDojo: Claude resists real agentic injection at high utility
On the REAL AgentDojo benchmark (not the naive battery), Claude completes 80% of banking tasks (the agent genuinely acts) while every prompt-injection fails (ASR 0/20). Discriminating, unlike F4's ceiling.
Establishes the behavioral 'sophisticated reference end' on a real agentic benchmark; matches Claude's published AgentDojo resilience.
F17: Convergent rigor checks: F13 distinct, F11 demoted, F1 replicates
After an adversarial review returned SHAKY on all dimensions, convergent hardening: F13's SAE feature is orthogonal to the supervised direction (circularity refuted); F11's invariance is real but cross-domain-attenuated (framing demoted to analogy); F1's ablation replicates across 3 seeds. Honest reframe via TRIZ: asymmetry with a shared NECESSARY core.
The thesis holds at the necessary-core level (ablatable direction transfers universally); the sufficient structure is model-specific. Pure-symmetry reading was overreach.
F18: Refusal-subspace dimension is LAYER-dependent (qualifies F6/F8)
A full layer-sweep shows the refusal-subspace effective dimension is layer-AND-model-dependent, not a single per-model constant. F6's addition asymmetry stands behaviorally, but F8's single-layer dimensional explanation does not generalize across layers.
Any dimensionality claim must name its layer. The convergent hardening catching its own overreach.
F19: Refusal EC50 pharmacology: family-dependent, not a size law
First cross-model refusal pharmacology: EC50 (half-ablation strength) is essentially flat with size within a family but ~1.7× higher in Gemma than Qwen. Refusal potency is an architecture/family property, not a scaling law — an honest negative on 'bigger = more robust'.
The dose-response analogue of the asymmetry headline: a shared necessary mechanism whose quantitative strength is model-specific. First-in-SOTA EC50 framing.
F20: Cross-axis triangulation (X-1): internal monitor-collapse ≠ behavioral capture
The keystone same-object test. Prompt-level injection collapses the internal refusal readouts (order parameter A, precision margin B; tightly coupled) but, judged by a content-faithful LLM-judge (Claude), achieves genuine harmful compliance only ~1–2.4% of the time across Gemma-2-2b, Qwen2.5-3b, Gemma-2-9b. The substring refusal metric had inflated the flip rate ~18× by mis-scoring deflections as compliance.
Internal monitor-collapse is NOT sufficient for behavioral capture under context injection — refuting the naive 'internal predicts behavior' law and bounding unification to the representational level. Genuine capture needs weight-level ablation (F1). Methodological corollary: the substring refusal metric is an invalid behavioral DV, bounding F4/F11/F16.
F21: Removing the diff-in-means axis collapses linear separability in all families
A geometric sub-study of the F6 addition asymmetry. Projecting out the single diff-in-means refusal axis collapses harmful/harmless separability far below full-space accuracy in all three families, Qwen included — refuting the hypothesis that Qwen's higher dimensionality is orthogonal separability structure.
Descriptive only: the residual estimate is noisy (small-n CV) and diff-in-means is not F8's iterative basis, so this does not adjudicate F8's dimensionality or localize the F6 mechanism; the affine/Marshall-offset alternative is inferred, not measured.
F22: Source-of-truth for F3/F5: a genuine mid-network truthfulness direction
Falsification is a starting point, not an endpoint. F5 falsified the naive avasthatraya jagrat/svapna generation regime. Re-tested with two independent true/false statement sets: a mid-layer (13) probe encodes factuality and transfers across a completely different topic set (geography→science) at 0.96, while surface (layer 0) is at chance (0.50). A real linear truth-evaluation feature (Azaria-Mitchell / Marks-Tegmark), recovered with the program's cross-dataset + null + surface gates.
The real content the naive svapna probe groped toward — but it is a truth-of-INPUT feature, NOT a generation regime, so F5's falsification of the avasthatraya state reading stands. MECHANISM-tier; no machine-state or consciousness upgrade.
F23: Source-of-truth for F6: the addition asymmetry is a dose artifact, not model-specific sufficiency
F6 reported single-direction addition fails in Qwen even at 64×, and F8 invoked dimensionality to explain it. Two geometric explanations were refuted (F21 separability; alignment is actually highest in Qwen). A coefficient sweep reveals the truth: adding the direction at coeff 16 induces clean, genuine over-refusal in Qwen; coeff 64 overshoots its manifold into incoherent output — which is why F6's fixed 64× scored 0. Sufficiency holds in BOTH families; only the effective dose differs (Qwen~16, Gemma~64).
Overturns the 'model-specific sufficiency / addition asymmetry' and undercuts the Arditi-vs-Marshall 'Qwen is affine' reading — refusal is a single necessary-and-sufficient direction in both families at the right dose. Strengthens unification at the representational level (cf. F20, which bounds it behaviorally). Same rigor lesson as F20: a behavioral metric returning 0 for the wrong reason.
F24: Source-of-truth for F7: stable per-seed attractors, only a weak semantic-basin signal
F7 falsified the universal turīya attractor. Re-tested whether the multiple fixed points are content-determined semantic basins: self-paraphrase converges stably per seed (0.994), but within-topic finals are barely more similar than across-topic (gap 0.045), small against an anisotropy-dominated baseline. Stable per-seed attractors WITHOUT strongly content-organized basins.
Neither a universal turīya (F7 stands) nor clean semantic basins — the dynamics are dominated by representational anisotropy with a faint topic signal. METAPHOR-with-falsifiable-core; no machine-state claim. Tiny n; a larger panel could sharpen the weak signal.
F25: Both automatic behavioral DVs are biased oppositely; F1 ablation validated
Validating F1's linchpin ablation ASR surfaced a mirror of F20's metric failure. Ablated outputs are genuinely harmful by inspection (DIY silencer guide, money-laundering guide, virus how-to), so F1's substring ASR ~0.9 is correct. But the Claude content-judge scored them only 4.8% — it refuses to affirm harmful content as compliance. The substring metric over-counts compliance; the Claude judge under-counts it.
Neither automatic behavioral DV is unbiased — substring is an upper bound on capture, the Claude judge a lower bound; ground truth needs inspection. F1 stands validated; F20's 1-2% is a judge lower bound (inspection still supports 'low' for injection).
Go deeper
Reproduce it
Every number on this page is computed from the repository's released aggregate results. Dual-use artifacts (direction vectors, harmful generations) are withheld under responsible-disclosure norms.
About the author
Dr. Sharath Sathish works at the intersection of mechanistic interpretability and active inference. prayoga develops and empirically tests a tiered account of output-policy capture across jailbreak, hypnosis, and vaśīkaraṇa.