An interactive essay · Active Circuit Discovery
We trust large language models to write, reason, and decide — yet we can barely see how they do it. This is the story of an agent that borrows a 30-year-old theory of the brain to hunt, one experiment at a time, for the circuits hidden inside an AI's mind.
by Dr. Sharath Sathish · a guided tour for the curious and the technical alike
Here is the strange truth at the heart of modern AI: nobody designed a large language model's behaviour. We designed a learning process, poured in much of the internet, and a mind precipitated out — billions of numbers that somehow know grammar, geography, and how to code. We can run it. We can't open it.
It's the difference between an engineer's blueprint and a biologist's specimen. A circuit board is designed; you can read its schematic. A brain is grown; you have to dissect it to learn how it works. Today's models are far closer to the brain: grown, not authored.
A frontier model holds hundreds of billions of parameters. The natural unit to inspect — the neuron — is polysemantic: a single neuron fires for dozens of unrelated concepts at once, because the network packs far more features than it has neurons (a phenomenon called superposition). Behavioural testing — "red-teaming" — samples outputs but never reveals mechanism, the way a crash test never explains the engine. Regulators have noticed: the EU AI Act and NIST's AI risk framework both lean on transparency the field cannot yet deliver.
A small field is building that microscope. Mechanistic interpretability treats a trained network like an organism to be reverse-engineered: find the internal features it represents, and the circuits that wire those features into behaviour.
The first surprise is that there's structure to find at all. In vision models, researchers found neurons for curves, then circuits that assemble curves into car wheels. The same playbook moved to language. But there's a catch: look at a single neuron and you'll see nonsense — it lights up for "the Golden Gate Bridge," "Python errors," and "the year 1987." Concepts are superposed.
The fix is a kind of prism. A sparse autoencoder (or transcoder) decomposes the tangled activations back into many clean, single-meaning features — one for the Golden Gate Bridge, one for Python errors, and so on. This is the picture below: a real attribution graph (the haystack of features), trimmed to the most influential.
A model with hidden width d can represent D ≫ d features by placing them as nearly-orthogonal directions in activation space — superposition. Polysemantic neurons are the symptom. Sparse autoencoders (Bricken et al. 2023; Cunningham et al. 2024) learn an overcomplete dictionary of directions with an L1/JumpReLU sparsity penalty, recovering monosemantic features; transcoders extend this to predict the next layer, letting you trace features across layers. Gemma Scope (Lieberum et al. 2024) released millions of such features for Gemma 2 — the exact backend this paper builds on.
Make it concrete with the field's fruit-fly task — and the very prompt this paper probes:
"When John and Mary went to the store, John gave the bag to ___"
You answered Mary without thinking. So does the model — and in GPT-2 researchers mapped exactly how. A circuit of ~26 attention heads cooperates: some heads notice that "John" appears twice, others inhibit that repeated name, and a special group — the "name-mover heads" — copy the other name to the output. Ablate them and accuracy collapses from ~90% to chance. That is a circuit: a minimal, causal, nameable mechanism.
Indirect Object Identification spans all 12 layers of GPT-2 small: duplicate-token heads → inhibition heads → name-mover heads at layers 9–11 attending from the final position to the indirect-object token, copying it to the logits. ~6 "backup" heads provide redundancy, which is why single-head ablations often barely dent performance — a recurring theme in real circuits.
How do you show a part is causal, not just present? The same way a mechanic tests a theory about spark plugs — interfere, and watch what breaks. Three interventions, increasingly gentle:
Switch the feature off entirely. The bluntest, most informative probe — like pulling the spark plug to see if the engine dies.
Replace its value with one from a different run. A precise, confirmatory swap — does this signal carry that effect?
Amplify it. The gentlest nudge — turn it up and see how far behaviour bends (this is how Golden Gate Claude was made).
The yardstick throughout is KL divergence: a number for how much the model's output distribution moved after the intervention. Big move ⇒ the part mattered.
Ablation (zero or mean) measures necessity but can knock the model off-distribution. Activation patching (Meng et al.; Syed et al. 2023) swaps a clean activation into a corrupted run (or vice versa) for a sharper causal estimate. Causal scrubbing stress-tests a hypothesised circuit by resampling everything outside it. All reduce to one question: does intervening here change the output, measured as KL between clean and intervened logits?
Step away from AI for a moment. Cover half a line of text on your phone with your thumb — you still "see" the whole sentence. Your brain didn't photograph the words; it predicted them and showed you the guess. You only notice when a prediction is wrong.
This is the brain-as-prediction-machine view, and its boldest formulation is Karl Friston's Free Energy Principle: living systems act to minimise surprise — the gap between what they predict and what they sense. Perception updates the model; action changes the world to match the model. Both reduce the same quantity.
"Surprise" (−log p(o)) is intractable, so the brain minimises a tractable upper bound — variational free energy — by keeping an internal belief q(s) close to the true posterior p(s|o):
Minimise F by changing beliefs (perception) or by acting to change observations (active inference). See Friston (2010) and Parr, Pezzulo & Friston (2022).
Now the idea that makes everything click. When an active-inference agent weighs an action, it scores it on two things at once — and a single quantity, Expected Free Energy, combines them:
That's the whole explore-vs-exploit dilemma, resolved by one rule. Think of a student picking a class: photography (loved, but nothing new) versus philosophy (uncertain, but transformative). A good agent weighs both. Drag the slider — watch which experiment the agent prefers as its uncertainty changes:
{{ c.blurb }}
{{ efe.caption }}
EFE scores a policy by expected information gain plus expected preference:
Minimising the epistemic term means choosing experiments whose outcomes you can't yet predict — the most informative ones (MacKay 2003's optimal experimental design, recovered from first principles). The agent is built on discrete active inference (Da Costa et al. 2020) via the pymdp library (Heins et al. 2022), and learns its observation model online with Dirichlet (conjugate) updates. What's genuinely novel here isn't any single ingredient — it's wiring them into one loop for circuit discovery.
Two halves click together. A backend builds the haystack — an attribution graph of candidate features. An Active-Inference agent then decides, step by step, which feature to probe and how.
The agent holds a belief about every candidate feature along three axes — how important, which depth, and how influential — and it owns the three interventions you just met. Crucially, those actions differ in how much they teach: ablation is high-information (for exploring), steering is low-information (for confirming). So the explore-then-exploit rhythm isn't hand-coded — it falls out of Expected Free Energy.
From its generative model (A, B, C, D) the agent maintains a probabilistic belief over each feature's hidden state.
For every (feature, action) pair it computes Expected Free Energy — the blend of information gain and preference.
It executes the single best experiment via the feature_intervention API and measures the KL shift, activation, and connectivity.
It updates its beliefs and refines its own observation model online with a Dirichlet update — then repeats, sharper than before.
Three hidden state factors — importance {negligible…high}, layer-role {early/mid/late}, causal influence {weak…strong} — and three observation modalities (KL magnitude, activation, graph connectivity). The transition model B(a) encodes the action semantics: ablation has the widest, highest-entropy belief updates (max epistemic value), steering the narrowest. That single design choice is what makes exploration emerge. The per-step decision loop:
Theory is cheap; here is a real run on a real prompt, twenty experiments deep. Each bar is one intervention coloured by the action chosen; below it, the agent's belief entropy (its uncertainty) and expected free energy. Scrub through every step.
To grade discovery we use bounded oracle efficiency: how close the agent's chosen features come to a perfect oracle that already knows each feature's true effect. 100% = oracle-level. Compare methods, tasks, and both models.
{{ eff.caption }}
{{ eff.h1text }}
On Gemma-2-2B's IOI task the agent reaches 82.0%, decisively beating random selection. The gradient baseline (EAP) is the one to beat — a genuinely strong, cheap ranking — and the agent trades blows with the bandit. On Llama-3.2-1B it only reaches 52.1%, about level with random.
Attribution patching uses gradient information about the true output, so it's a strong static prior. The agent's edge is adaptivity under uncertainty — most valuable when signal is noisy or the budget tiny, less decisive when one gradient ranking is already near-perfect. In short, the POMDP reliably beats uninformed search and is competitive with the best heuristics, rather than dominating them.
One number looks spectacular: with steering on, the agent's cumulative effect reaches 1255% of the ablation oracle on Gemma IOI. Super-oracle discovery? No. And the data says why.
{{ rck.caption }}
We can pin it down. Take circuit-selected features and matched random controls, scale each from ×1.5 to ×10, and count how often the prediction flips:
{{ steer.caption }}
Amplifying features does change predictions — steering is real.
binomial p ≈ 2.6 × 10⁻¹² (Gemma, ×10)
Circuit-selected features are only marginally more steerable than random active ones — selectivity is not significant.
selected vs control p ≈ 0.31 (Gemma, ×10); pooled p ≈ 0.10
The sharpest failure is the most instructive. On Llama's multi-step reasoning task, efficiency collapses to 9.3% — below random.
The cause isn't the theory — it's resolution. The agent bins layers into three coarse thirds. Gemma has 26 layers; Llama only 16, so three bins smear together the early layers where reasoning lives. Refine to six bins and efficiency more than triples, to 37.8% — confirming the diagnosis.
Run the agent across five kinds of knowledge and it stays robust everywhere — but where the important features sit depends on the task and the model.
{{ dom.caption }}
Each bar = the top causal features for that domain, split by depth. Longer bars simply found more high-influence features.
On Gemma the most influential features cluster in early layers across every domain. On Llama the balance tips later for some (geography, maths) and stays early for others (logic, history). There's no single universal "facts-live-here" rule — and that's the useful part: rather than assuming a layout, the agent surfaces each task's layer fingerprint for inspection. Toggle the models to compare.
These are the verbatim prompts from the experiment code. For each knowledge domain the bar shows its layer fingerprint on Gemma — how the top causal features split across early / middle / late layers. Factual lookups and compositional reasoning leave visibly different prints.
The headline isn't a leaderboard win. It's a reframing: interpretability under a budget is an experimental-design problem, and a theory of the brain gives us a principled way to run those experiments.
Every number in this essay is computed from the paper's real released results — no figures were redrawn by hand or invented. Explore the code, reproduce the runs, or read the sources behind the ideas.
Dr. Sharath Sathish works at the intersection of mechanistic interpretability and active inference. Read the paper and the code, or follow the work below.