Transformer diagrams are usually drawn as stacks, where a token enters at the bottom, passes through attention and feed-forward blocks, and eventually becomes a distribution over the next token. This picture is architecturally correct, but it is not always the best picture for interpretation. It encourages us to ask what a layer represents, even though the model’s computation is distributed across components that repeatedly read from and write to a shared state.

Elhage et al. (2021) proposed a different unit of analysis. Because transformer blocks add their outputs to a residual stream, an attention-only transformer can be expanded into a sum of end-to-end paths, which explains why one can say the residual stream is an additive communication channel. Each attention head can then be separated into a query-key circuit, which determines where information is read from, and an output-value circuit, which determines what is written. In two-layer models, multiplying out the same equations reveals Q-, K-, and V-composition between heads.

This post begins with a concise reconstruction of that beautiful framework spearheaded by Anthropic’s Interpretability research team. It then asks what survives after removing its strongest simplification. The models studied in the original derivation had at most two layers and no MLPs. Contemporary transformers alternate attention with nonlinear feed-forward blocks, often use a gated MLP such as SwiGLU, and may replace each dense MLP with a sparse mixture of experts. The residual stream is still additive, but the operators that write to it are no longer fixed linear maps. To that end, the second part of this post proposes what is called a conditional residual-stream calculus, which formalizes a now well-known notion:

A nonlinear transformer still admits an exact path expansion for a realized forward pass, but its effective operators are indexed by the residual trajectory that activates them.

Using math, I intend to show that this change has an interpretive payoff. Notably:

Expanding an ordinary MLP exposes input-conditioned rank-one writers;
Expanding SwiGLU exposes two distinct routes by which earlier residual writes can compose with a later channel: through its content branch or through its gate; and
Expanding a mixture-of-experts layer introduces a second distinction, between changing what an expert computes and changing which expert is selected.

The algebraic identities will be separated throughout from local approximations and from mechanistic hypotheses that require causal validation.

Let’s start with a brief recap of what the residual stream is in a transformer.

The residual stream

Let a sequence contain $n$ positions and let the residual width be $d$. We collect the residual vectors into a matrix

$$X_\ell\in\mathbb R^{n\times d},$$

with one token position per row. Ignoring normalization for the moment, a decoder block performs two residual updates:¹1 LayerNorm and RMSNorm are themselves input-dependent nonlinear maps. They can be included in the forward-effective or Jacobian framework developed later, but doing so would obscure the structural difference between attention, MLPs, and expert routing that is the focus here. Their omission is a scope choice, not a claim that they are mechanistically irrelevant.

$$\begin{aligned} X_{\ell,\mathrm{attn}} &=X_\ell+\operatorname{Attn}_\ell(X_\ell),\\ X_{\ell+1} &=X_{\ell,\mathrm{attn}} +\operatorname{MLP}_\ell(X_{\ell,\mathrm{attn}}). \end{aligned}$$

Neither sublayer overwrites the state it receives. It computes a change and adds that change back. Recursively,

$$X_L =X_0 +\sum_{\ell=0}^{L-1}\operatorname{Attn}_\ell(X_\ell) +\sum_{\ell=0}^{L-1}\operatorname{MLP}_\ell(X_{\ell,\mathrm{attn}}).$$

This shared additive channel is the residual stream. It performs no computation by itself. Embeddings initialize it; attention heads and MLPs read from it and write to it; later components can read any combination of what earlier components have left there. Information can occupy different residual subspaces, persist through identity paths, be amplified, or be cancelled by a later write.

If $T\in\mathbb R^{n\times |V|}$ contains one-hot token vectors, the embedding and unembedding endpoints are

$$X_0=TW_E, \qquad Y=X_LW_U,$$

where $Y\in\mathbb R^{n\times |V|}$ contains the logits.²2 I use row vectors for token representations: $W_E\in\mathbb R^{|V|\times d}$ and $W_U\in\mathbb R^{d\times |V|}$. The original Transformer Circuits article often uses the transpose convention. Nothing substantive depends on this choice, but some matrix products consequently appear transposed relative to that article.

The additivity of the stream is what makes path expansion possible. If a sequence of residual blocks were linear, we could write

$$X_L=(I+F_{L-1})\cdots(I+F_1)(I+F_0)X_0$$

and distribute the product. Every resulting term would select either the identity route or the residual branch at every block. A product of layer operators would become a sum of paths from the embedding to the logits.

Attention is not globally linear because its attention pattern depends on its input. Once a pattern is fixed, however, an attention head is linear in the information it moves. That is enough structure to begin.

Why tensor products appear

Transformer activations have at least two relevant axes: position and residual feature. Attention mixes positions; learned projections transform features. Tensor-product notation lets us say which operation acts on which axis without flattening the whole sequence into one enormous vector.

For a position map $A\in\mathbb R^{n\times n}$ and a feature map $W\in\mathbb R^{d\times d}$, define the action

$$(A\otimes W)X:=AXW^\top.$$

Operationally:

$I_n\otimes W$ applies the same feature transformation at every position;
$A\otimes I_d$ moves information across positions without changing its residual coordinates;
$A\otimes W$ does both.

The notation also remembers that the two axes compose independently:

$$(A\otimes B)(C\otimes D) =(AC)\otimes(BD).$$

This mixed-product rule is the reason tensor products become useful rather than decorative. It lets positional routes and feature-space routes be multiplied separately while remaining part of one end-to-end path.

One attention head

For head $h$, take

$$W_Q^h,W_K^h,W_V^h\in\mathbb R^{d_h\times d}, \qquad W_O^h\in\mathbb R^{d\times d_h}.$$

The head’s query-key score matrix is

$$S^h(X) =\frac{X(W_Q^h)^\top W_K^hX^\top}{\sqrt{d_h}} =\frac{XW_{QK}^hX^\top}{\sqrt{d_h}},$$

where

$$W_{QK}^h:=(W_Q^h)^\top W_K^h.$$

Causal masking and a row-wise softmax turn the scores into the attention pattern $A^h(X)$. The value and output projections always occur together, so define

$$W_{OV}^h:=W_O^hW_V^h.$$

The head output becomes

$$H^h(X) =A^h(X)X(W_{OV}^h)^\top =\bigl(A^h(X)\otimes W_{OV}^h\bigr)X.$$

This factorization separates two operations:

$W_{QK}^h$ determines the attention pattern: where does the head read?
$W_{OV}^h$ determines the residual update: what does the head read and write?

Queries, keys, values, and output vectors remain useful implementation-level objects. For circuit analysis, however, the composite matrices are often more fundamental: $W_Q$ and $W_K$ only affect the forward pass through their product, as do $W_O$ and $W_V$.

From layers to paths

Zero layers: the direct path

A transformer with no blocks maps tokens directly through embedding and unembedding:

$$Y=TW_EW_U.$$

The vocabulary-space matrix

$$C_{EU}:=W_EW_U\in\mathbb R^{|V|\times |V|}$$

describes how the current token directly changes the next-token logits. Because this path does not move information between positions, it can express a token-level bigram contribution but no contextual dependency.

One layer: QK and OV circuits

For a one-layer attention-only transformer,

$$X_1 =\left[ I_n\otimes I_d +\sum_{h\in H_1}A^h(X_0)\otimes W_{OV}^h \right]X_0.$$

The logits are therefore

$$Y =TW_EW_U +\sum_{h\in H_1} A^h(X_0)TW_E(W_{OV}^h)^\top W_U.$$

Each head term contains two vocabulary-level circuits. The first is

$$C_{QK}^h :=W_EW_{QK}^hW_E^\top.$$

An entry $(C_{QK}^h)_{qk}$ is the pre-softmax score between a destination token type $q$ and source token type $k$, ignoring positional contributions. The second is

$$C_{OV}^h :=W_E(W_{OV}^h)^\top W_U.$$

A row of $C_{OV}^h$ says how attending completely to one source token would change the output logits at the destination.

Together, the two circuits describe a skip-trigram:

$$[\text{source}]\;\cdots\;[\text{destination}] \longrightarrow [\text{output}].$$

The QK circuit determines whether the destination reads the source; the OV circuit determines how that source changes the output distribution. This is an exact description of the one-layer attention-only model, but not automatically a compact human interpretation. For a vocabulary of tens of thousands of tokens, each expanded circuit contains billions of entries.

Two layers: three axes and composition

The second layer introduces a qualitatively new phenomenon: its heads read a residual stream that already contains first-layer head outputs.

For a second-layer head $h_2$, its score matrix is

$$S^{h_2} =X_1W_{QK}^{h_2}X_1^\top.$$

The first-layer expansion is substituted into both arguments of this bilinear form. Multiplication produces four classes of terms:

neither side reads a first-layer head;
only the query side reads one;
only the key side reads one; and
both sides read first-layer heads.

At this point, an operator such as $A\otimes W$ is no longer enough. A second-layer attention score has a query-position axis, a key-position axis, and a residual-feature interaction. Terms therefore take the form

$$A_q\otimes A_k\otimes W.$$

The expression highlighted in the original derivation,

$$\sum_{h_q\in H_1} A^{h_q}\otimes I\otimes \left(W_{OV}^{h_q}W_E\right)^\top,$$

tracks the terms in which a first-layer head writes to the later query while the key side remains on its direct path. The first tensor factor transports query-side information across positions, the identity leaves key-side positions unchanged, and the last factor tracks the feature content written from the embedding through the first-layer OV map.³3 This displayed term follows the original article’s orientation so that it can be compared directly with its two-layer expansion. Under the row-vector convention used elsewhere in this post, the same map can be written with the corresponding products transposed.

When this path enters the later $W_{QK}^{h_2}$, the earlier head influences the later query. This is Q-composition. The term with the two positional factors reversed gives K-composition. In both cases, a first-layer head changes where a second-layer head attends.

There is a third form. If the second-layer value computation reads information written by a first-layer head, their OV maps compose:

$$W_{OV}^{h_2}W_{OV}^{h_1}.$$

This is V-composition. With attention patterns frozen, the positional maps compose as well, producing

$$(A^{h_2}A^{h_1}) \otimes (W_{OV}^{h_2}W_{OV}^{h_1}).$$

The product behaves like a new, longer-range virtual attention head. No physical head has those parameters; the effective head is an end-to-end path through two actual heads.

This is the main achievement of the original calculus. Q-, K-, and V-composition are not imposed as interpretive labels after inspecting examples. They appear because the residual updates are substituted into later QK and OV computations and the resulting products are expanded.

A short circuit gallery

The path language has supported several canonical mechanistic-interpretability results. The examples below are landmarks, not a survey.

Induction heads

An induction pattern has the form

$$[A][B]\;\cdots\;[A] \longrightarrow[B].$$

In the minimal two-layer account, an earlier previous-token head shifts information so that a later head’s keys represent what followed each earlier token. The later query can then match the current $A$ against previous $A$-related keys and copy the associated $B$. This is naturally expressed as K-composition between heads. Elhage et al. (2021) derive the circuit in small attention-only models; Olsson et al. (2022) provide causal evidence in small models and broader, more qualified evidence connecting induction heads to in-context learning.

Indirect object identification (IOI)

Consider a prompt such as:

When Mary and John went to the store, John gave a drink to …

GPT-2 Small tends to predict Mary: the name that is not the repeated subject. Wang et al. (2023) identify an indirect-object-identification (IOI) circuit containing 26 attention heads grouped into seven functional classes, including duplicate-token heads, induction heads, inhibition heads, and name-mover heads. The circuit exhibits Q-, K-, and V-composition, but its discovery also required causal interventions and explicit tests of faithfulness, completeness, and minimality. That is an important methodological boundary: algebra suggests possible paths; empirical circuit analysis must establish which paths implement the behavior.

Greater-than

Hanna, Liu, and Variengien (2023) analyze how GPT-2 Small completes prompts such as “The war lasted from 1732 to 17…” with two-digit years greater than 32. Unlike the attention-only examples, the resulting circuit assigns a central role to late MLPs, which promote valid end years in vocabulary space. This is precisely the kind of circuit for which an attention-only path calculus is incomplete: the model’s decisive transformation occurs inside a nonlinear residual writer.

Where the attention-only calculus stops

For a position-wise MLP,

$$M(x)=W_{\mathrm{out}}\phi(W_{\mathrm{in}}x),$$

the residual update is still additive:

$$x^+=x+M(x).$$

But $M$ is not a fixed matrix. If we expand $x$ as a sum of earlier residual writes, the nonlinearity generally prevents us from distributing $\phi$ across that sum:

$$\phi(x_1+x_2) \neq \phi(x_1)+\phi(x_2).$$

There are at least three different questions one might now ask:

Forward decomposition: which terms sum exactly to the output observed on this input?
Differential decomposition: how would an infinitesimal change propagate from this input?
Causal decomposition: what happens if a component or path is changed by a finite intervention?

These questions coincide for a globally linear map. They diverge for nonlinear blocks. Much confusion about nonlinear “circuits” comes from answering one while speaking as if one had answered all three.

Ordinary MLPs as conditional operators

The native neuron expansion

Let the hidden width be $m$. Write the rows of $W_{\mathrm{in}}$ as $k_i^\top$ and the columns of $W_{\mathrm{out}}$ as $v_i$. Then

$$M(x) =\sum_{i=1}^m \phi(k_i^\top x)v_i.$$

Each hidden unit has a read direction $k_i$, an input-dependent scalar coefficient $\phi(k_i^\top x)$, and a write direction $v_i$. This identity motivates the view of feed-forward layers as key-value memories: the first matrix detects input patterns and the second supplies value vectors that are combined into the output (Geva et al. 2021). It also gives an exact neuron-level decomposition before any approximation or learned dictionary is introduced.

Exactness does not guarantee interpretability. Native neurons can be polysemantic, while meaningful features can be represented in superposition across many neurons. Sparse autoencoders and transcoders attempt to recover more interpretable units, but they trade the exact native decomposition for a learned approximation (Elhage et al. 2022; Dunefsky, Chlenski, and Nanda 2024). We will return to that distinction after deriving the native operator.

Why a diagonal gate appears

Set

$$z=W_{\mathrm{in}}x.$$

The two weight matrices mix coordinates, but $\phi$ acts on each hidden coordinate independently. Once $z$ is known, the activation does not rotate one hidden coordinate into another; it scales each coordinate by an amount determined by that coordinate. The matrix representation of independent coordinate-wise gains is diagonal.

Define

$$D_{\phi}^{\mathrm{fwd}}(z) :=\operatorname{diag}\!\left( \gamma_\phi(z_1),\ldots,\gamma_\phi(z_m) \right),$$

where

$$\gamma_\phi(s) := \begin{cases} \phi(s)/s,&s\neq0,\\ 0,&s=0\text{ and }\phi(0)=0. \end{cases}$$

Coordinate-wise,

$$\phi(z_i)=\gamma_\phi(z_i)z_i,$$

and therefore

$$\phi(z)=D_{\phi}^{\mathrm{fwd}}(z)z.$$

Substituting $z=W_{\mathrm{in}}x$ gives

$$M(x) =W_{\mathrm{out}} D_{\phi}^{\mathrm{fwd}}(W_{\mathrm{in}}x) W_{\mathrm{in}}x.$$

Define the forward-effective MLP operator

$$B_M^{\mathrm{fwd}}(x) :=W_{\mathrm{out}} D_{\phi}^{\mathrm{fwd}}(W_{\mathrm{in}}x) W_{\mathrm{in}}.$$

Then

$$M(x)=B_M^{\mathrm{fwd}}(x)x$$

exactly. The MLP has not become globally linear. Rather, its nonlinear forward pass has selected an effective linear operator for this particular $x$.

Biases turn this into an affine identity. They can be displayed explicitly or absorbed into a homogeneous coordinate $\tilde x=(x,1)$.⁴4 The bias-free notation keeps the path structure visible. With $z=W_{\mathrm{in}}x+b_{\mathrm{in}}$, one obtains $M(x)=W_{\mathrm{out}}D_\phi^{\mathrm{fwd}}(z)(W_{\mathrm{in}}x+b_{\mathrm{in}})+b_{\mathrm{out}}$. Augmenting the state with a constant coordinate rewrites the affine map as a linear map on the augmented space.

ReLU: an exact linear region

For ReLU,

$$\operatorname{ReLU}(s)=\mathbf 1[s>0]s,$$

$$D_{\mathrm{ReLU}}^{\mathrm{fwd}}(z) =\operatorname{diag}(\mathbf 1[z_i>0]).$$

The mask is constant within each activation region. As long as a perturbation does not cause any preactivation to cross zero, the MLP is exactly linear there:

$$M(x)= W_{\mathrm{out}} D_{\mathrm{ReLU}}^{\mathrm{fwd}}(z) W_{\mathrm{in}}x.$$

A ReLU MLP—and a network built only from affine maps and ReLUs—is therefore piecewise affine. A full transformer still contains the softmax nonlinearity in attention. Conditional on fixed attention patterns and a fixed ReLU region, ordinary residual path expansion applies with fixed operators; crossing either kind of boundary changes which paths are active.

GELU: an exact soft gate

For exact GELU,

$$\operatorname{GELU}(s)=s\Phi(s),$$

where $\Phi$ is the standard normal cumulative distribution function. Hence

$$D_{\mathrm{GELU}}^{\mathrm{fwd}}(z) =\operatorname{diag}(\Phi(z_i)).$$

The same identity remains exact:

$$M(x) =W_{\mathrm{out}} \operatorname{diag}(\Phi(W_{\mathrm{in}}x)) W_{\mathrm{in}}x.$$

Unlike the ReLU mask, the GELU gains vary smoothly with $x$. We no longer have finitely bounded activation regions with one constant operator. We have a continuum of input-conditioned operators.

A forward path expansion

Let $Z\in\mathbb R^{n\times d}$ denote the state presented to an MLP, with token rows $z_i$, and set $B_{\ell,i}^{\mathrm{fwd}}:=B_{M_\ell}^{\mathrm{fwd}}(z_i)$. The full sequence-level MLP operator is block diagonal across positions:

$$\mathcal B_\ell^{\mathrm{fwd}}(X_\ell) =\bigoplus_{i=1}^n B_{\ell,i}^{\mathrm{fwd}} =\sum_{i=1}^n P_i\otimes B_{\ell,i}^{\mathrm{fwd}},$$

where $P_i$ projects onto position $i$.⁵5 A static position-wise linear layer has the simpler form $I_n\otimes B$. For a nonlinear MLP, different tokens usually select different effective matrices, so $\bigoplus_i B(x_i)$ is more precise than $I_n\otimes B(x)$.

To preserve the actual order of the sublayers, first define the realized attention operator

$$\mathcal A_\ell(X_\ell) :=\sum_{h\in H_\ell} A_\ell^h(X_\ell)\otimes W_{OV,\ell}^h$$

so that $X_{\ell,\mathrm{attn}}=[I+\mathcal A_\ell(X_\ell)]X_\ell$. One complete attention-then-MLP block is therefore

$$\mathcal R_\ell(X_\ell) := \left[ I+\mathcal B_\ell^{\mathrm{fwd}}(X_{\ell,\mathrm{attn}}) \right] \left[ I+\mathcal A_\ell(X_\ell) \right].$$

Multiplying the two factors already exposes four within-block routes: the identity, attention alone, the MLP reading the pre-attention residual, and the MLP reading an attention-head write. This last product would be lost if the two sequential sublayers were treated as parallel residual branches.

Along the actual trajectory,

$$X_L =\mathcal R_{L-1}(X_{L-1}) \cdots \mathcal R_0(X_0)X_0.$$

After evaluating every attention pattern and MLP gate on that trajectory, distributing this product yields an exact sum of forward paths. The price is that the path operators are conditional on the complete forward pass. If one path is ablated, later $A_\ell^h$ and $\mathcal B_\ell^{\mathrm{fwd}}$ may change. Keeping them frozen answers a frozen-gate counterfactual, not necessarily the behavior of the intervened model.

To summarize, “exact summand of a realized forward pass” does not imply “independently executable causal mechanism.”

Forward, differential, and causal paths

The operator that reconstructs $M(x)$ is not generally the operator that propagates a perturbation.

For the ordinary MLP,

$$J_M(x) =W_{\mathrm{out}} \operatorname{diag}\!\left( \phi'(W_{\mathrm{in}}x) \right) W_{\mathrm{in}}.$$

For a small $\delta$,

$$M(x+\delta) =M(x)+J_M(x)\delta+O(\|\delta\|^2).$$

For ReLU away from an activation boundary, the forward and differential gates coincide. For GELU they do not:

$$\frac{\operatorname{GELU}(s)}{s}=\Phi(s), \qquad \operatorname{GELU}'(s)=\Phi(s)+s\varphi(s),$$

where $\varphi$ is the standard normal density. The forward-effective operator reconstructs the value of the MLP at $x$; the Jacobian describes its infinitesimal sensitivity at $x$.

For a finite change from $x_0$ to $x_1$, the fundamental theorem of calculus gives

$$F(x_1)-F(x_0) =\int_0^1 J_F\!\left(x_0+t(x_1-x_0)\right) (x_1-x_0)\,dt.$$

Expanding the Jacobian with the chain rule produces a sum of differential paths under the integral. This is exact for the chosen baseline and interpolation path, but it attributes a change rather than decomposing the unperturbed computation. Integrated gradients is the corresponding attribution method at input-feature level (Sundararajan, Taly, and Yan 2017).

Other circuit methods occupy different positions:

Attribution patching estimates a finite activation intervention using a first-order gradient calculation. It is efficient but approximate (Syed, Rager, and Conmy 2023).
Activation patching measures the model under an actual intervention, but one intervention does not decompose the output into an additive set of independent causes.
Path patching tests whether a selected computational path mediates a behavior under a specified counterfactual distribution (Goldowsky-Dill et al. 2023).

These methods are complementary. A conditional forward expansion supplies candidate structure; differential scores help prioritize paths; interventions test mechanistic claims.

SwiGLU: gate-content composition

The preceding derivation treats an MLP as a collection of input-conditioned writers. A gated MLP introduces a richer interaction. In SwiGLU, two linear reads from the residual stream meet multiplicatively before the result is projected back.

Ignoring biases,

$$M_{\mathrm{SwiGLU}}(x) =W_D\left[ \operatorname{SiLU}(W_Gx) \odot (W_Ux) \right].$$

Here $W_G$ supplies the gate branch, $W_U$ the content or “up” branch, and $W_D$ the down projection into the residual stream. Let

$$g=W_Gx, \qquad u=W_Ux.$$

The Hadamard product can be written as a diagonal action:

$$\operatorname{SiLU}(g)\odot u =\operatorname{diag}(\operatorname{SiLU}(g))u.$$

Therefore,

$$M_{\mathrm{SwiGLU}}(x) =W_D \operatorname{diag}(\operatorname{SiLU}(W_Gx)) W_Ux.$$

This is again an exact forward-effective operator:

$$B_{\mathrm{SwiGLU}}^{\mathrm{fwd}}(x) :=W_D \operatorname{diag}(\operatorname{SiLU}(W_Gx)) W_U.$$

The diagonal matrix now has a particularly direct meaning: each gate coordinate scales the corresponding content coordinate before the down projection mixes the surviving channels into the residual stream.

One channel, three roles

Let $g_j^\top$ and $u_j^\top$ be rows of $W_G$ and $W_U$, and let $d_j$ be the corresponding column of $W_D$. Then

$$M_{\mathrm{SwiGLU}}(x) =\sum_j \operatorname{SiLU}(g_j^\top x) (u_j^\top x)d_j.$$

Each channel has three roles:

$g_j$ is a gate reader;
$u_j$ is a content reader;
$d_j$ is a residual writer.

Since $\operatorname{SiLU}(s)=s\sigma(s)$,

$$M_{\mathrm{SwiGLU}}(x) =\sum_j \sigma(g_j^\top x) (g_j^\top x) (u_j^\top x)d_j.$$

A channel is therefore a softly gated bilinear interaction between two residual directions. It does not merely ask whether one pattern is present and then retrieve a fixed value. It can combine a gate-side feature and a content-side feature multiplicatively before deciding what to write.

Pure bilinear layers have been proposed as more tractable objects for mechanistic interpretability because their interactions admit an exact third-order tensor description (Sharkey 2023). SwiGLU is not purely bilinear: the sigmoid factor varies with the gate preactivation. But the comparison shows where its additional complexity lies. Conditional on the sigmoid gain, the channel contains an explicit bilinear feature interaction.

Expanding the Jacobian

The local Jacobian reveals two routes through which an earlier residual write can affect the channel. Differentiating gives

$$J_{\mathrm{SwiGLU}}(x) =W_D\left[ \operatorname{diag}(\operatorname{SiLU}(g))W_U +\operatorname{diag}\!\left( u\odot\operatorname{SiLU}'(g) \right)W_G \right].$$

The first term propagates a perturbation through $W_U$ while holding the gate value locally fixed. The second propagates it through $W_G$, changing how strongly the content branch is admitted.

This suggests two forms of composition:

Content composition

An earlier component writes a residual direction that a later SwiGLU channel reads through $W_U$. It changes the content carried through the current gate:

$$\delta x \xrightarrow{W_U} \delta u \xrightarrow{\operatorname{diag}(\operatorname{SiLU}(g))} \delta\text{ write}.$$

Gate composition

An earlier component writes a residual direction that the channel reads through $W_G$. It changes the gain applied to a potentially different content direction:

$$\delta x \xrightarrow{W_G} \delta g \xrightarrow{u\odot\operatorname{SiLU}'(g)} \delta\text{ write}.$$

The terminology is proposed here descriptively; it is not established MI nomenclature. The claim of two Jacobian terms is an exact mathematical statement. The stronger claim—that trained transformers use stable, interpretable gate-content circuits—is a mechanistic conjecture.

The analogy with attention is nevertheless useful. Q- and K-composition distinguish two inputs to a bilinear attention score. SwiGLU gate- and content composition distinguish two inputs to a gated multiplicative feature interaction. In both cases, substituting earlier residual writes into a later multilinear computation reveals qualitatively different path types.

Features, transcoders, and conditional transforms

The native SwiGLU decomposition identifies channels, but it does not guarantee that those channels align with human concepts. Modern feature-based MI offers two nearby approaches.

Dunefsky, Chlenski, and Nanda (2024) train sparse transcoders to approximate an MLP’s input-output map. Active transcoder features become sparse computational nodes, allowing feature-to-feature paths through MLPs to be analyzed. Anthropic’s later Circuit Tracing uses cross-layer transcoders to build prompt-specific attribution graphs in a replacement model. These graphs make nonlinear computations more legible, but they inherit reconstruction error, feature-splitting concerns, and differences between the replacement model and the original network.

A second line is even closer to the operator developed here. Anthropic’s preliminary sparse mixtures of linear transforms replace an MLP with sparsely active low-rank transformations:

$$\widehat M(x) =\sum_t \phi(e_t^\top x-b_t) U_tV_tx.$$

The activating condition and the linear transformation are separated. This can preserve geometric computations that a rank-one transcoder may fragment into many lookup-like features. The method is a learned surrogate rather than an exact factorization of the native MLP, but it supports the same general picture: nonlinear MLP computation can be studied as a sparse, input-conditioned collection of feature transformations.

Recent work has also begun to unify attention and MLP credit through a common lookup-style decomposition and backward recursion (Chen, van Stein, and Plaat 2026). None of these approaches, by itself, supplies a globally fixed residual-stream algebra for the native nonlinear model. Together they provide practical counterparts to the conditional calculus: learned features suggest interpretable nodes, conditional transforms suggest interpretable edges, and interventions test whether the resulting graph is faithful.

Mixture of experts: routing and content

A sparse mixture-of-experts layer replaces one dense MLP with a collection of expert MLPs and a learned router. For one token residual $x$, write

$$M_{\mathrm{MoE}}(x) =\sum_{e\in S(x)} \pi_e(x)E_e(x),$$

where $S(x)$ is the selected top-$k$ expert set, $\pi_e(x)$ is the router weight, and $E_e$ is an expert—often itself a gated MLP. This is the basic sparse-gating form introduced by Shazeer et al. (2017) and subsequently simplified or scaled in architectures such as Switch Transformers and Mixtral.

Suppose each expert has an exact forward-effective operator

$$E_e(x)=B_e^{\mathrm{fwd}}(x)x.$$

Then

$$M_{\mathrm{MoE}}(x) =\left[ \sum_{e\in S(x)} \pi_e(x)B_e^{\mathrm{fwd}}(x) \right]x.$$

The MoE operator is thus conditioned twice:

the router selects and weights experts;
each selected expert activates its own internal nonlinear operator.

At sequence level, this remains position-local:

$$\mathcal B_{\mathrm{MoE}}(X) =\bigoplus_{i=1}^n \left[ \sum_{e\in S(x_i)} \pi_e(x_i)B_e^{\mathrm{fwd}}(x_i) \right].$$

This contrasts cleanly with attention. An attention head uses $A(X)$ to transport information across token positions and $W_{OV}$ to transform it. An MoE layer does not ordinarily transport information between positions; it routes each token’s local computation across parameterized expert subnetworks.

The MoE Jacobian

Within a region where the selected set $S(x)$ is unchanged, differentiate the MoE output:

$$J_{\mathrm{MoE}}(x) =\sum_{e\in S(x)} \left[ \pi_e(x)J_{E_e}(x) +E_e(x)\nabla\pi_e(x)^\top \right].$$

Again, the expansion exposes two mathematically distinct paths.

Expert-content composition

The term

$$\pi_e(x)J_{E_e}(x)$$

describes an upstream perturbation changing what an already selected expert computes. If the expert is SwiGLU, this term further decomposes into its gate and content routes.

Router composition

The term

$$E_e(x)\nabla\pi_e(x)^\top$$

describes an upstream perturbation changing the router weight, thereby modulating an entire expert output. A residual direction can matter not because an expert reads it as content, but because the router uses it to decide which computation should be applied.

Hard top-$k$ selection adds a discrete case: at a routing boundary, an arbitrarily small change can replace one expert with another, and the ordinary Jacobian is undefined at the switch.⁶6 Production MoEs may also impose expert-capacity limits and drop or reroute tokens. In those systems the realized operation can depend on other tokens in the batch, not only on the token residual being routed. That systems-level complication is outside the baseline calculus here.

This suggests a MoE circuit vocabulary (read as block: selection computation / content transformation):

Attention: QK selects source positions / OV moves and writes information
Dense SwiGLU: gate branch modulates channels / content branch is transformed and written
Sparse MoE: router selects expert subnetworks / experts transform and write information

The analogy should not be forced. QK attention computes a relation between two token positions; an MoE router usually computes a choice over parameter blocks for one position. But both architectures separate which computation is activated from what the activated computation writes.

Recent work has begun to attribute routing decisions to earlier transformer components and reports persistent promoting and inhibiting influences on later routes (Li et al. 2026). Other work redesigns MoEs for greater intrinsic interpretability rather than deriving circuits from ordinary pretrained MoEs (Yang et al. 2025). I am not aware of an established MoE analogue of the full QK/OV and head-composition calculus. The routing/content split above should therefore be read as an exact local decomposition followed by a conjectural interpretive program, not as a settled account of expert specialization.⁷7 This literature claim reflects a search conducted in August 2026 across the Transformer Circuits thread, arXiv, OpenReview, the ACL Anthology, and citation chains from MoE interpretability papers. It is an absence-of-evidence claim, not proof that no unpublished or differently named formalism exists.

What is “genuinely general”?

The diagonal formulas exploit the structure of familiar activations, but an exact input-conditioned operator exists more generally. Let $F:\mathbb R^d\to\mathbb R^d$ be differentiable and suppose $F(0)=0$. Define its radial effective operator

$$\overline J_F(x) :=\int_0^1J_F(tx)\,dt.$$

Then

$$F(x) =\int_0^1\frac{d}{dt}F(tx)\,dt =\left[\int_0^1J_F(tx)\,dt\right]x =\overline J_F(x)x.$$

Thus any differentiable residual writer can be inserted into an exact conditional product

$$x_{\ell+1} =\left[I+\overline J_{F_\ell}(x_\ell)\right]x_\ell.$$

Evaluating these operators on the realized trajectory and multiplying out gives a formally general forward path calculus.

Generality alone is not enough. The radial construction selects one effective operator, but the equation $B(x)x=F(x)$ does not determine a unique matrix from a single input-output pair; other baselines or integration paths produce other decompositions. The resulting operator may also be dense and no easier to interpret than the original function. A useful calculus needs structured factorizations that expose the architecture’s natural choices:

attention separates positional selection from residual transformation;
an ordinary MLP separates scalar activation from rank-one writes;
SwiGLU separates gate reads, content reads, and residual writes;
an MoE separates routing from expert computation.

These factorizations are what turn an existence identity into a candidate language of mechanisms.

References

Chen, P.-K., van Stein, N., & Plaat, A. (2026). “Every Component is a Lookup: Token Attribution and Composition from a Single Decomposition.” arXiv:2605.23393.
Dunefsky, J., Chlenski, P., & Nanda, N. (2024). “Transcoders Find Interpretable LLM Feature Circuits.” arXiv:2406.11944.
Elhage, N., et al. (2021). “A Mathematical Framework for Transformer Circuits.” Transformer Circuits Thread. Article.
Elhage, N., et al. (2022). “Toy Models of Superposition.” Transformer Circuits Thread. Article.
Fedus, W., Zoph, B., & Shazeer, N. (2021). “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.” arXiv:2101.03961.
Geva, M., Schuster, R., Berant, J., & Levy, O. (2021). “Transformer Feed-Forward Layers Are Key-Value Memories.” EMNLP 2021, 5484–5495. ACL Anthology.
Goldowsky-Dill, N., MacLeod, C., Sato, L., & Arora, A. (2023). “Localizing Model Behavior with Path Patching.” arXiv:2304.05969.
Hanna, M., Liu, O., & Variengien, A. (2023). “How does GPT-2 compute greater-than? Interpreting mathematical abilities in a pre-trained language model.” arXiv:2305.00586.
Jiang, A. Q., et al. (2024). “Mixtral of Experts.” arXiv:2401.04088.
Li, W., Zhang, L., Endo, T., & Wahib, M. (2026). “Understanding Cross-layer Contributions to Mixture-of-Experts Routing in LLMs.” ICLR 2026. Conference page.
Lindsey, J., et al. (2025). “Circuit Tracing: Revealing Computational Graphs in Language Models.” Transformer Circuits Thread. Methods.
Lindsey, J., Chen, B., Pearce, A., Hydrie, S., & Conerly, T. (2025). “Sparse mixtures of linear transforms.” Transformer Circuits Thread. Research update.
Olsson, C., et al. (2022). “In-context Learning and Induction Heads.” arXiv:2209.11895.
Sharkey, L. (2023). “A Technical Note on Bilinear Layers for Interpretability.” arXiv:2305.03452.
Shazeer, N., et al. (2017). “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer.” arXiv:1701.06538.
Sundararajan, M., Taly, A., & Yan, Q. (2017). “Axiomatic Attribution for Deep Networks.” arXiv:1703.01365.
Syed, A., Rager, C., & Conmy, A. (2023). “Attribution Patching Outperforms Automated Circuit Discovery.” arXiv:2310.10348.
Wang, K. R., Variengien, A., Conmy, A., Shlegeris, B., & Steinhardt, J. (2023). “Interpretability in the Wild: A Circuit for Indirect Object Identification in GPT-2 Small.” ICLR 2023. arXiv:2211.00593.
Yang, X., et al. (2025). “Mixture of Experts Made Intrinsically Interpretable.” arXiv:2503.07639.