Residual Stream Calculus
We start with the attention-only path expansion as beautifully explicated in mathematical framework of transformers and extend it to a conditional calculus for nonlinear MLPs, SwiGLU, and mixture-of-experts transformers.
Posts
Policy Optimization for RL and LLM Alignment — Theory
The optimization story behind aligning language models with human preferences
Variational Autoencoders
About my favorite generative model of all time!