About
Dimitri Mérault
I've been building iOS apps since 2009 — almost since the beginning of the App Store. Step by Token is a different kind of project: born from my interest in LLMs, and from the conviction that explaining them is the best way to truly understand them.
When language models started becoming part of my daily life, I wanted to go beyond the vague metaphors. Building the visualizations on this site forced me to dig into the mechanisms in detail — that's the way I learned best.
I care about well-crafted interfaces and honest explanations. Step by Token tries to be both at once.
The companion project
Step by Token explains how it works. The Loss Curve shows how you build one.
The two sites are meant to be read together, though neither needs the other. Here you manipulate visualisations to build mechanical intuition; there you write the code, run it in the browser, and walk away with a small transformer training on your own machine.
The method
Every chapter starts from a concrete intuition, never from a formula. If a concept can be shown interactively, it is: each of the 21 chapters has at least one manipulable visualisation, and building them is how I understood best what I was describing.
The path is cumulative. The tokens of chapter 02 become vectors in 03, look at each other in 04, and travel through a Transformer block in 05. Nothing is assumed before it has been explained.
The promise is never to cheat on difficulty: when a mechanism is counter-intuitive, the text says so rather than smoothing it over.
Inspirations
The form owes a lot to Daniel Shiffman’s The Nature of Code: manipulable sketches as the teaching unit, an open voice, and the patience to make one concept interesting before moving to the next.
On substance, the work of Jay Alammar, Distill and Anthropic on interpretability made legible outside the labs the very mechanisms this site tries to explain in turn.
Credits
Built with Next.js, React, Tailwind CSS and MDX. The visualisations are written in plain SVG and CSS, with no charting library.
Typography: Inter and JetBrains Mono, served via next/font.