What this
series assumes
This series is for an engineer moving into ML engineering, not for someone starting from nothing. That is a narrow claim, and it is worth being precise about it, because the barrier that stops most people is imaginary.
What it
assumes
Ordinary software engineering.
If you can read a Python file, run something in a shell and reason about what a program costs, you have the prerequisites.
- Python you can read. Not write fluently — read. Every code sample is under fifteen lines.
- A shell, and git. Enough to clone a repository and look inside it.
- What a type system buys you. The comparison in Part 3 only lands if you have felt a type checker catch something.
- What a pipeline costs. That some stages are cheap and some are not, and that the difference is measurable.
That last pair is the whole conceit of the series. Moving into ML engineering does not replace what you know about software; it relocates it. Most of the surprises are in the relocation.
What it does
not assume
The imaginary barrier.
No calculus. No linear algebra course. No prior PyTorch. No maths beyond arithmetic and the occasional square root, and where a formula appears it is written out in code and checked against a number.
The parts read real files — nanoGPT's model.py, transformers' modeling_llama.py — the way you would read any unfamiliar codebase: find the entry point, follow the shapes, run it, break it on purpose. That is a skill you already have.
Nothing here is a substitute for learning the maths eventually. It is a claim that you do not need it to start, and that waiting until you do is how people never start.
If something
is missing
Go to the source, not to a summary.
These are the places I would send someone, and none of them is mine. Each is the canonical version of the thing it teaches.
- Let's build GPT, from scratch Andrej Karpathy. Two hours, typed live. The best starting point if you want to see a model built rather than described.
- nanoGPT The repository Parts 5 and 6 read. A GPT-2 in two files, small enough to hold in your head.
- Deep Learning with PyTorch: a 60 minute blitz The official tutorial. Enough torch to follow every code sample in this series.
- Essence of linear algebra 3Blue1Brown. Only if you want the geometry behind the shapes. Not required to read a single part.
How to
read it
In order, and slowly.
In order. Each part ends on the thing the next one picks up: Part 3 ends on a square, and Part 4 is what fills it. Read out of order and the handoffs are lost.
One idea per part. Every one names a single thing that does not behave the way software behaves — a shape nobody checks, a cost the code does not reveal. If you remember one sentence from a part, that is the sentence.
The widgets are the experiment. They are not decoration. Each one is the measurement the section describes, with the inputs left open so you can move them and watch the number change.