2 series · 11 pieces
Series
Multi-part essays: read in order, or jump to any part. Each series has a thread holding it together; the parts compound.
From code to weights:
ML fundamentals for engineers
You already know what a lookup table is, what a pipeline costs, what a type system buys you. Moving into ML engineering does not replace that knowledge; it relocates it. Each part takes one concept, explains it from the engineering side, reads one real file, and names the single thing that does not behave the way software behaves.
ML fundamentals
A model doesn't read text: what a tokenizer decides for you
The tokenizer ships inside the model, not in front of it. Reading tiktoken's BPE for what it fixes before any weight exists: vocabulary, cost, context limit.
ML fundamentals
An embedding is a lookup table, and everything else is how you fill it
A row of a matrix indexed by id. The science is how it gets filled; the engineering is that there is no schema migration: a new embedder invalidates your index.
ML fundamentals
Shapes are the type system of ML, and nobody checks them for you
[B, T, d] is a function signature. The runtime never verifies it, so a wrong axis does not raise: it broadcasts, returns plausible numbers, and ships.
ML fundamentals
Attention is a learned weighted average, and the cost is in the square
A soft dictionary whose weights the model computes from its own input. The idea is cheap; the weight matrix is T by T, so the cost comes from the data, not the code.
ML fundamentals
A transformer block is six tensors and a bus: read it like a pipeline
Six tensors on a residual stream, read line by line with shapes in the margin. Two thirds of the weights sit in the feed-forward; the cost sits in attention.
ML fundamentals
GPT-2 is the transformer plus five decisions, and inference is a different program
One artifact, two programs. Training is one parallel forward pass over the sequence; generation is a sequential loop with a cache. Same weights, different bugs.
ML fundamentals
From GPT-2 to LLaMA: four changes and the reason for each
Rotary positions, RMSNorm, gated feed-forward, grouped-query attention. Four local edits bought with cost or stability, not theory; none portable across weights.
ML fundamentals
Loss and gradients without the formulas: backprop is a graph walk
Backprop is a reverse-topological walk applying the chain rule. Read as code, it explains exploding and vanishing, and why the bug is a curve a thousand steps later.
ML fundamentals
The learning rate is the hyperparameter that breaks production
Adam and the schedule are engineering decisions, not mathematics. Nothing validates the config: a bad value diverges thirty hours in and takes the run with it.
ML fundamentals
Fine-tuning doesn't add knowledge: four training phases by the weights they touch
The phases differ in which weights move and on what data. Fine-tuning changes behaviour, not knowledge; a weight patch is not a diff, and nothing legible remains.
ML fundamentals
Serving is a memory problem: prefill, decode, and the KV-cache
Serving a model is memory bandwidth and scheduling, not modelling. A GPU at thirty per cent with a blown p99 is the normal case, and the reason is in the arithmetic.
ML fundamentals
At scale the memory decides: mixed precision and the three parallelisms
Scaling training is dividing memory, not compute. Mixed precision and the data, tensor and pipeline splits make the states fit; each GPU you add adds communication.
Reading the source:
how OSS tools judge LLM agents
Every LLM-as-judge tool fights the same thing: variance. Few say so out loud. I read the source of seven OSS evaluators: five strategies, straight from the code, not the docs. Each contains the variance a different way; each pays for it somewhere. The series ends where they all stop short: none gates a build on whether the judge still agrees with a human.
Evaluation methodology
Variance is the problem with LLM-as-judge, and tools don't agree on how to fix it
Reading the source code of seven OSS LLM-as-judge tools: variance manifests across four axes, and the seven approaches don't share a framework; they share a problem.
Evaluation methodology
Binary verdicts: how Ragas keeps judges honest
Ragas decomposes complex judgments into atomic yes/no questions. The trade-off: you trade information for variance reduction.
Evaluation methodology
From rubric to graph: how DeepEval splits a judgment
DeepEval encodes evaluation as a graph traversal. Each node is a simpler decision than the overall judgment.
Evaluation methodology
When judges show their uncertainty: DeepEval and ARES
DeepEval weights scores by token probabilities. ARES applies Prediction-Powered Inference for formal confidence intervals. Two routes to the same goal.
Evaluation methodology
The negative rubric: telling a judge what NOT to evaluate
TruLens uses rubrics that explicitly tell the judge what falls outside scope. The pattern is unusually explicit.
Evaluation methodology
Deterministic where you can, judge where you must
Tool calls deserve deterministic comparison. Goal completion needs LLM-based assessment. Some tools draw the line; others don't.
Evaluation methodology
Who calibrates the judge? The gap nothing fills
Six posts, five strategies, five tools. Two measure judge-vs-human agreement; none gates a CI build on it with a published confidence band. That's the gap.