Trust, but Trace

2 series · 11 pieces

Series

Multi-part essays: read in order, or jump to any part. Each series has a thread holding it together; the parts compound.

Series

From code to weights:
ML fundamentals for engineers

You already know what a lookup table is, what a pipeline costs, what a type system buys you. Moving into ML engineering does not replace that knowledge; it relocates it. Each part takes one concept, explains it from the engineering side, reads one real file, and names the single thing that does not behave the way software behaves.

4 of 12 published
Part 1/12

ML fundamentals

A model doesn't read text: what a tokenizer decides for you

The tokenizer ships inside the model, not in front of it. Reading tiktoken's BPE for what it fixes before any weight exists: vocabulary, cost, context limit.

9 min
Part 2/12

ML fundamentals

An embedding is a lookup table, and everything else is how you fill it

A row of a matrix indexed by id. The science is how it gets filled; the engineering is that there is no schema migration: a new embedder invalidates your index.

11 min
Part 3/12

ML fundamentals

Shapes are the type system of ML, and nobody checks them for you

[B, T, d] is a function signature. The runtime never verifies it, so a wrong axis does not raise: it broadcasts, returns plausible numbers, and ships.

12 min
Part 4/12

ML fundamentals

Attention is a learned weighted average, and the cost is in the square

A soft dictionary whose weights the model computes from its own input. The idea is cheap; the weight matrix is T by T, so the cost comes from the data, not the code.

Coming
Part 5/12

ML fundamentals

A transformer block is six tensors and a bus: read it like a pipeline

Six tensors on a residual stream, read line by line with shapes in the margin. Two thirds of the weights sit in the feed-forward; the cost sits in attention.

Available soon
Part 6/12

ML fundamentals

GPT-2 is the transformer plus five decisions, and inference is a different program

One artifact, two programs. Training is one parallel forward pass over the sequence; generation is a sequential loop with a cache. Same weights, different bugs.

Available soon
Part 7/12

ML fundamentals

From GPT-2 to LLaMA: four changes and the reason for each

Rotary positions, RMSNorm, gated feed-forward, grouped-query attention. Four local edits bought with cost or stability, not theory; none portable across weights.

Available soon
Part 8/12

ML fundamentals

Loss and gradients without the formulas: backprop is a graph walk

Backprop is a reverse-topological walk applying the chain rule. Read as code, it explains exploding and vanishing, and why the bug is a curve a thousand steps later.

Available soon
Part 9/12

ML fundamentals

The learning rate is the hyperparameter that breaks production

Adam and the schedule are engineering decisions, not mathematics. Nothing validates the config: a bad value diverges thirty hours in and takes the run with it.

Available soon
Part 10/12

ML fundamentals

Fine-tuning doesn't add knowledge: four training phases by the weights they touch

The phases differ in which weights move and on what data. Fine-tuning changes behaviour, not knowledge; a weight patch is not a diff, and nothing legible remains.

Available soon
Part 11/12

ML fundamentals

Serving is a memory problem: prefill, decode, and the KV-cache

Serving a model is memory bandwidth and scheduling, not modelling. A GPU at thirty per cent with a blown p99 is the normal case, and the reason is in the arithmetic.

Available soon
Part 12/12

ML fundamentals

At scale the memory decides: mixed precision and the three parallelisms

Scaling training is dividing memory, not compute. Mixed precision and the data, tensor and pipeline splits make the states fit; each GPU you add adds communication.

Available soon
Series

Reading the source:
how OSS tools judge LLM agents

Every LLM-as-judge tool fights the same thing: variance. Few say so out loud. I read the source of seven OSS evaluators: five strategies, straight from the code, not the docs. Each contains the variance a different way; each pays for it somewhere. The series ends where they all stop short: none gates a build on whether the judge still agrees with a human.

7 of 7 published
Part 1/7

Evaluation methodology

Variance is the problem with LLM-as-judge, and tools don't agree on how to fix it

Reading the source code of seven OSS LLM-as-judge tools: variance manifests across four axes, and the seven approaches don't share a framework; they share a problem.

6 min
Part 2/7

Evaluation methodology

Binary verdicts: how Ragas keeps judges honest

Ragas decomposes complex judgments into atomic yes/no questions. The trade-off: you trade information for variance reduction.

8 min
Part 3/7

Evaluation methodology

From rubric to graph: how DeepEval splits a judgment

DeepEval encodes evaluation as a graph traversal. Each node is a simpler decision than the overall judgment.

10 min
Part 4/7

Evaluation methodology

When judges show their uncertainty: DeepEval and ARES

DeepEval weights scores by token probabilities. ARES applies Prediction-Powered Inference for formal confidence intervals. Two routes to the same goal.

9 min
Part 5/7

Evaluation methodology

The negative rubric: telling a judge what NOT to evaluate

TruLens uses rubrics that explicitly tell the judge what falls outside scope. The pattern is unusually explicit.

8 min
Part 6/7

Evaluation methodology

Deterministic where you can, judge where you must

Tool calls deserve deterministic comparison. Goal completion needs LLM-based assessment. Some tools draw the line; others don't.

9 min
Part 7/7

Evaluation methodology

Who calibrates the judge? The gap nothing fills

Six posts, five strategies, five tools. Two measure judge-vs-human agreement; none gates a CI build on it with a published confidence band. That's the gap.

11 min