Topic · 3 pieces
ML fundamentals
The pieces a model is actually made of (tokenizers, embeddings, tensors, attention, training and serving), read from the engineering side, for people arriving from software or from data science.
← All writing0112 min → 0211 min → 039 min →
Shapes are the type system of ML, and nobody checks them for you
[B, T, d] is a function signature. The runtime never verifies it, so a wrong axis does not raise: it broadcasts, returns plausible numbers, and ships.
An embedding is a lookup table, and everything else is how you fill it
A row of a matrix indexed by id. The science is how it gets filled; the engineering is that there is no schema migration: a new embedder invalidates your index.
A model doesn't read text: what a tokenizer decides for you
The tokenizer ships inside the model, not in front of it. Reading tiktoken's BPE for what it fixes before any weight exists: vocabulary, cost, context limit.