Trust, but Trace
Topic · 1 piece

Production ML

What it takes to make ML systems reliable in deployment — observability, replays, contracts, traces, and the unglamorous discipline around them.

← All writing
01
Evaluation methodology

Variance is the problem with LLM-as-judge — and tools don't agree on how to fix it

Reading the source code of seven OSS LLM-as-judge tools: variance manifests across four axes, and the seven approaches don't share a framework — they share a problem.

6 min →