Field Notes № 01The Harness Is the Product

Nine things to do on Monday

Drawn directly from the findings in this issue. Your progress is remembered in this browser only, and is not visible to anyone else.

Write down which of your invariants a model currently gets a vote on. Move them out of its reach. Finding 08
Run your next evaluation against a second foundation model, even if you have no intention of switching. Finding 04
Take your three most expensive model calls down the triage ladder and see where they stop. The triage ladder
Confirm your judge is not from the same model family as the thing it judges. Finding 07
Decompose one high-level judge into subgoals, and find which of them become verifiable in code. Finding 07
Turn off sampling on one agent workflow and read the full traces for a week. Finding 09
Pick a metric you are proud of and ask which outcome it permits to get worse. Every model is looking for the Yes
Establish whether your agents hold ambient credentials or receive them per call. Finding 08
Name the one thing in your system that breaks the day chain-of-thought becomes unreadable. Finding 10