The Harness Is the Product

01Two years ago, the harness did not exist

That was said flatly, and nobody on the panel objected. As recently as the start of this year the category barely had a name. The open harnesses people now reach for by default are months old, not years.

One panelist offered a way to think about where this ends up. If all software eventually carries a dependency on a model, in the same way that all software today carries dependencies on libraries, then all software eventually carries a harness to work with that model. On that reading the harness is not a fashion of 2026. It is a permanent layer of the stack that happens to be arriving in public, in real time, with no specification written for it.

02A good harness can make a bad model a good model

This was the line of the night, and it inverts how most teams budget. A harness, as one panelist put it, gives a model superpowers, and it does that mostly by compensating for things models still cannot do. Memory was named as the most painful of those in the near term. A model has a hard ceiling on what it can hold. A harness can remember what you said yesterday, what you said last week, what a colleague on your team did, and what the previous query returned.

The practical consequence is worth stating plainly. The ceiling on your system’s quality is not set by the model you chose. A team waiting for the next frontier release to fix a quality problem is, more often than not, waiting for the wrong thing.

03Reinforcement learning no longer trains the model. It trains the harness

Reinforcement learning is now the dominant way agents get trained for specific applications. But what enters the training loop is no longer the model by itself. It is the model together with its memory, its tools and its orchestration loop, evaluated as a single unit.

This is a larger shift than it sounds. It means the artifact you are tuning is the harness. It also means you cannot tune it at all without an evaluation system good enough to score the whole assembly, repeatedly, with no human in the middle. Several panelists arrived at the same conclusion from different directions: the eval platform is not a quality function bolted on at the end, it is the precondition for improving anything.

04Portability is a bill, and evaluation is the invoice

No single foundation model holds the leaderboard on every metric for more than a few months. The panel treated this as settled and drew the engineering conclusion from it: design the pipeline so the foundation model is a parameter rather than a premise. One panelist described evaluating against at least two different foundation models on a continuous basis, specifically to detect whether a recipe is genuinely portable or has quietly overfitted to one vendor’s behaviour.

The part most teams skip is the price. Independence from any one model is bought entirely with investment in evaluation. Without a robust eval platform you can claim to be model-agnostic, but you cannot demonstrate it, which means you cannot act on it in the month the leaderboard changes.

05Push the model to the leaves and code the loop

There is a strong industry habit of putting a language model in charge of orchestration. More than one panelist had tried it and walked it back. One described hallucination rates falling as they moved the model out of the orchestration loop and down to the leaves, to the specific points where reasoning was actually required, with the rest of the loop written as ordinary code.

The rule of thumb offered was roughly eighty-twenty. Eighty per cent code: invariants, typed artifacts, interfaces, boundaries that cannot be crossed, and every decision committed to disk before the next step is permitted to run. The model handles the twenty per cent that needs judgment.

The clarifying example was deployment. If you are rolling software across data centres, that is a hard-coded orchestrated path with a rollback at the end of it. It requires no reasoning whatsoever. Putting a model in front of it buys nothing and costs you determinism.

06Take the evaluation out of the model wherever you can

The most repeated piece of practical advice in the room, and the one most at odds with current fashion. Where the evaluation can be made deterministic, make it deterministic: a rubric, a rule, a hard gate in continuous integration that no model gets a vote on. Generative use cases will need judges. Task-oriented use cases frequently do not, and succeed with ordinary test suites.

The framing was simply to go back and apply the best practices of software engineering first, and reach for a judge only when the thing being judged genuinely has no single right answer.

07Never let a model family grade its own homework

When a judge is unavoidable, the first rule came without hedging: it must be from a different model family. A different vendor, an adversarial family, at an absolute minimum a different version. The model generating the output must not be the model judging it. The failure mode is correlated error and self-reference bias, and it is invisible in your metrics precisely because both sides share it.

Two refinements followed. Where a question has several defensible answers, and the panel was clear that in generative work it often does, a single judge is the wrong instrument. Use several, or a meta-judge that evaluates their output.

And decompose the goal. Rather than judging whether a high-level output was correct, break it into subgoals and sub-subgoals. Two things happen. Some of those subgoals turn out to be verifiable outside the model loop entirely, in code. And when something does go wrong, the blast radius is scoped to one subgoal instead of the whole system.

08You cannot ask an agent not to. You have to make it impossible

Asked whether a non-bypassable policy gate exists, the panel’s immediate answer was that it does not. Not at the level of the model. What exists is a substrate you can build so that the question never reaches the model’s judgment in the first place.

One panelist described the architecture used for agents running unattended for days at a time on shared production infrastructure:

• Least privilege as the starting state. The agent’s container is permitted to do nothing.
• No ambient credentials. Identity is injected at call time, per invocation, by a proxy sitting in front of the agent.
• Separate identities for reading and for writing, scoped per subsystem.
• Every outbound call routed through an egress proxy with an explicit allow list.
• Invariants as the last gate, independent of anything the agent reasoned its way toward.

That last point is the one to carry out of the room. The invariant is not a policy the agent is asked to respect. It is a wall the agent cannot reach through, and it holds regardless of what the model concluded it ought to do.

A second panelist came at the same problem from the enterprise side and reached a parallel conclusion: observability is not sufficient. You need an agent watching the agent, making a security judgment on each tool call and each action, with every action validated and gated before it takes effect. Actions are the exciting part of this technology. They are also the part that cannot be rolled back.

09Stop sampling. The signal is in the steps you threw away

Sampled tracing works for conventional services because a thousand web requests look substantially alike, so a sample represents the population. Agents break that assumption. A single missed step can corrupt an entire reasoning chain, and the trace that would have told you is the one you discarded.

By several accounts, capturing full traces in real time is no longer the bottleneck. Analysing them in real time is. The work has moved to engines that can evaluate large trace volumes as they arrive, because postmortem analysis only tells you what went wrong after the user has already had the experience.

The companion finding: evaluate the intermediate steps, not only the input and the final output. Those middle steps carry more information about accuracy than either end of the chain, and in a multi-agent system they are the only way to work out which sub-agent made the mistake and what the root cause was.

10The chain of thought is a temporary gift

Nearly every verification and evaluation technique running in production today depends on reasoning a human can read. Latent reasoning removes it.

The mechanism, as one panelist explained it: when a model does ordinary chain-of-thought reasoning it repeatedly projects its hidden state out into language tokens and reads them back in, and that projection loses information. Latent reasoning skips the translation. The hidden state goes straight back into the model. It is faster, it loses less, and it is opaque.

The panel’s assessment was that this is close, and that the right time to design for it is now rather than on arrival. The prescription came as an image. You cannot see a black hole, but you can see the stars around it. So build systems with interim steps that cause real-world mutations, and track the mutations. Force the system to predict actions rather than tokens, then verify the actions.

Two counterweights were offered, and both deserve to be held alongside it. First, most enterprise use cases had outcome-oriented evaluation in place long before reasoning traces existed, and still do. If what you measure is whether the business outcome moved, opacity in the reasoning may cost you nothing at all. Second, the traces are a security asset and not only a debugging one. While you can read what a model is reasoning toward, you can intervene before a tool call that should not happen goes out. That capability disappears with the traces.