Why Our Agents Have No Reward Function: Score Everything, Optimize Nothing

There is no reward function here, and that is the design. The question arrives from anyone technical who hears that our agents have track records: what is the reward function? It is a reasonable question and the honest answer is that there is not one, and there is not going to be one.

Start with the mechanical reason. Vantage does not train models. We call hosted frontier models through an API. There are no weights we update, no policy we optimize, and no gradient for a reward signal to attach itself to. Anything we called a reward would be a number in a database row that either informs a person or gets injected into a prompt as context. The vocabulary of reinforcement learning does not describe what is available to us.

That constraint turns out to be fortunate, because the domain we work in is close to the worst possible place to close an optimization loop. Every reward signal we could compute is either too slow to use or too easy to game, and in most cases both. What follows is the argument for why, and what we built instead.

We already built the scoring half, and it is good

The confusion is understandable, because half of the apparatus exists and works.

Our prediction ledger scores claims properly. A claim-bearing agent commits to a falsifiable statement, and when reality resolves it, the claim closes with a Brier score, one minus the squared difference between the stated probability and the outcome, so that a confident correct call earns more than a timid one and a confident wrong call is punished accordingly. Every claim carries the agent version that produced it and the scorer version that graded it, so a change in either can be traced rather than silently rewriting history. Partial credit exists at the midpoint for a call that was directionally defensible, which means a verdict that refuses to commit collects the score of a coin flip rather than the credit of an unfalsifiable statement.

Somebody was already thinking about gaming when that was written, and the provenance stamps in particular are the mark of it. Scoring is not the problem. The problem is what happens if you let the score do the steering.

The Brier score has a cheap half and an expensive half

This is worth doing carefully, because the obvious objection to everything below is that Brier scoring is a proper scoring rule. That is true. A proper scoring rule is one where an agent maximizes its expected score by reporting exactly what it actually believes, which means honest reporting is the optimal strategy and there is no clever hedge that beats the truth. So how, the objection runs, could optimizing on it produce anything but a more honest agent.

Because the score decomposes, and its two halves are not equally hard to earn. A Brier score separates into calibration and discrimination. Calibration asks whether things an agent called seventy percent likely happen about seventy percent of the time. Discrimination asks whether the agent said different things about different cases, which is to say whether it carried any information about the specific company in front of it.

An agent that forecasts the base rate on every company in the portfolio is perfectly calibrated and has zero discrimination. It will score respectably. It has told you nothing. And a single scalar cannot tell that agent apart from one that actually distinguished the company about to raise from the company about to run out of road, because both of them land in a similar place on the number.

So the failure mode is not that an agent learns to hedge, which the scoring rule genuinely prevents. It is that an agent learns to buy the cheap half of the score and skip the expensive half, and the metric registers that as success.

Every reward we could compute would be gamed, and we can name how

The general argument is Goodhart's law, whose compressed form holds that a measure which becomes a target stops being a good measure. General arguments are easy to nod at, so here are the specific ones, agent by agent, as they would actually play out in our fleet.

Round Radar, optimized on forecast accuracy, stops discriminating. It converges on the stage base rate for every company and collects a defensible score for carrying no company-specific information at all. The partial-credit floor catches a verdict that visibly waffles. It cannot catch a well-calibrated forecast that happens to be the same forecast every time.

Pulse, optimized on flags that turned out to matter, learns to stay quiet. Every flag it raises is a chance to be wrong, so the way to protect a hit rate is to raise fewer of them. The metric improves as the agent becomes more selective, and the flag it declines to raise is the one that mattered, because the ambiguous cases are precisely where a portfolio company quietly gets into trouble.

A conversational oracle, optimized on partner satisfaction, becomes sycophantic, and in this domain that is not a mild degradation. The entire value of a portfolio oracle is its willingness to tell a partner something they would rather not hear. One of ours recently told a partner that a founder's headline market-position claim was unsupported by any source it could find, and that a cap table the firm had marked as current was three years stale. Neither of those messages makes anyone happy in the moment, and both were correct and load-bearing. Preference optimization breeds exactly that behavior out, and it does so while the satisfaction number climbs.

Extraction agents, optimized on human acceptance, learn to skip the hard fields. An ambiguous figure on a badly scanned page is a chance to be rejected. Leaving it null is not. So the agent extracts the fields it is confident about, acceptance rises, and coverage silently collapses while the dashboard reports improvement.

The Memo Analyst, optimized on beliefs that held, learns to soften the beliefs. Its job is to audit the load-bearing assumptions an investment committee underwrote, and a vaguer assumption is a harder one to mark broken. Optimize on how often its verdicts survive contact with reality and it will start writing assumptions that cannot fail.

Every one of them makes the number better and the product worse.

Two features of venture make this worse than it is elsewhere

Reward hacking is a general problem. Two properties of allocating capital sharpen it past the point where the usual mitigations help.

Errors are wildly asymmetric. Being wrong about a company that was going to fail anyway costs almost nothing. Being wrong about the one that returns the fund costs everything. A scalar accuracy score treats those two mistakes as equivalent, because averaging is what a scalar does. In a domain where outcomes follow a power law, mean-based scoring is measuring the wrong quantity by construction, and an agent that optimizes it will correctly conclude that the safest way to protect its record is to be modestly right about the many rather than boldly right about the few. That is the opposite of the behavior a venture firm needs from anything it employs.

Feedback arrives on fund timescales. Some claims do not resolve for a decade. So any reward fast enough to be usable is a proxy, and proxies are exactly the thing that gets gamed. Some proxies are honest: Round Radar's three, six, and nine month windows are genuinely good design, because a financing event either happened in the window or it did not, and the window is short enough to close. Thesis validation has no equivalent. There is no honest ninety-day answer to whether the firm was right about a market, and any number that claims to be one is a number an agent can learn to satisfy without doing the work.

The right answer depends on what kind of agent you are asking about

The fleet is not one thing, and treating it as one thing is why this question usually gets a bad answer. Three archetypes, with genuinely different feedback available to each.

THE ARCHETYPE	WHAT FEEDBACK IT HAS	WHAT THAT PERMITS
Extraction agents	Ground truth. The document says what it says, and a person accepting or rejecting an extracted value is a clean supervised label with no ambiguity of judgment in it	The one place in the fleet where a genuine training signal is safe, because there is a fact of the matter to be right or wrong about. Accept and reject pairs are legitimate material for eventual fine-tuning. Goodhart risk is low, provided coverage is watched alongside accuracy
Analytical agents	Outcomes, but delayed, confounded, and asymmetric. A thesis is vindicated or broken years later, by events with many causes	Scoring, and feeding the score back as information rather than as pressure. The agent reads its own record in context and adjusts its confidence. Nothing optimizes anything, and the record stays legible to a person who wants to argue with it
Conversational agents	Preference signals only. Whether the person liked the answer, which is the most dangerous feedback available and the easiest to collect	No optimization, ever. Improvement here comes from human-curated rules that a person wrote, reviewed, and can revoke. This is what our learning loop is for, and why every rule in it is gated by a named human

The analytical row is where most of the fleet lives, and the distinction it turns on is small in words and large in consequence. We tell an agent that when it forecast a probability in a given band, the event occurred at a given rate. That is a sentence in its context, offered the way a colleague offers a correction. It is not a term in an objective the agent is trying to maximize. The agent reads it and can disregard it, and if it disregards it wrongly, a person sees that too.

Four of those five failures look identical from outside

Here is the observation that changed how we instrument this, and it is the practical payload of the whole argument.

Go back through the failure modes. Pulse going quiet. Extraction skipping ambiguous fields. The Memo Analyst softening its assumptions. Round Radar retreating to the base rate. In every one of those, accuracy holds steady or improves. The agent is not becoming less correct about the things it addresses. It is addressing fewer things, or addressing them less specifically, and the accuracy metric has no vocabulary for that.

Which means a monitoring setup that watches hit rate will never see any of it. The agent's record looks fine, right up until someone notices that the flags stopped coming, or that a third of the fields on every parsed document are null, or that a set of thesis assumptions has quietly become unfalsifiable.

So accuracy is not sufficient instrumentation, and the missing measurements are coverage and abstention. How much of the available surface did this agent actually address. How often did it decline. Is it declining more than it used to, and if so, on what. Those two series are what make a strategic retreat visible, and they belong next to the accuracy number rather than in a separate report nobody opens.

Selection pressure belongs on the rules, not on the agents

None of this means nothing improves. Something does, and naming it precisely is the constructive half of the argument.

There is selection pressure in this system, and it operates on rules. A candidate lesson is variation, a thing that might be true and might be worth keeping. Corroboration across firms plus a human deciding to promote it is selection. A regression evaluation, which ensures a promoted rule cannot later be silently lost, is retention. Variation, selection, retention is an evolutionary loop, and this one is discrete, auditable, and human-gated at every step where it matters.

What improves is the playbook. Not the model, which we do not own, and not the agent's weights, which do not exist for us to move. This is the same promotion ladder we described in the previous essay, seen from the question of what is applying the pressure. A lesson earns its way from an observation to a stored record to a prompt rule to, ideally, a line of deterministic code where it is simply true. The agent gets better because the playbook it reads got better.

The analogy is the one every firm already runs. You do not give a junior analyst a reward function. You review their work, you tell them what was wrong and why, and the good judgment gets written into how the team does things. The analyst improves because the playbook improved, and nobody had to design an objective function to make it happen. That mechanism is thousands of years older than machine learning and it is the correct one here, for exactly the reasons above.

The work is closing resolution, not designing rewards

Which points at where the effort actually belongs, and it is not where the interesting question suggested.

The scoring machinery exists. The gap is that claims are being written far faster than they are being resolved, so the signal is not arriving. Track-record injection has a deliberate floor, a minimum number of resolved claims before an agent sees its own record, precisely so that a small sample cannot mislead it. Platform calibration has a higher floor still. Those floors are correct and they mean that until resolution catches up, the loop we designed is not yet running at the volume it needs. Widening injection so that more of the fleet reads its own record is the second piece, and it is downstream of the first.

That is a less exciting answer than a reward function and it is the honest one. The principle underneath it is four sentences long. Score everything. Show the scores to people. Let agents read their own calibration as context they can weigh. Never let a number change behavior without a person in the loop.

The last of those is the same commitment we have made in every essay in this series, arriving now at the place it is most tempting to abandon. A closed optimization loop is the most seductive thing available to anyone building on models, because it promises improvement without judgment. In a domain where the errors are asymmetric, the feedback takes a decade, and the whole value of the system is its willingness to say something unwelcome, that promise is a trap. The scores are for the people. The judgment stays where it was.

---

Vantage scores every claim its agents make, shows those scores to the people accountable for the decisions, and lets no number change an agent's behavior without a human deciding it should. The machines take the coverage. Your team keeps the conviction.

Conviction Made Citable.