AI Agents: The Budget is The Design

Our agents returned with answers that were incorrect. Not subtly wrong: verdicts that a reviewer could see were built on partial pictures, comparables missing entries a first-year analyst would have found, a conviction score that read like it had stopped thinking halfway through. Our first thought, something must be wrong in the models; the prompt needs work; the agent isn't well trained.

So we traced the runs and the models were fine. The prompts were fine. The agents had been doing exactly what we built them to do, right up until the moment they ran out of something: out of turns, out of tokens, out of freshness, out of time. And here is the part that matters more than the failures themselves: most of them failed silently. The agent hit its ceiling, rounded off whatever it had, and delivered it in the same confident voice it uses when the work is actually done.

The lessons took us a while to fully diagnose and fully digest. Budgets are not just about cost-control. They are part of the agent's design, as load-bearing as the prompt and less examined than many other components in the stack. This essay is about what those budgets are, what happens when they are wrong, and why tuning them turned out to be some of the most consequential engineering we’ve done thus far.

Every agent lives on an allowance

Start with what a budget is doing there in the first place. In our working definition, an agent is a model that runs a loop with the resources it was given: tools it may call, data it may reach, turns it may take, money it may spend. The resources make the agent possible. The budgets make it governable, auditable, and improvable over time. An agent with no ceilings is an agent whose behavior nobody has priced, whose worst case is unbounded, and whose runs can't be compared, because no two of them operated under the same constraints.

But a budget's deeper function is epistemic, and the industry mostly hasn't noticed. Every budget is an answer, written in advance, to the question of how much evidence is enough. Give a research agent 15 turns, and you have declared that a defensible verdict in its domain costs about 15 rounds of looking. Give a different agent six turns with one audited escalation to 12, and you have declared what a focused answer costs, and what an unusually hard one is allowed to cost, and that anything beyond that needs a logged reason. These are core design parameters. They are judgments about diligence, encoded as numbers, and they deserve the same scrutiny as any other judgment in the system.

A budget is a critical judgment about how much evidence is enough.

The budgets an agent actually runs on

Every turn budget, input cap, output cap, timeout, and concurrency limit in our code was a number we chose, for good reasons, on the day the feature shipped. Ours were tuned to the data of the firms we had at the time. Then the firms grew. One client's portfolio now spans 3,483 documents, with single companies whose records run to megabytes; another arrived with a 2.8 gigabyte document folder. The budgets did not grow with them, because nothing forced anyone to re-decide a number that lives as a bare constant in one of 150 files. A budget that encoded good judgment at launch encoded a different judgment two quarters later, and nobody had made the second judgment. When people say budget, they usually mean the dollar cap. In our system and in most systems with AI Agents, there are actually several more to consider: Turn budgets cap the loop: how many think-act-observe cycles a run may take before it must stop. Ours range from 50 turns for comparable-company discovery and 40 for credit-agreement parsing, down through 15 for thesis assessment, to six for the fund oracle's first-pass deep read of a company, with one retry at 12. Two to four turns cover the classification-style tasks. The spread is the point: a turn budget encodes what a defensible answer costs in that discipline, and discovery really does cost more looking than classification. The deep read's six is the strictest in the fleet, and it works only because its instructions demand a clearly partial answer over a padded one.

Input budgets cap how much of a document the model gets to read. Ours run from 120,000 characters for board materials and investor updates, through 60,000 for credit agreements, down to 30,000 for legal documents and valuation-marks imports, and 3,500 for news articles. Every one of them was cutting silently. A legal document's later clauses, the back half of a long marks sheet: past the cap, the text was simply not there, and the model, which cannot miss what it never saw, read the beginning and reasoned as though it were the whole.

Output budgets cap how long the model's answer may be. Our de facto standard is 16,000 tokens, but the audit found about 25 calls capped at 2,000 tokens or less, one as low as 96. The dangerous ones are the calls that must return structured data: an email parser at 1,200 tokens, an attachment classifier at 2,000, a sender-matching model at 400. Truncated prose is degraded. Truncated JSON is unreadable, full stop: the cut does not weaken the answer, it deletes it. Worse, the model vendor's own guidance is not to set this cap at all, because it also counts the model's thinking, so a low cap can consume the entire allowance on reasoning and return nothing. Time budgets cap wall-clock. A valuation letter parse gets six minutes. Most jobs get five to eight, firm-wide jobs 12 with up to three attempts, background work 30, and a run with no completion after 60 minutes is marked abandoned. One of our earliest ceiling bugs lived here: a 30-second command timeout caused a publishing job to be retried after it had already half-succeeded, duplicating its entries. The timeout was innocent. The retry it triggered was not.

Concurrency budgets cap parallelism: six document reads at a time, valuation letters one at a time, three retries on a rate limit. The letters run one at a time because we learned the alternative empirically, when parallel calls hit the vendor's rate limits and 40 percent of a recovery batch failed. A concurrency budget is a treaty with a limit you don't own.

Escalation budgets cap the exceptions: one retry at double the allowance for the deep read, with the reason logged, because an exception that repeats without a record is not an exception. It is the new budget, set by nobody.

The silent ceiling is the dangerous one

The real story is what the agent does at the moment it hits a ceiling, because there are two possibilities, and the difference between them is the difference between a system you can trust and one you can't.

The honest path: the agent stops, and the stopping is a first-class alert. It reports that it ran out and what it ran out of, names what it did not get to, states its confidence accordingly, and the surface downstream shows the run as stopped rather than finished.

The silent path: the agent hits the ceiling and improvises a landing. It writes the verdict from what it has, rounds the confidence up to its usual register, and delivers. Nothing in the output distinguishes a completed analysis from an interrupted one, and that is the most dangerous artifact in applied AI: a confident partial answer. It reads exactly like a thorough one. It gets cited like one. And when it is wrong, the postmortem blames the model's judgment, when the judgment never got to finish.

A budget hit in silence turns an interrupted analysis into a confident one. Nothing downstream can tell the difference, and everything downstream builds on it.

This is why we now treat every ceiling as an instrumented event. A run records which budgets it touched, which it exhausted, and whether it completed or stopped. Exhaustion rates are tracked per agent and per budget type, because a ceiling that is hit constantly is not a safeguard. It is the operating condition, and an agent operating at its ceiling every day is an agent whose diligence you have quietly capped without deciding to.

Wait, why not uncap just everything?

The obvious escape from all of this is abundance. Token prices are falling, context windows are enormous and growing, and the models get cheaper by the quarter. So why ration at all? Give every agent 10 times the headroom it could plausibly need, let it read every document it wants for as long as it wants, and the whole tuning problem dissolves. It is a serious objection, and the honest first answer is that cost really is the weakest reason for ceilings. If budgets were only about money, uncapping would eventually be right. The first real reason is that accuracy does not rise with abundance. It rises, plateaus, and then falls. A context window is enormous and still a fixed denominator, and every marginal document competes with the ones that matter: models attend unevenly across long contexts, the middle goes dim, and a curated evidence block reliably beats the same block buried in 10 times its volume of adjacent material. Handing an agent everything it might want to read is not generosity. It is burying the signal and billing yourself for the shovel.

The second reason is what open-ended room does to judgment. An agent that may always take one more turn never has to commit. Verdicts drift toward the asymptote of one more look, cost and variance climb, and the trace becomes a wander no reviewer can follow and no two runs of which are comparable. Scarcity is also where prioritization becomes visible: an agent with six turns must decide what matters most, and that ranking sits in the trace where a person can audit it. An agent with unlimited turns never has to rank anything, and its diligence becomes unreadable precisely because it is unlimited.

The third reason is the decisive one, and September made it for us. Removing your own ceilings does not remove ceilings. The context window is a ceiling. The vendor's rate limit is a ceiling, as the letter recovery learned at 40 percent. The tool-result cap that arrived underneath us was a ceiling nobody in our building chose. Declining to set budgets does not produce an agent without limits. It produces an agent whose limits were set by physics, pricing pages, and another company's release schedule, at positions nobody chose and nobody instrumented, which is exactly the silent-failure machine this essay opened with.

There is no such thing as an uncapped agent. There is only a ceiling somebody chose and a ceiling nobody did.

Run the experiment on a person

If the architecture argument feels abstract, run it as a management experiment. Hand a strong analyst an assignment, and answer their three scoping questions the way an uncapped agent is answered. How much time should I spend? You decide. How long should the report be? You decide. When is it due? Whenever it's done. That isn't empowerment. You have transferred the scoping of the work to the person doing it and kept the grading for yourself, and the three questions were never logistics: time is a turn budget, length is a token budget, a deadline is a deadline. Scope is where the delegator writes down how much the work matters, and withholding it doesn't erase that judgment. It makes the most junior person in the exchange improvise it.

Now give the same unscoped assignment to two people. One returns four sharp pages in two days, the other 40 exhaustive pages in three weeks. Which is better? There's no answer, because they performed different assignments: the variable you needed held constant is the one you let float. Scale that to 30 agents running thousands of times a quarter, each run deciding its own diligence, and no two work products are comparable, no baseline exists, and no track record can accumulate, because a track record is a series of comparable trials and you have arranged to never run two. Nobody manages people this way, and not out of distrust: scope is what makes delegated work evaluable. Budgets are the act of remembering that about the machines. You decide is not a gift. It is a rubric you refused to write down, applied to work you will grade anyway.

What we are doing about it

The repair program follows directly from the audit, and each piece answers a specific failure above. First, one limits file: every budget in the platform becomes a named constant with a one-line reason and its source, in a single place, so that a number chosen for a small firm's data can be found and re-decided when a 3,483-document firm arrives, instead of hiding as a bare integer in one of 150 files. A budget you cannot find is a budget you cannot tune.

Second, no silent cuts anywhere. The announced-cut pattern that already protects the oracle becomes the standard for every reader: when text is trimmed, the model is told how much it saw and where the rest lives, and the tightest document readers move to chunked reading so the back half of a marks sheet exists at all. Third, the low output caps come off the structured-data calls, per the vendor's own guidance, because a cap that can silently convert a parser's answer into unreadable fragments is not a safeguard. Fourth, every ceiling hit becomes a recorded event: turn budget exhausted, input truncated, output cut, each surfaced in the fleet's health view, so a limit announces itself to us before a client does. And underneath all four, the practice the audit really demanded: budgets are parameters now, watched like parameters. Exhaustion rates per agent per budget type, read against accuracy, because the two explain each other. A budget set too tight shows up as ceilings hit while reviewers add findings the agent should have reached. A budget set too loose shows up as cost and wander past the knee with no accuracy gain. The right number sits just past that knee, it moves as the models and the firms move, and finding it once is worth little. The practice of re-finding it is the asset.

The evidence it’s working

This essay has argued that the problem with budgets was unmeasured claims, so the fixes should be held to their own standard. Here is the evidence from one production instance, in the first week after the repairs landed, with each claim carrying its number.

Prevention first. The gate tests now run against production code, and all 49 pass: 44 checking agent tool budgets, four checking voice-tool input schemas, and one that fails the build outright on any non-vision output cap below 4,096 tokens. That last test is the guarantee this essay's output-budget section was asking for: the class of failure where a parser's answer gets silently amputated can't return without breaking the build. The ceilings stopped being folklore in 150 files and became tested code.

Observability second. The limit-hits ledger went live on September 27 and recorded roughly 700 ceiling events from real use in its first week, which is 700 events that previously happened in silence. The repeat-pattern alerts are firing too: two reached my inbox this week, one for news turn budgets and one for governance document reads, each naming a ceiling that real work is leaning on. Ceilings now reach us before they reach a client, which inverts the discovery order this essay opened with. And the oracle's evidence budget is holding against the platform cap that started all of this: five tool results came in oversized during the window, the largest at 41,714 bytes, and each was trimmed to the 32-kilobyte budget with a note telling the model what was left out and which tool fetches the rest. Under the roughly 51-kilobyte silent cut, every time, with the cut announced instead of hidden.

Capacity third. This is where the before-and-after is starkest. Seventeen document readers now read in chunks, including the parsers for financial documents, cap tables, legal documents, and valuation marks, the exact readers whose 30,000-character caps were silently discarding the back halves of long documents. The financial-document window grew from 25,000 characters to 200,000, an eightfold expansion. And here is the detail we find most satisfying: the new window has already been reached, twice, by a single document of 560,663 characters. Both times, the reader reported its coverage instead of cutting silently. That pair of hits is the whole essay in one event. The point was never that agents stop hitting ceilings. It is that a ceiling hit stopped being a secret.

What this evidence is, and what it is not. These are receipts of prevention, observability, and honest behavior at the limits: the silent-failure class this essay described is structurally closed in this instance. What they are not, yet, is an accuracy comparison, because accuracy here gets graded the way everything on this platform gets graded, through resolved outcomes over time. The limit-hits ledger now makes a real baseline possible: review-edit rates and claim scores on the affected agents, before and after, once the coming quarters produce enough resolutions to say something honest. We would rather publish that number when reality has had time to produce it than dress this week's telemetry up as a verdict.

The cheapest capability upgrade we shipped

The industry's instinct, when an agent underperforms, is to reach for a bigger model, a longer prompt, a new framework. Our trace pointed somewhere less glamorous. The oracle that invented a figure was a strong agent reading 2 percent of a company. The parsers producing thin analyses were strong models reading the front third of long documents. Fixing the evidence budget, chunking the readers, lifting a handful of output caps, and making every exhaustion loud was the cheapest capability upgrade we shipped this year, and no part of it touched a model.

There is a broader point underneath, and it is where this essay has been heading. Everyone building agents is, whether they know it or not, running a firm of employees on expense accounts. The accounts were set on each employee's first day, by whoever hired them, and filed in 150 different drawers. The employees never mention them, the landlord changes the building's limits without notice, and when an account runs dry mid-assignment, the work comes back polished and incomplete, and the review meeting blames the employee. No one would run a human team this way. The budgets are where your agents actually live: what they may read, how long they may think, when they must stop, and whether anyone hears about it. Set them deliberately, write them down in one place, instrument them loudly, and revisit them on a schedule, because the alternative is a fleet whose diligence is capped by numbers nobody remembers choosing, on a floor that can move without asking.

Vantage is built on the conviction that an agent is only as honest as its budgets, and that a ceiling hit in silence is a lie told by architecture.

Conviction Made Citable.