AI Model Temperature: What it is and What it means for Judgment

Inside every large language model is a setting called temperature: a single number, usually between zero and one, that controls how much randomness the model uses when it chooses its next word.

Most AI platforms set it once, globally, and never speak of it again. At Vantage, we think that is a mistake. Every agent in our fleet carries a temperature chosen for its task, and those settings become inputs to a learning loop rather than constants in a config file.

What is AI model temperature?

At every step of generating text, an AI model doesn’t decide on one next word. It produces a probability distribution over its entire vocabulary. After "the capital of France is," the token for "Paris" might carry 95 percent of the probability, "located" 2 percent, "a" 1 percent, with thousands of others splitting the remainder. Temperature is a dial applied to that distribution before the model picks.

Low temperature sharpens the distribution. Likely tokens become more dominant, and at zero the model simply takes the single most probable token every time. The output becomes deterministic, or close to it: ask the same question twice, get essentially the same answer.

High temperature flattens the distribution. The gap between the likely and the unlikely narrows, so the model samples from a wider spread of plausible continuations. Same question twice, noticeably different answers.

The name comes from physics: In thermodynamics, temperature governs how much particles jitter around their most stable state: cold systems settle, hot systems explore. Token sampling uses the same equation.

Temperature doesn’t make a model smarter or dumber, and it doesn’t control creativity in any deep sense. What the model knows is the same at every setting. Temperature only changes how it rolls the dice when choosing each word.

A field guide to the scale

At 0, the model takes the single most probable token every time. Ask it to describe a company's quarter and you get the same sentences on every run. This is the setting for arithmetic, extraction, and anything an auditor might re-run.

At 0.1 to 0.2, the model is nearly deterministic but no longer frozen. Two runs agree on every fact and conclusion; a word choice or sentence boundary may differ. The output still behaves like a measurement.

At 0.3 to 0.5, the model writes like a careful professional with latitude. The substance is stable across runs, but the framing, emphasis, and connective tissue vary. Ask twice and you get two well-formed memos that agree with each other while reading differently. Most conversational AI products you have used run somewhere in this band.

At 0.7 to 1.0, variety becomes the product. The model reaches for less probable phrasings and less obvious angles, which is what you want when the task is imagining different futures and not what you want when the task is extracting the numbers from a filing. At 1.0 the model samples from its raw, unmodified distribution, every token drawn at exactly the probability the model assigned it.

Above 1.0, the dial keeps turning but the returns invert. The distribution flattens past the model's own judgment, tokens it rated unlikely for good reason start getting picked, and coherence decays: first the prose loosens, then the logic, then the grammar. By 2.0 the output is word salad. No serious analytical system runs here; the territory exists mostly to demonstrate why the dial matters.

The full dial runs from frozen to incoherent, and the entire analytical range lives in its bottom half. Within that half, the difference between 0.1 and 0.7 is the difference between a measurement and a brainstorm.

Why one setting cannot serve every task

For work with a verifiable right answer, like extraction, classification, computation, or reading a financial statement, randomness is pure liability. You want the model's best single judgment, reproducibly. Run the same board deck through the same parser twice and the extracted figures should match, byte for byte. A fund's auditor doesn’t want to hear that the pipeline produces different numbers on different days.

For work whose value lies in range, framing scenarios, surfacing angles, drafting narrative, some randomness is the point. The second-most-likely phrasing is often the more interesting one, and a model held at zero produces the same safe synthesis every time, which narrows what a human reader gets to consider.

So the question is not "what temperature should the AI run at?" but "what temperature does this task deserve?" A platform that runs everything at one setting is accepting noisier extraction, flatter writing, or a little of each.

How the Vantage fleet is tuned

Every reactive agent in the Vantage fleet, those that read, analyze, and report on each portfolio company, carries a temperature chosen for the kind of judgment it renders.

Verdict reads run near zero (0.1). Financial Analysis, the parsing pass of Deck Storyteller, and the Product and Technology Analyst all render verdicts on evidence: what the numbers are, what the deck claims, what the technology is. These are reads where two runs should agree, so they run nearly cold.

Reconstruction runs low (0.2). Company Timeline and House View reassemble what happened and what the firm concluded. The facts are fixed; the assembly requires slight judgment in ordering and emphasis, but not invention. Cold, with a degree of freedom.

Synthesis runs moderate (0.3 to 0.4). Company Profile, Peer Benchmarks, Predictive Signals, News Briefing, Valuation Insights, Talent Assessment, and the synthesis pass of Deck Storyteller weave many sources into narrative. Here phrasing, framing, and connection carry value, so these agents run warm enough to write well while staying anchored to cited evidence. The oracles sit in this band too: Pythia converses at 0.4, and the fund-level Narrative agent, whose job is prose, runs at 0.5.

Generation runs hot (0.7). The Liquidity agent runs fat above the rest, at 0.7 because it performs the most generative task in the fleet. Where every other agent produces a verdict, a classification, or a reconstruction of facts, the Liquidity agent writes. Its output is three distinct scenario narratives, downside, base, and upside, each a full prose paragraph painting a different future, delivered in partner voice: direct, specific, conviction-driven.

Run that task cold and you get flat, templated prose: three futures that read like one with the adjectives swapped. That defeats the purpose of scenario work, which is to make a reader feel how different each outcome would be. The heat is what produces that difference. The numbers underneath each scenario don't come from this agent at all; they arrive from the cold ones upstream. The 0.7 governs only the writing, and in this case the writing is the deliverable.

The closer an agent's output sits to a number an auditor might rely on, the colder it runs. The more its value depends on people weighing alternatives, the warmer. Choosing a temperature amounts to choosing what kind of claim an agent is allowed to make.

What the spread proves

Publishing a tuning doctrine invites a fair question: is this real engineering or a tidy story written after the fact?

The spread itself is the evidence. A platform that never asked the temperature question runs everything at one vendor default, and its settings, if you could see them, would be a flat line. Our agents run from 0.1 to 0.7, and each setting traces back to a reason about the task.

Verdicts are cold because auditors re-run them, reconstruction is low because facts are fixed, synthesis is moderate because framing carries value, scenario generation is hot because three futures that read alike are not three futures. When one agent sits far from the others, there is a reason you can read, and disagree with, and hold us to. That is the same principle that runs through the platform, the same reason every claim in Engram carries provenance: systems earn trust not by being uniform but by being inspectable.

Temperature as a learned parameter

Each of these settings were chosen by people, informed by the task and checked by evals. The next step, already underway, is to close the loop.

Vantage agents write their outputs back into the record: predictions with dates, assessments with citations, forecasts that are later joined by their resolutions. That means every agent builds a track record, which is what is needed to tune with. If the predictive agents' resolved forecasts show that a warmer setting surfaces signals a colder one misses, the setting should move. If a synthesis agent's citation-verification rate degrades above a threshold, it should cool. The same evidence decides larger questions than temperature: which underlying model each agent should pull from, as new model generations arrive and are benchmarked against its actual task.

This is the learning loop: settings that begin as engineering judgment become parameters adjusted by measured performance, per agent, per task, on the client's own work. Each agent's output becomes evidence for how it should be adjusted.

The point beneath the dial

Every degree of freedom an agent is given should be a decision someone made, stated clearly enough to be audited and instrumented well enough to be improved. Temperature is the easiest place to show that, because it's one dial with consequences you can see in the output. But the same standard governs how each agent's underlying model is selected, how its analysis is grounded in source material, and how every claim it makes traces back to that source.

As the learning loop matures, fewer of these decisions will rest on engineering judgment and more will rest on measured results. Either way, the reason behind each one stays on the record, where a client can read it, disagree with it, and hold us to it.

---

Vantage is the operating system for the modern venture firm: We put AI where it holds the asymmetric advantage and keep humans where they are irreplaceable. Book a demo with our team to see what it looks like.

Conviction Made Citable.