Standard evaluations tell us whether an agent got the right result. For systems acting in changing physical environments, we also need to know whether they understood the world well enough to keep getting the right result.
Consider a warehouse robot approaching an intersection between two aisles.
It slows down before crossing. On its face, this is the correct behavior.
But there are several very different reasons it might have done so. It may have detected a worker just outside its direct path and predicted that the worker could enter the intersection. It may have noticed that one of its cameras is partially obstructed and treated the area as uncertain. Or it may have simply learned that slowing down at this visual pattern usually improves its reward, without representing why.
These differences may not matter in a familiar warehouse. They matter when the environment changes.
If the worker approaches from an unusual direction, if a shelf blocks the camera, if the route is rearranged, or if the robot's sensor degrades, a policy that looked safe under ordinary evaluation may fail immediately. The agent reached the correct outcome before, but not necessarily because it had an accurate and updateable understanding of the situation.
This is a central challenge for embodied AI.
Robots and other agents operating in the physical world do not act on complete information. Their observations are partial, their sensors are imperfect, and the environment changes as they act within it. A mistake early in a task can alter what the agent sees later. An assumption that was harmless at one moment can become dangerous several steps afterward.
For these systems, safety cannot be established only by observing behavior. It also depends on whether the agent can maintain a sufficiently accurate picture of the world, recognize what it does not know, and revise its understanding when reality contradicts it.
At Pelagic Platforms, we call this epistemic verification.
The gap in existing evaluation
Most AI evaluation is built around outcomes.
We ask whether a model answered a question correctly, whether an agent completed a browser task, whether a robot picked up an object, or whether a vehicle avoided an obstacle. These are useful measures. They tell us whether the system achieved a desired result on a particular distribution of tasks.
But an outcome alone does not tell us what the system understood.
An agent can succeed by using a shortcut that does not generalize. It can act correctly because the environment happens to match a familiar pattern. It can rely on stale or incomplete information without recognizing that its confidence is unjustified. It can pass a benchmark while losing track of a changing world.
This is not a hypothetical concern unique to robotics. In many agentic systems, an answer can look plausible while being based on incomplete retrieval, an incorrect assumption, or an unverified intermediate step. The resulting output may be useful in familiar cases, but it becomes unreliable as the task grows longer, the environment changes, or an initially hidden detail becomes important.
Embodied AI makes this problem more acute because incorrect assumptions have physical consequences.
A household robot may place an object safely on a counter without recognizing that the counter has been moved. A factory agent may navigate a route correctly without noticing that a machine's operating state has changed. A delivery robot may infer that a path is clear because it has not observed an obstruction, rather than because it has evidence that the path is clear.
These are different failure modes, but they share a common structure: the agent's behavior appears acceptable until the world stops resembling the setting in which that behavior was learned.
From behavioral data to epistemic data
Most training data teaches systems by example.
A text input maps to an answer. An image maps to a label. A video or pixel stream maps to an action. A browser page maps to a click. A prompt maps to a response. A tool call maps to an execution.
This data is necessary. It teaches an agent what successful behavior looks like.
But it usually does not record the information structure behind that behavior.
It does not tell us:
- What did the agent actually observe?
- What information was unavailable?
- What did the agent believe about the world?
- How certain was it?
- Which observations did its decision depend on?
- Did reality unfold as the agent expected?
- When it was wrong, did it update?
This is the missing layer that epistemic data is designed to capture.
Epistemic data records not only what an agent did, but the evidence available to it while it acted. It makes it possible to distinguish between an agent that acted correctly because it understood a situation and one that acted correctly for a reason that will not survive a change in conditions.
For embodied systems, this distinction is particularly important. The physical world is not a sequence of isolated examples. It is a partially observed process in which the state of the environment changes, information arrives over time, and early decisions affect later possibilities.
A system that cannot track these changes reliably may still appear capable on short tasks. Over longer horizons, its errors compound.
Why long-horizon interaction changes the problem
Many existing benchmarks evaluate an agent on a short sequence: observe a state, choose an action, and measure whether the result was correct.
Real-world embodied tasks are different.
A robot may need to remember where an object was last seen, infer whether another actor has moved it, recognize that a tool output is outdated, or decide that it does not have enough information to proceed. It may need to revise a plan because a previous action changed the environment in an unexpected way.
The critical safety question is therefore not only whether the robot acted safely at a particular step.
It is whether it maintained an accurate enough model of the evolving world to know when its next action is justified.
This is difficult to determine from trajectories alone. Two agents may take the same action and achieve the same outcome, while relying on very different internal assumptions. One may have noticed the relevant evidence. The other may have succeeded by coincidence.
The difference often appears only after the environment changes.
That is why evaluation needs to test more than task completion. It needs to test whether an agent can maintain, update, and act on an evidence-backed representation of a world that is incomplete and changing.
What epistemic verification measures
Epistemic verification begins by placing an agent in long-horizon interactive environments where observations are incomplete, the world state changes, and earlier mistakes can affect later decisions.
We then capture the execution trace available to the agent: its observations, memory, tool outputs, decisions, and the outcomes that follow. From this trace, we reconstruct a structured record of the agent's epistemic state.
This is not an attempt to claim direct access to every latent representation inside a model. Instead, it is an evidence-backed account of what the agent could have known at a given point in time, what it was justified in believing, and whether its subsequent actions were grounded in the right information.
The resulting questions are more demanding than a standard success metric:
- Did the agent notice the observation that mattered?
- Did it distinguish uncertainty from absence of risk?
- Did it preserve an accurate picture of the world over time?
- Did its actions rely on evidence that was still valid?
- When reality diverged from its expectations, did it update?
- Did that update generalize to related situations?
These questions make previously hidden failures visible.
An agent may succeed on a task while relying on a visual shortcut. It may act confidently after losing track of an object. It may treat missing information as evidence that nothing has changed. It may fail only after a small intervention exposes that its model of the world was never robust.
Standard behavioral evaluation can miss these cases because the final outcome looks acceptable.
Epistemic verification is designed to reveal them before deployment makes them consequential.
Measuring the intelligence beneath a score
A benchmark score tells us whether a model completed a specified task. It does not necessarily tell us whether the model has become more reliable in the ways that matter for deployment.
This becomes especially important when comparing successive versions of an agent.
A new model may improve on average task success while becoming worse at tracking uncertainty. It may complete more tasks but rely more often on brittle shortcuts. It may appear more decisive while becoming less sensitive to evidence that should change its plan.
Without an epistemic layer, these tradeoffs are difficult to see.
Epistemic evaluation can produce a more useful kind of model comparison: an intelligence diff.
Instead of asking only whether a model's aggregate score increased, we can ask:
- What did it learn to notice?
- Where did its world model become more accurate?
- Which kinds of uncertainty does it now handle better?
- Where did it become overconfident?
- Which assumptions remain unproven?
- Did it gain capability by improving understanding, or by exploiting a narrow feature of the evaluation?
This does not replace conventional benchmarks. It adds the information needed to interpret them.
Building systems that remain correctable
The longer-term importance of epistemic verification is not only better measurement. It is better correction.
Once an agent's intelligence gap is visible, it can be turned into something operational: a targeted test, a training example, a release condition, or a runtime intervention. The next model version can then be evaluated on whether it actually learned the missing capability, rather than simply achieving a higher headline score.
This matters for safety because advanced agents will increasingly operate in conditions their developers cannot fully anticipate.
They will encounter new layouts, tools, people, incentives, and environmental dynamics. No static dataset can contain every relevant situation in advance. No fixed set of behavioral rules can exhaustively specify what safe action looks like in every context.
A safe agent therefore needs more than a policy that behaves well under known conditions. It needs to preserve the ability to recognize when its assumptions no longer hold and to incorporate corrective evidence when the world proves it wrong.
That is what it means for a system to remain teachable.
An agent that performs well today but cannot meaningfully update tomorrow is not robustly safe. It is only well matched to the conditions that produced it.
Toward epistemic safety for embodied AI
The most important question in embodied AI is not simply whether a robot can act.
It is whether it can act while remaining grounded in reality.
As systems move from demonstrations into warehouses, factories, homes, laboratories, and public spaces, we will need stronger evidence than a successful trajectory or a high benchmark score. We will need to know what the agent observed, what it inferred, what it missed, how certain it was, and whether it can revise its understanding when conditions change.
That is the role of epistemic verification.
It gives us a way to test whether an agent is operating from an accurate and updateable picture of the world, rather than merely producing behavior that happens to look correct.
For embodied AI, this is not a secondary measurement problem.
It is a safety requirement.
The systems we deploy in the physical world must not only act safely under the conditions we tested. They must remain able to learn when those conditions no longer hold.