World models - and what AgentWorldBench measures

Why does this sit here and not with the Tiny Agents? Because the interesting question about a world model is a benchmark question. What does the number above it actually say? A world model does not act at all. It plays the environment that gets acted in, which puts it in the same building block as AgentLens - with the question of how you judge an agent in the first place.

The inversion

An ordinary agent model gets a history and predicts the next action. Which tool, which arguments.

A world model gets the same history plus the action and predicts the next observation. What the server would have answered. What would have stood in the terminal. How the page looks after the click.

Same build, different role. The agent plays the actor, the world model plays the world. That is all it is.

Qwen-AgentWorld covers seven such environments. MCP, search, terminal, SWE, web, operating system, Android. Each has its own format in which action and observation are expressed, and the model is trained to answer in exactly that format.

The numbers, and what they do not say

On AgentWorldBench, the yardstick shipped with it, these are the values as published on 24/06/2026:

Those are third-party figures. The vendor published them itself, nobody re-measured them here. AgentLens measures something else and is no use as a cross-check.

More important than the ranking is what gets scored there at all. AgentWorldBench tests simulation quality. How close does the predicted observation come to the one a real system would have delivered? A high value means the model plays environment convincingly. About the ability to act it says nothing. A model that predicts the answer of a CRM server perfectly has not worked off a single job by doing so.

The gap of 0.46 points to the next general-purpose model is also narrower than a table makes it look. What is remarkable is rather that a specialised model keeps up with the big ones at all - and that the big ones are apparently quite decent world models on the side.

What you use it for

Practice environments nobody has to build. The strongest case, and the reason an endpoint like that stands on this platform. A scenario for a customer's industry is a text file instead of a system, and the larger of the two here holds 48 tools from eight domains, with no schema, no migration, no test data. Whoever wants to reset a world creates a new one.

Prediction as preview. Your agent can run a risky call against the world model first and look at the predicted observation before it really fires it off. Cheaper than a rollback. And more honest than a self-assessment.

Data generation, with a caveat. You can use histories from an emulated world as training material. What comes out of it is material about a fiction, though, and whoever trains a specialist on it has to test him somewhere other than in that same fiction. See below.

Where the limit runs

A world model invents environments the way a language model invents facts - plausible, format-true, unmarked. Three things follow from that.

It drifts. The longer a session runs, the more history has to go into the prompt, and at some point it no longer fits. That is not an error you configure away, that is the build. So the endpoint here records per call how much history still went in.

It has no truth. Two fresh worlds from the same scenario give the same firm two different customer numbers, and both answers look equally right. That is demonstrated in the piece The world, or a model of the world.

It cannot grade. A task counts as passed when a state has demonstrably come about. In an invented world there is none. What you hold in your hand afterwards is a conversation log that agrees with itself.

Out of that comes the rule that stands in the way in several places in this project: Practise in the invented world, measure in the real one. Measuring happens against Simhaven, where rows in a database sit behind the endpoint that a grader can read.

How it is built here

The endpoint /welt sits in the same MCP container as the Simhaven worlds and follows the same rules. Session token instead of operator token, a fresh server per session, its own scope. The scenarios are data (mcp/world/scenarios/*.json). Two of them, one large, one narrow.

Four caps bound a world. Calls per session, context length, timeout, session duration - every tool call is an LLM call with growing context, after all, and a forgotten loop would otherwise be a GPU bill. Counting happens before checking. Otherwise running at the cap would be free.

The model runs as its own small permanent service next to the large inference slot, with its own environment variables. The reason for that could be inspected on the machine itself on 06/09/2026. The large slot was empty that day, and every request to the house model came back with HTTP 500. An endpoint that has to stand reliably in a training day must not hang on something like that.

You can create a world at /weltmodell.