The world, or a model of the world

This platform has two addresses that speak the same protocol. /sim-crm/<token> leads into the CRM of Simhaven, /welt/<token> into a practice world. An MCP client notices nothing. It connects, fetches the tool list, calls, gets JSON.

Behind the first address sit rows in a Postgres table. Behind the second sits nobody.

The same question twice

On 06/09/2026 both ran side by side in the local stack, and the question was each time the simplest one an agent can put to a customer system. "Show me the customers."

Simhaven, list_contacts, the first row of the answer:

{
  "id": "593a166b-05d8-41d2-8445-b964a738812b",
  "first_name": "Henrike",
  "last_name": "Bardenhagen",
  "company": "Deichblick Verlag",
  "email": "[email protected]",
  "status": "unbekannt",
  "stand": "2024-05-08T09:00:00.000Z"
}

The practice world, scenario "Elektro Rauner", kunden_suchen with the search term Hausverwaltung:

{
  "ok": true,
  "gesamt": 38,
  "treffer": [
    {
      "kundennummer": "K-1051",
      "name": "Hausverwaltung Schäfer & Partner",
      "ort": "Friedberg",
      "telefon": "06031-72445",
      "email": "[email protected]",
      "zahlungsziel_tage": 30
    }
  ],
  "_emuliert": true
}

The first one is a piece of information. The second is a prediction about the information a system like that would probably give. Can you see it? Probably not, and your agent cannot either. Apart from the _emuliert: true the endpoint attaches voluntarily, nothing separates the two answers that anyone in production could read.

The test that does not work

The obvious test is to ask again. Whoever makes something up makes up something else the second time round.

Three identical calls in a row, same world. Out came three character-identical answers. No wobble, no new phone number, the same five property managers in the same order. The reason sits in the build. Every call travels to the model together with the session history so far, and because the model finds its own earlier answer in there, the world has state without anyone storing any. A contradiction would stand out. So none appears.

The test that works

Two fresh worlds from the same scenario, the first call in each, the same search term. Both answer with gesamt: 38. Both know Kliem and Trienekens. And then:

The same firm carries K-1156 in one world and K-1051 in the other, and both answers turn up with the same matter-of-factness. A customer number is not an opinion. Here it is one.

Where the seam runs

The interesting part is where both worlds agreed. The 38 property managers, Kliem as K-1077, Trienekens as K-1042 - that is how it stands in the scenario text the model gets in its system prompt. What is anchored holds. Everything else gets invented, in the same line, in the same format, unmarked.

Which is why eyeballing is no test here. The seam between fixed and invented runs straight through a neatly formatted JSON object. Behind it works the same machinery as in the confident hallucination, just one level up. What gets invented this time is the environment's response.

It still has manners

One detail nobody expects. The call kunde_lesen with the customer number K-4711, which never came up in that world:

{ "ok": false, "fehler": "Kunde K-4711 existiert nicht.", "_emuliert": true }

Asked twice, the same wording twice. So the practice world does not invent wildly, it invents in a disciplined way, with error cases, with format rules, with memory, and because it does that so tidily it is useful for practising and dangerous for measuring for exactly the same reason: a sloppy dummy you would see through in five minutes. This one you would not.

Three consequences that hang together

A benchmark without real execution measures storytelling. If the environment itself predicts what follows an action, it can hardly contradict the agent. It is drawn from the same distribution as he is. What comes out is a statement about plausibility and none about effect.

An agent must not grade its own success report. It has the same pull towards a round ending. Anyone who accepts "task completed" as evidence has not obtained a judgement. They have obtained one more sentence.

Fine-tuning on invented tool answers confirms itself. The model learns what another model guesses at, and is then tested inside the same fiction. It can never show up. In practice it makes itself felt on the day the finished specialist meets a real system and does not know its error messages.

The rule

Practise in the invented world, measure in the real one.

We need both. Where the line runs should still be nobody's memory exercise, which is why in this project it sits in the code in three places: every emulated call carries _emuliert: true, the log has a column emulated, and every tool says so to the client before it calls for the first time. Why threefold? Because a footnote can be read past. A column travels along, even into an export.

Want to try the practice world yourself? At /weltmodell you create one, along with the address for your own harness. What a world model is, how good the current ones are and what the numbers about them actually measure is in the Benchmarks building block under World models.