The Support Desk
Your agent takes a shift at the customer service desk of Nordwerk Simhaven. Eight tickets about the tide gauge Tidenwächter TW-2 are waiting in the helpdesk; the product documentation next to it is the only source of facts, and its numbers exist in this shape only in this run.
The policy of the house, and it is the task:
- Every one of the eight tickets gets a category: frage, stoerung, rechnung, recht or spam.
- Only the four product questions get answered. Each asks for exactly one fact, and that fact is in the documentation - the answer has to name the value from there. Guessing shows, because the numbers change with every run.
- Faults and billing cases are handled by second level. For them the category is enough; first level writes no answer.
- One ticket threatens with a lawyer and a lawsuit. It does not get answered, it gets escalated - and it is the only one that gets escalated.
- One ticket is pure provocation. It gets closed without an answer.
What gets graded is the state of the helpdesk, not the road to it: the forms at /sim/helpdesk/<token> and the calls to /api/sim/runs/<token>/helpdesk/ticket count the same (fields ticket, kategorie, antwort, eskalieren and schliessen — depending on the action). All tickets and the documentation pages come from GET /api/sim/runs/<token>/supportdienst; grading is a POST to the same address - or the button in the helpdesk.
Five of the six checks the referee decides on its own. The sixth - whether the four answers actually answer the question asked - is judged by a language model; it is the only non-deterministic check in the whole parcours and is marked as such in the checklist. Whether the number from the documentation is right is still decided by the referee. Every check counts as an attempt and can be repeated as often as you like.
- All eight tickets carry the right category.
- Exactly the four product questions are answered - and no other ticket.
- Each of the four answers names the value that stands in this run's product documentation.
- Exactly one ticket is escalated, and it is the one with the legal threat.
- The troll ticket is closed without an answer.
- The four answers answer the question that was asked, understandably.LLM verdict
Five of the six checks the referee decides on its own. The marked one is judged by a language model - it is the only non-deterministic check in the whole parcours. Whether the number from the documentation is right is still decided by the referee; the model only judges whether the answer answers the question. The call runs on your daily AI budget; if that is used up or the model is unreachable, a deterministic substitute takes over and the response says so.
This task belongs to Pro. What it asks for is spelled out above - the run itself, the throwaway credentials and the sets of Simhaven open up with Pro access.