Building agents in ChatGPT - so why a platform?

The short answer

Yes, you can click an agent together in ChatGPT. You can give it an instruction, drop some files next to it, hook up a few actions and use it. That is not a compromise and not a beginner's path - for a whole class of tasks it is the right solution, and if you are done in an hour, you did nothing wrong.

There is only one thing at the end of that sentence: until it gets serious. Serious doesn't mean "big" here, it means something very concrete - the agent should do the same thing again tomorrow, it should do it while nobody is watching, and if it doesn't, somebody should notice. That is exactly where clicking things together stops carrying the weight, and not because the tool is bad, but because the questions that show up then aren't answered inside the tool at all.

This page tries not to assert that but to show it. Four cases familiar to anyone who has used a clicked-together agent for longer than a week - and for each one, the place on this platform that covers what's missing behind it.

What is genuinely good about it

The concession first, and it is meant seriously.

A custom GPT takes care of all the machinery that sits in the LLM call and the message history: the call, the record, building the context, executing tools. You write an instruction - technically a system prompt - and get a running agent for it. That is an enormous lever, and the same arithmetic holds for a finished agent you set up instead of building. We have one ourselves: Hermes is the reference agent a lot of the material here is demonstrated on.

The price is no surprise either: you don't just get the machinery for free, you get it hidden. As long as everything runs, that is an advantage. It stops being one the moment something doesn't run - because then you need some idea of where to look, and that part doesn't come with it.

Case 1: The agent forgets mid-run what it was supposed to do

The pattern is always the same. At the start the agent sticks to its instruction; twenty minutes into the work it drifts back into small talk, ignores formatting rules and answers the last message instead of the task. No error, no warning - it is simply somewhere else.

The reason isn't a mood, it is arithmetic. The model has no memory; what looks like memory is the message history, which the harness resends in full on every call. That list grows with every tool output, every attached document, every intermediate answer - the context window does not grow with it. At some point the actual instruction sits so far back that it competes with twenty pages of fresh material. What you are seeing is exactly that.

What's missing is called context management: deciding what travels along in each call and what does not. Summarise instead of appending, select instead of dragging everything along, restate the instruction instead of hoping. A clicked-together interface has no switch for this, because it isn't a setting, it is a design decision. It is covered under message history and - as soon as knowledge has to outlast a single run - under memory.

Case 2: A tool call fails and the agent carries on regardless

The second case is the most expensive one, because it doesn't look like a failure. The calendar query times out, the agent gets an empty response - and writes: "Nothing is scheduled for tomorrow." It isn't lying; it was never taught that empty and nothing are two different things. The result is a fluent, factually wrong statement at the head of a chain, and everything after it builds on that.

The place where this gets decided is not the model, it is the harness. The model only proposes a call - it is executed by the code around it, and that code also decides what comes back when it fails: a usable message, an abort, a second attempt with different parameters. An agent that never gets to see the error can't work around it either.

In production this turns into a volume problem: a single tool call almost always succeeds, twenty in a row almost never do, because the rates multiply. 95 percent per step is 36 percent after twenty steps. What you build against that - retries, intermediate checks, tools that return something usable on failure - is covered under tool calling in production. And everything the agent must not do in the first place belongs to guardrails.

Case 3: The same prompt returns something different today than yesterday

You change one line in the instruction because one case went badly. Afterwards that case goes better. Whether the other seven still work you don't know - you didn't look at them. Two weeks later a model is updated in the background, and the same instruction behaves differently. Nobody noticed, because there is nothing that could notice.

That is the difference between trying things and measuring them. An agent whose behaviour cannot be reproduced is not a reliable agent, it is a gamble with a good hit rate. Reproducibility starts at the LLM call - temperature, stop words and model version are settings, not constants of nature - and ends with a fixed set of test cases you run after every change. How to compare models and setups without fooling yourself is covered under benchmarks; what such a measurement looks like in practice you can play through on a single variable in the input-format lab.

Case 4: It runs with three documents and collapses at three hundred

The demo is convincing: three PDFs uploaded, questions asked, every answer lands. With three hundred documents the same agent answers more slowly, more expensively and worse - it cites the wrong paper, misses the right one and suddenly seems dumber than before.

With three documents everything simply fits into the context at once. Selection isn't a job then, it doesn't arise. With three hundred, selection is the only job: which five excerpts travel along for this one question? That is memory - search, selection, placement in the prompt - and it is the point where an upload turns into a system.

The same holds for tools instead of documents. Out of two hundred available capabilities the model only sees the ones you show it, and the outputs of the rest flood its context. Why selection is the actual construction work here is covered under tiny agents and the compression boundary.

The difference in one sentence

Clicked-together agents are good at working once. Everything beyond that - again tomorrow, unattended, demonstrably, with more material than in the demo - hangs on decisions made outside the tool: what travels along in the context? What happens when something fails? How do we notice it got worse? What must the agent never do?

That is four questions, not forty. They have names, they have known answers, and they are precisely the material this platform lays out in three blocks: the harness, everything that stands next to it, and what it all runs on.

Where to start

Not with a framework, and not by throwing your custom GPT away. Keep it - from here on it is your benchmark.

The cheapest first step is to see the machinery once yourself: a single LLM call, then a message history around it, then one tool and a loop. After that you read every finished product for what it is - a collection of defaults for exactly the questions above.

The fundamentals of every building block are free to read, without an account, and start at fifteen minutes. If you want to come in one step earlier: What is an AI agent? answers the question before this one - what an agent even is and which parts it is made of. You can build all of it along real code in the tutorials.