Measure the problem

The usual way into this subject is a claim: "too many tools blow up the context". This course starts one step earlier, because in that form the claim is false. Tools do not blow up the context. Their outputs do.

That is not hair-splitting, it is the difference between two solutions. Whoever sees the first problem builds a filter. Whoever sees the second builds a boundary. And whoever only builds the filter has solved the cheaper of the two.

So we count first. This chapter ends with an output you will be quoting several times later on.

The goal

Two problems, two magnitudes. A tool definition costs roughly 200 tokens. A tool output costs, depending on the tool, a hundred to a thousand times that. As long as both are called "the context problem", you look for the fix in the wrong place.

The scaffolding

Create your repo. Two levels, because tiny-agents is both the project and the package inside it - the same layout as any Python package:

tiny-agents/                ← your repo
├── tiny/                   ← the package
│   └── __init__.py         (empty)
└── scripts/
    └── 01_token_vergleich.py

mkdir -p tiny-agents/tiny tiny-agents/scripts
cd tiny-agents
touch tiny/__init__.py
python -m venv .venv && source .venv/bin/activate
pip install httpx pyyaml numpy tiktoken

tiktoken is the counter. It is calibrated on OpenAI models and yours is probably a different one - so the numbers are accurate to within a few percent, not exact. For a factor of 100 that is more than enough, and the alternative would be tracking down the right tokenizer for every model. Say so in the output and the number stops lying.

Step 1: what a tool definition really costs

A definition is the JSON schema that ends up in the tools field of the request. Not the call, not the answer - the pure description that rides along in every single turn.

# scripts/01_token_vergleich.py
import json
import tiktoken

ENC = tiktoken.get_encoding("cl100k_base")


def tokens(text: str) -> int:
    return len(ENC.encode(text))


def tool_definition(i: int) -> dict:
    """A realistically verbose definition, the kind a real tool has."""
    return {
        "type": "function",
        "function": {
            "name": f"crm_lookup_contact_{i}",
            "description": (
                "Looks up a contact in the CRM by name, email address or customer "
                "number and returns master data, the associated company and the "
                "most recent activities. Use this tool when a specific person is "
                "being asked about. Do not use it for company searches and not to "
                "create new contacts."
            ),
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "Search term: name, email or customer number.",
                    },
                    "limit": {
                        "type": "integer",
                        "description": "Maximum number of hits, default 5.",
                    },
                    "include_activities": {
                        "type": "boolean",
                        "description": "If true, the most recent activities are included.",
                    },
                },
                "required": ["query"],
            },
        },
    }


def cost_of_tool_list(n: int) -> int:
    schema = [tool_definition(i) for i in range(n)]
    return tokens(json.dumps(schema, ensure_ascii=False))

Call that for a single tool and look at the number before you read on. It sits somewhere between 150 and 250. That is the price of the model knowing this tool exists - regardless of whether it is ever used.

Step 2: what a single tool output costs

Now the other side. We fetch a real page and convert its text content into tokens. No library for that, just the standard library plus httpx:

# scripts/01_token_vergleich.py (continued)
import re
import httpx

DROP_TAGS = re.compile(r"<(script|style)[^>]*>.*?</\1>", re.S | re.I)
DROP_MARKUP = re.compile(r"<[^>]+>")
WHITESPACE = re.compile(r"\n{3,}")


def page_as_text(url: str) -> str:
    html = httpx.get(url, timeout=30, follow_redirects=True).text
    without_scripts = DROP_TAGS.sub(" ", html)
    text = DROP_MARKUP.sub(" ", without_scripts)
    return WHITESPACE.sub("\n\n", text).strip()


def cost_of_page(url: str) -> tuple[int, int]:
    raw = httpx.get(url, timeout=30, follow_redirects=True).text
    return tokens(raw), tokens(page_as_text(url))

Two numbers, because both occur in practice: some tools push raw HTML into the context (the bad case), good ones push extracted text (the normal case). The difference between them is the first and cheapest compression there is - and it still is not enough.

Step 3: put them side by side

# scripts/01_token_vergleich.py (end)
URL = "https://en.wikipedia.org/wiki/Tool"

def main() -> None:
    raw, text = cost_of_page(URL)
    print("Counter: tiktoken/cl100k_base (approximation, model-dependent +/- a few percent)\n")
    print(f"{'What':<44}{'Tokens':>10}")
    print("-" * 54)
    for n in (1, 5, 25, 50, 100, 200):
        print(f"{n:>3} tool definitions in the schema{'':<11}{cost_of_tool_list(n):>10}")
    print("-" * 54)
    print(f"{'ONE page, raw HTML':<44}{raw:>10}")
    print(f"{'ONE page, text extracted':<44}{text:>10}")
    print("-" * 54)
    print(f"Factor: one page equals {text / cost_of_tool_list(1):.0f} tool definitions.")


if __name__ == "__main__":
    main()

python scripts/01_token_vergleich.py

What you see

The output looks roughly like this (your numbers will differ, that is fine):

What                                            Tokens
------------------------------------------------------
  1 tool definitions in the schema                 193
  5 tool definitions in the schema                 957
 25 tool definitions in the schema                4772
 50 tool definitions in the schema                9542
100 tool definitions in the schema               19082
200 tool definitions in the schema               38162
------------------------------------------------------
ONE page, raw HTML                              104233
ONE page, text extracted                         21804
------------------------------------------------------
Factor: one page equals 113 tool definitions.

Three things are now established, and they carry the rest of the course.

First: two hundred tools are expensive but not absurd. With modern context windows, 38,000 tokens are annoying, not fatal. Which is exactly why dynamic filtering works as a solution to this problem - it turns 38,000 into about 1,000.

Second: a single page costs as much as a hundred tools. And unlike the tool list it is added per call. After three scrapes a 128k window is full, no matter how many tools were registered.

Third: filtering helps with the first line and changes nothing about the second. The five filtered tools then run in the same context the dialogue lives in, and dump their output right there.

If it looks different for you

If your page comes to only 2,000 tokens, you picked a small page. Try a product page with a price table, a PDF-to-text result, or an API response with a hundred records. The point is not the specific number but that the distribution is skewed: definitions are uniformly expensive, outputs are unpredictably expensive. The outlier is the problem, not the average.

Keep the script. In chapter 06 you use it to measure your digest, and in chapter 08 it is the basis of the whole measurement series.