Build your own tiny agents system

An agent with five tools works. An agent with two hundred does not - and for two entirely different reasons that keep getting lumped together. The Tiny Agents building block explains both reasons and the architecture that separates them. This course builds it.

By the end you have a directory tiny-agents/ in which a concierge decides without a single LLM call who handles a request, and a runner that starts the chosen agent with a fresh context, its own one to five tools and a budget that is not up for negotiation. Plus an artifact store that turns 38,000 tokens of scraped text into a digest of 800 without throwing the rest away.

The rule: no framework

One rule holds the course together, and it is the same one as in Build your own PyTorch: don't import it, build it.

Allowed are an HTTP client, a YAML parser and NumPy. So httpx (or requests), pyyaml, numpy. Add tiktoken for counting if you like - that is bookkeeping, not architecture.

Forbidden are agent frameworks of any kind: LangChain, LlamaIndex, CrewAI, AutoGen, smolagents, the Agents SDK. They would take exactly the three parts off your hands that this course is about: routing, context management, budget enforcement.

Explicitly not forbidden, but not needed: a vector database. The router in this course keeps its examples in a Python list and computes cosine similarity with a NumPy one-liner. For a few hundred examples that is not the budget version but the honest one: you get to see that Qdrant is an optimisation here and not a building block. When it starts to pay off is covered at the end of chapter 03.

What you need

Python 3.11 or newer. Everything else is one pip install line.

An OpenAI-compatible endpoint. This is the only external dependency of the course, and it is deliberately the weakest one possible: anything that speaks /v1/chat/completions and /v1/embeddings will do. Locally via Ollama, rented via OpenRouter, self-hosted via vLLM or LiteLLM. The course never talks to a vendor SDK, only to two URLs.

export TINY_BASE_URL="http://localhost:11434/v1"   # Ollama locally
export TINY_API_KEY="ollama"                       # Ollama ignores it, others do not
export TINY_CHAT_MODEL="qwen3:8b"
export TINY_EMBED_MODEL="bge-m3"

A chat model that can call tools. From chapter 05 onwards you need tools in the request. Small models are often worse at this than their model card claims - if your runner counts ghost calls by the dozen, that is a model finding first and not a bug in your code. Which is precisely why chapter 05 counts them at all.

A multilingual embedding model. This is the one place where the wrong choice ruins the whole course: an English-leaning model such as all-MiniLM-L6-v2 collapses when routing German queries, and you will spend hours looking for the fault in your threshold. The recommendation is bge-m3 - it runs locally in Ollama (ollama pull bge-m3) and is available at most providers too. The alternative is multilingual-e5-large, which expects prefixes (query: before the query, passage: before the example). Forget them and you lose retrieval quality without anything visibly breaking - one of the nastier classes of bug.

The layout

By the end of chapter 08 your repo looks like this. It grows there chapter by chapter, so for now you only create tiny-agents/ and tiny/:

tiny-agents/                ← your repo
├── registry/               ← the cards, from chapter 02
│   ├── web-reader.card.yaml
│   ├── crm-writer.card.yaml
│   └── docs-finder.card.yaml
├── tiny/                   ← the package
│   ├── __init__.py         (empty)
│   ├── llm.py              (03) OpenAI-compatible client
│   ├── cards.py            (02) format and validator
│   ├── index.py            (03) embeddings and cosine
│   ├── concierge.py        (03, 04) the router
│   ├── tools.py            (05) the tool registry
│   ├── runner.py           (05, 06) invocation loop
│   └── artifacts.py        (07) handle and expand()
└── scripts/
    ├── 01_token_vergleich.py
    ├── 03_router_eval.py
    └── 08_abc.py

Every chapter ends with a working state. Not with a green test, but with a command you type and an output you read: a number, a ranking, a refusal. That is deliberate - this course builds a system whose entire value hangs on numbers you do not know in advance.

What this course deliberately does not build

The prototype, not the production system. Three things a real tiny agents system needs do not appear here, and the end of chapter 08 says so again at length:

• Traces in a database. The course logs to stdout and to a JSONL file. Anyone who wants to evaluate routing decisions over weeks needs Postgres.
• Learned routing. Writing confirmed routings back into the index is phase 2 - and impossible without traces anyway.
• Gap clustering. That every abstain is the roadmap for the next agent is something you build as a log line in chapter 04. Grouping those lines automatically is a small project of its own.

The subject matter behind all of this lives in the seven chapters of the Tiny Agents building block, from "Two problems, two magnitudes" to "What it costs". You can take the course without them. If you want to know why a decision goes one way and not the other, the reasoning is there.