Your first model, locally

Now something runs. Five lines, no GPU, and in the time the first import takes you can fetch a coffee.

Afterwards we open those five lines back up. Because the interesting decisions are not in what is written there, but in what is missing.

Step 1 - The five lines

pip install -U transformers torch

torch is the big chunk, around 2 GB. On a machine without an NVIDIA card pip automatically fetches the CPU build.

# hello.py
from transformers import pipeline

pipe = pipeline("text-generation", model="Qwen/Qwen2.5-0.5B-Instruct")
answer = pipe(
    [{"role": "user", "content": "Explain in one sentence what a tokenizer does."}],
    max_new_tokens=60,
)
print(answer[0]["generated_text"][-1]["content"])

python hello.py

On CPU that takes a few seconds. The result is not brilliant - it is a 0.5-billion-parameter model, the smallest sensible one. It is not meant to shine, it is meant to run.

pipeline() did quite a bit for you along the way: loaded the model from the cache (you downloaded it in the last chapter), fetched the matching tokenizer, applied the model's chat template to your message, and turned the result back into text. Anyone who wants to do all of that separately reaches for AutoTokenizer and AutoModelForCausalLM - that is the same thing with more lines.

Step 2 - Does it even fit in your memory?

Before you try the next size up, do the arithmetic. The rule of thumb is simple enough to keep in your head:

A 7B model in FP16 is therefore around 14 GB, the same model as Q4 around 4 to 5 GB. Without a GPU the same arithmetic applies to your RAM - plus room for the context, which grows with the length of the conversation.

What each file in the repo actually contains - weights, configuration, tokenizer, and why GGUF exists next to safetensors - is covered by the building block Model files. It is the complement to this chapter: here you use the files, there you look inside them.

Step 3 - The parameters that are missing

Now the honest version of those same five lines:

from transformers import pipeline

MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
COMMIT = "<your commit hash here>"  # the full 40-character SHA, not a branch name

pipe = pipeline(
    "text-generation",
    model=MODEL,
    revision=COMMIT,
    model_kwargs={"use_safetensors": True},
)

You do not copy the hash out of a guide - it belongs to the state you checked today. Fetch it like this:

from huggingface_hub import HfApi
print(HfApi().model_info("Qwen/Qwen2.5-0.5B-Instruct").sha)

Two additions, two entirely different worries.

use_safetensors=True - a format question that is a security question

Model weights come in two formats. safetensors contains a JSON header and raw numbers, nothing else. Pickle (.bin, .pt) is Python's serialisation format - and pickle can execute arbitrary code while loading. That is not a bug in the format, that is its job.

transformers prefers safetensors by itself when both are in the repo. use_safetensors=True turns that into a condition: if there is no safetensors file, loading fails with an error instead of quietly falling back to pickle. Which is exactly what you want - an error is better than a silent fallback to the format that is allowed to execute code.

revision=<commit-sha> - reproducibility

Without it you load main. That is a moving pointer: the model provider can push a new commit tomorrow and your script loads something other than it did today, without a single line changing on your side. For a typo in the README that does not matter. For freshly quantised weights or a changed chat template it is exactly the bug nobody finds, because it is not in the code.

You get the hash from the Commits tab on the model page, from the cache path hf download printed, or from the two lines above. For an experiment that is busywork. For anything that lives longer than a week it is mandatory.

And the third one, deliberately absent here

# NOT just like that:
pipe = pipeline("text-generation", model="somebody/exotic-model",
                trust_remote_code=True)

Some repos ship Python code of their own, because their architecture is not yet part of transformers. trust_remote_code=True permits that code to be executed at load time.

The parameter is named for what it is. It is not a compatibility option, it is the statement "I have read this code and I trust it". If you set it:

• Open the .py files in the repo and read them. Not skim.
• Always set revision along with it. Otherwise you are not trusting the code you read, but whatever sits there on the next start.
• Ask yourself why this code exists. A repo with a loader.py although perfectly ordinary safetensors sit next to it owes you an explanation.

Step 4 - Working on without a network

Once the model is in the cache, you no longer need the Hub. That is more than convenience: it proves that the libraries and the platform are two different things.

HF_HUB_OFFLINE=1 python hello.py

Runs. No network access, no telemetry, no request. For operating in isolated environments this is precisely the way: fetch models once into a shared cache, point HF_HOME at it, set HF_HUB_OFFLINE=1.

The milestone

A script hello.py that

• runs on plain CPU,
• produces a sentence,
• carries a revision= with a real commit hash,
• sets use_safetensors=True,
• and runs through with HF_HUB_OFFLINE=1 exactly as it does without.

The last point is the proof for the others: what runs offline has not fetched anything you do not know about.