Account, token and the CLI

From here on you need an account. It costs nothing, and the free tier is enough for everything in this course.

This chapter has three outcomes: an account, a token with exactly the rights you need, and a working command line - which is called something other than what almost every guide older than a year says.

Step 1 - Create the account

Register at huggingface.co/joinExternal - Opens in a new tab, confirm your email address, done. Your username is also your namespace: everything you publish later is called yourname/something. It can be changed afterwards, but then links break - pick one you want to keep.

Step 2 - Install the library

python -m venv .venv && source .venv/bin/activate
pip install -U huggingface_hub

On Windows the second half of the first line is .venv\\Scripts\\activate.

The package brings two things: the Python client (HfApi, snapshot_download and relatives) and the command-line tool.

Step 3 - The CLI is called hf

Check that it is there:

hf version
hf --help

The calls you need in this course all follow the same pattern:

Step 4 - A token that may do as little as possible

You create tokens at huggingface.co/settings/tokensExternal - Opens in a new tab. There are three kinds, and the order in the form tempts you into the wrong choice:

• read - may read everything you have access to, including private repos.
• write - may additionally write, everywhere, in every repo of yours.
• fine-grained - may do exactly what you tick, and only in the repos or organisations you select.

Take fine-grained. That is also the official recommendation for production use, and the reason is unspectacular: sooner or later a token ends up in a log, a screenshot or a commit by accident. What happens then depends entirely on what the token is allowed to do.

For this chapter and the following ones you need:

• Read access to contents of all public gated repos (in case you ever want a gated model)
• Write access to contents/settings - but only for your own namespace, and honestly only from chapter 04 onwards. Until then reading is enough.

Sign in:

hf auth login
# paste the token (the input stays invisible), then:
hf auth whoami
# → yourusername

The token then sits in ~/.cache/huggingface/token. In scripts and CI you do not take it from there but from the environment variable HF_TOKEN - then it is nowhere in your code.

Step 5 - The first download and where it lands

hf download Qwen/Qwen2.5-0.5B-Instruct

Around 1 GB, half a minute to two minutes depending on your connection. At the end the command prints a path, and it looks roughly like this:

~/.cache/huggingface/hub/models--Qwen--Qwen2.5-0.5B-Instruct/snapshots/<commit-sha>/

Three things are packed into that path:

1. ~/.cache/huggingface/hub is the central cache. Every library and every script on your machine uses the same one - a model is never fetched twice, no matter which project asks for it.
2. models--Org--Name is the encoded repo name. The double hyphens stand in for the slashes.
3. snapshots/<commit-sha> is the decisive part: the cache is organised by commits, not by version numbers. That is exactly why you can pin to a commit in chapter 03 at no cost.

If you want the cache somewhere else - on a bigger disk, in a shared directory on a server - set HF_HOME:

export HF_HOME=/data/huggingface
hf download Qwen/Qwen2.5-0.5B-Instruct

Everything moves there, including the token. On a server where several services need the same models, that is the first line you write.

Fetching only parts of a repo works too, and with large models that is the difference between 15 GB and 150 GB:

hf download Qwen/Qwen2.5-0.5B-Instruct --include "*.safetensors" "*.json"

Why that was faster than you expected

For a long time large files on the Hub lived in Git LFS. Since 23 May 2025, Xet is the default for new accounts and organisations - a different storage backend that cuts files into content-defined chunks and transfers only the chunks that changed.

In practice: when a model provider re-uploads an 8 GB weight file because something small changed, you do not download 8 GB again next time. You do not have to do anything for this - pip install -U huggingface_hub brings the hf_xet package along and it takes effect automatically. You only need to know about it when someone shows you a guide containing git lfs install: that is the old road, it still works, but it is no longer the fast one.

The milestone

Three lines, three pieces of evidence:

hf auth whoami
du -sh ~/.cache/huggingface/hub
ls ~/.cache/huggingface/hub

The first names your username. The second tells you how much space the cache occupies. The third shows the directory with the encoded repo name inside it.

If you can explain all three outputs - who you are, where the files live, how big they are - the chapter is done. Incidentally, this is also the answer to "why is my system disk full": the cache does not tidy itself up.