Renting compute without owning a GPU

A model on your laptop is honest but slow. A GPU of your own is fast but expensive and idle most of the time. In between lies the road almost everyone takes: rent compute, by the second.

Hugging Face offers two very different things for this, and the difference matters enough to state up front.

Inference Providers is a gateway. You send a request to one address, and Hugging Face routes it to one of a dozen compute providers. Nothing runs at Hugging Face itself. You are billed for compute time consumed, with no markup.

Inference Endpoints is rented hardware of your own. One model, one machine, your choice of provider and region, billed by the hour - even while it does nothing.

The first is the prototyping road, the second the operations road.

Step 1 - Change one line

The gateway speaks the OpenAI protocol. If you have OpenAI code lying around somewhere, you need exactly one change: the base_url.

pip install -U openai
export HF_TOKEN=hf_...   # your token from chapter 02

# router.py
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
)

answer = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Name three advantages of Parquet over CSV."}],
)
print(answer.choices[0].message.content)

python router.py

A 120-billion-parameter model just did the computing for you, and your machine did nothing but make an HTTPS request.

Step 2 - Choosing the provider

It gets interesting at the model name. It can carry a suffix:

model="openai/gpt-oss-120b"          # the Hub's default
model="openai/gpt-oss-120b:groq"     # this provider, explicitly
model="openai/gpt-oss-120b:fastest"  # whichever is currently fastest

After the colon comes the provider. More than a dozen are connected - Cerebras, Groq, Together, Fireworks, Nebius, Novita, Replicate, Cohere, SambaNova, Scaleway and others, plus hf-inference, the in-house one. The model page lists on the right which providers serve that model.

This is more than a speed switch. It is where you decide where your data is processed - and therefore the transition to the next section.

Step 3 - Where does your data end up?

For anything involving personal data this is the only question that counts. There are three honest answers, and they differ a great deal.

Self-hosted. The model on your own hardware, with transformers, vLLM, llama.cpp or TGI. The data never leaves your infrastructure. That is the best position and the only one that needs no paperwork. The price: you look after everything.

Inference Endpoints in an EU region. Dedicated, managed machines with a choice of provider and region; the EU option is currently AWS eu-west-1 in Ireland. Certified, with a data processing agreement, private networking available. This is the usual compromise when you do not want to self-host.

Inference Providers. Your request is passed on to a partner, and that partner's privacy policy applies in addition to Hugging Face's. Here the provider choice helps: Scaleway is a French, European provider in the gateway.

model="…:scaleway"   # the same request, processed in Europe

One more thing is relevant for the EU AI Act: whoever deploys a model has different obligations from whoever builds it - but transparency about what you use is part of the job in both cases. The model card from chapter 01 is the instrument for that too.

Step 4 - When the prototype becomes operations

As soon as requests arrive regularly, the gateway becomes the wrong choice: no guaranteed capacity, no fixed provider, no regional commitment. That is when you take an Inference Endpoint.

That is a model on a machine that is yours for the duration. You pick the model, cloud provider, region and hardware; you get a URL of your own, likewise OpenAI-compatible. It starts at a few cents per hour for CPU; GPU tiers begin at around $0.50 per hour and reach far upwards (as of August 2026).

Its most important property is called scale to zero: if no request arrives for a while - 15 minutes by default - the endpoint scales down to zero replicas and stops costing anything. The next request wakes it, and that request waits for the cold start. For load profiles with gaps this is the difference between $40 and $400 a month; for a service where the first request has to be fast, it is the wrong setting.

Endpoints can also be managed from code - create, pause, scale - via huggingface_hub. That is the way to go when an endpoint belongs to a nightly batch run and should not exist during the day.

The decision aid

Which engine is a candidate for that and what distinguishes them is covered by the building block Inference engines.

The milestone

A script that asks the same question twice: once without a suffix, once with :scaleway or another explicitly chosen provider. Both answers are printed, together with the elapsed time.

import os, time
from openai import OpenAI

client = OpenAI(base_url="https://router.huggingface.co/v1",
                api_key=os.environ["HF_TOKEN"])

question = [{"role": "user", "content": "One sentence: what is scale to zero?"}]

for model in ["openai/gpt-oss-120b", "openai/gpt-oss-120b:fastest"]:
    start = time.time()
    r = client.chat.completions.create(model=model, messages=question)
    print(f"{model}: {time.time() - start:.1f}s")
    print(r.choices[0].message.content, "\\n")

The evidence is not the answer but the diff: between the two calls lies a suffix, and behind the suffix lies a different data centre.