Hugging Face ❤️ TorchTPU: TPUs, Natively

If you ever tried to run PyTorch on a TPU, you probably ported your code to torch_xla, or you ended up having to learn JAX. With TorchTPU , a native PyTorch backend, TPU is now just another device. It opens up native support of TPU in the Hugging Face ecosystem. Transformers, Diffusers and Kernels run natively on TPU. Pick a model from the Hub, send it to "tpu", and you're done.

Wait, what's a TPU? 🤔

TPU stands for Tensor Processing Unit . TPUs are Google's own AI accelerators. They are chips built for one job: the matrix math that neural networks spend most of their time on. Google has been designing them for about ten years, and uses them to train and serve its own models, including Gemini.

TPUs are available on Google Cloud , from a single chip up to pods of thousands of chips. They shine when you need a lot of compute, for training or for serving at scale.

The catch has always been software. The TPU stack grew up around TensorFlow and JAX, and PyTorch users had to work harder to get in.

The old way

Until now, running PyTorch on TPUs meant going through PyTorch/XLA (torch_xla). It worked, but it had its quirks. Tensors were lazy (the tensors, not you), so your code didn't always run when you thought it did. Small changes could trigger graph recompilations, and debugging was hard to reason about.

To make life easier, we built Optimum TPU. It gave Hugging Face models inference and training support on TPUs. Now that TorchTPU brings native support, Optimum TPU is not required anymore to run Hugging Face models with Transformers and Diffusers.

Enter TorchTPU 🚀

TorchTPU adds TPU as a new backend to PyTorch. You write regular PyTorch code. It comes with Pallas kernels tuned for TPUs, support for distributed training, and a torch.compile integration for when you want more speed. Installing it on a TPU VM takes one command:

pip install torch-tpu --extra-index-url https://download.pytorch.org/whl/cpu

The extra index gives you the CPU build of torch. Without it, pip pulls the much bigger CUDA build, which you don't need on a TPU.

Your favourite libraries, now on TPU

Transformers

Transformers v5 was built around one idea: be the source of truth for model definitions, so they can run anywhere. "Anywhere" now includes TPUs. To load a model on a TPU, pass device_map="tpu". That's it.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen3.5-9B" model = AutoModelForCausalLM.from_pretrained( model_id, device_map= "tpu" , # will load model directly on the TPU ) tokenizer = AutoTokenizer.from_pretrained(model_id) inputs = tokenizer( "The future of AI is" , return_tensors= "pt" ).to( "tpu" ) out = model.generate(**inputs, max_new_tokens= 64 ) print (tokenizer.decode(out[ 0 ], skip_special_tokens= True ))

The model runs in eager mode by default. You can also use torch.compile for better performance, or run it in a distributed setup.

Images? Videos? Diffusers paints them on TPUs 🎨

TPU is now a first-class device in Diffusers. For prototyping or debugging, eager mode only needs pipe.to("tpu"). For deployment, compiled mode adds one compile call. Your pipelines and code stay the same, and there's no TPU fork to maintain.

import torch from diffusers import StableDiffusion3Pipeline

pipe = StableDiffusion3Pipeline.from_pretrained( "stabilityai/stable-diffusion-3.5-medium" , dtype=torch.bfloat16 ) pipe.to( "tpu" ) # eager mode: that's the whole change # optional: compiled mode pipe.transformer. compile (backend= "tpu" , fullgraph= True , dynamic= False )

image = pipe( "a corgi astronaut planting a flag on a pizza moon" ).images[ 0 ]

The first call compiles, and every call after that with the same image size reuses the graph.