# Optimum for Intel® Gaudi® AI Accelerator

Optimum for Intel Gaudi AI accelerator is the interface between Hugging Face libraries (Transformers, Diffusers, Accelerate,...) and [Intel Gaudi AI Accelerators (HPUs)](https://docs.habana.ai/en/latest/index.html).
It provides a set of tools that enable easy model loading, training and inference on single- and multi-HPU settings for various downstream tasks as shown in the table below.

  
    Tutorials
      Learn the basics and become familiar with training transformers on HPUs with 🤗 Optimum. Start here if you are using 🤗 Optimum for Intel Gaudi for the first time!
    
    How-to guides
      Practical guides to help you achieve a specific goal. Take a look at these guides to learn how to use 🤗 Optimum for Intel Gaudi to solve real-world problems.
    
  

The Intel Gaudi AI accelerator family currently includes three product generations:
[Intel Gaudi 1](https://habana.ai/products/gaudi/),
[Intel Gaudi 2](https://habana.ai/products/gaudi2/), and
[Intel Gaudi 3](https://habana.ai/products/gaudi3/).
Each server is equipped with 8 devices, known as Habana Processing Units (HPUs), providing 128GB of memory on Gaudi 3,
96GB on Gaudi 2, and 32GB on the first-gen Gaudi. For more details on the underlying hardware architecture, check out the
[Gaudi Architecture Overview](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html).
Optimum for Intel Gaudi library is fully compatible with all three generations of Gaudi accelerators.

For in-depth examples of running workloads on Gaudi, explore the following blog posts:
- [Benchmarking Intel Gaudi 2 with NVIDIA A100 GPUs](https://huggingface.co/blog/habana-gaudi-2-benchmark)
- [Accelerating Vision-Language Models: BridgeTower on Habana Gaudi2](https://huggingface.co/blog/bridgetower)

The following model architectures, tasks and device distributions have been validated for Optimum for Intel Gaudi:

In the tables below, ✅ means single-card, multi-card and DeepSpeed have all been validated.

- Transformers:

| Architecture                                          | Training                      | Inference                                       | Tasks                                                                                                                                                                                                                                                           |
|:------------------------------------------------------|:-----------------------------:|:-----------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| BERT                                                  | ✅                            | ✅                                              | [text classification](/examples/text-classification)[question answering](/examples/question-answering)[language modeling](/examples/language-modeling)[text feature extraction](/examples/text-feature-extraction) |
| RoBERTa                                               | ✅                            | ✅                                              | [question answering](/examples/question-answering)[language modeling](/examples/language-modeling)                                                                                                                                   |
| ALBERT                                                | ✅                            | ✅                                              | [question answering](/examples/question-answering)[language modeling](/examples/language-modeling)                                                                                                                                   |
| DistilBERT                                            | ✅                            | ✅                                              | [question answering](/examples/question-answering)[language modeling](/examples/language-modeling)                                                                                                                                   |
| GPT2                                                  | ✅                            | ✅                                              | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| BLOOM(Z)                                              |                               | DeepSpeed                     | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| StarCoder / StarCoder2                                | ✅                            | Single-card                   | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| GPT-J                                                 | DeepSpeed   | Single cardDeepSpeed | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| GPT-Neo                                               |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| GPT-NeoX                                              | DeepSpeed   | DeepSpeed                     | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| OPT                                                   |                               | DeepSpeed                     | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Llama 2 / CodeLlama / Llama 3 / Llama Guard / Granite | ✅                            | ✅                                              | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)[question answering](/examples/question-answering)[text classification](/examples/text-classification) (Llama Guard)   |
| StableLM                                              |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Falcon                                                | LoRA        | ✅                                              | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| CodeGen                                               |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| MPT                                                   |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Mistral                                               |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Phi                                                   | ✅                            | Single card                   | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| Mixtral                                               |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Persimmon                                             |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Qwen2 / Qwen3                                         | Single card | Single card                   | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| Qwen2-MoE                                             |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Gemma                                                 | ✅                            | Single card                   | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| Gemma2                                                |                               | ✅                                              | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Gemma3                                                |                               | ✅                                              | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| XGLM                                                  |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Cohere                                                |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| T5 / Flan T5                                          | ✅                            | ✅                                              | [summarization](/examples/summarization)[translation](/examples/translation)[question answering](/examples/question-answering#fine-tuning-t5-on-squad20)                                                                    |
| BART                                                  |                               | Single card                   | [summarization](/examples/summarization)[translation](/examples/translation)[question answering](/examples/question-answering#fine-tuning-t5-on-squad20)                                                                    |
| ViT                                                   | ✅                            | ✅                                              | [image classification](/examples/image-classification)                                                                                                                                                                                        |
| Swin                                                  | ✅                            | ✅                                              | [image classification](/examples/image-classification)                                                                                                                                                                                        |
| Wav2Vec2                                              | ✅                            | ✅                                              | [audio classification](/examples/audio-classification)[speech recognition](/examples/speech-recognition)                                                                                                                             |
| Whisper                                               | ✅                            | ✅                                              | [speech recognition](/examples/speech-recognition)                                                                                                                                                                                            |
| SpeechT5                                              |                               | Single card                   | [text to speech](/examples/text-to-speech)                                                                                                                                                                                                    |
| CLIP                                                  | ✅                            | ✅                                              | [contrastive image-text training](/examples/contrastive-image-text)                                                                                                                                                                           |
| BridgeTower                                           | ✅                            | ✅                                              | [contrastive image-text training](/examples/contrastive-image-text)                                                                                                                                                                           |
| ESMFold                                               |                               | Single card                   | [protein folding](/examples/protein-folding)                                                                                                                                                                                                  |
| Blip                                                  |                               | Single card                   | [visual question answering](/examples/visual-question-answering)[image to text](/examples/image-to-text)                                                                                                                             |
| OWLViT                                                |                               | Single card                   | [zero shot object detection](/examples/zero-shot-object-detection)                                                                                                                                                                            |
| ClipSeg                                               |                               | Single card                   | [object segmentation](/examples/object-segementation)                                                                                                                                                                                         |
| Llava / Llava-next / Llava-onevision                  |                               | Single card                   | [image to text](/examples/image-to-text)                                                                                                                                                                                                      |
| idefics2                                              | LoRA        | Single card                   | [image to text](/examples/image-to-text)                                                                                                                                                                                                      |
| Paligemma                                             |                               | Single card                   | [image to text](/examples/image-to-text)                                                                                                                                                                                                      |
| Segment Anything Model                                |                               | Single card                   | [object segmentation](/examples/object-segementation)                                                                                                                                                                                         |
| VideoMAE                                              |                               | Single card                   | [Video classification](/examples/video-classification)                                                                                                                                                                                        |
| TableTransformer                                      |                               | Single card                   | [table object detection](/examples/table-detection)                                                                                                                                                                                           |
| DETR                                                  |                               | Single card                   | [object detection](/examples/object-detection)                                                                                                                                                                                                |
| Mllama                                                | LoRA        | ✅                                              | [image to text](/examples/image-to-text)                                                                                                                                                                                                      |
| MiniCPM3                                              |                               | Single card                   | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| Baichuan2                                             | DeepSpeed   | Single card                   | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| DeepSeek-V2                                           | ✅                            | ✅                                              | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| DeepSeek-V3 / Moonlight                               |                               | ✅                                              | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| ChatGLM                                               | DeepSpeed   | Single card                   | [language modeling](/examples/language-modeling)[text generation](/examples/text-generation)                                                                                                                                         |
| Qwen2-VL                                              |                               |  Single card                  | [image to text](/examples/image-to-text)                                                                                                                                                                                                      |
| Qwen2.5-VL                                            |                               |  Single card                  | [image to text](/examples/image-to-text)                                                                                                                                                                                                      |
| VideoLLaVA                                            |                               | Single card                   | [Video comprehension](/examples/video-comprehension)                                                                                                                                                                                          |
| GLM-4V                                                |                               |  Single card                  | [image to text](/examples/image-to-text)                                                                                                                                                                                                      |
| Arctic                                                |                               |  DeepSpeed                    | [text generation](/examples/text-generation)                                                                                                                                                                                                  |
| GPT-OSS                                               |                               |  DeepSpeed                    | [text generation](/examples/text-generation)                                                                                                                                                                                                  |

- Diffusers

| Architecture               | Training.              | Inference                     | Tasks                                                                                                                                                                                         |
|----------------------------|:----------------------:|:-----------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Stable Diffusion           | ✅                  | ✅                         | [text-to-image generation](/examples/stable-diffusion)                                                                                                                      |
| Stable Diffusion XL        | ✅                  | ✅                         | [text-to-image generation](/examples/stable-diffusion)                                                                                                                      |
| Stable Diffusion Depth2img |                        | Single card | [depth-to-image generation](/examples/stable-diffusion)                                                                                                                     |
| Stable Diffusion 3         | ✅                  | ✅                         | [text-to-image generation](/examples/stable-diffusion#stable-diffusion-3-and-35-sd3)                                                                                        |
| LDM3D                      |                        | Single card | [text-to-image generation](/examples/stable-diffusion)                                                                                                                      |
| FLUX.1                     | LoRA | Single card | [text-to-image generation](/examples/stable-diffusion)                                                                                                                      |
| Text to Video              |                        | Single card | [text-to-video generation](/examples/stable-diffusion#text-to-video-generation)                                                                                             |
| Image to Video             |                        | Single card | [image-to-video generation](/examples/stable-diffusion#image-to-video-generation)                                                                                           |
| i2vgen-xl                  |                        | Single card | [image-to-video generation](/examples/stable-diffusion#I2vgen-xl)                                                                                                           |
| Wan                        |                        | ✅                         | [text-to-video generation](/examples/stable-diffusion#text-to-video-with-wan-22)[image-to-video generation](/examples/stable-diffusion#image-to-video-with-wan-22) |

- PyTorch Image Models/TIMM:

| Architecture        | Training | Inference                     | Tasks                                                                     |
|---------------------|:--------:|:-----------------------------:|:--------------------------------------------------------------------------|
| FastViT             |          | Single card |  [image classification](/examples/image-classification) |

- TRL:

| Architecture     | Training | Inference | Tasks                                            |
|------------------|:--------:|:---------:|:-------------------------------------------------|
| Llama 2          | ✅       |           | [DPO Pipeline](/examples/trl)  |
| Llama 2          | ✅       |           | [PPO Pipeline](/examples/trl)  |
| Stable Diffusion | ✅       |           | [DDPO Pipeline](/examples/trl) |

Other models and tasks supported by the 🤗 Transformers and 🤗 Diffusers library may also work.
You can refer to this [section](https://github.com/huggingface/optimum-habana#how-to-use-it) for using them with 🤗 Optimum for Intel Gaudi.
In addition, [this page](/examples) explains how to modify any [example](https://github.com/huggingface/transformers/tree/main/examples/pytorch) from the 🤗 Transformers library to make it work with 🤗 Optimum for Intel Gaudi.

