GPUs

Current Status
This page is intended to represent the current state (for deployments in the wild) of how to provide GPU capability within an OpenStack cloud. We also provide links (with comments on currency) to older information on this topic. The future section looks at what is or might be coming upstream.

Today there are two primary possibilities for integrating GPU acceleration "natively" into an OpenStack cloud without any external services. These are:
 * Ironic bare-metal cloud having GPU compute nodes
 * Nova VM-based compute (e.g. Libvirt+KVM) with PCI passthrough

GPU passthrough known good/working & bad/not-working configs
Working GPU passthrough into virtual machines (as possible with PCI passthrough support in Nova) requires specific hardware features which are not always reliably implemented or available. This page aims to collect knowing working and particularly *not*-working hardware combinations of platform and GPU models so that existing and potential deployers can knowledge-share and avoid expensive/nasty surprises. Systems vendors may also be able to use this to indicate compatibility.

The same GPU may appear as both working and not working due to platform and or software issues.

Working
Please add details in the format below, if the format needs adjusting then please do so consistently for all existing entries.    
 * notes

NVIDIA P100 Dell R730 QEMU-KVM Ubuntu Trusty with Libvirt and QEMU-KVM since Liberty, Ubuntu Xenial

NVIDIA P100 (3 GPU's) Penguin (Gigabyte Motherboard) Ubuntu Openstack Stein Ubuntu 20.04 Mellanox drivers interfere with QEMU and compute node cannot use libvirt. To override: wget http://archive.ubuntu.com/ubuntu/pool/main/r/rdma-core/libibverbs1_28.0-1ubuntu1_amd64.deb dpkg -i libibverbs1_28.0-1ubuntu1_amd64.deb apt --fix-broken -y install
 * BUG ***

NVIDIA K80 Dell R720, Dell R730, Dell C4130, SM R354.v5 QEMU-KVM Ubuntu Trusty with Libvirt and QEMU-KVM since Liberty, Ubuntu Xenial

NVIDIA K40 Dell R720, Dell R730 QEMU-KVM Ubuntu Trusty with Libvirt and QEMU-KVM since Liberty, Ubuntu Xenial

NVIDIA GRID K1 Dell R720, Dell R730, SM R354.v5 QEMU-KVM Ubuntu Trusty, Ubuntu Xenial
 * Drivers for the K1 stop at version 367. So if 370+ are installed the card does not work as expected. (Trusty, CentOS, Xenial guests)

NVIDIA GRID K2 Dell R720 QEMU-KVM Ubuntu Trusty, Ubuntu Xenial

NVIDIA Quadro 5000 Dell R720 QEMU-KVM Ubuntu Trusty, Ubuntu Xenial

NVIDIA GTX 770 Intel i7 3770 QEMU-KVM Debian Jessie

Not working
   

NVIDIA P100 Dell R730 QEMU-KVM Ubuntu Trusty with Libvirt and QEMU-KVM before Liberty, Ubuntu Xenial

NVIDIA K80 Dell R720, Dell R730, Dell C4130 QEMU-KVM Ubuntu Trusty with Libvirt and QEMU-KVM before Liberty, Ubuntu Xenial

NVIDIA K40 Dell R720, Dell R730 QEMU-KVM Ubuntu Trusty with Libvirt and QEMU-KVM before Liberty, Ubuntu Xenial

References / Community Efforts

 * OVH's talk at OpenStack Summit Sydney: https://www.openstack.org/summit/sydney-2017/summit-schedule/events/19493
 * Scientific WG/SIG: https://wiki.openstack.org/wiki/ScientificWGGPUs
 * Notebook demonstrating how to setup and test GPU passthrough with Devstack: https://notebooks.azure.com/jmain-nv/libraries/openstack-lab

Outdated References

 * https://wiki.openstack.org/wiki/HeterogeneousGpuAcceleratorSupport