Virtualization advanced ⏱ 90 minutes

GPU Passthrough: Complete NVIDIA and AMD Guide

Pass GPUs to VMs for gaming and AI. IOMMU groups, vfio-pci, and troubleshooting.

By homelabbing.io • 2025-12-17
#gpu-passthrough#vfio#nvidia#amd#gaming

GPU passthrough allows you to dedicate a physical graphics card directly to a virtual machine, providing near-native performance for gaming, machine learning, CAD, video editing, and other GPU-intensive workloads. This comprehensive guide covers both NVIDIA and AMD GPU passthrough on Proxmox VE and standalone KVM/libvirt systems.

Understanding GPU Passthrough

GPU passthrough uses IOMMU (Input-Output Memory Management Unit) technology to allow virtual machines to directly access physical hardware. When properly configured, a VM can use a GPU as if it were installed in a physical machine, with minimal performance overhead (typically 1-5%).

Prerequisites

Before starting, ensure your hardware supports GPU passthrough:

CPU Requirements:

  • Intel: VT-d support (most Core i5/i7/i9 from 2nd gen onwards)
  • AMD: AMD-Vi support (most Ryzen and EPYC processors)

Motherboard Requirements:

  • IOMMU support in BIOS/UEFI
  • Ideally, separate IOMMU groups for each PCIe slot
  • Chipset with sufficient PCIe lanes

GPU Requirements:

  • A dedicated GPU for passthrough (cannot be your host’s primary display)
  • NVIDIA: GeForce, Quadro, or Tesla cards (see NVIDIA Code 43 workaround below)
  • AMD: Most modern Radeon cards work well
  • For dual-GPU setups: integrated GPU for host, discrete GPU for VM

System Requirements:

  • Minimum 16GB RAM (32GB+ recommended)
  • At least one additional GPU or integrated graphics for the host
  • UEFI boot mode (not legacy BIOS)
  • Latest motherboard firmware

Part 1: IOMMU Configuration

Step 1: Enable IOMMU in BIOS/UEFI

Reboot your system and enter BIOS/UEFI settings (usually DEL, F2, or F12 during boot):

Intel Systems:

  1. Navigate to Advanced settings
  2. Find “VT-d” or “Intel Virtualization Technology for Directed I/O”
  3. Enable VT-d
  4. Save and exit

AMD Systems:

  1. Navigate to Advanced settings
  2. Find “AMD-Vi” or “IOMMU”
  3. Enable AMD-Vi/IOMMU
  4. Save and exit

Step 2: Enable IOMMU in GRUB Bootloader

For Proxmox and Debian-based systems:

# Edit GRUB configuration
nano /etc/default/grub

# Find the GRUB_CMDLINE_LINUX_DEFAULT line and modify it:
# For Intel CPUs:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

# For AMD CPUs:
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

# Optional: Add video=efifb:off if you have display issues
# For problematic setups:
# GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt video=efifb:off"

# Update GRUB
update-grub

# Reboot
reboot

What these parameters do:

  • intel_iommu=on or amd_iommu=on: Enables IOMMU
  • iommu=pt: Uses passthrough mode (better performance)
  • video=efifb:off: Prevents EFI framebuffer from claiming the GPU

Step 3: Verify IOMMU is Enabled

After rebooting:

# Check if IOMMU is enabled
dmesg | grep -e DMAR -e IOMMU

# You should see output like:
# [    0.000000] DMAR: IOMMU enabled
# or
# [    0.000000] AMD-Vi: IOMMU enabled

# List IOMMU groups
find /sys/kernel/iommu_groups/ -type l

# View detailed IOMMU groups
#!/bin/bash
shopt -s nullglob
for g in /sys/kernel/iommu_groups/*; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done
done

Understanding IOMMU Groups: Your GPU should ideally be in its own IOMMU group. If it shares a group with other devices, you may need to pass through the entire group or use the ACS override patch (not recommended for production).

Example of a good IOMMU group:

IOMMU Group 1:
    01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3070] [10de:2484]
    01:00.1 Audio device [0403]: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b]

Part 2: Configuring VFIO

Step 4: Load VFIO Modules

# Edit modules file
nano /etc/modules

# Add these lines:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Step 5: Blacklist GPU Drivers on Host

Prevent the host from loading drivers for your passthrough GPU:

# Create blacklist file
nano /etc/modprobe.d/blacklist.conf

# For NVIDIA GPUs, add:
blacklist nouveau
blacklist nvidia
blacklist nvidiafb
blacklist nvidia_drm
blacklist nvidia_modeset

# For AMD GPUs, add:
blacklist radeon
blacklist amdgpu

# Save and exit

Step 6: Bind GPU to VFIO-PCI

Find your GPU’s device IDs:

# List PCI devices
lspci -nn | grep -i vga
lspci -nn | grep -i nvidia
lspci -nn | grep -i amd

# Example output:
# 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3070] [10de:2484]
# 01:00.1 Audio device [0403]: NVIDIA Corporation GA104 HD Audio [10de:228b]

Note the device IDs in square brackets (e.g., 10de:2484 and 10de:228b for NVIDIA RTX 3070).

# Configure VFIO to claim these devices
nano /etc/modprobe.d/vfio.conf

# Add your device IDs (replace with your actual IDs):
options vfio-pci ids=10de:2484,10de:228b disable_vga=1

# For AMD GPU example:
# options vfio-pci ids=1002:67df,1002:aaf0 disable_vga=1

# Save and exit

Step 7: Update Initramfs

# Update initramfs to include changes
update-initramfs -u -k all

# Reboot
reboot

Step 8: Verify VFIO Binding

After reboot:

# Check if GPU is bound to vfio-pci
lspci -k | grep -A 3 -i "vga\|3d\|nvidia\|amd"

# You should see:
# Kernel driver in use: vfio-pci
# instead of nouveau, nvidia, or amdgpu

# Verify VFIO devices
dmesg | grep -i vfio

# List VFIO devices
ls -la /dev/vfio/

Part 3: Proxmox VM Configuration

Step 9: Create a VM in Proxmox

  1. Create VM via Web UI:

    • Click “Create VM”
    • General: Set VM ID and name (e.g., “Gaming-VM”)
    • OS: Select your ISO (Windows 10/11 or Linux)
    • System:
      • BIOS: OVMF (UEFI)
      • Add EFI Disk: Yes
      • SCSI Controller: VirtIO SCSI
      • Machine: q35
    • Disks:
      • Size: 100GB minimum
      • Cache: Write back (for better performance)
      • Discard: Enabled (for SSD trim support)
    • CPU:
      • Sockets: 1
      • Cores: 6-8 (half your physical cores)
      • Type: host
    • Memory: 16384MB minimum (16GB)
    • Network: VirtIO (semi-virtualized)

  2. DO NOT START THE VM YET

Step 10: Add GPU to VM via CLI

# SSH into Proxmox host
# Find your GPU's PCI address
lspci | grep -i vga

# Example output: 01:00.0 VGA compatible controller

# Edit VM configuration (replace 100 with your VM ID)
nano /etc/pve/qemu-server/100.conf

# Add these lines:
hostpci0: 0000:01:00,pcie=1,x-vga=1

# For GPU with audio (recommended):
hostpci0: 0000:01:00,pcie=1,x-vga=1,multifunction=on
hostpci1: 0000:01:00.1,pcie=1

# Additional recommended settings:
cpu: host,hidden=1,flags=+pcid
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'

Configuration Explained:

  • hostpci0: First PCIe passthrough device
  • 0000:01:00: PCI address of GPU
  • pcie=1: Present device as PCIe (required for modern GPUs)
  • x-vga=1: Enable VGA passthrough
  • hidden=1: Hide virtualization from guest (NVIDIA requirement)
  • hv_vendor_id=NV43FIX: Workaround for NVIDIA Code 43 error
  • kvm=off: Hide KVM from guest (helps with NVIDIA driver)

Step 11: ROM File (If Needed)

Some GPUs require a ROM file for passthrough:

# Download GPU ROM from TechPowerUp VBIOS database
# https://www.techpowerup.com/vgabios/

# Upload ROM file to Proxmox
scp gpu-rom.bin root@YOUR-PROXMOX-IP:/usr/share/kvm/

# Add to VM config
nano /etc/pve/qemu-server/100.conf

# Modify hostpci line:
hostpci0: 0000:01:00,pcie=1,x-vga=1,romfile=gpu-rom.bin

Step 12: Install Guest OS

  1. Start the VM
  2. Install Windows 10/11 or Linux
  3. During Windows installation, load VirtIO drivers when prompted:
    • Download VirtIO ISO from Proxmox
    • Attach as second CD drive
    • Load storage and network drivers

Step 13: Install GPU Drivers

For NVIDIA in Windows:

1. Download latest Game Ready or Studio drivers from NVIDIA
2. Install normally
3. If you get "Code 43" error in Device Manager:
   - Verify VM config has cpu: host,hidden=1
   - Verify args line includes hv_vendor_id=NV43FIX,kvm=off
   - Reboot VM

For AMD in Windows:

1. Download AMD Adrenalin drivers
2. Install normally
3. AMD cards generally work without special configuration

For Linux guests:

# NVIDIA (Ubuntu/Debian)
sudo apt update
sudo apt install nvidia-driver-545

# AMD (Ubuntu/Debian)
# Usually works out of the box with open-source drivers
# For proprietary drivers:
sudo apt install amdgpu-pro

Part 4: Advanced Configuration

CPU Pinning for Better Performance

Pin VM cores to specific physical CPU cores:

# Check CPU topology
lscpu -e

# Edit VM config
nano /etc/pve/qemu-server/100.conf

# Add CPU affinity (example for cores 4-11 on a 12-core CPU)
# Keep cores 0-3 for host
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off' -smp 8,sockets=1,cores=8,threads=1 -vcpu vcpunum=0,affinity=4 -vcpu vcpunum=1,affinity=5 -vcpu vcpunum=2,affinity=6 -vcpu vcpunum=3,affinity=7 -vcpu vcpunum=4,affinity=8 -vcpu vcpunum=5,affinity=9 -vcpu vcpunum=6,affinity=10 -vcpu vcpunum=7,affinity=11

Huge Pages for Better Memory Performance

# Calculate huge pages (16GB VM = 8192 pages of 2MB)
# Edit sysctl
nano /etc/sysctl.conf

# Add:
vm.nr_hugepages = 8192

# Apply
sysctl -p

# Verify
cat /proc/meminfo | grep Huge

# Edit VM config
nano /etc/pve/qemu-server/100.conf

# Modify memory line:
memory: 16384
hugepages: 1024

Looking Glass for Zero-Latency Display

Looking Glass allows you to view the VM’s GPU output on your host without monitor switching:

# On Proxmox host
apt install build-essential cmake libgl1-mesa-dev libfontconfig1-dev libspice-protocol-dev libx11-dev

# Add shared memory device to VM
nano /etc/pve/qemu-server/100.conf

# Add:
args: -device ivshmem-plain,memdev=ivshmem,bus=pcie.0 -object memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M

# Create shared memory file at boot
nano /etc/tmpfiles.d/looking-glass.conf

# Add:
f /dev/shm/looking-glass 0660 root kvm -

# Create file now
touch /dev/shm/looking-glass
chown root:kvm /dev/shm/looking-glass
chmod 660 /dev/shm/looking-glass

# Inside Windows VM, install Looking Glass client
# Download from: https://looking-glass.io/

Audio Passthrough

For better audio quality, pass through a USB audio device or use PulseAudio:

# Method 1: USB audio passthrough (easiest)
# Add USB audio device in Proxmox web UI

# Method 2: PulseAudio
nano /etc/pve/qemu-server/100.conf

# Add:
args: -audiodev pa,id=snd0,server=/run/user/0/pulse/native -device ich9-intel-hda -device hda-output,audiodev=snd0

Part 5: KVM/libvirt Configuration (Non-Proxmox)

For standalone KVM setups using virt-manager:

Create VM with virt-manager

# Install required packages
sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager ovmf

# Start libvirt
sudo systemctl enable --now libvirtd

# Add user to libvirt group
sudo usermod -aG libvirt $USER
sudo usermod -aG kvm $USER

# Create VM using virt-manager
virt-manager
  1. Create new VM
  2. Firmware: UEFI x86_64
  3. Chipset: Q35
  4. Install OS

Add GPU via virt-manager

  1. Open VM details
  2. Add Hardware > PCI Host Device
  3. Select your GPU (and GPU audio device)
  4. Apply

Edit XML for NVIDIA Fix

# Edit VM XML
sudo virsh edit VM-NAME

# Add to <features> section:
<features>
  <hyperv>
    <vendor_id state='on' value='1234567890ab'/>
  </hyperv>
  <kvm>
    <hidden state='on'/>
  </kvm>
</features>

# Modify <cpu> section:
<cpu mode='host-passthrough' check='none'>
  <topology sockets='1' cores='8' threads='1'/>
</cpu>

Part 6: Troubleshooting

Issue: Code 43 Error (NVIDIA)

Symptoms: NVIDIA driver shows error Code 43 in Windows Device Manager

Solutions:

# 1. Verify VM config has these lines:
cpu: host,hidden=1
args: -cpu 'host,hv_vendor_id=NV43FIX,kvm=off'

# 2. Ensure UEFI boot mode (not legacy BIOS)

# 3. Try different NVIDIA driver versions (some work better)

# 4. Check if GPU ROM is needed
# Download from TechPowerUp and add to config

# 5. Verify GPU is properly bound to vfio-pci
lspci -k | grep -A 3 -i nvidia

Issue: VM Won’t Start / Black Screen

Solutions:

# 1. Check if GPU is bound to vfio-pci
lspci -k

# 2. Verify IOMMU groups
# GPU should be in isolated group

# 3. Add video=efifb:off to GRUB
nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt video=efifb:off"
update-grub

# 4. Try different machine type
# In VM config: machine: q35 or machine: pc-q35-6.1

# 5. Check VM logs
journalctl -u pveproxy
tail -f /var/log/syslog

Issue: Poor Performance

Solutions:

# 1. Enable CPU host mode
cpu: host

# 2. Use CPU pinning (see advanced section)

# 3. Enable huge pages

# 4. Check CPU governor
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
# Should be "performance" not "powersave"
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# 5. Disable CPU mitigations (reduces security, increases performance)
# Add to GRUB: mitigations=off

# 6. Enable MSI (Message Signaled Interrupts) in Windows VM
# Download MSI Utility Mode v3

Issue: GPU Reset Bug

Symptoms: VM works first time but fails to start after shutdown

Solutions:

# 1. Vendor reset bug - common with some AMD cards

# 2. Use vendor-reset kernel module
git clone https://github.com/gnif/vendor-reset
cd vendor-reset
make
make install
echo "vendor-reset" > /etc/modules-load.d/vendor-reset.conf

# 3. Alternative: Don't shutdown VM, use reboot instead

# 4. Update motherboard BIOS/UEFI

# 5. Try different GPU if problem persists

Issue: No Audio from GPU

Solutions:

# 1. Ensure GPU audio device is also passed through
# Both GPU and GPU audio should be in VM config

# 2. Check Windows Device Manager
# GPU audio should appear under "Sound, video and game controllers"

# 3. Install NVIDIA/AMD audio drivers

# 4. Set GPU audio as default playback device in Windows

Issue: IOMMU Group Contains Multiple Devices

Not Recommended Solution (Security Risk):

# ACS Override Patch - breaks IOMMU isolation
# Only use if you understand the security implications

# Add to GRUB:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction"
update-grub
reboot

Recommended Solution:

  • Use a motherboard with better IOMMU grouping
  • Pass through the entire IOMMU group
  • Use different PCIe slots

Part 7: Best Practices for Homelabs

Multiple GPU Setup

For running multiple VMs with GPU passthrough:

# Example: 2 VMs with dedicated GPUs
# Host: iGPU (Intel/AMD integrated)
# VM1: NVIDIA RTX 3070
# VM2: AMD RX 6700 XT

# Bind both GPUs to vfio-pci
options vfio-pci ids=10de:2484,10de:228b,1002:73df,1002:ab28

Resource Allocation Strategy

For a 12-core/24-thread system with 64GB RAM:

  • Host: 4 cores, 16GB RAM
  • VM1 (Gaming): 6 cores, 24GB RAM, NVIDIA RTX 3070
  • VM2 (Workstation): 4 cores, 16GB RAM, AMD RX 6700 XT
  • Remaining: 8GB for other VMs/containers

Backup Considerations

# GPU passthrough VMs can be backed up normally
# But exclude GPU state from backups

# In Proxmox:
vzdump 100 --mode snapshot --compress zstd

# VM snapshots work but may have issues
# Better to backup the disk and config separately

Power Management

# Enable PCIe power management
nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_aspm=force"

# Note: May cause issues with some GPUs
# Test thoroughly

Monitoring

# Monitor GPU usage in VM (from host)
watch -n 1 nvidia-smi

# For AMD:
watch -n 1 radeontop

# Monitor VM performance
qm monitor VMID
# Type: info status

Performance Benchmarks

Expected performance compared to bare metal:

  • Gaming: 95-98% performance
  • 3D Rendering: 96-99% performance
  • Machine Learning: 97-99% performance
  • Video Encoding: 96-98% performance

Performance loss mainly comes from:

  1. CPU virtualization overhead (1-2%)
  2. Memory access patterns (1-2%)
  3. Storage I/O if not optimized (0-3%)

Security Considerations

Important Notes:

  1. GPU passthrough bypasses hypervisor isolation
  2. Malware in VM could potentially access GPU memory
  3. Always run untrusted VMs without GPU passthrough
  4. Keep GPU firmware updated
  5. IOMMU provides DMA protection - never disable it

Conclusion

GPU passthrough transforms virtual machines into powerful workstations capable of gaming, content creation, and GPU-accelerated computing. While the setup requires careful configuration and compatible hardware, the result is near-native performance with the flexibility of virtualization.

Key Takeaways:

  • Always verify IOMMU support before purchasing hardware
  • NVIDIA cards require extra configuration to avoid Code 43
  • AMD cards generally work more smoothly
  • CPU pinning and huge pages provide noticeable performance improvements
  • Keep separate GPUs for host and guests when possible

Next Steps:

  • Experiment with Looking Glass for seamless desktop integration
  • Try USB device passthrough for peripherals
  • Explore PCI passthrough for other devices (sound cards, capture cards)
  • Set up automated VM deployment with Terraform and Packer (see our other guides)

Happy virtualizing!