Building the Kernel Path Between a CPU and an Accelerator
1. Introduction
Every accelerator — a GPU, an FPGA, an AI inference chip — is, from the kernel’s point of view, just another device sitting behind the PCIe bus with its own address space, its own notion of memory coherency, and its own completion semantics. The interesting engineering does not happen inside the accelerator’s silicon; it happens in the thin, unforgiving layer of kernel code that has to reconcile three incompatible worldviews at once: the CPU’s virtual memory model, the IOMMU’s device address space, and the accelerator’s own internal memory controller. Get that layer wrong and you don’t get a crash — you get silent data corruption, an IOMMU fault storm three days into a training run, or a doorbell write that the hardware never sees because a memory barrier was missing.
This lesson builds that layer from first principles. We will construct, piece by piece, the kernel driver architecture that lets a userspace process hand a buffer to an accelerator, have the IOMMU translate it safely, have the device execute work against it, and have the CPU find out when the work is done — all without trusting the device, without stalling the scheduler, and without corrupting memory that another CPU might be touching concurrently.
2. Historical Background
Linux’s accelerator story did not start with GPUs. It started with the humbler problem of network interface cards doing DMA into kernel buffers, solved in the 1990s with the dma_alloc_coherent API. The first real “heterogeneous compute” pattern arrived with InfiniBand RDMA in the early 2000s, which forced the kernel to solve pinned-memory registration and zero-copy transfer between a device and arbitrary userspace virtual addresses — a problem structurally identical to what a GPU driver needs today.
GPU compute changed the scale of the problem. Nvidia’s proprietary driver and the open amdgpu/nouveau drivers had to support gigabyte-scale buffers, buffer sharing between multiple consumers (a buffer produced by a video decoder and consumed by a compositor and a neural network, all as separate processes), and years of incremental work led to DMA-BUF (merged in Linux 3.3) as the generic buffer-sharing framework. When unified-memory GPU compute (CUDA UVA, ROCm) needed the accelerator to fault on pages that were still resident in normal process memory, the kernel grew HMM — Heterogeneous Memory Management (merged in 4.14), extending mmu_notifier and introducing ZONE_DEVICE pages so device memory could be represented as ordinary-looking struct page entries. FPGAs rode the same rails through the accel subsystem and DRM’s render-node model. The AI accelerator wave of the last few years (custom ASICs, NPUs) mostly reuses this exact stack rather than inventing a new one, because the hard problems — address translation, coherency, buffer lifetime — were already solved.
3. The Systems-Level Problem
Strip away the marketing and an accelerator workload reduces to four kernel-level problems that must all be solved correctly and in the right order:
Memory description — the accelerator needs a physically contiguous (or IOMMU-scatter-gather-described) view of a buffer that a userspace process addresses virtually.
Address translation and protection — the device must not be able to read or write memory it wasn’t granted, and the IOMMU is the only trust boundary between a possibly-hostile or buggy device and the rest of physical memory.
Coherency — CPU caches and, in some designs, the accelerator’s own caches must agree on the contents of shared memory before either side reads it.
Completion notification without stalling the CPU — a compute kernel might run for microseconds or minutes; the driver must let the calling thread block cheaply, or better, let userspace poll without ever calling into the kernel on the hot path.
Every accelerator driver on Linux — amdgpu, habanalabs, i915, the various NPU drivers under drivers/accel/ — is a variation on solving these four problems for a specific piece of silicon.
4. Linux Kernel Architecture
