PCIe ATS Moves Address Translation Caching Into the Device

An IOMMU can translate DMA addresses on behalf of a device, but that arrangement puts translation machinery in the path of device memory traffic. PCIe Address Translation Services (ATS) adds another option: a capable device can request a translation and retain the result in its own translation cache. Later transactions can carry the translated address instead of requiring the same translation work at the IOMMU for every access.

That changes more than lookup placement. Translation state now exists inside the endpoint, so software that changes an address-space mapping must account for cached device translations as part of the invalidation protocol.

ATS separates translation requests from translated memory requests

Without ATS, a device commonly emits a DMA address that the platform IOMMU translates before the request reaches memory. With ATS enabled, the device can issue an Address Translation Request for an address it intends to use. The translation agent returns an Address Translation Completion containing translation information that the device may cache according to the applicable PCIe rules.

A simplified path looks like this:

device
  |
  | Address Translation Request
  v
IOMMU / translation agent
  |
  | Address Translation Completion
  v
device translation cache
  |
  | translated memory request
  v
platform memory path

The device-side cache is often described as a device TLB in operating-system documentation. Its exact organization is an implementation detail. ATS specifies protocol behavior around obtaining and invalidating translations; it does not require every endpoint to implement the same cache geometry or replacement policy.

A cached translation extends mapping lifetime beyond the IOMMU lookup

Consider a process address that currently resolves to physical page A. An ATS-capable device obtains that translation and stores it locally. Software later changes the mapping so the same virtual address refers to page B.

Changing the CPU page table or an IOMMU translation structure does not by itself erase the endpoint’s cached entry. Until the relevant invalidation protocol has completed, the device may still possess translation state derived from the earlier mapping.

earlier state:
virtual address V -> page A
device cache: V -> page A

mapping changes:
virtual address V -> page B
device cache: V -> page A   stale state must be retired

This is the central systems consequence of ATS. Translation caching can move work away from repeated IOMMU lookups, but mapping changes acquire a distributed consistency obligation. Software cannot treat page-table modification and device-cache retirement as the same event.

Invalidation crosses the PCIe boundary

ATS defines invalidation messaging so stale endpoint translations can be removed. Platform and operating-system code coordinate this machinery rather than allowing a mapping to disappear while a device can continue using an obsolete cached result.

Linux Shared Virtual Addressing documentation describes this relationship directly: ATS permits devices to cache translations, while IOMMU integration and memory-management notification mechanisms keep device translation state synchronized with CPU address-space changes. The precise path depends on the IOMMU driver, architecture, device capabilities, and the address-sharing mode in use.

Completion matters. An invalidation request traveling toward a device and the device finishing the required invalidation are distinct points in time. Memory backing an old mapping cannot safely be repurposed merely because software has initiated retirement of the translation. The applicable platform and PCIe contracts determine the ordering required before reuse.

This resembles an IOTLB invalidation problem, but the cache location is different. An IOMMU IOTLB is translation state inside the remapping unit. ATS permits translation state inside the endpoint. Systems that use both can therefore have several layers of derived address state whose lifetime must remain consistent with authoritative mappings.

PASID gives translation requests process context

ATS becomes especially significant with Shared Virtual Addressing (SVA). In that model, a device can operate on addresses from a process address space rather than relying only on a separate DMA mapping namespace.

PCIe Process Address Space ID (PASID) supplies context that lets platform translation machinery distinguish address spaces associated with device transactions. Linux documents PASID as a prerequisite for SVA alongside the relevant IOMMU support. A device transaction tagged for one process must resolve in that process’s translation context rather than in an unrelated address space.

ATS and PASID solve different parts of the path. PASID identifies an address-space context. ATS provides a protocol for obtaining and caching translations. Neither property alone means a device can recover from a missing resident page.

PRI covers faults that ATS alone cannot resolve

A translation request can encounter an address whose mapping cannot currently produce a usable translation. PCIe Page Request Interface (PRI) complements ATS by allowing a capable device to request that software service a page-related condition.

Linux’s SVA model combines these pieces. A device can request translation through ATS; when the address requires paging work, PRI provides the request path needed to involve the operating system. After the mapping becomes available, the device requests a translation again before continuing.

That distinction keeps the roles clear:

PASID -> identifies the address-space context
ATS   -> obtains and caches address translations
PRI   -> requests service for addressability conditions

Support is conditional. Device capability, IOMMU capability, firmware configuration, kernel support, and driver policy all participate. The presence of one PCIe capability does not imply that a complete SVA path is active.

Device translation caches change the cost boundary

ATS can reduce repeated translation traffic for workloads where a device reuses address translations, but no universal latency or throughput gain follows from the feature alone. Results depend on access locality, endpoint cache behavior, IOMMU implementation, invalidation frequency, PCIe topology, and workload characteristics.

Frequent mapping churn can also increase invalidation activity. A design that gains from cached translations during stable access can face a different cost profile when address spaces change rapidly. These are implementation and workload effects, not fixed properties of the protocol.

The more durable architectural point is about state placement. ATS lets an endpoint retain derived address translations. Once that state exists beyond the IOMMU, mapping lifetime and invalidation must cover the endpoint as well. The performance opportunity and the consistency obligation come from the same mechanism.