PCIe ACS Controls Whether Peer Traffic Can Bypass the IOMMU

An IOMMU can restrict DMA only for transactions that reach its translation and permission checks. PCI Express complicates that boundary because two endpoints under the same hierarchy can exchange peer-to-peer traffic without sending every transaction through the root complex. A switch may be able to route a request directly from one downstream port to another.

PCIe Access Control Services (ACS) adds controls for that routing boundary. Depending on the component and supported ACS features, the fabric can validate a request, block selected translated traffic, redirect peer requests or completions upstream, or constrain their egress. The result is not merely a routing preference. ACS can determine whether two devices are separable at the IOMMU boundary.

A peer path can stay below the root port

Consider two endpoints attached to the same PCIe switch. Device A issues a Memory Write targeting an address exposed by device B. If the switch can route that request directly to B, the transaction does not need to travel up to the root port first.

                 root complex
                      |
                  root port
                      |
                    switch
                   /      \
             device A    device B
                   \------>/
                  local P2P path

That local path is useful for peer-to-peer DMA. It can also create an isolation problem when system software expects the IOMMU to mediate traffic between the endpoints. An IOMMU located upstream cannot enforce a policy on a transaction that never reaches it.

The relevant boundary therefore includes the PCIe topology, not only the IOMMU page tables. A correct translation table does not compensate for an interconnect path that bypasses the translation unit.

Request redirect changes the path seen by the platform

ACS P2P Request Redirect can force applicable peer requests toward the upstream port instead of allowing a component to forward them directly between downstream peers. P2P Completion Redirect provides corresponding control for completions.

A simplified redirected path looks like this:

device A
   |
   v
 PCIe switch
   |
   | ACS redirect
   v
upstream path
   |
IOMMU / root complex policy boundary
   |
   v
destination path

The exact route and enforcement behavior depend on the platform topology and the ACS capability implemented by each component. ACS is a set of controls rather than one universal switch. Source Validation, Translation Blocking, P2P Request Redirect, P2P Completion Redirect, Upstream Forwarding, P2P Egress Control, and Direct Translated P2P are distinct ACS functions.

This distinction matters when inspecting hardware. Seeing an ACS capability does not establish that every ACS function exists, is enabled, or provides the isolation property required by a particular deployment.

IOMMU groups reflect paths that cannot be isolated safely

Linux exposes IOMMU groups as sets of devices that form a minimum isolation unit for IOMMU-based ownership. VFIO documentation notes that interconnect topology can reduce isolation granularity even when the IOMMU itself can distinguish individual requesters. A bridge without suitable ACS behavior can permit transactions between devices without reaching the IOMMU.

That can place several endpoints in the same IOMMU group.

IOMMU group 18
  0000:41:00.0
  0000:42:00.0

The grouping is not simply an administrative choice to keep related hardware together. It represents an isolation constraint discovered from the device and interconnect topology. Assigning only one member to an untrusted userspace driver or virtual machine can be unsafe if another member provides a path around the intended DMA boundary.

ACS-capable routing components can permit finer separation when their supported and configured controls establish the required isolation. Other factors still matter, including multifunction devices, bridges that hide requester identity, and platform-specific IOMMU behavior.

Disabling redirect can improve P2P reachability while reducing isolation

Linux provides a disable_acs_redir= kernel parameter for selected PCI devices. Kernel documentation states that forcing ACS redirect capabilities off can allow peer traffic through bridges without forcing it upstream. The same documentation warns that doing so removes isolation between devices and can place more devices in an IOMMU group.

That trade is structural. A direct peer path may be desirable for workloads that intentionally move data between devices, such as certain storage, accelerator, or network configurations. But the shorter path and the stronger isolation boundary can require opposite routing behavior.

This does not imply that ACS should always be enabled or that direct P2P should always be disabled. The required setting follows from the ownership model. Devices trusted as one unit can have different routing requirements from devices assigned to separate guests or separate untrusted processes.

Linux PCI peer-to-peer DMA support also evaluates topology because peer routing across host bridges is not uniformly defined by PCIe. ACS settings can determine whether a transaction remains within a well-defined PCIe hierarchy or is redirected toward a host bridge. P2P feasibility and isolation therefore share part of the same topology analysis.

Source Validation checks requester identity at a boundary

ACS Source Validation addresses another part of the problem. PCIe transactions carry requester identity, and isolation mechanisms often rely on that identity to select translation or permission context. A component supporting Source Validation can check whether a received request carries a requester identity that is valid for the ingress path.

This is different from P2P Request Redirect. Redirect controls where traffic is forwarded. Source Validation checks whether the source identity is acceptable at the point where the request arrives.

Those functions can contribute to the same isolation goal, but they operate on different properties. Treating ACS as a single boolean feature hides that distinction and can produce incorrect conclusions from capability listings.

ACS does not turn every topology into per-device isolation

ACS cannot repair every source of shared fate in a PCIe hierarchy. A multifunction device may contain internal paths between functions that are outside an upstream switch’s control. A legacy PCI bridge can obscure requester granularity. A platform may also impose IOMMU or root-complex constraints that keep several devices inseparable.

For device assignment, the useful question is therefore not whether one switch reports ACS. The relevant property is whether the complete path provides an isolation boundary that system software can rely on.

Linux encodes that result in its IOMMU grouping rather than requiring userspace to infer safety from one capability bit. Tools can expose ACS state for diagnosis, but forcing a different grouping without matching hardware isolation changes the security assumption rather than creating new isolation.

Routing policy is part of DMA isolation

IOMMU protection is often described as an address-translation problem: bind a device to a translation domain, map only approved pages, and reject everything else. PCIe peer routing adds a prior condition. The transaction must actually cross the enforcement point for those mappings to matter.

ACS supplies fabric controls that help preserve that condition across PCIe switches and bridges. Its practical effect appears in IOMMU group boundaries, VFIO device assignment, and P2P routing decisions. The durable rule is topological: DMA isolation depends on every path a device transaction can take, not only on the page tables programmed into the IOMMU.