Isolation of DMA devices

Updated: April 19, 2023

The SMMUMAN supports spatial isolation of DMA devices on systems with IOMMU/SMMUs.

Isolation

The figure below presents the four isolation axes that can be implemented in a software system. Spatial isolation is fundamental to all other forms of isolation.

Figure 1. The isolation axes: spatial, resource, data, temporal.

In a QNX system, the OS kernel (procnto) uses MMU page tables to control attempts to access memory. In a virtualized system with the QNX Neutrino OS plus its virtualization extension (a libmod_qvm variant) running as the host, a second layer of page tables (or intermediate page tables) is used to manage guest access to memory.

MMUs can’t be used to manage Direct Memory Access (DMA) device attempts to access memory, however. The following section describes a different hardware mechanism, called an IOMMU/SMMU, which manages DMA device access to memory.

Note: On ARM platforms, intermediate page tables are known as Stage 2 page tables; on x86 platforms they are known as Extended Page Tables (EPT)).

DMA devices and IOMMU/SMMUs

A DMA device (e.g., GPU, network card, sound card) initiates reads and writes to memory without involving the CPU in the transaction.

That is, the CPU doesn't control a DMA device's access to memory. Instead, a DMA device takes control of the memory bus to gain direct access to system memory. Since no CPU is implicated in the memory access, an OS can’t manage a DMA device’s access to system memory without support from hardware IOMMU/SMMUs.

Most importantly, the protections the OS can provide against incorrect (and possibly malicious) access to memory by requests that go through the CPU do not apply to access requests from DMA devices. The OS cannot ensure that a DMA device is prevented from accessing memory it is not authorized to access.

A System Memory Management Unit (IOMMU/SMMU) is a hardware component that provides translation and access control for non-CPU initiated reads and writes (e.g., from DMA devices), similar to the translation and access control page tables provide for CPU-initiated reads and writes.

Hardware limitations

Note that on some boards the IOMMU/SMMU hardware doesn't provide the smmuman service the information it requires to map individual devices and report their attempted transgressions.

For example, on some ARM boards the IOMMU doesn't provide the smmuman service the information it requires to identify individual PCI devices. Similarly, on x86 boards the VT-d hardware can't identify or control individual MMIO devices that do DMA. Additionally, some boards might not have enough session identifiers (SIDs) to be able to assign a unique SID to every hardware device, so multiple hardware devices may have to share an SID.

Pass-through DMA devices in virtualized systems

In a virtualized system, pass-through devices are devices that are “owned” by a guest running in a VM. A driver in the guest controls the device hardware directly.

For a DMA device to be usable as a pass-through device in a virtualized system, an IOMMU/SMMU is required for the following reasons: