[mirror_ubuntu-bionic-kernel.git] / Documentation / Intel-IOMMU.txt

===================
Linux IOMMU Support
===================

The architecture spec can be obtained from the below location.

http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf

This guide gives a quick cheat sheet for some basic understanding.

Some Keywords

- DMAR - DMA remapping
- DRHD - DMA Remapping Hardware Unit Definition
- RMRR - Reserved memory Region Reporting Structure
- ZLR  - Zero length reads from PCI devices
- IOVA - IO Virtual address.

Basic stuff
-----------

ACPI enumerates and lists the different DMA engines in the platform, and
device scope relationships between PCI devices and which DMA engine  controls
them.

What is RMRR?
-------------

There are some devices the BIOS controls, for e.g USB devices to perform
PS2 emulation. The regions of memory used for these devices are marked
reserved in the e820 map. When we turn on DMA translation, DMA to those
regions will fail. Hence BIOS uses RMRR to specify these regions along with
devices that need to access these regions. OS is expected to setup
unity mappings for these regions for these devices to access these regions.

How is IOVA generated?
----------------------

Well behaved drivers call pci_map_*() calls before sending command to device
that needs to perform DMA. Once DMA is completed and mapping is no longer
required, device performs a pci_unmap_*() calls to unmap the region.

The Intel IOMMU driver allocates a virtual address per domain. Each PCIE
device has its own domain (hence protection). Devices under p2p bridges
share the virtual address with all devices under the p2p bridge due to
transaction id aliasing for p2p bridges.

IOVA generation is pretty generic. We used the same technique as vmalloc()
but these are not global address spaces, but separate for each domain.
Different DMA engines may support different number of domains.

We also allocate guard pages with each mapping, so we can attempt to catch
any overflow that might happen.


Graphics Problems?
------------------
If you encounter issues with graphics devices, you can try adding
option intel_iommu=igfx_off to turn off the integrated graphics engine.
If this fixes anything, please ensure you file a bug reporting the problem.

Some exceptions to IOVA
-----------------------
Interrupt ranges are not address translated, (0xfee00000 - 0xfeefffff).
The same is true for peer to peer transactions. Hence we reserve the
address from PCI MMIO ranges so they are not allocated for IOVA addresses.


Fault reporting
---------------
When errors are reported, the DMA engine signals via an interrupt. The fault
reason and device that caused it with fault reason is printed on console.

See below for sample.


Boot Message Sample
-------------------

Something like this gets printed indicating presence of DMAR tables
in ACPI.

ACPI: DMAR (v001 A M I  OEMDMAR  0x00000001 MSFT 0x00000097) @ 0x000000007f5b5ef0

When DMAR is being processed and initialized by ACPI, prints DMAR locations
and any RMRR's processed::

	ACPI DMAR:Host address width 36
	ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000
	ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000
	ACPI DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000
	ACPI DMAR:RMRR base: 0x00000000000ed000 end: 0x00000000000effff
	ACPI DMAR:RMRR base: 0x000000007f600000 end: 0x000000007fffffff

When DMAR is enabled for use, you will notice..

PCI-DMA: Using DMAR IOMMU

Fault reporting
---------------

::

	DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000
	DMAR:[fault reason 05] PTE Write access is not set
	DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000
	DMAR:[fault reason 05] PTE Write access is not set

TBD
----

- For compatibility testing, could use unity map domain for all devices, just
  provide a 1-1 for all useful memory under a single domain for all devices.
- API for paravirt ops for abstracting functionality for VMM folks.
Commit	Line	Data
45d85146	1	===================
ba395927 KA	2	Linux IOMMU Support
	3	===================
	4
	5	The architecture spec can be obtained from the below location.
	6
6f3cdb38	7	http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf
ba395927 KA	8
	9	This guide gives a quick cheat sheet for some basic understanding.
	10
	11	Some Keywords
	12
45d85146 MCC	13	- DMAR - DMA remapping
	14	- DRHD - DMA Remapping Hardware Unit Definition
	15	- RMRR - Reserved memory Region Reporting Structure
	16	- ZLR - Zero length reads from PCI devices
	17	- IOVA - IO Virtual address.
ba395927 KA	18
	19	Basic stuff
	20	-----------
	21
	22	ACPI enumerates and lists the different DMA engines in the platform, and
	23	device scope relationships between PCI devices and which DMA engine controls
	24	them.
	25
	26	What is RMRR?
	27	-------------
	28
	29	There are some devices the BIOS controls, for e.g USB devices to perform
	30	PS2 emulation. The regions of memory used for these devices are marked
	31	reserved in the e820 map. When we turn on DMA translation, DMA to those
	32	regions will fail. Hence BIOS uses RMRR to specify these regions along with
	33	devices that need to access these regions. OS is expected to setup
	34	unity mappings for these regions for these devices to access these regions.
	35
	36	How is IOVA generated?
45d85146	37	----------------------
ba395927 KA	38
	39	Well behaved drivers call pci_map_*() calls before sending command to device
	40	that needs to perform DMA. Once DMA is completed and mapping is no longer
	41	required, device performs a pci_unmap_*() calls to unmap the region.
	42
	43	The Intel IOMMU driver allocates a virtual address per domain. Each PCIE
	44	device has its own domain (hence protection). Devices under p2p bridges
	45	share the virtual address with all devices under the p2p bridge due to
	46	transaction id aliasing for p2p bridges.
	47
	48	IOVA generation is pretty generic. We used the same technique as vmalloc()
	49	but these are not global address spaces, but separate for each domain.
	50	Different DMA engines may support different number of domains.
	51
d9195881	52	We also allocate guard pages with each mapping, so we can attempt to catch
ba395927 KA	53	any overflow that might happen.
	54
	55
	56	Graphics Problems?
	57	------------------
	58	If you encounter issues with graphics devices, you can try adding
	59	option intel_iommu=igfx_off to turn off the integrated graphics engine.
0c02a20f	60	If this fixes anything, please ensure you file a bug reporting the problem.
e820482c	61
ba395927 KA	62	Some exceptions to IOVA
	63	-----------------------
	64	Interrupt ranges are not address translated, (0xfee00000 - 0xfeefffff).
	65	The same is true for peer to peer transactions. Hence we reserve the
	66	address from PCI MMIO ranges so they are not allocated for IOVA addresses.
	67
3460a6d9 KA	68
	69	Fault reporting
	70	---------------
	71	When errors are reported, the DMA engine signals via an interrupt. The fault
	72	reason and device that caused it with fault reason is printed on console.
	73
	74	See below for sample.
	75
	76
ba395927 KA	77	Boot Message Sample
	78	-------------------
	79
	80	Something like this gets printed indicating presence of DMAR tables
	81	in ACPI.
	82
	83	ACPI: DMAR (v001 A M I OEMDMAR 0x00000001 MSFT 0x00000097) @ 0x000000007f5b5ef0
	84
	85	When DMAR is being processed and initialized by ACPI, prints DMAR locations
45d85146	86	and any RMRR's processed::
ba395927	87
45d85146 MCC	88	ACPI DMAR:Host address width 36
	89	ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000
	90	ACPI DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed91000
	91	ACPI DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000
	92	ACPI DMAR:RMRR base: 0x00000000000ed000 end: 0x00000000000effff
	93	ACPI DMAR:RMRR base: 0x000000007f600000 end: 0x000000007fffffff
ba395927 KA	94
	95	When DMAR is enabled for use, you will notice..
	96
	97	PCI-DMA: Using DMAR IOMMU
	98
3460a6d9 KA	99	Fault reporting
	100	---------------
	101
45d85146 MCC	102	::
	103
	104	DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000
	105	DMAR:[fault reason 05] PTE Write access is not set
	106	DMAR:[DMA Write] Request device [00:02.0] fault addr 6df084000
	107	DMAR:[fault reason 05] PTE Write access is not set
3460a6d9	108
ba395927 KA	109	TBD
	110	----
	111
	112	- For compatibility testing, could use unity map domain for all devices, just
	113	provide a 1-1 for all useful memory under a single domain for all devices.
d9195881	114	- API for paravirt ops for abstracting functionality for VMM folks.