Commit graph

222 commits

Author SHA1 Message Date
Linus Torvalds
dcded10f6d Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx
* 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx: (30 commits)
  DMAENGINE: at_hdmac: locking fixlet
  DMAENGINE: pch_dma: kill another usage of __raw_{read|write}l
  dma: dmatest: fix potential sign bug
  ioat2: catch and recover from broken vtd configurations v6
  DMAENGINE: add runtime slave control to COH 901 318 v3
  DMAENGINE: add runtime slave config to DMA40 v3
  DMAENGINE: generic slave channel control v3
  dmaengine: Driver for Topcliff PCH DMA controller
  intel_mid: Add Mrst & Mfld DMA Drivers
  drivers/dma: Eliminate a NULL pointer dereference
  dma/timb_dma: compile warning on 32 bit
  DMAENGINE: ste_dma40: support older silicon
  DMAENGINE: ste_dma40: support disabling physical channels
  DMAENGINE: ste_dma40: no disabled phy channels on ux500
  DMAENGINE: ste_dma40: fix suspend bug
  DMAENGINE: ste_dma40: add DB8500 memcpy channels
  DMAENGINE: ste_dma40: no flow control on memcpy
  DMAENGINE: ste_dma40: arch updates for LCLA and LCPA
  DMAENGINE: ste_dma40: allocate LCLA dynamically
  DMAENGINE: ste_dma40: no premature stop
  ...

Fix up trivial conflicts in arch/arm/mach-ux500/devices-db8500.c
2010-08-09 21:00:07 -07:00
Dan Williams
556ab45f9a ioat2: catch and recover from broken vtd configurations v6
On some platforms (MacPro3,1) the BIOS assigns the ioatdma device to the
incorrect iommu causing faults when the driver initializes.  Add a quirk
to catch this misconfiguration and try falling back to untranslated
operation (which works in the MacPro3,1 case).

Assuming there are other platforms with misconfigured iommus teach the
ioatdma driver to treat initialization failures as non-fatal (just fail
the driver load and emit a warning instead of triggering a BUG_ON).

This can be classified as a boot regression since 2.6.32 on affected
platforms since the ioatdma module did not autoload prior to that
kernel.

Cc: <stable@kernel.org>
Acked-by: David Woodhouse <David.Woodhouse@intel.com>
Reported-by: Chris Li <lkml@chrisli.org>
Tested-by: Chris Li <lkml@chrisli.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2010-08-04 14:18:17 -07:00
Tom Lyon
323f99cbc3 iommu-api: Extension to check for interrupt remapping
This patch allows IOMMU users to determine whether the
hardware and software support safe, isolated interrupt
remapping.  Not all Intel IOMMUs have the hardware, and the
software for AMD is not there yet.

Signed-off-by: Tom Lyon <pugs@cisco.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2010-07-19 15:44:25 +02:00
David Woodhouse
2d9e667efd intel-iommu: Force-disable IOMMU for iGFX on broken Cantiga revisions.
Certain revisions of this chipset appear to be broken. There is a shadow
GTT which mirrors the real GTT but contains pre-translated physical
addresses, for performance reasons. When a GTT update happens, the
translations are done once and the resulting physical addresses written
back to the shadow GTT.

Except sometimes, the physical address is actually written back to the
_real_ GTT, not the shadow GTT. Thus we start to see faults when that
physical address is fed through translation again.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-06-15 10:57:57 +01:00
Jiri Slaby
00dfff77e7 intel-iommu: Fix double lock in get_domain_for_dev()
stanse found the following double lock.

In get_domain_for_dev:
  spin_lock_irqsave(&device_domain_lock, flags);
  domain_exit(domain);
    domain_remove_dev_info(domain);
      spin_lock_irqsave(&device_domain_lock, flags);
      spin_unlock_irqrestore(&device_domain_lock, flags);
  spin_unlock_irqrestore(&device_domain_lock, flags);

This happens when the domain is created by another CPU at the same time 
as this function is creating one, and the other CPU wins the race to 
attach it to the device in question, so we have to destroy our own 
newly-created one.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-06-15 10:50:46 +01:00
Sheng Yang
25cbff1660 intel-iommu: Fix reference by physical address in intel_iommu_attach_device()
Commit a99c47a2 "intel-iommu: errors with smaller iommu widths" replace the
dmar_domain->pgd with the first entry of page table when iommu's supported
width is smaller than dmar_domain's. But it use physical address directly
for new dmar_domain->pgd...

This result in KVM oops with VT-d on some machines.

Reported-by: Allen Kay <allen.m.kay@intel.com>
Cc: Tom Lyon <pugs@cisco.com>
Signed-off-by: Sheng Yang <sheng@linux.intel.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-06-15 10:40:02 +01:00
Linus Torvalds
0961d6581c Merge git://git.infradead.org/iommu-2.6
* git://git.infradead.org/iommu-2.6:
  intel-iommu: Set a more specific taint flag for invalid BIOS DMAR tables
  intel-iommu: Combine the BIOS DMAR table warning messages
  panic: Add taint flag TAINT_FIRMWARE_WORKAROUND ('I')
  panic: Allow warnings to set different taint flags
  intel-iommu: intel_iommu_map_range failed at very end of address space
  intel-iommu: errors with smaller iommu widths
  intel-iommu: Fix boot inside 64bit virtualbox with io-apic disabled
  intel-iommu: use physfn to search drhd for VF
  intel-iommu: Print out iommu seq_id
  intel-iommu: Don't complain that ACPI_DMAR_SCOPE_TYPE_IOAPIC is not supported
  intel-iommu: Avoid global flushes with caching mode.
  intel-iommu: Use correct domain ID when caching mode is enabled
  intel-iommu mistakenly uses offset_pfn when caching mode is enabled
  intel-iommu: use for_each_set_bit()
  intel-iommu: Fix section mismatch dmar_ir_support() uses dmar_tbl.
2010-05-21 17:25:01 -07:00
Tom Lyon
8954da1f82 intel-iommu: intel_iommu_map_range failed at very end of address space
intel_iommu_map_range() doesn't allow allocation at the very end of the
address space; that code has been simplified and corrected.

Signed-off-by: Tom Lyon <pugs@cisco.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-05-17 08:34:20 +01:00
Tom Lyon
a99c47a228 intel-iommu: errors with smaller iommu widths
When using iommu_domain_alloc with the Intel iommu, the domain address
width is always initialized to 48 bits (agaw 2).  This domain->agaw value
is then used by pfn_to_dma_pte to (always) build a 4 level page table.
However, not all systems support iommu width of 48 or 4 level page tables.
In particular, the Core i5-660 and i5-670 support an address width of 36
bits (not 39!), an agaw of only 1, and only 3 level page tables.

This version of the patch simply lops off extra levels of the page tables
if the agaw value of the iommu is less than what is currently allocated
for the domain (in intel_iommu_attach_device).  If there were already
allocated addresses above what the new iommu can handle, EFAULT is
returned.

Signed-off-by: Tom Lyon <pugs@cisco.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-05-17 08:34:18 +01:00
Yinghai Lu
680a752462 intel-iommu: Print out iommu seq_id
more info on system with more than one IOMMU

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-04-09 16:33:13 +01:00
Nadav Amit
78d5f0f500 intel-iommu: Avoid global flushes with caching mode.
While it may be efficient on real hardware, emulation of global
invalidations is very expensive as all shadow entries must be examined.
This patch changes the behaviour when caching mode is enabled (which is
the case when IOMMU emulation takes place). In this case, page specific
invalidation is used instead.

Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-04-09 15:54:41 +01:00
Nadav Amit
82653633b6 intel-iommu: Use correct domain ID when caching mode is enabled
In caching-mode mappings of pages (changes from non-present to present)
require invalidation.
Currently, this IOTLB flush is performed with domain ID of zero.
This is not according to the VT-d spec and causes big problems for
emulating software.
This patch uses the correct domain ID in IOTLB flushes.
Device IOTLB invalidation is performed only on present to non-present
changes. This decision is now based on explicit parameter instead of
zero domain-ID.

Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-04-09 15:39:21 +01:00
Nadav Amit
8bdd77dd4e intel-iommu mistakenly uses offset_pfn when caching mode is enabled
intel_map_sg used offset_pfn which was set to zero when invalidating the IOTLB.
intel_map_sg now uses size variable for this matter.

Signed-off-by: Nadav Amit <nadav.amit@gmail.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-04-09 15:38:47 +01:00
Akinobu Mita
a45946abb8 intel-iommu: use for_each_set_bit()
Replace open-coded loop with for_each_set_bit().

Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2010-04-09 15:24:11 +01:00
Joerg Roedel
b146a1c9f7 VT-d: Change {un}map_range functions to implement {un}map interface
This patch changes the iommu-api functions for mapping and
unmapping page ranges to use the new page-size based
interface. This allows to remove the range based functions
later.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2010-03-07 18:01:11 +01:00
Joerg Roedel
4abc14a733 iommu-api: Rename ->{un}map function pointers to ->{un}map_range
The new function pointer names match better with the
top-level functions of the iommu-api which are using them.
Main intention of this change is to make the ->{un}map
pointer names free for two new mapping functions.

Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2010-03-07 18:01:11 +01:00
Linus Torvalds
d661d76b02 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6:
  PCI/cardbus: Add a fixup hook and fix powerpc
  PCI: change PCI nomenclature in drivers/pci/ (non-comment changes)
  PCI: change PCI nomenclature in drivers/pci/ (comment changes)
  PCI: fix section mismatch on update_res()
  PCI: add Intel 82599 Virtual Function specific reset method
  PCI: add Intel USB specific reset method
  PCI: support device-specific reset methods
  PCI: Handle case when no pci device can provide cache line size hint
  PCI/PM: Propagate wake-up enable for PCIe devices too
  vgaarbiter: fix a typo in the vgaarbiter Documentation
2009-12-30 13:13:24 -08:00
Stefan Assmann
45e829ea41 PCI: change PCI nomenclature in drivers/pci/ (comment changes)
Changing occurrences of variants of PCI-X and PCIe to the PCI-SIG
terms listed in the "Trademark and Logo Usage Guidelines".
http://www.pcisig.com/developers/procedures/logos/Trademark_and_Logo_Usage_Guidelines_updated_112206.pdf

Patch is limited to drivers/pci/ and changes concern comments only.

Signed-off-by: Stefan Assmann <sassmann@redhat.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2009-12-16 13:37:53 -08:00
Linus Torvalds
a79960e576 Merge git://git.infradead.org/iommu-2.6
* git://git.infradead.org/iommu-2.6:
  implement early_io{re,un}map for ia64
  Revert "Intel IOMMU: Avoid memory allocation failures in dma map api calls"
  intel-iommu: ignore page table validation in pass through mode
  intel-iommu: Fix oops with intel_iommu=igfx_off
  intel-iommu: Check for an RMRR which ends before it starts.
  intel-iommu: Apply BIOS sanity checks for interrupt remapping too.
  intel-iommu: Detect DMAR in hyperspace at probe time.
  dmar: Fix build failure without NUMA, warn on bogus RHSA tables and don't abort
  iommu: Allocate dma-remapping structures using numa locality info
  intr_remap: Allocate intr-remapping table using numa locality info
  dmar: Allocate queued invalidation structure using numa locality info
  dmar: support for parsing Remapping Hardware Static Affinity structure
2009-12-16 10:11:38 -08:00
Linus Torvalds
11bd04f6f3 Merge branch 'linux-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6
* 'linux-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6: (109 commits)
  PCI: fix coding style issue in pci_save_state()
  PCI: add pci_request_acs
  PCI: fix BUG_ON triggered by logical PCIe root port removal
  PCI: remove ifdefed pci_cleanup_aer_correct_error_status
  PCI: unconditionally clear AER uncorr status register during cleanup
  x86/PCI: claim SR-IOV BARs in pcibios_allocate_resource
  PCI: portdrv: remove redundant definitions
  PCI: portdrv: remove unnecessary struct pcie_port_data
  PCI: portdrv: minor cleanup for pcie_port_device_register
  PCI: portdrv: add missing irq cleanup
  PCI: portdrv: enable device before irq initialization
  PCI: portdrv: cleanup service irqs initialization
  PCI: portdrv: check capabilities first
  PCI: portdrv: move PME capability check
  PCI: portdrv: remove redundant pcie type calculation
  PCI: portdrv: cleanup pcie_device registration
  PCI: portdrv: remove redundant pcie_port_device_probe
  PCI: Always set prefetchable base/limit upper32 registers
  PCI: read-modify-write the pcie device control register when initiating pcie flr
  PCI: show dma_mask bits in /sys
  ...

Fixed up conflicts in:
	arch/x86/kernel/amd_iommu_init.c
	drivers/pci/dmar.c
	drivers/pci/hotplug/acpiphp_glue.c
2009-12-11 12:18:16 -08:00
KOSAKI Motohiro
354bb65e6e Revert "Intel IOMMU: Avoid memory allocation failures in dma map api calls"
commit eb3fa7cb51 said Intel IOMMU

    Intel IOMMU driver needs memory during DMA map calls to setup its
    internal page tables and for other data structures.  As we all know
    that these DMA map calls are mostly called in the interrupt context
    or with the spinlock held by the upper level drivers(network/storage
    drivers), so in order to avoid any memory allocation failure due to
    low memory issues, this patch makes memory allocation by temporarily
    setting PF_MEMALLOC flags for the current task before making memory
    allocation calls.

    We evaluated mempools as a backup when kmem_cache_alloc() fails
    and found that mempools are really not useful here because
     1) We don't know for sure how much to reserve in advance
     2) And mempools are not useful for GFP_ATOMIC case (as we call
        memory alloc functions with GFP_ATOMIC)

    (akpm: point 2 is wrong...)

The above description doesn't justify to waste system emergency memory
at all. Non MM subsystem must not use PF_MEMALLOC. Memory reclaim need
few memory, anyone must not prevent it. Otherwise the system cause
mysterious hang-up and/or OOM Killer invokation.

Plus, akpm already pointed out what we should do.

Then, this patch revert it.

Cc: Keshavamurthy Anil S <anil.s.keshavamurthy@intel.com>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-12-08 10:12:04 +00:00
Chris Wright
1672af1164 intel-iommu: ignore page table validation in pass through mode
We are seeing a bug when booting w/ iommu=pt with current upstream
(bisect blames 19943b0e30 "intel-iommu:
Unify hardware and software passthrough support).

The issue is specific to this loop during identity map initialization
of each device:

domain_context_mapping_one(si_domain, ..., CONTEXT_TT_PASS_THROUGH)
...
		/* Skip top levels of page tables for
		* iommu which has less agaw than default.
		*/
		for (agaw = domain->agaw; agaw != iommu->agaw; agaw--) {
			pgd = phys_to_virt(dma_pte_addr(pgd));
			if (!dma_pte_present(pgd)) {      <------ failing here
				spin_unlock_irqrestore(&iommu->lock, flags);
			return -ENOMEM;
		}

This box has 2 iommu's in it.  The catchall iommu has MGAW == 48, and
SAGAW == 4.  The other iommu has MGAW == 39, SAGAW == 2.

The device that's failing the above pgd test is the only device connected
to the non-catchall iommu, which has a smaller address width than the
domain default.  This test is not necessary since the context is in PT
mode and the ASR is ignored.

Thanks to Don Dutile for discovering and debugging this one.

Cc: stable@kernel.org
Signed-off-by: Chris Wright <chrisw@sous-sol.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-12-08 10:03:25 +00:00
David Woodhouse
44cd613c0e intel-iommu: Fix oops with intel_iommu=igfx_off
The hotplug notifier will call find_domain() to see if the device in
question has been assigned an IOMMU domain. However, this should never
be called for devices with a "dummy" domain, such as graphics devices
when intel_iommu=igfx_off is set and the corresponding IOMMU isn't even
initialised. If you do that, it'll oops as it dereferences the (-1)
pointer.

The notifier function should check iommu_no_mapping() for the
device before doing anything else.

Cc: stable@kernel.org
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-12-08 10:03:06 +00:00
David Woodhouse
5595b528b4 intel-iommu: Check for an RMRR which ends before it starts.
Some HP BIOSes report an RMRR region (a region which needs a 1:1 mapping
in the IOMMU for a given device) which has an end address lower than its
start address. Detect that and warn, rather than triggering the
BUG() in dma_pte_clear_range().

Cc: stable@kernel.org
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-12-08 10:02:52 +00:00
David Woodhouse
ec20849193 Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
Merge the BIOS workarounds from 2.6.32, and the swiotlb fallback on failure.
2009-12-08 09:59:24 +00:00
Linus Torvalds
7b626acb8f Merge branch 'core-iommu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
* 'core-iommu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip: (63 commits)
  x86, Calgary IOMMU quirk: Find nearest matching Calgary while walking up the PCI tree
  x86/amd-iommu: Remove amd_iommu_pd_table
  x86/amd-iommu: Move reset_iommu_command_buffer out of locked code
  x86/amd-iommu: Cleanup DTE flushing code
  x86/amd-iommu: Introduce iommu_flush_device() function
  x86/amd-iommu: Cleanup attach/detach_device code
  x86/amd-iommu: Keep devices per domain in a list
  x86/amd-iommu: Add device bind reference counting
  x86/amd-iommu: Use dev->arch->iommu to store iommu related information
  x86/amd-iommu: Remove support for domain sharing
  x86/amd-iommu: Rearrange dma_ops related functions
  x86/amd-iommu: Move some pte allocation functions in the right section
  x86/amd-iommu: Remove iommu parameter from dma_ops_domain_alloc
  x86/amd-iommu: Use get_device_id and check_device where appropriate
  x86/amd-iommu: Move find_protection_domain to helper functions
  x86/amd-iommu: Simplify get_device_resources()
  x86/amd-iommu: Let domain_for_device handle aliases
  x86/amd-iommu: Remove iommu specific handling from dma_ops path
  x86/amd-iommu: Remove iommu parameter from __(un)map_single
  x86/amd-iommu: Make alloc_new_range aware of multiple IOMMUs
  ...
2009-12-05 09:49:07 -08:00
Kenji Kaneshige
5f4d91a122 PCI: use pci_is_pcie() in pci core
Change for PCI core to use pci_is_pcie() instead of checking
pci_dev->is_pcie.

Signed-off-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>
2009-11-24 15:25:16 -08:00
Fenghua Yu
99dcadede4 intel-iommu: Support PCIe hot-plug
To support PCIe hot plug in IOMMU, we register a notifier to respond to device
change action.

When the notifier gets BUS_NOTIFY_UNBOUND_DRIVER, it removes the device
from its DMAR domain.

A hot added device will be added into an IOMMU domain when it first does IOMMU
op. So there is no need to add more code for hot add.

Without the patch, after a hot-remove, a hot-added device on the same
slot will not work.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Tested-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-11-12 02:28:45 +00:00
Alex Williamson
e8bb910d1b intel-iommu: Obey coherent_dma_mask for alloc_coherent on passthrough
The model for IOMMU passthrough is that decent devices that can cope
with DMA to all of memory get passthrough; crappy devices with a limited
dma_mask don't -- they get to use the IOMMU anyway.

This is done on the basis that IOMMU passthrough is usually wanted for
performance reasons, and it's only the decent PCI devices that you
really care about performance for, while the crappy 32-bit ones like
your USB controller can just use the IOMMU and you won't really care.

Unfortunately, the check for this was only looking at dev->dma_mask, not
at dev->coherent_dma_mask. And some devices have a 32-bit
coherent_dma_mask even though they have a full 64-bit dma_mask.

Even more unfortunately, fixing that simple oversight would upset
certain broken HP devices. Not only do they have a 32-bit
coherent_dma_mask, but they also have a tendency to do stray DMA to
unmapped addresses. And then they die when they take the DMA fault they
so richly deserve.

So if we do the 'correct' fix, it'll mean that affected users have to
disable IOMMU support completely on "a large percentage of servers from
a major vendor."

Personally, I have little sympathy -- given that this is the _same_
'major vendor' who is shipping machines which claim to have IOMMU
support but have obviously never _once_ booted a VT-d capable OS to do
any form of QA. But strictly speaking, it _would_ be a regression even
though it only ever worked by fluke.

For 2.6.33, we'll come up with a quirk which gives swiotlb support
for this particular device, and other devices with an inadequate
coherent_dma_mask will just get normal IOMMU mapping.

The simplest fix for 2.6.32, though, is just to jump through some hoops
to try to allocate coherent DMA memory for such devices in a place that
they can reach. We'd use dma_generic_alloc_coherent() for this if it
existed on IA64.

Signed-off-by: Alex Williamson <alex.williamson@hp.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-11-12 02:10:34 +00:00
FUJITA Tomonori
75f1cdf1dd x86: Handle HW IOMMU initialization failure gracefully
If HW IOMMU initialization fails (Intel VT-d often does this,
typically due to BIOS bugs), we fall back to nommu. It doesn't
work for the majority since nowadays we have more than 4GB
memory so we must use swiotlb instead of nommu.

The problem is that it's too late to initialize swiotlb when HW
IOMMU initialization fails. We need to allocate swiotlb memory
earlier from bootmem allocator. Chris explained the issue in
detail:

  http://marc.info/?l=linux-kernel&m=125657444317079&w=2

The current x86 IOMMU initialization sequence is too complicated
and handling the above issue makes it more hacky.

This patch changes x86 IOMMU initialization sequence to handle
the above issue cleanly.

The new x86 IOMMU initialization sequence are:

1. we initialize the swiotlb (and setting swiotlb to 1) in the case
   of (max_pfn > MAX_DMA32_PFN && !no_iommu). dma_ops is set to
   swiotlb_dma_ops or nommu_dma_ops. if swiotlb usage is forced by
   the boot option, we finish here.

2. we call the detection functions of all the IOMMUs

3. the detection function sets x86_init.iommu.iommu_init to the
   IOMMU initialization function (so we can avoid calling the
   initialization functions of all the IOMMUs needlessly).

4. if the IOMMU initialization function doesn't need to swiotlb
   then sets swiotlb to zero (e.g. the initialization is
   sucessful).

5. if we find that swiotlb is set to zero, we free swiotlb
   resource.

Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: chrisw@sous-sol.org
Cc: dwmw2@infradead.org
Cc: joerg.roedel@amd.com
Cc: muli@il.ibm.com
LKML-Reference: <1257849980-22640-10-git-send-email-fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-11-10 12:32:07 +01:00
Suresh Siddha
4c923d4714 iommu: Allocate dma-remapping structures using numa locality info
Allocate dma-remapping structures using numa locality info. On platforms
having remapping hardware units span different nodes, this enables
optimized dma-remapping transalation structures access by remapping hardware.

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-10-05 07:55:33 +01:00
David Woodhouse
e0fc7e0b4b intel-iommu: Yet another BIOS workaround: Isoch DMAR unit with no TLB space
Asus decided to ship a BIOS which configures sound DMA to go via the
dedicated IOMMU unit, but assigns precisely zero TLB entries to that
unit. Which causes the whole thing to deadlock, including the DMA
traffic on the _other_ IOMMU units. Nice one.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-09-30 09:12:17 -07:00
Linus Torvalds
b09a75fc5e Merge git://git.infradead.org/iommu-2.6
* git://git.infradead.org/iommu-2.6: (23 commits)
  intel-iommu: Disable PMRs after we enable translation, not before
  intel-iommu: Kill DMAR_BROKEN_GFX_WA option.
  intel-iommu: Fix integer wrap on 32 bit kernels
  intel-iommu: Fix integer overflow in dma_pte_{clear_range,free_pagetable}()
  intel-iommu: Limit DOMAIN_MAX_PFN to fit in an 'unsigned long'
  intel-iommu: Fix kernel hang if interrupt remapping disabled in BIOS
  intel-iommu: Disallow interrupt remapping if not all ioapics covered
  intel-iommu: include linux/dmi.h to use dmi_ routines
  pci/dmar: correct off-by-one error in dmar_fault()
  intel-iommu: Cope with yet another BIOS screwup causing crashes
  intel-iommu: iommu init error path bug fixes
  intel-iommu: Mark functions with __init
  USB: Work around BIOS bugs by quiescing USB controllers earlier
  ia64: IOMMU passthrough mode shouldn't trigger swiotlb init
  intel-iommu: make domain_add_dev_info() call domain_context_mapping()
  intel-iommu: Unify hardware and software passthrough support
  intel-iommu: Cope with broken HP DC7900 BIOS
  iommu=pt is a valid early param
  intel-iommu: double kfree()
  intel-iommu: Kill pointless intel_unmap_single() function
  ...

Fixed up trivial include lines conflict in drivers/pci/intel-iommu.c
2009-09-23 10:06:10 -07:00
David Woodhouse
b94996c99c intel-iommu: Disable PMRs after we enable translation, not before
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-09-19 15:28:12 -07:00
Benjamin LaHaise
64de5af000 intel-iommu: Fix integer wrap on 32 bit kernels
The following 64 bit promotions are necessary to handle memory above the
4GiB boundary correctly.

[dwmw2: Fix the second part not to need 64-bit arithmetic at all]

Signed-off-by: Benjamin LaHaise <ben.lahaise@neterion.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-09-19 09:26:20 -07:00
David Woodhouse
59c36286b7 intel-iommu: Fix integer overflow in dma_pte_{clear_range,free_pagetable}()
If end_pfn is equal to (unsigned long)-1, then the loop will never end.

Seen on 32-bit kernel, but could have happened on 64-bit too once we get
hardware that supports 64-bit guest addresses.

Change both functions to a 'do {} while' loop with the test at the end,
and check for the PFN having wrapper round to zero.

Reported-by: Benjamin LaHaise <ben.lahaise@neterion.com>
Tested-by: Benjamin LaHaise <ben.lahaise@neterion.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-09-19 07:36:28 -07:00
David Woodhouse
2ebe31513f intel-iommu: Limit DOMAIN_MAX_PFN to fit in an 'unsigned long'
This means we're limited to 44-bit addresses on 32-bit kernels, and
makes it sane for us to use 'unsigned long' for PFNs throughout.

Which is just as well, really, since we already do that.

Reported-by: Benjamin LaHaise <ben.lahaise@neterion.com>
Tested-by: Benjamin LaHaise <ben.lahaise@neterion.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-09-19 07:34:04 -07:00
Ingo Molnar
936e894a97 Merge commit 'v2.6.31-rc8' into x86/txt
Conflicts:
	arch/x86/kernel/reboot.c
	security/Kconfig

Merge reason: resolve the conflicts, bump up from rc3 to rc8.

Signed-off-by: Ingo Molnar <mingo@elte.hu>
2009-09-02 08:17:56 +02:00
Shane Wang
69575d3886 x86, intel_txt: clean up the impact on generic code, unbreak non-x86
Move tboot.h from asm to linux to fix the build errors of intel_txt
patch on non-X86 platforms. Remove the tboot code from generic code
init/main.c and kernel/cpu.c.

Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2009-09-01 18:25:07 -07:00
Stephen Rothwell
adb2fe0277 intel-iommu: include linux/dmi.h to use dmi_ routines
This file needs to include linux/dmi.h directly rather than relying on
it being pulled in from elsewhere.

Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-31 10:50:01 +01:00
David Woodhouse
2ff729f544 intel-iommu: Cope with yet another BIOS screwup causing crashes
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-26 14:25:41 +01:00
Donald Dutile
94a91b5051 intel-iommu: iommu init error path bug fixes
The kcalloc() failure path in iommu_init_domains() calls
free_dmar_iommu(), which assumes that ->domains, ->domain_ids,
and ->lock have been properly initialized.

Add checks in free_[dmar]_iommu to not use ->domains,->domain_ids
if not alloced. Move the lock init to prior to the kcalloc()'s,
so it is valid in free_context_table() when free_dmar_iommu() invokes
it at the end.

Patch based on iommu-2.6,
commit 132032274a

Signed-off-by: Donald Dutile <ddutile@redhat.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-24 08:56:26 +01:00
Matt Kraai
071e13746f intel-iommu: Mark functions with __init
Mark si_domain_init and iommu_prepare_static_identity_mapping with
__init, to eliminate the following warnings:

WARNING: drivers/pci/built-in.o(.text+0xf1f4): Section mismatch in reference from the function si_domain_init() to the function .init.text:si_domain_work_fn()
The function si_domain_init() references
the function __init si_domain_work_fn().
This is often because si_domain_init lacks a __init
annotation or the annotation of si_domain_work_fn is wrong.

WARNING: drivers/pci/built-in.o(.text+0xe340): Section mismatch in reference from the function iommu_prepare_static_identity_mapping() to the function .init.text:si_domain_init()
The function iommu_prepare_static_identity_mapping() references
the function __init si_domain_init().
This is often because iommu_prepare_static_identity_mapping lacks a __init
annotation or the annotation of si_domain_init is wrong.

Signed-off-by: Matt Kraai <kraai@ftbfs.org>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-24 08:54:34 +01:00
David Woodhouse
5fe60f4e58 intel-iommu: make domain_add_dev_info() call domain_context_mapping()
All callers of the former were also calling the latter, in one order or
the other, and failing to correctly clean up if the second returned
failure.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-09 10:53:50 +01:00
David Woodhouse
a131bc1855 Merge branch 'master' of /pub/scm/linux/kernel/git/torvalds/linux-2.6
Pull fixes in from 2.6.31 so that people testing the iommu-2.6.git tree
no longer trip over bugs which were already fixed (sorry, Horms).
2009-08-08 11:26:15 +01:00
Sheng Yang
c5b1525533 intel-iommu: Fix enabling snooping feature by mistake
Two defects work together result in KVM device passthrough randomly can't
work:
1. iommu_snooping is not initialized to zero when vm_iommu_init() called.
So it is possible to get a random value.
2. One line added by commit 2c2e2c38("IOMMU Identity Mapping Support")
change the code path, let it bypass domain_update_iommu_cap(), as well as
missing the increment of domain iommu reference count.

The latter is also likely to cause a leak of domains on repeated VMM 
assignment and deassignment.

Signed-off-by: Sheng Yang <sheng@linux.intel.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-06 11:35:50 +01:00
Fenghua Yu
33041ec049 intel-iommu: Mask physical address to correct page size in intel_map_single()
The physical address passed to domain_pfn_mapping() should be rounded 
down to the start of the MM page, not the VT-d page.

This issue causes kernel panic on PAGE_SIZE>VTD_PAGE_SIZE platforms e.g. ia64
platforms.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-05 09:15:48 +01:00
Fenghua Yu
f532959b77 intel-iommu: Correct sglist size calculation.
In domain_sg_mapping(), use aligned_nrpages() instead of hand-coded
rounding code for calculating the size of each sg elem. This means that
on IA64 we correctly round up to the MM page size, not just to the VT-d
page size.

Also remove the incorrect mm_to_dma_pfn() when intel_map_sg() calls
domain_sg_mapping() -- the 'size' variable is in VT-d pages already.

Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-05 08:59:47 +01:00
David Woodhouse
19943b0e30 intel-iommu: Unify hardware and software passthrough support
This makes the hardware passthrough mode work a lot more like the
software version, so that the behaviour of a kernel with 'iommu=pt'
is the same whether the hardware supports passthrough or not.

In particular:
 - We use a single si_domain for the pass-through devices.
 - 32-bit devices can be taken out of the pass-through domain so that
   they don't have to use swiotlb.
 - Devices will work again after being removed from a KVM guest.
 - A potential oops on OOM (in init_context_pass_through()) is fixed.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
2009-08-04 16:19:23 +01:00
Joseph Cihula
a59b50e995 intel_txt: Force IOMMU on for Intel TXT launch
The tboot module will DMA protect all of memory in order to ensure the that
kernel will be able to initialize without compromise (from DMA).  Consequently,
the kernel must enable Intel Virtualization Technology for Directed I/O
(VT-d or Intel IOMMU) in order to replace this broad protection with the
appropriate page-granular protection.  Otherwise DMA devices will be unable
to read or write from memory and the kernel will eventually panic.

Because runtime IOMMU support is configurable by command line options, this
patch will force it to be enabled regardless of the options specified, and will
log a message if it was required to force it on.

 dmar.c        |    7 +++++++
 intel-iommu.c |   17 +++++++++++++++--
 2 files changed, 22 insertions(+), 2 deletions(-)

Signed-off-by: Joseph Cihula <joseph.cihula@intel.com>
Signed-off-by: Shane Wang <shane.wang@intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: H. Peter Anvin <hpa@zytor.com>
2009-07-21 11:50:42 -07:00