On xsc3, L2 cache ops are possible only on virtual addresses. The code
is rearranged so to have a linear progression requiring the least amount
of pte setups in the highmem case. To protect the virtual mapping so
created, interrupts must be disabled currently up to a page worth of
address range.
The interrupt disabling is done in a way to minimize the overhead within
the inner loop. The alternative would consist in separate code for
the highmem and non highmem compilation which is less preferable.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
The choice is between looping over the physical range and performing
single cache line operations, or to map highmem pages somewhere, as
cache range ops are possible only on virtual addresses.
Because L2 range ops are much faster, we go with the later by factoring
the physical-to-virtual address conversion and use a fixmap entry for it
in the HIGHMEM case.
Possible future optimizations to avoid the pte setup cost:
- do the pte setup for highmem pages only
- determine a threshold for doing a line-by-line processing on physical
addresses when the range is small
Signed-off-by: Nicolas Pitre <nico@marvell.com>
This is a helper to be used by the DMA mapping API to handle cache
maintenance for memory identified by a page structure instead of a
virtual address. Those pages may or may not be highmem pages, and
when they're highmem pages, they may or may not be virtually mapped.
When they're not mapped then there is no L1 cache to worry about. But
even in that case the L2 cache must be processed since unmapped highmem
pages can still be L2 cached.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
The kmap virtual area borrows a 2MB range at the top of the 16MB area
below PAGE_OFFSET currently reserved for kernel modules and/or the
XIP kernel. This 2MB corresponds to the range covered by 2 consecutive
second-level page tables, or a single pmd entry as seen by the Linux
page table abstraction. Because XIP kernels are unlikely to be seen
on systems needing highmem support, there shouldn't be any shortage of
VM space for modules (14 MB for modules is still way more than twice the
typical usage).
Because the virtual mapping of highmem pages can go away at any moment
after kunmap() is called on them, we need to bypass the delayed cache
flushing provided by flush_dcache_page() in that case.
The atomic kmap versions are based on fixmaps, and
__cpuc_flush_dcache_page() is used directly in that case.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
This is the minimum fixmap interface expected to be implemented by
architectures supporting highmem.
We have a second level page table already allocated and covering
0xfff00000-0xffffffff because the exception vector page is located
at 0xffff0000, and various cache tricks already use some entries above
0xffff0000. Therefore the PTEs covering 0xfff00000-0xfffeffff are free
to be used.
However the XScale cache flushing code already uses virtual addresses
between 0xfffe0000 and 0xfffeffff.
So this reserves the 0xfff00000-0xfffdffff range for fixmap stuff.
The Documentation/arm/memory.txt information is updated accordingly,
including the information about the actual top of DMA memory mapping
region which didn't match the code.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
The current use of these macros works well when the conversion is
entirely linear. In this case, we can be assured that the following
holds true:
__va(p + s) - s = __va(p)
However, this is not always the case, especially when there is a
non-linear conversion (eg, when there is a 3.5GB hole in memory.)
In this case, if 's' is the size of the region (eg, PAGE_SIZE) and
'p' is the final page, the above is most definitely not true.
So, we must ensure that __va() and __pa() are only used with valid
kernel direct mapped RAM addresses. This patch tweaks the code
to achieve this.
Tested-by: Charles Moschel <fred99@carolina.rr.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This is a fix for the following crash observed in 2.6.29-rc3:
http://lkml.org/lkml/2009/1/29/150
On ARM it doesn't make sense to trace a naked function because then
mcount is called without stack and frame pointer being set up and there
is no chance to restore the lr register to the value before mcount was
called.
Reported-by: Matthias Kaehlcke <matthias@kaehlcke.net>
Tested-by: Matthias Kaehlcke <matthias@kaehlcke.net>
Cc: Abhishek Sagar <sagar.abhishek@gmail.com>
Cc: Steven Rostedt <rostedt@home.goodmis.org>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
The target of the strex instruction to clear the exlusive monitor
is currently the top of the stack. If the store succeeeds this
corrupts r0 in pt_regs. Use the next stack location instead of
the current one to prevent any chance of corrupting an in-use
address.
Signed-off-by: Seth Forshee <seth.forshee@gmail.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
In the non highmem case, if two memory banks of 1GB each are provided,
the second bank would evade suppression since its virtual base would
be 0. Fix this by disallowing any memory bank which virtual base
address is found to be lower than PAGE_OFFSET.
Reported-by: Lennert Buytenhek <buytenh@marvell.com>
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
When there are multiple L1-aliasing userland mappings of the same physical
page, we currently remap each of them uncached, to prevent VIVT cache
aliasing issues. (E.g. writes to one of the mappings not being immediately
visible via another mapping.) However, when we do this remapping, there
could still be stale data in the L2 cache, and an uncached mapping might
bypass L2 and go straight to RAM. This would cause reads from such
mappings to see old data (until the dirty L2 line is eventually evicted.)
This issue is solved by forcing a L2 cache flush whenever the shared page
is made L1 uncacheable.
Ideally, we would make L1 uncacheable and L2 cacheable as L2 is PIPT. But
Feroceon does not support that combination, and the TEX=5 C=0 B=0 encoding
for XSc3 doesn't appear to work in practice.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Tomi Valkeinen reports:
Running with latest linux-omap kernel on OMAP3 SDP board, I have
problem with iounmap(). It looks like iounmap() does not properly
free large areas. Below is a test which fails for me in 6-7 loops.
for (i = 0; i < 200; ++i) {
vaddr = ioremap(paddr, size);
if (!vaddr) {
printk("couldn't ioremap\n");
break;
}
iounmap(vaddr);
}
The changes to vmalloc.c weren't reflected in the ARM ioremap
implementation. Turns out the fix is rather simple.
Tested-by: Tomi Valkeinen <tomi.valkeinen@nokia.com>
Tested-by: Matt Gerassimoff <mgeras@gmail.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Without this, the pxa2xx-flash driver cannot be used as a module.
Reported-by: Chris Lawrence <chrisdl@netspace.net.au>
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Rename ARM's struct vm_region so that I can introduce my own global version
for NOMMU. It's feasible that the ARM version may wish to use my global one
instead.
The NOMMU vm_region struct defines areas of the physical memory map that are
under mmap. This may include chunks of RAM or regions of memory mapped
devices, such as flash. It is also used to retain copies of file content so
that shareable private memory mappings of files can be made. As such, it may
be compatible with what is described in the banner comment for ARM's vm_region
struct.
Signed-off-by: David Howells <dhowells@redhat.com>
As noted by Akinobu Mita in patch b1fceac2b9,
alloc_bootmem and related functions never return NULL and always return a
zeroed region of memory. Thus a NULL test or memset after calls to these
functions is unnecessary.
This was fixed using the following semantic patch.
(http://www.emn.fr/x-info/coccinelle/)
// <smpl>
@@
expression E;
statement S;
@@
E = \(alloc_bootmem\|alloc_bootmem_low\|alloc_bootmem_pages\|alloc_bootmem_low_pages\|alloc_bootmem_node\|alloc_bootmem_low_pages_node\|alloc_bootmem_pages_node\)(...)
... when != E
(
- BUG_ON (E == NULL);
|
- if (E == NULL) S
)
@@
expression E,E1;
@@
E = \(alloc_bootmem\|alloc_bootmem_low\|alloc_bootmem_pages\|alloc_bootmem_low_pages\|alloc_bootmem_node\|alloc_bootmem_low_pages_node\|alloc_bootmem_pages_node\)(...)
... when != E
- memset(E,0,E1);
// </smpl>
Signed-off-by: Julia Lawall <julia@diku.dk>
Cc: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
On ARMv6 and later CPUs, it is possible for userspace processes to
get stuck on a misaligned load or store due to the "ignore fault"
setting; unlike previous CPUs, retrying the instruction without
the 'A' bit set does not always cause the load to succeed.
We have no real option but to default to fixing up alignment faults
on these CPUs, and having the CPU fix up those misaligned accesses
which it can.
Reported-by: Wolfgang Grandegger <wg@grandegger.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
PXA935 has changed its implementor ID from Intel to Marvell, this
patch modifies arch/arm/boot/compressed/head.S and proc-xsc3.S to
support a smooth bootup.
Signed-off-by: Eric Miao <eric.miao@marvell.com>
This patch adds the necessary definitions and Kconfig entries to enable
Cortex-A9 (ARMv7 SMP) tiles on the RealView/EB board.
Signed-off-by: Jon Callan <Jon.Callan@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Update to use the asm/sections.h header rather than declaring these
symbols ourselves. Change __data_start to _data to conform with the
naming found within asm/sections.h.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
The restriction on !CONFIG_HIGHMEM is unneeded since page tables are
currently never allocated with highmem pages, and actually disable PTE
dump whenever highmem is configured. Let's have a dynamic test to better
describe the current limitation instead.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Commit 8d5796d2ec allows for the vmalloc
area to be resized from the kernel cmdline. Make sure it cannot overlap
with RAM entirely.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Make free_area() arguments pfn based, and return number of freed pages.
This will simplify highmem initialization later.
Also, codepages, datapages and initpages are actually codesize, datasize
and initsize.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Doing so will greatly simplify the bootmem initialization code as each
bank is therefore entirely lowmem or highmem with no crossing between
those zones.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Currently there are two instances of struct meminfo: one in
kernel/setup.c marked __initdata, and another in mm/init.c with
permanent storage. Let's keep only the later to directly populate
the permanent version from arm_add_memory().
Also move common validation tests between the MMU and non-MMU cases
into arm_add_memory() to remove some duplication. Protection against
overflowing the membank array is also moved in there in order to cover
the kernel cmdline parsing path as well.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
In all cases the kaddr is assigned an input register even though it is
modified in the assembly code. Let's assign a new variable to the
modified value and mark those inline asm with volatile otherwise they
get optimized away because the output variable is otherwise not used.
Also fix a few conversion errors in copypage-feroceon.c and
copypage-v4mc.c.
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
For similar reasons as copy_user_page(), we want to avoid the
additional kmap_atomic if it's unnecessary.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
We used to override the copy_user_page() function. However, this
is not only inefficient, it also causes additional complexity for
highmem support, since we convert from a struct page to a kernel
direct mapped address and back to a struct page again.
Moreover, with highmem support, we end up pointlessly setting up
kmap entries for pages which we're going to remap. So, push the
kmapping down into the copypage implementation files where it's
required.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Rather than:
config CPU_BLAH
bool
depends on ARCH_FOO || MACH_BAR
default y if ARCH_FOO || MACH_BAR
arrange for ARCH_FOO and MACH_BAR to select CPU_BLAH directly.
Acked-by: Nicolas Pitre <nico@marvell.com>
Acked-by: Andrew Victor <linux@maxim.org.za>
Acked-by: Brian Swetland <swetland@google.com>
Acked-by: Eric Miao <eric.miao@marvell.com>
Acked-by: Nicolas Bellido <ml@acolin.be>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
As suggested by Andrew Morton, remove memzero() - it's not supported
on other architectures so use of it is a potential build breaking bug.
Since the compiler optimizes memset(x,0,n) to __memzero() perfectly
well, we don't miss out on the underlying benefits of memzero().
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Since WFI may cause the processor to enter a low-power mode, data may
still be in the write buffer. This patch adds a DSB (or DWB) to the
cpu_(v6|v7)_do_idle functions before the WFI.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Mikael Pettersson reported:
The 2.6.28-rc kernels fail to detect PCI device 0000:00:01.0
(the first ethernet port) on my Thecus n2100 XScale box.
There is however still a strange "ghost" device that gets partially
detected in 2.6.28-rc2 vanilla.
The IOP321 manual says:
The user designates the memory region containing the OCCDR as
non-cacheable and non-bufferable from the IntelR XScaleTM core.
This guarantees that all load/stores to the OCCDR are only of
DWORD quantities.
Ensure that the OCCDR is so mapped.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Same fix as commit c7cf72dcad: when 'start' and 'end' are less than a
cacheline apart and 'start' is unaligned we are done after cleaning and
invalidating the first cacheline.
Cc: <stable@kernel.org>
Signed-off-by: Nicolas Pitre <nico@marvell.com>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
When 'start' and 'end' are less than a cacheline apart and 'start' is
unaligned we are done after cleaning and invalidating the first
cacheline. So check for (start < end) which will not walk off into
invalid address ranges when (start > end).
This issue was caught by drivers/dma/dmatest.
2.6.27 is susceptible.
Cc: <stable@kernel.org>
Cc: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Cc: Lothar WaÃ<9f>mann <LW@KARO-electronics.de>
Cc: Lennert Buytenhek <buytenh@marvell.com>
Cc: Eric Miao <eric.miao@marvell.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
As a result of the ptebits changes, we ended up marking device mappings
as normal memory on ARMv7 CPUs, resulting in undesirable behaviour with
serial ports and the like. While reviewing the section mapping table
entries, other errors in the memory type settings for devices were
detected and confirmed to prevent Xscale3 platforms booting.
Tested on:
OMAP34xx (ARMv7),
OMAP24xx (ARMv6),
OMAP16xx (ARM926T, ARMv5),
PXA311 (Xscale3),
PXA272 (Xscale),
PXA255 (Xscale),
IXP42x (Xscale),
S3C2410 (ARM920T, ARMv4T),
ARM720T (ARMv4T)
StrongARM-110 (ARMv4)
Acked-by: Tony Lindgren <tony@atomide.com>
Tested-by: Robert Jarzmik <robert.jarzmik@free.fr>
Tested-by: Mike Rapoport <mike@compulab.co.il>
Tested-by: Ben Dooks <ben-linux@fluff.org>
Tested-by: Anders Grafström <grfstrm@users.sourceforge.net>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
As of 73bdf0a60e, the kernel needs
to know where modules are located in the virtual address space.
On ARM, we located this region between MODULE_START and MODULE_END.
Unfortunately, everyone else calls it MODULES_VADDR and MODULES_END.
Update ARM to use the same naming, so is_vmalloc_or_module_addr()
can work properly. Also update the comment on mm/vmalloc.c to
reflect that ARM also places modules in a separate region from the
vmalloc space.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This patch adds the SMP/nAMP mode setting to __v7_setup and also sets
TTBR to shared page table walks if SMP is enabled. The PTWs are also
marked inner cacheable for both SMP and UP modes (setting this is fine
even if the CPU doesn't support the feature).
Signed-off-by: Jon Callan <Jon.Callan@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The flush_cache_all function on ARMv7 is implemented as a series of
cache operations by set/way. These are not guaranteed to be ordered with
previous memory accesses, requiring a DMB. This patch also adds barriers
for the TLB operations in compressed/head.S
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
