Fold the SB-1 specific implementation of clear_page/copy_page in the
generic version, and rewrite that one in tlbex style. The immediate
benefits:
- It converts the compile-time workaround for SB-1 pass 1 prefetches
to a more efficient run-time check.
- It allows adjustment of loop unfolling, which helps to reduce the
number of redundant cdex cache ops.
- It fixes some esoteric cornercases (the cache line length calculations
can go wrong, and support for 64k pages without prefetch instructions
will overflow the addiu immediate).
- Somewhat better guesses of "good" prefetch values.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
flush_cache_vmap / flush_cache_vunmap were calling flush_cache_all which -
having been deprecated - turned into a nop ...
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Early 4KEc were MIPS32r1 and therefore need some love to get a TLB
refill handler.
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Fix type mismatch warnings for 64-bit kernel builds which trigger -Werror.
The problem affects only SB-1 kernels with CONFIG_SIBYTE_DMA_PAGEOPS
enabled.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
kmap_coherent will only work correctly if the page it is called on is
not marked dirty. If it's dirty the kernel address of the page should
be used instead of a temporary mapping.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
So far flush_cache_range() did't consider the I-cache largely because it
did rarely ever matter to real world code. This was working primarily
because normally code and data are don't share the same pages - with the
exception of MIPS16 code which uses address constants embedded between
the code. The following sequence of events may break the code:
o MIPS16 executable being loaded
o dynamic linker relocates the address constants embedded into the code:
o Uses mprotect(2) to make code pages PROT_READ|PROT_WRITE
o Performs the actual relocations by writing to the pages which likely
are COW. Because no PROT_EXEC is set I-cache coherence will not be
considered.
o Uses mprotect(2) to switch code pages back to PROT_READ|PROT_EXEC.
This results in a call to flush_cache_range() which also does not
consider I-caches.
o => executing the page just having been relocated may now result in the
I-cache getting refilled with stale data from memory.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This patch moves the micro-assembler in a separate implementation, as
it is useful for further run-time optimizations. The only change in
behaviour is cutting down printk noise at kernel startup time.
Checkpatch complains about macro parameters which aren't protected by
parentheses. I believe this is a flaw in checkpatch, the paste operator
used in those macros won't work with parenthesised parameters.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This patch adds IDs for new Au1200 variants: Au1210 and Au1250.
They are essentially identical to the Au1200 except for the Au1210
which has a different SoC-ID in the PRId register [bits 31:24].
The Au1250 is a "Au1200 V0.2".
Signed-off-by: Manuel Lauss <mano@roarinelk.homelinux.net>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Not cache coherent R10k systems (like IP28) need to do real cache
invalidates in dma_cache_sync().
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
R10k non coherent machines need a real dma cache invalidate to get rid of
speculative stores in cache. For other machines this promises a slight
speedup.
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This complements the generic R4000/R4400 errata workaround code and adds
bits for the daddiu problem. In most places it just modifies handwritten
assembly code so that the assembler is allowed to use a temporary register
as daddiu may now be treated as a macro that expands to a sequence of li
and daddu. It is the AT register or, where AT is unavailable or used
explicitly for another purpose, an explicitly-named register is selected,
using the .set at=<reg> feature added recently to gas. This feature is
only used if CONFIG_CPU_DADDI_WORKAROUNDS has been set, so if the
workaround remains disabled, the required version of binutils stays
unchanged.
Similarly, daddiu instructions put in branch delay slots in noreorder
fragments are now taken out of them and the assembler is allowed to
reorder them itself as possible (which it does making the whole idea of
scheduling them into delay slots manually questionable).
Also in the very few places where such a simple conversion was not
possible, a handcoded longer sequence is implemented.
Other than that there are changes to code responsible for building the
TLB fault and page clear/copy handlers to avoid daddiu as appropriate.
These are only effective if the erratum is verified to be present at the
run time.
Finally there is a trivial update to __delay(), because it uses daddiu in
a branch delay slot.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Sibyte SOCs only have 32-bit PCI. Due to the sparse use of the address
space only the first 1GB of memory is mapped at physical addresses
below 1GB. If a system has more than 1GB of memory 32-bit DMA will
not be able to reach all of it.
For now this patch is good enough to keep Sibyte users happy but it seems
eventually something like swiotlb will be needed for Sibyte.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The recent switch of the Sibyte SOCs from the processor specific cache
managment code in c-sb1.c to c-r4k.c lost this old hack
[MIPS] Hack for SB1 cache issues
Removing flush_icache_page a while ago broke SB1 which was using an empty
flush_data_cache_page function. This glues things well enough so a more
efficient but also more intrusive solution can be found later.
Signed-Off-By: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
in the hope it was no longer needed. As it turns it still is so resurrect
it until there is a better solution.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
When GDB writes a breakpoint into address area of inferior process the
kernel needs to invalidate the modified memory in the inferior which
is done by calling flush_cache_page which in turns calls
r4k_flush_cache_page and local_r4k_flush_cache_page for VSMP or SMTC
kernel via r4k_on_each_cpu().
As the VSMP and SMTC SMP kernels for 34K are running on a single shared
caches it is possible to get away without interprocessor function calls.
This optimization is implemented in r4k_on_each_cpu, so
local_r4k_flush_cache_page is only ever called on the local CPU.
This is where the following code in local_r4k_flush_cache_page() strikes:
/*
* If ownes no valid ASID yet, cannot possibly have gotten
* this page into the cache.
*/
if (cpu_context(smp_processor_id(), mm) == 0)
return;
On VSMP and SMTC had a function of cpu_context() for each CPU(TC).
So in case another CPU than the CPU executing local_r4k_cache_flush_page
has not accessed the mm but one of the other CPUs has there may be data
to be flushed in the cache yet local_r4k_cache_flush_page will falsely
return leaving the I-cache inconsistent for the breakpoint.
While the issue was discovered with GDB it also exists in
local_r4k_flush_cache_range() and local_r4k_flush_cache().
Fixed by introducing a new function has_valid_asid which on MT kernels
returns true if a mm is active on any processor in the system.
This is relativly expensive since for memory acccesses in that loop
cache misses have to be assumed but it seems the most viable solution
for 2.6.23 and older -stable kernels.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Contrary to the belief of some, the R3000 and related processors did have
caches, both a data and an instruction cache. Here is an implementation
of r3k_flush_cache_page(), which is the processor-specific back-end for
flush_cache_range(), done according to the spec in
Documentation/cachetlb.txt.
While at it, remove an unused local function: get_phys_page(), do some
trivial formatting fixes and modernise debugging facilities.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
is_init() is an ambiguous name for the pid==1 check. Split it into
is_global_init() and is_container_init().
A cgroup init has it's tsk->pid == 1.
A global init also has it's tsk->pid == 1 and it's active pid namespace
is the init_pid_ns. But rather than check the active pid namespace,
compare the task structure with 'init_pid_ns.child_reaper', which is
initialized during boot to the /sbin/init process and never changes.
Changelog:
2.6.22-rc4-mm2-pidns1:
- Use 'init_pid_ns.child_reaper' to determine if a given task is the
global init (/sbin/init) process. This would improve performance
and remove dependence on the task_pid().
2.6.21-mm2-pidns2:
- [Sukadev Bhattiprolu] Changed is_container_init() calls in {powerpc,
ppc,avr32}/traps.c for the _exception() call to is_global_init().
This way, we kill only the cgroup if the cgroup's init has a
bug rather than force a kernel panic.
[akpm@linux-foundation.org: fix comment]
[sukadev@us.ibm.com: Use is_global_init() in arch/m32r/mm/fault.c]
[bunk@stusta.de: kernel/pid.c: remove unused exports]
[sukadev@us.ibm.com: Fix capability.c to work with threaded init]
Signed-off-by: Serge E. Hallyn <serue@us.ibm.com>
Signed-off-by: Sukadev Bhattiprolu <sukadev@us.ibm.com>
Acked-by: Pavel Emelianov <xemul@openvz.org>
Cc: Eric W. Biederman <ebiederm@xmission.com>
Cc: Cedric Le Goater <clg@fr.ibm.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Cc: Herbert Poetzel <herbert@13thfloor.at>
Cc: Kirill Korotaev <dev@sw.ru>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
dma_cache_(wback|inv|wback_inv) were the earliest attempt on a generalized
cache managment API for I/O purposes. Originally it was basically the raw
MIPS low level cache API exported to the entire world. The API has
suffered from a lack of documentation, was not very widely used unlike it's
more modern brothers and can easily be replaced by dma_cache_sync. So
remove it rsp. turn the surviving bits back into an arch private API, as
discussed on linux-arch.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Acked-by: Paul Mundt <lethal@linux-sh.org>
Acked-by: Paul Mackerras <paulus@samba.org>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Kyle McMartin <kyle@parisc-linux.org>
Acked-by: Haavard Skinnemoen <hskinnemoen@atmel.com>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* 'upstream' of git://ftp.linux-mips.org/pub/scm/upstream-linus:
[MIPS] Increase cp0 compare clockevent min_delta_ns from 0x30 to 0x300.
[MIPS] Cache: Provide more information on cache policy on bootup.
[MIPS] Fix aliasing bug in copy_user_highpage, take 2.
[MIPS] VPE loader: convert from struct class_ device to struct device
[MIPS] MIPSsim: Fix booting from NFS root
[MIPS] Alchemy: Get rid of au1xxx_irq_map_t.
[MIPS] Alchemy: Get rid of au_ffz().
[MIPS] Alchemy: Get rid of au_ffs().
[MIPS] Alchemy: cleanup interrupt code.
[MIPS] Lasat: Fix build by conversion to irq_cpu.c.
[MIPS] Lasat: Add #ifndef ... #endif include warpper to lasatint.h.
[MIPS] IP22: Enable -Werror.
[MIPS] IP22: Fix warning.
[MIPS] IP22: Complain if requesting the front panel irq failed.
[MIPS] vmlinux.lds.S: Handle KPROBES_TEXT.
[MIPS] vmlinux.lds.S: Fix handling of .notes in final link.
[MIPS] vmlinux.lds.S: Remove duplicate comment.
[MIPS] MSP71XX: Add workarounds file.
[MIPS] IP32: Fix build by conversion to irq_cpu.c.
This should help making bug reports for the gadzillion of cores with all
their configuration and synthesis options more useful.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Turns out b868868ae0 wasn't quite right.
When called for a page that isn't marked dirty it would artificially
create an alias instead of doing the obvious thing and access the page
via KSEG0.
The same issue also exists in copy_to_user_page and copy_from_user_page
which was causing the machine to die under rare circumstances for example
when running ps if the BUG_ON() assertion added by the earlier fix was
getting triggered.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
We have had complaints where a threaded application is left in a bad state
after one of it's threads is killed when we hit a VM: out_of_memory
condition.
Killing just one of the process threads can leave the application in a bad
state, whereas killing the entire process group would allow for the
application to restart, or be otherwise handled, and makes it very obvious
that something has gone wrong.
This change allows the entire process group to be taken down, rather
than just the one thread.
Signed-off-by: Will Schmidt <will_schmidt@vnet.ibm.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ian Molton <spyro@f2s.com>
Cc: Haavard Skinnemoen <hskinnemoen@atmel.com>
Cc: Mikael Starvik <starvik@axis.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Andi Kleen <ak@suse.de>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Hirokazu Takata <takata@linux-m32r.org>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Roman Zippel <zippel@linux-m68k.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Kyle McMartin <kyle@mcmartin.ca>
Cc: Matthew Wilcox <willy@debian.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Kazumoto Kojima <kkojima@rr.iij4u.or.jp>
Cc: Richard Curnow <rc@rc0.org.uk>
Cc: William Lee Irwin III <wli@holomorphy.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Chris Zankel <chris@zankel.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Dump the generated code for clear/copy page calls like it is done for TLB
fault handlers. Useful for debugging.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
On MP configurations it's highly dubious what this code will actually
affect since blasting away cachelines may or may not do the right
thing wrt. cache coherency.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
It may not be perfect yet but the SB1 code is badly borken and has
horrible performance issues.
Downside: This seriously breaks support for pass 1 parts of the BCM1250
where indexed cacheops don't work quite reliable but I seem to be the
last one on the planet with a pass 1 part anyway.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Note that the BCM4710 does not support the wait instruction, this
is not a mistake in the code.
It originally comes from the OpenWrt patches.
Cc: Michael Buesch <mb@bu3sch.de>
Cc: Felix Fietkau <nbd@openwrt.org>
Cc: Florian Schirmer <jolt@tuxbox.org>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Restore a load from KSEG1 done as a workaround for an R4600 v2
erratum, dropped with 211be16de99a7424e66c0b6c0d00e2c970508ac2.
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Copy_user_highpage was written assuming it was only being called for
breaking COW pages in which case the source page isn't cached as in
marked cachable under it kernel virtual address. If it is called anyway
the aliasing avoidance strategy implemented by kmap_coherent will fail.
Avoid the use of kmap_coherent for pages marked dirty and to avoid
another instance of this sort of bug, place a BUG_ON in kmap_coherent.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
