Add arch specific dev_archdata to struct device
Adds an arch specific struct dev_arch to struct device. This enables
architecture to add specific fields to every device in the system, like
DMA operation pointers, NUMA node ID, firmware specific data, etc...
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: Andi Kleen <ak@suse.de>
Acked-By: David Howells <dhowells@redhat.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Make clocksource_mips public and get rid of mips_hpt_read,
mips_hpt_mask.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Now we have both function and macro version of do_IRQ() and the former
is used only by DEC and non-preemptive kernel. This patch makes
everyone use the macro version and removes the function version.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Further incorporation of generic irq framework. Replacing __do_IRQ()
by proper flow handler would make the irq handling path a bit simpler
and faster.
* use generic_handle_irq() instead of __do_IRQ().
* use handle_level_irq for obvious level-type irq chips.
* use handle_percpu_irq for irqs marked as IRQ_PER_CPU.
* setup .eoi routine for irq chips possibly used with handle_percpu_irq.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This is a big irq cleanup patch.
* Use set_irq_chip() to register irq_chip.
* Initialize .mask, .unmask, .mask_ack field. Functions for these
method are already exist in most case.
* Do not initialize .startup, .shutdown, .enable, .disable fields if
default routines provided by irq_chip_set_defaults() were suitable.
* Remove redundant irq_desc initializations.
* Remove unnecessary local_irq_save/local_irq_restore, spin_lock.
With this cleanup, it would be easy to switch to slightly lightwait
irq flow handlers (handle_level_irq(), etc.) instead of __do_IRQ().
Though whole this patch is quite large, changes in each irq_chip are
not quite simple. Please review and test on your platform. Thanks.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Currently nobody outside time.c require mips_hpt_init(). Remove it
and call c0_hpt_timer_init() directly if R4k counter was used for
timer interrupt.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This patch has rewritten GALILEO_INL/GALILEO_OUTL using GT_READ/GT_WRITE.
This patch tested on Cobalt Qube2.
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This would get rid of some warnings about "long" vs. "long long".
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
This patch introduces __pa_symbol() macro which should be used to
calculate the physical address of kernel symbols. It also relies
on RELOC_HIDE() to avoid any compiler's oddities when doing
arithmetics on symbols.
Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
During early boot mem init, some configs couldn't use __pa() to
convert virtual into physical addresses. Specially for 64 bit
kernel cases when CONFIG_BUILD_ELF64=n. This patch make __pa()
work for _all_ configs and thus make CPHYSADDR() useless.
Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
__pa() was used by virt_to_page() and virt_addr_valid(). These
latter are used when kernel is initialised so __pa() is not
appropriate, we use virt_to_phys() instead.
Futhermore __pa() is going to take care of CKSEG0/XKPHYS
address mix for 64 bit kernels. This makes __pa() more complex
than virt_to_phys() and this extra work is not needed by
virt_to_page() and virt_addr_valid().
Eventually it consolidates virt_to_phys() prototype by making
its argument 'const'. this avoids some warnings that was due
to some virt_to_page() usages which pass const pointer.
Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The Au1xx IDE controller driver doesn't compile:
CC drivers/ide/mips/au1xxx-ide.o
/linux-2.6.19-rc6-work/drivers/ide/mips/au1xxx-ide.c:480: error: conflicting types for 'auide_ddma_tx_callback'
include2/asm/mach-au1x00/au1xxx_ide.h:174: error: previous declaration of 'auide_ddma_tx_callback' was here
/linux-2.6.19-rc6-work/drivers/ide/mips/au1xxx-ide.c:486: error: conflicting types for 'auide_ddma_rx_callback'
include2/asm/mach-au1x00/au1xxx_ide.h:176: error: previous declaration of 'auide_ddma_rx_callback' was here
Signed-off-by: Manuel Lauss <mano@roarinelk.homelinux.net>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Since we already moved to GENERIC_TIME, we should implement alternatives
of old do_gettimeoffset routines to get sub-jiffies resolution from
gettimeofday(). This patch includes:
* MIPS clocksource support (based on works by Manish Lachwani).
* remove unused gettimeoffset routines and related codes.
* remove unised 64bit do_div64_32().
* simplify mips_hpt_init. (no argument needed, __init tag)
* simplify c0_hpt_timer_init. (no need to write to c0_count)
* remove some hpt_init routines.
* mips_hpt_mask variable to specify bitmask of hpt value.
* convert jmr3927_do_gettimeoffset to jmr3927_hpt_read.
* convert ip27_do_gettimeoffset to ip27_hpt_read.
* convert bcm1480_do_gettimeoffset to bcm1480_hpt_read.
* simplify sb1250 hpt functions. (no need to subtract and shift)
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The current implementation uses a sequence of a cacheflush and a copy.
This is racy in case of a multithreaded debuggee and renders GDB
virtually unusable.
Aside this fixes a performance hog rendering access to /proc/cmdline very
slow and resulting in a enough cache stalls for the 34K AP/SP programming
model to make the bare metal code on the non-Linux VPE miss RT deadlines.
The main part of this patch was originally written by Ralf Baechle;
Atushi Nemoto did the the debugging.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
With the existing prototype the following code:
const void __iomem *io = ioremap();
x = readb(io);
iounmap(io);
did result in a warning.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* 'upstream' of git://ftp.linux-mips.org/pub/scm/upstream-linus:
[MIPS] Pass NULL not 0 for pointer value.
[MIPS] IP27: Make declaration of setup_replication_mask a proper prototype.
[MIPS] BigSur: More useful defconfig.
[MIPS] Cleanup definitions of speed_t and tcflag_t.
[MIPS] Fix compilation warnings in arch/mips/sibyte/bcm1480/smp.c
[MIPS] Optimize and cleanup get_saved_sp, set_saved_sp
[MIPS] <asm/irq.h> does not need pt_regs anymore.
[MIPS] Workaround for bug in gcc -EB / -EL options.
[MIPS] Fix timer setup for Jazz
If CONFIG_BUILD_ELF64 was not selected and gcc had -msym32 option
(i.e. 4.0 or newer), there is no point to use %highest, %higher for
kernel symbols.
This patch also fixes 64-bit SMTC version of get_saved_sp() which is
broken but harmless since there is no such CPUs for now.
A bonus is set_saved_sp() and SMP version of get_saved_sp() are more
readable now.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
There's nothing arch-specific about check_signature(), so move it to
<linux/io.h>. Use a cross between the Alpha and i386 implementations as
the generic one.
Signed-off-by: Matthew Wilcox <willy@parisc-linux.org>
Acked-by: Alan Cox <alan@redhat.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Make sure cpu_has_fpu (which uses smp_processor_id()) is used only in
atomic context.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Use the new typedef for interrupt handler function pointers rather than
actually spelling out the full thing each time. This was scripted with the
following small shell script:
#!/bin/sh
egrep -nHrl -e 'irqreturn_t[ ]*[(][*]' $* |
while read i
do
echo $i
perl -pi -e 's/irqreturn_t\s*[(]\s*[*]\s*([_a-zA-Z0-9]*)\s*[)]\s*[(]\s*int\s*,\s*void\s*[*]\s*[)]/irq_handler_t \1/g' $i || exit $?
done
Signed-Off-By: David Howells <dhowells@redhat.com>
Maintain a per-CPU global "struct pt_regs *" variable which can be used instead
of passing regs around manually through all ~1800 interrupt handlers in the
Linux kernel.
The regs pointer is used in few places, but it potentially costs both stack
space and code to pass it around. On the FRV arch, removing the regs parameter
from all the genirq function results in a 20% speed up of the IRQ exit path
(ie: from leaving timer_interrupt() to leaving do_IRQ()).
Where appropriate, an arch may override the generic storage facility and do
something different with the variable. On FRV, for instance, the address is
maintained in GR28 at all times inside the kernel as part of general exception
handling.
Having looked over the code, it appears that the parameter may be handed down
through up to twenty or so layers of functions. Consider a USB character
device attached to a USB hub, attached to a USB controller that posts its
interrupts through a cascaded auxiliary interrupt controller. A character
device driver may want to pass regs to the sysrq handler through the input
layer which adds another few layers of parameter passing.
I've build this code with allyesconfig for x86_64 and i386. I've runtested the
main part of the code on FRV and i386, though I can't test most of the drivers.
I've also done partial conversion for powerpc and MIPS - these at least compile
with minimal configurations.
This will affect all archs. Mostly the changes should be relatively easy.
Take do_IRQ(), store the regs pointer at the beginning, saving the old one:
struct pt_regs *old_regs = set_irq_regs(regs);
And put the old one back at the end:
set_irq_regs(old_regs);
Don't pass regs through to generic_handle_irq() or __do_IRQ().
In timer_interrupt(), this sort of change will be necessary:
- update_process_times(user_mode(regs));
- profile_tick(CPU_PROFILING, regs);
+ update_process_times(user_mode(get_irq_regs()));
+ profile_tick(CPU_PROFILING);
I'd like to move update_process_times()'s use of get_irq_regs() into itself,
except that i386, alone of the archs, uses something other than user_mode().
Some notes on the interrupt handling in the drivers:
(*) input_dev() is now gone entirely. The regs pointer is no longer stored in
the input_dev struct.
(*) finish_unlinks() in drivers/usb/host/ohci-q.c needs checking. It does
something different depending on whether it's been supplied with a regs
pointer or not.
(*) Various IRQ handler function pointers have been moved to type
irq_handler_t.
Signed-Off-By: David Howells <dhowells@redhat.com>
(cherry picked from 1b16e7ac850969f38b375e511e3fa2f474a33867 commit)
This patch fixes the compilation errors on PNX8550 and hard-to-track
bug in interrupt handling.
It also corresponds to the latest changes in PNX8550 serial driver.
Signed-off-by: Vitaly Wool <vitalywool@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* git://git.kernel.org/pub/scm/linux/kernel/git/bunk/trivial: (39 commits)
Add missing maintainer countries in CREDITS
Fix bytes <-> kilobytes typo in Kconfig for ramdisk
fix a typo in Documentation/pi-futex.txt
BUG_ON conversion for fs/xfs/
BUG_ON() conversion in fs/nfsd/
BUG_ON conversion for fs/reiserfs
BUG_ON cleanups in arch/i386
BUG_ON cleanup in drivers/net/tokenring/
BUG_ON cleanup for drivers/md/
kerneldoc-typo in led-class.c
debugfs: spelling fix
rcutorture: Fix incorrect description of default for nreaders parameter
parport: Remove space in function calls
Michal Wronski: update contact info
Spelling fix: "control" instead of "cotrol"
reboot parameter in Documentation/kernel-parameters.txt
Fix copy&waste bug in comment in scripts/kernel-doc
remove duplicate "until" from kernel/workqueue.c
ite_gpio fix tabbage
fix file specification in comments
...
Fixed trivial path conflicts due to removed files:
arch/mips/dec/boot/decstation.c, drivers/char/ite_gpio.c
Many files include the filename at the beginning, serveral used a wrong one.
Signed-off-by: Uwe Zeisberger <Uwe_Zeisberger@digi.com>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
As per feature-removal-schedule.txt.
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Acked-by: Alan Cox <alan@redhat.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The stack pointer in MIPS/gcc should always 8 bytes aligned on
entry to any routines. Therefore pt_regs structure must be
aligned to 8-byte boundary too.
Instead of creating dummy fields to achieve this alignment, this
patch let gcc doing it. Therefore 'smtc_pad' field can be safely
removed.
Signed-off-by: Franck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Introduce the child field in sched_domain struct and use it in
sched_balance_self().
We will also use this field in cleaning up the sched group cpu_power
setup(done in a different patch) code.
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Acked-by: Ingo Molnar <mingo@elte.hu>
Acked-by: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The last in-kernel user of errno is gone, so we should remove the definition
and everything referring to it. This also removes the now-unused lib/execve.c
file that was introduced earlier.
Also remove every trace of __KERNEL_SYSCALLS__ that still remained in the
kernel.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Andi Kleen <ak@muc.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ian Molton <spyro@f2s.com>
Cc: Mikael Starvik <starvik@axis.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Hirokazu Takata <takata.hirokazu@renesas.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Kyle McMartin <kyle@mcmartin.ca>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Kazumoto Kojima <kkojima@rr.iij4u.or.jp>
Cc: Richard Curnow <rc@rc0.org.uk>
Cc: William Lee Irwin III <wli@holomorphy.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Miles Bader <uclinux-v850@lsi.nec.co.jp>
Cc: Chris Zankel <chris@zankel.net>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Roman Zippel <zippel@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
If the PC was ret_from_irq or ret_from_exception, there will be no
more normal stackframe. Instead of stopping the unwinding, use PC and
RA saved by an exception handler to continue unwinding into the
interrupted context. This also simplifies the CONFIG_STACKTRACE code.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Implement stacktrace interface by using unwind_stack() and enable lockdep
support in Kconfig.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
In handle_sys and its variants, we must reload some registers which
might be clobbered by trace_hardirqs_on().
Also we must make sure trace_hardirqs_on() called in kernel level (not
exception level).
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
On systems running with virtual cpus there is optimization potential in
regard to spinlocks and rw-locks. If the virtual cpu that has taken a lock
is known to a cpu that wants to acquire the same lock it is beneficial to
yield the timeslice of the virtual cpu in favour of the cpu that has the
lock (directed yield).
With CONFIG_PREEMPT="n" this can be implemented by the architecture without
common code changes. Powerpc already does this.
With CONFIG_PREEMPT="y" the lock loops are coded with _raw_spin_trylock,
_raw_read_trylock and _raw_write_trylock in kernel/spinlock.c. If the lock
could not be taken cpu_relax is called. A directed yield is not possible
because cpu_relax doesn't know anything about the lock. To be able to
yield the lock in favour of the current lock holder variants of cpu_relax
for spinlocks and rw-locks are needed. The new _raw_spin_relax,
_raw_read_relax and _raw_write_relax primitives differ from cpu_relax
insofar that they have an argument: a pointer to the lock structure.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Haavard Skinnemoen <hskinnemoen@atmel.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
MIPS is the only port to call its fstatat()-related syscalls
"__NR_fstatat". Now I can see why that might be seen as every
other port being wrong, but I think for o32, it is at best confusing.
__NR_fstat provides a plain (32-bit) stat while __NR_fstatat provides a
64-bit stat. Changing the name to __NR_fstatat64 would make things more
explicit, match x86, and make the glibc port slightly easier.
The current name is more appropriate for n32 and n64, but it would be
appropriate for other 64-bit targets too, and those targets have chosen
to call it __NR_newfstatat instead. Using the same name for MIPS would
again be more consistent and make the glibc port slightly easier.
I'm not wedded to this idea if the current names are preferred,
but FWIW...
Signed-off-by: Richard Sandiford <richard@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The code in pgtable-64.h assumes TASK_SIZE is always bigger than a first
level PGDIR_SIZE. This is not the case for 64K pages, where task size is
40 bits (1TB) and a pgd entry can map 42 bits. This leads to
USER_PTRS_PER_PGD being zero for 64K pages.
Signed-off-by: Peter Watkins <treestem@gmail.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
excite_fpga.h, like all platform headers, really belongs in the
platform header directory.
Signed-off-by: Thomas Koeller <thomas.koeller@baslerweb.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The following change updates the Atlas interrupt handling to match that
of Malta. Tested with a 5Kc and a 34Kf successfully.
Signed-off-by: Maciej W. Rozycki <macro@mips.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Atlas maps its RTC chip in the host mmio space rather than using the
"traditional" location in the PCI/ISA port space. A change that has
happened to the generic RTC header requires to define ARCH_RTC_LOCATION
now.
Signed-off-by: Maciej W. Rozycki <macro@mips.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* export asm/sgidefs.h
* include asm/isadep.h only if in kernel
* do not export contents of asm/timex.h and asm/user.h
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The third argument of au1xxx_dbdma_chan_alloc's callback function is not
used anywhere.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Cc: David Howells <dhowells@redhat.com>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
One of the changes necessary for shared page tables is to standardize the
pxx_page macros. pte_page and pmd_page have always returned the struct
page associated with their entry, while pte_page_kernel and pmd_page_kernel
have returned the kernel virtual address. pud_page and pgd_page, on the
other hand, return the kernel virtual address.
Shared page tables needs pud_page and pgd_page to return the actual page
structures. There are very few actual users of these functions, so it is
simple to standardize their usage.
Since this is basic cleanup, I am submitting these changes as a standalone
patch. Per Hugh Dickins' comments about it, I am also changing the
pxx_page_kernel macros to pxx_page_vaddr to clarify their meaning.
Signed-off-by: Dave McCracken <dmccr@us.ibm.com>
Cc: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
MIPS asm/page.h unconditionally includes <asm-generic/memory_model.h>, which
doesn't exist in userspace. Move an #endif /* __KERNEL__ */ down a few lines
to prevent that.
Also, remove the broken definition of PAGE_SIZE which is never going to be
correct -- in the absence of PAGE_SIZE, non-broken userspace will fall back to
using sysconf() or getpagesize() instead.
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
set_wmb should not be used in the kernel because it just confuses the
code more and has no benefit. Since it is not currently used in the
kernel this patch removes it so that new code does not include it.
All archs define set_wmb(var, value) to do { var = value; wmb(); }
while(0) except ia64 and sparc which use a mb() instead. But this is
still moot since it is not used anyway.
Hasn't been tested on any archs but x86 and x86_64 (and only compiled
tested)
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
1. MIPS should select SPARSEMEM_STATIC since allocating bootmem in
memory_present() will corrupt bootmap area.
2. pfn_valid() for SPARSEMEM is defined in linux/mmzone.h
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Handle memory-mapped chips properly, needed for example on DECstations.
This support was in Linux 2.4 but for some reason got lost in 2.6. This
patch is taken directly from the linux-mips repository.
[akpm@osdl.org: cleanup]
Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>
Signed-off-by: Martin Michlmayr <tbm@cyrius.com>
Cc: Paul Gortmaker <penguin@muskoka.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Use the new IRQF_ constants and remove the SA_INTERRUPT define
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patch implements an API whereby an application can determine the
label of its peer's Unix datagram sockets via the auxiliary data mechanism of
recvmsg.
Patch purpose:
This patch enables a security-aware application to retrieve the
security context of the peer of a Unix datagram socket. The application
can then use this security context to determine the security context for
processing on behalf of the peer who sent the packet.
Patch design and implementation:
The design and implementation is very similar to the UDP case for INET
sockets. Basically we build upon the existing Unix domain socket API for
retrieving user credentials. Linux offers the API for obtaining user
credentials via ancillary messages (i.e., out of band/control messages
that are bundled together with a normal message). To retrieve the security
context, the application first indicates to the kernel such desire by
setting the SO_PASSSEC option via getsockopt. Then the application
retrieves the security context using the auxiliary data mechanism.
An example server application for Unix datagram socket should look like this:
toggle = 1;
toggle_len = sizeof(toggle);
setsockopt(sockfd, SOL_SOCKET, SO_PASSSEC, &toggle, &toggle_len);
recvmsg(sockfd, &msg_hdr, 0);
if (msg_hdr.msg_controllen > sizeof(struct cmsghdr)) {
cmsg_hdr = CMSG_FIRSTHDR(&msg_hdr);
if (cmsg_hdr->cmsg_len <= CMSG_LEN(sizeof(scontext)) &&
cmsg_hdr->cmsg_level == SOL_SOCKET &&
cmsg_hdr->cmsg_type == SCM_SECURITY) {
memcpy(&scontext, CMSG_DATA(cmsg_hdr), sizeof(scontext));
}
}
sock_setsockopt is enhanced with a new socket option SOCK_PASSSEC to allow
a server socket to receive security context of the peer.
Testing:
We have tested the patch by setting up Unix datagram client and server
applications. We verified that the server can retrieve the security context
using the auxiliary data mechanism of recvmsg.
Signed-off-by: Catherine Zhang <cxzhang@watson.ibm.com>
Acked-by: Acked-by: James Morris <jmorris@namei.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
* 'upstream' of git://ftp.linux-mips.org/pub/scm/upstream-linus: (33 commits)
[MIPS] Add missing backslashes to macro definitions.
[MIPS] Death list of board support to be removed after 2.6.18.
[MIPS] Remove BSD and Sys V compat data types.
[MIPS] ioc3.h: Uses u8, so include <linux/types.h>.
[MIPS] 74K: Assume it will also have an AR bit in config7
[MIPS] Treat CPUs with AR bit as physically indexed.
[MIPS] Oprofile: Support VSMP on 34K.
[MIPS] MIPS32/MIPS64 S-cache fix and cleanup
[MIPS] excite: PCI makefile needs to use += if it wants a chance to work.
[MIPS] excite: plat_setup -> plat_mem_setup.
[MIPS] au1xxx: export dbdma functions
[MIPS] au1xxx: dbdma, no sleeping under spin_lock
[MIPS] au1xxx: fix PSC_SMBTXRX_RSR.
[MIPS] Early printk for IP27.
[MIPS] Fix handling of 0 length I & D caches.
[MIPS] Typo fixes.
[MIPS] MIPS32/MIPS64 secondary cache management
[MIPS] Fix FIXADDR_TOP for TX39/TX49.
[MIPS] Remove first timer interrupt setup in wrppmc_timer_setup()
[MIPS] Fix configuration of R2 CPU features and multithreading.
...
FIXADDR_TOP is used for HIGHMEM and for upper limit of vmalloc area on
32bit kernel. TX39XX and TX49XX have "reserved" segment in CKSEG3
area. 0xff000000-0xff3fffff on TX49XX and 0xff000000-0xfffeffff on
TX39XX are reserved (unmapped, uncached) therefore can not be used as
mapped area.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Add ->retrigger() irq op to consolidate hw_irq_resend() implementations.
(Most architectures had it defined to NOP anyway.)
NOTE: ia64 needs testing. i386 and x86_64 tested.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
I'm testing glibc on MIPS64, little-endian, N32, O32 and N64 multilibs.
Among the NPTL test failures seen are some arising from sigsuspend problems
for N32: it blocks the wrong signals, so SIGCANCEL (SIGRTMIN) is blocked
despite glibc's carefully excluding it from sets of signals to block.
Specifically, testing suggests it blocks signal N^32 instead of signal N,
so (in the example tested) blocking SIGUSR1 (17) blocks signal 49 instead.
glibc's sigset_t uses an array of unsigned long, as does the kernel.
In both cases, signal N+1 is represented as
(1UL << (N % (8 * sizeof (unsigned long)))) in word number
(N / (8 * sizeof (unsigned long))).
Thus the N32 glibc uses an array of 32-bit words and the N64 kernel uses an
array of 64-bit words. For little-endian, the layout is the same, with
signals 1-32 in the first 4 bytes, signals 33-64 in the second, etc.; for
big-endian, userspace has that layout while in the kernel each 8 bytes have
the two halves swapped from the userspace layout.
The N32 sigsuspend syscall uses sigset_from_compat to convert the userspace
sigset to kernel format. If __COMPAT_ENDIAN_SWAP__ is *not* set, this uses
logic of the form
set->sig[0] = compat->sig[0] | (((long)compat->sig[1]) << 32 )
to convert the userspace sigset to a kernel one. This looks correct to me
for both big and little endian, given that in userspace compat->sig[1] will
represent signals 33-64, and so will the high 32 bits of set->sig[0] in the
kernel. If however __COMPAT_ENDIAN_SWAP__ *is* set, as it is for
__MIPSEL__, it uses
set->sig[0] = compat->sig[1] | (((long)compat->sig[0]) << 32 );
which seems incorrect for both big and little endian, and would
explain the observed symptoms.
This code is the only use of __COMPAT_ENDIAN_SWAP__, so if incorrect
then that macro serves no purpose, in which case something like the
following patch would seem appropriate to remove it.
Signed-off-by: Joseph Myers <joseph@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The floppy driver is already calling add_disk_randomness as it should, so this
was redundant.
Signed-off-by: Matt Mackall <mpm@selenic.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add missing PSC #define's required for the drivers using PSC on DBAu1550
board (also fixing Au1550 PSC3 address) and all Au1200-based boards as
well. Make the OSS driver use the correct PSC definitions fo each board.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
VGA_MAP_MEM translates to ioremap() on some architectures. It makes sense
to do this to vga_vram_base, because we're going to access memory between
vga_vram_base and vga_vram_end.
But it doesn't really make sense to map starting at vga_vram_end, because
we aren't going to access memory starting there. On ia64, which always has
to be different, ioremapping vga_vram_end gives you something completely
incompatible with ioremapped vga_vram_start, so vga_vram_size ends up being
nonsense.
As a bonus, we often know the size up front, so we can use ioremap()
correctly, rather than giving it a zero size.
Signed-off-by: Bjorn Helgaas <bjorn.helgaas@hp.com>
Cc: "Antonino A. Daplas" <adaplas@pol.net>
Cc: "Luck, Tony" <tony.luck@intel.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
* git://git.infradead.org/hdrcleanup-2.6: (63 commits)
[S390] __FD_foo definitions.
Switch to __s32 types in joystick.h instead of C99 types for consistency.
Add <sys/types.h> to headers included for userspace in <linux/input.h>
Move inclusion of <linux/compat.h> out of user scope in asm-x86_64/mtrr.h
Remove struct fddi_statistics from user view in <linux/if_fddi.h>
Move user-visible parts of drivers/s390/crypto/z90crypt.h to include/asm-s390
Revert include/media changes: Mauro says those ioctls are only used in-kernel(!)
Include <linux/types.h> and use __uXX types in <linux/cramfs_fs.h>
Use __uXX types in <linux/i2o_dev.h>, include <linux/ioctl.h> too
Remove private struct dx_hash_info from public view in <linux/ext3_fs.h>
Include <linux/types.h> and use __uXX types in <linux/affs_hardblocks.h>
Use __uXX types in <linux/divert.h> for struct divert_blk et al.
Use __u32 for elf_addr_t in <asm-powerpc/elf.h>, not u32. It's user-visible.
Remove PPP_FCS from user view in <linux/ppp_defs.h>, remove __P mess entirely
Use __uXX types in user-visible structures in <linux/nbd.h>
Don't use 'u32' in user-visible struct ip_conntrack_old_tuple.
Use __uXX types for S390 DASD volume label definitions which are user-visible
S390 BIODASDREADCMB ioctl should use __u64 not u64 type.
Remove unneeded inclusion of <linux/time.h> from <linux/ufs_fs.h>
Fix private integer types used in V4L2 ioctls.
...
Manually resolve conflict in include/linux/mtd/physmap.h
Make HZ configurable. DECSTATION can select 128/256/1024 HZ, JAZZ can
only select 100 HZ, others can select 100/128/250/256/1000/1024 HZ if
not explicitly specified). Also remove all mach-xxx/param.h files and
update all defconfigs according to current HZ value.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Several machines have the R4K cache macro name spelled incorrectly. Namely,
they have cpu_has_4kcache defined instead of cpu_has_4k_cache.
Signed-off-by: Joshua Kinard <kumba@gentoo.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Keeping in accordance with other machines, IP27 and IP32 lack a few
macros. IP27 lacks cpu_has_4kex & cpu_has_4k_cache macros while IP32
lacks just the cpu_has_4k_cache macro.
Signed-off-by: Joshua Kinard <kumba@gentoo.org>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Added support for RM200C machines with big endian firmware
Added support for RM200-C40 (R5000 support)
Signed-off-by: Florian Lohoff <flo@rfc822.org>
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Historically plat_mem_setup did the entire platform initialization. This
was rather impractical because it meant plat_mem_setup had to get away
without any kind of memory allocator. To keep old code from breaking
plat_setup was just renamed to plat_setup and a second platform
initialization hook for anything else was introduced.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
kern_addr_valid is currently only being used in kmem_ptr_validate which
is making some vague attempt at verfying the validity of an address.
Only IA-64, PARISC and x86-64 actually make some actual effort to verify
the validity of the pointer. Most architecture definitions of
kern_addr_valid() just define it as 1; the Alpha and CONFIG_DISCONTIGMEM
on i386 and MIPS even as 0; the 0-definition will result in
kmem_ptr_validate always failing which in turn will cause d_validate to
always fail. d_validate's only two users are smbfs and ncpfs, so the
0 definition ended breaking those ...
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
The struct mips_fpu_soft_struct and mips_fpu_hard_struct are
completely same now and the kernel fpu emulator assumes that. This
patch unifies them to mips_fpu_struct and get rid of mips_fpu_union.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Support for the GT-64120-based Wind River 4KC PPMC Evaluation board.
Signed-off-by: Rongkai.Zhan <Rongkai.zhan@windriver.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
I found that NPTL's pthread_cond_signal() does not work properly on
kernels compiled by gcc 4.1.x. I suppose inline asm for
__futex_atomic_op() was wrong. I suppose:
1. "=&r" constraint should be used for oldval.
2. Instead of "r" (uaddr), "=R" (*uaddr) for output and "R" (*uaddr)
for input should be used.
3. "memory" should be added to the clobber list.
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
These include nothing more than the basic set of files listed in
asm-generic/Kbuild.asm. Any extra arch-specific files will need to be
added.
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
Move memory_present() in arch/mips/kernel/setup.c. When using sparsemem
extreme, this function does an allocate for bootmem. This would always
fail since init_bootmem hasn't been called yet.
Move memory_present after free_bootmem. This only marks actual memory
ranges as present instead of the entire address space.
Signed-off-by: Chad Reese <creese@caviumnetworks.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Fix the non-linear memory mapping done via remap_file_pages() -- it
didn't work on any MIPS CPU because the page offset clashing with
_PAGE_FILE and some other page protection bits which should have been left
zeros for this kind of pages.
Signed-off-by: Konstantin Baydarov <kbaidarov@ru.mvista.com>
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
With 64-bit physical address enabled, 'swapon' was causing kernel oops on
Alchemy CPUs (MIPS32) because of the swap entry type field corrupting the
_PAGE_FILE bit in 'pte_low' field. So, switch to storing the swap entry in
'pte_high' field using all its bits except _PAGE_GLOBAL and _PAGE_VALID which
gives 25 bits for the swap entry offset.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
A while ago prom_prepare_cpus was replaced by plat_prepare_cpus but
the declaration has stayed unchanged.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Increase alignment of BogoMIPS loop to 8 bytes. Having the delay loop
overlap cache line boundaries may cause instable delays.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
* master.kernel.org:/pub/scm/linux/kernel/git/davem/sparc-2.6:
[SPARC64]: Fix D-cache corruption in mremap
[SPARC64]: Make smp_processor_id() functional before start_kernel()
If we move a mapping from one virtual address to another,
and this changes the virtual color of the mapping to those
pages, we can see corrupt data due to D-cache aliasing.
Check for and deal with this by overriding the move_pte()
macro. Set things up so that other platforms can cleanly
override the move_pte() macro too.
Signed-off-by: David S. Miller <davem@davemloft.net>
A small bugfix for up to now unused instruction definitions, and a
somewhat larger update to cover MIPS32R2 instructions.
Signed-off-by: Thiemo Seufer <ths@networkno.de>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Rename the 64-bit sc_hi and sc_lo arrays to use the same names
as the 32-bit struct sigcontext (sc_mdhi, sc_hi1, et cetera).
Signed-off-by: Daniel Jacobowitz <dan@codesourcery.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
o Implement futex_atomic_op_inuser() operation
o Don't use the R10000-ll/sc bug workaround version for every processor.
branch likely is deprecated and some historic ll/sc processors don't
implement it. In any case it's slow.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Nothing exciting; Linux just didn't know it yet so this is most adding
a value to a case statement.
Signed-off-by: Chris Dearman <chris@mips.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
In case of CONFIG_64BIT_PHYS_ADDR, set_pte() and pte_clear() functions
only set _PAGE_GLOBAL bit in the pte_low field of the buddy PTEs,
forgetting to propagate ito to pte_high. Thus, the both pages might not
really be made global for the CPU (since it AND's the G-bit of the
odd / even PTEs together to decide whether they're global or not). Thus,
if only a single page is allocated via vmalloc() or ioremap(), it's not
really global for CPU (and it must be, since this is kernel mapping),
and thus its ASID is compared against the current process' one -- so,
we'll get into trouble sooner or later... Also, pte_none() will fail
on global pages because _PAGE_GLOBAL bit is set in both pte_low and
pte_high, and pte_val() will return u64 value consisting of those fields
concateneted.
Signed-off-by: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
These aren't needed by glibc or klibc, and they're broken in some cases
anyway. The uClibc folks are apparently switching over to stop using
them too (now that we agreed that they should be dropped, at least).
Signed-off-by: David Woodhouse <dwmw2@infradead.org>
With recent rewrite for generic bitops, fls() for 32bit kernel with
MIPS64_CPU is broken. Also, ffs(), fls() should be defined the same
way as the libc and compiler built-in routines (returns int instead of
unsigned long).
Signed-off-by: Atsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Gcc might emit an absolute address for the the "m" constraint which
gas unfortunately does not permit.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Current implementations define NODES_SHIFT in include/asm-xxx/numnodes.h for
each arch. Its definition is sometimes configurable. Indeed, ia64 defines 5
NODES_SHIFT values in the current git tree. But it looks a bit messy.
SGI-SN2(ia64) system requires 1024 nodes, and the number of nodes already has
been changeable by config. Suitable node's number may be changed in the
future even if it is other architecture. So, I wrote configurable node's
number.
This patch set defines just default value for each arch which needs multi
nodes except ia64. But, it is easy to change to configurable if necessary.
On ia64 the number of nodes can be already configured in generic ia64 and SN2
config. But, NODES_SHIFT is defined for DIG64 and HP'S machine too. So, I
changed it so that all platforms can be configured via CONFIG_NODES_SHIFT. It
would be simpler.
See also: http://marc.theaimsgroup.com/?l=linux-kernel&m=114358010523896&w=2
Signed-off-by: Yasunori Goto <y-goto@jp.fujitsu.com>
Cc: Hirokazu Takata <takata@linux-m32r.org>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Andi Kleen <ak@muc.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Kyle McMartin <kyle@mcmartin.ca>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Jack Steiner <steiner@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
* master.kernel.org:/home/rmk/linux-2.6-serial:
[SERIAL] Provide Cirrus EP93xx AMBA PL010 serial support.
[SERIAL] amba-pl010: allow platforms to specify modem control method
[SERIAL] Remove obsoleted au1x00_uart driver
[SERIAL] Small time UART configuration fix for AU1100 processor
- fix: initialize the robust list(s) to NULL in copy_process.
- doc update
- cleanup: rename _inuser to _inatomic
- __user cleanups and other small cleanups
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Arjan van de Ven <arjan@infradead.org>
Cc: Ulrich Drepper <drepper@redhat.com>
Cc: Andi Kleen <ak@muc.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patchset provides a new (written from scratch) implementation of robust
futexes, called "lightweight robust futexes". We believe this new
implementation is faster and simpler than the vma-based robust futex solutions
presented before, and we'd like this patchset to be adopted in the upstream
kernel. This is version 1 of the patchset.
Background
----------
What are robust futexes? To answer that, we first need to understand what
futexes are: normal futexes are special types of locks that in the
noncontended case can be acquired/released from userspace without having to
enter the kernel.
A futex is in essence a user-space address, e.g. a 32-bit lock variable
field. If userspace notices contention (the lock is already owned and someone
else wants to grab it too) then the lock is marked with a value that says
"there's a waiter pending", and the sys_futex(FUTEX_WAIT) syscall is used to
wait for the other guy to release it. The kernel creates a 'futex queue'
internally, so that it can later on match up the waiter with the waker -
without them having to know about each other. When the owner thread releases
the futex, it notices (via the variable value) that there were waiter(s)
pending, and does the sys_futex(FUTEX_WAKE) syscall to wake them up. Once all
waiters have taken and released the lock, the futex is again back to
'uncontended' state, and there's no in-kernel state associated with it. The
kernel completely forgets that there ever was a futex at that address. This
method makes futexes very lightweight and scalable.
"Robustness" is about dealing with crashes while holding a lock: if a process
exits prematurely while holding a pthread_mutex_t lock that is also shared
with some other process (e.g. yum segfaults while holding a pthread_mutex_t,
or yum is kill -9-ed), then waiters for that lock need to be notified that the
last owner of the lock exited in some irregular way.
To solve such types of problems, "robust mutex" userspace APIs were created:
pthread_mutex_lock() returns an error value if the owner exits prematurely -
and the new owner can decide whether the data protected by the lock can be
recovered safely.
There is a big conceptual problem with futex based mutexes though: it is the
kernel that destroys the owner task (e.g. due to a SEGFAULT), but the kernel
cannot help with the cleanup: if there is no 'futex queue' (and in most cases
there is none, futexes being fast lightweight locks) then the kernel has no
information to clean up after the held lock! Userspace has no chance to clean
up after the lock either - userspace is the one that crashes, so it has no
opportunity to clean up. Catch-22.
In practice, when e.g. yum is kill -9-ed (or segfaults), a system reboot is
needed to release that futex based lock. This is one of the leading
bugreports against yum.
To solve this problem, 'Robust Futex' patches were created and presented on
lkml: the one written by Todd Kneisel and David Singleton is the most advanced
at the moment. These patches all tried to extend the futex abstraction by
registering futex-based locks in the kernel - and thus give the kernel a
chance to clean up.
E.g. in David Singleton's robust-futex-6.patch, there are 3 new syscall
variants to sys_futex(): FUTEX_REGISTER, FUTEX_DEREGISTER and FUTEX_RECOVER.
The kernel attaches such robust futexes to vmas (via
vma->vm_file->f_mapping->robust_head), and at do_exit() time, all vmas are
searched to see whether they have a robust_head set.
Lots of work went into the vma-based robust-futex patch, and recently it has
improved significantly, but unfortunately it still has two fundamental
problems left:
- they have quite complex locking and race scenarios. The vma-based
patches had been pending for years, but they are still not completely
reliable.
- they have to scan _every_ vma at sys_exit() time, per thread!
The second disadvantage is a real killer: pthread_exit() takes around 1
microsecond on Linux, but with thousands (or tens of thousands) of vmas every
pthread_exit() takes a millisecond or more, also totally destroying the CPU's
L1 and L2 caches!
This is very much noticeable even for normal process sys_exit_group() calls:
the kernel has to do the vma scanning unconditionally! (this is because the
kernel has no knowledge about how many robust futexes there are to be cleaned
up, because a robust futex might have been registered in another task, and the
futex variable might have been simply mmap()-ed into this process's address
space).
This huge overhead forced the creation of CONFIG_FUTEX_ROBUST, but worse than
that: the overhead makes robust futexes impractical for any type of generic
Linux distribution.
So it became clear to us, something had to be done. Last week, when Thomas
Gleixner tried to fix up the vma-based robust futex patch in the -rt tree, he
found a handful of new races and we were talking about it and were analyzing
the situation. At that point a fundamentally different solution occured to
me. This patchset (written in the past couple of days) implements that new
solution. Be warned though - the patchset does things we normally dont do in
Linux, so some might find the approach disturbing. Parental advice
recommended ;-)
New approach to robust futexes
------------------------------
At the heart of this new approach there is a per-thread private list of robust
locks that userspace is holding (maintained by glibc) - which userspace list
is registered with the kernel via a new syscall [this registration happens at
most once per thread lifetime]. At do_exit() time, the kernel checks this
user-space list: are there any robust futex locks to be cleaned up?
In the common case, at do_exit() time, there is no list registered, so the
cost of robust futexes is just a simple current->robust_list != NULL
comparison. If the thread has registered a list, then normally the list is
empty. If the thread/process crashed or terminated in some incorrect way then
the list might be non-empty: in this case the kernel carefully walks the list
[not trusting it], and marks all locks that are owned by this thread with the
FUTEX_OWNER_DEAD bit, and wakes up one waiter (if any).
The list is guaranteed to be private and per-thread, so it's lockless. There
is one race possible though: since adding to and removing from the list is
done after the futex is acquired by glibc, there is a few instructions window
for the thread (or process) to die there, leaving the futex hung. To protect
against this possibility, userspace (glibc) also maintains a simple per-thread
'list_op_pending' field, to allow the kernel to clean up if the thread dies
after acquiring the lock, but just before it could have added itself to the
list. Glibc sets this list_op_pending field before it tries to acquire the
futex, and clears it after the list-add (or list-remove) has finished.
That's all that is needed - all the rest of robust-futex cleanup is done in
userspace [just like with the previous patches].
Ulrich Drepper has implemented the necessary glibc support for this new
mechanism, which fully enables robust mutexes. (Ulrich plans to commit these
changes to glibc-HEAD later today.)
Key differences of this userspace-list based approach, compared to the vma
based method:
- it's much, much faster: at thread exit time, there's no need to loop
over every vma (!), which the VM-based method has to do. Only a very
simple 'is the list empty' op is done.
- no VM changes are needed - 'struct address_space' is left alone.
- no registration of individual locks is needed: robust mutexes dont need
any extra per-lock syscalls. Robust mutexes thus become a very lightweight
primitive - so they dont force the application designer to do a hard choice
between performance and robustness - robust mutexes are just as fast.
- no per-lock kernel allocation happens.
- no resource limits are needed.
- no kernel-space recovery call (FUTEX_RECOVER) is needed.
- the implementation and the locking is "obvious", and there are no
interactions with the VM.
Performance
-----------
I have benchmarked the time needed for the kernel to process a list of 1
million (!) held locks, using the new method [on a 2GHz CPU]:
- with FUTEX_WAIT set [contended mutex]: 130 msecs
- without FUTEX_WAIT set [uncontended mutex]: 30 msecs
I have also measured an approach where glibc does the lock notification [which
it currently does for !pshared robust mutexes], and that took 256 msecs -
clearly slower, due to the 1 million FUTEX_WAKE syscalls userspace had to do.
(1 million held locks are unheard of - we expect at most a handful of locks to
be held at a time. Nevertheless it's nice to know that this approach scales
nicely.)
Implementation details
----------------------
The patch adds two new syscalls: one to register the userspace list, and one
to query the registered list pointer:
asmlinkage long
sys_set_robust_list(struct robust_list_head __user *head,
size_t len);
asmlinkage long
sys_get_robust_list(int pid, struct robust_list_head __user **head_ptr,
size_t __user *len_ptr);
List registration is very fast: the pointer is simply stored in
current->robust_list. [Note that in the future, if robust futexes become
widespread, we could extend sys_clone() to register a robust-list head for new
threads, without the need of another syscall.]
So there is virtually zero overhead for tasks not using robust futexes, and
even for robust futex users, there is only one extra syscall per thread
lifetime, and the cleanup operation, if it happens, is fast and
straightforward. The kernel doesnt have any internal distinction between
robust and normal futexes.
If a futex is found to be held at exit time, the kernel sets the highest bit
of the futex word:
#define FUTEX_OWNER_DIED 0x40000000
and wakes up the next futex waiter (if any). User-space does the rest of
the cleanup.
Otherwise, robust futexes are acquired by glibc by putting the TID into the
futex field atomically. Waiters set the FUTEX_WAITERS bit:
#define FUTEX_WAITERS 0x80000000
and the remaining bits are for the TID.
Testing, architecture support
-----------------------------
I've tested the new syscalls on x86 and x86_64, and have made sure the parsing
of the userspace list is robust [ ;-) ] even if the list is deliberately
corrupted.
i386 and x86_64 syscalls are wired up at the moment, and Ulrich has tested the
new glibc code (on x86_64 and i386), and it works for his robust-mutex
testcases.
All other architectures should build just fine too - but they wont have the
new syscalls yet.
Architectures need to implement the new futex_atomic_cmpxchg_inuser() inline
function before writing up the syscalls (that function returns -ENOSYS right
now).
This patch:
Add placeholder futex_atomic_cmpxchg_inuser() implementations to every
architecture that supports futexes. It returns -ENOSYS.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Acked-by: Ulrich Drepper <drepper@redhat.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
As announced in feature-removal-schedule.txt.
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
