Techwizz/kernel-fxtec-pro1x
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

MIPS KVM Guest FPU & SIMD (MSA) Support

Browse source 
Add guest FPU and MIPS SIMD Architecture (MSA) support to MIPS KVM, by
 enabling the host FPU/MSA while in guest mode. This adds two new KVM
 capabilities, KVM_CAP_MIPS_FPU & KVM_CAP_MIPS_MSA, and supports the 3 FP
 register modes (FR=0, FR=1, FRE=1), and 128-bit MSA vector registers,
 with lazy FPU/MSA context save and restore.
 
 Some required MIPS FP/MSA fixes are merged in from a branch in the MIPS
 tree first.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJVFc3aAAoJEGwLaZPeOHZ6DjwQAI4yYxOD3QmyIGMekQpFj3iE
 5DhOpkXGTC6UHm2bI9ftvfkpfRVTc3ow4EnY+ytx4YIXjSnNn/xHnYmNJxVWQimQ
 jPCUuTQhPTX7BcYd/TGwu6inXe3f1I0vP7oLXUXA239wc/b/Y6KOfYJdewEGefqC
 2kPxd/Towtx+HfGb8bicXYpHYDP0JrFfUClDtCHIsWmm2tlv4rkT0TD9gkhRjkP+
 hQTyZlWF2279Cmk2c+r5ou7HRqK2EnUnry1N5YuY7+hKnqBvMW3mD8XNMvzIqKIJ
 CksFLKtHBx4OJojP3q6HX5BYFRzCItG3uUirQ3xZTzjZBEl8nKTKWIAoPtT7STgu
 qe9WCIelMvLp0tW1zVodS1L9tYXm2ijbqcrzAM0XaaysBiT3YrgN6R/m8fHyT3VP
 0oqg8ozUTLPj6kjH6CsVAgjLhRvWFCZB+YH8kqRphmLXOXCWDmkdbaLL16R7kjGQ
 ajsx9lSkqFicUgycTpmlXlY6VdvPMekGbxBewf2H6PMuVHIO3LPnD80Z190yoMZs
 9KzNQAFoy2uJs4nHVupa+dgqViQHMFt4wFGLg9+qnolcclA9PCQ/6nnMqSc2jQJ7
 B8UGqesUR5j/9KD6vXE6Y9v7qZtTEc7NLwWVVfzXaHRmBIaFlWP9o0rJnFm3pGsZ
 WDC/bktf1XzCsHvCwIrS
 =jsOv
 -----END PGP SIGNATURE-----

Merge tag 'kvm_mips_20150327' of git://git.kernel.org/pub/scm/linux/kernel/git/jhogan/kvm-mips into kvm-next

MIPS KVM Guest FPU & SIMD (MSA) Support

Add guest FPU and MIPS SIMD Architecture (MSA) support to MIPS KVM, by
enabling the host FPU/MSA while in guest mode. This adds two new KVM
capabilities, KVM_CAP_MIPS_FPU & KVM_CAP_MIPS_MSA, and supports the 3 FP
register modes (FR=0, FR=1, FRE=1), and 128-bit MSA vector registers,
with lazy FPU/MSA context save and restore.

Some required MIPS FP/MSA fixes are merged in from a branch in the MIPS
tree first.
This commit is contained in:
Paolo Bonzini 2015-03-30 16:37:06 +02:00
commit 65844bb0a7
23 changed files with 1875 additions and 364 deletions

54
Documentation/virtual/kvm/api.txt
View file

@ -1967,15 +1967,25 @@ registers, find a list below:
MIPS | KVM_REG_MIPS_CP0_STATUS | 32
MIPS | KVM_REG_MIPS_CP0_CAUSE | 32
MIPS | KVM_REG_MIPS_CP0_EPC | 64
MIPS | KVM_REG_MIPS_CP0_PRID | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG1 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG2 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG3 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG4 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG5 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG7 | 32
MIPS | KVM_REG_MIPS_CP0_ERROREPC | 64
MIPS | KVM_REG_MIPS_COUNT_CTL | 64
MIPS | KVM_REG_MIPS_COUNT_RESUME | 64
MIPS | KVM_REG_MIPS_COUNT_HZ | 64
MIPS | KVM_REG_MIPS_FPR_32(0..31) | 32
MIPS | KVM_REG_MIPS_FPR_64(0..31) | 64
MIPS | KVM_REG_MIPS_VEC_128(0..31) | 128
MIPS | KVM_REG_MIPS_FCR_IR | 32
MIPS | KVM_REG_MIPS_FCR_CSR | 32
MIPS | KVM_REG_MIPS_MSA_IR | 32
MIPS | KVM_REG_MIPS_MSA_CSR | 32
ARM registers are mapped using the lower 32 bits. The upper 16 of that
is the register group type, or coprocessor number:
@ -2029,6 +2039,25 @@ patterns depending on whether they're 32-bit or 64-bit registers:
MIPS KVM control registers (see above) have the following id bit patterns:
0x7030 0000 0002 <reg:16>
MIPS FPU registers (see KVM_REG_MIPS_FPR_{32,64}() above) have the following
id bit patterns depending on the size of the register being accessed. They are
always accessed according to the current guest FPU mode (Status.FR and
Config5.FRE), i.e. as the guest would see them, and they become unpredictable
if the guest FPU mode is changed. MIPS SIMD Architecture (MSA) vector
registers (see KVM_REG_MIPS_VEC_128() above) have similar patterns as they
overlap the FPU registers:
0x7020 0000 0003 00 <0:3> <reg:5> (32-bit FPU registers)
0x7030 0000 0003 00 <0:3> <reg:5> (64-bit FPU registers)
0x7040 0000 0003 00 <0:3> <reg:5> (128-bit MSA vector registers)
MIPS FPU control registers (see KVM_REG_MIPS_FCR_{IR,CSR} above) have the
following id bit patterns:
0x7020 0000 0003 01 <0:3> <reg:5>
MIPS MSA control registers (see KVM_REG_MIPS_MSA_{IR,CSR} above) have the
following id bit patterns:
0x7020 0000 0003 02 <0:3> <reg:5>
4.69 KVM_GET_ONE_REG
@ -3293,6 +3322,31 @@ Parameters: none
This capability enables the in-kernel irqchip for s390. Please refer to
"4.24 KVM_CREATE_IRQCHIP" for details.
6.9 KVM_CAP_MIPS_FPU
Architectures: mips
Target: vcpu
Parameters: args[0] is reserved for future use (should be 0).
This capability allows the use of the host Floating Point Unit by the guest. It
allows the Config1.FP bit to be set to enable the FPU in the guest. Once this is
done the KVM_REG_MIPS_FPR_* and KVM_REG_MIPS_FCR_* registers can be accessed
(depending on the current guest FPU register mode), and the Status.FR,
Config5.FRE bits are accessible via the KVM API and also from the guest,
depending on them being supported by the FPU.
6.10 KVM_CAP_MIPS_MSA
Architectures: mips
Target: vcpu
Parameters: args[0] is reserved for future use (should be 0).
This capability allows the use of the MIPS SIMD Architecture (MSA) by the guest.
It allows the Config3.MSAP bit to be set to enable the use of MSA by the guest.
Once this is done the KVM_REG_MIPS_VEC_* and KVM_REG_MIPS_MSA_* registers can be
accessed, and the Config5.MSAEn bit is accessible via the KVM API and also from
the guest.
7. Capabilities that can be enabled on VMs
------------------------------------------

128
arch/mips/include/asm/asmmacro-32.h
View file

@ -16,38 +16,38 @@
.set push
SET_HARDFLOAT
cfc1 \tmp, fcr31
swc1 $f0, THREAD_FPR0_LS64(\thread)
swc1 $f1, THREAD_FPR1_LS64(\thread)
swc1 $f2, THREAD_FPR2_LS64(\thread)
swc1 $f3, THREAD_FPR3_LS64(\thread)
swc1 $f4, THREAD_FPR4_LS64(\thread)
swc1 $f5, THREAD_FPR5_LS64(\thread)
swc1 $f6, THREAD_FPR6_LS64(\thread)
swc1 $f7, THREAD_FPR7_LS64(\thread)
swc1 $f8, THREAD_FPR8_LS64(\thread)
swc1 $f9, THREAD_FPR9_LS64(\thread)
swc1 $f10, THREAD_FPR10_LS64(\thread)
swc1 $f11, THREAD_FPR11_LS64(\thread)
swc1 $f12, THREAD_FPR12_LS64(\thread)
swc1 $f13, THREAD_FPR13_LS64(\thread)
swc1 $f14, THREAD_FPR14_LS64(\thread)
swc1 $f15, THREAD_FPR15_LS64(\thread)
swc1 $f16, THREAD_FPR16_LS64(\thread)
swc1 $f17, THREAD_FPR17_LS64(\thread)
swc1 $f18, THREAD_FPR18_LS64(\thread)
swc1 $f19, THREAD_FPR19_LS64(\thread)
swc1 $f20, THREAD_FPR20_LS64(\thread)
swc1 $f21, THREAD_FPR21_LS64(\thread)
swc1 $f22, THREAD_FPR22_LS64(\thread)
swc1 $f23, THREAD_FPR23_LS64(\thread)
swc1 $f24, THREAD_FPR24_LS64(\thread)
swc1 $f25, THREAD_FPR25_LS64(\thread)
swc1 $f26, THREAD_FPR26_LS64(\thread)
swc1 $f27, THREAD_FPR27_LS64(\thread)
swc1 $f28, THREAD_FPR28_LS64(\thread)
swc1 $f29, THREAD_FPR29_LS64(\thread)
swc1 $f30, THREAD_FPR30_LS64(\thread)
swc1 $f31, THREAD_FPR31_LS64(\thread)
swc1 $f0, THREAD_FPR0(\thread)
swc1 $f1, THREAD_FPR1(\thread)
swc1 $f2, THREAD_FPR2(\thread)
swc1 $f3, THREAD_FPR3(\thread)
swc1 $f4, THREAD_FPR4(\thread)
swc1 $f5, THREAD_FPR5(\thread)
swc1 $f6, THREAD_FPR6(\thread)
swc1 $f7, THREAD_FPR7(\thread)
swc1 $f8, THREAD_FPR8(\thread)
swc1 $f9, THREAD_FPR9(\thread)
swc1 $f10, THREAD_FPR10(\thread)
swc1 $f11, THREAD_FPR11(\thread)
swc1 $f12, THREAD_FPR12(\thread)
swc1 $f13, THREAD_FPR13(\thread)
swc1 $f14, THREAD_FPR14(\thread)
swc1 $f15, THREAD_FPR15(\thread)
swc1 $f16, THREAD_FPR16(\thread)
swc1 $f17, THREAD_FPR17(\thread)
swc1 $f18, THREAD_FPR18(\thread)
swc1 $f19, THREAD_FPR19(\thread)
swc1 $f20, THREAD_FPR20(\thread)
swc1 $f21, THREAD_FPR21(\thread)
swc1 $f22, THREAD_FPR22(\thread)
swc1 $f23, THREAD_FPR23(\thread)
swc1 $f24, THREAD_FPR24(\thread)
swc1 $f25, THREAD_FPR25(\thread)
swc1 $f26, THREAD_FPR26(\thread)
swc1 $f27, THREAD_FPR27(\thread)
swc1 $f28, THREAD_FPR28(\thread)
swc1 $f29, THREAD_FPR29(\thread)
swc1 $f30, THREAD_FPR30(\thread)
swc1 $f31, THREAD_FPR31(\thread)
sw \tmp, THREAD_FCR31(\thread)
.set pop
.endm
@ -56,38 +56,38 @@
.set push
SET_HARDFLOAT
lw \tmp, THREAD_FCR31(\thread)
lwc1 $f0, THREAD_FPR0_LS64(\thread)
lwc1 $f1, THREAD_FPR1_LS64(\thread)
lwc1 $f2, THREAD_FPR2_LS64(\thread)
lwc1 $f3, THREAD_FPR3_LS64(\thread)
lwc1 $f4, THREAD_FPR4_LS64(\thread)
lwc1 $f5, THREAD_FPR5_LS64(\thread)
lwc1 $f6, THREAD_FPR6_LS64(\thread)
lwc1 $f7, THREAD_FPR7_LS64(\thread)
lwc1 $f8, THREAD_FPR8_LS64(\thread)
lwc1 $f9, THREAD_FPR9_LS64(\thread)
lwc1 $f10, THREAD_FPR10_LS64(\thread)
lwc1 $f11, THREAD_FPR11_LS64(\thread)
lwc1 $f12, THREAD_FPR12_LS64(\thread)
lwc1 $f13, THREAD_FPR13_LS64(\thread)
lwc1 $f14, THREAD_FPR14_LS64(\thread)
lwc1 $f15, THREAD_FPR15_LS64(\thread)
lwc1 $f16, THREAD_FPR16_LS64(\thread)
lwc1 $f17, THREAD_FPR17_LS64(\thread)
lwc1 $f18, THREAD_FPR18_LS64(\thread)
lwc1 $f19, THREAD_FPR19_LS64(\thread)
lwc1 $f20, THREAD_FPR20_LS64(\thread)
lwc1 $f21, THREAD_FPR21_LS64(\thread)
lwc1 $f22, THREAD_FPR22_LS64(\thread)
lwc1 $f23, THREAD_FPR23_LS64(\thread)
lwc1 $f24, THREAD_FPR24_LS64(\thread)
lwc1 $f25, THREAD_FPR25_LS64(\thread)
lwc1 $f26, THREAD_FPR26_LS64(\thread)
lwc1 $f27, THREAD_FPR27_LS64(\thread)
lwc1 $f28, THREAD_FPR28_LS64(\thread)
lwc1 $f29, THREAD_FPR29_LS64(\thread)
lwc1 $f30, THREAD_FPR30_LS64(\thread)
lwc1 $f31, THREAD_FPR31_LS64(\thread)
lwc1 $f0, THREAD_FPR0(\thread)
lwc1 $f1, THREAD_FPR1(\thread)
lwc1 $f2, THREAD_FPR2(\thread)
lwc1 $f3, THREAD_FPR3(\thread)
lwc1 $f4, THREAD_FPR4(\thread)
lwc1 $f5, THREAD_FPR5(\thread)
lwc1 $f6, THREAD_FPR6(\thread)
lwc1 $f7, THREAD_FPR7(\thread)
lwc1 $f8, THREAD_FPR8(\thread)
lwc1 $f9, THREAD_FPR9(\thread)
lwc1 $f10, THREAD_FPR10(\thread)
lwc1 $f11, THREAD_FPR11(\thread)
lwc1 $f12, THREAD_FPR12(\thread)
lwc1 $f13, THREAD_FPR13(\thread)
lwc1 $f14, THREAD_FPR14(\thread)
lwc1 $f15, THREAD_FPR15(\thread)
lwc1 $f16, THREAD_FPR16(\thread)
lwc1 $f17, THREAD_FPR17(\thread)
lwc1 $f18, THREAD_FPR18(\thread)
lwc1 $f19, THREAD_FPR19(\thread)
lwc1 $f20, THREAD_FPR20(\thread)
lwc1 $f21, THREAD_FPR21(\thread)
lwc1 $f22, THREAD_FPR22(\thread)
lwc1 $f23, THREAD_FPR23(\thread)
lwc1 $f24, THREAD_FPR24(\thread)
lwc1 $f25, THREAD_FPR25(\thread)
lwc1 $f26, THREAD_FPR26(\thread)
lwc1 $f27, THREAD_FPR27(\thread)
lwc1 $f28, THREAD_FPR28(\thread)
lwc1 $f29, THREAD_FPR29(\thread)
lwc1 $f30, THREAD_FPR30(\thread)
lwc1 $f31, THREAD_FPR31(\thread)
ctc1 \tmp, fcr31
.set pop
.endm

218
arch/mips/include/asm/asmmacro.h
View file

@ -60,22 +60,22 @@
.set push
SET_HARDFLOAT
cfc1 \tmp, fcr31
sdc1 $f0, THREAD_FPR0_LS64(\thread)
sdc1 $f2, THREAD_FPR2_LS64(\thread)
sdc1 $f4, THREAD_FPR4_LS64(\thread)
sdc1 $f6, THREAD_FPR6_LS64(\thread)
sdc1 $f8, THREAD_FPR8_LS64(\thread)
sdc1 $f10, THREAD_FPR10_LS64(\thread)
sdc1 $f12, THREAD_FPR12_LS64(\thread)
sdc1 $f14, THREAD_FPR14_LS64(\thread)
sdc1 $f16, THREAD_FPR16_LS64(\thread)
sdc1 $f18, THREAD_FPR18_LS64(\thread)
sdc1 $f20, THREAD_FPR20_LS64(\thread)
sdc1 $f22, THREAD_FPR22_LS64(\thread)
sdc1 $f24, THREAD_FPR24_LS64(\thread)
sdc1 $f26, THREAD_FPR26_LS64(\thread)
sdc1 $f28, THREAD_FPR28_LS64(\thread)
sdc1 $f30, THREAD_FPR30_LS64(\thread)
sdc1 $f0, THREAD_FPR0(\thread)
sdc1 $f2, THREAD_FPR2(\thread)
sdc1 $f4, THREAD_FPR4(\thread)
sdc1 $f6, THREAD_FPR6(\thread)
sdc1 $f8, THREAD_FPR8(\thread)
sdc1 $f10, THREAD_FPR10(\thread)
sdc1 $f12, THREAD_FPR12(\thread)
sdc1 $f14, THREAD_FPR14(\thread)
sdc1 $f16, THREAD_FPR16(\thread)
sdc1 $f18, THREAD_FPR18(\thread)
sdc1 $f20, THREAD_FPR20(\thread)
sdc1 $f22, THREAD_FPR22(\thread)
sdc1 $f24, THREAD_FPR24(\thread)
sdc1 $f26, THREAD_FPR26(\thread)
sdc1 $f28, THREAD_FPR28(\thread)
sdc1 $f30, THREAD_FPR30(\thread)
sw \tmp, THREAD_FCR31(\thread)
.set pop
.endm
@ -84,22 +84,22 @@
.set push
.set mips64r2
SET_HARDFLOAT
sdc1 $f1, THREAD_FPR1_LS64(\thread)
sdc1 $f3, THREAD_FPR3_LS64(\thread)
sdc1 $f5, THREAD_FPR5_LS64(\thread)
sdc1 $f7, THREAD_FPR7_LS64(\thread)
sdc1 $f9, THREAD_FPR9_LS64(\thread)
sdc1 $f11, THREAD_FPR11_LS64(\thread)
sdc1 $f13, THREAD_FPR13_LS64(\thread)
sdc1 $f15, THREAD_FPR15_LS64(\thread)
sdc1 $f17, THREAD_FPR17_LS64(\thread)
sdc1 $f19, THREAD_FPR19_LS64(\thread)
sdc1 $f21, THREAD_FPR21_LS64(\thread)
sdc1 $f23, THREAD_FPR23_LS64(\thread)
sdc1 $f25, THREAD_FPR25_LS64(\thread)
sdc1 $f27, THREAD_FPR27_LS64(\thread)
sdc1 $f29, THREAD_FPR29_LS64(\thread)
sdc1 $f31, THREAD_FPR31_LS64(\thread)
sdc1 $f1, THREAD_FPR1(\thread)
sdc1 $f3, THREAD_FPR3(\thread)
sdc1 $f5, THREAD_FPR5(\thread)
sdc1 $f7, THREAD_FPR7(\thread)
sdc1 $f9, THREAD_FPR9(\thread)
sdc1 $f11, THREAD_FPR11(\thread)
sdc1 $f13, THREAD_FPR13(\thread)
sdc1 $f15, THREAD_FPR15(\thread)
sdc1 $f17, THREAD_FPR17(\thread)
sdc1 $f19, THREAD_FPR19(\thread)
sdc1 $f21, THREAD_FPR21(\thread)
sdc1 $f23, THREAD_FPR23(\thread)
sdc1 $f25, THREAD_FPR25(\thread)
sdc1 $f27, THREAD_FPR27(\thread)
sdc1 $f29, THREAD_FPR29(\thread)
sdc1 $f31, THREAD_FPR31(\thread)
.set pop
.endm
@ -118,22 +118,22 @@
.set push
SET_HARDFLOAT
lw \tmp, THREAD_FCR31(\thread)
ldc1 $f0, THREAD_FPR0_LS64(\thread)
ldc1 $f2, THREAD_FPR2_LS64(\thread)
ldc1 $f4, THREAD_FPR4_LS64(\thread)
ldc1 $f6, THREAD_FPR6_LS64(\thread)
ldc1 $f8, THREAD_FPR8_LS64(\thread)
ldc1 $f10, THREAD_FPR10_LS64(\thread)
ldc1 $f12, THREAD_FPR12_LS64(\thread)
ldc1 $f14, THREAD_FPR14_LS64(\thread)
ldc1 $f16, THREAD_FPR16_LS64(\thread)
ldc1 $f18, THREAD_FPR18_LS64(\thread)
ldc1 $f20, THREAD_FPR20_LS64(\thread)
ldc1 $f22, THREAD_FPR22_LS64(\thread)
ldc1 $f24, THREAD_FPR24_LS64(\thread)
ldc1 $f26, THREAD_FPR26_LS64(\thread)
ldc1 $f28, THREAD_FPR28_LS64(\thread)
ldc1 $f30, THREAD_FPR30_LS64(\thread)
ldc1 $f0, THREAD_FPR0(\thread)
ldc1 $f2, THREAD_FPR2(\thread)
ldc1 $f4, THREAD_FPR4(\thread)
ldc1 $f6, THREAD_FPR6(\thread)
ldc1 $f8, THREAD_FPR8(\thread)
ldc1 $f10, THREAD_FPR10(\thread)
ldc1 $f12, THREAD_FPR12(\thread)
ldc1 $f14, THREAD_FPR14(\thread)
ldc1 $f16, THREAD_FPR16(\thread)
ldc1 $f18, THREAD_FPR18(\thread)
ldc1 $f20, THREAD_FPR20(\thread)
ldc1 $f22, THREAD_FPR22(\thread)
ldc1 $f24, THREAD_FPR24(\thread)
ldc1 $f26, THREAD_FPR26(\thread)
ldc1 $f28, THREAD_FPR28(\thread)
ldc1 $f30, THREAD_FPR30(\thread)
ctc1 \tmp, fcr31
.endm
@ -141,22 +141,22 @@
.set push
.set mips64r2
SET_HARDFLOAT
ldc1 $f1, THREAD_FPR1_LS64(\thread)
ldc1 $f3, THREAD_FPR3_LS64(\thread)
ldc1 $f5, THREAD_FPR5_LS64(\thread)
ldc1 $f7, THREAD_FPR7_LS64(\thread)
ldc1 $f9, THREAD_FPR9_LS64(\thread)
ldc1 $f11, THREAD_FPR11_LS64(\thread)
ldc1 $f13, THREAD_FPR13_LS64(\thread)
ldc1 $f15, THREAD_FPR15_LS64(\thread)
ldc1 $f17, THREAD_FPR17_LS64(\thread)
ldc1 $f19, THREAD_FPR19_LS64(\thread)
ldc1 $f21, THREAD_FPR21_LS64(\thread)
ldc1 $f23, THREAD_FPR23_LS64(\thread)
ldc1 $f25, THREAD_FPR25_LS64(\thread)
ldc1 $f27, THREAD_FPR27_LS64(\thread)
ldc1 $f29, THREAD_FPR29_LS64(\thread)
ldc1 $f31, THREAD_FPR31_LS64(\thread)
ldc1 $f1, THREAD_FPR1(\thread)
ldc1 $f3, THREAD_FPR3(\thread)
ldc1 $f5, THREAD_FPR5(\thread)
ldc1 $f7, THREAD_FPR7(\thread)
ldc1 $f9, THREAD_FPR9(\thread)
ldc1 $f11, THREAD_FPR11(\thread)
ldc1 $f13, THREAD_FPR13(\thread)
ldc1 $f15, THREAD_FPR15(\thread)
ldc1 $f17, THREAD_FPR17(\thread)
ldc1 $f19, THREAD_FPR19(\thread)
ldc1 $f21, THREAD_FPR21(\thread)
ldc1 $f23, THREAD_FPR23(\thread)
ldc1 $f25, THREAD_FPR25(\thread)
ldc1 $f27, THREAD_FPR27(\thread)
ldc1 $f29, THREAD_FPR29(\thread)
ldc1 $f31, THREAD_FPR31(\thread)
.set pop
.endm
@ -211,6 +211,22 @@
.endm
#ifdef TOOLCHAIN_SUPPORTS_MSA
.macro _cfcmsa rd, cs
.set push
.set mips32r2
.set msa
cfcmsa \rd, $\cs
.set pop
.endm
.macro _ctcmsa cd, rs
.set push
.set mips32r2
.set msa
ctcmsa $\cd, \rs
.set pop
.endm
.macro ld_d wd, off, base
.set push
.set mips32r2
@ -227,35 +243,35 @@
.set pop
.endm
.macro copy_u_w rd, ws, n
.macro copy_u_w ws, n
.set push
.set mips32r2
.set msa
copy_u.w \rd, $w\ws[\n]
copy_u.w $1, $w\ws[\n]
.set pop
.endm
.macro copy_u_d rd, ws, n
.macro copy_u_d ws, n
.set push
.set mips64r2
.set msa
copy_u.d \rd, $w\ws[\n]
copy_u.d $1, $w\ws[\n]
.set pop
.endm
.macro insert_w wd, n, rs
.macro insert_w wd, n
.set push
.set mips32r2
.set msa
insert.w $w\wd[\n], \rs
insert.w $w\wd[\n], $1
.set pop
.endm
.macro insert_d wd, n, rs
.macro insert_d wd, n
.set push
.set mips64r2
.set msa
insert.d $w\wd[\n], \rs
insert.d $w\wd[\n], $1
.set pop
.endm
#else
@ -283,7 +299,7 @@
/*
* Temporary until all toolchains in use include MSA support.
*/
.macro cfcmsa rd, cs
.macro _cfcmsa rd, cs
.set push
.set noat
SET_HARDFLOAT
@ -293,7 +309,7 @@
.set pop
.endm
.macro ctcmsa cd, rs
.macro _ctcmsa cd, rs
.set push
.set noat
SET_HARDFLOAT
@ -320,44 +336,36 @@
.set pop
.endm
.macro copy_u_w rd, ws, n
.macro copy_u_w ws, n
.set push
.set noat
SET_HARDFLOAT
.insn
.word COPY_UW_MSA_INSN | (\n << 16) | (\ws << 11)
/* move triggers an assembler bug... */
or \rd, $1, zero
.set pop
.endm
.macro copy_u_d rd, ws, n
.macro copy_u_d ws, n
.set push
.set noat
SET_HARDFLOAT
.insn
.word COPY_UD_MSA_INSN | (\n << 16) | (\ws << 11)
/* move triggers an assembler bug... */
or \rd, $1, zero
.set pop
.endm
.macro insert_w wd, n, rs
.macro insert_w wd, n
.set push
.set noat
SET_HARDFLOAT
/* move triggers an assembler bug... */
or $1, \rs, zero
.word INSERT_W_MSA_INSN | (\n << 16) | (\wd << 6)
.set pop
.endm
.macro insert_d wd, n, rs
.macro insert_d wd, n
.set push
.set noat
SET_HARDFLOAT
/* move triggers an assembler bug... */
or $1, \rs, zero
.word INSERT_D_MSA_INSN | (\n << 16) | (\wd << 6)
.set pop
.endm
@ -399,7 +407,7 @@
.set push
.set noat
SET_HARDFLOAT
cfcmsa $1, MSA_CSR
_cfcmsa $1, MSA_CSR
sw $1, THREAD_MSA_CSR(\thread)
.set pop
.endm
@ -409,7 +417,7 @@
.set noat
SET_HARDFLOAT
lw $1, THREAD_MSA_CSR(\thread)
ctcmsa MSA_CSR, $1
_ctcmsa MSA_CSR, $1
.set pop
ld_d 0, THREAD_FPR0, \thread
ld_d 1, THREAD_FPR1, \thread
@ -452,9 +460,6 @@
insert_w \wd, 2
insert_w \wd, 3
#endif
.if 31-\wd
msa_init_upper (\wd+1)
.endif
.endm
.macro msa_init_all_upper
@ -463,6 +468,37 @@
SET_HARDFLOAT
not $1, zero
msa_init_upper 0
msa_init_upper 1
msa_init_upper 2
msa_init_upper 3
msa_init_upper 4
msa_init_upper 5
msa_init_upper 6
msa_init_upper 7
msa_init_upper 8
msa_init_upper 9
msa_init_upper 10
msa_init_upper 11
msa_init_upper 12
msa_init_upper 13
msa_init_upper 14
msa_init_upper 15
msa_init_upper 16
msa_init_upper 17
msa_init_upper 18
msa_init_upper 19
msa_init_upper 20
msa_init_upper 21
msa_init_upper 22
msa_init_upper 23
msa_init_upper 24
msa_init_upper 25
msa_init_upper 26
msa_init_upper 27
msa_init_upper 28
msa_init_upper 29
msa_init_upper 30
msa_init_upper 31
.set pop
.endm

20
arch/mips/include/asm/fpu.h
View file

@ -48,6 +48,12 @@ enum fpu_mode {
#define FPU_FR_MASK 0x1
};
#define __disable_fpu() \
do { \
clear_c0_status(ST0_CU1); \
disable_fpu_hazard(); \
} while (0)
static inline int __enable_fpu(enum fpu_mode mode)
{
int fr;
@ -86,7 +92,12 @@ static inline int __enable_fpu(enum fpu_mode mode)
enable_fpu_hazard();
/* check FR has the desired value */
return (!!(read_c0_status() & ST0_FR) == !!fr) ? 0 : SIGFPE;
if (!!(read_c0_status() & ST0_FR) == !!fr)
return 0;
/* unsupported FR value */
__disable_fpu();
return SIGFPE;
default:
BUG();
@ -95,12 +106,6 @@ static inline int __enable_fpu(enum fpu_mode mode)
return SIGFPE;
}
#define __disable_fpu() \
do { \
clear_c0_status(ST0_CU1); \
disable_fpu_hazard(); \
} while (0)
#define clear_fpu_owner() clear_thread_flag(TIF_USEDFPU)
static inline int __is_fpu_owner(void)
@ -170,6 +175,7 @@ static inline void lose_fpu(int save)
}
disable_msa();
clear_thread_flag(TIF_USEDMSA);
__disable_fpu();
} else if (is_fpu_owner()) {
if (save)
_save_fp(current);

3
arch/mips/include/asm/kdebug.h
View file

@ -10,7 +10,8 @@ enum die_val {
DIE_RI,
DIE_PAGE_FAULT,
DIE_BREAK,
DIE_SSTEPBP
DIE_SSTEPBP,
DIE_MSAFP
};
#endif /* _ASM_MIPS_KDEBUG_H */

125
arch/mips/include/asm/kvm_host.h
View file

@ -21,10 +21,10 @@
/* MIPS KVM register ids */
#define MIPS_CP0_32(_R, _S) \
(KVM_REG_MIPS | KVM_REG_SIZE_U32 | 0x10000 | (8 * (_R) + (_S)))
(KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
#define MIPS_CP0_64(_R, _S) \
(KVM_REG_MIPS | KVM_REG_SIZE_U64 | 0x10000 | (8 * (_R) + (_S)))
(KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U64 | (8 * (_R) + (_S)))
#define KVM_REG_MIPS_CP0_INDEX MIPS_CP0_32(0, 0)
#define KVM_REG_MIPS_CP0_ENTRYLO0 MIPS_CP0_64(2, 0)
@ -42,11 +42,14 @@
#define KVM_REG_MIPS_CP0_STATUS MIPS_CP0_32(12, 0)
#define KVM_REG_MIPS_CP0_CAUSE MIPS_CP0_32(13, 0)
#define KVM_REG_MIPS_CP0_EPC MIPS_CP0_64(14, 0)
#define KVM_REG_MIPS_CP0_PRID MIPS_CP0_32(15, 0)
#define KVM_REG_MIPS_CP0_EBASE MIPS_CP0_64(15, 1)
#define KVM_REG_MIPS_CP0_CONFIG MIPS_CP0_32(16, 0)
#define KVM_REG_MIPS_CP0_CONFIG1 MIPS_CP0_32(16, 1)
#define KVM_REG_MIPS_CP0_CONFIG2 MIPS_CP0_32(16, 2)
#define KVM_REG_MIPS_CP0_CONFIG3 MIPS_CP0_32(16, 3)
#define KVM_REG_MIPS_CP0_CONFIG4 MIPS_CP0_32(16, 4)
#define KVM_REG_MIPS_CP0_CONFIG5 MIPS_CP0_32(16, 5)
#define KVM_REG_MIPS_CP0_CONFIG7 MIPS_CP0_32(16, 7)
#define KVM_REG_MIPS_CP0_XCONTEXT MIPS_CP0_64(20, 0)
#define KVM_REG_MIPS_CP0_ERROREPC MIPS_CP0_64(30, 0)
@ -119,6 +122,10 @@ struct kvm_vcpu_stat {
u32 syscall_exits;
u32 resvd_inst_exits;
u32 break_inst_exits;
u32 trap_inst_exits;
u32 msa_fpe_exits;
u32 fpe_exits;
u32 msa_disabled_exits;
u32 flush_dcache_exits;
u32 halt_successful_poll;
u32 halt_wakeup;
@ -138,6 +145,10 @@ enum kvm_mips_exit_types {
SYSCALL_EXITS,
RESVD_INST_EXITS,
BREAK_INST_EXITS,
TRAP_INST_EXITS,
MSA_FPE_EXITS,
FPE_EXITS,
MSA_DISABLED_EXITS,
FLUSH_DCACHE_EXITS,
MAX_KVM_MIPS_EXIT_TYPES
};
@ -206,6 +217,8 @@ struct mips_coproc {
#define MIPS_CP0_CONFIG1_SEL 1
#define MIPS_CP0_CONFIG2_SEL 2
#define MIPS_CP0_CONFIG3_SEL 3
#define MIPS_CP0_CONFIG4_SEL 4
#define MIPS_CP0_CONFIG5_SEL 5
/* Config0 register bits */
#define CP0C0_M 31
@ -262,31 +275,6 @@ struct mips_coproc {
#define CP0C3_SM 1
#define CP0C3_TL 0
/* Have config1, Cacheable, noncoherent, write-back, write allocate*/
#define MIPS_CONFIG0 \
((1 << CP0C0_M) | (0x3 << CP0C0_K0))
/* Have config2, no coprocessor2 attached, no MDMX support attached,
no performance counters, watch registers present,
no code compression, EJTAG present, no FPU, no watch registers */
#define MIPS_CONFIG1 \
((1 << CP0C1_M) | \
(0 << CP0C1_C2) | (0 << CP0C1_MD) | (0 << CP0C1_PC) | \
(0 << CP0C1_WR) | (0 << CP0C1_CA) | (1 << CP0C1_EP) | \
(0 << CP0C1_FP))
/* Have config3, no tertiary/secondary caches implemented */
#define MIPS_CONFIG2 \
((1 << CP0C2_M))
/* No config4, no DSP ASE, no large physaddr (PABITS),
no external interrupt controller, no vectored interrupts,
no 1kb pages, no SmartMIPS ASE, no trace logic */
#define MIPS_CONFIG3 \
((0 << CP0C3_M) | (0 << CP0C3_DSPP) | (0 << CP0C3_LPA) | \
(0 << CP0C3_VEIC) | (0 << CP0C3_VInt) | (0 << CP0C3_SP) | \
(0 << CP0C3_SM) | (0 << CP0C3_TL))
/* MMU types, the first four entries have the same layout as the
CP0C0_MT field. */
enum mips_mmu_types {
@ -321,7 +309,9 @@ enum mips_mmu_types {
*/
#define T_TRAP 13 /* Trap instruction */
#define T_VCEI 14 /* Virtual coherency exception */
#define T_MSAFPE 14 /* MSA floating point exception */
#define T_FPE 15 /* Floating point exception */
#define T_MSADIS 21 /* MSA disabled exception */
#define T_WATCH 23 /* Watch address reference */
#define T_VCED 31 /* Virtual coherency data */
@ -374,6 +364,9 @@ struct kvm_mips_tlb {
long tlb_lo1;
};
#define KVM_MIPS_FPU_FPU 0x1
#define KVM_MIPS_FPU_MSA 0x2
#define KVM_MIPS_GUEST_TLB_SIZE 64
struct kvm_vcpu_arch {
void *host_ebase, *guest_ebase;
@ -395,6 +388,8 @@ struct kvm_vcpu_arch {
/* FPU State */
struct mips_fpu_struct fpu;
/* Which FPU state is loaded (KVM_MIPS_FPU_*) */
unsigned int fpu_inuse;
/* COP0 State */
struct mips_coproc *cop0;
@ -441,6 +436,9 @@ struct kvm_vcpu_arch {
/* WAIT executed */
int wait;
u8 fpu_enabled;
u8 msa_enabled;
};
@ -482,11 +480,15 @@ struct kvm_vcpu_arch {
#define kvm_read_c0_guest_config1(cop0) (cop0->reg[MIPS_CP0_CONFIG][1])
#define kvm_read_c0_guest_config2(cop0) (cop0->reg[MIPS_CP0_CONFIG][2])
#define kvm_read_c0_guest_config3(cop0) (cop0->reg[MIPS_CP0_CONFIG][3])
#define kvm_read_c0_guest_config4(cop0) (cop0->reg[MIPS_CP0_CONFIG][4])
#define kvm_read_c0_guest_config5(cop0) (cop0->reg[MIPS_CP0_CONFIG][5])
#define kvm_read_c0_guest_config7(cop0) (cop0->reg[MIPS_CP0_CONFIG][7])
#define kvm_write_c0_guest_config(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][0] = (val))
#define kvm_write_c0_guest_config1(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][1] = (val))
#define kvm_write_c0_guest_config2(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][2] = (val))
#define kvm_write_c0_guest_config3(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][3] = (val))
#define kvm_write_c0_guest_config4(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][4] = (val))
#define kvm_write_c0_guest_config5(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][5] = (val))
#define kvm_write_c0_guest_config7(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][7] = (val))
#define kvm_read_c0_guest_errorepc(cop0) (cop0->reg[MIPS_CP0_ERROR_PC][0])
#define kvm_write_c0_guest_errorepc(cop0, val) (cop0->reg[MIPS_CP0_ERROR_PC][0] = (val))
@ -567,6 +569,31 @@ static inline void _kvm_atomic_change_c0_guest_reg(unsigned long *reg,
kvm_set_c0_guest_ebase(cop0, ((val) & (change))); \
}
/* Helpers */
static inline bool kvm_mips_guest_can_have_fpu(struct kvm_vcpu_arch *vcpu)
{
return (!__builtin_constant_p(cpu_has_fpu) || cpu_has_fpu) &&
vcpu->fpu_enabled;
}
static inline bool kvm_mips_guest_has_fpu(struct kvm_vcpu_arch *vcpu)
{
return kvm_mips_guest_can_have_fpu(vcpu) &&
kvm_read_c0_guest_config1(vcpu->cop0) & MIPS_CONF1_FP;
}
static inline bool kvm_mips_guest_can_have_msa(struct kvm_vcpu_arch *vcpu)
{
return (!__builtin_constant_p(cpu_has_msa) || cpu_has_msa) &&
vcpu->msa_enabled;
}
static inline bool kvm_mips_guest_has_msa(struct kvm_vcpu_arch *vcpu)
{
return kvm_mips_guest_can_have_msa(vcpu) &&
kvm_read_c0_guest_config3(vcpu->cop0) & MIPS_CONF3_MSA;
}
struct kvm_mips_callbacks {
int (*handle_cop_unusable)(struct kvm_vcpu *vcpu);
@ -578,6 +605,10 @@ struct kvm_mips_callbacks {
int (*handle_syscall)(struct kvm_vcpu *vcpu);
int (*handle_res_inst)(struct kvm_vcpu *vcpu);
int (*handle_break)(struct kvm_vcpu *vcpu);
int (*handle_trap)(struct kvm_vcpu *vcpu);
int (*handle_msa_fpe)(struct kvm_vcpu *vcpu);
int (*handle_fpe)(struct kvm_vcpu *vcpu);
int (*handle_msa_disabled)(struct kvm_vcpu *vcpu);
int (*vm_init)(struct kvm *kvm);
int (*vcpu_init)(struct kvm_vcpu *vcpu);
int (*vcpu_setup)(struct kvm_vcpu *vcpu);
@ -596,6 +627,8 @@ struct kvm_mips_callbacks {
const struct kvm_one_reg *reg, s64 *v);
int (*set_one_reg)(struct kvm_vcpu *vcpu,
const struct kvm_one_reg *reg, s64 v);
int (*vcpu_get_regs)(struct kvm_vcpu *vcpu);
int (*vcpu_set_regs)(struct kvm_vcpu *vcpu);
};
extern struct kvm_mips_callbacks *kvm_mips_callbacks;
int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks);
@ -606,6 +639,19 @@ int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
/* Trampoline ASM routine to start running in "Guest" context */
extern int __kvm_mips_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
/* FPU/MSA context management */
void __kvm_save_fpu(struct kvm_vcpu_arch *vcpu);
void __kvm_restore_fpu(struct kvm_vcpu_arch *vcpu);
void __kvm_restore_fcsr(struct kvm_vcpu_arch *vcpu);
void __kvm_save_msa(struct kvm_vcpu_arch *vcpu);
void __kvm_restore_msa(struct kvm_vcpu_arch *vcpu);
void __kvm_restore_msa_upper(struct kvm_vcpu_arch *vcpu);
void __kvm_restore_msacsr(struct kvm_vcpu_arch *vcpu);
void kvm_own_fpu(struct kvm_vcpu *vcpu);
void kvm_own_msa(struct kvm_vcpu *vcpu);
void kvm_drop_fpu(struct kvm_vcpu *vcpu);
void kvm_lose_fpu(struct kvm_vcpu *vcpu);
/* TLB handling */
uint32_t kvm_get_kernel_asid(struct kvm_vcpu *vcpu);
@ -711,6 +757,26 @@ extern enum emulation_result kvm_mips_emulate_bp_exc(unsigned long cause,
struct kvm_run *run,
struct kvm_vcpu *vcpu);
extern enum emulation_result kvm_mips_emulate_trap_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu);
extern enum emulation_result kvm_mips_emulate_msafpe_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu);
extern enum emulation_result kvm_mips_emulate_fpe_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu);
extern enum emulation_result kvm_mips_emulate_msadis_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu);
extern enum emulation_result kvm_mips_complete_mmio_load(struct kvm_vcpu *vcpu,
struct kvm_run *run);
@ -749,6 +815,11 @@ enum emulation_result kvm_mips_emulate_load(uint32_t inst,
struct kvm_run *run,
struct kvm_vcpu *vcpu);
unsigned int kvm_mips_config1_wrmask(struct kvm_vcpu *vcpu);
unsigned int kvm_mips_config3_wrmask(struct kvm_vcpu *vcpu);
unsigned int kvm_mips_config4_wrmask(struct kvm_vcpu *vcpu);
unsigned int kvm_mips_config5_wrmask(struct kvm_vcpu *vcpu);
/* Dynamic binary translation */
extern int kvm_mips_trans_cache_index(uint32_t inst, uint32_t *opc,
struct kvm_vcpu *vcpu);

2
arch/mips/include/asm/processor.h
View file

@ -105,7 +105,7 @@ union fpureg {
#ifdef CONFIG_CPU_LITTLE_ENDIAN
# define FPR_IDX(width, idx) (idx)
#else
# define FPR_IDX(width, idx) ((FPU_REG_WIDTH / (width)) - 1 - (idx))
# define FPR_IDX(width, idx) ((idx) ^ ((64 / (width)) - 1))
#endif
#define BUILD_FPR_ACCESS(width) \

160
arch/mips/include/uapi/asm/kvm.h
View file

@ -36,77 +36,85 @@ struct kvm_regs {
/*
* for KVM_GET_FPU and KVM_SET_FPU
*
* If Status[FR] is zero (32-bit FPU), the upper 32-bits of the FPRs
* are zero filled.
*/
struct kvm_fpu {
__u64 fpr[32];
__u32 fir;
__u32 fccr;
__u32 fexr;
__u32 fenr;
__u32 fcsr;
__u32 pad;
};
/*
* For MIPS, we use KVM_SET_ONE_REG and KVM_GET_ONE_REG to access CP0
* For MIPS, we use KVM_SET_ONE_REG and KVM_GET_ONE_REG to access various
* registers. The id field is broken down as follows:
*
* bits[2..0] - Register 'sel' index.
* bits[7..3] - Register 'rd' index.
* bits[15..8] - Must be zero.
* bits[31..16] - 1 -> CP0 registers.
* bits[51..32] - Must be zero.
* bits[63..52] - As per linux/kvm.h
* bits[51..32] - Must be zero.
* bits[31..16] - Register set.
*
* Register set = 0: GP registers from kvm_regs (see definitions below).
*
* Register set = 1: CP0 registers.
* bits[15..8] - Must be zero.
* bits[7..3] - Register 'rd' index.
* bits[2..0] - Register 'sel' index.
*
* Register set = 2: KVM specific registers (see definitions below).
*
* Register set = 3: FPU / MSA registers (see definitions below).
*
* Other sets registers may be added in the future. Each set would
* have its own identifier in bits[31..16].
*
* The registers defined in struct kvm_regs are also accessible, the
* id values for these are below.
*/
#define KVM_REG_MIPS_R0 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 0)
#define KVM_REG_MIPS_R1 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 1)
#define KVM_REG_MIPS_R2 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 2)
#define KVM_REG_MIPS_R3 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 3)
#define KVM_REG_MIPS_R4 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 4)
#define KVM_REG_MIPS_R5 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 5)
#define KVM_REG_MIPS_R6 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 6)
#define KVM_REG_MIPS_R7 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 7)
#define KVM_REG_MIPS_R8 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 8)
#define KVM_REG_MIPS_R9 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 9)
#define KVM_REG_MIPS_R10 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 10)
#define KVM_REG_MIPS_R11 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 11)
#define KVM_REG_MIPS_R12 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 12)
#define KVM_REG_MIPS_R13 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 13)
#define KVM_REG_MIPS_R14 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 14)
#define KVM_REG_MIPS_R15 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 15)
#define KVM_REG_MIPS_R16 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 16)
#define KVM_REG_MIPS_R17 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 17)
#define KVM_REG_MIPS_R18 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 18)
#define KVM_REG_MIPS_R19 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 19)
#define KVM_REG_MIPS_R20 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 20)
#define KVM_REG_MIPS_R21 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 21)
#define KVM_REG_MIPS_R22 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 22)
#define KVM_REG_MIPS_R23 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 23)
#define KVM_REG_MIPS_R24 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 24)
#define KVM_REG_MIPS_R25 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 25)
#define KVM_REG_MIPS_R26 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 26)
#define KVM_REG_MIPS_R27 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 27)
#define KVM_REG_MIPS_R28 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 28)
#define KVM_REG_MIPS_R29 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 29)
#define KVM_REG_MIPS_R30 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 30)
#define KVM_REG_MIPS_R31 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 31)
#define KVM_REG_MIPS_GP (KVM_REG_MIPS | 0x0000000000000000ULL)
#define KVM_REG_MIPS_CP0 (KVM_REG_MIPS | 0x0000000000010000ULL)
#define KVM_REG_MIPS_KVM (KVM_REG_MIPS | 0x0000000000020000ULL)
#define KVM_REG_MIPS_FPU (KVM_REG_MIPS | 0x0000000000030000ULL)
#define KVM_REG_MIPS_HI (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 32)
#define KVM_REG_MIPS_LO (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 33)
#define KVM_REG_MIPS_PC (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 34)
/* KVM specific control registers */
/*
* KVM_REG_MIPS_GP - General purpose registers from kvm_regs.
*/
#define KVM_REG_MIPS_R0 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 0)
#define KVM_REG_MIPS_R1 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 1)
#define KVM_REG_MIPS_R2 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 2)
#define KVM_REG_MIPS_R3 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 3)
#define KVM_REG_MIPS_R4 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 4)
#define KVM_REG_MIPS_R5 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 5)
#define KVM_REG_MIPS_R6 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 6)
#define KVM_REG_MIPS_R7 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 7)
#define KVM_REG_MIPS_R8 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 8)
#define KVM_REG_MIPS_R9 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 9)
#define KVM_REG_MIPS_R10 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 10)
#define KVM_REG_MIPS_R11 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 11)
#define KVM_REG_MIPS_R12 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 12)
#define KVM_REG_MIPS_R13 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 13)
#define KVM_REG_MIPS_R14 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 14)
#define KVM_REG_MIPS_R15 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 15)
#define KVM_REG_MIPS_R16 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 16)
#define KVM_REG_MIPS_R17 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 17)
#define KVM_REG_MIPS_R18 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 18)
#define KVM_REG_MIPS_R19 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 19)
#define KVM_REG_MIPS_R20 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 20)
#define KVM_REG_MIPS_R21 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 21)
#define KVM_REG_MIPS_R22 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 22)
#define KVM_REG_MIPS_R23 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 23)
#define KVM_REG_MIPS_R24 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 24)
#define KVM_REG_MIPS_R25 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 25)
#define KVM_REG_MIPS_R26 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 26)
#define KVM_REG_MIPS_R27 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 27)
#define KVM_REG_MIPS_R28 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 28)
#define KVM_REG_MIPS_R29 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 29)
#define KVM_REG_MIPS_R30 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 30)
#define KVM_REG_MIPS_R31 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 31)
#define KVM_REG_MIPS_HI (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 32)
#define KVM_REG_MIPS_LO (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 33)
#define KVM_REG_MIPS_PC (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 34)
/*
* KVM_REG_MIPS_KVM - KVM specific control registers.
*/
/*
* CP0_Count control
@ -118,8 +126,7 @@ struct kvm_fpu {
* safely without losing time or guest timer interrupts.
* Other: Reserved, do not change.
*/
#define KVM_REG_MIPS_COUNT_CTL (KVM_REG_MIPS | KVM_REG_SIZE_U64 | \
0x20000 | 0)
#define KVM_REG_MIPS_COUNT_CTL (KVM_REG_MIPS_KVM | KVM_REG_SIZE_U64 | 0)
#define KVM_REG_MIPS_COUNT_CTL_DC 0x00000001
/*
@ -131,15 +138,46 @@ struct kvm_fpu {
* emulated.
* Modifications to times in the future are rejected.
*/
#define KVM_REG_MIPS_COUNT_RESUME (KVM_REG_MIPS | KVM_REG_SIZE_U64 | \
0x20000 | 1)
#define KVM_REG_MIPS_COUNT_RESUME (KVM_REG_MIPS_KVM | KVM_REG_SIZE_U64 | 1)
/*
* CP0_Count rate in Hz
* Specifies the rate of the CP0_Count timer in Hz. Modifications occur without
* discontinuities in CP0_Count.
*/
#define KVM_REG_MIPS_COUNT_HZ (KVM_REG_MIPS | KVM_REG_SIZE_U64 | \
0x20000 | 2)
#define KVM_REG_MIPS_COUNT_HZ (KVM_REG_MIPS_KVM | KVM_REG_SIZE_U64 | 2)
/*
* KVM_REG_MIPS_FPU - Floating Point and MIPS SIMD Architecture (MSA) registers.
*
* bits[15..8] - Register subset (see definitions below).
* bits[7..5] - Must be zero.
* bits[4..0] - Register number within register subset.
*/
#define KVM_REG_MIPS_FPR (KVM_REG_MIPS_FPU | 0x0000000000000000ULL)
#define KVM_REG_MIPS_FCR (KVM_REG_MIPS_FPU | 0x0000000000000100ULL)
#define KVM_REG_MIPS_MSACR (KVM_REG_MIPS_FPU | 0x0000000000000200ULL)
/*
* KVM_REG_MIPS_FPR - Floating point / Vector registers.
*/
#define KVM_REG_MIPS_FPR_32(n) (KVM_REG_MIPS_FPR | KVM_REG_SIZE_U32 | (n))
#define KVM_REG_MIPS_FPR_64(n) (KVM_REG_MIPS_FPR | KVM_REG_SIZE_U64 | (n))
#define KVM_REG_MIPS_VEC_128(n) (KVM_REG_MIPS_FPR | KVM_REG_SIZE_U128 | (n))
/*
* KVM_REG_MIPS_FCR - Floating point control registers.
*/
#define KVM_REG_MIPS_FCR_IR (KVM_REG_MIPS_FCR | KVM_REG_SIZE_U32 | 0)
#define KVM_REG_MIPS_FCR_CSR (KVM_REG_MIPS_FCR | KVM_REG_SIZE_U32 | 31)
/*
* KVM_REG_MIPS_MSACR - MIPS SIMD Architecture (MSA) control registers.
*/
#define KVM_REG_MIPS_MSA_IR (KVM_REG_MIPS_MSACR | KVM_REG_SIZE_U32 | 0)
#define KVM_REG_MIPS_MSA_CSR (KVM_REG_MIPS_MSACR | KVM_REG_SIZE_U32 | 1)
/*
* KVM MIPS specific structures and definitions

105
arch/mips/kernel/asm-offsets.c
View file

@ -167,72 +167,6 @@ void output_thread_fpu_defines(void)
OFFSET(THREAD_FPR30, task_struct, thread.fpu.fpr[30]);
OFFSET(THREAD_FPR31, task_struct, thread.fpu.fpr[31]);
/* the least significant 64 bits of each FP register */
OFFSET(THREAD_FPR0_LS64, task_struct,
thread.fpu.fpr[0].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR1_LS64, task_struct,
thread.fpu.fpr[1].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR2_LS64, task_struct,
thread.fpu.fpr[2].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR3_LS64, task_struct,
thread.fpu.fpr[3].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR4_LS64, task_struct,
thread.fpu.fpr[4].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR5_LS64, task_struct,
thread.fpu.fpr[5].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR6_LS64, task_struct,
thread.fpu.fpr[6].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR7_LS64, task_struct,
thread.fpu.fpr[7].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR8_LS64, task_struct,
thread.fpu.fpr[8].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR9_LS64, task_struct,
thread.fpu.fpr[9].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR10_LS64, task_struct,
thread.fpu.fpr[10].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR11_LS64, task_struct,
thread.fpu.fpr[11].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR12_LS64, task_struct,
thread.fpu.fpr[12].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR13_LS64, task_struct,
thread.fpu.fpr[13].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR14_LS64, task_struct,
thread.fpu.fpr[14].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR15_LS64, task_struct,
thread.fpu.fpr[15].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR16_LS64, task_struct,
thread.fpu.fpr[16].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR17_LS64, task_struct,
thread.fpu.fpr[17].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR18_LS64, task_struct,
thread.fpu.fpr[18].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR19_LS64, task_struct,
thread.fpu.fpr[19].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR20_LS64, task_struct,
thread.fpu.fpr[20].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR21_LS64, task_struct,
thread.fpu.fpr[21].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR22_LS64, task_struct,
thread.fpu.fpr[22].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR23_LS64, task_struct,
thread.fpu.fpr[23].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR24_LS64, task_struct,
thread.fpu.fpr[24].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR25_LS64, task_struct,
thread.fpu.fpr[25].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR26_LS64, task_struct,
thread.fpu.fpr[26].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR27_LS64, task_struct,
thread.fpu.fpr[27].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR28_LS64, task_struct,
thread.fpu.fpr[28].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR29_LS64, task_struct,
thread.fpu.fpr[29].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR30_LS64, task_struct,
thread.fpu.fpr[30].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FPR31_LS64, task_struct,
thread.fpu.fpr[31].val64[FPR_IDX(64, 0)]);
OFFSET(THREAD_FCR31, task_struct, thread.fpu.fcr31);
OFFSET(THREAD_MSA_CSR, task_struct, thread.fpu.msacsr);
BLANK();
@ -470,6 +404,45 @@ void output_kvm_defines(void)
OFFSET(VCPU_LO, kvm_vcpu_arch, lo);
OFFSET(VCPU_HI, kvm_vcpu_arch, hi);
OFFSET(VCPU_PC, kvm_vcpu_arch, pc);
BLANK();
OFFSET(VCPU_FPR0, kvm_vcpu_arch, fpu.fpr[0]);
OFFSET(VCPU_FPR1, kvm_vcpu_arch, fpu.fpr[1]);
OFFSET(VCPU_FPR2, kvm_vcpu_arch, fpu.fpr[2]);
OFFSET(VCPU_FPR3, kvm_vcpu_arch, fpu.fpr[3]);
OFFSET(VCPU_FPR4, kvm_vcpu_arch, fpu.fpr[4]);
OFFSET(VCPU_FPR5, kvm_vcpu_arch, fpu.fpr[5]);
OFFSET(VCPU_FPR6, kvm_vcpu_arch, fpu.fpr[6]);
OFFSET(VCPU_FPR7, kvm_vcpu_arch, fpu.fpr[7]);
OFFSET(VCPU_FPR8, kvm_vcpu_arch, fpu.fpr[8]);
OFFSET(VCPU_FPR9, kvm_vcpu_arch, fpu.fpr[9]);
OFFSET(VCPU_FPR10, kvm_vcpu_arch, fpu.fpr[10]);
OFFSET(VCPU_FPR11, kvm_vcpu_arch, fpu.fpr[11]);
OFFSET(VCPU_FPR12, kvm_vcpu_arch, fpu.fpr[12]);
OFFSET(VCPU_FPR13, kvm_vcpu_arch, fpu.fpr[13]);
OFFSET(VCPU_FPR14, kvm_vcpu_arch, fpu.fpr[14]);
OFFSET(VCPU_FPR15, kvm_vcpu_arch, fpu.fpr[15]);
OFFSET(VCPU_FPR16, kvm_vcpu_arch, fpu.fpr[16]);
OFFSET(VCPU_FPR17, kvm_vcpu_arch, fpu.fpr[17]);
OFFSET(VCPU_FPR18, kvm_vcpu_arch, fpu.fpr[18]);
OFFSET(VCPU_FPR19, kvm_vcpu_arch, fpu.fpr[19]);
OFFSET(VCPU_FPR20, kvm_vcpu_arch, fpu.fpr[20]);
OFFSET(VCPU_FPR21, kvm_vcpu_arch, fpu.fpr[21]);
OFFSET(VCPU_FPR22, kvm_vcpu_arch, fpu.fpr[22]);
OFFSET(VCPU_FPR23, kvm_vcpu_arch, fpu.fpr[23]);
OFFSET(VCPU_FPR24, kvm_vcpu_arch, fpu.fpr[24]);
OFFSET(VCPU_FPR25, kvm_vcpu_arch, fpu.fpr[25]);
OFFSET(VCPU_FPR26, kvm_vcpu_arch, fpu.fpr[26]);
OFFSET(VCPU_FPR27, kvm_vcpu_arch, fpu.fpr[27]);
OFFSET(VCPU_FPR28, kvm_vcpu_arch, fpu.fpr[28]);
OFFSET(VCPU_FPR29, kvm_vcpu_arch, fpu.fpr[29]);
OFFSET(VCPU_FPR30, kvm_vcpu_arch, fpu.fpr[30]);
OFFSET(VCPU_FPR31, kvm_vcpu_arch, fpu.fpr[31]);
OFFSET(VCPU_FCR31, kvm_vcpu_arch, fpu.fcr31);
OFFSET(VCPU_MSA_CSR, kvm_vcpu_arch, fpu.msacsr);
BLANK();
OFFSET(VCPU_COP0, kvm_vcpu_arch, cop0);
OFFSET(VCPU_GUEST_KERNEL_ASID, kvm_vcpu_arch, guest_kernel_asid);
OFFSET(VCPU_GUEST_USER_ASID, kvm_vcpu_arch, guest_user_asid);

15
arch/mips/kernel/genex.S
View file

@ -360,12 +360,15 @@ NESTED(nmi_handler, PT_SIZE, sp)
.set mips1
SET_HARDFLOAT
cfc1 a1, fcr31
li a2, ~(0x3f << 12)
and a2, a1
ctc1 a2, fcr31
.set pop
TRACE_IRQS_ON
STI
CLI
TRACE_IRQS_OFF
.endm
.macro __build_clear_msa_fpe
_cfcmsa a1, MSA_CSR
CLI
TRACE_IRQS_OFF
.endm
.macro __build_clear_ade
@ -426,7 +429,7 @@ NESTED(nmi_handler, PT_SIZE, sp)
BUILD_HANDLER cpu cpu sti silent /* #11 */
BUILD_HANDLER ov ov sti silent /* #12 */
BUILD_HANDLER tr tr sti silent /* #13 */
BUILD_HANDLER msa_fpe msa_fpe sti silent /* #14 */
BUILD_HANDLER msa_fpe msa_fpe msa_fpe silent /* #14 */
BUILD_HANDLER fpe fpe fpe silent /* #15 */
BUILD_HANDLER ftlb ftlb none silent /* #16 */
BUILD_HANDLER msa msa sti silent /* #21 */

30
arch/mips/kernel/ptrace.c
View file

@ -46,6 +46,26 @@
#define CREATE_TRACE_POINTS
#include <trace/events/syscalls.h>
static void init_fp_ctx(struct task_struct *target)
{
/* If FP has been used then the target already has context */
if (tsk_used_math(target))
return;
/* Begin with data registers set to all 1s... */
memset(&target->thread.fpu.fpr, ~0, sizeof(target->thread.fpu.fpr));
/* ...and FCSR zeroed */
target->thread.fpu.fcr31 = 0;
/*
* Record that the target has "used" math, such that the context
* just initialised, and any modifications made by the caller,
* aren't discarded.
*/
set_stopped_child_used_math(target);
}
/*
* Called by kernel/ptrace.c when detaching..
*
@ -142,6 +162,7 @@ int ptrace_setfpregs(struct task_struct *child, __u32 __user *data)
if (!access_ok(VERIFY_READ, data, 33 * 8))
return -EIO;
init_fp_ctx(child);
fregs = get_fpu_regs(child);
for (i = 0; i < 32; i++) {
@ -439,6 +460,8 @@ static int fpr_set(struct task_struct *target,
/* XXX fcr31 */
init_fp_ctx(target);
if (sizeof(target->thread.fpu.fpr[i]) == sizeof(elf_fpreg_t))
return user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&target->thread.fpu,
@ -660,12 +683,7 @@ long arch_ptrace(struct task_struct *child, long request,
case FPR_BASE ... FPR_BASE + 31: {
union fpureg *fregs = get_fpu_regs(child);
if (!tsk_used_math(child)) {
/* FP not yet used */
memset(&child->thread.fpu, ~0,
sizeof(child->thread.fpu));
child->thread.fpu.fcr31 = 0;
}
init_fp_ctx(child);
#ifdef CONFIG_32BIT
if (test_thread_flag(TIF_32BIT_FPREGS)) {
/*

2
arch/mips/kernel/r4k_fpu.S
View file

@ -34,7 +34,6 @@
.endm
.set noreorder
.set MIPS_ISA_ARCH_LEVEL_RAW
LEAF(_save_fp_context)
.set push
@ -103,6 +102,7 @@ LEAF(_save_fp_context)
/* Save 32-bit process floating point context */
LEAF(_save_fp_context32)
.set push
.set MIPS_ISA_ARCH_LEVEL_RAW
SET_HARDFLOAT
cfc1 t1, fcr31

33
arch/mips/kernel/traps.c
View file

@ -701,6 +701,13 @@ asmlinkage void do_ov(struct pt_regs *regs)
int process_fpemu_return(int sig, void __user *fault_addr)
{
/*
* We can't allow the emulated instruction to leave any of the cause
* bits set in FCSR. If they were then the kernel would take an FP
* exception when restoring FP context.
*/
current->thread.fpu.fcr31 &= ~FPU_CSR_ALL_X;
if (sig == SIGSEGV || sig == SIGBUS) {
struct siginfo si = {0};
si.si_addr = fault_addr;
@ -781,6 +788,11 @@ asmlinkage void do_fpe(struct pt_regs *regs, unsigned long fcr31)
if (notify_die(DIE_FP, "FP exception", regs, 0, regs_to_trapnr(regs),
SIGFPE) == NOTIFY_STOP)
goto out;
/* Clear FCSR.Cause before enabling interrupts */
write_32bit_cp1_register(CP1_STATUS, fcr31 & ~FPU_CSR_ALL_X);
local_irq_enable();
die_if_kernel("FP exception in kernel code", regs);
if (fcr31 & FPU_CSR_UNI_X) {
@ -804,18 +816,12 @@ asmlinkage void do_fpe(struct pt_regs *regs, unsigned long fcr31)
sig = fpu_emulator_cop1Handler(regs, &current->thread.fpu, 1,
&fault_addr);
/*
* We can't allow the emulated instruction to leave any of
* the cause bit set in $fcr31.
*/
current->thread.fpu.fcr31 &= ~FPU_CSR_ALL_X;
/* If something went wrong, signal */
process_fpemu_return(sig, fault_addr);
/* Restore the hardware register state */
own_fpu(1); /* Using the FPU again. */
/* If something went wrong, signal */
process_fpemu_return(sig, fault_addr);
goto out;
} else if (fcr31 & FPU_CSR_INV_X)
info.si_code = FPE_FLTINV;
@ -1392,13 +1398,22 @@ asmlinkage void do_cpu(struct pt_regs *regs)
exception_exit(prev_state);
}
asmlinkage void do_msa_fpe(struct pt_regs *regs)
asmlinkage void do_msa_fpe(struct pt_regs *regs, unsigned int msacsr)
{
enum ctx_state prev_state;
prev_state = exception_enter();
if (notify_die(DIE_MSAFP, "MSA FP exception", regs, 0,
regs_to_trapnr(regs), SIGFPE) == NOTIFY_STOP)
goto out;
/* Clear MSACSR.Cause before enabling interrupts */
write_msa_csr(msacsr & ~MSA_CSR_CAUSEF);
local_irq_enable();
die_if_kernel("do_msa_fpe invoked from kernel context!", regs);
force_sig(SIGFPE, current);
out:
exception_exit(prev_state);
}

8
arch/mips/kvm/Makefile
View file

@ -1,13 +1,15 @@
# Makefile for KVM support for MIPS
#
common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
kvm-objs := $(common-objs) mips.o emulate.o locore.o \
common-objs-$(CONFIG_CPU_HAS_MSA) += msa.o
kvm-objs := $(common-objs-y) mips.o emulate.o locore.o \
interrupt.o stats.o commpage.o \
dyntrans.o trap_emul.o
dyntrans.o trap_emul.o fpu.o
obj-$(CONFIG_KVM) += kvm.o
obj-y += callback.o tlb.o

332
arch/mips/kvm/emulate.c
View file

@ -884,6 +884,84 @@ enum emulation_result kvm_mips_emul_tlbp(struct kvm_vcpu *vcpu)
return EMULATE_DONE;
}
/**
* kvm_mips_config1_wrmask() - Find mask of writable bits in guest Config1
* @vcpu: Virtual CPU.
*
* Finds the mask of bits which are writable in the guest's Config1 CP0
* register, by userland (currently read-only to the guest).
*/
unsigned int kvm_mips_config1_wrmask(struct kvm_vcpu *vcpu)
{
unsigned int mask = 0;
/* Permit FPU to be present if FPU is supported */
if (kvm_mips_guest_can_have_fpu(&vcpu->arch))
mask |= MIPS_CONF1_FP;
return mask;
}
/**
* kvm_mips_config3_wrmask() - Find mask of writable bits in guest Config3
* @vcpu: Virtual CPU.
*
* Finds the mask of bits which are writable in the guest's Config3 CP0
* register, by userland (currently read-only to the guest).
*/
unsigned int kvm_mips_config3_wrmask(struct kvm_vcpu *vcpu)
{
/* Config4 is optional */
unsigned int mask = MIPS_CONF_M;
/* Permit MSA to be present if MSA is supported */
if (kvm_mips_guest_can_have_msa(&vcpu->arch))
mask |= MIPS_CONF3_MSA;
return mask;
}
/**
* kvm_mips_config4_wrmask() - Find mask of writable bits in guest Config4
* @vcpu: Virtual CPU.
*
* Finds the mask of bits which are writable in the guest's Config4 CP0
* register, by userland (currently read-only to the guest).
*/
unsigned int kvm_mips_config4_wrmask(struct kvm_vcpu *vcpu)
{
/* Config5 is optional */
return MIPS_CONF_M;
}
/**
* kvm_mips_config5_wrmask() - Find mask of writable bits in guest Config5
* @vcpu: Virtual CPU.
*
* Finds the mask of bits which are writable in the guest's Config5 CP0
* register, by the guest itself.
*/
unsigned int kvm_mips_config5_wrmask(struct kvm_vcpu *vcpu)
{
unsigned int mask = 0;
/* Permit MSAEn changes if MSA supported and enabled */
if (kvm_mips_guest_has_msa(&vcpu->arch))
mask |= MIPS_CONF5_MSAEN;
/*
* Permit guest FPU mode changes if FPU is enabled and the relevant
* feature exists according to FIR register.
*/
if (kvm_mips_guest_has_fpu(&vcpu->arch)) {
if (cpu_has_fre)
mask |= MIPS_CONF5_FRE;
/* We don't support UFR or UFE */
}
return mask;
}
enum emulation_result kvm_mips_emulate_CP0(uint32_t inst, uint32_t *opc,
uint32_t cause, struct kvm_run *run,
struct kvm_vcpu *vcpu)
@ -1021,18 +1099,114 @@ enum emulation_result kvm_mips_emulate_CP0(uint32_t inst, uint32_t *opc,
kvm_mips_write_compare(vcpu,
vcpu->arch.gprs[rt]);
} else if ((rd == MIPS_CP0_STATUS) && (sel == 0)) {
kvm_write_c0_guest_status(cop0,
vcpu->arch.gprs[rt]);
unsigned int old_val, val, change;
old_val = kvm_read_c0_guest_status(cop0);
val = vcpu->arch.gprs[rt];
change = val ^ old_val;
/* Make sure that the NMI bit is never set */
val &= ~ST0_NMI;
/*
* Make sure that CU1 and NMI bits are
* never set
* Don't allow CU1 or FR to be set unless FPU
* capability enabled and exists in guest
* configuration.
*/
kvm_clear_c0_guest_status(cop0,
(ST0_CU1 | ST0_NMI));
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
val &= ~(ST0_CU1 | ST0_FR);
/*
* Also don't allow FR to be set if host doesn't
* support it.
*/
if (!(current_cpu_data.fpu_id & MIPS_FPIR_F64))
val &= ~ST0_FR;
/* Handle changes in FPU mode */
preempt_disable();
/*
* FPU and Vector register state is made
* UNPREDICTABLE by a change of FR, so don't
* even bother saving it.
*/
if (change & ST0_FR)
kvm_drop_fpu(vcpu);
/*
* If MSA state is already live, it is undefined
* how it interacts with FR=0 FPU state, and we
* don't want to hit reserved instruction
* exceptions trying to save the MSA state later
* when CU=1 && FR=1, so play it safe and save
* it first.
*/
if (change & ST0_CU1 && !(val & ST0_FR) &&
vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA)
kvm_lose_fpu(vcpu);
/*
* Propagate CU1 (FPU enable) changes
* immediately if the FPU context is already
* loaded. When disabling we leave the context
* loaded so it can be quickly enabled again in
* the near future.
*/
if (change & ST0_CU1 &&
vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)
change_c0_status(ST0_CU1, val);
preempt_enable();
kvm_write_c0_guest_status(cop0, val);
#ifdef CONFIG_KVM_MIPS_DYN_TRANS
kvm_mips_trans_mtc0(inst, opc, vcpu);
/*
* If FPU present, we need CU1/FR bits to take
* effect fairly soon.
*/
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
kvm_mips_trans_mtc0(inst, opc, vcpu);
#endif
} else if ((rd == MIPS_CP0_CONFIG) && (sel == 5)) {
unsigned int old_val, val, change, wrmask;
old_val = kvm_read_c0_guest_config5(cop0);
val = vcpu->arch.gprs[rt];
/* Only a few bits are writable in Config5 */
wrmask = kvm_mips_config5_wrmask(vcpu);
change = (val ^ old_val) & wrmask;
val = old_val ^ change;
/* Handle changes in FPU/MSA modes */
preempt_disable();
/*
* Propagate FRE changes immediately if the FPU
* context is already loaded.
*/
if (change & MIPS_CONF5_FRE &&
vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)
change_c0_config5(MIPS_CONF5_FRE, val);
/*
* Propagate MSAEn changes immediately if the
* MSA context is already loaded. When disabling
* we leave the context loaded so it can be
* quickly enabled again in the near future.
*/
if (change & MIPS_CONF5_MSAEN &&
vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA)
change_c0_config5(MIPS_CONF5_MSAEN,
val);
preempt_enable();
kvm_write_c0_guest_config5(cop0, val);
} else if ((rd == MIPS_CP0_CAUSE) && (sel == 0)) {
uint32_t old_cause, new_cause;
@ -1970,6 +2144,146 @@ enum emulation_result kvm_mips_emulate_bp_exc(unsigned long cause,
return er;
}
enum emulation_result kvm_mips_emulate_trap_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_vcpu_arch *arch = &vcpu->arch;
enum emulation_result er = EMULATE_DONE;
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
kvm_write_c0_guest_epc(cop0, arch->pc);
kvm_set_c0_guest_status(cop0, ST0_EXL);
if (cause & CAUSEF_BD)
kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
else
kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
kvm_debug("Delivering TRAP @ pc %#lx\n", arch->pc);
kvm_change_c0_guest_cause(cop0, (0xff),
(T_TRAP << CAUSEB_EXCCODE));
/* Set PC to the exception entry point */
arch->pc = KVM_GUEST_KSEG0 + 0x180;
} else {
kvm_err("Trying to deliver TRAP when EXL is already set\n");
er = EMULATE_FAIL;
}
return er;
}
enum emulation_result kvm_mips_emulate_msafpe_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_vcpu_arch *arch = &vcpu->arch;
enum emulation_result er = EMULATE_DONE;
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
kvm_write_c0_guest_epc(cop0, arch->pc);
kvm_set_c0_guest_status(cop0, ST0_EXL);
if (cause & CAUSEF_BD)
kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
else
kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
kvm_debug("Delivering MSAFPE @ pc %#lx\n", arch->pc);
kvm_change_c0_guest_cause(cop0, (0xff),
(T_MSAFPE << CAUSEB_EXCCODE));
/* Set PC to the exception entry point */
arch->pc = KVM_GUEST_KSEG0 + 0x180;
} else {
kvm_err("Trying to deliver MSAFPE when EXL is already set\n");
er = EMULATE_FAIL;
}
return er;
}
enum emulation_result kvm_mips_emulate_fpe_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_vcpu_arch *arch = &vcpu->arch;
enum emulation_result er = EMULATE_DONE;
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
kvm_write_c0_guest_epc(cop0, arch->pc);
kvm_set_c0_guest_status(cop0, ST0_EXL);
if (cause & CAUSEF_BD)
kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
else
kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
kvm_debug("Delivering FPE @ pc %#lx\n", arch->pc);
kvm_change_c0_guest_cause(cop0, (0xff),
(T_FPE << CAUSEB_EXCCODE));
/* Set PC to the exception entry point */
arch->pc = KVM_GUEST_KSEG0 + 0x180;
} else {
kvm_err("Trying to deliver FPE when EXL is already set\n");
er = EMULATE_FAIL;
}
return er;
}
enum emulation_result kvm_mips_emulate_msadis_exc(unsigned long cause,
uint32_t *opc,
struct kvm_run *run,
struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_vcpu_arch *arch = &vcpu->arch;
enum emulation_result er = EMULATE_DONE;
if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
/* save old pc */
kvm_write_c0_guest_epc(cop0, arch->pc);
kvm_set_c0_guest_status(cop0, ST0_EXL);
if (cause & CAUSEF_BD)
kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
else
kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
kvm_debug("Delivering MSADIS @ pc %#lx\n", arch->pc);
kvm_change_c0_guest_cause(cop0, (0xff),
(T_MSADIS << CAUSEB_EXCCODE));
/* Set PC to the exception entry point */
arch->pc = KVM_GUEST_KSEG0 + 0x180;
} else {
kvm_err("Trying to deliver MSADIS when EXL is already set\n");
er = EMULATE_FAIL;
}
return er;
}
/* ll/sc, rdhwr, sync emulation */
#define OPCODE 0xfc000000
@ -2176,6 +2490,10 @@ enum emulation_result kvm_mips_check_privilege(unsigned long cause,
case T_SYSCALL:
case T_BREAK:
case T_RES_INST:
case T_TRAP:
case T_MSAFPE:
case T_FPE:
case T_MSADIS:
break;
case T_COP_UNUSABLE:

122
arch/mips/kvm/fpu.S Normal file
View file

@ -0,0 +1,122 @@
/*
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
* FPU context handling code for KVM.
*
* Copyright (C) 2015 Imagination Technologies Ltd.
*/
#include <asm/asm.h>
#include <asm/asm-offsets.h>
#include <asm/fpregdef.h>
#include <asm/mipsregs.h>
#include <asm/regdef.h>
.set noreorder
.set noat
LEAF(__kvm_save_fpu)
.set push
.set mips64r2
SET_HARDFLOAT
mfc0 t0, CP0_STATUS
sll t0, t0, 5 # is Status.FR set?
bgez t0, 1f # no: skip odd doubles
nop
sdc1 $f1, VCPU_FPR1(a0)
sdc1 $f3, VCPU_FPR3(a0)
sdc1 $f5, VCPU_FPR5(a0)
sdc1 $f7, VCPU_FPR7(a0)
sdc1 $f9, VCPU_FPR9(a0)
sdc1 $f11, VCPU_FPR11(a0)
sdc1 $f13, VCPU_FPR13(a0)
sdc1 $f15, VCPU_FPR15(a0)
sdc1 $f17, VCPU_FPR17(a0)
sdc1 $f19, VCPU_FPR19(a0)
sdc1 $f21, VCPU_FPR21(a0)
sdc1 $f23, VCPU_FPR23(a0)
sdc1 $f25, VCPU_FPR25(a0)
sdc1 $f27, VCPU_FPR27(a0)
sdc1 $f29, VCPU_FPR29(a0)
sdc1 $f31, VCPU_FPR31(a0)
1: sdc1 $f0, VCPU_FPR0(a0)
sdc1 $f2, VCPU_FPR2(a0)
sdc1 $f4, VCPU_FPR4(a0)
sdc1 $f6, VCPU_FPR6(a0)
sdc1 $f8, VCPU_FPR8(a0)
sdc1 $f10, VCPU_FPR10(a0)
sdc1 $f12, VCPU_FPR12(a0)
sdc1 $f14, VCPU_FPR14(a0)
sdc1 $f16, VCPU_FPR16(a0)
sdc1 $f18, VCPU_FPR18(a0)
sdc1 $f20, VCPU_FPR20(a0)
sdc1 $f22, VCPU_FPR22(a0)
sdc1 $f24, VCPU_FPR24(a0)
sdc1 $f26, VCPU_FPR26(a0)
sdc1 $f28, VCPU_FPR28(a0)
jr ra
sdc1 $f30, VCPU_FPR30(a0)
.set pop
END(__kvm_save_fpu)
LEAF(__kvm_restore_fpu)
.set push
.set mips64r2
SET_HARDFLOAT
mfc0 t0, CP0_STATUS
sll t0, t0, 5 # is Status.FR set?
bgez t0, 1f # no: skip odd doubles
nop
ldc1 $f1, VCPU_FPR1(a0)
ldc1 $f3, VCPU_FPR3(a0)
ldc1 $f5, VCPU_FPR5(a0)
ldc1 $f7, VCPU_FPR7(a0)
ldc1 $f9, VCPU_FPR9(a0)
ldc1 $f11, VCPU_FPR11(a0)
ldc1 $f13, VCPU_FPR13(a0)
ldc1 $f15, VCPU_FPR15(a0)
ldc1 $f17, VCPU_FPR17(a0)
ldc1 $f19, VCPU_FPR19(a0)
ldc1 $f21, VCPU_FPR21(a0)
ldc1 $f23, VCPU_FPR23(a0)
ldc1 $f25, VCPU_FPR25(a0)
ldc1 $f27, VCPU_FPR27(a0)
ldc1 $f29, VCPU_FPR29(a0)
ldc1 $f31, VCPU_FPR31(a0)
1: ldc1 $f0, VCPU_FPR0(a0)
ldc1 $f2, VCPU_FPR2(a0)
ldc1 $f4, VCPU_FPR4(a0)
ldc1 $f6, VCPU_FPR6(a0)
ldc1 $f8, VCPU_FPR8(a0)
ldc1 $f10, VCPU_FPR10(a0)
ldc1 $f12, VCPU_FPR12(a0)
ldc1 $f14, VCPU_FPR14(a0)
ldc1 $f16, VCPU_FPR16(a0)
ldc1 $f18, VCPU_FPR18(a0)
ldc1 $f20, VCPU_FPR20(a0)
ldc1 $f22, VCPU_FPR22(a0)
ldc1 $f24, VCPU_FPR24(a0)
ldc1 $f26, VCPU_FPR26(a0)
ldc1 $f28, VCPU_FPR28(a0)
jr ra
ldc1 $f30, VCPU_FPR30(a0)
.set pop
END(__kvm_restore_fpu)
LEAF(__kvm_restore_fcsr)
.set push
SET_HARDFLOAT
lw t0, VCPU_FCR31(a0)
/*
* The ctc1 must stay at this offset in __kvm_restore_fcsr.
* See kvm_mips_csr_die_notify() which handles t0 containing a value
* which triggers an FP Exception, which must be stepped over and
* ignored since the set cause bits must remain there for the guest.
*/
ctc1 t0, fcr31
jr ra
nop
.set pop
END(__kvm_restore_fcsr)

38
arch/mips/kvm/locore.S
View file

@ -36,6 +36,8 @@
#define PT_HOST_USERLOCAL PT_EPC
#define CP0_DDATA_LO $28,3
#define CP0_CONFIG3 $16,3
#define CP0_CONFIG5 $16,5
#define CP0_EBASE $15,1
#define CP0_INTCTL $12,1
@ -353,6 +355,42 @@ NESTED (MIPSX(GuestException), CALLFRAME_SIZ, ra)
LONG_L k0, VCPU_HOST_EBASE(k1)
mtc0 k0,CP0_EBASE
/*
* If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
* trigger FPE for pending exceptions.
*/
.set at
and v1, v0, ST0_CU1
beqz v1, 1f
nop
.set push
SET_HARDFLOAT
cfc1 t0, fcr31
sw t0, VCPU_FCR31(k1)
ctc1 zero,fcr31
.set pop
.set noat
1:
#ifdef CONFIG_CPU_HAS_MSA
/*
* If MSA is enabled, save MSACSR and clear it so that later
* instructions don't trigger MSAFPE for pending exceptions.
*/
mfc0 t0, CP0_CONFIG3
ext t0, t0, 28, 1 /* MIPS_CONF3_MSAP */
beqz t0, 1f
nop
mfc0 t0, CP0_CONFIG5
ext t0, t0, 27, 1 /* MIPS_CONF5_MSAEN */
beqz t0, 1f
nop
_cfcmsa t0, MSA_CSR
sw t0, VCPU_MSA_CSR(k1)
_ctcmsa MSA_CSR, zero
1:
#endif
/* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
.set at
and v0, v0, ~(ST0_EXL | KSU_USER | ST0_IE)

472
arch/mips/kvm/mips.c
View file

@ -11,6 +11,7 @@
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/kdebug.h>
#include <linux/module.h>
#include <linux/vmalloc.h>
#include <linux/fs.h>
@ -48,6 +49,10 @@ struct kvm_stats_debugfs_item debugfs_entries[] = {
{ "syscall", VCPU_STAT(syscall_exits), KVM_STAT_VCPU },
{ "resvd_inst", VCPU_STAT(resvd_inst_exits), KVM_STAT_VCPU },
{ "break_inst", VCPU_STAT(break_inst_exits), KVM_STAT_VCPU },
{ "trap_inst", VCPU_STAT(trap_inst_exits), KVM_STAT_VCPU },
{ "msa_fpe", VCPU_STAT(msa_fpe_exits), KVM_STAT_VCPU },
{ "fpe", VCPU_STAT(fpe_exits), KVM_STAT_VCPU },
{ "msa_disabled", VCPU_STAT(msa_disabled_exits), KVM_STAT_VCPU },
{ "flush_dcache", VCPU_STAT(flush_dcache_exits), KVM_STAT_VCPU },
{ "halt_successful_poll", VCPU_STAT(halt_successful_poll), KVM_STAT_VCPU },
{ "halt_wakeup", VCPU_STAT(halt_wakeup), KVM_STAT_VCPU },
@ -504,10 +509,13 @@ static u64 kvm_mips_get_one_regs[] = {
KVM_REG_MIPS_CP0_STATUS,
KVM_REG_MIPS_CP0_CAUSE,
KVM_REG_MIPS_CP0_EPC,
KVM_REG_MIPS_CP0_PRID,
KVM_REG_MIPS_CP0_CONFIG,
KVM_REG_MIPS_CP0_CONFIG1,
KVM_REG_MIPS_CP0_CONFIG2,
KVM_REG_MIPS_CP0_CONFIG3,
KVM_REG_MIPS_CP0_CONFIG4,
KVM_REG_MIPS_CP0_CONFIG5,
KVM_REG_MIPS_CP0_CONFIG7,
KVM_REG_MIPS_CP0_ERROREPC,
@ -520,10 +528,14 @@ static int kvm_mips_get_reg(struct kvm_vcpu *vcpu,
const struct kvm_one_reg *reg)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct mips_fpu_struct *fpu = &vcpu->arch.fpu;
int ret;
s64 v;
s64 vs[2];
unsigned int idx;
switch (reg->id) {
/* General purpose registers */
case KVM_REG_MIPS_R0 ... KVM_REG_MIPS_R31:
v = (long)vcpu->arch.gprs[reg->id - KVM_REG_MIPS_R0];
break;
@ -537,6 +549,67 @@ static int kvm_mips_get_reg(struct kvm_vcpu *vcpu,
v = (long)vcpu->arch.pc;
break;
/* Floating point registers */
case KVM_REG_MIPS_FPR_32(0) ... KVM_REG_MIPS_FPR_32(31):
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
idx = reg->id - KVM_REG_MIPS_FPR_32(0);
/* Odd singles in top of even double when FR=0 */
if (kvm_read_c0_guest_status(cop0) & ST0_FR)
v = get_fpr32(&fpu->fpr[idx], 0);
else
v = get_fpr32(&fpu->fpr[idx & ~1], idx & 1);
break;
case KVM_REG_MIPS_FPR_64(0) ... KVM_REG_MIPS_FPR_64(31):
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
idx = reg->id - KVM_REG_MIPS_FPR_64(0);
/* Can't access odd doubles in FR=0 mode */
if (idx & 1 && !(kvm_read_c0_guest_status(cop0) & ST0_FR))
return -EINVAL;
v = get_fpr64(&fpu->fpr[idx], 0);
break;
case KVM_REG_MIPS_FCR_IR:
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
v = boot_cpu_data.fpu_id;
break;
case KVM_REG_MIPS_FCR_CSR:
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
v = fpu->fcr31;
break;
/* MIPS SIMD Architecture (MSA) registers */
case KVM_REG_MIPS_VEC_128(0) ... KVM_REG_MIPS_VEC_128(31):
if (!kvm_mips_guest_has_msa(&vcpu->arch))
return -EINVAL;
/* Can't access MSA registers in FR=0 mode */
if (!(kvm_read_c0_guest_status(cop0) & ST0_FR))
return -EINVAL;
idx = reg->id - KVM_REG_MIPS_VEC_128(0);
#ifdef CONFIG_CPU_LITTLE_ENDIAN
/* least significant byte first */
vs[0] = get_fpr64(&fpu->fpr[idx], 0);
vs[1] = get_fpr64(&fpu->fpr[idx], 1);
#else
/* most significant byte first */
vs[0] = get_fpr64(&fpu->fpr[idx], 1);
vs[1] = get_fpr64(&fpu->fpr[idx], 0);
#endif
break;
case KVM_REG_MIPS_MSA_IR:
if (!kvm_mips_guest_has_msa(&vcpu->arch))
return -EINVAL;
v = boot_cpu_data.msa_id;
break;
case KVM_REG_MIPS_MSA_CSR:
if (!kvm_mips_guest_has_msa(&vcpu->arch))
return -EINVAL;
v = fpu->msacsr;
break;
/* Co-processor 0 registers */
case KVM_REG_MIPS_CP0_INDEX:
v = (long)kvm_read_c0_guest_index(cop0);
break;
@ -573,8 +646,8 @@ static int kvm_mips_get_reg(struct kvm_vcpu *vcpu,
case KVM_REG_MIPS_CP0_EPC:
v = (long)kvm_read_c0_guest_epc(cop0);
break;
case KVM_REG_MIPS_CP0_ERROREPC:
v = (long)kvm_read_c0_guest_errorepc(cop0);
case KVM_REG_MIPS_CP0_PRID:
v = (long)kvm_read_c0_guest_prid(cop0);
break;
case KVM_REG_MIPS_CP0_CONFIG:
v = (long)kvm_read_c0_guest_config(cop0);
@ -588,9 +661,18 @@ static int kvm_mips_get_reg(struct kvm_vcpu *vcpu,
case KVM_REG_MIPS_CP0_CONFIG3:
v = (long)kvm_read_c0_guest_config3(cop0);
break;
case KVM_REG_MIPS_CP0_CONFIG4:
v = (long)kvm_read_c0_guest_config4(cop0);
break;
case KVM_REG_MIPS_CP0_CONFIG5:
v = (long)kvm_read_c0_guest_config5(cop0);
break;
case KVM_REG_MIPS_CP0_CONFIG7:
v = (long)kvm_read_c0_guest_config7(cop0);
break;
case KVM_REG_MIPS_CP0_ERROREPC:
v = (long)kvm_read_c0_guest_errorepc(cop0);
break;
/* registers to be handled specially */
case KVM_REG_MIPS_CP0_COUNT:
case KVM_REG_MIPS_COUNT_CTL:
@ -612,6 +694,10 @@ static int kvm_mips_get_reg(struct kvm_vcpu *vcpu,
u32 v32 = (u32)v;
return put_user(v32, uaddr32);
} else if ((reg->id & KVM_REG_SIZE_MASK) == KVM_REG_SIZE_U128) {
void __user *uaddr = (void __user *)(long)reg->addr;
return copy_to_user(uaddr, vs, 16);
} else {
return -EINVAL;
}
@ -621,7 +707,10 @@ static int kvm_mips_set_reg(struct kvm_vcpu *vcpu,
const struct kvm_one_reg *reg)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
u64 v;
struct mips_fpu_struct *fpu = &vcpu->arch.fpu;
s64 v;
s64 vs[2];
unsigned int idx;
if ((reg->id & KVM_REG_SIZE_MASK) == KVM_REG_SIZE_U64) {
u64 __user *uaddr64 = (u64 __user *)(long)reg->addr;
@ -635,11 +724,16 @@ static int kvm_mips_set_reg(struct kvm_vcpu *vcpu,
if (get_user(v32, uaddr32) != 0)
return -EFAULT;
v = (s64)v32;
} else if ((reg->id & KVM_REG_SIZE_MASK) == KVM_REG_SIZE_U128) {
void __user *uaddr = (void __user *)(long)reg->addr;
return copy_from_user(vs, uaddr, 16);
} else {
return -EINVAL;
}
switch (reg->id) {
/* General purpose registers */
case KVM_REG_MIPS_R0:
/* Silently ignore requests to set $0 */
break;
@ -656,6 +750,64 @@ static int kvm_mips_set_reg(struct kvm_vcpu *vcpu,
vcpu->arch.pc = v;
break;
/* Floating point registers */
case KVM_REG_MIPS_FPR_32(0) ... KVM_REG_MIPS_FPR_32(31):
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
idx = reg->id - KVM_REG_MIPS_FPR_32(0);
/* Odd singles in top of even double when FR=0 */
if (kvm_read_c0_guest_status(cop0) & ST0_FR)
set_fpr32(&fpu->fpr[idx], 0, v);
else
set_fpr32(&fpu->fpr[idx & ~1], idx & 1, v);
break;
case KVM_REG_MIPS_FPR_64(0) ... KVM_REG_MIPS_FPR_64(31):
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
idx = reg->id - KVM_REG_MIPS_FPR_64(0);
/* Can't access odd doubles in FR=0 mode */
if (idx & 1 && !(kvm_read_c0_guest_status(cop0) & ST0_FR))
return -EINVAL;
set_fpr64(&fpu->fpr[idx], 0, v);
break;
case KVM_REG_MIPS_FCR_IR:
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
/* Read-only */
break;
case KVM_REG_MIPS_FCR_CSR:
if (!kvm_mips_guest_has_fpu(&vcpu->arch))
return -EINVAL;
fpu->fcr31 = v;
break;
/* MIPS SIMD Architecture (MSA) registers */
case KVM_REG_MIPS_VEC_128(0) ... KVM_REG_MIPS_VEC_128(31):
if (!kvm_mips_guest_has_msa(&vcpu->arch))
return -EINVAL;
idx = reg->id - KVM_REG_MIPS_VEC_128(0);
#ifdef CONFIG_CPU_LITTLE_ENDIAN
/* least significant byte first */
set_fpr64(&fpu->fpr[idx], 0, vs[0]);
set_fpr64(&fpu->fpr[idx], 1, vs[1]);
#else
/* most significant byte first */
set_fpr64(&fpu->fpr[idx], 1, vs[0]);
set_fpr64(&fpu->fpr[idx], 0, vs[1]);
#endif
break;
case KVM_REG_MIPS_MSA_IR:
if (!kvm_mips_guest_has_msa(&vcpu->arch))
return -EINVAL;
/* Read-only */
break;
case KVM_REG_MIPS_MSA_CSR:
if (!kvm_mips_guest_has_msa(&vcpu->arch))
return -EINVAL;
fpu->msacsr = v;
break;
/* Co-processor 0 registers */
case KVM_REG_MIPS_CP0_INDEX:
kvm_write_c0_guest_index(cop0, v);
break;
@ -686,6 +838,9 @@ static int kvm_mips_set_reg(struct kvm_vcpu *vcpu,
case KVM_REG_MIPS_CP0_EPC:
kvm_write_c0_guest_epc(cop0, v);
break;
case KVM_REG_MIPS_CP0_PRID:
kvm_write_c0_guest_prid(cop0, v);
break;
case KVM_REG_MIPS_CP0_ERROREPC:
kvm_write_c0_guest_errorepc(cop0, v);
break;
@ -693,6 +848,12 @@ static int kvm_mips_set_reg(struct kvm_vcpu *vcpu,
case KVM_REG_MIPS_CP0_COUNT:
case KVM_REG_MIPS_CP0_COMPARE:
case KVM_REG_MIPS_CP0_CAUSE:
case KVM_REG_MIPS_CP0_CONFIG:
case KVM_REG_MIPS_CP0_CONFIG1:
case KVM_REG_MIPS_CP0_CONFIG2:
case KVM_REG_MIPS_CP0_CONFIG3:
case KVM_REG_MIPS_CP0_CONFIG4:
case KVM_REG_MIPS_CP0_CONFIG5:
case KVM_REG_MIPS_COUNT_CTL:
case KVM_REG_MIPS_COUNT_RESUME:
case KVM_REG_MIPS_COUNT_HZ:
@ -703,6 +864,33 @@ static int kvm_mips_set_reg(struct kvm_vcpu *vcpu,
return 0;
}
static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
struct kvm_enable_cap *cap)
{
int r = 0;
if (!kvm_vm_ioctl_check_extension(vcpu->kvm, cap->cap))
return -EINVAL;
if (cap->flags)
return -EINVAL;
if (cap->args[0])
return -EINVAL;
switch (cap->cap) {
case KVM_CAP_MIPS_FPU:
vcpu->arch.fpu_enabled = true;
break;
case KVM_CAP_MIPS_MSA:
vcpu->arch.msa_enabled = true;
break;
default:
r = -EINVAL;
break;
}
return r;
}
long kvm_arch_vcpu_ioctl(struct file *filp, unsigned int ioctl,
unsigned long arg)
{
@ -760,6 +948,15 @@ long kvm_arch_vcpu_ioctl(struct file *filp, unsigned int ioctl,
r = kvm_vcpu_ioctl_interrupt(vcpu, &irq);
break;
}
case KVM_ENABLE_CAP: {
struct kvm_enable_cap cap;
r = -EFAULT;
if (copy_from_user(&cap, argp, sizeof(cap)))
goto out;
r = kvm_vcpu_ioctl_enable_cap(vcpu, &cap);
break;
}
default:
r = -ENOIOCTLCMD;
}
@ -868,11 +1065,30 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
switch (ext) {
case KVM_CAP_ONE_REG:
case KVM_CAP_ENABLE_CAP:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
r = KVM_COALESCED_MMIO_PAGE_OFFSET;
break;
case KVM_CAP_MIPS_FPU:
r = !!cpu_has_fpu;
break;
case KVM_CAP_MIPS_MSA:
/*
* We don't support MSA vector partitioning yet:
* 1) It would require explicit support which can't be tested
* yet due to lack of support in current hardware.
* 2) It extends the state that would need to be saved/restored
* by e.g. QEMU for migration.
*
* When vector partitioning hardware becomes available, support
* could be added by requiring a flag when enabling
* KVM_CAP_MIPS_MSA capability to indicate that userland knows
* to save/restore the appropriate extra state.
*/
r = cpu_has_msa && !(boot_cpu_data.msa_id & MSA_IR_WRPF);
break;
default:
r = 0;
break;
@ -1119,6 +1335,30 @@ int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu)
ret = kvm_mips_callbacks->handle_break(vcpu);
break;
case T_TRAP:
++vcpu->stat.trap_inst_exits;
trace_kvm_exit(vcpu, TRAP_INST_EXITS);
ret = kvm_mips_callbacks->handle_trap(vcpu);
break;
case T_MSAFPE:
++vcpu->stat.msa_fpe_exits;
trace_kvm_exit(vcpu, MSA_FPE_EXITS);
ret = kvm_mips_callbacks->handle_msa_fpe(vcpu);
break;
case T_FPE:
++vcpu->stat.fpe_exits;
trace_kvm_exit(vcpu, FPE_EXITS);
ret = kvm_mips_callbacks->handle_fpe(vcpu);
break;
case T_MSADIS:
++vcpu->stat.msa_disabled_exits;
trace_kvm_exit(vcpu, MSA_DISABLED_EXITS);
ret = kvm_mips_callbacks->handle_msa_disabled(vcpu);
break;
default:
kvm_err("Exception Code: %d, not yet handled, @ PC: %p, inst: 0x%08x BadVaddr: %#lx Status: %#lx\n",
exccode, opc, kvm_get_inst(opc, vcpu), badvaddr,
@ -1146,12 +1386,233 @@ int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu)
}
}
if (ret == RESUME_GUEST) {
/*
* If FPU / MSA are enabled (i.e. the guest's FPU / MSA context
* is live), restore FCR31 / MSACSR.
*
* This should be before returning to the guest exception
* vector, as it may well cause an [MSA] FP exception if there
* are pending exception bits unmasked. (see
* kvm_mips_csr_die_notifier() for how that is handled).
*/
if (kvm_mips_guest_has_fpu(&vcpu->arch) &&
read_c0_status() & ST0_CU1)
__kvm_restore_fcsr(&vcpu->arch);
if (kvm_mips_guest_has_msa(&vcpu->arch) &&
read_c0_config5() & MIPS_CONF5_MSAEN)
__kvm_restore_msacsr(&vcpu->arch);
}
/* Disable HTW before returning to guest or host */
htw_stop();
return ret;
}
/* Enable FPU for guest and restore context */
void kvm_own_fpu(struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
unsigned int sr, cfg5;
preempt_disable();
sr = kvm_read_c0_guest_status(cop0);
/*
* If MSA state is already live, it is undefined how it interacts with
* FR=0 FPU state, and we don't want to hit reserved instruction
* exceptions trying to save the MSA state later when CU=1 && FR=1, so
* play it safe and save it first.
*
* In theory we shouldn't ever hit this case since kvm_lose_fpu() should
* get called when guest CU1 is set, however we can't trust the guest
* not to clobber the status register directly via the commpage.
*/
if (cpu_has_msa && sr & ST0_CU1 && !(sr & ST0_FR) &&
vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA)
kvm_lose_fpu(vcpu);
/*
* Enable FPU for guest
* We set FR and FRE according to guest context
*/
change_c0_status(ST0_CU1 | ST0_FR, sr);
if (cpu_has_fre) {
cfg5 = kvm_read_c0_guest_config5(cop0);
change_c0_config5(MIPS_CONF5_FRE, cfg5);
}
enable_fpu_hazard();
/* If guest FPU state not active, restore it now */
if (!(vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)) {
__kvm_restore_fpu(&vcpu->arch);
vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_FPU;
}
preempt_enable();
}
#ifdef CONFIG_CPU_HAS_MSA
/* Enable MSA for guest and restore context */
void kvm_own_msa(struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
unsigned int sr, cfg5;
preempt_disable();
/*
* Enable FPU if enabled in guest, since we're restoring FPU context
* anyway. We set FR and FRE according to guest context.
*/
if (kvm_mips_guest_has_fpu(&vcpu->arch)) {
sr = kvm_read_c0_guest_status(cop0);
/*
* If FR=0 FPU state is already live, it is undefined how it
* interacts with MSA state, so play it safe and save it first.
*/
if (!(sr & ST0_FR) &&
(vcpu->arch.fpu_inuse & (KVM_MIPS_FPU_FPU |
KVM_MIPS_FPU_MSA)) == KVM_MIPS_FPU_FPU)
kvm_lose_fpu(vcpu);
change_c0_status(ST0_CU1 | ST0_FR, sr);
if (sr & ST0_CU1 && cpu_has_fre) {
cfg5 = kvm_read_c0_guest_config5(cop0);
change_c0_config5(MIPS_CONF5_FRE, cfg5);
}
}
/* Enable MSA for guest */
set_c0_config5(MIPS_CONF5_MSAEN);
enable_fpu_hazard();
switch (vcpu->arch.fpu_inuse & (KVM_MIPS_FPU_FPU | KVM_MIPS_FPU_MSA)) {
case KVM_MIPS_FPU_FPU:
/*
* Guest FPU state already loaded, only restore upper MSA state
*/
__kvm_restore_msa_upper(&vcpu->arch);
vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_MSA;
break;
case 0:
/* Neither FPU or MSA already active, restore full MSA state */
__kvm_restore_msa(&vcpu->arch);
vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_MSA;
if (kvm_mips_guest_has_fpu(&vcpu->arch))
vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_FPU;
break;
default:
break;
}
preempt_enable();
}
#endif
/* Drop FPU & MSA without saving it */
void kvm_drop_fpu(struct kvm_vcpu *vcpu)
{
preempt_disable();
if (cpu_has_msa && vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA) {
disable_msa();
vcpu->arch.fpu_inuse &= ~KVM_MIPS_FPU_MSA;
}
if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU) {
clear_c0_status(ST0_CU1 | ST0_FR);
vcpu->arch.fpu_inuse &= ~KVM_MIPS_FPU_FPU;
}
preempt_enable();
}
/* Save and disable FPU & MSA */
void kvm_lose_fpu(struct kvm_vcpu *vcpu)
{
/*
* FPU & MSA get disabled in root context (hardware) when it is disabled
* in guest context (software), but the register state in the hardware
* may still be in use. This is why we explicitly re-enable the hardware
* before saving.
*/
preempt_disable();
if (cpu_has_msa && vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA) {
set_c0_config5(MIPS_CONF5_MSAEN);
enable_fpu_hazard();
__kvm_save_msa(&vcpu->arch);
/* Disable MSA & FPU */
disable_msa();
if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)
clear_c0_status(ST0_CU1 | ST0_FR);
vcpu->arch.fpu_inuse &= ~(KVM_MIPS_FPU_FPU | KVM_MIPS_FPU_MSA);
} else if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU) {
set_c0_status(ST0_CU1);
enable_fpu_hazard();
__kvm_save_fpu(&vcpu->arch);
vcpu->arch.fpu_inuse &= ~KVM_MIPS_FPU_FPU;
/* Disable FPU */
clear_c0_status(ST0_CU1 | ST0_FR);
}
preempt_enable();
}
/*
* Step over a specific ctc1 to FCSR and a specific ctcmsa to MSACSR which are
* used to restore guest FCSR/MSACSR state and may trigger a "harmless" FP/MSAFP
* exception if cause bits are set in the value being written.
*/
static int kvm_mips_csr_die_notify(struct notifier_block *self,
unsigned long cmd, void *ptr)
{
struct die_args *args = (struct die_args *)ptr;
struct pt_regs *regs = args->regs;
unsigned long pc;
/* Only interested in FPE and MSAFPE */
if (cmd != DIE_FP && cmd != DIE_MSAFP)
return NOTIFY_DONE;
/* Return immediately if guest context isn't active */
if (!(current->flags & PF_VCPU))
return NOTIFY_DONE;
/* Should never get here from user mode */
BUG_ON(user_mode(regs));
pc = instruction_pointer(regs);
switch (cmd) {
case DIE_FP:
/* match 2nd instruction in __kvm_restore_fcsr */
if (pc != (unsigned long)&__kvm_restore_fcsr + 4)
return NOTIFY_DONE;
break;
case DIE_MSAFP:
/* match 2nd/3rd instruction in __kvm_restore_msacsr */
if (!cpu_has_msa ||
pc < (unsigned long)&__kvm_restore_msacsr + 4 ||
pc > (unsigned long)&__kvm_restore_msacsr + 8)
return NOTIFY_DONE;
break;
}
/* Move PC forward a little and continue executing */
instruction_pointer(regs) += 4;
return NOTIFY_STOP;
}
static struct notifier_block kvm_mips_csr_die_notifier = {
.notifier_call = kvm_mips_csr_die_notify,
};
int __init kvm_mips_init(void)
{
int ret;
@ -1161,6 +1622,8 @@ int __init kvm_mips_init(void)
if (ret)
return ret;
register_die_notifier(&kvm_mips_csr_die_notifier);
/*
* On MIPS, kernel modules are executed from "mapped space", which
* requires TLBs. The TLB handling code is statically linked with
@ -1173,7 +1636,6 @@ int __init kvm_mips_init(void)
kvm_mips_release_pfn_clean = kvm_release_pfn_clean;
kvm_mips_is_error_pfn = is_error_pfn;
pr_info("KVM/MIPS Initialized\n");
return 0;
}
@ -1185,7 +1647,7 @@ void __exit kvm_mips_exit(void)
kvm_mips_release_pfn_clean = NULL;
kvm_mips_is_error_pfn = NULL;
pr_info("KVM/MIPS unloaded\n");
unregister_die_notifier(&kvm_mips_csr_die_notifier);
}
module_init(kvm_mips_init);

161
arch/mips/kvm/msa.S Normal file
View file

@ -0,0 +1,161 @@
/*
* This file is subject to the terms and conditions of the GNU General Public
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
* MIPS SIMD Architecture (MSA) context handling code for KVM.
*
* Copyright (C) 2015 Imagination Technologies Ltd.
*/
#include <asm/asm.h>
#include <asm/asm-offsets.h>
#include <asm/asmmacro.h>
#include <asm/regdef.h>
.set noreorder
.set noat
LEAF(__kvm_save_msa)
st_d 0, VCPU_FPR0, a0
st_d 1, VCPU_FPR1, a0
st_d 2, VCPU_FPR2, a0
st_d 3, VCPU_FPR3, a0
st_d 4, VCPU_FPR4, a0
st_d 5, VCPU_FPR5, a0
st_d 6, VCPU_FPR6, a0
st_d 7, VCPU_FPR7, a0
st_d 8, VCPU_FPR8, a0
st_d 9, VCPU_FPR9, a0
st_d 10, VCPU_FPR10, a0
st_d 11, VCPU_FPR11, a0
st_d 12, VCPU_FPR12, a0
st_d 13, VCPU_FPR13, a0
st_d 14, VCPU_FPR14, a0
st_d 15, VCPU_FPR15, a0
st_d 16, VCPU_FPR16, a0
st_d 17, VCPU_FPR17, a0
st_d 18, VCPU_FPR18, a0
st_d 19, VCPU_FPR19, a0
st_d 20, VCPU_FPR20, a0
st_d 21, VCPU_FPR21, a0
st_d 22, VCPU_FPR22, a0
st_d 23, VCPU_FPR23, a0
st_d 24, VCPU_FPR24, a0
st_d 25, VCPU_FPR25, a0
st_d 26, VCPU_FPR26, a0
st_d 27, VCPU_FPR27, a0
st_d 28, VCPU_FPR28, a0
st_d 29, VCPU_FPR29, a0
st_d 30, VCPU_FPR30, a0
st_d 31, VCPU_FPR31, a0
jr ra
nop
END(__kvm_save_msa)
LEAF(__kvm_restore_msa)
ld_d 0, VCPU_FPR0, a0
ld_d 1, VCPU_FPR1, a0
ld_d 2, VCPU_FPR2, a0
ld_d 3, VCPU_FPR3, a0
ld_d 4, VCPU_FPR4, a0
ld_d 5, VCPU_FPR5, a0
ld_d 6, VCPU_FPR6, a0
ld_d 7, VCPU_FPR7, a0
ld_d 8, VCPU_FPR8, a0
ld_d 9, VCPU_FPR9, a0
ld_d 10, VCPU_FPR10, a0
ld_d 11, VCPU_FPR11, a0
ld_d 12, VCPU_FPR12, a0
ld_d 13, VCPU_FPR13, a0
ld_d 14, VCPU_FPR14, a0
ld_d 15, VCPU_FPR15, a0
ld_d 16, VCPU_FPR16, a0
ld_d 17, VCPU_FPR17, a0
ld_d 18, VCPU_FPR18, a0
ld_d 19, VCPU_FPR19, a0
ld_d 20, VCPU_FPR20, a0
ld_d 21, VCPU_FPR21, a0
ld_d 22, VCPU_FPR22, a0
ld_d 23, VCPU_FPR23, a0
ld_d 24, VCPU_FPR24, a0
ld_d 25, VCPU_FPR25, a0
ld_d 26, VCPU_FPR26, a0
ld_d 27, VCPU_FPR27, a0
ld_d 28, VCPU_FPR28, a0
ld_d 29, VCPU_FPR29, a0
ld_d 30, VCPU_FPR30, a0
ld_d 31, VCPU_FPR31, a0
jr ra
nop
END(__kvm_restore_msa)
.macro kvm_restore_msa_upper wr, off, base
.set push
.set noat
#ifdef CONFIG_64BIT
ld $1, \off(\base)
insert_d \wr, 1
#elif defined(CONFIG_CPU_LITTLE_ENDIAN)
lw $1, \off(\base)
insert_w \wr, 2
lw $1, (\off+4)(\base)
insert_w \wr, 3
#else /* CONFIG_CPU_BIG_ENDIAN */
lw $1, (\off+4)(\base)
insert_w \wr, 2
lw $1, \off(\base)
insert_w \wr, 3
#endif
.set pop
.endm
LEAF(__kvm_restore_msa_upper)
kvm_restore_msa_upper 0, VCPU_FPR0 +8, a0
kvm_restore_msa_upper 1, VCPU_FPR1 +8, a0
kvm_restore_msa_upper 2, VCPU_FPR2 +8, a0
kvm_restore_msa_upper 3, VCPU_FPR3 +8, a0
kvm_restore_msa_upper 4, VCPU_FPR4 +8, a0
kvm_restore_msa_upper 5, VCPU_FPR5 +8, a0
kvm_restore_msa_upper 6, VCPU_FPR6 +8, a0
kvm_restore_msa_upper 7, VCPU_FPR7 +8, a0
kvm_restore_msa_upper 8, VCPU_FPR8 +8, a0
kvm_restore_msa_upper 9, VCPU_FPR9 +8, a0
kvm_restore_msa_upper 10, VCPU_FPR10+8, a0
kvm_restore_msa_upper 11, VCPU_FPR11+8, a0
kvm_restore_msa_upper 12, VCPU_FPR12+8, a0
kvm_restore_msa_upper 13, VCPU_FPR13+8, a0
kvm_restore_msa_upper 14, VCPU_FPR14+8, a0
kvm_restore_msa_upper 15, VCPU_FPR15+8, a0
kvm_restore_msa_upper 16, VCPU_FPR16+8, a0
kvm_restore_msa_upper 17, VCPU_FPR17+8, a0
kvm_restore_msa_upper 18, VCPU_FPR18+8, a0
kvm_restore_msa_upper 19, VCPU_FPR19+8, a0
kvm_restore_msa_upper 20, VCPU_FPR20+8, a0
kvm_restore_msa_upper 21, VCPU_FPR21+8, a0
kvm_restore_msa_upper 22, VCPU_FPR22+8, a0
kvm_restore_msa_upper 23, VCPU_FPR23+8, a0
kvm_restore_msa_upper 24, VCPU_FPR24+8, a0
kvm_restore_msa_upper 25, VCPU_FPR25+8, a0
kvm_restore_msa_upper 26, VCPU_FPR26+8, a0
kvm_restore_msa_upper 27, VCPU_FPR27+8, a0
kvm_restore_msa_upper 28, VCPU_FPR28+8, a0
kvm_restore_msa_upper 29, VCPU_FPR29+8, a0
kvm_restore_msa_upper 30, VCPU_FPR30+8, a0
kvm_restore_msa_upper 31, VCPU_FPR31+8, a0
jr ra
nop
END(__kvm_restore_msa_upper)
LEAF(__kvm_restore_msacsr)
lw t0, VCPU_MSA_CSR(a0)
/*
* The ctcmsa must stay at this offset in __kvm_restore_msacsr.
* See kvm_mips_csr_die_notify() which handles t0 containing a value
* which triggers an MSA FP Exception, which must be stepped over and
* ignored since the set cause bits must remain there for the guest.
*/
_ctcmsa MSA_CSR, t0
jr ra
nop
END(__kvm_restore_msacsr)

4
arch/mips/kvm/stats.c
View file

@ -25,6 +25,10 @@ char *kvm_mips_exit_types_str[MAX_KVM_MIPS_EXIT_TYPES] = {
"System Call",
"Reserved Inst",
"Break Inst",
"Trap Inst",
"MSA FPE",
"FPE",
"MSA Disabled",
"D-Cache Flushes",
};

6
arch/mips/kvm/tlb.c
View file

@ -733,6 +733,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
}
}
/* restore guest state to registers */
kvm_mips_callbacks->vcpu_set_regs(vcpu);
local_irq_restore(flags);
}
@ -751,6 +754,9 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
vcpu->arch.preempt_entryhi = read_c0_entryhi();
vcpu->arch.last_sched_cpu = cpu;
/* save guest state in registers */
kvm_mips_callbacks->vcpu_get_regs(vcpu);
if (((cpu_context(cpu, current->mm) ^ asid_cache(cpu)) &
ASID_VERSION_MASK)) {
kvm_debug("%s: Dropping MMU Context: %#lx\n", __func__,

199
arch/mips/kvm/trap_emul.c
View file

@ -39,16 +39,30 @@ static gpa_t kvm_trap_emul_gva_to_gpa_cb(gva_t gva)
static int kvm_trap_emul_handle_cop_unusable(struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_run *run = vcpu->run;
uint32_t __user *opc = (uint32_t __user *) vcpu->arch.pc;
unsigned long cause = vcpu->arch.host_cp0_cause;
enum emulation_result er = EMULATE_DONE;
int ret = RESUME_GUEST;
if (((cause & CAUSEF_CE) >> CAUSEB_CE) == 1)
er = kvm_mips_emulate_fpu_exc(cause, opc, run, vcpu);
else
if (((cause & CAUSEF_CE) >> CAUSEB_CE) == 1) {
/* FPU Unusable */
if (!kvm_mips_guest_has_fpu(&vcpu->arch) ||
(kvm_read_c0_guest_status(cop0) & ST0_CU1) == 0) {
/*
* Unusable/no FPU in guest:
* deliver guest COP1 Unusable Exception
*/
er = kvm_mips_emulate_fpu_exc(cause, opc, run, vcpu);
} else {
/* Restore FPU state */
kvm_own_fpu(vcpu);
er = EMULATE_DONE;
}
} else {
er = kvm_mips_emulate_inst(cause, opc, run, vcpu);
}
switch (er) {
case EMULATE_DONE:
@ -330,6 +344,107 @@ static int kvm_trap_emul_handle_break(struct kvm_vcpu *vcpu)
return ret;
}
static int kvm_trap_emul_handle_trap(struct kvm_vcpu *vcpu)
{
struct kvm_run *run = vcpu->run;
uint32_t __user *opc = (uint32_t __user *)vcpu->arch.pc;
unsigned long cause = vcpu->arch.host_cp0_cause;
enum emulation_result er = EMULATE_DONE;
int ret = RESUME_GUEST;
er = kvm_mips_emulate_trap_exc(cause, opc, run, vcpu);
if (er == EMULATE_DONE) {
ret = RESUME_GUEST;
} else {
run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
ret = RESUME_HOST;
}
return ret;
}
static int kvm_trap_emul_handle_msa_fpe(struct kvm_vcpu *vcpu)
{
struct kvm_run *run = vcpu->run;
uint32_t __user *opc = (uint32_t __user *)vcpu->arch.pc;
unsigned long cause = vcpu->arch.host_cp0_cause;
enum emulation_result er = EMULATE_DONE;
int ret = RESUME_GUEST;
er = kvm_mips_emulate_msafpe_exc(cause, opc, run, vcpu);
if (er == EMULATE_DONE) {
ret = RESUME_GUEST;
} else {
run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
ret = RESUME_HOST;
}
return ret;
}
static int kvm_trap_emul_handle_fpe(struct kvm_vcpu *vcpu)
{
struct kvm_run *run = vcpu->run;
uint32_t __user *opc = (uint32_t __user *)vcpu->arch.pc;
unsigned long cause = vcpu->arch.host_cp0_cause;
enum emulation_result er = EMULATE_DONE;
int ret = RESUME_GUEST;
er = kvm_mips_emulate_fpe_exc(cause, opc, run, vcpu);
if (er == EMULATE_DONE) {
ret = RESUME_GUEST;
} else {
run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
ret = RESUME_HOST;
}
return ret;
}
/**
* kvm_trap_emul_handle_msa_disabled() - Guest used MSA while disabled in root.
* @vcpu: Virtual CPU context.
*
* Handle when the guest attempts to use MSA when it is disabled.
*/
static int kvm_trap_emul_handle_msa_disabled(struct kvm_vcpu *vcpu)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_run *run = vcpu->run;
uint32_t __user *opc = (uint32_t __user *) vcpu->arch.pc;
unsigned long cause = vcpu->arch.host_cp0_cause;
enum emulation_result er = EMULATE_DONE;
int ret = RESUME_GUEST;
if (!kvm_mips_guest_has_msa(&vcpu->arch) ||
(kvm_read_c0_guest_status(cop0) & (ST0_CU1 | ST0_FR)) == ST0_CU1) {
/*
* No MSA in guest, or FPU enabled and not in FR=1 mode,
* guest reserved instruction exception
*/
er = kvm_mips_emulate_ri_exc(cause, opc, run, vcpu);
} else if (!(kvm_read_c0_guest_config5(cop0) & MIPS_CONF5_MSAEN)) {
/* MSA disabled by guest, guest MSA disabled exception */
er = kvm_mips_emulate_msadis_exc(cause, opc, run, vcpu);
} else {
/* Restore MSA/FPU state */
kvm_own_msa(vcpu);
er = EMULATE_DONE;
}
switch (er) {
case EMULATE_DONE:
ret = RESUME_GUEST;
break;
case EMULATE_FAIL:
run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
ret = RESUME_HOST;
break;
default:
BUG();
}
return ret;
}
static int kvm_trap_emul_vm_init(struct kvm *kvm)
{
return 0;
@ -351,8 +466,9 @@ static int kvm_trap_emul_vcpu_setup(struct kvm_vcpu *vcpu)
* guest will come up as expected, for now we simulate a MIPS 24kc
*/
kvm_write_c0_guest_prid(cop0, 0x00019300);
kvm_write_c0_guest_config(cop0,
MIPS_CONFIG0 | (0x1 << CP0C0_AR) |
/* Have config1, Cacheable, noncoherent, write-back, write allocate */
kvm_write_c0_guest_config(cop0, MIPS_CONF_M | (0x3 << CP0C0_K0) |
(0x1 << CP0C0_AR) |
(MMU_TYPE_R4000 << CP0C0_MT));
/* Read the cache characteristics from the host Config1 Register */
@ -368,10 +484,18 @@ static int kvm_trap_emul_vcpu_setup(struct kvm_vcpu *vcpu)
(1 << CP0C1_WR) | (1 << CP0C1_CA));
kvm_write_c0_guest_config1(cop0, config1);
kvm_write_c0_guest_config2(cop0, MIPS_CONFIG2);
/* MIPS_CONFIG2 | (read_c0_config2() & 0xfff) */
kvm_write_c0_guest_config3(cop0, MIPS_CONFIG3 | (0 << CP0C3_VInt) |
(1 << CP0C3_ULRI));
/* Have config3, no tertiary/secondary caches implemented */
kvm_write_c0_guest_config2(cop0, MIPS_CONF_M);
/* MIPS_CONF_M | (read_c0_config2() & 0xfff) */
/* Have config4, UserLocal */
kvm_write_c0_guest_config3(cop0, MIPS_CONF_M | MIPS_CONF3_ULRI);
/* Have config5 */
kvm_write_c0_guest_config4(cop0, MIPS_CONF_M);
/* No config6 */
kvm_write_c0_guest_config5(cop0, 0);
/* Set Wait IE/IXMT Ignore in Config7, IAR, AR */
kvm_write_c0_guest_config7(cop0, (MIPS_CONF7_WII) | (1 << 10));
@ -416,6 +540,7 @@ static int kvm_trap_emul_set_one_reg(struct kvm_vcpu *vcpu,
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
int ret = 0;
unsigned int cur, change;
switch (reg->id) {
case KVM_REG_MIPS_CP0_COUNT:
@ -444,6 +569,44 @@ static int kvm_trap_emul_set_one_reg(struct kvm_vcpu *vcpu,
kvm_write_c0_guest_cause(cop0, v);
}
break;
case KVM_REG_MIPS_CP0_CONFIG:
/* read-only for now */
break;
case KVM_REG_MIPS_CP0_CONFIG1:
cur = kvm_read_c0_guest_config1(cop0);
change = (cur ^ v) & kvm_mips_config1_wrmask(vcpu);
if (change) {
v = cur ^ change;
kvm_write_c0_guest_config1(cop0, v);
}
break;
case KVM_REG_MIPS_CP0_CONFIG2:
/* read-only for now */
break;
case KVM_REG_MIPS_CP0_CONFIG3:
cur = kvm_read_c0_guest_config3(cop0);
change = (cur ^ v) & kvm_mips_config3_wrmask(vcpu);
if (change) {
v = cur ^ change;
kvm_write_c0_guest_config3(cop0, v);
}
break;
case KVM_REG_MIPS_CP0_CONFIG4:
cur = kvm_read_c0_guest_config4(cop0);
change = (cur ^ v) & kvm_mips_config4_wrmask(vcpu);
if (change) {
v = cur ^ change;
kvm_write_c0_guest_config4(cop0, v);
}
break;
case KVM_REG_MIPS_CP0_CONFIG5:
cur = kvm_read_c0_guest_config5(cop0);
change = (cur ^ v) & kvm_mips_config5_wrmask(vcpu);
if (change) {
v = cur ^ change;
kvm_write_c0_guest_config5(cop0, v);
}
break;
case KVM_REG_MIPS_COUNT_CTL:
ret = kvm_mips_set_count_ctl(vcpu, v);
break;
@ -459,6 +622,18 @@ static int kvm_trap_emul_set_one_reg(struct kvm_vcpu *vcpu,
return ret;
}
static int kvm_trap_emul_vcpu_get_regs(struct kvm_vcpu *vcpu)
{
kvm_lose_fpu(vcpu);
return 0;
}
static int kvm_trap_emul_vcpu_set_regs(struct kvm_vcpu *vcpu)
{
return 0;
}
static struct kvm_mips_callbacks kvm_trap_emul_callbacks = {
/* exit handlers */
.handle_cop_unusable = kvm_trap_emul_handle_cop_unusable,
@ -470,6 +645,10 @@ static struct kvm_mips_callbacks kvm_trap_emul_callbacks = {
.handle_syscall = kvm_trap_emul_handle_syscall,
.handle_res_inst = kvm_trap_emul_handle_res_inst,
.handle_break = kvm_trap_emul_handle_break,
.handle_trap = kvm_trap_emul_handle_trap,
.handle_msa_fpe = kvm_trap_emul_handle_msa_fpe,
.handle_fpe = kvm_trap_emul_handle_fpe,
.handle_msa_disabled = kvm_trap_emul_handle_msa_disabled,
.vm_init = kvm_trap_emul_vm_init,
.vcpu_init = kvm_trap_emul_vcpu_init,
@ -483,6 +662,8 @@ static struct kvm_mips_callbacks kvm_trap_emul_callbacks = {
.irq_clear = kvm_mips_irq_clear_cb,
.get_one_reg = kvm_trap_emul_get_one_reg,
.set_one_reg = kvm_trap_emul_set_one_reg,
.vcpu_get_regs = kvm_trap_emul_vcpu_get_regs,
.vcpu_set_regs = kvm_trap_emul_vcpu_set_regs,
};
int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks)

2
include/uapi/linux/kvm.h
View file

@ -802,6 +802,8 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_S390_MEM_OP 108
#define KVM_CAP_S390_USER_STSI 109
#define KVM_CAP_S390_SKEYS 110
#define KVM_CAP_MIPS_FPU 111
#define KVM_CAP_MIPS_MSA 112
#ifdef KVM_CAP_IRQ_ROUTING