29c8d9eba5
This patch series adds a driver for a paravirtual RDMA device. The device is developed for VMware's Virtual Machines and allows existing RDMA applications to continue to use existing Verbs API when deployed in VMs on ESXi. We recently did a presentation in the OFA Workshop [1] regarding this device. Description and RDMA Support ============================ The virtual device is exposed as a dual function PCIe device. One part is a virtual network device (VMXNet3) which provides networking properties like MAC, IP addresses to the RDMA part of the device. The networking properties are used to register GIDs required by RDMA applications to communicate. These patches add support and the all required infrastructure for letting applications use such a device. We support the mandatory Verbs API as well as the base memory management extensions (Local Inv, Send with Inv and Fast Register Work Requests). We currently support both Reliable Connected and Unreliable Datagram QPs but do not support Shared Receive Queues (SRQs). Also, we support the following types of Work Requests: o Send/Receive (with or without Immediate Data) o RDMA Write (with or without Immediate Data) o RDMA Read o Local Invalidate o Send with Invalidate o Fast Register Work Requests This version only adds support for version 1 of RoCE. We will add RoCEv2 support in a future patch. We do support registration of both MAC-based and IP-based GIDs. I have also created a git tree for our user-level driver [2]. Testing ======= We have tested this internally for various types of Guest OS - Red Hat, Centos, Ubuntu 12.04/14.04/16.04, Oracle Enterprise Linux, SLES 12 using backported versions of this driver. The tests included several runs of the performance tests (included with OFED), Intel MPI PingPong benchmark on OpenMPI, krping for FRWRs. Mellanox has been kind enough to test the backported version of the driver internally on their hardware using a VMware provided ESX build. I have also applied and tested this with Doug's k.o/for-4.9 branch (commit 5603910b). Note, that this patch series should be applied all together. I split out the commits so that it may be easier to review. PVRDMA Resources ================ [1] OFA Workshop Presentation - https://openfabrics.org/images/eventpresos/2016presentations/102parardma.pdf [2] Libpvrdma User-level library - http://git.openfabrics.org/?p=~aditr/libpvrdma.git;a=summary Reviewed-by: Jorgen Hansen <jhansen@vmware.com> Reviewed-by: George Zhang <georgezhang@vmware.com> Reviewed-by: Aditya Sarwade <asarwade@vmware.com> Reviewed-by: Bryan Tan <bryantan@vmware.com> Reviewed-by: Leon Romanovsky <leonro@mellanox.com> Signed-off-by: Adit Ranadive <aditr@vmware.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
95 lines
3 KiB
Text
95 lines
3 KiB
Text
menuconfig INFINIBAND
|
|
tristate "InfiniBand support"
|
|
depends on PCI || BROKEN
|
|
depends on HAS_IOMEM
|
|
depends on NET
|
|
depends on INET
|
|
depends on m || IPV6 != m
|
|
select IRQ_POLL
|
|
---help---
|
|
Core support for InfiniBand (IB). Make sure to also select
|
|
any protocols you wish to use as well as drivers for your
|
|
InfiniBand hardware.
|
|
|
|
if INFINIBAND
|
|
|
|
config INFINIBAND_USER_MAD
|
|
tristate "InfiniBand userspace MAD support"
|
|
depends on INFINIBAND
|
|
---help---
|
|
Userspace InfiniBand Management Datagram (MAD) support. This
|
|
is the kernel side of the userspace MAD support, which allows
|
|
userspace processes to send and receive MADs. You will also
|
|
need libibumad from <http://www.openfabrics.org/downloads/management/>.
|
|
|
|
config INFINIBAND_USER_ACCESS
|
|
tristate "InfiniBand userspace access (verbs and CM)"
|
|
select ANON_INODES
|
|
---help---
|
|
Userspace InfiniBand access support. This enables the
|
|
kernel side of userspace verbs and the userspace
|
|
communication manager (CM). This allows userspace processes
|
|
to set up connections and directly access InfiniBand
|
|
hardware for fast-path operations. You will also need
|
|
libibverbs, libibcm and a hardware driver library from
|
|
<http://www.openfabrics.org/git/>.
|
|
|
|
config INFINIBAND_USER_MEM
|
|
bool
|
|
depends on INFINIBAND_USER_ACCESS != n
|
|
default y
|
|
|
|
config INFINIBAND_ON_DEMAND_PAGING
|
|
bool "InfiniBand on-demand paging support"
|
|
depends on INFINIBAND_USER_MEM
|
|
select MMU_NOTIFIER
|
|
default y
|
|
---help---
|
|
On demand paging support for the InfiniBand subsystem.
|
|
Together with driver support this allows registration of
|
|
memory regions without pinning their pages, fetching the
|
|
pages on demand instead.
|
|
|
|
config INFINIBAND_ADDR_TRANS
|
|
bool
|
|
depends on INFINIBAND
|
|
default y
|
|
|
|
config INFINIBAND_ADDR_TRANS_CONFIGFS
|
|
bool
|
|
depends on INFINIBAND_ADDR_TRANS && CONFIGFS_FS && !(INFINIBAND=y && CONFIGFS_FS=m)
|
|
default y
|
|
---help---
|
|
ConfigFS support for RDMA communication manager (CM).
|
|
This allows the user to config the default GID type that the CM
|
|
uses for each device, when initiaing new connections.
|
|
|
|
source "drivers/infiniband/hw/mthca/Kconfig"
|
|
source "drivers/infiniband/hw/qib/Kconfig"
|
|
source "drivers/infiniband/hw/cxgb3/Kconfig"
|
|
source "drivers/infiniband/hw/cxgb4/Kconfig"
|
|
source "drivers/infiniband/hw/i40iw/Kconfig"
|
|
source "drivers/infiniband/hw/mlx4/Kconfig"
|
|
source "drivers/infiniband/hw/mlx5/Kconfig"
|
|
source "drivers/infiniband/hw/nes/Kconfig"
|
|
source "drivers/infiniband/hw/ocrdma/Kconfig"
|
|
source "drivers/infiniband/hw/vmw_pvrdma/Kconfig"
|
|
source "drivers/infiniband/hw/usnic/Kconfig"
|
|
source "drivers/infiniband/hw/hns/Kconfig"
|
|
|
|
source "drivers/infiniband/ulp/ipoib/Kconfig"
|
|
|
|
source "drivers/infiniband/ulp/srp/Kconfig"
|
|
source "drivers/infiniband/ulp/srpt/Kconfig"
|
|
|
|
source "drivers/infiniband/ulp/iser/Kconfig"
|
|
source "drivers/infiniband/ulp/isert/Kconfig"
|
|
|
|
source "drivers/infiniband/sw/rdmavt/Kconfig"
|
|
source "drivers/infiniband/sw/rxe/Kconfig"
|
|
|
|
source "drivers/infiniband/hw/hfi1/Kconfig"
|
|
|
|
source "drivers/infiniband/hw/qedr/Kconfig"
|
|
|
|
endif # INFINIBAND
|