Commit graph

172 commits

Author SHA1 Message Date
Dotan Barak
abb6e9ba17 IB/mthca: Return actual capacity from create_srq
Have mthca's create_srq method return the actual capacity of the SRQ
that gets created.  Also update comments in <rdma/ib_verbs.h> to
clarify that this is what is expected from ib_create_srq().

Signed-off-by: Dotan Barak <dotanb@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:16 -08:00
Roland Dreier
00df1b2c8b IB/mthca: Bump driver version and release date
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:15 -08:00
Eli Cohen
8ebe5077e3 IB/mthca: Support for query QP and SRQ
Implement the query_qp and query_srq methods in mthca.

Signed-off-by: Eli Cohen <eli@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:15 -08:00
Roland Dreier
d844183d9c IB/mthca: Convert to use ib_modify_qp_is_ok()
Use ib_modify_qp_is_ok() in mthca, and delete the big table of
attributes for queue pair state transitions.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:13 -08:00
Roland Dreier
3fa1fa3e80 IB/mthca: Generate SQ drained events when requested
Add low-level driver support to ib_mthca so that consumers can request
a "send queue drained" event be generated when a transiton to the SQD
state completes.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:12 -08:00
Or Gerlitz
d36f34aadf IB: Enable FMR pool user to set page size
This patch allows the consumer to set the page size of "pages" mapped
by the pool FMRs, which is a feature already existing in the base
verbs API.  On the cosmetic side it changes ib_fmr_attr.page_size field
to be named page_shift.

Signed-off-by: Or Gerlitz <ogerlitz@voltaire.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:10 -08:00
Roland Dreier
6dfc3901b0 IB/mthca: Add modify_device method to set node description
Add a modify_device method to mthca, which implements setting the node
description.  This makes the writable "node_desc" sysfs attribute work
for Mellanox HCAs.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:10 -08:00
Roland Dreier
2fa5e2ebbe IB/mthca: Whitespace cleanups
Remove trailing whitespace and fix indentation that with spaces
instead of tabs.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:09 -08:00
Roland Dreier
4885bf64bc IB/mthca: Add device-specific support for resizing CQs
Add low-level driver support for resizing CQs (both kernel and
userspace) to mthca.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:08 -08:00
Roland Dreier
399d792129 IB/mthca: Get rid of might_sleep() annotations
The might_sleep() annotations in mthca are silly -- they all occur
shortly before calls that will end up in core functions like kmalloc()
that will print the same warning in an unsafe context anyway.  In
fact, beyond cluttering the source, we're actually bloating text with
CONFIG_DEBUG_SPINLOCK_SLEEP and/or CONFIG_PREEMPT_VOLUNTARY set.

With both options set, getting rid of the might_sleep()s saves a lot:
add/remove: 0/0 grow/shrink: 0/7 up/down: 0/-171 (-171)
function                                     old     new   delta
mthca_pd_alloc                               132     109     -23
mthca_init_cq                                969     946     -23
mthca_mr_alloc                               592     568     -24
mthca_pd_free                                 67      42     -25
mthca_free_mr                                219     194     -25
mthca_free_cq                                570     545     -25
mthca_fmr_alloc                              742     716     -26

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:07 -08:00
Roland Dreier
d9b98b0f11 IB/mthca: Make functions that never fail return void
The function mthca_free_err_wqe() can never fail, so get rid of its
return value.  That means handle_error_cqe() doesn't have to check
what mthca_free_err_wqe() returns, which means it can't fail either
and doesn't have to return anything either.  All this results in
simpler source code and a slight object code improvement:

add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-10 (-10)
function                                     old     new   delta
mthca_free_err_wqe                            83      81      -2
mthca_poll_cq                               1758    1750      -8

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-20 10:08:07 -08:00
Roland Dreier
7d2babc487 IB/mthca: bump driver version and release date
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-02-13 12:19:44 -08:00
Roland Dreier
f295c79b67 IB/mthca: Don't print debugging info until we have all values
When debugging is enabled, the mthca_QUERY_DEV_LIM() firmware command
function prints out some of the device limits that it queries.
However the debugging prints happen before all of the fields are
extracted from the firmware response, so some of the values that get
printed are uninitialized junk.  Move the prints to the end of the
function to fix this.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-02-10 18:02:44 -08:00
Roland Dreier
fd9cfdd11b IB/mthca: Semaphore to mutex conversions
Convert semaphores to mutexes in mthca.  Leave firmware command
interface poll_sem and event_sem as semaphores.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-30 16:45:11 -08:00
Michael S. Tsirkin
e3aa31c517 IB/mthca: Don't cancel commands on a signal
We have run into the following problem: if a task receives a signal
while in the process of e.g. destroying a resource (which could be
because the relevant file was closed) mthca could bail out from trying
to take a command interface semaphore without performing the
appropriate command to tell hardware that the resource is being
destroyed.

As a result we see messages like
 ib_mthca 0000:04:00.0: HW2SW_CQ failed (-4)

In this case, hardware could access the resource after the memory has
been freed, possibly causing memory corruption.

A simple solution is to replace down_interruptible() by down() in
command interface activation.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
[ It's also not safe to bail out on multicast table operations, since
  they may be invoked on the cleanup path too.  So use down() for
  mcg_table.sem too. ]
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-30 16:22:29 -08:00
Michael S. Tsirkin
cbd2981a97 IB/mthca: Relax UAR size check
There are some cards around that have UAR (user access region) size
different from 8 MB.  Relax our sanity check to make sure that the PCI
BAR is big enough to access the UAR size reported by the device
firmware instead.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-30 15:20:35 -08:00
Michael S. Tsirkin
f9e61929e5 IB/mthca: Use correct GID in MADs sent on port 2
mthca_create_ah() includes the port number in the GID index. The reverse
needs to be done in mthca_read_ah().

Noted by Hal Rosenstock.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-21 14:02:59 -08:00
Michael S. Tsirkin
9eacee2ac6 IB/mthca: Initialize grh_present before using it
build_mlx_header() was using sqp->ud_header.grh_present before it was
initialized by mthca_read_ah().  Furthermore, header->grh_present is
set by ib_ud_header_init, so there's no need to set it again in
mthca_read_ah().

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-12 15:55:41 -08:00
Michael S. Tsirkin
c063a06835 IB/mthca: Cosmetic: use the ALIGN macro
Use the ALIGN macro to simplify some rounding code.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-12 15:43:58 -08:00
Jack Morgenstein
17e2e81951 IB/mthca: Fix memory leaks in error handling
Fix memory leaks in mthca_create_qp() and mthca_create_srq()
error handling.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-12 15:35:15 -08:00
Ishai Rabinovitz
59f174faff IB/mthca: Fix memory leak of multicast group structures
Convert "/ (1 << lg)" to ">> lg" for a slight code size reduction.

add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-24 (-24)
function                                     old     new   delta
mthca_map_cmd                                613     589     -24

Signed-off-by: Ishai Rabinovitz <ishai@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-12 15:24:51 -08:00
Sean Hefty
cf311cd49a IB: Add node_guid to struct ib_device
Add a node_guid field to struct ib_device.  It is the responsibility
of the low-level driver to initialize this field before registering a
device with the midlayer.  Convert everyone to looking at this field
instead of calling ib_query_device() when all they want is the node
GUID, and remove the node_guid field from struct ib_device_attr.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-10 07:39:34 -08:00
Roland Dreier
87635b71b5 IB/mthca: Factor common MAD initialization code
Factor out common code for initializing MAD packets, which is shared
by many query routines in mthca_provider.c, into init_query_mad().

add/remove: 1/0 grow/shrink: 0/4 up/down: 16/-44 (-28)
function                                     old     new   delta
init_query_mad                                 -      16     +16
mthca_query_port                             521     518      -3
mthca_query_pkey                             301     294      -7
mthca_query_device                           648     641      -7
mthca_query_gid                              453     426     -27

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-09 15:29:53 -08:00
Roland Dreier
105e50a5e8 IB/mthca: kzalloc conversions
Convert kmalloc()/memset(,0,) pairs to kzalloc().

Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-09 15:21:21 -08:00
Michael S. Tsirkin
92898522e3 IB/mthca: prevent event queue overrun
I am seeing EQ overruns in SDP stress tests: if the CQ completion
handler arms a CQ, this could generate more EQEs, so that EQ will
never get empty and consumer index will never get updated.

This is similiar to what we have with command interface:
		/*
		 * cmd_event() may add more commands.
		 * The card will think the queue has overflowed if
		 * we don't tell it we've been processing events.
		 */
However, for completion events, we *don't* want to update the consumer
index on each event. So, perform EQ doorbell coalescing: allocate EQs
with some spare EQEs, and update once we run out of them.

The value 0x80 was selected to avoid any performance impact.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-09 14:04:40 -08:00
Michael S. Tsirkin
6627fa662e IB/mthca: fix page shift calculation in mthca_reg_phys_mr()
For all pages except possibly the last one, the byte beyond the buffer
end must be page aligned.  Therefore, when computing the page shift to
use, OR the end addresses of the buffers as well as the start
addresses into the mask we check.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-09 13:50:57 -08:00
Linus Torvalds
5367f2d67c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband 2006-01-08 20:18:44 -08:00
Tim Schmielau
de25968cc8 [PATCH] fix more missing includes
Include fixes for 2.6.14-git11.  Should allow to remove sched.h from
module.h on i386, x86_64, arm, ia64, ppc, ppc64, and s390.  Probably more
to come since I haven't yet checked the other archs.

Signed-off-by: Tim Schmielau <tim@physik3.uni-rostock.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-01-08 20:13:45 -08:00
Dotan Barak
4de144bf72 IB/mthca: Add support for automatic path migration (APM)
Add code to modify QP operation to handle setting alternate paths for
connected QPs.

Signed-off-by: Dotan Barak <dotanb@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-06 13:23:58 -08:00
Michael S. Tsirkin
0f8e8f9607 IB/mthca: Fill in vendor_err field in completion with error
Fill vendor_err field in completion with error.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-06 13:13:32 -08:00
Jack Morgenstein
5ceb74557c IB/mthca: multiple fixes for multicast group handling
Multicast group management fixes:
. Fix leak of mailbox memory in error handling on multicast group operations.
. Free AMGM indices at detach and in attach error handling.
. Fix amount to shift for aligning next_gid_index in mailbox: it
  starts at bit 6, not bit 5.
. Allocate AMGM index after end of MGM table, in the range num_mgms to
  multicast table size - 1. Add some BUG_ON checks to catch cases
  where the index falls in the MGM hash area.
. Initialize the list of QPs in a newly-allocated group from AMGM to 0
  This is necessary since when a group is moved from AMGM to MGM (in the
  case where the MGM entry has been emptied of QPs), the AMGM entry is
  not reset to 0 (and we don't want an extra command to do that).

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-06 13:11:07 -08:00
Jack Morgenstein
0d3b525fff IB/mthca: fix for RTR-to-RTS transition in modify QP
PKEY_INDEX is not a legal parameter in the RTR->RTS transition.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-06 13:03:43 -08:00
Jack Morgenstein
0364ffc3e8 IB/mthca: fix for SQEr-to-RTS transition in modify QP
Fixes to SQEr->RTS transition in modify_qp:
1. The flag IB_QP_ACCESS_FLAGS is optional for UC qps
2. The SQEr state is not supported for RC qps

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-06 13:01:27 -08:00
Jack Morgenstein
5b3bc7a681 IB/mthca: max_inline_data handling tweaks
Fix a case where copying max_inline_data from a successful create_qp
capabilities output to create_qp input could cause EINVAL error:

mthca_set_qp_size must check max_inline_data directly against
max_desc_sz; checking qp->sq.max_gs is wrong since max_inline_data
depends on the qp type and does not involve max_sg.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-06 12:57:30 -08:00
Michael S. Tsirkin
466200562c IB/mthca: create_eq with size not a power of 2
Fix mthca_create_eq for when the EQ size is not a power of 2.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-05 16:17:38 -08:00
Jack Morgenstein
38d1e79347 IB/mthca: check port validity in modify_qp
Modify_qp should check that the physical port number provided
is a legal value.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-05 16:13:46 -08:00
Jack Morgenstein
aa2f936779 IB/mthca: check return value in mthca_dev_lim call
Check error return on call to mthca_dev_lim for Tavor
(as is done for memfree).

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-05 16:12:01 -08:00
Jack Morgenstein
1d7d2f6f47 IB/mthca: fix WQE size calculation in create-srq
Thinko: 64 bytes is the minimum SRQ WQE size (not the maximum).

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-04 14:42:39 -08:00
Jack Morgenstein
c4342d8a4d IB/mthca: Fix corner cases in max_rd_atomic value handling in modify QP
sae and sre bits should only be set when setting sra_max.  Further, in
the old code, if the caller specifies max_rd_atomic = 0, the sre and
sae bits are still set, with the result that the QP ends up with
max_rd_atomic = 1 in effect.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-15 19:59:01 -08:00
Jack Morgenstein
d1646f86a2 IB/mthca: Fix IB_QP_ACCESS_FLAGS handling.
This patch corrects some corner cases in managing the RAE/RRE bits in
the mthca qp context.  These bits need to be zero if the user requests
max_dest_rd_atomic of zero.  The bits need to be restored to the value
implied by the qp access flags attribute in a previous (or the
current) modify-qp command if the dest_rd_atomic variable is changed
to non-zero.

In the current implementation, the following scenario will not work:
RESET-to-INIT 	set QP access flags to all disabled (zeroes)
INIT-to-RTR     set max_dest_rd_atomic=10, AND
		set qp_access_flags = IB_ACCESS_REMOTE_READ | IB_ACCESS_REMOTE_ATOMIC

The current code will incorrectly take the access-flags value set in
the RESET-to-INIT transition.

We can simplify, and correct, this IB_QP_ACCESS_FLAGS handling: it is
always safe to set qp access flags in the firmware command if either
of IB_QP_MAX_DEST_RD_ATOMIC or IB_QP_ACCESS_FLAGS is set, so let's
just set it to the correct value, always.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-15 14:36:24 -08:00
Jack Morgenstein
576d2e4e40 IB/mthca: Fix SRQ cleanup during QP destroy
When cleaning up a CQ for a QP attached to SRQ, need to free an SRQ
WQE only if the CQE is a receive completion.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-15 14:20:23 -08:00
Michael S. Tsirkin
6c7d2a75b5 IB/mthca: Fix thinko in mthca_table_find()
break only escapes from the innermost loop, and we want to escape both
loops and return an answer.  Noticed by Ishai Rabinovitch.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-15 13:55:50 -08:00
Jack Morgenstein
44b5b03033 IB/mthca: don't change driver's copy of attributes if modify QP fails
Only change the driver's copy of the QP attributes in modify QP after
checking the modify QP command completed successfully.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-09 16:40:14 -08:00
Jack Morgenstein
6aa2e4e806 IB/mthca: correct log2 calculation
Fix thinko in rd_atomic calculation: ffs(x) - 1 does not find the next
power of 2 -- it should be fls(x - 1).

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-09 16:38:04 -08:00
Jack Morgenstein
94361cf74a IB/mthca: check RDMA limits
Add limit checking on rd_atomic and dest_rd_atomic attributes:
especially for max_dest_rd_atomic, a value that is larger than HCA
capability can cause RDB overflow and corruption of another QP.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-09 16:32:21 -08:00
Jack Morgenstein
52d0df153c IB/mthca: fix memory user DB table leak
Free the memory allocated in mthca_init_user_db_tab() when releasing
the db_tab in mthca_cleanup_user_db_tab().

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-12-09 13:48:50 -08:00
Jack Morgenstein
a3c8ab4fe8 IB/mthca: fix QP size limits for mem-free HCAs
Unlike tavor, the max work queue size is an exact power of 2 for arbel
mode, despite what the documentation (of the QUERY_DEV_LIM firmware
command) says.  Without this patch, on Arbel, we can start with a QP
of a valid size and get above the reported limit after rounding to the
next power of two.

Signed-off-by: Jack Morgenstein <jackm@mellanox.co.il>
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-11-30 09:55:22 -08:00
Michael S. Tsirkin
e0ae9ecf46 IB/mthca: fix posting of send lists of length >= 255 on mem-free HCAs
On mem-free HCAs, when posting a long list of send requests, a
doorbell must be rung every 255 requests.  Add code to handle this.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-11-29 11:33:46 -08:00
Michael S. Tsirkin
187a25863f IB/mthca: reset QP's last pointers when transitioning to reset state
last pointer is not updated when QP is modified to reset state.  This
causes data corruption if WQEs are already posted on the queue.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-11-28 11:19:43 -08:00
Linus Torvalds
9b152d53b7 Merge branch 'for-linus' of master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband 2005-11-18 14:33:03 -08:00