Leon Romanovsky [Mon, 27 Apr 2020 15:46:20 +0000 (18:46 +0300)]
RDMA/mlx5: Store QP type in the vendor QP structure
QP type is stored in the IB/core QP struct, but it doesn't have all the
needed information, like internal QP type used in the driver itself.
Update mlx5_ib to have cached QP type which includes both IBTA and
Mellanox specific one.
Such change allows us to make even further cleanup of QP creation flow.
Jason Gunthorpe [Wed, 29 Apr 2020 00:44:51 +0000 (21:44 -0300)]
Merge branch 'mlx5_ib_qp_refactor_1' into rdma.git for-next
Leon Romanovsky says:
====================
This is first part of series which tries to return some sanity to
mlx5_ib_create_qp() function. Such refactoring is required to make
extension of that function with less worries of breaking driver.
Extra goal of such refactoring is to ensure that QP is allocated at the
beginning of function and released at the end. It will allow us to move QP
allocation to be under IB/core responsibility.
====================
Based on the mlx5-next branch at
git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux
Due to dependencies
* branch 'mlx5_ib_qp_refactor_1': (66 commits)
RDMA/mlx5: Process all vendor flags in one place
RDMA/mlx5: Return all configured create flags through query QP
RDMA/mlx5: Change scatter CQE flag to be set like other vendor flags
RDMA/mlx5: Use flags_en mechanism to mark QP created with WQE signature
RDMA/mlx5: Process create QP flags in one place
RDMA/mlx5: Delete create QP flags obfuscation
RDMA/mlx5: Initial separation of RAW_PACKET QP from common flow
RDMA/mlx5: Remove second copy from user for non RSS RAW QPs
RDMA/mlx5: Move DRIVER QP flags check into separate function
RDMA/mlx5: Update all DRIVER QP places to use QP subtype
RDMA/mlx5: Split scatter CQE configuration for DCT QP
RDMA/mlx5: Separate create QP flows to be based on type
RDMA/mlx5: Set QP subtype immediately when it is known
RDMA/mlx5: Avoid setting redundant NULL for XRC QPs
RDMA/mlx5: Prepare QP allocation for future removal
RDMA/mlx5: Perform check if QP creation flow is valid
RDMA/mlx5: Delete impossible GSI port check
RDMA/mlx5: Organize QP types checks in one place
Leon Romanovsky [Mon, 27 Apr 2020 15:46:16 +0000 (18:46 +0300)]
RDMA/mlx5: Change scatter CQE flag to be set like other vendor flags
In similar way to wqe_sig, the scat_cqe was treated differently from
other create QP vendor flags. Change it to be similar to other flags
and use flags_en mechanism.
Leon Romanovsky [Mon, 27 Apr 2020 15:46:15 +0000 (18:46 +0300)]
RDMA/mlx5: Use flags_en mechanism to mark QP created with WQE signature
MLX5_QP_FLAG_SIGNATURE is exposed to the users but in the kernel
the create_qp flow treated it differently from other MLX5_QP_FLAG_*s.
Fix it by ditching wq_sig boolean variable and use general flag_en
mechanism.
Leon Romanovsky [Mon, 27 Apr 2020 15:46:14 +0000 (18:46 +0300)]
RDMA/mlx5: Process create QP flags in one place
create_flags is checked in too many places and scattered across all
the code, consolidate all the checks inside one function, so we will
be easily see the flow. As part of such change, delete unreachable code,
because IB/core is responsible sanitize the input.
Leon Romanovsky [Mon, 27 Apr 2020 15:46:13 +0000 (18:46 +0300)]
RDMA/mlx5: Delete create QP flags obfuscation
There is no point in redefinition of stable and exposed to users create
flags. Their values won't be changed and it is equal to used by the
mlx5. Delete the mlx5 definitions and use IB/core fields.
Leon Romanovsky [Mon, 27 Apr 2020 15:46:09 +0000 (18:46 +0300)]
RDMA/mlx5: Update all DRIVER QP places to use QP subtype
Instead of overwriting QP init attributes with driver QP subtype,
use that subtype directly. This change will allow us to remove
logic which cached QP init attributes.
Leon Romanovsky [Mon, 27 Apr 2020 15:46:06 +0000 (18:46 +0300)]
RDMA/mlx5: Set QP subtype immediately when it is known
There is no need to delay QP subtype assignment to the end of the
create_qp() function and it is better to move it to be immediately
after it is checked so we would be able to rewrite later checks
to be based on it and not on over-written struct ib_qp_init_attr.
Leon Romanovsky [Mon, 27 Apr 2020 15:46:01 +0000 (18:46 +0300)]
RDMA/mlx5: Organize QP types checks in one place
Perform check if QP type is supported in one place at the beginning of
the create_qp function instead of current implementation with checks
buried inside of the code.
Raed Salem [Fri, 24 Apr 2020 19:45:10 +0000 (12:45 -0700)]
net/mlx5: TX WQE Add trailer insertion field
Add new TX WQE field for Connect-X6DX trailer insertion support,
when set, the HW adds a trailer to the packet, the WQE trailer
association flags are used to set to HW the header which the
trailer belongs.
Signed-off-by: Raed Salem <raeds@mellanox.com> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
net/mlx5: Add structure layout and defines for MFRL register
Add needed structure layouts and defines for MFRL (Management Firmware
Reset Level) register. This structure will be used for the firmware
upgrade and reset flow in the downstream patches.
Raed Salem [Fri, 24 Apr 2020 19:45:05 +0000 (12:45 -0700)]
net/mlx5: Refactor imm_inval_pkey field in cqe struct
The imm_inval_pkey field can hold four different types of data,
depends on the usage, the data could be one of the below:
- Immediate field of the received message
- Invalidate rkey
- Pkey of the packet
- Flow table metadata
Current implementation doesn't reflect the intended usage of the
field at usage time.
Reflect the different types by replace this field with a union,
modify code where this field is used to reflect its intended
usage.
net/mlx5: Use aligned variable while allocating ICM memory
The alignment value is part of the input structure, so use it and spare
extra memory allocation when is not needed.
Now, using the new ability when allocating icm for Direct-Rule
insertion. Signed-off-by: Ariel Levkovich <lariel@mellanox.com> Signed-off-by: Erez Shitrit <erezsh@mellanox.com> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Lang Cheng [Wed, 15 Apr 2020 08:14:33 +0000 (16:14 +0800)]
RDMA/hns: Simplify the cqe code of poll cq
Encapsulate codes to get status of cqe into a function and use map table
instead of switch-case to reduce cyclomatic complexity of
hns_roce_v2_poll_one().
Leon Romanovsky [Sun, 19 Apr 2020 13:20:46 +0000 (16:20 +0300)]
RDMA/bnxt: Delete 'nq_ptr' variable which is not used
The variable "nq_ptr" is set but never used, this generates the following
warning while compiling kernel with W=1 option.
drivers/infiniband/hw/bnxt_re/qplib_fp.c: In function 'bnxt_qplib_service_nq':
drivers/infiniband/hw/bnxt_re/qplib_fp.c:303:25: warning:
variable 'nq_ptr' set but not used [-Wunused-but-set-variable]
303 | struct nq_base *nqe, **nq_ptr;
|
Xi Wang [Mon, 13 Apr 2020 11:58:09 +0000 (19:58 +0800)]
RDMA/hns: Support 0 hop addressing for WQE buffer
Add the zero hop addressing support by using new mtr interface for WQE
buffer and simple mtr invoking process, so WQE buffer can support hopnum
between 0 to 3.
Xi Wang [Mon, 13 Apr 2020 11:58:07 +0000 (19:58 +0800)]
RDMA/hns: Optimize hns buffer allocation flow
When the value of nbufs is 1, the buffer is in direct mode, which may cause
confusion. So optimizes current codes to make it easier to maintain and
understand.
Xi Wang [Mon, 13 Apr 2020 11:58:06 +0000 (19:58 +0800)]
RDMA/hns: Add support for addressing when hopnum is 0
Currently, WQE and EQE table have already used the mtr interface to config
and access memory by multi-hop addressing when hopnum is from 1 to 3. But
if hopnum is 0, each table need write its own but repetitive logic, and
many duplicate code exists in the mtr interfaces invoke process.
So wraps the public logic as 3 functions: hns_roce_mtr_create(),
hns_roce_mtr_destroy() and hns_roce_mtr_map() to support hopnum ranges from
0 to 3. In addition, makes the mtr interfaces easier to use.
Leon Romanovsky [Thu, 19 Mar 2020 09:43:59 +0000 (11:43 +0200)]
net/mlx5: Refactor HCA capability set flow
Reduce the amount of kzalloc/kfree cycles by allocating
command structure in the parent function and leverage the
knowledge that set_caps() is called for HCA capabilities
only with specific HW structure as parameter to calculate
mailbox size.
Acked-by: Saeed Mahameed <saeedm@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Leon Romanovsky [Tue, 7 Apr 2020 13:09:15 +0000 (16:09 +0300)]
net/mlx5: Delete not-used cmd header
The structures defined in the cmd header are not used and can be safely
removed from the driver. This patch removes that file and deletes all
relevant includes.
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Leon Romanovsky [Fri, 3 Apr 2020 10:31:18 +0000 (13:31 +0300)]
RDMA/mlx5: Delete Q counter allocations command
Remove mlx5_ib implementation of Q counter allocation logic
together with cleaning boolean which controlled validity of the
counter. It is not needed, because counter_id == 0 means that
counter is not valid.
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Leon Romanovsky [Fri, 3 Apr 2020 08:28:28 +0000 (11:28 +0300)]
net/mlx5: Remove Q counter low level helper APIs
mlx5 core users are encouraged to use low level API (mlx5_cmd_exec)
without the need of helper functions, do this for q counters, remove
helper functions and call mlx5_cmd_exec directly from users.
This will help reduce the total amount of code and reduction of the
mlx5_core symbol table.
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Leon Romanovsky [Mon, 6 Apr 2020 12:53:06 +0000 (15:53 +0300)]
net/mlx5: Replace hand written QP context struct with automatic getters
By changing debugfs to not use any QP related API, convert the debugfs
code to use MLX5_GET() directly to access QP context instead of hand
written struct.
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Leon Romanovsky [Mon, 6 Apr 2020 05:40:52 +0000 (08:40 +0300)]
net/mlx5: Remove empty QP and CQ events handlers
The QP and CQ events functions do nothing except printing some debug
messages. There is nothing to do with this knowledge and such events,
so remove them.
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Leon Romanovsky [Sun, 5 Apr 2020 17:57:00 +0000 (20:57 +0300)]
net/mlx5: Provide simplified command interfaces
Many mlx5_cmd_exec() callers are not interested in the output from that
command or have standard in/out structures. Those callers simply allocate
those structure on the stack and use sizeof() to provide in/out arguments.
In this naive approach provide simplified versions of mlx5_cmd_exec().
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
RDMA/bnxt_re: Simplify obtaining queue entry from hw ring
Restructring the data path and control path queue management code to
simplify the way a queue element is extracted from the hardware ring.
Introduced a new function which will give a pointer to the next ring item
depending upon the current cons/prod index in the hardware queue.
Further, there are hardcoding when size of queue entry is calculated,
replacing it with an inline function. This function would be easier to
expand if need going forward.
The code section to initialize the PSN search areas has also been
restructured and couple of functions has been added there.
Adding fast path support data structure into hardware HSI. These
structures are header only definition of RQE/SRQE/SQE. This is to help
calculating the size of hardware wqe size.
Getting rid of the repeated code in the driver when deciding on the page
size of the hardware ring memory. A new common function would translate
the ring page size into device specific page size.
Zou Wei [Mon, 13 Apr 2020 02:42:04 +0000 (10:42 +0800)]
IB/qib: Remove unused variable ret
This patch fixes below warnings reported by coccicheck
drivers/infiniband/hw/qib/qib_iba7322.c:6878:8-11:
Unneeded variable: "ret". Return "0" on line 6907
drivers/infiniband/hw/qib/qib_iba7322.c:2378:5-8:
Unneeded variable: "ret". Return "0" on line 2513
Jason Gunthorpe [Tue, 7 Apr 2020 00:44:26 +0000 (21:44 -0300)]
RDMA/uverbs: Make the event_queue fds return POLLERR when disassociated
If is_closed is set, and the event list is empty, then read() will return
-EIO without blocking. After setting is_closed in
ib_uverbs_free_event_queue(), we do trigger a wake_up on the poll_wait,
but the fops->poll() function does not check it, so poll will continue to
sleep on an empty list.
The RXE driver doesn't set vendor_id and user space applications see
zeros. This causes to pyverbs tests to fail with the following traceback,
because the expectation is to have valid vendor_id.
Traceback (most recent call last):
File "tests/test_device.py", line 51, in test_query_device
self.verify_device_attr(attr)
File "tests/test_device.py", line 77, in verify_device_attr
assert attr.vendor_id != 0
In order to fix it, we will set vendor_id 0XFFFFFF, according to the IBTA
v1.4 A3.3.1 VENDOR INFORMATION section.
"""
A vendor that produces a generic controller (i.e., one that supports a
standard I/O protocol such as SRP), which does not have vendor specific
device drivers, may use the value of 0xFFFFFF in the VendorID field.
"""
This sorts the actual field names too, potentially causing even more
chaos and confusion at merge time if you have edited the MAINTAINERS
file. But the end result is a more consistent layout, and hopefully
it's a one-time pain minimized by doing this just before the -rc1
release.
They are all supposed to be sorted, but people who add new entries don't
always know the alphabet. Plus sometimes the entry names get edited,
and people don't then re-order the entry.
Let's see how painful this will be for merging purposes (the MAINTAINERS
file is often edited in various different trees), but Joe claims there's
relatively few patches in -next that touch this, and doing it just
before -rc1 is likely the best time. Fingers crossed.