What: /sys/class/ata_...
-Date: August 2008
-Contact: Gwendal Grignou<gwendal@google.com>
Description:
-
-Provide a place in sysfs for storing the ATA topology of the system. This allows
-retrieving various information about ATA objects.
+ Provide a place in sysfs for storing the ATA topology of the
+ system. This allows retrieving various information about ATA
+ objects.
Files under /sys/class/ata_port
-------------------------------
- For each port, a directory ataX is created where X is the ata_port_id of
- the port. The device parent is the ata host device.
+For each port, a directory ataX is created where X is the ata_port_id of the
+port. The device parent is the ata host device.
-idle_irq (read)
- Number of IRQ received by the port while idle [some ata HBA only].
+What: /sys/class/ata_port/ataX/nr_pmp_links
+What: /sys/class/ata_port/ataX/idle_irq
+Date: May, 2010
+KernelVersion: v2.6.37
+Contact: Gwendal Grignou <gwendal@chromium.org>
+Description:
+ nr_pmp_links: (RO) If a SATA Port Multiplier (PM) is
+ connected, the number of links behind it.
-nr_pmp_links (read)
+ idle_irq: (RO) Number of IRQ received by the port while
+ idle [some ata HBA only].
- If a SATA Port Multiplier (PM) is connected, number of link behind it.
+
+What: /sys/class/ata_port/ataX/port_no
+Date: May, 2013
+KernelVersion: v3.11
+Contact: Gwendal Grignou <gwendal@chromium.org>
+Description:
+ (RO) Host local port number. While registering host controller,
+ port numbers are tracked based upon number of ports available on
+ the controller. This attribute is needed by udev for composing
+ persistent links in /dev/disk/by-path.
Files under /sys/class/ata_link
-------------------------------
- Behind each port, there is a ata_link. If there is a SATA PM in the
- topology, 15 ata_link objects are created.
-
- If a link is behind a port, the directory name is linkX, where X is
- ata_port_id of the port.
- If a link is behind a PM, its name is linkX.Y where X is ata_port_id
- of the parent port and Y the PM port.
+Behind each port, there is a ata_link. If there is a SATA PM in the topology, 15
+ata_link objects are created.
-hw_sata_spd_limit
+If a link is behind a port, the directory name is linkX, where X is ata_port_id
+of the port. If a link is behind a PM, its name is linkX.Y where X is
+ata_port_id of the parent port and Y the PM port.
- Maximum speed supported by the connected SATA device.
-sata_spd_limit
+What: /sys/class/ata_link/linkX[.Y]/hw_sata_spd_limit
+What: /sys/class/ata_link/linkX[.Y]/sata_spd_limit
+What: /sys/class/ata_link/linkX[.Y]/sata_spd
+Date: May, 2010
+KernelVersion: v2.6.37
+Contact: Gwendal Grignou <gwendal@chromium.org>
+Description:
+ hw_sata_spd_limit: (RO) Maximum speed supported by the
+ connected SATA device.
- Maximum speed imposed by libata.
+ sata_spd_limit: (RO) Maximum speed imposed by libata.
-sata_spd
+ sata_spd: (RO) Current speed of the link
+ eg. 1.5, 3 Gbps etc.
- Current speed of the link [1.5, 3Gps,...].
Files under /sys/class/ata_device
---------------------------------
- Behind each link, up to two ata device are created.
- The name of the directory is devX[.Y].Z where:
- - X is ata_port_id of the port where the device is connected,
- - Y the port of the PM if any, and
- - Z the device id: for PATA, there is usually 2 devices [0,1],
- only 1 for SATA.
-
-class
- Device class. Can be "ata" for disk, "atapi" for packet device,
- "pmp" for PM, or "none" if no device was found behind the link.
-
-dma_mode
+Behind each link, up to two ata devices are created.
+The name of the directory is devX[.Y].Z where:
+- X is ata_port_id of the port where the device is connected,
+- Y the port of the PM if any, and
+- Z the device id: for PATA, there is usually 2 devices [0,1], only 1 for SATA.
+
+
+What: /sys/class/ata_device/devX[.Y].Z/spdn_cnt
+What: /sys/class/ata_device/devX[.Y].Z/gscr
+What: /sys/class/ata_device/devX[.Y].Z/ering
+What: /sys/class/ata_device/devX[.Y].Z/id
+What: /sys/class/ata_device/devX[.Y].Z/pio_mode
+What: /sys/class/ata_device/devX[.Y].Z/xfer_mode
+What: /sys/class/ata_device/devX[.Y].Z/dma_mode
+What: /sys/class/ata_device/devX[.Y].Z/class
+Date: May, 2010
+KernelVersion: v2.6.37
+Contact: Gwendal Grignou <gwendal@chromium.org>
+Description:
+ spdn_cnt: (RO) Number of times libata decided to lower the
+ speed of link due to errors.
- Transfer modes supported by the device when in DMA mode.
- Mostly used by PATA device.
+ gscr: (RO) Cached result of the dump of PM GSCR
+ register. Valid registers are:
-pio_mode
+ 0: SATA_PMP_GSCR_PROD_ID,
+ 1: SATA_PMP_GSCR_REV,
+ 2: SATA_PMP_GSCR_PORT_INFO,
+ 32: SATA_PMP_GSCR_ERROR,
+ 33: SATA_PMP_GSCR_ERROR_EN,
+ 64: SATA_PMP_GSCR_FEAT,
+ 96: SATA_PMP_GSCR_FEAT_EN,
+ 130: SATA_PMP_GSCR_SII_GPIO
- Transfer modes supported by the device when in PIO mode.
- Mostly used by PATA device.
+ Only valid if the device is a PM.
-xfer_mode
+ ering: (RO) Formatted output of the error ring of the
+ device.
- Current transfer mode.
+ id: (RO) Cached result of IDENTIFY command, as
+ described in ATA8 7.16 and 7.17. Only valid if
+ the device is not a PM.
-id
+ pio_mode: (RO) Transfer modes supported by the device when
+ in PIO mode. Mostly used by PATA device.
- Cached result of IDENTIFY command, as described in ATA8 7.16 and 7.17.
- Only valid if the device is not a PM.
+ xfer_mode: (RO) Current transfer mode
-gscr
+ dma_mode: (RO) Transfer modes supported by the device when
+ in DMA mode. Mostly used by PATA device.
- Cached result of the dump of PM GSCR register.
- Valid registers are:
- 0: SATA_PMP_GSCR_PROD_ID,
- 1: SATA_PMP_GSCR_REV,
- 2: SATA_PMP_GSCR_PORT_INFO,
- 32: SATA_PMP_GSCR_ERROR,
- 33: SATA_PMP_GSCR_ERROR_EN,
- 64: SATA_PMP_GSCR_FEAT,
- 96: SATA_PMP_GSCR_FEAT_EN,
- 130: SATA_PMP_GSCR_SII_GPIO
- Only valid if the device is a PM.
+ class: (RO) Device class. Can be "ata" for disk,
+ "atapi" for packet device, "pmp" for PM, or
+ "none" if no device was found behind the link.
-trim
- Shows the DSM TRIM mode currently used by the device. Valid
- values are:
- unsupported: Drive does not support DSM TRIM
- unqueued: Drive supports unqueued DSM TRIM only
- queued: Drive supports queued DSM TRIM
- forced_unqueued: Drive's queued DSM support is known to be
- buggy and only unqueued TRIM commands
- are sent
+What: /sys/class/ata_device/devX[.Y].Z/trim
+Date: May, 2015
+KernelVersion: v4.10
+Contact: Gwendal Grignou <gwendal@chromium.org>
+Description:
+ (RO) Shows the DSM TRIM mode currently used by the device. Valid
+ values are:
-spdn_cnt
+ unsupported: Drive does not support DSM TRIM
- Number of time libata decided to lower the speed of link due to errors.
+ unqueued: Drive supports unqueued DSM TRIM only
-ering
+ queued: Drive supports queued DSM TRIM
- Formatted output of the error ring of the device.
+ forced_unqueued: Drive's queued DSM support is known to
+ be buggy and only unqueued TRIM commands
+ are sent
--- /dev/null
+What: /sys/block/*/device/sw_activity
+Date: Jun, 2008
+KernelVersion: v2.6.27
+Contact: linux-ide@vger.kernel.org
+Description:
+ (RW) Used by drivers which support software controlled activity
+ LEDs.
+
+ It has the following valid values:
+
+ 0 OFF - the LED is not activated on activity
+ 1 BLINK_ON - the LED blinks on every 10ms when activity is
+ detected.
+ 2 BLINK_OFF - the LED is on when idle, and blinks off
+ every 10ms when activity is detected.
+
+ Note that the user must turn sw_activity OFF it they wish to
+ control the activity LED via the em_message file.
+
+
+What: /sys/block/*/device/unload_heads
+Date: Sep, 2008
+KernelVersion: v2.6.28
+Contact: linux-ide@vger.kernel.org
+Description:
+ (RW) Hard disk shock protection
+
+ Writing an integer value to this file will take the heads of the
+ respective drive off the platter and block all I/O operations
+ for the specified number of milliseconds.
+
+ - If the device does not support the unload heads feature,
+ access is denied with -EOPNOTSUPP.
+ - The maximal value accepted for a timeout is 30000
+ milliseconds.
+ - A previously set timeout can be cancelled and disk can resume
+ normal operation immediately by specifying a timeout of 0.
+ - Some hard drives only comply with an earlier version of the
+ ATA standard, but support the unload feature nonetheless.
+ There is no safe way Linux can detect these devices, so this
+ is not enabled by default. If it is known that your device
+ does support the unload feature, then you can tell the kernel
+ to enable it by writing -1. It can be disabled again by
+ writing -2.
+ - Values below -2 are rejected with -EINVAL
+
+ For more information, see
+ Documentation/laptops/disk-shock-protection.txt
+
+
+What: /sys/block/*/device/ncq_prio_enable
+Date: Oct, 2016
+KernelVersion: v4.10
+Contact: linux-ide@vger.kernel.org
+Description:
+ (RW) Write to the file to turn on or off the SATA ncq (native
+ command queueing) support. By default this feature is turned
+ off.
the direct i/o path to physical devices. This setting is
controller wide, affecting all configured logical drives on the
controller. This file is readable and writable.
+
+What: /sys/class/scsi_host/hostX/link_power_management_policy
+Date: Oct, 2007
+KernelVersion: v2.6.24
+Contact: linux-ide@vger.kernel.org
+Description:
+ (RW) This parameter allows the user to read and set the link
+ (interface) power management.
+
+ There are four possible options:
+
+ min_power: Tell the controller to try to make the link use the
+ least possible power when possible. This may sacrifice some
+ performance due to increased latency when coming out of lower
+ power states.
+
+ max_performance: Generally, this means no power management.
+ Tell the controller to have performance be a priority over power
+ management.
+
+ medium_power: Tell the controller to enter a lower power state
+ when possible, but do not enter the lowest power state, thus
+ improving latency over min_power setting.
+
+ med_power_with_dipm: Identical to the existing medium_power
+ setting except that it enables dipm (device initiated power
+ management) on top, which makes it match the Windows IRST (Intel
+ Rapid Storage Technology) driver settings. This setting is also
+ close to min_power, except that:
+ a) It does not use host-initiated slumber mode, but it does
+ allow device-initiated slumber
+ b) It does not enable low power device sleep mode (DevSlp).
+
+What: /sys/class/scsi_host/hostX/em_message
+What: /sys/class/scsi_host/hostX/em_message_type
+Date: Jun, 2008
+KernelVersion: v2.6.27
+Contact: linux-ide@vger.kernel.org
+Description:
+ em_message: (RW) Enclosure management support. For the LED
+ protocol, writes and reads correspond to the LED message format
+ as defined in the AHCI spec.
+
+ The user must turn sw_activity (under /sys/block/*/device/) OFF
+ it they wish to control the activity LED via the em_message
+ file.
+
+ em_message_type: (RO) Displays the current enclosure management
+ protocol that is being used by the driver (for eg. LED, SAF-TE,
+ SES-2, SGPIO etc).
+
+What: /sys/class/scsi_host/hostX/ahci_port_cmd
+What: /sys/class/scsi_host/hostX/ahci_host_caps
+What: /sys/class/scsi_host/hostX/ahci_host_cap2
+Date: Mar, 2010
+KernelVersion: v2.6.35
+Contact: linux-ide@vger.kernel.org
+Description:
+ [to be documented]
+
+What: /sys/class/scsi_host/hostX/ahci_host_version
+Date: Mar, 2010
+KernelVersion: v2.6.35
+Contact: linux-ide@vger.kernel.org
+Description:
+ (RO) Display the version of the AHCI spec implemented by the
+ host.
+
+What: /sys/class/scsi_host/hostX/em_buffer
+Date: Apr, 2010
+KernelVersion: v2.6.35
+Contact: linux-ide@vger.kernel.org
+Description:
+ (RW) Allows access to AHCI EM (enclosure management) buffer
+ directly if the host supports EM.
+
+ For eg. the AHCI driver supports SGPIO EM messages but the
+ SATA/AHCI specs do not define the SGPIO message format of the EM
+ buffer. Different hardware(HW) vendors may have different
+ definitions. With the em_buffer attribute, this issue can be
+ solved by allowing HW vendors to provide userland drivers and
+ tools for their SGPIO initiators.
+
+What: /sys/class/scsi_host/hostX/em_message_supported
+Date: Oct, 2009
+KernelVersion: v2.6.39
+Contact: linux-ide@vger.kernel.org
+Description:
+ (RO) Displays supported enclosure management message types.
Associate an event fd to an AFU interrupt so that the user process
can be notified when the AFU sends an interrupt.
+OCXL_IOCTL_GET_METADATA:
+
+ Obtains configuration information from the card, such at the size of
+ MMIO areas, the AFU version, and the PASID for the current context.
+
mmap
----
The clang option "-fno-jump-tables" can be used to disable
switch table generation.
+ - For clang -target bpf, it is guaranteed that pointer or long /
+ unsigned long types will always have a width of 64 bit, no matter
+ whether underlying clang binary or default target (or kernel) is
+ 32 bit. However, when native clang target is used, then it will
+ compile these types based on the underlying architecture's conventions,
+ meaning in case of 32 bit architecture, pointer or long / unsigned
+ long types e.g. in BPF context structure will have width of 32 bit
+ while the BPF LLVM back end still operates in 64 bit. The native
+ target is mostly needed in tracing for the case of walking pt_regs
+ or other kernel structures where CPU's register width matters.
+ Otherwise, clang -target bpf is generally recommended.
+
You should use default target when:
- Your program includes a header file, e.g., ptrace.h, which eventually
- ddc: phandle to the hdmi ddc node
- phy: phandle to the hdmi phy node
- samsung,syscon-phandle: phandle for system controller node for PMU.
+- #sound-dai-cells: should be 0.
Required properties for Exynos 4210, 4212, 5420 and 5433:
- clocks: list of clock IDs from SoC clock driver.
interrupts.
Optional properties:
-- clocks: Optional reference to the clock used by the XOR engine.
+- clocks: Optional reference to the clocks used by the XOR engine.
+- clock-names: mandatory if there is a second clock, in this case the
+ name must be "core" for the first clock and "reg" for the second
+ one
+
Example:
compatible = "marvell,mv88e6085";
reg = <0>;
reset-gpios = <&gpio5 1 GPIO_ACTIVE_LOW>;
- };
- mdio {
- #address-cells = <1>;
- #size-cells = <0>;
- switch1phy0: switch1phy0@0 {
- reg = <0>;
- interrupt-parent = <&switch0>;
- interrupts = <0 IRQ_TYPE_LEVEL_HIGH>;
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ switch1phy0: switch1phy0@0 {
+ reg = <0>;
+ interrupt-parent = <&switch0>;
+ interrupts = <0 IRQ_TYPE_LEVEL_HIGH>;
+ };
};
};
};
compatible = "marvell,mv88e6390";
reg = <0>;
reset-gpios = <&gpio5 1 GPIO_ACTIVE_LOW>;
- };
- mdio {
- #address-cells = <1>;
- #size-cells = <0>;
- switch1phy0: switch1phy0@0 {
- reg = <0>;
- interrupt-parent = <&switch0>;
- interrupts = <0 IRQ_TYPE_LEVEL_HIGH>;
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ switch1phy0: switch1phy0@0 {
+ reg = <0>;
+ interrupt-parent = <&switch0>;
+ interrupts = <0 IRQ_TYPE_LEVEL_HIGH>;
+ };
};
- };
- mdio1 {
- compatible = "marvell,mv88e6xxx-mdio-external";
- #address-cells = <1>;
- #size-cells = <0>;
- switch1phy9: switch1phy0@9 {
- reg = <9>;
+ mdio1 {
+ compatible = "marvell,mv88e6xxx-mdio-external";
+ #address-cells = <1>;
+ #size-cells = <0>;
+ switch1phy9: switch1phy0@9 {
+ reg = <9>;
+ };
};
};
};
--- /dev/null
+* NI XGE Ethernet controller
+
+Required properties:
+- compatible: Should be "ni,xge-enet-2.00"
+- reg: Address and length of the register set for the device
+- interrupts: Should contain tx and rx interrupt
+- interrupt-names: Should be "rx" and "tx"
+- phy-mode: See ethernet.txt file in the same directory.
+- phy-handle: See ethernet.txt file in the same directory.
+- nvmem-cells: Phandle of nvmem cell containing the MAC address
+- nvmem-cell-names: Should be "address"
+
+Examples (10G generic PHY):
+ nixge0: ethernet@40000000 {
+ compatible = "ni,xge-enet-2.00";
+ reg = <0x40000000 0x6000>;
+
+ nvmem-cells = <ð1_addr>;
+ nvmem-cell-names = "address";
+
+ interrupts = <0 29 IRQ_TYPE_LEVEL_HIGH>, <0 30 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "rx", "tx";
+ interrupt-parent = <&intc>;
+
+ phy-mode = "xgmii";
+ phy-handle = <ðernet_phy1>;
+
+ ethernet_phy1: ethernet-phy@4 {
+ compatible = "ethernet-phy-ieee802.3-c45";
+ reg = <4>;
+ };
+ };
SoC-specific version corresponding to the platform first followed by
the generic version.
-- reg: offset and length of (1) the register block and (2) the stream buffer.
+- reg: Offset and length of (1) the register block and (2) the stream buffer.
+ The region for the register block is mandatory.
+ The region for the stream buffer is optional, as it is only present on
+ R-Car Gen2 and RZ/G1 SoCs, and on R-Car H3 (R8A7795), M3-W (R8A7796),
+ and M3-N (R8A77965).
- interrupts: A list of interrupt-specifiers, one for each entry in
interrupt-names.
If interrupt-names is not present, an interrupt specifier
- "socionext,uniphier-pxs2-ave4" : for PXs2 SoC
- "socionext,uniphier-ld11-ave4" : for LD11 SoC
- "socionext,uniphier-ld20-ave4" : for LD20 SoC
+ - "socionext,uniphier-pxs3-ave4" : for PXs3 SoC
- reg: Address where registers are mapped and size of region.
- interrupts: Should contain the MAC interrupt.
- phy-mode: See ethernet.txt in the same directory. Allow to choose
configured in FS mode;
- "st,stm32f4x9-hsotg": The DWC2 USB HS controller instance in STM32F4x9 SoCs
configured in HS mode;
- - "st,stm32f7xx-hsotg": The DWC2 USB HS controller instance in STM32F7xx SoCs
+ - "st,stm32f7-hsotg": The DWC2 USB HS controller instance in STM32F7 SoCs
configured in HS mode;
- reg : Should contain 1 register range (address and length)
- interrupts : Should contain 1 interrupt
- compatible: Must contain one of the following:
- "renesas,r8a7795-usb3-peri"
- "renesas,r8a7796-usb3-peri"
+ - "renesas,r8a77965-usb3-peri"
- "renesas,rcar-gen3-usb3-peri" for a generic R-Car Gen3 compatible
device
- "renesas,usbhs-r8a7794" for r8a7794 (R-Car E2) compatible device
- "renesas,usbhs-r8a7795" for r8a7795 (R-Car H3) compatible device
- "renesas,usbhs-r8a7796" for r8a7796 (R-Car M3-W) compatible device
+ - "renesas,usbhs-r8a77965" for r8a77965 (R-Car M3-N) compatible device
- "renesas,usbhs-r8a77995" for r8a77995 (R-Car D3) compatible device
- "renesas,usbhs-r7s72100" for r7s72100 (RZ/A1) compatible device
- "renesas,rcar-gen2-usbhs" for R-Car Gen2 or RZ/G1 compatible devices
- "renesas,xhci-r8a7793" for r8a7793 SoC
- "renesas,xhci-r8a7795" for r8a7795 SoC
- "renesas,xhci-r8a7796" for r8a7796 SoC
+ - "renesas,xhci-r8a77965" for r8a77965 SoC
- "renesas,rcar-gen2-xhci" for a generic R-Car Gen2 or RZ/G1 compatible
device
- "renesas,rcar-gen3-xhci" for a generic R-Car Gen3 compatible device
--- /dev/null
+Intel(R) Ethernet Connection E800 Series Linux Driver
+===================================================================
+
+Intel ice Linux driver.
+Copyright(c) 2018 Intel Corporation.
+
+Contents
+========
+- Enabling the driver
+- Support
+
+The driver in this release supports Intel's E800 Series of products. For
+more information, visit Intel's support page at http://support.intel.com.
+
+Enabling the driver
+===================
+
+The driver is enabled via the standard kernel configuration system,
+using the make command:
+
+ Make oldconfig/silentoldconfig/menuconfig/etc.
+
+The driver is located in the menu structure at:
+
+ -> Device Drivers
+ -> Network device support (NETDEVICES [=y])
+ -> Ethernet driver support
+ -> Intel devices
+ -> Intel(R) Ethernet Connection E800 Series Support
+
+Support
+=======
+
+For general information, go to the Intel support website at:
+
+ http://support.intel.com
+
+If an issue is identified with the released source code, please email
+the maintainer listed in the MAINTAINERS file.
--- /dev/null
+Net DIM - Generic Network Dynamic Interrupt Moderation
+======================================================
+
+Author:
+ Tal Gilboa <talgi@mellanox.com>
+
+
+Contents
+=========
+
+- Assumptions
+- Introduction
+- The Net DIM Algorithm
+- Registering a Network Device to DIM
+- Example
+
+Part 0: Assumptions
+======================
+
+This document assumes the reader has basic knowledge in network drivers
+and in general interrupt moderation.
+
+
+Part I: Introduction
+======================
+
+Dynamic Interrupt Moderation (DIM) (in networking) refers to changing the
+interrupt moderation configuration of a channel in order to optimize packet
+processing. The mechanism includes an algorithm which decides if and how to
+change moderation parameters for a channel, usually by performing an analysis on
+runtime data sampled from the system. Net DIM is such a mechanism. In each
+iteration of the algorithm, it analyses a given sample of the data, compares it
+to the previous sample and if required, it can decide to change some of the
+interrupt moderation configuration fields. The data sample is composed of data
+bandwidth, the number of packets and the number of events. The time between
+samples is also measured. Net DIM compares the current and the previous data and
+returns an adjusted interrupt moderation configuration object. In some cases,
+the algorithm might decide not to change anything. The configuration fields are
+the minimum duration (microseconds) allowed between events and the maximum
+number of wanted packets per event. The Net DIM algorithm ascribes importance to
+increase bandwidth over reducing interrupt rate.
+
+
+Part II: The Net DIM Algorithm
+===============================
+
+Each iteration of the Net DIM algorithm follows these steps:
+1. Calculates new data sample.
+2. Compares it to previous sample.
+3. Makes a decision - suggests interrupt moderation configuration fields.
+4. Applies a schedule work function, which applies suggested configuration.
+
+The first two steps are straightforward, both the new and the previous data are
+supplied by the driver registered to Net DIM. The previous data is the new data
+supplied to the previous iteration. The comparison step checks the difference
+between the new and previous data and decides on the result of the last step.
+A step would result as "better" if bandwidth increases and as "worse" if
+bandwidth reduces. If there is no change in bandwidth, the packet rate is
+compared in a similar fashion - increase == "better" and decrease == "worse".
+In case there is no change in the packet rate as well, the interrupt rate is
+compared. Here the algorithm tries to optimize for lower interrupt rate so an
+increase in the interrupt rate is considered "worse" and a decrease is
+considered "better". Step #2 has an optimization for avoiding false results: it
+only considers a difference between samples as valid if it is greater than a
+certain percentage. Also, since Net DIM does not measure anything by itself, it
+assumes the data provided by the driver is valid.
+
+Step #3 decides on the suggested configuration based on the result from step #2
+and the internal state of the algorithm. The states reflect the "direction" of
+the algorithm: is it going left (reducing moderation), right (increasing
+moderation) or standing still. Another optimization is that if a decision
+to stay still is made multiple times, the interval between iterations of the
+algorithm would increase in order to reduce calculation overhead. Also, after
+"parking" on one of the most left or most right decisions, the algorithm may
+decide to verify this decision by taking a step in the other direction. This is
+done in order to avoid getting stuck in a "deep sleep" scenario. Once a
+decision is made, an interrupt moderation configuration is selected from
+the predefined profiles.
+
+The last step is to notify the registered driver that it should apply the
+suggested configuration. This is done by scheduling a work function, defined by
+the Net DIM API and provided by the registered driver.
+
+As you can see, Net DIM itself does not actively interact with the system. It
+would have trouble making the correct decisions if the wrong data is supplied to
+it and it would be useless if the work function would not apply the suggested
+configuration. This does, however, allow the registered driver some room for
+manoeuvre as it may provide partial data or ignore the algorithm suggestion
+under some conditions.
+
+
+Part III: Registering a Network Device to DIM
+==============================================
+
+Net DIM API exposes the main function net_dim(struct net_dim *dim,
+struct net_dim_sample end_sample). This function is the entry point to the Net
+DIM algorithm and has to be called every time the driver would like to check if
+it should change interrupt moderation parameters. The driver should provide two
+data structures: struct net_dim and struct net_dim_sample. Struct net_dim
+describes the state of DIM for a specific object (RX queue, TX queue,
+other queues, etc.). This includes the current selected profile, previous data
+samples, the callback function provided by the driver and more.
+Struct net_dim_sample describes a data sample, which will be compared to the
+data sample stored in struct net_dim in order to decide on the algorithm's next
+step. The sample should include bytes, packets and interrupts, measured by
+the driver.
+
+In order to use Net DIM from a networking driver, the driver needs to call the
+main net_dim() function. The recommended method is to call net_dim() on each
+interrupt. Since Net DIM has a built-in moderation and it might decide to skip
+iterations under certain conditions, there is no need to moderate the net_dim()
+calls as well. As mentioned above, the driver needs to provide an object of type
+struct net_dim to the net_dim() function call. It is advised for each entity
+using Net DIM to hold a struct net_dim as part of its data structure and use it
+as the main Net DIM API object. The struct net_dim_sample should hold the latest
+bytes, packets and interrupts count. No need to perform any calculations, just
+include the raw data.
+
+The net_dim() call itself does not return anything. Instead Net DIM relies on
+the driver to provide a callback function, which is called when the algorithm
+decides to make a change in the interrupt moderation parameters. This callback
+will be scheduled and run in a separate thread in order not to add overhead to
+the data flow. After the work is done, Net DIM algorithm needs to be set to
+the proper state in order to move to the next iteration.
+
+
+Part IV: Example
+=================
+
+The following code demonstrates how to register a driver to Net DIM. The actual
+usage is not complete but it should make the outline of the usage clear.
+
+my_driver.c:
+
+#include <linux/net_dim.h>
+
+/* Callback for net DIM to schedule on a decision to change moderation */
+void my_driver_do_dim_work(struct work_struct *work)
+{
+ /* Get struct net_dim from struct work_struct */
+ struct net_dim *dim = container_of(work, struct net_dim,
+ work);
+ /* Do interrupt moderation related stuff */
+ ...
+
+ /* Signal net DIM work is done and it should move to next iteration */
+ dim->state = NET_DIM_START_MEASURE;
+}
+
+/* My driver's interrupt handler */
+int my_driver_handle_interrupt(struct my_driver_entity *my_entity, ...)
+{
+ ...
+ /* A struct to hold current measured data */
+ struct net_dim_sample dim_sample;
+ ...
+ /* Initiate data sample struct with current data */
+ net_dim_sample(my_entity->events,
+ my_entity->packets,
+ my_entity->bytes,
+ &dim_sample);
+ /* Call net DIM */
+ net_dim(&my_entity->dim, dim_sample);
+ ...
+}
+
+/* My entity's initialization function (my_entity was already allocated) */
+int my_driver_init_my_entity(struct my_driver_entity *my_entity, ...)
+{
+ ...
+ /* Initiate struct work_struct with my driver's callback function */
+ INIT_WORK(&my_entity->dim.work, my_driver_do_dim_work);
+ ...
+}
TCP segmentation allows a device to segment a single frame into multiple
frames with a data payload size specified in skb_shinfo()->gso_size.
-When TCP segmentation requested the bit for either SKB_GSO_TCP or
-SKB_GSO_TCP6 should be set in skb_shinfo()->gso_type and
+When TCP segmentation requested the bit for either SKB_GSO_TCPV4 or
+SKB_GSO_TCPV6 should be set in skb_shinfo()->gso_type and
skb_shinfo()->gso_size should be set to a non-zero value.
TCP segmentation is dependent on support for the use of partial checksum
Therefore, any code in the core networking stack must be aware of the
possibility that gso_size will be GSO_BY_FRAGS and handle that case
-appropriately. (For size checks, the skb_gso_validate_*_len family of
-helpers do this automatically.)
+appropriately.
+
+There are some helpers to make this easier:
+
+ - skb_is_gso(skb) && skb_is_gso_sctp(skb) is the best way to see if
+ an skb is an SCTP GSO skb.
+
+ - For size checks, the skb_gso_validate_*_len family of helpers correctly
+ considers GSO_BY_FRAGS.
+
+ - For manipulating packets, skb_increase_gso_size and skb_decrease_gso_size
+ will check for GSO_BY_FRAGS and WARN if asked to manipulate these skbs.
This also affects drivers with the NETIF_F_FRAGLIST & NETIF_F_GSO_SCTP bits
set. Note also that NETIF_F_GSO_SCTP is included in NETIF_F_GSO_SOFTWARE.
setsockopt(sock, SOL_TLS, TLS_TX, &crypto_info, sizeof(crypto_info));
+Transmit and receive are set separately, but the setup is the same, using either
+TLS_TX or TLS_RX.
+
Sending TLS application data
----------------------------
-ENOMEM and some data was left on the socket buffer from a previous
call using MSG_MORE, the MSG_MORE data is left on the socket buffer.
+Receiving TLS application data
+------------------------------
+
+After setting the TLS_RX socket option, all recv family socket calls
+are decrypted using TLS parameters provided. A full TLS record must
+be received before decryption can happen.
+
+ char buffer[16384];
+ recv(sock, buffer, 16384);
+
+Received data is decrypted directly in to the user buffer if it is
+large enough, and no additional allocations occur. If the userspace
+buffer is too small, data is decrypted in the kernel and copied to
+userspace.
+
+EINVAL is returned if the TLS version in the received message does not
+match the version passed in setsockopt.
+
+EMSGSIZE is returned if the received message is too big.
+
+EBADMSG is returned if decryption failed for any other reason.
+
Send TLS control messages
-------------------------
Control message data should be provided unencrypted, and will be
encrypted by the kernel.
+Receiving TLS control messages
+------------------------------
+
+TLS control messages are passed in the userspace buffer, with message
+type passed via cmsg. If no cmsg buffer is provided, an error is
+returned if a control message is received. Data messages may be
+received without a cmsg buffer set.
+
+ char buffer[16384];
+ char cmsg[CMSG_SPACE(sizeof(unsigned char))];
+ struct msghdr msg = {0};
+ msg.msg_control = cmsg;
+ msg.msg_controllen = sizeof(cmsg);
+
+ struct iovec msg_iov;
+ msg_iov.iov_base = buffer;
+ msg_iov.iov_len = 16384;
+
+ msg.msg_iov = &msg_iov;
+ msg.msg_iovlen = 1;
+
+ int ret = recvmsg(sock, &msg, 0 /* flags */);
+
+ struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
+ if (cmsg->cmsg_level == SOL_TLS &&
+ cmsg->cmsg_type == TLS_GET_RECORD_TYPE) {
+ int record_type = *((unsigned char *)CMSG_DATA(cmsg));
+ // Do something with record_type, and control message data in
+ // buffer.
+ //
+ // Note that record_type may be == to application data (23).
+ } else {
+ // Buffer contains application data.
+ }
+
+recv will never return data from mixed types of TLS records.
+
Integrating in to userspace TLS library
---------------------------------------
A patchset to OpenSSL to use ktls as the record layer is here:
-https://github.com/Mellanox/tls-openssl
+https://github.com/Mellanox/openssl/commits/tls_rx2
An example of calling send directly after a handshake using
gnutls. Since it doesn't implement a full record layer, control
messages are not supported:
-https://github.com/Mellanox/tls-af_ktls_tool
+https://github.com/ktls/af_ktls-tool/commits/RX
- Adjust clock frequency
+ Ancillary clock features
- - One short or periodic alarms, with signal delivery to user program
- Time stamp external events
- Period output signals configurable from user space
- Synchronization of the Linux system time via the PPS subsystem
User space programs may control the clock using standardized
ioctls. A program may query, enable, configure, and disable the
ancillary clock features. User space can receive time stamped
- events via blocking read() and poll(). One shot and periodic
- signals may be configured via the POSIX timer_settime() system
- call.
+ events via blocking read() and poll().
** Writing clock drivers
from docutils import nodes, statemachine
from docutils.statemachine import ViewList
-from docutils.parsers.rst import directives
-from sphinx.util.compat import Directive
+from docutils.parsers.rst import directives, Directive
from sphinx.ext.autodoc import AutodocReporter
__version__ = '1.0'
F: Documentation/networking/ixgbevf.txt
F: Documentation/networking/i40e.txt
F: Documentation/networking/i40evf.txt
+F: Documentation/networking/ice.txt
F: drivers/net/ethernet/intel/
F: drivers/net/ethernet/intel/*/
F: include/linux/avf/virtchnl.h
F: include/linux/nvmem-consumer.h
F: include/linux/nvmem-provider.h
+NXP SGTL5000 DRIVER
+M: Fabio Estevam <fabio.estevam@nxp.com>
+L: alsa-devel@alsa-project.org (moderated for non-subscribers)
+S: Maintained
+F: Documentation/devicetree/bindings/sound/sgtl5000.txt
+F: sound/soc/codecs/sgtl5000*
+
NXP TDA998X DRM DRIVER
M: Russell King <linux@armlinux.org.uk>
S: Supported
F: include/linux/oprofile.h
ORACLE CLUSTER FILESYSTEM 2 (OCFS2)
-M: Mark Fasheh <mfasheh@versity.com>
+M: Mark Fasheh <mark@fasheh.com>
M: Joel Becker <jlbec@evilplan.org>
L: ocfs2-devel@oss.oracle.com (moderated for non-subscribers)
W: http://ocfs2.wiki.kernel.org
PER-CPU MEMORY ALLOCATOR
M: Tejun Heo <tj@kernel.org>
M: Christoph Lameter <cl@linux.com>
+M: Dennis Zhou <dennisszhou@gmail.com>
T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git
S: Maintained
F: include/linux/percpu*.h
L: alsa-devel@alsa-project.org (moderated for non-subscribers)
S: Supported
F: sound/soc/samsung/
+F: Documentation/devicetree/bindings/sound/samsung*
SAMSUNG EXYNOS PSEUDO RANDOM NUMBER GENERATOR (RNG) DRIVER
M: Krzysztof Kozlowski <krzk@kernel.org>
VERSION = 4
PATCHLEVEL = 16
SUBLEVEL = 0
-EXTRAVERSION = -rc4
+EXTRAVERSION = -rc6
NAME = Fearless Coyote
# *DOCUMENTATION*
# disable invalid "can't wrap" optimizations for signed / pointers
KBUILD_CFLAGS += $(call cc-option,-fno-strict-overflow)
+# clang sets -fmerge-all-constants by default as optimization, but this
+# is non-conforming behavior for C and in fact breaks the kernel, so we
+# need to disable it here generally.
+KBUILD_CFLAGS += $(call cc-option,-fno-merge-all-constants)
+
+# for gcc -fno-merge-all-constants disables everything, but it is fine
+# to have actual conforming behavior enabled.
+KBUILD_CFLAGS += $(call cc-option,-fmerge-constants)
+
# Make sure -fstack-check isn't enabled (like gentoo apparently did)
KBUILD_CFLAGS += $(call cc-option,-fno-stack-check,)
case PSCI_CONDUIT_HVC:
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
- if (res.a0)
+ if ((int)res.a0 < 0)
return 0;
cb = call_hvc_arch_workaround_1;
smccc_start = __smccc_workaround_1_hvc_start;
case PSCI_CONDUIT_SMC:
arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
- if (res.a0)
+ if ((int)res.a0 < 0)
return 0;
cb = call_smc_arch_workaround_1;
smccc_start = __smccc_workaround_1_smc_start;
{
int ret = 0;
- vcpu_load(vcpu);
-
trace_kvm_set_guest_debug(vcpu, dbg->control);
if (dbg->control & ~KVM_GUESTDBG_VALID_MASK) {
}
out:
- vcpu_put(vcpu);
return ret;
}
* The following mapping attributes may be updated in live
* kernel mappings without the need for break-before-make.
*/
- static const pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE;
+ static const pteval_t mask = PTE_PXN | PTE_RDONLY | PTE_WRITE | PTE_NG;
/* creating or taking down mappings is always safe */
if (old == 0 || new == 0)
if ((old | new) & PTE_CONT)
return false;
- /* Transitioning from Global to Non-Global is safe */
- if (((old ^ new) == PTE_NG) && (new & PTE_NG))
- return true;
+ /* Transitioning from Non-Global to Global is unsafe */
+ if (old & ~new & PTE_NG)
+ return false;
return ((old ^ new) & ~mask) == 0;
}
pmd_clear(pmdp);
return 1;
}
+
+int pud_free_pmd_page(pud_t *pud)
+{
+ return pud_none(*pud);
+}
+
+int pmd_free_pte_page(pmd_t *pmd)
+{
+ return pmd_none(*pmd);
+}
#ifndef __H8300_BYTEORDER_H__
#define __H8300_BYTEORDER_H__
-#define __BIG_ENDIAN __ORDER_BIG_ENDIAN__
#include <linux/byteorder/big_endian.h>
#endif
select HAVE_FTRACE_MCOUNT_RECORD
select HAVE_FUNCTION_GRAPH_TRACER
select HAVE_FUNCTION_TRACER
+ select NO_BOOTMEM
select HAVE_MEMBLOCK
select HAVE_MEMBLOCK_NODE_MAP
select HAVE_OPROFILE
config OPT_LIB_FUNCTION
bool "Optimalized lib function"
- depends on CPU_LITTLE_ENDIAN
default y
help
Allows turn on optimalized library function (memcpy and memmove).
config OPT_LIB_ASM
bool "Optimalized lib function ASM"
depends on OPT_LIB_FUNCTION && (XILINX_MICROBLAZE0_USE_BARREL = 1)
+ depends on CPU_BIG_ENDIAN
default n
help
Allows turn on optimalized library function (memcpy and memmove).
void machine_halt(void);
void machine_power_off(void);
-extern void *alloc_maybe_bootmem(size_t size, gfp_t mask);
extern void *zalloc_maybe_bootmem(size_t size, gfp_t mask);
# endif /* __ASSEMBLY__ */
* between mem locations with size of xfer spec'd in bytes
*/
-#ifdef __MICROBLAZEEL__
-#error Microblaze LE not support ASM optimized lib func. Disable OPT_LIB_ASM.
-#endif
-
#include <linux/linkage.h>
.text
.globl memcpy
#ifndef CONFIG_MMU
unsigned int __page_offset;
EXPORT_SYMBOL(__page_offset);
-
-#else
-static int init_bootmem_done;
#endif /* CONFIG_MMU */
char *klimit = _end;
void __init setup_memory(void)
{
- unsigned long map_size;
struct memblock_region *reg;
#ifndef CONFIG_MMU
pr_info("%s: max_low_pfn: %#lx\n", __func__, max_low_pfn);
pr_info("%s: max_pfn: %#lx\n", __func__, max_pfn);
- /*
- * Find an area to use for the bootmem bitmap.
- * We look for the first area which is at least
- * 128kB in length (128kB is enough for a bitmap
- * for 4GB of memory, using 4kB pages), plus 1 page
- * (in case the address isn't page-aligned).
- */
- map_size = init_bootmem_node(NODE_DATA(0),
- PFN_UP(TOPHYS((u32)klimit)), min_low_pfn, max_low_pfn);
- memblock_reserve(PFN_UP(TOPHYS((u32)klimit)) << PAGE_SHIFT, map_size);
-
/* Add active regions with valid PFNs */
for_each_memblock(memory, reg) {
unsigned long start_pfn, end_pfn;
&memblock.memory, 0);
}
- /* free bootmem is whole main memory */
- free_bootmem_with_active_regions(0, max_low_pfn);
-
- /* reserve allocate blocks */
- for_each_memblock(reserved, reg) {
- unsigned long top = reg->base + reg->size - 1;
-
- pr_debug("reserved - 0x%08x-0x%08x, %lx, %lx\n",
- (u32) reg->base, (u32) reg->size, top,
- memory_start + lowmem_size - 1);
-
- if (top <= (memory_start + lowmem_size - 1)) {
- reserve_bootmem(reg->base, reg->size, BOOTMEM_DEFAULT);
- } else if (reg->base < (memory_start + lowmem_size - 1)) {
- unsigned long trunc_size = memory_start + lowmem_size -
- reg->base;
- reserve_bootmem(reg->base, trunc_size, BOOTMEM_DEFAULT);
- }
- }
-
/* XXX need to clip this if using highmem? */
sparse_memory_present_with_active_regions(0);
-#ifdef CONFIG_MMU
- init_bootmem_done = 1;
-#endif
paging_init();
}
/* This is only called until mem_init is done. */
void __init *early_get_page(void)
{
- void *p;
- if (init_bootmem_done) {
- p = alloc_bootmem_pages(PAGE_SIZE);
- } else {
- /*
- * Mem start + kernel_tlb -> here is limit
- * because of mem mapping from head.S
- */
- p = __va(memblock_alloc_base(PAGE_SIZE, PAGE_SIZE,
- memory_start + kernel_tlb));
- }
- return p;
+ /*
+ * Mem start + kernel_tlb -> here is limit
+ * because of mem mapping from head.S
+ */
+ return __va(memblock_alloc_base(PAGE_SIZE, PAGE_SIZE,
+ memory_start + kernel_tlb));
}
#endif /* CONFIG_MMU */
-void * __ref alloc_maybe_bootmem(size_t size, gfp_t mask)
-{
- if (mem_init_done)
- return kmalloc(size, mask);
- else
- return alloc_bootmem(size);
-}
-
void * __ref zalloc_maybe_bootmem(size_t size, gfp_t mask)
{
void *p;
}
board_data = kzalloc(BOARD_CONFIG_BUFSZ, GFP_KERNEL);
+ if (!board_data)
+ goto error;
ath25_board.config = (struct ath25_boarddata *)board_data;
memcpy_fromio(board_data, bcfg, 0x100);
if (broken_boarddata) {
}
host_data = kzalloc(sizeof(*host_data), GFP_KERNEL);
+ if (!host_data)
+ return -ENOMEM;
raw_spin_lock_init(&host_data->lock);
addr = of_get_address(ciu_node, 0, NULL, NULL);
return;
}
- if (request_irq(IPI0_IRQ, bmips_ipi_interrupt, IRQF_PERCPU,
- "smp_ipi0", NULL))
+ if (request_irq(IPI0_IRQ, bmips_ipi_interrupt,
+ IRQF_PERCPU | IRQF_NO_SUSPEND, "smp_ipi0", NULL))
panic("Can't request IPI0 interrupt");
- if (request_irq(IPI1_IRQ, bmips_ipi_interrupt, IRQF_PERCPU,
- "smp_ipi1", NULL))
+ if (request_irq(IPI1_IRQ, bmips_ipi_interrupt,
+ IRQF_PERCPU | IRQF_NO_SUSPEND, "smp_ipi1", NULL))
panic("Can't request IPI1 interrupt");
}
config LEMOTE_FULOONG2E
bool "Lemote Fuloong(2e) mini-PC"
select ARCH_SPARSEMEM_ENABLE
+ select ARCH_MIGHT_HAVE_PC_PARPORT
+ select ARCH_MIGHT_HAVE_PC_SERIO
select CEVT_R4K
select CSRC_R4K
select SYS_HAS_CPU_LOONGSON2E
config LEMOTE_MACH2F
bool "Lemote Loongson 2F family machines"
select ARCH_SPARSEMEM_ENABLE
+ select ARCH_MIGHT_HAVE_PC_PARPORT
+ select ARCH_MIGHT_HAVE_PC_SERIO
select BOARD_SCACHE
select BOOT_ELF32
select CEVT_R4K if ! MIPS_EXTERNAL_TIMER
config LOONGSON_MACH3X
bool "Generic Loongson 3 family machines"
select ARCH_SPARSEMEM_ENABLE
+ select ARCH_MIGHT_HAVE_PC_PARPORT
+ select ARCH_MIGHT_HAVE_PC_SERIO
select GENERIC_ISA_DMA_SUPPORT_BROKEN
select BOOT_ELF32
select BOARD_SCACHE
rp3440, etc. So, avoid it if the mm isn't too big. */
if ((!IS_ENABLED(CONFIG_SMP) || !arch_irqs_disabled()) &&
mm_total_size(mm) >= parisc_cache_flush_threshold) {
- flush_tlb_all();
+ if (mm->context)
+ flush_tlb_all();
flush_cache_all();
return;
}
pfn = pte_pfn(*ptep);
if (!pfn_valid(pfn))
continue;
+ if (unlikely(mm->context))
+ flush_tlb_page(vma, addr);
__flush_cache_page(vma, addr, PFN_PHYS(pfn));
}
}
void flush_cache_range(struct vm_area_struct *vma,
unsigned long start, unsigned long end)
{
+ pgd_t *pgd;
+ unsigned long addr;
+
if ((!IS_ENABLED(CONFIG_SMP) || !arch_irqs_disabled()) &&
end - start >= parisc_cache_flush_threshold) {
- flush_tlb_range(vma, start, end);
+ if (vma->vm_mm->context)
+ flush_tlb_range(vma, start, end);
flush_cache_all();
return;
}
- flush_user_dcache_range_asm(start, end);
- if (vma->vm_flags & VM_EXEC)
- flush_user_icache_range_asm(start, end);
- flush_tlb_range(vma, start, end);
+ if (vma->vm_mm->context == mfsp(3)) {
+ flush_user_dcache_range_asm(start, end);
+ if (vma->vm_flags & VM_EXEC)
+ flush_user_icache_range_asm(start, end);
+ flush_tlb_range(vma, start, end);
+ return;
+ }
+
+ pgd = vma->vm_mm->pgd;
+ for (addr = vma->vm_start; addr < vma->vm_end; addr += PAGE_SIZE) {
+ unsigned long pfn;
+ pte_t *ptep = get_ptep(pgd, addr);
+ if (!ptep)
+ continue;
+ pfn = pte_pfn(*ptep);
+ if (pfn_valid(pfn)) {
+ if (unlikely(vma->vm_mm->context))
+ flush_tlb_page(vma, addr);
+ __flush_cache_page(vma, addr, PFN_PHYS(pfn));
+ }
+ }
}
void
flush_cache_page(struct vm_area_struct *vma, unsigned long vmaddr, unsigned long pfn)
{
- BUG_ON(!vma->vm_mm->context);
-
if (pfn_valid(pfn)) {
- flush_tlb_page(vma, vmaddr);
+ if (likely(vma->vm_mm->context))
+ flush_tlb_page(vma, vmaddr);
__flush_cache_page(vma, vmaddr, PFN_PHYS(pfn));
}
}
libfdt := fdt.c fdt_ro.c fdt_wip.c fdt_sw.c fdt_rw.c fdt_strerror.c
libfdtheader := fdt.h libfdt.h libfdt_internal.h
-$(addprefix $(obj)/,$(libfdt) libfdt-wrapper.o simpleboot.o epapr.o opal.o): \
+$(addprefix $(obj)/,$(libfdt) libfdt-wrapper.o simpleboot.o epapr.o opal.o \
+ treeboot-akebono.o treeboot-currituck.o treeboot-iss4xx.o): \
$(addprefix $(obj)/,$(libfdtheader))
src-wlib-y := string.S crt0.S stdio.c decompress.c main.c \
.mmu = 0,
.hash_ext = 0,
.radix_ext = 0,
- .byte22 = 0,
},
/* option vector 6: IBM PAPR hints */
kmem_cache_free(kvm_pte_cache, ptep);
}
+/* Like pmd_huge() and pmd_large(), but works regardless of config options */
+static inline int pmd_is_leaf(pmd_t pmd)
+{
+ return !!(pmd_val(pmd) & _PAGE_PTE);
+}
+
static int kvmppc_create_pte(struct kvm *kvm, pte_t pte, unsigned long gpa,
unsigned int level, unsigned long mmu_seq)
{
else
new_pmd = pmd_alloc_one(kvm->mm, gpa);
- if (level == 0 && !(pmd && pmd_present(*pmd)))
+ if (level == 0 && !(pmd && pmd_present(*pmd) && !pmd_is_leaf(*pmd)))
new_ptep = kvmppc_pte_alloc();
/* Check if we might have been invalidated; let the guest retry if so */
new_pmd = NULL;
}
pmd = pmd_offset(pud, gpa);
- if (pmd_large(*pmd)) {
- /* Someone else has instantiated a large page here; retry */
- ret = -EAGAIN;
- goto out_unlock;
- }
- if (level == 1 && !pmd_none(*pmd)) {
+ if (pmd_is_leaf(*pmd)) {
+ unsigned long lgpa = gpa & PMD_MASK;
+
+ /*
+ * If we raced with another CPU which has just put
+ * a 2MB pte in after we saw a pte page, try again.
+ */
+ if (level == 0 && !new_ptep) {
+ ret = -EAGAIN;
+ goto out_unlock;
+ }
+ /* Valid 2MB page here already, remove it */
+ old = kvmppc_radix_update_pte(kvm, pmdp_ptep(pmd),
+ ~0UL, 0, lgpa, PMD_SHIFT);
+ kvmppc_radix_tlbie_page(kvm, lgpa, PMD_SHIFT);
+ if (old & _PAGE_DIRTY) {
+ unsigned long gfn = lgpa >> PAGE_SHIFT;
+ struct kvm_memory_slot *memslot;
+ memslot = gfn_to_memslot(kvm, gfn);
+ if (memslot && memslot->dirty_bitmap)
+ kvmppc_update_dirty_map(memslot,
+ gfn, PMD_SIZE);
+ }
+ } else if (level == 1 && !pmd_none(*pmd)) {
/*
* There's a page table page here, but we wanted
* to install a large page. Tell the caller and let
} else {
page = pages[0];
pfn = page_to_pfn(page);
- if (PageHuge(page)) {
- page = compound_head(page);
- pte_size <<= compound_order(page);
+ if (PageCompound(page)) {
+ pte_size <<= compound_order(compound_head(page));
/* See if we can insert a 2MB large-page PTE here */
if (pte_size >= PMD_SIZE &&
- (gpa & PMD_MASK & PAGE_MASK) ==
- (hva & PMD_MASK & PAGE_MASK)) {
+ (gpa & (PMD_SIZE - PAGE_SIZE)) ==
+ (hva & (PMD_SIZE - PAGE_SIZE))) {
level = 1;
pfn &= ~((PMD_SIZE >> PAGE_SHIFT) - 1);
}
}
/* See if we can provide write access */
if (writing) {
- /*
- * We assume gup_fast has set dirty on the host PTE.
- */
pgflags |= _PAGE_WRITE;
} else {
local_irq_save(flags);
ptep = find_current_mm_pte(current->mm->pgd,
hva, NULL, NULL);
- if (ptep && pte_write(*ptep) && pte_dirty(*ptep))
+ if (ptep && pte_write(*ptep))
pgflags |= _PAGE_WRITE;
local_irq_restore(flags);
}
pte = pfn_pte(pfn, __pgprot(pgflags));
ret = kvmppc_create_pte(kvm, pte, gpa, level, mmu_seq);
}
- if (ret == 0 || ret == -EAGAIN)
- ret = RESUME_GUEST;
if (page) {
- /*
- * We drop pages[0] here, not page because page might
- * have been set to the head page of a compound, but
- * we have to drop the reference on the correct tail
- * page to match the get inside gup()
- */
- put_page(pages[0]);
+ if (!ret && (pgflags & _PAGE_WRITE))
+ set_page_dirty_lock(page);
+ put_page(page);
}
+
+ if (ret == 0 || ret == -EAGAIN)
+ ret = RESUME_GUEST;
return ret;
}
continue;
pmd = pmd_offset(pud, 0);
for (im = 0; im < PTRS_PER_PMD; ++im, ++pmd) {
- if (pmd_huge(*pmd)) {
+ if (pmd_is_leaf(*pmd)) {
pmd_clear(pmd);
continue;
}
*/
trace_hardirqs_on();
- guest_enter();
+ guest_enter_irqoff();
srcu_idx = srcu_read_lock(&vc->kvm->srcu);
srcu_read_unlock(&vc->kvm->srcu, srcu_idx);
- guest_exit();
-
trace_hardirqs_off();
set_irq_happened(trap);
kvmppc_set_host_core(pcpu);
local_irq_enable();
+ guest_exit();
/* Let secondaries go back to the offline loop */
for (i = 0; i < controlled_threads; ++i) {
goto up_out;
psize = vma_kernel_pagesize(vma);
- porder = __ilog2(psize);
up_read(¤t->mm->mmap_sem);
/* We can handle 4k, 64k or 16M pages in the VRMA */
- err = -EINVAL;
- if (!(psize == 0x1000 || psize == 0x10000 ||
- psize == 0x1000000))
- goto out_srcu;
+ if (psize >= 0x1000000)
+ psize = 0x1000000;
+ else if (psize >= 0x10000)
+ psize = 0x10000;
+ else
+ psize = 0x1000;
+ porder = __ilog2(psize);
senc = slb_pgsize_encoding(psize);
kvm->arch.vrma_slb_v = senc | SLB_VSID_B_1T |
stw r12, STACK_SLOT_TRAP(r1)
bl kvmhv_commence_exit
nop
- lwz r12, STACK_SLOT_TRAP(r1)
b kvmhv_switch_to_host
/*
secondary_too_late:
li r12, 0
+ stw r12, STACK_SLOT_TRAP(r1)
cmpdi r4, 0
beq 11f
stw r12, VCPU_TRAP(r4)
3: stw r5,VCPU_SLB_MAX(r9)
guest_bypass:
+ stw r12, STACK_SLOT_TRAP(r1)
mr r3, r12
/* Increment exit count, poke other threads to exit */
bl kvmhv_commence_exit
nop
ld r9, HSTATE_KVM_VCPU(r13)
- lwz r12, VCPU_TRAP(r9)
/* Stop others sending VCPU interrupts to this physical CPU */
li r0, -1
* POWER7/POWER8 guest -> host partition switch code.
* We don't have to lock against tlbies but we do
* have to coordinate the hardware threads.
+ * Here STACK_SLOT_TRAP(r1) contains the trap number.
*/
kvmhv_switch_to_host:
/* Secondary threads wait for primary to do partition switch */
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
/* If HMI, call kvmppc_realmode_hmi_handler() */
+ lwz r12, STACK_SLOT_TRAP(r1)
cmpwi r12, BOOK3S_INTERRUPT_HMI
bne 27f
bl kvmppc_realmode_hmi_handler
nop
cmpdi r3, 0
- li r12, BOOK3S_INTERRUPT_HMI
/*
* At this point kvmppc_realmode_hmi_handler may have resync-ed
* the TB, and if it has, we must not subtract the guest timebase
lwz r8, KVM_SPLIT_DO_RESTORE(r3)
cmpwi r8, 0
beq 47f
- stw r12, STACK_SLOT_TRAP(r1)
bl kvmhv_p9_restore_lpcr
nop
- lwz r12, STACK_SLOT_TRAP(r1)
b 48f
47:
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
li r0, KVM_GUEST_MODE_NONE
stb r0, HSTATE_IN_GUEST(r13)
+ lwz r12, STACK_SLOT_TRAP(r1) /* return trap # in r12 */
ld r0, SFS+PPC_LR_STKOFF(r1)
addi r1, r1, SFS
mtlr r0
int kvmppc_handle_load128_by2x64(struct kvm_run *run, struct kvm_vcpu *vcpu,
unsigned int rt, int is_default_endian)
{
- enum emulation_result emulated;
+ enum emulation_result emulated = EMULATE_DONE;
while (vcpu->arch.mmio_vmx_copy_nums) {
emulated = __kvmppc_handle_load(run, vcpu, rt, 8,
kvm_sigset_deactivate(vcpu);
+#ifdef CONFIG_ALTIVEC
out:
+#endif
vcpu_put(vcpu);
return r;
}
_ASCE_USER_BITS | _ASCE_TYPE_SEGMENT;
/* pgd_alloc() did not account this pmd */
mm_inc_nr_pmds(mm);
+ mm_inc_nr_puds(mm);
}
crst_table_init((unsigned long *) mm->pgd, pgd_entry_type(mm));
return 0;
#include <asm/processor.h>
#include <asm/cache.h>
#include <asm/ctl_reg.h>
+#include <asm/dwarf.h>
#include <asm/errno.h>
#include <asm/ptrace.h>
#include <asm/thread_info.h>
.hidden \name
.type \name,@function
\name:
- .cfi_startproc
+ CFI_STARTPROC
#ifdef CONFIG_HAVE_MARCH_Z10_FEATURES
exrl 0,0f
#else
#endif
j .
0: br \reg
- .cfi_endproc
+ CFI_ENDPROC
.endm
GEN_BR_THUNK __s390x_indirect_jump_r1use_r9,%r9,%r1
UPDATE_VTIME %r8,%r9,__LC_SYNC_ENTER_TIMER
BPENTER __TI_flags(%r12),_TIF_ISOLATE_BP
stmg %r0,%r7,__PT_R0(%r11)
- # clear user controlled register to prevent speculative use
- xgr %r0,%r0
mvc __PT_R8(64,%r11),__LC_SAVE_AREA_SYNC
mvc __PT_PSW(16,%r11),__LC_SVC_OLD_PSW
mvc __PT_INT_CODE(4,%r11),__LC_SVC_ILC
stg %r14,__PT_FLAGS(%r11)
.Lsysc_do_svc:
+ # clear user controlled register to prevent speculative use
+ xgr %r0,%r0
# load address of system call table
lg %r10,__THREAD_sysc_table(%r13,%r12)
llgh %r8,__PT_INT_CODE+2(%r11)
stg %r15,__LC_SYSTEM_TIMER
0: # update accounting time stamp
mvc __LC_LAST_UPDATE_TIMER(8),__LC_SYNC_ENTER_TIMER
+ BPENTER __TI_flags(%r12),_TIF_ISOLATE_BP
# set up saved register r11
lg %r15,__LC_KERNEL_STACK
la %r9,STACK_FRAME_OVERHEAD(%r15)
#include <linux/module.h>
#include <asm/nospec-branch.h>
-int nospec_call_disable = IS_ENABLED(EXPOLINE_OFF);
-int nospec_return_disable = !IS_ENABLED(EXPOLINE_FULL);
+int nospec_call_disable = IS_ENABLED(CONFIG_EXPOLINE_OFF);
+int nospec_return_disable = !IS_ENABLED(CONFIG_EXPOLINE_FULL);
static int __init nospectre_v2_setup_early(char *str)
{
{ "deliver_prefix_signal", VCPU_STAT(deliver_prefix_signal) },
{ "deliver_restart_signal", VCPU_STAT(deliver_restart_signal) },
{ "deliver_program_interruption", VCPU_STAT(deliver_program_int) },
+ { "deliver_io_interrupt", VCPU_STAT(deliver_io_int) },
{ "exit_wait_state", VCPU_STAT(exit_wait_state) },
{ "instruction_epsw", VCPU_STAT(instruction_epsw) },
{ "instruction_gs", VCPU_STAT(instruction_gs) },
/* we still need the basic sca for the ipte control */
vcpu->arch.sie_block->scaoh = (__u32)(((__u64)sca) >> 32);
vcpu->arch.sie_block->scaol = (__u32)(__u64)sca;
+ return;
}
read_lock(&vcpu->kvm->arch.sca_lock);
if (vcpu->kvm->arch.use_esca) {
pte_unmap(pte);
}
-void set_pmd_at(struct mm_struct *mm, unsigned long addr,
- pmd_t *pmdp, pmd_t pmd)
-{
- pmd_t orig = *pmdp;
-
- *pmdp = pmd;
+static void __set_pmd_acct(struct mm_struct *mm, unsigned long addr,
+ pmd_t orig, pmd_t pmd)
+{
if (mm == &init_mm)
return;
}
}
+void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+ pmd_t *pmdp, pmd_t pmd)
+{
+ pmd_t orig = *pmdp;
+
+ *pmdp = pmd;
+ __set_pmd_acct(mm, addr, orig, pmd);
+}
+
static inline pmd_t pmdp_establish(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp, pmd_t pmd)
{
do {
old = *pmdp;
} while (cmpxchg64(&pmdp->pmd, old.pmd, pmd.pmd) != old.pmd);
+ __set_pmd_acct(vma->vm_mm, address, old, pmd);
return old;
}
it can be used to assist security vulnerability exploitation.
This setting can be changed at boot time via the kernel command
- line parameter vsyscall=[native|emulate|none].
+ line parameter vsyscall=[emulate|none].
On a system with recent enough glibc (2.14 or newer) and no
static binaries, you can say None without a performance penalty
If unsure, select "Emulate".
- config LEGACY_VSYSCALL_NATIVE
- bool "Native"
- help
- Actual executable code is located in the fixed vsyscall
- address mapping, implementing time() efficiently. Since
- this makes the mapping executable, it can be used during
- security vulnerability exploitation (traditionally as
- ROP gadgets). This configuration is not recommended.
-
config LEGACY_VSYSCALL_EMULATE
bool "Emulate"
help
pushq 2*8(%rdi) /* regs->ip */
pushq 1*8(%rdi) /* regs->orig_ax */
- movq (%rdi), %rdi /* restore %rdi */
-
- pushq %rdi /* pt_regs->di */
+ pushq (%rdi) /* pt_regs->di */
pushq %rsi /* pt_regs->si */
pushq %rdx /* pt_regs->dx */
pushq %rcx /* pt_regs->cx */
TRACE_IRQS_ON
jmp swapgs_restore_regs_and_return_to_usermode
END(entry_INT80_compat)
-
-ENTRY(stub32_clone)
- /*
- * The 32-bit clone ABI is: clone(..., int tls_val, int *child_tidptr).
- * The 64-bit clone ABI is: clone(..., int *child_tidptr, int tls_val).
- *
- * The native 64-bit kernel's sys_clone() implements the latter,
- * so we need to swap arguments here before calling it:
- */
- xchg %r8, %rcx
- jmp sys_clone
-ENDPROC(stub32_clone)
#
0 i386 restart_syscall sys_restart_syscall
1 i386 exit sys_exit
-2 i386 fork sys_fork sys_fork
+2 i386 fork sys_fork
3 i386 read sys_read
4 i386 write sys_write
5 i386 open sys_open compat_sys_open
6 i386 close sys_close
-7 i386 waitpid sys_waitpid sys32_waitpid
+7 i386 waitpid sys_waitpid compat_sys_x86_waitpid
8 i386 creat sys_creat
9 i386 link sys_link
10 i386 unlink sys_unlink
69 i386 ssetmask sys_ssetmask
70 i386 setreuid sys_setreuid16
71 i386 setregid sys_setregid16
-72 i386 sigsuspend sys_sigsuspend sys_sigsuspend
+72 i386 sigsuspend sys_sigsuspend
73 i386 sigpending sys_sigpending compat_sys_sigpending
74 i386 sethostname sys_sethostname
75 i386 setrlimit sys_setrlimit compat_sys_setrlimit
87 i386 swapon sys_swapon
88 i386 reboot sys_reboot
89 i386 readdir sys_old_readdir compat_sys_old_readdir
-90 i386 mmap sys_old_mmap sys32_mmap
+90 i386 mmap sys_old_mmap compat_sys_x86_mmap
91 i386 munmap sys_munmap
92 i386 truncate sys_truncate compat_sys_truncate
93 i386 ftruncate sys_ftruncate compat_sys_ftruncate
117 i386 ipc sys_ipc compat_sys_ipc
118 i386 fsync sys_fsync
119 i386 sigreturn sys_sigreturn sys32_sigreturn
-120 i386 clone sys_clone stub32_clone
+120 i386 clone sys_clone compat_sys_x86_clone
121 i386 setdomainname sys_setdomainname
122 i386 uname sys_newuname
123 i386 modify_ldt sys_modify_ldt
177 i386 rt_sigtimedwait sys_rt_sigtimedwait compat_sys_rt_sigtimedwait
178 i386 rt_sigqueueinfo sys_rt_sigqueueinfo compat_sys_rt_sigqueueinfo
179 i386 rt_sigsuspend sys_rt_sigsuspend
-180 i386 pread64 sys_pread64 sys32_pread
-181 i386 pwrite64 sys_pwrite64 sys32_pwrite
+180 i386 pread64 sys_pread64 compat_sys_x86_pread
+181 i386 pwrite64 sys_pwrite64 compat_sys_x86_pwrite
182 i386 chown sys_chown16
183 i386 getcwd sys_getcwd
184 i386 capget sys_capget
187 i386 sendfile sys_sendfile compat_sys_sendfile
188 i386 getpmsg
189 i386 putpmsg
-190 i386 vfork sys_vfork sys_vfork
+190 i386 vfork sys_vfork
191 i386 ugetrlimit sys_getrlimit compat_sys_getrlimit
192 i386 mmap2 sys_mmap_pgoff
-193 i386 truncate64 sys_truncate64 sys32_truncate64
-194 i386 ftruncate64 sys_ftruncate64 sys32_ftruncate64
-195 i386 stat64 sys_stat64 sys32_stat64
-196 i386 lstat64 sys_lstat64 sys32_lstat64
-197 i386 fstat64 sys_fstat64 sys32_fstat64
+193 i386 truncate64 sys_truncate64 compat_sys_x86_truncate64
+194 i386 ftruncate64 sys_ftruncate64 compat_sys_x86_ftruncate64
+195 i386 stat64 sys_stat64 compat_sys_x86_stat64
+196 i386 lstat64 sys_lstat64 compat_sys_x86_lstat64
+197 i386 fstat64 sys_fstat64 compat_sys_x86_fstat64
198 i386 lchown32 sys_lchown
199 i386 getuid32 sys_getuid
200 i386 getgid32 sys_getgid
# 222 is unused
# 223 is unused
224 i386 gettid sys_gettid
-225 i386 readahead sys_readahead sys32_readahead
+225 i386 readahead sys_readahead compat_sys_x86_readahead
226 i386 setxattr sys_setxattr
227 i386 lsetxattr sys_lsetxattr
228 i386 fsetxattr sys_fsetxattr
247 i386 io_getevents sys_io_getevents compat_sys_io_getevents
248 i386 io_submit sys_io_submit compat_sys_io_submit
249 i386 io_cancel sys_io_cancel
-250 i386 fadvise64 sys_fadvise64 sys32_fadvise64
+250 i386 fadvise64 sys_fadvise64 compat_sys_x86_fadvise64
# 251 is available for reuse (was briefly sys_set_zone_reclaim)
252 i386 exit_group sys_exit_group
253 i386 lookup_dcookie sys_lookup_dcookie compat_sys_lookup_dcookie
269 i386 fstatfs64 sys_fstatfs64 compat_sys_fstatfs64
270 i386 tgkill sys_tgkill
271 i386 utimes sys_utimes compat_sys_utimes
-272 i386 fadvise64_64 sys_fadvise64_64 sys32_fadvise64_64
+272 i386 fadvise64_64 sys_fadvise64_64 compat_sys_x86_fadvise64_64
273 i386 vserver
274 i386 mbind sys_mbind
275 i386 get_mempolicy sys_get_mempolicy compat_sys_get_mempolicy
297 i386 mknodat sys_mknodat
298 i386 fchownat sys_fchownat
299 i386 futimesat sys_futimesat compat_sys_futimesat
-300 i386 fstatat64 sys_fstatat64 sys32_fstatat
+300 i386 fstatat64 sys_fstatat64 compat_sys_x86_fstatat
301 i386 unlinkat sys_unlinkat
302 i386 renameat sys_renameat
303 i386 linkat sys_linkat
311 i386 set_robust_list sys_set_robust_list compat_sys_set_robust_list
312 i386 get_robust_list sys_get_robust_list compat_sys_get_robust_list
313 i386 splice sys_splice
-314 i386 sync_file_range sys_sync_file_range sys32_sync_file_range
+314 i386 sync_file_range sys_sync_file_range compat_sys_x86_sync_file_range
315 i386 tee sys_tee
316 i386 vmsplice sys_vmsplice compat_sys_vmsplice
317 i386 move_pages sys_move_pages compat_sys_move_pages
321 i386 signalfd sys_signalfd compat_sys_signalfd
322 i386 timerfd_create sys_timerfd_create
323 i386 eventfd sys_eventfd
-324 i386 fallocate sys_fallocate sys32_fallocate
+324 i386 fallocate sys_fallocate compat_sys_x86_fallocate
325 i386 timerfd_settime sys_timerfd_settime compat_sys_timerfd_settime
326 i386 timerfd_gettime sys_timerfd_gettime compat_sys_timerfd_gettime
327 i386 signalfd4 sys_signalfd4 compat_sys_signalfd4
#define CREATE_TRACE_POINTS
#include "vsyscall_trace.h"
-static enum { EMULATE, NATIVE, NONE } vsyscall_mode =
-#if defined(CONFIG_LEGACY_VSYSCALL_NATIVE)
- NATIVE;
-#elif defined(CONFIG_LEGACY_VSYSCALL_NONE)
+static enum { EMULATE, NONE } vsyscall_mode =
+#ifdef CONFIG_LEGACY_VSYSCALL_NONE
NONE;
#else
EMULATE;
if (str) {
if (!strcmp("emulate", str))
vsyscall_mode = EMULATE;
- else if (!strcmp("native", str))
- vsyscall_mode = NATIVE;
else if (!strcmp("none", str))
vsyscall_mode = NONE;
else
WARN_ON_ONCE(address != regs->ip);
- /* This should be unreachable in NATIVE mode. */
- if (WARN_ON(vsyscall_mode == NATIVE))
- return false;
-
if (vsyscall_mode == NONE) {
warn_bad_vsyscall(KERN_INFO, regs,
"vsyscall attempted with vsyscall=none");
if (vsyscall_mode != NONE) {
__set_fixmap(VSYSCALL_PAGE, physaddr_vsyscall,
- vsyscall_mode == NATIVE
- ? PAGE_KERNEL_VSYSCALL
- : PAGE_KERNEL_VVAR);
+ PAGE_KERNEL_VVAR);
set_vsyscall_pgtable_user_bits(swapper_pg_dir);
}
};
static struct attribute *skx_upi_uncore_formats_attr[] = {
- &format_attr_event_ext.attr,
+ &format_attr_event.attr,
&format_attr_umask_ext.attr,
&format_attr_edge.attr,
&format_attr_inv.attr,
#define AA(__x) ((unsigned long)(__x))
-asmlinkage long sys32_truncate64(const char __user *filename,
- unsigned long offset_low,
- unsigned long offset_high)
+COMPAT_SYSCALL_DEFINE3(x86_truncate64, const char __user *, filename,
+ unsigned long, offset_low, unsigned long, offset_high)
{
return sys_truncate(filename, ((loff_t) offset_high << 32) | offset_low);
}
-asmlinkage long sys32_ftruncate64(unsigned int fd, unsigned long offset_low,
- unsigned long offset_high)
+COMPAT_SYSCALL_DEFINE3(x86_ftruncate64, unsigned int, fd,
+ unsigned long, offset_low, unsigned long, offset_high)
{
return sys_ftruncate(fd, ((loff_t) offset_high << 32) | offset_low);
}
return 0;
}
-asmlinkage long sys32_stat64(const char __user *filename,
- struct stat64 __user *statbuf)
+COMPAT_SYSCALL_DEFINE2(x86_stat64, const char __user *, filename,
+ struct stat64 __user *, statbuf)
{
struct kstat stat;
int ret = vfs_stat(filename, &stat);
return ret;
}
-asmlinkage long sys32_lstat64(const char __user *filename,
- struct stat64 __user *statbuf)
+COMPAT_SYSCALL_DEFINE2(x86_lstat64, const char __user *, filename,
+ struct stat64 __user *, statbuf)
{
struct kstat stat;
int ret = vfs_lstat(filename, &stat);
return ret;
}
-asmlinkage long sys32_fstat64(unsigned int fd, struct stat64 __user *statbuf)
+COMPAT_SYSCALL_DEFINE2(x86_fstat64, unsigned int, fd,
+ struct stat64 __user *, statbuf)
{
struct kstat stat;
int ret = vfs_fstat(fd, &stat);
return ret;
}
-asmlinkage long sys32_fstatat(unsigned int dfd, const char __user *filename,
- struct stat64 __user *statbuf, int flag)
+COMPAT_SYSCALL_DEFINE4(x86_fstatat, unsigned int, dfd,
+ const char __user *, filename,
+ struct stat64 __user *, statbuf, int, flag)
{
struct kstat stat;
int error;
unsigned int offset;
};
-asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *arg)
+COMPAT_SYSCALL_DEFINE1(x86_mmap, struct mmap_arg_struct32 __user *, arg)
{
struct mmap_arg_struct32 a;
a.offset>>PAGE_SHIFT);
}
-asmlinkage long sys32_waitpid(compat_pid_t pid, unsigned int __user *stat_addr,
- int options)
+COMPAT_SYSCALL_DEFINE3(x86_waitpid, compat_pid_t, pid, unsigned int __user *,
+ stat_addr, int, options)
{
return compat_sys_wait4(pid, stat_addr, options, NULL);
}
/* warning: next two assume little endian */
-asmlinkage long sys32_pread(unsigned int fd, char __user *ubuf, u32 count,
- u32 poslo, u32 poshi)
+COMPAT_SYSCALL_DEFINE5(x86_pread, unsigned int, fd, char __user *, ubuf,
+ u32, count, u32, poslo, u32, poshi)
{
return sys_pread64(fd, ubuf, count,
((loff_t)AA(poshi) << 32) | AA(poslo));
}
-asmlinkage long sys32_pwrite(unsigned int fd, const char __user *ubuf,
- u32 count, u32 poslo, u32 poshi)
+COMPAT_SYSCALL_DEFINE5(x86_pwrite, unsigned int, fd, const char __user *, ubuf,
+ u32, count, u32, poslo, u32, poshi)
{
return sys_pwrite64(fd, ubuf, count,
((loff_t)AA(poshi) << 32) | AA(poslo));
* Some system calls that need sign extended arguments. This could be
* done by a generic wrapper.
*/
-long sys32_fadvise64_64(int fd, __u32 offset_low, __u32 offset_high,
- __u32 len_low, __u32 len_high, int advice)
+COMPAT_SYSCALL_DEFINE6(x86_fadvise64_64, int, fd, __u32, offset_low,
+ __u32, offset_high, __u32, len_low, __u32, len_high,
+ int, advice)
{
return sys_fadvise64_64(fd,
(((u64)offset_high)<<32) | offset_low,
advice);
}
-asmlinkage ssize_t sys32_readahead(int fd, unsigned off_lo, unsigned off_hi,
- size_t count)
+COMPAT_SYSCALL_DEFINE4(x86_readahead, int, fd, unsigned int, off_lo,
+ unsigned int, off_hi, size_t, count)
{
return sys_readahead(fd, ((u64)off_hi << 32) | off_lo, count);
}
-asmlinkage long sys32_sync_file_range(int fd, unsigned off_low, unsigned off_hi,
- unsigned n_low, unsigned n_hi, int flags)
+COMPAT_SYSCALL_DEFINE6(x86_sync_file_range, int, fd, unsigned int, off_low,
+ unsigned int, off_hi, unsigned int, n_low,
+ unsigned int, n_hi, int, flags)
{
return sys_sync_file_range(fd,
((u64)off_hi << 32) | off_low,
((u64)n_hi << 32) | n_low, flags);
}
-asmlinkage long sys32_fadvise64(int fd, unsigned offset_lo, unsigned offset_hi,
- size_t len, int advice)
+COMPAT_SYSCALL_DEFINE5(x86_fadvise64, int, fd, unsigned int, offset_lo,
+ unsigned int, offset_hi, size_t, len, int, advice)
{
return sys_fadvise64_64(fd, ((u64)offset_hi << 32) | offset_lo,
len, advice);
}
-asmlinkage long sys32_fallocate(int fd, int mode, unsigned offset_lo,
- unsigned offset_hi, unsigned len_lo,
- unsigned len_hi)
+COMPAT_SYSCALL_DEFINE6(x86_fallocate, int, fd, int, mode,
+ unsigned int, offset_lo, unsigned int, offset_hi,
+ unsigned int, len_lo, unsigned int, len_hi)
{
return sys_fallocate(fd, mode, ((u64)offset_hi << 32) | offset_lo,
((u64)len_hi << 32) | len_lo);
}
+
+/*
+ * The 32-bit clone ABI is CONFIG_CLONE_BACKWARDS
+ */
+COMPAT_SYSCALL_DEFINE5(x86_clone, unsigned long, clone_flags,
+ unsigned long, newsp, int __user *, parent_tidptr,
+ unsigned long, tls_val, int __user *, child_tidptr)
+{
+ return sys_clone(clone_flags, newsp, parent_tidptr, child_tidptr,
+ tls_val);
+}
#define X86_FEATURE_VPCLMULQDQ (16*32+10) /* Carry-Less Multiplication Double Quadword */
#define X86_FEATURE_AVX512_VNNI (16*32+11) /* Vector Neural Network Instructions */
#define X86_FEATURE_AVX512_BITALG (16*32+12) /* Support for VPOPCNT[B,W] and VPSHUF-BITQMB instructions */
+#define X86_FEATURE_TME (16*32+13) /* Intel Total Memory Encryption */
#define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14) /* POPCNT for vectors of DW/QW */
#define X86_FEATURE_LA57 (16*32+16) /* 5-level page tables */
#define X86_FEATURE_RDPID (16*32+22) /* RDPID instruction */
/* Intel-defined CPU features, CPUID level 0x00000007:0 (EDX), word 18 */
#define X86_FEATURE_AVX512_4VNNIW (18*32+ 2) /* AVX-512 Neural Network Instructions */
#define X86_FEATURE_AVX512_4FMAPS (18*32+ 3) /* AVX-512 Multiply Accumulation Single precision */
+#define X86_FEATURE_PCONFIG (18*32+18) /* Intel PCONFIG */
#define X86_FEATURE_SPEC_CTRL (18*32+26) /* "" Speculation Control (IBRS + IBPB) */
#define X86_FEATURE_INTEL_STIBP (18*32+27) /* "" Single Thread Indirect Branch Predictors */
#define X86_FEATURE_ARCH_CAPABILITIES (18*32+29) /* IA32_ARCH_CAPABILITIES MSR (Intel) */
enum ucode_state {
UCODE_OK = 0,
+ UCODE_NEW,
UCODE_UPDATED,
UCODE_NFOUND,
UCODE_ERROR,
* otherwise we'll run out of registers. We don't care about CET
* here, anyway.
*/
-# define CALL_NOSPEC ALTERNATIVE("call *%[thunk_target]\n", \
+# define CALL_NOSPEC \
+ ALTERNATIVE( \
+ ANNOTATE_RETPOLINE_SAFE \
+ "call *%[thunk_target]\n", \
" jmp 904f;\n" \
" .align 16\n" \
"901: call 903f;\n" \
#define __PAGE_KERNEL_RO (__PAGE_KERNEL & ~_PAGE_RW)
#define __PAGE_KERNEL_RX (__PAGE_KERNEL_EXEC & ~_PAGE_RW)
#define __PAGE_KERNEL_NOCACHE (__PAGE_KERNEL | _PAGE_NOCACHE)
-#define __PAGE_KERNEL_VSYSCALL (__PAGE_KERNEL_RX | _PAGE_USER)
#define __PAGE_KERNEL_VVAR (__PAGE_KERNEL_RO | _PAGE_USER)
#define __PAGE_KERNEL_LARGE (__PAGE_KERNEL | _PAGE_PSE)
#define __PAGE_KERNEL_LARGE_EXEC (__PAGE_KERNEL_EXEC | _PAGE_PSE)
#define PAGE_KERNEL_NOCACHE __pgprot(__PAGE_KERNEL_NOCACHE | _PAGE_ENC)
#define PAGE_KERNEL_LARGE __pgprot(__PAGE_KERNEL_LARGE | _PAGE_ENC)
#define PAGE_KERNEL_LARGE_EXEC __pgprot(__PAGE_KERNEL_LARGE_EXEC | _PAGE_ENC)
-#define PAGE_KERNEL_VSYSCALL __pgprot(__PAGE_KERNEL_VSYSCALL | _PAGE_ENC)
#define PAGE_KERNEL_VVAR __pgprot(__PAGE_KERNEL_VVAR | _PAGE_ENC)
#define PAGE_KERNEL_IO __pgprot(__PAGE_KERNEL_IO)
#if defined(CONFIG_X86_64)
extern char __end_rodata_hpage_align[];
+extern char __entry_trampoline_start[], __entry_trampoline_end[];
#endif
#endif /* _ASM_X86_SECTIONS_H */
#include <asm/ia32.h>
/* ia32/sys_ia32.c */
-asmlinkage long sys32_truncate64(const char __user *, unsigned long, unsigned long);
-asmlinkage long sys32_ftruncate64(unsigned int, unsigned long, unsigned long);
+asmlinkage long compat_sys_x86_truncate64(const char __user *, unsigned long,
+ unsigned long);
+asmlinkage long compat_sys_x86_ftruncate64(unsigned int, unsigned long,
+ unsigned long);
-asmlinkage long sys32_stat64(const char __user *, struct stat64 __user *);
-asmlinkage long sys32_lstat64(const char __user *, struct stat64 __user *);
-asmlinkage long sys32_fstat64(unsigned int, struct stat64 __user *);
-asmlinkage long sys32_fstatat(unsigned int, const char __user *,
+asmlinkage long compat_sys_x86_stat64(const char __user *,
+ struct stat64 __user *);
+asmlinkage long compat_sys_x86_lstat64(const char __user *,
+ struct stat64 __user *);
+asmlinkage long compat_sys_x86_fstat64(unsigned int, struct stat64 __user *);
+asmlinkage long compat_sys_x86_fstatat(unsigned int, const char __user *,
struct stat64 __user *, int);
struct mmap_arg_struct32;
-asmlinkage long sys32_mmap(struct mmap_arg_struct32 __user *);
+asmlinkage long compat_sys_x86_mmap(struct mmap_arg_struct32 __user *);
-asmlinkage long sys32_waitpid(compat_pid_t, unsigned int __user *, int);
+asmlinkage long compat_sys_x86_waitpid(compat_pid_t, unsigned int __user *,
+ int);
-asmlinkage long sys32_pread(unsigned int, char __user *, u32, u32, u32);
-asmlinkage long sys32_pwrite(unsigned int, const char __user *, u32, u32, u32);
+asmlinkage long compat_sys_x86_pread(unsigned int, char __user *, u32, u32,
+ u32);
+asmlinkage long compat_sys_x86_pwrite(unsigned int, const char __user *, u32,
+ u32, u32);
-long sys32_fadvise64_64(int, __u32, __u32, __u32, __u32, int);
-long sys32_vm86_warning(void);
+asmlinkage long compat_sys_x86_fadvise64_64(int, __u32, __u32, __u32, __u32,
+ int);
-asmlinkage ssize_t sys32_readahead(int, unsigned, unsigned, size_t);
-asmlinkage long sys32_sync_file_range(int, unsigned, unsigned,
- unsigned, unsigned, int);
-asmlinkage long sys32_fadvise64(int, unsigned, unsigned, size_t, int);
-asmlinkage long sys32_fallocate(int, int, unsigned,
- unsigned, unsigned, unsigned);
+asmlinkage ssize_t compat_sys_x86_readahead(int, unsigned int, unsigned int,
+ size_t);
+asmlinkage long compat_sys_x86_sync_file_range(int, unsigned int, unsigned int,
+ unsigned int, unsigned int,
+ int);
+asmlinkage long compat_sys_x86_fadvise64(int, unsigned int, unsigned int,
+ size_t, int);
+asmlinkage long compat_sys_x86_fallocate(int, int, unsigned int, unsigned int,
+ unsigned int, unsigned int);
+asmlinkage long compat_sys_x86_clone(unsigned long, unsigned long, int __user *,
+ unsigned long, int __user *);
/* ia32/ia32_signal.c */
asmlinkage long sys32_sigreturn(void);
#define INTR_TYPE_NMI_INTR (2 << 8) /* NMI */
#define INTR_TYPE_HARD_EXCEPTION (3 << 8) /* processor exception */
#define INTR_TYPE_SOFT_INTR (4 << 8) /* software interrupt */
+#define INTR_TYPE_PRIV_SW_EXCEPTION (5 << 8) /* ICE breakpoint - undocumented */
#define INTR_TYPE_SOFT_EXCEPTION (6 << 8) /* software exception */
/* GUEST_INTERRUPTIBILITY_INFO flags. */
__u64 synd; /* MCA_SYND MSR: only valid on SMCA systems */
__u64 ipid; /* MCA_IPID MSR: only valid on SMCA systems */
__u64 ppin; /* Protected Processor Inventory Number */
+ __u32 microcode;/* Microcode revision */
};
#define MCE_GET_RECORD_LEN _IOR('M', 1, int)
/*
* Early microcode releases for the Spectre v2 mitigation were broken.
* Information taken from;
- * - https://newsroom.intel.com/wp-content/uploads/sites/11/2018/01/microcode-update-guidance.pdf
+ * - https://newsroom.intel.com/wp-content/uploads/sites/11/2018/03/microcode-update-guidance.pdf
* - https://kb.vmware.com/s/article/52345
* - Microcode revisions observed in the wild
* - Release note from 20180108 microcode release
{ INTEL_FAM6_KABYLAKE_MOBILE, 0x09, 0x80 },
{ INTEL_FAM6_SKYLAKE_X, 0x03, 0x0100013e },
{ INTEL_FAM6_SKYLAKE_X, 0x04, 0x0200003c },
- { INTEL_FAM6_SKYLAKE_DESKTOP, 0x03, 0xc2 },
{ INTEL_FAM6_BROADWELL_CORE, 0x04, 0x28 },
{ INTEL_FAM6_BROADWELL_GT3E, 0x01, 0x1b },
{ INTEL_FAM6_BROADWELL_XEON_D, 0x02, 0x14 },
{
int i;
+ /*
+ * We know that the hypervisor lie to us on the microcode version so
+ * we may as well hope that it is running the correct version.
+ */
+ if (cpu_has(c, X86_FEATURE_HYPERVISOR))
+ return false;
+
for (i = 0; i < ARRAY_SIZE(spectre_bad_microcodes); i++) {
if (c->x86_model == spectre_bad_microcodes[i].model &&
c->x86_stepping == spectre_bad_microcodes[i].stepping)
static DEFINE_MUTEX(mce_log_mutex);
+/* sysfs synchronization */
+static DEFINE_MUTEX(mce_sysfs_mutex);
+
#define CREATE_TRACE_POINTS
#include <trace/events/mce.h>
if (this_cpu_has(X86_FEATURE_INTEL_PPIN))
rdmsrl(MSR_PPIN, m->ppin);
+
+ m->microcode = boot_cpu_data.microcode;
}
DEFINE_PER_CPU(struct mce, injectm);
*/
pr_emerg(HW_ERR "PROCESSOR %u:%x TIME %llu SOCKET %u APIC %x microcode %x\n",
m->cpuvendor, m->cpuid, m->time, m->socketid, m->apicid,
- cpu_data(m->extcpu).microcode);
+ m->microcode);
}
static void print_mce(struct mce *m)
if (kstrtou64(buf, 0, &new) < 0)
return -EINVAL;
+ mutex_lock(&mce_sysfs_mutex);
if (mca_cfg.ignore_ce ^ !!new) {
if (new) {
/* disable ce features */
on_each_cpu(mce_enable_ce, (void *)1, 1);
}
}
+ mutex_unlock(&mce_sysfs_mutex);
+
return size;
}
if (kstrtou64(buf, 0, &new) < 0)
return -EINVAL;
+ mutex_lock(&mce_sysfs_mutex);
if (mca_cfg.cmci_disabled ^ !!new) {
if (new) {
/* disable cmci */
on_each_cpu(mce_enable_ce, NULL, 1);
}
}
+ mutex_unlock(&mce_sysfs_mutex);
+
return size;
}
struct device_attribute *attr,
const char *buf, size_t size)
{
- ssize_t ret = device_store_int(s, attr, buf, size);
+ unsigned long old_check_interval = check_interval;
+ ssize_t ret = device_store_ulong(s, attr, buf, size);
+
+ if (check_interval == old_check_interval)
+ return ret;
+
+ if (check_interval < 1)
+ check_interval = 1;
+
+ mutex_lock(&mce_sysfs_mutex);
mce_restart();
+ mutex_unlock(&mce_sysfs_mutex);
+
return ret;
}
return -EINVAL;
ret = load_microcode_amd(true, x86_family(cpuid_1_eax), desc.data, desc.size);
- if (ret != UCODE_OK)
+ if (ret > UCODE_UPDATED)
return -EINVAL;
return 0;
static enum ucode_state
load_microcode_amd(bool save, u8 family, const u8 *data, size_t size)
{
+ struct ucode_patch *p;
enum ucode_state ret;
/* free old equiv table */
free_equiv_cpu_table();
ret = __load_microcode_amd(family, data, size);
-
- if (ret != UCODE_OK)
+ if (ret != UCODE_OK) {
cleanup();
+ return ret;
+ }
-#ifdef CONFIG_X86_32
- /* save BSP's matching patch for early load */
- if (save) {
- struct ucode_patch *p = find_patch(0);
- if (p) {
- memset(amd_ucode_patch, 0, PATCH_MAX_SIZE);
- memcpy(amd_ucode_patch, p->data, min_t(u32, ksize(p->data),
- PATCH_MAX_SIZE));
- }
+ p = find_patch(0);
+ if (!p) {
+ return ret;
+ } else {
+ if (boot_cpu_data.microcode == p->patch_id)
+ return ret;
+
+ ret = UCODE_NEW;
}
-#endif
+
+ /* save BSP's matching patch for early load */
+ if (!save)
+ return ret;
+
+ memset(amd_ucode_patch, 0, PATCH_MAX_SIZE);
+ memcpy(amd_ucode_patch, p->data, min_t(u32, ksize(p->data), PATCH_MAX_SIZE));
+
return ret;
}
#define pr_fmt(fmt) "microcode: " fmt
#include <linux/platform_device.h>
+#include <linux/stop_machine.h>
#include <linux/syscore_ops.h>
#include <linux/miscdevice.h>
#include <linux/capability.h>
#include <linux/firmware.h>
#include <linux/kernel.h>
+#include <linux/delay.h>
#include <linux/mutex.h>
#include <linux/cpu.h>
+#include <linux/nmi.h>
#include <linux/fs.h>
#include <linux/mm.h>
*/
static DEFINE_MUTEX(microcode_mutex);
+/*
+ * Serialize late loading so that CPUs get updated one-by-one.
+ */
+static DEFINE_SPINLOCK(update_lock);
+
struct ucode_cpu_info ucode_cpu_info[NR_CPUS];
struct cpu_info_ctx {
return ret;
}
-struct apply_microcode_ctx {
- enum ucode_state err;
-};
-
static void apply_microcode_local(void *arg)
{
- struct apply_microcode_ctx *ctx = arg;
+ enum ucode_state *err = arg;
- ctx->err = microcode_ops->apply_microcode(smp_processor_id());
+ *err = microcode_ops->apply_microcode(smp_processor_id());
}
static int apply_microcode_on_target(int cpu)
{
- struct apply_microcode_ctx ctx = { .err = 0 };
+ enum ucode_state err;
int ret;
- ret = smp_call_function_single(cpu, apply_microcode_local, &ctx, 1);
- if (!ret)
- ret = ctx.err;
-
+ ret = smp_call_function_single(cpu, apply_microcode_local, &err, 1);
+ if (!ret) {
+ if (err == UCODE_ERROR)
+ ret = 1;
+ }
return ret;
}
/* fake device for request_firmware */
static struct platform_device *microcode_pdev;
-static enum ucode_state reload_for_cpu(int cpu)
+/*
+ * Late loading dance. Why the heavy-handed stomp_machine effort?
+ *
+ * - HT siblings must be idle and not execute other code while the other sibling
+ * is loading microcode in order to avoid any negative interactions caused by
+ * the loading.
+ *
+ * - In addition, microcode update on the cores must be serialized until this
+ * requirement can be relaxed in the future. Right now, this is conservative
+ * and good.
+ */
+#define SPINUNIT 100 /* 100 nsec */
+
+static int check_online_cpus(void)
{
- struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
- enum ucode_state ustate;
+ if (num_online_cpus() == num_present_cpus())
+ return 0;
- if (!uci->valid)
- return UCODE_OK;
+ pr_err("Not all CPUs online, aborting microcode update.\n");
+
+ return -EINVAL;
+}
+
+static atomic_t late_cpus_in;
+static atomic_t late_cpus_out;
+
+static int __wait_for_cpus(atomic_t *t, long long timeout)
+{
+ int all_cpus = num_online_cpus();
+
+ atomic_inc(t);
- ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev, true);
- if (ustate != UCODE_OK)
- return ustate;
+ while (atomic_read(t) < all_cpus) {
+ if (timeout < SPINUNIT) {
+ pr_err("Timeout while waiting for CPUs rendezvous, remaining: %d\n",
+ all_cpus - atomic_read(t));
+ return 1;
+ }
- return apply_microcode_on_target(cpu);
+ ndelay(SPINUNIT);
+ timeout -= SPINUNIT;
+
+ touch_nmi_watchdog();
+ }
+ return 0;
+}
+
+/*
+ * Returns:
+ * < 0 - on error
+ * 0 - no update done
+ * 1 - microcode was updated
+ */
+static int __reload_late(void *info)
+{
+ int cpu = smp_processor_id();
+ enum ucode_state err;
+ int ret = 0;
+
+ /*
+ * Wait for all CPUs to arrive. A load will not be attempted unless all
+ * CPUs show up.
+ * */
+ if (__wait_for_cpus(&late_cpus_in, NSEC_PER_SEC))
+ return -1;
+
+ spin_lock(&update_lock);
+ apply_microcode_local(&err);
+ spin_unlock(&update_lock);
+
+ if (err > UCODE_NFOUND) {
+ pr_warn("Error reloading microcode on CPU %d\n", cpu);
+ return -1;
+ /* siblings return UCODE_OK because their engine got updated already */
+ } else if (err == UCODE_UPDATED || err == UCODE_OK) {
+ ret = 1;
+ } else {
+ return ret;
+ }
+
+ /*
+ * Increase the wait timeout to a safe value here since we're
+ * serializing the microcode update and that could take a while on a
+ * large number of CPUs. And that is fine as the *actual* timeout will
+ * be determined by the last CPU finished updating and thus cut short.
+ */
+ if (__wait_for_cpus(&late_cpus_out, NSEC_PER_SEC * num_online_cpus()))
+ panic("Timeout during microcode update!\n");
+
+ return ret;
+}
+
+/*
+ * Reload microcode late on all CPUs. Wait for a sec until they
+ * all gather together.
+ */
+static int microcode_reload_late(void)
+{
+ int ret;
+
+ atomic_set(&late_cpus_in, 0);
+ atomic_set(&late_cpus_out, 0);
+
+ ret = stop_machine_cpuslocked(__reload_late, NULL, cpu_online_mask);
+ if (ret > 0)
+ microcode_check();
+
+ return ret;
}
static ssize_t reload_store(struct device *dev,
const char *buf, size_t size)
{
enum ucode_state tmp_ret = UCODE_OK;
- bool do_callback = false;
+ int bsp = boot_cpu_data.cpu_index;
unsigned long val;
ssize_t ret = 0;
- int cpu;
ret = kstrtoul(buf, 0, &val);
if (ret)
if (val != 1)
return size;
- get_online_cpus();
- mutex_lock(µcode_mutex);
- for_each_online_cpu(cpu) {
- tmp_ret = reload_for_cpu(cpu);
- if (tmp_ret > UCODE_NFOUND) {
- pr_warn("Error reloading microcode on CPU %d\n", cpu);
-
- /* set retval for the first encountered reload error */
- if (!ret)
- ret = -EINVAL;
- }
+ tmp_ret = microcode_ops->request_microcode_fw(bsp, µcode_pdev->dev, true);
+ if (tmp_ret != UCODE_NEW)
+ return size;
- if (tmp_ret == UCODE_UPDATED)
- do_callback = true;
- }
+ get_online_cpus();
- if (!ret && do_callback)
- microcode_check();
+ ret = check_online_cpus();
+ if (ret)
+ goto put;
+ mutex_lock(µcode_mutex);
+ ret = microcode_reload_late();
mutex_unlock(µcode_mutex);
+
+put:
put_online_cpus();
- if (!ret)
+ if (ret >= 0)
ret = size;
return ret;
if (system_state != SYSTEM_RUNNING)
return UCODE_NFOUND;
- ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev,
- refresh_fw);
-
- if (ustate == UCODE_OK) {
+ ustate = microcode_ops->request_microcode_fw(cpu, µcode_pdev->dev, refresh_fw);
+ if (ustate == UCODE_NEW) {
pr_debug("CPU%d updated upon init\n", cpu);
apply_microcode_on_target(cpu);
}
if (!mc)
return 0;
+ /*
+ * Save us the MSR write below - which is a particular expensive
+ * operation - when the other hyperthread has updated the microcode
+ * already.
+ */
+ rev = intel_get_microcode_revision();
+ if (rev >= mc->hdr.rev) {
+ uci->cpu_sig.rev = rev;
+ return UCODE_OK;
+ }
+
+ /*
+ * Writeback and invalidate caches before updating microcode to avoid
+ * internal issues depending on what the microcode is updating.
+ */
+ native_wbinvd();
+
/* write microcode via MSR 0x79 */
native_wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc->bits);
static enum ucode_state apply_microcode_intel(int cpu)
{
+ struct ucode_cpu_info *uci = ucode_cpu_info + cpu;
+ struct cpuinfo_x86 *c = &cpu_data(cpu);
struct microcode_intel *mc;
- struct ucode_cpu_info *uci;
- struct cpuinfo_x86 *c;
static int prev_rev;
u32 rev;
if (WARN_ON(raw_smp_processor_id() != cpu))
return UCODE_ERROR;
- uci = ucode_cpu_info + cpu;
- mc = uci->mc;
+ /* Look for a newer patch in our cache: */
+ mc = find_patch(uci);
if (!mc) {
- /* Look for a newer patch in our cache: */
- mc = find_patch(uci);
+ mc = uci->mc;
if (!mc)
return UCODE_NFOUND;
}
+ /*
+ * Save us the MSR write below - which is a particular expensive
+ * operation - when the other hyperthread has updated the microcode
+ * already.
+ */
+ rev = intel_get_microcode_revision();
+ if (rev >= mc->hdr.rev) {
+ uci->cpu_sig.rev = rev;
+ c->microcode = rev;
+ return UCODE_OK;
+ }
+
+ /*
+ * Writeback and invalidate caches before updating microcode to avoid
+ * internal issues depending on what the microcode is updating.
+ */
+ native_wbinvd();
+
/* write microcode via MSR 0x79 */
wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc->bits);
prev_rev = rev;
}
- c = &cpu_data(cpu);
-
uci->cpu_sig.rev = rev;
c->microcode = rev;
unsigned int leftover = size;
unsigned int curr_mc_size = 0, new_mc_size = 0;
unsigned int csig, cpf;
+ enum ucode_state ret = UCODE_OK;
while (leftover) {
struct microcode_header_intel mc_header;
new_mc = mc;
new_mc_size = mc_size;
mc = NULL; /* trigger new vmalloc */
+ ret = UCODE_NEW;
}
ucode_ptr += mc_size;
pr_debug("CPU%d found a matching microcode update with version 0x%x (current=0x%x)\n",
cpu, new_rev, uci->cpu_sig.rev);
- return UCODE_OK;
+ return ret;
}
static int get_ucode_fw(void *to, const void *from, size_t n)
/*
* this changes the io permissions bitmap in the current task.
*/
-asmlinkage long sys_ioperm(unsigned long from, unsigned long num, int turn_on)
+SYSCALL_DEFINE3(ioperm, unsigned long, from, unsigned long, num, int, turn_on)
{
struct thread_struct *t = ¤t->thread;
struct tss_struct *tss;
bool arch_within_kprobe_blacklist(unsigned long addr)
{
+ bool is_in_entry_trampoline_section = false;
+
+#ifdef CONFIG_X86_64
+ is_in_entry_trampoline_section =
+ (addr >= (unsigned long)__entry_trampoline_start &&
+ addr < (unsigned long)__entry_trampoline_end);
+#endif
return (addr >= (unsigned long)__kprobes_text_start &&
addr < (unsigned long)__kprobes_text_end) ||
(addr >= (unsigned long)__entry_text_start &&
- addr < (unsigned long)__entry_text_end);
+ addr < (unsigned long)__entry_text_end) ||
+ is_in_entry_trampoline_section;
}
int __init arch_init_kprobes(void)
BUILD_BUG_ON(offsetof(compat_siginfo_t, _sifields) != 3 * sizeof(int));
#define CHECK_CSI_OFFSET(name) BUILD_BUG_ON(_sifields_offset != offsetof(compat_siginfo_t, _sifields.name))
+ BUILD_BUG_ON(offsetof(siginfo_t, si_signo) != 0);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_errno) != 4);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_code) != 8);
+
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_signo) != 0);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_errno) != 4);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_code) != 8);
/*
* Ensure that the size of each si_field never changes.
* If it does, it is a sign that the
CHECK_CSI_SIZE (_kill, 2*sizeof(int));
CHECK_SI_SIZE (_kill, 2*sizeof(int));
+ BUILD_BUG_ON(offsetof(siginfo_t, si_pid) != 0x10);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_uid) != 0x14);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_pid) != 0xC);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_uid) != 0x10);
+
CHECK_CSI_OFFSET(_timer);
CHECK_CSI_SIZE (_timer, 3*sizeof(int));
CHECK_SI_SIZE (_timer, 6*sizeof(int));
+ BUILD_BUG_ON(offsetof(siginfo_t, si_tid) != 0x10);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_overrun) != 0x14);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_value) != 0x18);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_tid) != 0x0C);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_overrun) != 0x10);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_value) != 0x14);
+
CHECK_CSI_OFFSET(_rt);
CHECK_CSI_SIZE (_rt, 3*sizeof(int));
CHECK_SI_SIZE (_rt, 4*sizeof(int));
+ BUILD_BUG_ON(offsetof(siginfo_t, si_pid) != 0x10);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_uid) != 0x14);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_value) != 0x18);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_pid) != 0x0C);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_uid) != 0x10);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_value) != 0x14);
+
CHECK_CSI_OFFSET(_sigchld);
CHECK_CSI_SIZE (_sigchld, 5*sizeof(int));
CHECK_SI_SIZE (_sigchld, 8*sizeof(int));
+ BUILD_BUG_ON(offsetof(siginfo_t, si_pid) != 0x10);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_uid) != 0x14);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_status) != 0x18);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_utime) != 0x20);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_stime) != 0x28);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_pid) != 0x0C);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_uid) != 0x10);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_status) != 0x14);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_utime) != 0x18);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_stime) != 0x1C);
+
#ifdef CONFIG_X86_X32_ABI
CHECK_CSI_OFFSET(_sigchld_x32);
CHECK_CSI_SIZE (_sigchld_x32, 7*sizeof(int));
/* no _sigchld_x32 in the generic siginfo_t */
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, _sifields._sigchld_x32._utime) != 0x18);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, _sifields._sigchld_x32._stime) != 0x20);
#endif
CHECK_CSI_OFFSET(_sigfault);
CHECK_CSI_SIZE (_sigfault, 4*sizeof(int));
CHECK_SI_SIZE (_sigfault, 8*sizeof(int));
+ BUILD_BUG_ON(offsetof(siginfo_t, si_addr) != 0x10);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_addr) != 0x0C);
+
+ BUILD_BUG_ON(offsetof(siginfo_t, si_addr_lsb) != 0x18);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_addr_lsb) != 0x10);
+
+ BUILD_BUG_ON(offsetof(siginfo_t, si_lower) != 0x20);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_upper) != 0x28);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_lower) != 0x14);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_upper) != 0x18);
+
+ BUILD_BUG_ON(offsetof(siginfo_t, si_pkey) != 0x20);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_pkey) != 0x14);
+
CHECK_CSI_OFFSET(_sigpoll);
CHECK_CSI_SIZE (_sigpoll, 2*sizeof(int));
CHECK_SI_SIZE (_sigpoll, 4*sizeof(int));
+ BUILD_BUG_ON(offsetof(siginfo_t, si_band) != 0x10);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_fd) != 0x18);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_band) != 0x0C);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_fd) != 0x10);
+
CHECK_CSI_OFFSET(_sigsys);
CHECK_CSI_SIZE (_sigsys, 3*sizeof(int));
CHECK_SI_SIZE (_sigsys, 4*sizeof(int));
+ BUILD_BUG_ON(offsetof(siginfo_t, si_call_addr) != 0x10);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_syscall) != 0x18);
+ BUILD_BUG_ON(offsetof(siginfo_t, si_arch) != 0x1C);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_call_addr) != 0x0C);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_syscall) != 0x10);
+ BUILD_BUG_ON(offsetof(compat_siginfo_t, si_arch) != 0x14);
+
/* any new si_fields should be added here */
}
return;
check_vip:
- if (VEFLAGS & X86_EFLAGS_VIP) {
+ if ((VEFLAGS & (X86_EFLAGS_VIP | X86_EFLAGS_VIF)) ==
+ (X86_EFLAGS_VIP | X86_EFLAGS_VIF)) {
save_v86_state(regs, VM86_STI);
return;
}
#ifdef CONFIG_X86_64
. = ALIGN(PAGE_SIZE);
+ VMLINUX_SYMBOL(__entry_trampoline_start) = .;
_entry_trampoline = .;
*(.entry_trampoline)
. = ALIGN(PAGE_SIZE);
+ VMLINUX_SYMBOL(__entry_trampoline_end) = .;
ASSERT(. - _entry_trampoline == PAGE_SIZE, "entry trampoline is too big");
#endif
else
pte_access &= ~ACC_WRITE_MASK;
+ if (!kvm_is_mmio_pfn(pfn))
+ spte |= shadow_me_mask;
+
spte |= (u64)pfn << PAGE_SHIFT;
- spte |= shadow_me_mask;
if (pte_access & ACC_WRITE_MASK) {
(INTR_TYPE_HARD_EXCEPTION | MC_VECTOR | INTR_INFO_VALID_MASK);
}
+/* Undocumented: icebp/int1 */
+static inline bool is_icebp(u32 intr_info)
+{
+ return (intr_info & (INTR_INFO_INTR_TYPE_MASK | INTR_INFO_VALID_MASK))
+ == (INTR_TYPE_PRIV_SW_EXCEPTION | INTR_INFO_VALID_MASK);
+}
+
static inline bool cpu_has_vmx_msr_bitmap(void)
{
return vmcs_config.cpu_based_exec_ctrl & CPU_BASED_USE_MSR_BITMAPS;
(KVM_GUESTDBG_SINGLESTEP | KVM_GUESTDBG_USE_HW_BP))) {
vcpu->arch.dr6 &= ~15;
vcpu->arch.dr6 |= dr6 | DR6_RTM;
- if (!(dr6 & ~DR6_RESERVED)) /* icebp */
+ if (is_icebp(intr_info))
skip_emulated_instruction(vcpu);
kvm_queue_exception(vcpu, DB_VECTOR);
if (!pmd_k)
return -1;
- if (pmd_huge(*pmd_k))
+ if (pmd_large(*pmd_k))
return 0;
pte_k = pte_offset_kernel(pmd_k, address);
if (pud_none(*pud) || pud_pfn(*pud) != pud_pfn(*pud_ref))
BUG();
- if (pud_huge(*pud))
+ if (pud_large(*pud))
return 0;
pmd = pmd_offset(pud, address);
if (pmd_none(*pmd) || pmd_pfn(*pmd) != pmd_pfn(*pmd_ref))
BUG();
- if (pmd_huge(*pmd))
+ if (pmd_large(*pmd))
return 0;
pte_ref = pte_offset_kernel(pmd_ref, address);
#define PAGE_INUSE 0xFD
-static void __meminit free_pagetable(struct page *page, int order,
- struct vmem_altmap *altmap)
+static void __meminit free_pagetable(struct page *page, int order)
{
unsigned long magic;
unsigned int nr_pages = 1 << order;
- if (altmap) {
- vmem_altmap_free(altmap, nr_pages);
- return;
- }
-
/* bootmem page has reserved flag */
if (PageReserved(page)) {
__ClearPageReserved(page);
free_pages((unsigned long)page_address(page), order);
}
-static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd,
+static void __meminit free_hugepage_table(struct page *page,
struct vmem_altmap *altmap)
+{
+ if (altmap)
+ vmem_altmap_free(altmap, PMD_SIZE / PAGE_SIZE);
+ else
+ free_pagetable(page, get_order(PMD_SIZE));
+}
+
+static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd)
{
pte_t *pte;
int i;
}
/* free a pte talbe */
- free_pagetable(pmd_page(*pmd), 0, altmap);
+ free_pagetable(pmd_page(*pmd), 0);
spin_lock(&init_mm.page_table_lock);
pmd_clear(pmd);
spin_unlock(&init_mm.page_table_lock);
}
-static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud,
- struct vmem_altmap *altmap)
+static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud)
{
pmd_t *pmd;
int i;
}
/* free a pmd talbe */
- free_pagetable(pud_page(*pud), 0, altmap);
+ free_pagetable(pud_page(*pud), 0);
spin_lock(&init_mm.page_table_lock);
pud_clear(pud);
spin_unlock(&init_mm.page_table_lock);
}
-static void __meminit free_pud_table(pud_t *pud_start, p4d_t *p4d,
- struct vmem_altmap *altmap)
+static void __meminit free_pud_table(pud_t *pud_start, p4d_t *p4d)
{
pud_t *pud;
int i;
}
/* free a pud talbe */
- free_pagetable(p4d_page(*p4d), 0, altmap);
+ free_pagetable(p4d_page(*p4d), 0);
spin_lock(&init_mm.page_table_lock);
p4d_clear(p4d);
spin_unlock(&init_mm.page_table_lock);
static void __meminit
remove_pte_table(pte_t *pte_start, unsigned long addr, unsigned long end,
- struct vmem_altmap *altmap, bool direct)
+ bool direct)
{
unsigned long next, pages = 0;
pte_t *pte;
* freed when offlining, or simplely not in use.
*/
if (!direct)
- free_pagetable(pte_page(*pte), 0, altmap);
+ free_pagetable(pte_page(*pte), 0);
spin_lock(&init_mm.page_table_lock);
pte_clear(&init_mm, addr, pte);
page_addr = page_address(pte_page(*pte));
if (!memchr_inv(page_addr, PAGE_INUSE, PAGE_SIZE)) {
- free_pagetable(pte_page(*pte), 0, altmap);
+ free_pagetable(pte_page(*pte), 0);
spin_lock(&init_mm.page_table_lock);
pte_clear(&init_mm, addr, pte);
if (IS_ALIGNED(addr, PMD_SIZE) &&
IS_ALIGNED(next, PMD_SIZE)) {
if (!direct)
- free_pagetable(pmd_page(*pmd),
- get_order(PMD_SIZE),
- altmap);
+ free_hugepage_table(pmd_page(*pmd),
+ altmap);
spin_lock(&init_mm.page_table_lock);
pmd_clear(pmd);
page_addr = page_address(pmd_page(*pmd));
if (!memchr_inv(page_addr, PAGE_INUSE,
PMD_SIZE)) {
- free_pagetable(pmd_page(*pmd),
- get_order(PMD_SIZE),
- altmap);
+ free_hugepage_table(pmd_page(*pmd),
+ altmap);
spin_lock(&init_mm.page_table_lock);
pmd_clear(pmd);
}
pte_base = (pte_t *)pmd_page_vaddr(*pmd);
- remove_pte_table(pte_base, addr, next, altmap, direct);
- free_pte_table(pte_base, pmd, altmap);
+ remove_pte_table(pte_base, addr, next, direct);
+ free_pte_table(pte_base, pmd);
}
/* Call free_pmd_table() in remove_pud_table(). */
IS_ALIGNED(next, PUD_SIZE)) {
if (!direct)
free_pagetable(pud_page(*pud),
- get_order(PUD_SIZE),
- altmap);
+ get_order(PUD_SIZE));
spin_lock(&init_mm.page_table_lock);
pud_clear(pud);
if (!memchr_inv(page_addr, PAGE_INUSE,
PUD_SIZE)) {
free_pagetable(pud_page(*pud),
- get_order(PUD_SIZE),
- altmap);
+ get_order(PUD_SIZE));
spin_lock(&init_mm.page_table_lock);
pud_clear(pud);
pmd_base = pmd_offset(pud, 0);
remove_pmd_table(pmd_base, addr, next, direct, altmap);
- free_pmd_table(pmd_base, pud, altmap);
+ free_pmd_table(pmd_base, pud);
}
if (direct)
* to adapt for boot-time switching between 4 and 5 level page tables.
*/
if (CONFIG_PGTABLE_LEVELS == 5)
- free_pud_table(pud_base, p4d, altmap);
+ free_pud_table(pud_base, p4d);
}
if (direct)
return 0;
}
+
+/**
+ * pud_free_pmd_page - Clear pud entry and free pmd page.
+ * @pud: Pointer to a PUD.
+ *
+ * Context: The pud range has been unmaped and TLB purged.
+ * Return: 1 if clearing the entry succeeded. 0 otherwise.
+ */
+int pud_free_pmd_page(pud_t *pud)
+{
+ pmd_t *pmd;
+ int i;
+
+ if (pud_none(*pud))
+ return 1;
+
+ pmd = (pmd_t *)pud_page_vaddr(*pud);
+
+ for (i = 0; i < PTRS_PER_PMD; i++)
+ if (!pmd_free_pte_page(&pmd[i]))
+ return 0;
+
+ pud_clear(pud);
+ free_page((unsigned long)pmd);
+
+ return 1;
+}
+
+/**
+ * pmd_free_pte_page - Clear pmd entry and free pte page.
+ * @pmd: Pointer to a PMD.
+ *
+ * Context: The pmd range has been unmaped and TLB purged.
+ * Return: 1 if clearing the entry succeeded. 0 otherwise.
+ */
+int pmd_free_pte_page(pmd_t *pmd)
+{
+ pte_t *pte;
+
+ if (pmd_none(*pmd))
+ return 1;
+
+ pte = (pte_t *)pmd_page_vaddr(*pmd);
+ pmd_clear(pmd);
+ free_page((unsigned long)pte);
+
+ return 1;
+}
#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
}
/*
- * Clone the ESPFIX P4D into the user space visinble page table
+ * Clone the ESPFIX P4D into the user space visible page table
*/
static void __init pti_setup_espfix64(void)
{
* may converge on the last pass. In such case do one more
* pass to emit the final image
*/
- for (pass = 0; pass < 10 || image; pass++) {
+ for (pass = 0; pass < 20 || image; pass++) {
proglen = do_jit(prog, addrs, image, oldproglen, &ctx);
if (proglen <= 0) {
image = NULL;
}
}
oldproglen = proglen;
+ cond_resched();
}
if (bpf_jit_enable > 1)
res.start = gas->address;
if (gas->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) {
res.flags = IORESOURCE_MEM;
- res.end = res.start + ALIGN(gas->access_width, 4);
+ res.end = res.start + ALIGN(gas->access_width, 4) - 1;
} else if (gas->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
res.flags = IORESOURCE_IO;
- res.end = res.start + gas->access_width;
+ res.end = res.start + gas->access_width - 1;
} else {
pr_warn("Unsupported address space: %u\n",
gas->space_id);
static bool battery_driver_registered;
static int battery_bix_broken_package;
static int battery_notification_delay_ms;
-static int battery_full_discharging;
static unsigned int cache_time = 1000;
module_param(cache_time, uint, 0644);
MODULE_PARM_DESC(cache_time, "cache time in milliseconds");
return -ENODEV;
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
- if (battery->state & ACPI_BATTERY_STATE_DISCHARGING) {
- if (battery_full_discharging && battery->rate_now == 0)
- val->intval = POWER_SUPPLY_STATUS_FULL;
- else
- val->intval = POWER_SUPPLY_STATUS_DISCHARGING;
- } else if (battery->state & ACPI_BATTERY_STATE_CHARGING)
+ if (battery->state & ACPI_BATTERY_STATE_DISCHARGING)
+ val->intval = POWER_SUPPLY_STATUS_DISCHARGING;
+ else if (battery->state & ACPI_BATTERY_STATE_CHARGING)
val->intval = POWER_SUPPLY_STATUS_CHARGING;
else if (acpi_battery_is_charged(battery))
val->intval = POWER_SUPPLY_STATUS_FULL;
return 0;
}
-static int __init battery_full_discharging_quirk(const struct dmi_system_id *d)
-{
- battery_full_discharging = 1;
- return 0;
-}
-
static const struct dmi_system_id bat_dmi_table[] __initconst = {
{
.callback = battery_bix_broken_package_quirk,
DMI_MATCH(DMI_PRODUCT_NAME, "Aspire V5-573G"),
},
},
- {
- .callback = battery_full_discharging_quirk,
- .ident = "ASUS GL502VSK",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
- DMI_MATCH(DMI_PRODUCT_NAME, "GL502VSK"),
- },
- },
- {
- .callback = battery_full_discharging_quirk,
- .ident = "ASUS UX305LA",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
- DMI_MATCH(DMI_PRODUCT_NAME, "UX305LA"),
- },
- },
- {
- .callback = battery_full_discharging_quirk,
- .ident = "ASUS UX360UA",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
- DMI_MATCH(DMI_PRODUCT_NAME, "UX360UA"),
- },
- },
- {
- .callback = battery_full_discharging_quirk,
- .ident = "ASUS UX410UAK",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
- DMI_MATCH(DMI_PRODUCT_NAME, "UX410UAK"),
- },
- },
{},
};
else
ndr_desc->numa_node = NUMA_NO_NODE;
- if(acpi_desc->platform_cap & ACPI_NFIT_CAPABILITY_CACHE_FLUSH)
+ /*
+ * Persistence domain bits are hierarchical, if
+ * ACPI_NFIT_CAPABILITY_CACHE_FLUSH is set then
+ * ACPI_NFIT_CAPABILITY_MEM_FLUSH is implied.
+ */
+ if (acpi_desc->platform_cap & ACPI_NFIT_CAPABILITY_CACHE_FLUSH)
set_bit(ND_REGION_PERSIST_CACHE, &ndr_desc->flags);
-
- if (acpi_desc->platform_cap & ACPI_NFIT_CAPABILITY_MEM_FLUSH)
+ else if (acpi_desc->platform_cap & ACPI_NFIT_CAPABILITY_MEM_FLUSH)
set_bit(ND_REGION_PERSIST_MEMCTRL, &ndr_desc->flags);
list_for_each_entry(nfit_memdev, &acpi_desc->memdevs, list) {
*/
int acpi_map_pxm_to_online_node(int pxm)
{
- int node, n, dist, min_dist;
+ int node, min_node;
node = acpi_map_pxm_to_node(pxm);
if (node == NUMA_NO_NODE)
node = 0;
+ min_node = node;
if (!node_online(node)) {
- min_dist = INT_MAX;
+ int min_dist = INT_MAX, dist, n;
+
for_each_online_node(n) {
dist = node_distance(node, n);
if (dist < min_dist) {
min_dist = dist;
- node = n;
+ min_node = n;
}
}
}
- return node;
+ return min_node;
}
EXPORT_SYMBOL(acpi_map_pxm_to_online_node);
.driver_data = board_ahci_yes_fbs },
{ PCI_DEVICE(PCI_VENDOR_ID_MARVELL_EXT, 0x9230),
.driver_data = board_ahci_yes_fbs },
- { PCI_DEVICE(PCI_VENDOR_ID_TTI, 0x0642),
+ { PCI_DEVICE(PCI_VENDOR_ID_TTI, 0x0642), /* highpoint rocketraid 642L */
+ .driver_data = board_ahci_yes_fbs },
+ { PCI_DEVICE(PCI_VENDOR_ID_TTI, 0x0645), /* highpoint rocketraid 644L */
.driver_data = board_ahci_yes_fbs },
/* Promise */
if ((tmp & (PORT_CMD_START | PORT_CMD_LIST_ON)) == 0)
return 0;
+ /*
+ * Don't try to issue commands but return with ENODEV if the
+ * AHCI controller not available anymore (e.g. due to PCIe hot
+ * unplugging). Otherwise a 500ms delay for each port is added.
+ */
+ if (tmp == 0xffffffff) {
+ dev_err(ap->host->dev, "AHCI controller unavailable!\n");
+ return -ENODEV;
+ }
+
/* setting HBA to idle */
tmp &= ~PORT_CMD_START;
writel(tmp, port_mmio + PORT_CMD);
* 2) regulator for controlling the targets power (optional)
* 3) 0 - AHCI_MAX_CLKS clocks, as specified in the devs devicetree node,
* or for non devicetree enabled platforms a single clock
- * 4) phys (optional)
+ * 4) phys (optional)
*
* RETURNS:
* The allocated ahci_host_priv on success, otherwise an ERR_PTR value
{ "PIONEER DVD-RW DVR-212D", NULL, ATA_HORKAGE_NOSETXFER },
{ "PIONEER DVD-RW DVR-216D", NULL, ATA_HORKAGE_NOSETXFER },
+ /* Crucial BX100 SSD 500GB has broken LPM support */
+ { "CT500BX100SSD1", NULL, ATA_HORKAGE_NOLPM },
+
+ /* 512GB MX100 with MU01 firmware has both queued TRIM and LPM issues */
+ { "Crucial_CT512MX100*", "MU01", ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM |
+ ATA_HORKAGE_NOLPM, },
+ /* 512GB MX100 with newer firmware has only LPM issues */
+ { "Crucial_CT512MX100*", NULL, ATA_HORKAGE_ZERO_AFTER_TRIM |
+ ATA_HORKAGE_NOLPM, },
+
+ /* 480GB+ M500 SSDs have both queued TRIM and LPM issues */
+ { "Crucial_CT480M500*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM |
+ ATA_HORKAGE_NOLPM, },
+ { "Crucial_CT960M500*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM |
+ ATA_HORKAGE_NOLPM, },
+
/* devices that don't properly handle queued TRIM commands */
{ "Micron_M500_*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
ATA_HORKAGE_ZERO_AFTER_TRIM, },
ATA_HORKAGE_ZERO_AFTER_TRIM, },
{ "Crucial_CT*MX100*", "MU01", ATA_HORKAGE_NO_NCQ_TRIM |
ATA_HORKAGE_ZERO_AFTER_TRIM, },
- { "Samsung SSD 8*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
+ { "Samsung SSD 840*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM, },
+ { "Samsung SSD 850*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
ATA_HORKAGE_ZERO_AFTER_TRIM, },
{ "FCCT*M500*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
ATA_HORKAGE_ZERO_AFTER_TRIM, },
* We guarantee to LLDs that they will have at least one
* non-zero sg if the command is a data command.
*/
- if (WARN_ON_ONCE(ata_is_data(prot) &&
- (!qc->sg || !qc->n_elem || !qc->nbytes)))
+ if (ata_is_data(prot) && (!qc->sg || !qc->n_elem || !qc->nbytes))
goto sys_err;
if (ata_is_dma(prot) || (ata_is_pio(prot) &&
if (ap->pflags & ATA_PFLAG_LOADING)
ap->pflags &= ~ATA_PFLAG_LOADING;
- else if (ap->pflags & ATA_PFLAG_SCSI_HOTPLUG)
+ else if ((ap->pflags & ATA_PFLAG_SCSI_HOTPLUG) &&
+ !(ap->flags & ATA_FLAG_SAS_HOST))
schedule_delayed_work(&ap->hotplug_task, 0);
if (ap->pflags & ATA_PFLAG_RECOVERED)
goto invalid_fld;
}
+ /* We may not issue NCQ commands to devices not supporting NCQ */
+ if (ata_is_ncq(tf->protocol) && !ata_ncq_enabled(dev)) {
+ fp = 1;
+ goto invalid_fld;
+ }
+
/* sanity check for pio multi commands */
if ((cdb[1] & 0xe0) && !is_multi_taskfile(tf)) {
fp = 1;
#ifdef ATA_DEBUG
struct scsi_device *scsidev = cmd->device;
- DPRINTK("CDB (%u:%d,%d,%d) %9ph\n",
+ DPRINTK("CDB (%u:%d,%d,%lld) %9ph\n",
ap->print_id,
scsidev->channel, scsidev->id, scsidev->lun,
cmd->cmnd);
if (likely((scsi_op != ATA_16) || !atapi_passthru16)) {
/* relay SCSI command to ATAPI device */
int len = COMMAND_SIZE(scsi_op);
- if (unlikely(len > scmd->cmd_len || len > dev->cdb_len))
+ if (unlikely(len > scmd->cmd_len ||
+ len > dev->cdb_len ||
+ scmd->cmd_len > ATAPI_CDB_LEN))
goto bad_cdb_len;
xlat_func = atapi_xlat;
enum sata_rcar_type {
RCAR_GEN1_SATA,
RCAR_GEN2_SATA,
+ RCAR_GEN3_SATA,
RCAR_R8A7790_ES1_SATA,
};
ioaddr->command_addr = ioaddr->cmd_addr + (ATA_REG_CMD << 2);
}
-static void sata_rcar_init_controller(struct ata_host *host)
+static void sata_rcar_init_module(struct sata_rcar_priv *priv)
{
- struct sata_rcar_priv *priv = host->private_data;
void __iomem *base = priv->base;
u32 val;
- /* reset and setup phy */
- switch (priv->type) {
- case RCAR_GEN1_SATA:
- sata_rcar_gen1_phy_init(priv);
- break;
- case RCAR_GEN2_SATA:
- case RCAR_R8A7790_ES1_SATA:
- sata_rcar_gen2_phy_init(priv);
- break;
- default:
- dev_warn(host->dev, "SATA phy is not initialized\n");
- break;
- }
-
/* SATA-IP reset state */
val = ioread32(base + ATAPI_CONTROL1_REG);
val |= ATAPI_CONTROL1_RESET;
/* ack and mask */
iowrite32(0, base + SATAINTSTAT_REG);
iowrite32(0x7ff, base + SATAINTMASK_REG);
+
/* enable interrupts */
iowrite32(ATAPI_INT_ENABLE_SATAINT, base + ATAPI_INT_ENABLE_REG);
}
+static void sata_rcar_init_controller(struct ata_host *host)
+{
+ struct sata_rcar_priv *priv = host->private_data;
+
+ /* reset and setup phy */
+ switch (priv->type) {
+ case RCAR_GEN1_SATA:
+ sata_rcar_gen1_phy_init(priv);
+ break;
+ case RCAR_GEN2_SATA:
+ case RCAR_GEN3_SATA:
+ case RCAR_R8A7790_ES1_SATA:
+ sata_rcar_gen2_phy_init(priv);
+ break;
+ default:
+ dev_warn(host->dev, "SATA phy is not initialized\n");
+ break;
+ }
+
+ sata_rcar_init_module(priv);
+}
+
static const struct of_device_id sata_rcar_match[] = {
{
/* Deprecated by "renesas,sata-r8a7779" */
},
{
.compatible = "renesas,sata-r8a7795",
- .data = (void *)RCAR_GEN2_SATA
+ .data = (void *)RCAR_GEN3_SATA
},
{
.compatible = "renesas,rcar-gen2-sata",
},
{
.compatible = "renesas,rcar-gen3-sata",
- .data = (void *)RCAR_GEN2_SATA
+ .data = (void *)RCAR_GEN3_SATA
},
{ },
};
if (ret)
return ret;
- /* ack and mask */
- iowrite32(0, base + SATAINTSTAT_REG);
- iowrite32(0x7ff, base + SATAINTMASK_REG);
- /* enable interrupts */
- iowrite32(ATAPI_INT_ENABLE_SATAINT, base + ATAPI_INT_ENABLE_REG);
+ if (priv->type == RCAR_GEN3_SATA) {
+ sata_rcar_gen2_phy_init(priv);
+ sata_rcar_init_module(priv);
+ } else {
+ /* ack and mask */
+ iowrite32(0, base + SATAINTSTAT_REG);
+ iowrite32(0x7ff, base + SATAINTMASK_REG);
+
+ /* enable interrupts */
+ iowrite32(ATAPI_INT_ENABLE_SATAINT,
+ base + ATAPI_INT_ENABLE_REG);
+ }
ata_host_resume(host);
static void malta_update(struct img_ascii_lcd_ctx *ctx)
{
unsigned int i;
- int err;
+ int err = 0;
for (i = 0; i < ctx->cfg->num_chars; i++) {
err = regmap_write(ctx->regmap,
static void sead3_update(struct img_ascii_lcd_ctx *ctx)
{
unsigned int i;
- int err;
+ int err = 0;
for (i = 0; i < ctx->cfg->num_chars; i++) {
err = sead3_wait_lcd_idle(ctx);
/**
* img_ascii_lcd_scroll() - scroll the display by a character
- * @arg: really a pointer to the private data structure
+ * @t: really a pointer to the private data structure
*
* Scroll the current message along the LCD by one character, rearming the
* timer if required.
break;
input->rise_timer = 0;
input->state = INPUT_ST_RISING;
- /* no break here, fall through */
+ /* fall through */
case INPUT_ST_RISING:
if ((phys_curr & input->mask) != input->value) {
input->state = INPUT_ST_LOW;
}
input->high_timer = 0;
input->state = INPUT_ST_HIGH;
- /* no break here, fall through */
+ /* fall through */
case INPUT_ST_HIGH:
if (input_state_high(input))
break;
- /* no break here, fall through */
+ /* fall through */
case INPUT_ST_FALLING:
input_state_falling(input);
}
config BCMA_DRIVER_PCI_HOSTMODE
bool "Driver for PCI core working in hostmode"
- depends on MIPS && BCMA_DRIVER_PCI && PCI_DRIVERS_LEGACY
+ depends on MIPS && BCMA_DRIVER_PCI && PCI_DRIVERS_LEGACY && BCMA = y
help
PCI core hostmode operation (external PCI bus).
* Add some delay; allow resources to come up and settle.
* Delay is required for SoC (early init).
*/
- mdelay(2);
+ usleep_range(2000, 2500);
}
/* Disable to allow reading SPROM. Don't know the adventages of enabling it. */
{ PCI_DEVICE_SUB(PCI_VENDOR_ID_BROADCOM, 0x4365, PCI_VENDOR_ID_DELL, 0x0016) },
{ PCI_DEVICE_SUB(PCI_VENDOR_ID_BROADCOM, 0x4365, PCI_VENDOR_ID_DELL, 0x0018) },
{ PCI_DEVICE_SUB(PCI_VENDOR_ID_BROADCOM, 0x4365, PCI_VENDOR_ID_FOXCONN, 0xe092) },
+ { PCI_DEVICE_SUB(PCI_VENDOR_ID_BROADCOM, 0x4365, PCI_VENDOR_ID_HP, 0x804a) },
{ PCI_DEVICE(PCI_VENDOR_ID_BROADCOM, 0x43a0) },
{ PCI_DEVICE(PCI_VENDOR_ID_BROADCOM, 0x43a9) },
{ PCI_DEVICE(PCI_VENDOR_ID_BROADCOM, 0x43aa) },
struct iov_iter i;
ssize_t bw;
- iov_iter_bvec(&i, ITER_BVEC, bvec, 1, bvec->bv_len);
+ iov_iter_bvec(&i, ITER_BVEC | WRITE, bvec, 1, bvec->bv_len);
file_start_write(file);
bw = vfs_iter_write(file, &i, ppos, 0);
static int blkfront_setup_indirect(struct blkfront_ring_info *rinfo);
static void blkfront_gather_backend_features(struct blkfront_info *info);
+static int negotiate_mq(struct blkfront_info *info);
static int get_id_from_freelist(struct blkfront_ring_info *rinfo)
{
unsigned int i, max_page_order;
unsigned int ring_page_order;
+ if (!info)
+ return -ENODEV;
+
max_page_order = xenbus_read_unsigned(info->xbdev->otherend,
"max-ring-page-order", 0);
ring_page_order = min(xen_blkif_max_ring_order, max_page_order);
info->nr_ring_pages = 1 << ring_page_order;
+ err = negotiate_mq(info);
+ if (err)
+ goto destroy_blkring;
+
for (i = 0; i < info->nr_rings; i++) {
struct blkfront_ring_info *rinfo = &info->rinfo[i];
}
info->xbdev = dev;
- err = negotiate_mq(info);
- if (err) {
- kfree(info);
- return err;
- }
mutex_init(&info->mutex);
info->vdevice = vdevice;
blkif_free(info, info->connected == BLKIF_STATE_CONNECTED);
- err = negotiate_mq(info);
- if (err)
- return err;
-
err = talk_to_blkback(dev, info);
if (!err)
blk_mq_update_nr_hw_queues(&info->tag_set, info->nr_rings);
Say Y here to compile support for HCI over Qualcomm SMD into the
kernel or say M to compile as a module.
+config BT_HCIRSI
+ tristate "Redpine HCI support"
+ default n
+ select RSI_COEX
+ help
+ Redpine BT driver.
+ This driver handles BT traffic from upper layers and pass
+ to the RSI_91x coex module for further scheduling to device
+
+ Say Y here to compile support for HCI over Redpine into the
+ kernel or say M to compile as a module.
+
endmenu
obj-$(CONFIG_BT_HCIUART_NOKIA) += hci_nokia.o
+obj-$(CONFIG_BT_HCIRSI) += btrsi.o
+
btmrvl-y := btmrvl_main.o
btmrvl-$(CONFIG_DEBUG_FS) += btmrvl_debugfs.o
--- /dev/null
+/**
+ * Copyright (c) 2017 Redpine Signals Inc.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+#include <linux/version.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <net/bluetooth/bluetooth.h>
+#include <net/bluetooth/hci_core.h>
+#include <asm/unaligned.h>
+#include <net/rsi_91x.h>
+#include <net/genetlink.h>
+
+#define RSI_HEADROOM_FOR_BT_HAL 16
+#define RSI_FRAME_DESC_SIZE 16
+
+struct rsi_hci_adapter {
+ void *priv;
+ struct rsi_proto_ops *proto_ops;
+ struct hci_dev *hdev;
+};
+
+static int rsi_hci_open(struct hci_dev *hdev)
+{
+ return 0;
+}
+
+static int rsi_hci_close(struct hci_dev *hdev)
+{
+ return 0;
+}
+
+static int rsi_hci_flush(struct hci_dev *hdev)
+{
+ return 0;
+}
+
+static int rsi_hci_send_pkt(struct hci_dev *hdev, struct sk_buff *skb)
+{
+ struct rsi_hci_adapter *h_adapter = hci_get_drvdata(hdev);
+ struct sk_buff *new_skb = NULL;
+
+ switch (hci_skb_pkt_type(skb)) {
+ case HCI_COMMAND_PKT:
+ hdev->stat.cmd_tx++;
+ break;
+ case HCI_ACLDATA_PKT:
+ hdev->stat.acl_tx++;
+ break;
+ case HCI_SCODATA_PKT:
+ hdev->stat.sco_tx++;
+ break;
+ }
+
+ if (skb_headroom(skb) < RSI_HEADROOM_FOR_BT_HAL) {
+ /* Insufficient skb headroom - allocate a new skb */
+ new_skb = skb_realloc_headroom(skb, RSI_HEADROOM_FOR_BT_HAL);
+ if (unlikely(!new_skb))
+ return -ENOMEM;
+ bt_cb(new_skb)->pkt_type = hci_skb_pkt_type(skb);
+ kfree_skb(skb);
+ skb = new_skb;
+ }
+
+ return h_adapter->proto_ops->coex_send_pkt(h_adapter->priv, skb,
+ RSI_BT_Q);
+}
+
+static int rsi_hci_recv_pkt(void *priv, const u8 *pkt)
+{
+ struct rsi_hci_adapter *h_adapter = priv;
+ struct hci_dev *hdev = h_adapter->hdev;
+ struct sk_buff *skb;
+ int pkt_len = get_unaligned_le16(pkt) & 0x0fff;
+
+ skb = dev_alloc_skb(pkt_len);
+ if (!skb)
+ return -ENOMEM;
+
+ memcpy(skb->data, pkt + RSI_FRAME_DESC_SIZE, pkt_len);
+ skb_put(skb, pkt_len);
+ h_adapter->hdev->stat.byte_rx += skb->len;
+
+ hci_skb_pkt_type(skb) = pkt[14];
+
+ return hci_recv_frame(hdev, skb);
+}
+
+static int rsi_hci_attach(void *priv, struct rsi_proto_ops *ops)
+{
+ struct rsi_hci_adapter *h_adapter = NULL;
+ struct hci_dev *hdev;
+ int err = 0;
+
+ h_adapter = kzalloc(sizeof(*h_adapter), GFP_KERNEL);
+ if (!h_adapter)
+ return -ENOMEM;
+
+ h_adapter->priv = priv;
+ ops->set_bt_context(priv, h_adapter);
+ h_adapter->proto_ops = ops;
+
+ hdev = hci_alloc_dev();
+ if (!hdev) {
+ BT_ERR("Failed to alloc HCI device");
+ goto err;
+ }
+
+ h_adapter->hdev = hdev;
+
+ if (ops->get_host_intf(priv) == RSI_HOST_INTF_SDIO)
+ hdev->bus = HCI_SDIO;
+ else
+ hdev->bus = HCI_USB;
+
+ hci_set_drvdata(hdev, h_adapter);
+ hdev->dev_type = HCI_PRIMARY;
+ hdev->open = rsi_hci_open;
+ hdev->close = rsi_hci_close;
+ hdev->flush = rsi_hci_flush;
+ hdev->send = rsi_hci_send_pkt;
+
+ err = hci_register_dev(hdev);
+ if (err < 0) {
+ BT_ERR("HCI registration failed with errcode %d", err);
+ hci_free_dev(hdev);
+ goto err;
+ }
+
+ return 0;
+err:
+ h_adapter->hdev = NULL;
+ kfree(h_adapter);
+ return -EINVAL;
+}
+
+static void rsi_hci_detach(void *priv)
+{
+ struct rsi_hci_adapter *h_adapter = priv;
+ struct hci_dev *hdev;
+
+ if (!h_adapter)
+ return;
+
+ hdev = h_adapter->hdev;
+ if (hdev) {
+ hci_unregister_dev(hdev);
+ hci_free_dev(hdev);
+ h_adapter->hdev = NULL;
+ }
+
+ kfree(h_adapter);
+}
+
+const struct rsi_mod_ops rsi_bt_ops = {
+ .attach = rsi_hci_attach,
+ .detach = rsi_hci_detach,
+ .recv_pkt = rsi_hci_recv_pkt,
+};
+EXPORT_SYMBOL(rsi_bt_ops);
+
+static int rsi_91x_bt_module_init(void)
+{
+ return 0;
+}
+
+static void rsi_91x_bt_module_exit(void)
+{
+ return;
+}
+
+module_init(rsi_91x_bt_module_init);
+module_exit(rsi_91x_bt_module_exit);
+MODULE_AUTHOR("Redpine Signals Inc");
+MODULE_DESCRIPTION("RSI BT driver");
+MODULE_SUPPORTED_DEVICE("RSI-BT");
+MODULE_LICENSE("Dual BSD/GPL");
{ USB_DEVICE(0x0930, 0x0227), .driver_info = BTUSB_ATH3012 },
{ USB_DEVICE(0x0b05, 0x17d0), .driver_info = BTUSB_ATH3012 },
{ USB_DEVICE(0x0cf3, 0x0036), .driver_info = BTUSB_ATH3012 },
- { USB_DEVICE(0x0cf3, 0x3004), .driver_info = BTUSB_ATH3012 },
{ USB_DEVICE(0x0cf3, 0x3008), .driver_info = BTUSB_ATH3012 },
{ USB_DEVICE(0x0cf3, 0x311d), .driver_info = BTUSB_ATH3012 },
{ USB_DEVICE(0x0cf3, 0x311e), .driver_info = BTUSB_ATH3012 },
{ USB_DEVICE(0x0489, 0xe03c), .driver_info = BTUSB_ATH3012 },
/* QCA ROME chipset */
+ { USB_DEVICE(0x0cf3, 0x3004), .driver_info = BTUSB_QCA_ROME },
{ USB_DEVICE(0x0cf3, 0xe007), .driver_info = BTUSB_QCA_ROME },
{ USB_DEVICE(0x0cf3, 0xe009), .driver_info = BTUSB_QCA_ROME },
{ USB_DEVICE(0x0cf3, 0xe010), .driver_info = BTUSB_QCA_ROME },
*/
static const struct dmi_system_id btusb_needs_reset_resume_table[] = {
{
- /* Lenovo Yoga 920 (QCA Rome device 0cf3:e300) */
+ /* Dell OptiPlex 3060 (QCA ROME device 0cf3:e007) */
.matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "LENOVO"),
- DMI_MATCH(DMI_PRODUCT_VERSION, "Lenovo YOGA 920"),
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "OptiPlex 3060"),
},
},
{}
bt_dev_dbg(bdev, "Host wake IRQ");
- pm_request_resume(bdev->dev);
+ pm_runtime_get(bdev->dev);
+ pm_runtime_mark_last_busy(bdev->dev);
+ pm_runtime_put_autosuspend(bdev->dev);
return IRQ_HANDLED;
}
.usb_auto_sleep = 0,
.usb_resume_timeout = 0,
.break_to_host = 0,
- .pulsed_host_wake = 0,
+ .pulsed_host_wake = 1,
};
static int bcm_setup_sleep(struct hci_uart *hu)
} else if (!bcm->rx_skb) {
/* Delay auto-suspend when receiving completed packet */
mutex_lock(&bcm_device_lock);
- if (bcm->dev && bcm_device_exists(bcm->dev))
- pm_request_resume(bcm->dev->dev);
+ if (bcm->dev && bcm_device_exists(bcm->dev)) {
+ pm_runtime_get(bcm->dev->dev);
+ pm_runtime_mark_last_busy(bcm->dev->dev);
+ pm_runtime_put_autosuspend(bcm->dev->dev);
+ }
mutex_unlock(&bcm_device_lock);
}
static const struct bcm2835_pll_ana_bits bcm2835_ana_default = {
.mask0 = 0,
.set0 = 0,
- .mask1 = (u32)~(A2W_PLL_KI_MASK | A2W_PLL_KP_MASK),
+ .mask1 = A2W_PLL_KI_MASK | A2W_PLL_KP_MASK,
.set1 = (2 << A2W_PLL_KI_SHIFT) | (8 << A2W_PLL_KP_SHIFT),
- .mask3 = (u32)~A2W_PLL_KA_MASK,
+ .mask3 = A2W_PLL_KA_MASK,
.set3 = (2 << A2W_PLL_KA_SHIFT),
.fb_prediv_mask = BIT(14),
};
static const struct bcm2835_pll_ana_bits bcm2835_ana_pllh = {
- .mask0 = (u32)~(A2W_PLLH_KA_MASK | A2W_PLLH_KI_LOW_MASK),
+ .mask0 = A2W_PLLH_KA_MASK | A2W_PLLH_KI_LOW_MASK,
.set0 = (2 << A2W_PLLH_KA_SHIFT) | (2 << A2W_PLLH_KI_LOW_SHIFT),
- .mask1 = (u32)~(A2W_PLLH_KI_HIGH_MASK | A2W_PLLH_KP_MASK),
+ .mask1 = A2W_PLLH_KI_HIGH_MASK | A2W_PLLH_KP_MASK,
.set1 = (6 << A2W_PLLH_KP_SHIFT),
.mask3 = 0,
.set3 = 0,
~A2W_PLL_CTRL_PWRDN);
/* Take the PLL out of reset. */
+ spin_lock(&cprman->regs_lock);
cprman_write(cprman, data->cm_ctrl_reg,
cprman_read(cprman, data->cm_ctrl_reg) & ~CM_PLL_ANARST);
+ spin_unlock(&cprman->regs_lock);
/* Wait for the PLL to lock. */
timeout = ktime_add_ns(ktime_get(), LOCK_TIMEOUT_NS);
}
/* Unmask the reference clock from the oscillator. */
+ spin_lock(&cprman->regs_lock);
cprman_write(cprman, A2W_XOSC_CTRL,
cprman_read(cprman, A2W_XOSC_CTRL) |
data->reference_enable_mask);
+ spin_unlock(&cprman->regs_lock);
if (do_ana_setup_first)
bcm2835_pll_write_ana(cprman, data->ana_reg_base, ana);
.calc_pll = aspeed_ast2400_calc_pll,
};
+static int aspeed_clk_is_enabled(struct clk_hw *hw)
+{
+ struct aspeed_clk_gate *gate = to_aspeed_clk_gate(hw);
+ u32 clk = BIT(gate->clock_idx);
+ u32 enval = (gate->flags & CLK_GATE_SET_TO_DISABLE) ? 0 : clk;
+ u32 reg;
+
+ regmap_read(gate->map, ASPEED_CLK_STOP_CTRL, ®);
+
+ return ((reg & clk) == enval) ? 1 : 0;
+}
+
static int aspeed_clk_enable(struct clk_hw *hw)
{
struct aspeed_clk_gate *gate = to_aspeed_clk_gate(hw);
spin_lock_irqsave(gate->lock, flags);
+ if (aspeed_clk_is_enabled(hw)) {
+ spin_unlock_irqrestore(gate->lock, flags);
+ return 0;
+ }
+
if (gate->reset_idx >= 0) {
/* Put IP in reset */
regmap_update_bits(gate->map, ASPEED_RESET_CTRL, rst, rst);
spin_unlock_irqrestore(gate->lock, flags);
}
-static int aspeed_clk_is_enabled(struct clk_hw *hw)
-{
- struct aspeed_clk_gate *gate = to_aspeed_clk_gate(hw);
- u32 clk = BIT(gate->clock_idx);
- u32 reg;
-
- regmap_read(gate->map, ASPEED_CLK_STOP_CTRL, ®);
-
- return (reg & clk) ? 0 : 1;
-}
-
static const struct clk_ops aspeed_clk_gate_ops = {
.enable = aspeed_clk_enable,
.disable = aspeed_clk_disable,
{
lockdep_assert_held(&prepare_lock);
- if (!core)
+ if (!core) {
+ req->rate = 0;
return 0;
+ }
clk_core_init_rate_req(core, req);
trace_clk_set_phase(core, degrees);
- if (core->ops->set_phase)
+ if (core->ops->set_phase) {
ret = core->ops->set_phase(core->hw, degrees);
+ if (!ret)
+ core->phase = degrees;
+ }
trace_clk_set_phase_complete(core, degrees);
rate = 0;
core->rate = core->req_rate = rate;
+ /*
+ * Enable CLK_IS_CRITICAL clocks so newly added critical clocks
+ * don't get accidentally disabled when walking the orphan tree and
+ * reparenting clocks
+ */
+ if (core->flags & CLK_IS_CRITICAL) {
+ unsigned long flags;
+
+ clk_core_prepare(core);
+
+ flags = clk_enable_lock();
+ clk_core_enable(core);
+ clk_enable_unlock(flags);
+ }
+
/*
* walk the list of orphan clocks and reparent any that newly finds a
* parent.
*/
hlist_for_each_entry_safe(orphan, tmp2, &clk_orphan_list, child_node) {
struct clk_core *parent = __clk_init_parent(orphan);
- unsigned long flags;
/*
- * we could call __clk_set_parent, but that would result in a
- * redundant call to the .set_rate op, if it exists
+ * We need to use __clk_set_parent_before() and _after() to
+ * to properly migrate any prepare/enable count of the orphan
+ * clock. This is important for CLK_IS_CRITICAL clocks, which
+ * are enabled during init but might not have a parent yet.
*/
if (parent) {
/* update the clk tree topology */
- flags = clk_enable_lock();
- clk_reparent(orphan, parent);
- clk_enable_unlock(flags);
+ __clk_set_parent_before(orphan, parent);
+ __clk_set_parent_after(orphan, parent, NULL);
__clk_recalc_accuracies(orphan);
__clk_recalc_rates(orphan, 0);
}
if (core->ops->init)
core->ops->init(core->hw);
- if (core->flags & CLK_IS_CRITICAL) {
- unsigned long flags;
-
- clk_core_prepare(core);
-
- flags = clk_enable_lock();
- clk_core_enable(core);
- clk_enable_unlock(flags);
- }
-
kref_init(&core->ref);
out:
clk_pm_runtime_put(core);
return PTR_ERR(stub_clk_chan.mbox);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res)
+ return -EINVAL;
freq_reg = devm_ioremap(dev, res->start, resource_size(res));
if (!freq_reg)
return -ENOMEM;
static struct clk *clk[IMX5_CLK_END];
static struct clk_onecell_data clk_data;
-static struct clk ** const uart_clks[] __initconst = {
+static struct clk ** const uart_clks_mx51[] __initconst = {
+ &clk[IMX5_CLK_UART1_IPG_GATE],
+ &clk[IMX5_CLK_UART1_PER_GATE],
+ &clk[IMX5_CLK_UART2_IPG_GATE],
+ &clk[IMX5_CLK_UART2_PER_GATE],
+ &clk[IMX5_CLK_UART3_IPG_GATE],
+ &clk[IMX5_CLK_UART3_PER_GATE],
+ NULL
+};
+
+static struct clk ** const uart_clks_mx50_mx53[] __initconst = {
&clk[IMX5_CLK_UART1_IPG_GATE],
&clk[IMX5_CLK_UART1_PER_GATE],
&clk[IMX5_CLK_UART2_IPG_GATE],
clk_prepare_enable(clk[IMX5_CLK_TMAX1]);
clk_prepare_enable(clk[IMX5_CLK_TMAX2]); /* esdhc2, fec */
clk_prepare_enable(clk[IMX5_CLK_TMAX3]); /* esdhc1, esdhc4 */
-
- imx_register_uart_clocks(uart_clks);
}
static void __init mx50_clocks_init(struct device_node *np)
r = clk_round_rate(clk[IMX5_CLK_USBOH3_PER_GATE], 54000000);
clk_set_rate(clk[IMX5_CLK_USBOH3_PER_GATE], r);
+
+ imx_register_uart_clocks(uart_clks_mx50_mx53);
}
CLK_OF_DECLARE(imx50_ccm, "fsl,imx50-ccm", mx50_clocks_init);
val = readl(MXC_CCM_CLPCR);
val |= 1 << 23;
writel(val, MXC_CCM_CLPCR);
+
+ imx_register_uart_clocks(uart_clks_mx51);
}
CLK_OF_DECLARE(imx51_ccm, "fsl,imx51-ccm", mx51_clocks_init);
r = clk_round_rate(clk[IMX5_CLK_USBOH3_PER_GATE], 54000000);
clk_set_rate(clk[IMX5_CLK_USBOH3_PER_GATE], r);
+
+ imx_register_uart_clocks(uart_clks_mx50_mx53);
}
CLK_OF_DECLARE(imx53_ccm, "fsl,imx53-ccm", mx53_clocks_init);
struct clk_regmap_mux_div *a53cc;
struct regmap *regmap;
struct clk_init_data init = { };
- int ret;
+ int ret = -ENODEV;
regmap = dev_get_regmap(parent, NULL);
- if (IS_ERR(regmap)) {
- ret = PTR_ERR(regmap);
+ if (!regmap) {
dev_err(dev, "failed to get regmap: %d\n", ret);
return ret;
}
.features = CCU_FEATURE_FIXED_PREDIV,
.hw.init = CLK_HW_INIT_PARENTS("out-a",
clk_out_parents,
- &ccu_div_ops,
+ &ccu_mp_ops,
0),
},
};
.features = CCU_FEATURE_FIXED_PREDIV,
.hw.init = CLK_HW_INIT_PARENTS("out-b",
clk_out_parents,
- &ccu_div_ops,
+ &ccu_mp_ops,
0),
},
};
.features = CCU_FEATURE_FIXED_PREDIV,
.hw.init = CLK_HW_INIT_PARENTS("out-c",
clk_out_parents,
- &ccu_div_ops,
+ &ccu_mp_ops,
0),
},
};
static const struct omap_clkctrl_reg_data am3_l4_per_clkctrl_regs[] __initconst = {
{ AM3_CPGMAC0_CLKCTRL, NULL, CLKF_SW_SUP, "cpsw_125mhz_gclk", "cpsw_125mhz_clkdm" },
- { AM3_LCDC_CLKCTRL, NULL, CLKF_SW_SUP, "lcd_gclk", "lcdc_clkdm" },
+ { AM3_LCDC_CLKCTRL, NULL, CLKF_SW_SUP | CLKF_SET_RATE_PARENT, "lcd_gclk", "lcdc_clkdm" },
{ AM3_USB_OTG_HS_CLKCTRL, NULL, CLKF_SW_SUP, "usbotg_fck", "l3s_clkdm" },
{ AM3_TPTC0_CLKCTRL, NULL, CLKF_SW_SUP, "l3_gclk", "l3_clkdm" },
{ AM3_EMIF_CLKCTRL, NULL, CLKF_SW_SUP, "dpll_ddr_m2_div2_ck", "l3_clkdm" },
{ AM4_OCP2SCP0_CLKCTRL, NULL, CLKF_SW_SUP, "l4ls_gclk" },
{ AM4_OCP2SCP1_CLKCTRL, NULL, CLKF_SW_SUP, "l4ls_gclk" },
{ AM4_EMIF_CLKCTRL, NULL, CLKF_SW_SUP, "dpll_ddr_m2_ck", "emif_clkdm" },
- { AM4_DSS_CORE_CLKCTRL, NULL, CLKF_SW_SUP, "disp_clk", "dss_clkdm" },
+ { AM4_DSS_CORE_CLKCTRL, NULL, CLKF_SW_SUP | CLKF_SET_RATE_PARENT, "disp_clk", "dss_clkdm" },
{ AM4_CPGMAC0_CLKCTRL, NULL, CLKF_SW_SUP, "cpsw_125mhz_gclk", "cpsw_125mhz_clkdm" },
{ 0 },
};
init.parent_names = ®_data->parent;
init.num_parents = 1;
init.flags = 0;
+ if (reg_data->flags & CLKF_SET_RATE_PARENT)
+ init.flags |= CLK_SET_RATE_PARENT;
init.name = kasprintf(GFP_KERNEL, "%s:%s:%04x:%d",
node->parent->name, node->name,
reg_data->offset, 0);
config ATMEL_ST
bool "Atmel ST timer support" if COMPILE_TEST
+ depends on HAS_IOMEM
select TIMER_OF
select MFD_SYSCON
help
void __iomem *dma_base;
void __iomem *glob_base;
struct clk *clk;
+ struct clk *reg_clk;
struct tasklet_struct irq_tasklet;
struct list_head free_sw_desc;
struct dma_device dmadev;
if (ret)
return ret;
+ xor_dev->reg_clk = devm_clk_get(&pdev->dev, "reg");
+ if (PTR_ERR(xor_dev->reg_clk) != -ENOENT) {
+ if (!IS_ERR(xor_dev->reg_clk)) {
+ ret = clk_prepare_enable(xor_dev->reg_clk);
+ if (ret)
+ return ret;
+ } else {
+ return PTR_ERR(xor_dev->reg_clk);
+ }
+ }
+
xor_dev->clk = devm_clk_get(&pdev->dev, NULL);
- if (IS_ERR(xor_dev->clk) && PTR_ERR(xor_dev->clk) == -EPROBE_DEFER)
- return -EPROBE_DEFER;
+ if (IS_ERR(xor_dev->clk) && PTR_ERR(xor_dev->clk) == -EPROBE_DEFER) {
+ ret = EPROBE_DEFER;
+ goto disable_reg_clk;
+ }
if (!IS_ERR(xor_dev->clk)) {
ret = clk_prepare_enable(xor_dev->clk);
if (ret)
- return ret;
+ goto disable_reg_clk;
}
ret = platform_msi_domain_alloc_irqs(&pdev->dev, 1,
free_msi_irqs:
platform_msi_domain_free_irqs(&pdev->dev);
disable_clk:
- if (!IS_ERR(xor_dev->clk))
- clk_disable_unprepare(xor_dev->clk);
+ clk_disable_unprepare(xor_dev->clk);
+disable_reg_clk:
+ clk_disable_unprepare(xor_dev->reg_clk);
return ret;
}
rcar_dmac_chan_configure_desc(chan, desc);
- max_chunk_size = (RCAR_DMATCR_MASK + 1) << desc->xfer_shift;
+ max_chunk_size = RCAR_DMATCR_MASK << desc->xfer_shift;
/*
* Allocate and fill the transfer chunk descriptors. We own the only
platform_driver_unregister(&dcdbas_driver);
}
-module_init(dcdbas_init);
+subsys_initcall_sync(dcdbas_init);
module_exit(dcdbas_exit);
MODULE_DESCRIPTION(DRIVER_DESCRIPTION " (version " DRIVER_VERSION ")");
efi_guid_t linux_eventlog_guid = LINUX_EFI_TPM_EVENT_LOG_GUID;
efi_status_t status;
efi_physical_addr_t log_location, log_last_entry;
- struct linux_efi_tpm_eventlog *log_tbl;
+ struct linux_efi_tpm_eventlog *log_tbl = NULL;
unsigned long first_entry_addr, last_entry_addr;
size_t log_size, last_entry_size;
efi_bool_t truncated;
- void *tcg2_protocol;
+ void *tcg2_protocol = NULL;
status = efi_call_early(locate_protocol, &tcg2_guid, NULL,
&tcg2_protocol);
* GNU General Public License for more details.
*/
-#include <linux/clk.h>
#include <linux/err.h>
#include <linux/gpio.h>
#include <linux/init.h>
struct platform_device *pdev;
struct gpio_chip gpio_chip;
struct irq_chip irq_chip;
- struct clk *clk;
unsigned int irq_parent;
+ atomic_t wakeup_path;
bool has_both_edge_trigger;
- bool needs_clk;
};
#define IOINTSEL 0x00 /* General IO/Interrupt Switching Register */
}
}
- if (!p->clk)
- return 0;
-
if (on)
- clk_enable(p->clk);
+ atomic_inc(&p->wakeup_path);
else
- clk_disable(p->clk);
+ atomic_dec(&p->wakeup_path);
return 0;
}
struct gpio_rcar_info {
bool has_both_edge_trigger;
- bool needs_clk;
};
static const struct gpio_rcar_info gpio_rcar_info_gen1 = {
.has_both_edge_trigger = false,
- .needs_clk = false,
};
static const struct gpio_rcar_info gpio_rcar_info_gen2 = {
.has_both_edge_trigger = true,
- .needs_clk = true,
};
static const struct of_device_id gpio_rcar_of_table[] = {
ret = of_parse_phandle_with_fixed_args(np, "gpio-ranges", 3, 0, &args);
*npins = ret == 0 ? args.args[2] : RCAR_MAX_GPIO_PER_BANK;
p->has_both_edge_trigger = info->has_both_edge_trigger;
- p->needs_clk = info->needs_clk;
if (*npins == 0 || *npins > RCAR_MAX_GPIO_PER_BANK) {
dev_warn(&p->pdev->dev,
platform_set_drvdata(pdev, p);
- p->clk = devm_clk_get(dev, NULL);
- if (IS_ERR(p->clk)) {
- if (p->needs_clk) {
- dev_err(dev, "unable to get clock\n");
- ret = PTR_ERR(p->clk);
- goto err0;
- }
- p->clk = NULL;
- }
-
pm_runtime_enable(dev);
irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
return 0;
}
+static int __maybe_unused gpio_rcar_suspend(struct device *dev)
+{
+ struct gpio_rcar_priv *p = dev_get_drvdata(dev);
+
+ if (atomic_read(&p->wakeup_path))
+ device_set_wakeup_path(dev);
+
+ return 0;
+}
+
+static SIMPLE_DEV_PM_OPS(gpio_rcar_pm_ops, gpio_rcar_suspend, NULL);
+
static struct platform_driver gpio_rcar_device_driver = {
.probe = gpio_rcar_probe,
.remove = gpio_rcar_remove,
.driver = {
.name = "gpio_rcar",
+ .pm = &gpio_rcar_pm_ops,
.of_match_table = of_match_ptr(gpio_rcar_of_table),
}
};
size_t size;
u32 retry = 3;
+ if (amdgpu_acpi_pcie_notify_device_ready(adev))
+ return -EINVAL;
+
/* Get the device handle */
handle = ACPI_HANDLE(&adev->pdev->dev);
if (!handle)
/* don't do anything if sink is not display port, i.e.,
* passive dp->(dvi|hdmi) adaptor
*/
- if (dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT) {
- int saved_dpms = connector->dpms;
- /* Only turn off the display if it's physically disconnected */
- if (!amdgpu_display_hpd_sense(adev, amdgpu_connector->hpd.hpd)) {
- drm_helper_connector_dpms(connector, DRM_MODE_DPMS_OFF);
- } else if (amdgpu_atombios_dp_needs_link_train(amdgpu_connector)) {
- /* Don't try to start link training before we
- * have the dpcd */
- if (amdgpu_atombios_dp_get_dpcd(amdgpu_connector))
- return;
-
- /* set it to OFF so that drm_helper_connector_dpms()
- * won't return immediately since the current state
- * is ON at this point.
- */
- connector->dpms = DRM_MODE_DPMS_OFF;
- drm_helper_connector_dpms(connector, DRM_MODE_DPMS_ON);
- }
- connector->dpms = saved_dpms;
+ if (dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT &&
+ amdgpu_display_hpd_sense(adev, amdgpu_connector->hpd.hpd) &&
+ amdgpu_atombios_dp_needs_link_train(amdgpu_connector)) {
+ /* Don't start link training before we have the DPCD */
+ if (amdgpu_atombios_dp_get_dpcd(amdgpu_connector))
+ return;
+
+ /* Turn the connector off and back on immediately, which
+ * will trigger link training
+ */
+ drm_helper_connector_dpms(connector, DRM_MODE_DPMS_OFF);
+ drm_helper_connector_dpms(connector, DRM_MODE_DPMS_ON);
}
}
}
DRM_INFO("amdgpu: finishing device.\n");
adev->shutdown = true;
- if (adev->mode_info.mode_config_initialized)
- drm_crtc_force_disable_all(adev->ddev);
-
+ if (adev->mode_info.mode_config_initialized){
+ if (!amdgpu_device_has_dc_support(adev))
+ drm_crtc_force_disable_all(adev->ddev);
+ else
+ drm_atomic_helper_shutdown(adev->ddev);
+ }
amdgpu_ib_pool_fini(adev);
amdgpu_fence_driver_fini(adev);
amdgpu_fbdev_fini(adev);
struct amdgpu_bo *robj = gem_to_amdgpu_bo(gobj);
if (robj) {
- if (robj->gem_base.import_attach)
- drm_prime_gem_destroy(&robj->gem_base, robj->tbo.sg);
amdgpu_mn_unregister(robj);
amdgpu_bo_unref(&robj);
}
u16 firmware_flags;
/* pointer to backlight encoder */
struct amdgpu_encoder *bl_encoder;
+ u8 bl_level; /* saved backlight level */
struct amdgpu_audio audio; /* audio stuff */
int num_crtc; /* number of crtcs */
int num_hpd; /* number of hpd pins */
amdgpu_bo_kunmap(bo);
+ if (bo->gem_base.import_attach)
+ drm_prime_gem_destroy(&bo->gem_base, bo->tbo.sg);
drm_gem_object_release(&bo->gem_base);
amdgpu_bo_unref(&bo->parent);
if (!list_empty(&bo->shadow_list)) {
result = 0;
if (*pos < 12) {
- early[0] = amdgpu_ring_get_rptr(ring);
+ early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask;
early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask;
early[2] = ring->wptr & ring->buf_mask;
for (i = *pos / 4; i < 3 && size; i++) {
cancel_delayed_work_sync(&adev->uvd.idle_work);
- for (i = 0; i < adev->uvd.max_handles; ++i)
- if (atomic_read(&adev->uvd.handles[i]))
- break;
+ /* only valid for physical mode */
+ if (adev->asic_type < CHIP_POLARIS10) {
+ for (i = 0; i < adev->uvd.max_handles; ++i)
+ if (atomic_read(&adev->uvd.handles[i]))
+ break;
- if (i == AMDGPU_MAX_UVD_HANDLES)
- return 0;
+ if (i == adev->uvd.max_handles)
+ return 0;
+ }
size = amdgpu_bo_size(adev->uvd.vcpu_bo);
ptr = adev->uvd.cpu_addr;
#include <linux/backlight.h>
#include "bif/bif_4_1_d.h"
-static u8
+u8
amdgpu_atombios_encoder_get_backlight_level_from_reg(struct amdgpu_device *adev)
{
u8 backlight_level;
return backlight_level;
}
-static void
+void
amdgpu_atombios_encoder_set_backlight_level_to_reg(struct amdgpu_device *adev,
u8 backlight_level)
{
#ifndef __ATOMBIOS_ENCODER_H__
#define __ATOMBIOS_ENCODER_H__
+u8
+amdgpu_atombios_encoder_get_backlight_level_from_reg(struct amdgpu_device *adev);
+void
+amdgpu_atombios_encoder_set_backlight_level_to_reg(struct amdgpu_device *adev,
+ u8 backlight_level);
u8
amdgpu_atombios_encoder_get_backlight_level(struct amdgpu_encoder *amdgpu_encoder);
void
static int dce_v10_0_suspend(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ adev->mode_info.bl_level =
+ amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
+
return dce_v10_0_hw_fini(handle);
}
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
int ret;
+ amdgpu_atombios_encoder_set_backlight_level_to_reg(adev,
+ adev->mode_info.bl_level);
+
ret = dce_v10_0_hw_init(handle);
/* turn on the BL */
static int dce_v11_0_suspend(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ adev->mode_info.bl_level =
+ amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
+
return dce_v11_0_hw_fini(handle);
}
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
int ret;
+ amdgpu_atombios_encoder_set_backlight_level_to_reg(adev,
+ adev->mode_info.bl_level);
+
ret = dce_v11_0_hw_init(handle);
/* turn on the BL */
static int dce_v6_0_suspend(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ adev->mode_info.bl_level =
+ amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
+
return dce_v6_0_hw_fini(handle);
}
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
int ret;
+ amdgpu_atombios_encoder_set_backlight_level_to_reg(adev,
+ adev->mode_info.bl_level);
+
ret = dce_v6_0_hw_init(handle);
/* turn on the BL */
tmp |= DC_HPD1_INT_CONTROL__DC_HPD1_INT_ACK_MASK;
WREG32(mmDC_HPD1_INT_CONTROL + hpd_offsets[hpd], tmp);
schedule_work(&adev->hotplug_work);
- DRM_INFO("IH: HPD%d\n", hpd + 1);
+ DRM_DEBUG("IH: HPD%d\n", hpd + 1);
}
return 0;
static int dce_v8_0_suspend(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+ adev->mode_info.bl_level =
+ amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
+
return dce_v8_0_hw_fini(handle);
}
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
int ret;
+ amdgpu_atombios_encoder_set_backlight_level_to_reg(adev,
+ adev->mode_info.bl_level);
+
ret = dce_v8_0_hw_init(handle);
/* turn on the BL */
case CHIP_KAVERI:
adev->gfx.config.max_shader_engines = 1;
adev->gfx.config.max_tile_pipes = 4;
- if ((adev->pdev->device == 0x1304) ||
- (adev->pdev->device == 0x1305) ||
- (adev->pdev->device == 0x130C) ||
- (adev->pdev->device == 0x130F) ||
- (adev->pdev->device == 0x1310) ||
- (adev->pdev->device == 0x1311) ||
- (adev->pdev->device == 0x131C)) {
- adev->gfx.config.max_cu_per_sh = 8;
- adev->gfx.config.max_backends_per_se = 2;
- } else if ((adev->pdev->device == 0x1309) ||
- (adev->pdev->device == 0x130A) ||
- (adev->pdev->device == 0x130D) ||
- (adev->pdev->device == 0x1313) ||
- (adev->pdev->device == 0x131D)) {
- adev->gfx.config.max_cu_per_sh = 6;
- adev->gfx.config.max_backends_per_se = 2;
- } else if ((adev->pdev->device == 0x1306) ||
- (adev->pdev->device == 0x1307) ||
- (adev->pdev->device == 0x130B) ||
- (adev->pdev->device == 0x130E) ||
- (adev->pdev->device == 0x1315) ||
- (adev->pdev->device == 0x131B)) {
- adev->gfx.config.max_cu_per_sh = 4;
- adev->gfx.config.max_backends_per_se = 1;
- } else {
- adev->gfx.config.max_cu_per_sh = 3;
- adev->gfx.config.max_backends_per_se = 1;
- }
+ adev->gfx.config.max_cu_per_sh = 8;
+ adev->gfx.config.max_backends_per_se = 2;
adev->gfx.config.max_sh_per_se = 1;
adev->gfx.config.max_texture_channel_caches = 4;
adev->gfx.config.max_gprs = 256;
#include "amdgpu_uvd.h"
#include "amdgpu_vce.h"
#include "atom.h"
+#include "amd_pcie.h"
#include "amdgpu_powerplay.h"
#include "sid.h"
#include "si_ih.h"
{
struct pci_dev *root = adev->pdev->bus->self;
int bridge_pos, gpu_pos;
- u32 speed_cntl, mask, current_data_rate;
- int ret, i;
+ u32 speed_cntl, current_data_rate;
+ int i;
u16 tmp16;
if (pci_is_root_bus(adev->pdev->bus))
if (adev->flags & AMD_IS_APU)
return;
- ret = drm_pcie_get_speed_cap_mask(adev->ddev, &mask);
- if (ret != 0)
- return;
-
- if (!(mask & (DRM_PCIE_SPEED_50 | DRM_PCIE_SPEED_80)))
+ if (!(adev->pm.pcie_gen_mask & (CAIL_PCIE_LINK_SPEED_SUPPORT_GEN2 |
+ CAIL_PCIE_LINK_SPEED_SUPPORT_GEN3)))
return;
speed_cntl = RREG32_PCIE_PORT(PCIE_LC_SPEED_CNTL);
current_data_rate = (speed_cntl & LC_CURRENT_DATA_RATE_MASK) >>
LC_CURRENT_DATA_RATE_SHIFT;
- if (mask & DRM_PCIE_SPEED_80) {
+ if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN3) {
if (current_data_rate == 2) {
DRM_INFO("PCIE gen 3 link speeds already enabled\n");
return;
}
DRM_INFO("enabling PCIE gen 3 link speeds, disable with amdgpu.pcie_gen2=0\n");
- } else if (mask & DRM_PCIE_SPEED_50) {
+ } else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN2) {
if (current_data_rate == 1) {
DRM_INFO("PCIE gen 2 link speeds already enabled\n");
return;
if (!gpu_pos)
return;
- if (mask & DRM_PCIE_SPEED_80) {
+ if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN3) {
if (current_data_rate != 2) {
u16 bridge_cfg, gpu_cfg;
u16 bridge_cfg2, gpu_cfg2;
pci_read_config_word(adev->pdev, gpu_pos + PCI_EXP_LNKCTL2, &tmp16);
tmp16 &= ~0xf;
- if (mask & DRM_PCIE_SPEED_80)
+ if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN3)
tmp16 |= 3;
- else if (mask & DRM_PCIE_SPEED_50)
+ else if (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_GEN2)
tmp16 |= 2;
else
tmp16 |= 1;
#include "amdgpu_pm.h"
#include "amdgpu_dpm.h"
#include "amdgpu_atombios.h"
+#include "amd_pcie.h"
#include "sid.h"
#include "r600_dpm.h"
#include "si_dpm.h"
}
}
-static enum amdgpu_pcie_gen r600_get_pcie_gen_support(struct amdgpu_device *adev,
- u32 sys_mask,
- enum amdgpu_pcie_gen asic_gen,
- enum amdgpu_pcie_gen default_gen)
-{
- switch (asic_gen) {
- case AMDGPU_PCIE_GEN1:
- return AMDGPU_PCIE_GEN1;
- case AMDGPU_PCIE_GEN2:
- return AMDGPU_PCIE_GEN2;
- case AMDGPU_PCIE_GEN3:
- return AMDGPU_PCIE_GEN3;
- default:
- if ((sys_mask & DRM_PCIE_SPEED_80) && (default_gen == AMDGPU_PCIE_GEN3))
- return AMDGPU_PCIE_GEN3;
- else if ((sys_mask & DRM_PCIE_SPEED_50) && (default_gen == AMDGPU_PCIE_GEN2))
- return AMDGPU_PCIE_GEN2;
- else
- return AMDGPU_PCIE_GEN1;
- }
- return AMDGPU_PCIE_GEN1;
-}
-
static void r600_calculate_u_and_p(u32 i, u32 r_c, u32 p_b,
u32 *p, u32 *u)
{
table->ACPIState.levels[0].vddc.index,
&table->ACPIState.levels[0].std_vddc);
}
- table->ACPIState.levels[0].gen2PCIE = (u8)r600_get_pcie_gen_support(adev,
- si_pi->sys_pcie_mask,
- si_pi->boot_pcie_gen,
- AMDGPU_PCIE_GEN1);
+ table->ACPIState.levels[0].gen2PCIE =
+ (u8)amdgpu_get_pcie_gen_support(adev,
+ si_pi->sys_pcie_mask,
+ si_pi->boot_pcie_gen,
+ AMDGPU_PCIE_GEN1);
if (si_pi->vddc_phase_shed_control)
si_populate_phase_shedding_value(adev,
pl->vddc = le16_to_cpu(clock_info->si.usVDDC);
pl->vddci = le16_to_cpu(clock_info->si.usVDDCI);
pl->flags = le32_to_cpu(clock_info->si.ulFlags);
- pl->pcie_gen = r600_get_pcie_gen_support(adev,
- si_pi->sys_pcie_mask,
- si_pi->boot_pcie_gen,
- clock_info->si.ucPCIEGen);
+ pl->pcie_gen = amdgpu_get_pcie_gen_support(adev,
+ si_pi->sys_pcie_mask,
+ si_pi->boot_pcie_gen,
+ clock_info->si.ucPCIEGen);
/* patch up vddc if necessary */
ret = si_get_leakage_voltage_from_leakage_index(adev, pl->vddc,
struct si_power_info *si_pi;
struct atom_clock_dividers dividers;
int ret;
- u32 mask;
si_pi = kzalloc(sizeof(struct si_power_info), GFP_KERNEL);
if (si_pi == NULL)
eg_pi = &ni_pi->eg;
pi = &eg_pi->rv7xx;
- ret = drm_pcie_get_speed_cap_mask(adev->ddev, &mask);
- if (ret)
- si_pi->sys_pcie_mask = 0;
- else
- si_pi->sys_pcie_mask = mask;
+ si_pi->sys_pcie_mask =
+ (adev->pm.pcie_gen_mask & CAIL_PCIE_LINK_SPEED_SUPPORT_MASK) >>
+ CAIL_PCIE_LINK_SPEED_SUPPORT_SHIFT;
si_pi->force_pcie_gen = AMDGPU_PCIE_GEN_INVALID;
si_pi->boot_pcie_gen = si_get_current_pcie_speed(adev);
!is_mst_root_connector) {
/* Downstream Port status changed. */
if (dc_link_detect(dc_link, DETECT_REASON_HPDRX)) {
+
+ if (aconnector->fake_enable)
+ aconnector->fake_enable = false;
+
amdgpu_dm_update_connector_after_detect(aconnector);
dst.width = stream->timing.h_addressable;
dst.height = stream->timing.v_addressable;
- rmx_type = dm_state->scaling;
- if (rmx_type == RMX_ASPECT || rmx_type == RMX_OFF) {
- if (src.width * dst.height <
- src.height * dst.width) {
- /* height needs less upscaling/more downscaling */
- dst.width = src.width *
- dst.height / src.height;
- } else {
- /* width needs less upscaling/more downscaling */
- dst.height = src.height *
- dst.width / src.width;
+ if (dm_state) {
+ rmx_type = dm_state->scaling;
+ if (rmx_type == RMX_ASPECT || rmx_type == RMX_OFF) {
+ if (src.width * dst.height <
+ src.height * dst.width) {
+ /* height needs less upscaling/more downscaling */
+ dst.width = src.width *
+ dst.height / src.height;
+ } else {
+ /* width needs less upscaling/more downscaling */
+ dst.height = src.height *
+ dst.width / src.width;
+ }
+ } else if (rmx_type == RMX_CENTER) {
+ dst = src;
}
- } else if (rmx_type == RMX_CENTER) {
- dst = src;
- }
- dst.x = (stream->timing.h_addressable - dst.width) / 2;
- dst.y = (stream->timing.v_addressable - dst.height) / 2;
+ dst.x = (stream->timing.h_addressable - dst.width) / 2;
+ dst.y = (stream->timing.v_addressable - dst.height) / 2;
- if (dm_state->underscan_enable) {
- dst.x += dm_state->underscan_hborder / 2;
- dst.y += dm_state->underscan_vborder / 2;
- dst.width -= dm_state->underscan_hborder;
- dst.height -= dm_state->underscan_vborder;
+ if (dm_state->underscan_enable) {
+ dst.x += dm_state->underscan_hborder / 2;
+ dst.y += dm_state->underscan_vborder / 2;
+ dst.width -= dm_state->underscan_hborder;
+ dst.height -= dm_state->underscan_vborder;
+ }
}
stream->src = src;
if (aconnector == NULL) {
DRM_ERROR("aconnector is NULL!\n");
- goto drm_connector_null;
- }
-
- if (dm_state == NULL) {
- DRM_ERROR("dm_state is NULL!\n");
- goto dm_state_null;
+ return stream;
}
drm_connector = &aconnector->base;
*/
if (aconnector->mst_port) {
dm_dp_mst_dc_sink_create(drm_connector);
- goto mst_dc_sink_create_done;
+ return stream;
}
if (create_fake_sink(aconnector))
- goto stream_create_fail;
+ return stream;
}
stream = dc_create_stream_for_sink(aconnector->dc_sink);
if (stream == NULL) {
DRM_ERROR("Failed to create stream for sink!\n");
- goto stream_create_fail;
+ return stream;
}
list_for_each_entry(preferred_mode, &aconnector->base.modes, head) {
} else {
decide_crtc_timing_for_drm_display_mode(
&mode, preferred_mode,
- dm_state->scaling != RMX_OFF);
+ dm_state ? (dm_state->scaling != RMX_OFF) : false);
}
+ if (!dm_state)
+ drm_mode_set_crtcinfo(&mode, 0);
+
fill_stream_properties_from_drm_display_mode(stream,
&mode, &aconnector->base);
update_stream_scaling_settings(&mode, dm_state, stream);
drm_connector,
aconnector->dc_sink);
-stream_create_fail:
-dm_state_null:
-drm_connector_null:
-mst_dc_sink_create_done:
+ update_stream_signal(stream);
+
return stream;
}
return &state->base;
}
+
+static inline int dm_set_vblank(struct drm_crtc *crtc, bool enable)
+{
+ enum dc_irq_source irq_source;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+ struct amdgpu_device *adev = crtc->dev->dev_private;
+
+ irq_source = IRQ_TYPE_VBLANK + acrtc->otg_inst;
+ return dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY;
+}
+
+static int dm_enable_vblank(struct drm_crtc *crtc)
+{
+ return dm_set_vblank(crtc, true);
+}
+
+static void dm_disable_vblank(struct drm_crtc *crtc)
+{
+ dm_set_vblank(crtc, false);
+}
+
/* Implemented only the options currently availible for the driver */
static const struct drm_crtc_funcs amdgpu_dm_crtc_funcs = {
.reset = dm_crtc_reset_state,
.page_flip = drm_atomic_helper_page_flip,
.atomic_duplicate_state = dm_crtc_duplicate_state,
.atomic_destroy_state = dm_crtc_destroy_state,
+ .enable_vblank = dm_enable_vblank,
+ .disable_vblank = dm_disable_vblank,
};
static enum drm_connector_status
goto fail;
}
- stream = dc_create_stream_for_sink(dc_sink);
+ stream = create_stream_for_sink(aconnector, mode, NULL);
if (stream == NULL) {
DRM_ERROR("Failed to create stream for sink!\n");
goto fail;
if (!dm_plane_state->dc_state)
return 0;
+ if (!fill_rects_from_plane_state(state, dm_plane_state->dc_state))
+ return -EINVAL;
+
if (dc_validate_plane(dc, dm_plane_state->dc_state) == DC_OK)
return 0;
switch (aplane->base.type) {
case DRM_PLANE_TYPE_PRIMARY:
- aplane->base.format_default = true;
-
res = drm_universal_plane_init(
dm->adev->ddev,
&aplane->base,
bool pflip_needed = !state->allow_modeset;
int ret = 0;
- if (pflip_needed)
- return ret;
/* Add new planes */
for_each_oldnew_plane_in_state(state, plane, old_plane_state, new_plane_state, i) {
/* Remove any changed/removed planes */
if (!enable) {
+ if (pflip_needed)
+ continue;
if (!old_plane_crtc)
continue;
*lock_and_validation_needed = true;
} else { /* Add new planes */
+ struct dc_plane_state *dc_new_plane_state;
if (drm_atomic_plane_disabling(plane->state, new_plane_state))
continue;
if (!dm_new_crtc_state->stream)
continue;
+ if (pflip_needed)
+ continue;
WARN_ON(dm_new_plane_state->dc_state);
- dm_new_plane_state->dc_state = dc_create_plane_state(dc);
-
- DRM_DEBUG_DRIVER("Enabling DRM plane: %d on DRM crtc %d\n",
- plane->base.id, new_plane_crtc->base.id);
-
- if (!dm_new_plane_state->dc_state) {
+ dc_new_plane_state = dc_create_plane_state(dc);
+ if (!dc_new_plane_state) {
ret = -EINVAL;
return ret;
}
+ DRM_DEBUG_DRIVER("Enabling DRM plane: %d on DRM crtc %d\n",
+ plane->base.id, new_plane_crtc->base.id);
+
ret = fill_plane_attributes(
new_plane_crtc->dev->dev_private,
- dm_new_plane_state->dc_state,
+ dc_new_plane_state,
new_plane_state,
new_crtc_state);
- if (ret)
+ if (ret) {
+ dc_plane_state_release(dc_new_plane_state);
return ret;
+ }
-
+ /*
+ * Any atomic check errors that occur after this will
+ * not need a release. The plane state will be attached
+ * to the stream, and therefore part of the atomic
+ * state. It'll be released when the atomic state is
+ * cleaned.
+ */
if (!dc_add_plane_to_context(
dc,
dm_new_crtc_state->stream,
- dm_new_plane_state->dc_state,
+ dc_new_plane_state,
dm_state->context)) {
+ dc_plane_state_release(dc_new_plane_state);
ret = -EINVAL;
return ret;
}
+ dm_new_plane_state->dc_state = dc_new_plane_state;
+
/* Tell DC to do a full surface update every time there
* is a plane change. Inefficient, but works for now.
*/
return ret;
}
+static int dm_atomic_check_plane_state_fb(struct drm_atomic_state *state,
+ struct drm_crtc *crtc)
+{
+ struct drm_plane *plane;
+ struct drm_crtc_state *crtc_state;
+
+ WARN_ON(!drm_atomic_get_new_crtc_state(state, crtc));
+
+ drm_for_each_plane_mask(plane, state->dev, crtc->state->plane_mask) {
+ struct drm_plane_state *plane_state =
+ drm_atomic_get_plane_state(state, plane);
+
+ if (IS_ERR(plane_state))
+ return -EDEADLK;
+
+ crtc_state = drm_atomic_get_crtc_state(plane_state->state, crtc);
+ if (IS_ERR(crtc_state))
+ return PTR_ERR(crtc_state);
+
+ if (crtc->primary == plane && crtc_state->active) {
+ if (!plane_state->fb)
+ return -EINVAL;
+ }
+ }
+ return 0;
+}
+
static int amdgpu_dm_atomic_check(struct drm_device *dev,
struct drm_atomic_state *state)
{
goto fail;
for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
+ ret = dm_atomic_check_plane_state_fb(state, crtc);
+ if (ret)
+ goto fail;
+
if (!drm_atomic_crtc_needs_modeset(new_crtc_state) &&
!new_crtc_state->color_mgmt_changed)
continue;
struct cea_sad *sad = &sads[i];
edid_caps->audio_modes[i].format_code = sad->format;
- edid_caps->audio_modes[i].channel_count = sad->channels;
+ edid_caps->audio_modes[i].channel_count = sad->channels + 1;
edid_caps->audio_modes[i].sample_rate = sad->freq;
edid_caps->audio_modes[i].sample_size = sad->byte2;
}
void amdgpu_dm_set_irq_funcs(struct amdgpu_device *adev)
{
- if (adev->mode_info.num_crtc > 0)
- adev->crtc_irq.num_types = AMDGPU_CRTC_IRQ_VLINE1 + adev->mode_info.num_crtc;
- else
- adev->crtc_irq.num_types = 0;
+
+ adev->crtc_irq.num_types = adev->mode_info.num_crtc;
adev->crtc_irq.funcs = &dm_crtc_irq_funcs;
adev->pageflip_irq.num_types = adev->mode_info.num_crtc;
.link = aconnector->dc_link,
.sink_signal = SIGNAL_TYPE_DISPLAY_PORT_MST };
+ /*
+ * TODO: Need to further figure out why ddc.algo is NULL while MST port exists
+ */
+ if (!aconnector->port || !aconnector->port->aux.ddc.algo)
+ return;
+
edid = drm_dp_mst_get_edid(connector, &aconnector->mst_port->mst_mgr, aconnector->port);
if (!edid) {
return dal_irq_service_to_irq_source(dc->res_pool->irqs, src_id, ext_id);
}
-void dc_interrupt_set(struct dc *dc, enum dc_irq_source src, bool enable)
+bool dc_interrupt_set(struct dc *dc, enum dc_irq_source src, bool enable)
{
if (dc == NULL)
- return;
+ return false;
- dal_irq_service_set(dc->res_pool->irqs, src, enable);
+ return dal_irq_service_set(dc->res_pool->irqs, src, enable);
}
void dc_interrupt_ack(struct dc *dc, enum dc_irq_source src)
link->link_enc,
pipe_ctx->clock_source->id,
display_color_depth,
- pipe_ctx->stream->signal == SIGNAL_TYPE_HDMI_TYPE_A,
- pipe_ctx->stream->signal == SIGNAL_TYPE_DVI_DUAL_LINK,
+ pipe_ctx->stream->signal,
stream->phy_pix_clk);
if (pipe_ctx->stream->signal == SIGNAL_TYPE_HDMI_TYPE_A)
return true;
}
-/* Maximum TMDS single link pixel clock 165MHz */
-#define TMDS_MAX_PIXEL_CLOCK_IN_KHZ 165000
-
static void update_stream_engine_usage(
struct resource_context *res_ctx,
const struct resource_pool *pool,
/*******************************************************************************
* Private functions
******************************************************************************/
-#define TMDS_MAX_PIXEL_CLOCK_IN_KHZ_UPMOST 297000
-static void update_stream_signal(struct dc_stream_state *stream)
+void update_stream_signal(struct dc_stream_state *stream)
{
struct dc_sink *dc_sink = stream->sink;
stream->signal = dc_sink->sink_signal;
if (dc_is_dvi_signal(stream->signal)) {
- if (stream->timing.pix_clk_khz > TMDS_MAX_PIXEL_CLOCK_IN_KHZ_UPMOST &&
- stream->sink->sink_signal != SIGNAL_TYPE_DVI_SINGLE_LINK)
+ if (stream->ctx->dc->caps.dual_link_dvi &&
+ stream->timing.pix_clk_khz > TMDS_MAX_PIXEL_CLOCK &&
+ stream->sink->sink_signal != SIGNAL_TYPE_DVI_SINGLE_LINK)
stream->signal = SIGNAL_TYPE_DVI_DUAL_LINK;
else
stream->signal = SIGNAL_TYPE_DVI_SINGLE_LINK;
core_dc = stream->ctx->dc;
res_ctx = &core_dc->current_state->res_ctx;
+ stream->cursor_attributes = *attributes;
for (i = 0; i < MAX_PIPES; i++) {
struct pipe_ctx *pipe_ctx = &res_ctx->pipe_ctx[i];
continue;
- if (pipe_ctx->plane_res.ipp->funcs->ipp_cursor_set_attributes != NULL)
- pipe_ctx->plane_res.ipp->funcs->ipp_cursor_set_attributes(
- pipe_ctx->plane_res.ipp, attributes);
-
- if (pipe_ctx->plane_res.hubp != NULL &&
- pipe_ctx->plane_res.hubp->funcs->set_cursor_attributes != NULL)
- pipe_ctx->plane_res.hubp->funcs->set_cursor_attributes(
- pipe_ctx->plane_res.hubp, attributes);
-
- if (pipe_ctx->plane_res.mi != NULL &&
- pipe_ctx->plane_res.mi->funcs->set_cursor_attributes != NULL)
- pipe_ctx->plane_res.mi->funcs->set_cursor_attributes(
- pipe_ctx->plane_res.mi, attributes);
-
-
- if (pipe_ctx->plane_res.xfm != NULL &&
- pipe_ctx->plane_res.xfm->funcs->set_cursor_attributes != NULL)
- pipe_ctx->plane_res.xfm->funcs->set_cursor_attributes(
- pipe_ctx->plane_res.xfm, attributes);
-
- if (pipe_ctx->plane_res.dpp != NULL &&
- pipe_ctx->plane_res.dpp->funcs->set_cursor_attributes != NULL)
- pipe_ctx->plane_res.dpp->funcs->set_cursor_attributes(
- pipe_ctx->plane_res.dpp, attributes->color_format);
+ core_dc->hwss.set_cursor_attribute(pipe_ctx);
}
-
- stream->cursor_attributes = *attributes;
-
return true;
}
core_dc = stream->ctx->dc;
res_ctx = &core_dc->current_state->res_ctx;
+ stream->cursor_position = *position;
for (i = 0; i < MAX_PIPES; i++) {
struct pipe_ctx *pipe_ctx = &res_ctx->pipe_ctx[i];
- struct input_pixel_processor *ipp = pipe_ctx->plane_res.ipp;
- struct mem_input *mi = pipe_ctx->plane_res.mi;
- struct hubp *hubp = pipe_ctx->plane_res.hubp;
- struct dpp *dpp = pipe_ctx->plane_res.dpp;
- struct dc_cursor_position pos_cpy = *position;
- struct dc_cursor_mi_param param = {
- .pixel_clk_khz = stream->timing.pix_clk_khz,
- .ref_clk_khz = core_dc->res_pool->ref_clock_inKhz,
- .viewport_x_start = pipe_ctx->plane_res.scl_data.viewport.x,
- .viewport_width = pipe_ctx->plane_res.scl_data.viewport.width,
- .h_scale_ratio = pipe_ctx->plane_res.scl_data.ratios.horz
- };
if (pipe_ctx->stream != stream ||
(!pipe_ctx->plane_res.mi && !pipe_ctx->plane_res.hubp) ||
!pipe_ctx->plane_res.ipp)
continue;
- if (pipe_ctx->plane_state->address.type
- == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE)
- pos_cpy.enable = false;
-
- if (pipe_ctx->top_pipe && pipe_ctx->plane_state != pipe_ctx->top_pipe->plane_state)
- pos_cpy.enable = false;
-
-
- if (ipp != NULL && ipp->funcs->ipp_cursor_set_position != NULL)
- ipp->funcs->ipp_cursor_set_position(ipp, &pos_cpy, ¶m);
-
- if (mi != NULL && mi->funcs->set_cursor_position != NULL)
- mi->funcs->set_cursor_position(mi, &pos_cpy, ¶m);
-
- if (!hubp)
- continue;
-
- if (hubp->funcs->set_cursor_position != NULL)
- hubp->funcs->set_cursor_position(hubp, &pos_cpy, ¶m);
-
- if (dpp != NULL && dpp->funcs->set_cursor_position != NULL)
- dpp->funcs->set_cursor_position(dpp, &pos_cpy, ¶m, hubp->curs_attr.width);
-
+ core_dc->hwss.set_cursor_position(pipe_ctx);
}
- stream->cursor_position = *position;
-
return true;
}
bool dcc_const_color;
bool dynamic_audio;
bool is_apu;
+ bool dual_link_dvi;
};
struct dc_dcc_surface_param {
struct dc *dc,
uint32_t src_id,
uint32_t ext_id);
-void dc_interrupt_set(struct dc *dc, enum dc_irq_source src, bool enable);
+bool dc_interrupt_set(struct dc *dc, enum dc_irq_source src, bool enable);
void dc_interrupt_ack(struct dc *dc, enum dc_irq_source src);
enum dc_irq_source dc_get_hpd_irq_source_at_index(
struct dc *dc, uint32_t link_index);
*/
struct dc_stream_state *dc_create_stream_for_sink(struct dc_sink *dc_sink);
+void update_stream_signal(struct dc_stream_state *stream);
+
void dc_stream_retain(struct dc_stream_state *dc_stream);
void dc_stream_release(struct dc_stream_state *dc_stream);
SR(D2VGA_CONTROL), \
SR(D3VGA_CONTROL), \
SR(D4VGA_CONTROL), \
+ SR(VGA_TEST_CONTROL), \
SR(DC_IP_REQUEST_CNTL), \
BL_REG_LIST()
uint32_t D2VGA_CONTROL;
uint32_t D3VGA_CONTROL;
uint32_t D4VGA_CONTROL;
+ uint32_t VGA_TEST_CONTROL;
/* MMHUB registers. read only. temporary hack */
uint32_t VM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32;
uint32_t VM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32;
HWS_SF(, DOMAIN6_PG_STATUS, DOMAIN6_PGFSM_PWR_STATUS, mask_sh), \
HWS_SF(, DOMAIN7_PG_STATUS, DOMAIN7_PGFSM_PWR_STATUS, mask_sh), \
HWS_SF(, DC_IP_REQUEST_CNTL, IP_REQUEST_EN, mask_sh), \
+ HWS_SF(, D1VGA_CONTROL, D1VGA_MODE_ENABLE, mask_sh),\
+ HWS_SF(, D2VGA_CONTROL, D2VGA_MODE_ENABLE, mask_sh),\
+ HWS_SF(, D3VGA_CONTROL, D3VGA_MODE_ENABLE, mask_sh),\
+ HWS_SF(, D4VGA_CONTROL, D4VGA_MODE_ENABLE, mask_sh),\
+ HWS_SF(, VGA_TEST_CONTROL, VGA_TEST_ENABLE, mask_sh),\
+ HWS_SF(, VGA_TEST_CONTROL, VGA_TEST_RENDER_START, mask_sh),\
HWS_SF(, LVTMA_PWRSEQ_CNTL, LVTMA_BLON, mask_sh), \
HWS_SF(, LVTMA_PWRSEQ_STATE, LVTMA_PWRSEQ_TARGET_STATE_R, mask_sh)
type DCFCLK_GATE_DIS; \
type DCHUBBUB_GLOBAL_TIMER_REFDIV; \
type DENTIST_DPPCLK_WDIVIDER; \
- type DENTIST_DISPCLK_WDIVIDER;
+ type DENTIST_DISPCLK_WDIVIDER; \
+ type VGA_TEST_ENABLE; \
+ type VGA_TEST_RENDER_START; \
+ type D1VGA_MODE_ENABLE; \
+ type D2VGA_MODE_ENABLE; \
+ type D3VGA_MODE_ENABLE; \
+ type D4VGA_MODE_ENABLE;
struct dce_hwseq_shift {
HWSEQ_REG_FIELD_LIST(uint8_t)
#define DCE110_DIG_FE_SOURCE_SELECT_DIGF 0x20
#define DCE110_DIG_FE_SOURCE_SELECT_DIGG 0x40
-/* Minimum pixel clock, in KHz. For TMDS signal is 25.00 MHz */
-#define TMDS_MIN_PIXEL_CLOCK 25000
-/* Maximum pixel clock, in KHz. For TMDS signal is 165.00 MHz */
-#define TMDS_MAX_PIXEL_CLOCK 165000
-/* For current ASICs pixel clock - 600MHz */
-#define MAX_ENCODER_CLOCK 600000
-
enum {
DP_MST_UPDATE_MAX_RETRY = 50
};
{
struct bp_encoder_cap_info bp_cap_info = {0};
const struct dc_vbios_funcs *bp_funcs = init_data->ctx->dc_bios->funcs;
+ enum bp_result result = BP_RESULT_OK;
enc110->base.funcs = &dce110_lnk_enc_funcs;
enc110->base.ctx = init_data->ctx;
enc110->base.preferred_engine = ENGINE_ID_UNKNOWN;
}
+ /* default to one to mirror Windows behavior */
+ enc110->base.features.flags.bits.HDMI_6GB_EN = 1;
+
+ result = bp_funcs->get_encoder_cap_info(enc110->base.ctx->dc_bios,
+ enc110->base.id, &bp_cap_info);
+
/* Override features with DCE-specific values */
- if (BP_RESULT_OK == bp_funcs->get_encoder_cap_info(
- enc110->base.ctx->dc_bios, enc110->base.id,
- &bp_cap_info)) {
+ if (BP_RESULT_OK == result) {
enc110->base.features.flags.bits.IS_HBR2_CAPABLE =
bp_cap_info.DP_HBR2_EN;
enc110->base.features.flags.bits.IS_HBR3_CAPABLE =
bp_cap_info.DP_HBR3_EN;
enc110->base.features.flags.bits.HDMI_6GB_EN = bp_cap_info.HDMI_6GB_EN;
+ } else {
+ dm_logger_write(enc110->base.ctx->logger, LOG_WARNING,
+ "%s: Failed to get encoder_cap_info from VBIOS with error code %d!\n",
+ __func__,
+ result);
}
}
struct link_encoder *enc,
enum clock_source_id clock_source,
enum dc_color_depth color_depth,
- bool hdmi,
- bool dual_link,
+ enum signal_type signal,
uint32_t pixel_clock)
{
struct dce110_link_encoder *enc110 = TO_DCE110_LINK_ENC(enc);
cntl.engine_id = enc->preferred_engine;
cntl.transmitter = enc110->base.transmitter;
cntl.pll_id = clock_source;
- if (hdmi) {
- cntl.signal = SIGNAL_TYPE_HDMI_TYPE_A;
- cntl.lanes_number = 4;
- } else if (dual_link) {
- cntl.signal = SIGNAL_TYPE_DVI_DUAL_LINK;
+ cntl.signal = signal;
+ if (cntl.signal == SIGNAL_TYPE_DVI_DUAL_LINK)
cntl.lanes_number = 8;
- } else {
- cntl.signal = SIGNAL_TYPE_DVI_SINGLE_LINK;
+ else
cntl.lanes_number = 4;
- }
+
cntl.hpd_sel = enc110->base.hpd_source;
cntl.pixel_clock = pixel_clock;
struct link_encoder *enc,
enum clock_source_id clock_source,
enum dc_color_depth color_depth,
- bool hdmi,
- bool dual_link,
+ enum signal_type signal,
uint32_t pixel_clock);
/* enables DP PHY output */
return;
}
/* on other format-to do */
- if (params->flags.TRUNCATE_ENABLED == 0 ||
- params->flags.TRUNCATE_DEPTH == 2)
+ if (params->flags.TRUNCATE_ENABLED == 0)
return;
/*Set truncation depth and Enable truncation*/
REG_UPDATE_3(FMT_BIT_DEPTH_CONTROL,
FMT_TRUNCATE_EN, 1,
FMT_TRUNCATE_DEPTH,
- params->flags.TRUNCATE_MODE,
+ params->flags.TRUNCATE_DEPTH,
FMT_TRUNCATE_MODE,
- params->flags.TRUNCATE_DEPTH);
+ params->flags.TRUNCATE_MODE);
}
/**
* set_spatial_dither
* 1) set spatial dithering mode: pattern of seed
- * 2) set spatical dithering depth: 0 for 18bpp or 1 for 24bpp
+ * 2) set spatial dithering depth: 0 for 18bpp or 1 for 24bpp
* 3) set random seed
* 4) set random mode
* lfsr is reset every frame or not reset
dc->caps.max_downscale_ratio = 200;
dc->caps.i2c_speed_in_khz = 40;
dc->caps.max_cursor_size = 128;
+ dc->caps.dual_link_dvi = true;
for (i = 0; i < pool->base.pipe_count; i++) {
pool->base.timing_generators[i] =
struct dc_crtc_timing *timing = &pipe_ctx->stream->timing;
struct dc_link *link = pipe_ctx->stream->sink->link;
- /* 1. update AVI info frame (HDMI, DP)
- * we always need to update info frame
- */
+
uint32_t active_total_with_borders;
uint32_t early_control = 0;
struct timing_generator *tg = pipe_ctx->stream_res.tg;
- /* TODOFPGA may change to hwss.update_info_frame */
+ /* For MST, there are multiply stream go to only one link.
+ * connect DIG back_end to front_end while enable_stream and
+ * disconnect them during disable_stream
+ * BY this, it is logic clean to separate stream and link */
+ link->link_enc->funcs->connect_dig_be_to_fe(link->link_enc,
+ pipe_ctx->stream_res.stream_enc->id, true);
+
+ /* update AVI info frame (HDMI, DP)*/
+ /* TODO: FPGA may change to hwss.update_info_frame */
dce110_update_info_frame(pipe_ctx);
+
/* enable early control to avoid corruption on DP monitor*/
active_total_with_borders =
timing->h_addressable
pipe_ctx->stream_res.stream_enc->funcs->dp_audio_enable(pipe_ctx->stream_res.stream_enc);
}
- /* For MST, there are multiply stream go to only one link.
- * connect DIG back_end to front_end while enable_stream and
- * disconnect them during disable_stream
- * BY this, it is logic clean to separate stream and link */
- link->link_enc->funcs->connect_dig_be_to_fe(link->link_enc,
- pipe_ctx->stream_res.stream_enc->id, true);
+
+
}
* Check if FBC can be enabled
*/
static bool should_enable_fbc(struct dc *dc,
- struct dc_state *context)
+ struct dc_state *context,
+ uint32_t *pipe_idx)
{
- struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[0];
+ uint32_t i;
+ struct pipe_ctx *pipe_ctx = NULL;
+ struct resource_context *res_ctx = &context->res_ctx;
+
ASSERT(dc->fbc_compressor);
if (context->stream_count != 1)
return false;
+ for (i = 0; i < dc->res_pool->pipe_count; i++) {
+ if (res_ctx->pipe_ctx[i].stream) {
+ pipe_ctx = &res_ctx->pipe_ctx[i];
+ *pipe_idx = i;
+ break;
+ }
+ }
+
/* Only supports eDP */
if (pipe_ctx->stream->sink->link->connector_signal != SIGNAL_TYPE_EDP)
return false;
static void enable_fbc(struct dc *dc,
struct dc_state *context)
{
- if (should_enable_fbc(dc, context)) {
+ uint32_t pipe_idx = 0;
+
+ if (should_enable_fbc(dc, context, &pipe_idx)) {
/* Program GRPH COMPRESSED ADDRESS and PITCH */
struct compr_addr_and_pitch_params params = {0, 0, 0};
struct compressor *compr = dc->fbc_compressor;
- struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[0];
+ struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[pipe_idx];
+
params.source_view_width = pipe_ctx->stream->timing.h_addressable;
params.source_view_height = pipe_ctx->stream->timing.v_addressable;
}
}
+void dce110_set_cursor_position(struct pipe_ctx *pipe_ctx)
+{
+ struct dc_cursor_position pos_cpy = pipe_ctx->stream->cursor_position;
+ struct input_pixel_processor *ipp = pipe_ctx->plane_res.ipp;
+ struct mem_input *mi = pipe_ctx->plane_res.mi;
+ struct dc_cursor_mi_param param = {
+ .pixel_clk_khz = pipe_ctx->stream->timing.pix_clk_khz,
+ .ref_clk_khz = pipe_ctx->stream->ctx->dc->res_pool->ref_clock_inKhz,
+ .viewport_x_start = pipe_ctx->plane_res.scl_data.viewport.x,
+ .viewport_width = pipe_ctx->plane_res.scl_data.viewport.width,
+ .h_scale_ratio = pipe_ctx->plane_res.scl_data.ratios.horz
+ };
+
+ if (pipe_ctx->plane_state->address.type
+ == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE)
+ pos_cpy.enable = false;
+
+ if (pipe_ctx->top_pipe && pipe_ctx->plane_state != pipe_ctx->top_pipe->plane_state)
+ pos_cpy.enable = false;
+
+ if (ipp->funcs->ipp_cursor_set_position)
+ ipp->funcs->ipp_cursor_set_position(ipp, &pos_cpy, ¶m);
+ if (mi->funcs->set_cursor_position)
+ mi->funcs->set_cursor_position(mi, &pos_cpy, ¶m);
+}
+
+void dce110_set_cursor_attribute(struct pipe_ctx *pipe_ctx)
+{
+ struct dc_cursor_attributes *attributes = &pipe_ctx->stream->cursor_attributes;
+
+ if (pipe_ctx->plane_res.ipp->funcs->ipp_cursor_set_attributes)
+ pipe_ctx->plane_res.ipp->funcs->ipp_cursor_set_attributes(
+ pipe_ctx->plane_res.ipp, attributes);
+
+ if (pipe_ctx->plane_res.mi->funcs->set_cursor_attributes)
+ pipe_ctx->plane_res.mi->funcs->set_cursor_attributes(
+ pipe_ctx->plane_res.mi, attributes);
+
+ if (pipe_ctx->plane_res.xfm->funcs->set_cursor_attributes)
+ pipe_ctx->plane_res.xfm->funcs->set_cursor_attributes(
+ pipe_ctx->plane_res.xfm, attributes);
+}
+
static void ready_shared_resources(struct dc *dc, struct dc_state *context) {}
static void optimize_shared_resources(struct dc *dc) {}
.edp_backlight_control = hwss_edp_backlight_control,
.edp_power_control = hwss_edp_power_control,
.edp_wait_for_hpd_ready = hwss_edp_wait_for_hpd_ready,
+ .set_cursor_position = dce110_set_cursor_position,
+ .set_cursor_attribute = dce110_set_cursor_attribute
};
void dce110_hw_sequencer_construct(struct dc *dc)
return result;
}
+enum dc_status dce110_validate_plane(const struct dc_plane_state *plane_state,
+ struct dc_caps *caps)
+{
+ if (((plane_state->dst_rect.width * 2) < plane_state->src_rect.width) ||
+ ((plane_state->dst_rect.height * 2) < plane_state->src_rect.height))
+ return DC_FAIL_SURFACE_VALIDATE;
+
+ return DC_OK;
+}
+
static bool dce110_validate_surface_sets(
struct dc_state *context)
{
plane->src_rect.height > 1080))
return false;
+ /* we don't have the logic to support underlay
+ * only yet so block the use case where we get
+ * NV12 plane as top layer
+ */
+ if (j == 0)
+ return false;
+
/* irrespective of plane format,
* stream should be RGB encoded
*/
.link_enc_create = dce110_link_encoder_create,
.validate_guaranteed = dce110_validate_guaranteed,
.validate_bandwidth = dce110_validate_bandwidth,
+ .validate_plane = dce110_validate_plane,
.acquire_idle_pipe_for_layer = dce110_acquire_underlay,
.add_stream_to_ctx = dce110_add_stream_to_ctx,
.validate_global = dce110_validate_global
dc->caps.max_downscale_ratio = 200;
dc->caps.i2c_speed_in_khz = 100;
dc->caps.max_cursor_size = 128;
+ dc->caps.dual_link_dvi = true;
+
/*************************************************
* Create resources *
dc->caps.max_downscale_ratio = 200;
dc->caps.i2c_speed_in_khz = 100;
dc->caps.max_cursor_size = 128;
+ dc->caps.dual_link_dvi = true;
+
dc->debug = debug_defaults;
/*************************************************
dc->caps.max_downscale_ratio = 200;
dc->caps.i2c_speed_in_khz = 40;
dc->caps.max_cursor_size = 128;
+ dc->caps.dual_link_dvi = true;
/*************************************************
* Create resources *
static void disable_vga(
struct dce_hwseq *hws)
{
+ unsigned int in_vga1_mode = 0;
+ unsigned int in_vga2_mode = 0;
+ unsigned int in_vga3_mode = 0;
+ unsigned int in_vga4_mode = 0;
+
+ REG_GET(D1VGA_CONTROL, D1VGA_MODE_ENABLE, &in_vga1_mode);
+ REG_GET(D2VGA_CONTROL, D2VGA_MODE_ENABLE, &in_vga2_mode);
+ REG_GET(D3VGA_CONTROL, D3VGA_MODE_ENABLE, &in_vga3_mode);
+ REG_GET(D4VGA_CONTROL, D4VGA_MODE_ENABLE, &in_vga4_mode);
+
+ if (in_vga1_mode == 0 && in_vga2_mode == 0 &&
+ in_vga3_mode == 0 && in_vga4_mode == 0)
+ return;
+
REG_WRITE(D1VGA_CONTROL, 0);
REG_WRITE(D2VGA_CONTROL, 0);
REG_WRITE(D3VGA_CONTROL, 0);
REG_WRITE(D4VGA_CONTROL, 0);
+
+ /* HW Engineer's Notes:
+ * During switch from vga->extended, if we set the VGA_TEST_ENABLE and
+ * then hit the VGA_TEST_RENDER_START, then the DCHUBP timing gets updated correctly.
+ *
+ * Then vBIOS will have it poll for the VGA_TEST_RENDER_DONE and unset
+ * VGA_TEST_ENABLE, to leave it in the same state as before.
+ */
+ REG_UPDATE(VGA_TEST_CONTROL, VGA_TEST_ENABLE, 1);
+ REG_UPDATE(VGA_TEST_CONTROL, VGA_TEST_RENDER_START, 1);
}
static void dpp_pg_control(
&pipe_ctx->plane_res.scl_data.viewport_c);
}
+ if (pipe_ctx->stream->cursor_attributes.address.quad_part != 0) {
+ dc->hwss.set_cursor_position(pipe_ctx);
+ dc->hwss.set_cursor_attribute(pipe_ctx);
+ }
+
if (plane_state->update_flags.bits.full_update) {
/*gamut remap*/
program_gamut_remap(pipe_ctx);
return true;
}
-void dcn10_update_pending_status(struct pipe_ctx *pipe_ctx)
+static void dcn10_update_pending_status(struct pipe_ctx *pipe_ctx)
{
struct dc_plane_state *plane_state = pipe_ctx->plane_state;
struct timing_generator *tg = pipe_ctx->stream_res.tg;
}
}
-void dcn10_update_dchub(struct dce_hwseq *hws, struct dchub_init_data *dh_data)
+static void dcn10_update_dchub(struct dce_hwseq *hws, struct dchub_init_data *dh_data)
{
if (hws->ctx->dc->res_pool->hubbub != NULL)
hubbub1_update_dchub(hws->ctx->dc->res_pool->hubbub, dh_data);
}
+static void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx)
+{
+ struct dc_cursor_position pos_cpy = pipe_ctx->stream->cursor_position;
+ struct hubp *hubp = pipe_ctx->plane_res.hubp;
+ struct dpp *dpp = pipe_ctx->plane_res.dpp;
+ struct dc_cursor_mi_param param = {
+ .pixel_clk_khz = pipe_ctx->stream->timing.pix_clk_khz,
+ .ref_clk_khz = pipe_ctx->stream->ctx->dc->res_pool->ref_clock_inKhz,
+ .viewport_x_start = pipe_ctx->plane_res.scl_data.viewport.x,
+ .viewport_width = pipe_ctx->plane_res.scl_data.viewport.width,
+ .h_scale_ratio = pipe_ctx->plane_res.scl_data.ratios.horz
+ };
+
+ if (pipe_ctx->plane_state->address.type
+ == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE)
+ pos_cpy.enable = false;
+
+ if (pipe_ctx->top_pipe && pipe_ctx->plane_state != pipe_ctx->top_pipe->plane_state)
+ pos_cpy.enable = false;
+
+ hubp->funcs->set_cursor_position(hubp, &pos_cpy, ¶m);
+ dpp->funcs->set_cursor_position(dpp, &pos_cpy, ¶m, hubp->curs_attr.width);
+}
+
+static void dcn10_set_cursor_attribute(struct pipe_ctx *pipe_ctx)
+{
+ struct dc_cursor_attributes *attributes = &pipe_ctx->stream->cursor_attributes;
+
+ pipe_ctx->plane_res.hubp->funcs->set_cursor_attributes(
+ pipe_ctx->plane_res.hubp, attributes);
+ pipe_ctx->plane_res.dpp->funcs->set_cursor_attributes(
+ pipe_ctx->plane_res.dpp, attributes->color_format);
+}
+
static const struct hw_sequencer_funcs dcn10_funcs = {
.program_gamut_remap = program_gamut_remap,
.program_csc_matrix = program_csc_matrix,
.edp_backlight_control = hwss_edp_backlight_control,
.edp_power_control = hwss_edp_power_control,
.edp_wait_for_hpd_ready = hwss_edp_wait_for_hpd_ready,
+ .set_cursor_position = dcn10_set_cursor_position,
+ .set_cursor_attribute = dcn10_set_cursor_attribute
};
void (*enable_tmds_output)(struct link_encoder *enc,
enum clock_source_id clock_source,
enum dc_color_depth color_depth,
- bool hdmi,
- bool dual_link,
+ enum signal_type signal,
uint32_t pixel_clock);
void (*enable_dp_output)(struct link_encoder *enc,
const struct dc_link_settings *link_settings,
bool enable);
void (*edp_wait_for_hpd_ready)(struct dc_link *link, bool power_up);
+ void (*set_cursor_position)(struct pipe_ctx *pipe);
+ void (*set_cursor_attribute)(struct pipe_ctx *pipe);
+
};
void color_space_to_black_color(
core_dc->current_state->res_ctx.pipe_ctx[pipe_offset].stream_res.tg;
if (enable) {
- if (!tg->funcs->arm_vert_intr(tg, 2)) {
+ if (!tg || !tg->funcs->arm_vert_intr(tg, 2)) {
DC_ERROR("Failed to get VBLANK!\n");
return false;
}
struct link_encoder *enc,
enum clock_source_id clock_source,
enum dc_color_depth color_depth,
- bool hdmi,
- bool dual_link,
+ enum signal_type signal,
uint32_t pixel_clock) {}
static void virtual_link_encoder_enable_dp_output(
bool backlight_changed;
};
-enum {
- HDMI_PIXEL_CLOCK_IN_KHZ_297 = 297000,
- TMDS_PIXEL_CLOCK_IN_KHZ_165 = 165000
-};
-
/*
* DFS-bypass flag
*/
#ifndef __DC_SIGNAL_TYPES_H__
#define __DC_SIGNAL_TYPES_H__
+/* Minimum pixel clock, in KHz. For TMDS signal is 25.00 MHz */
+#define TMDS_MIN_PIXEL_CLOCK 25000
+/* Maximum pixel clock, in KHz. For TMDS signal is 165.00 MHz */
+#define TMDS_MAX_PIXEL_CLOCK 165000
+
enum signal_type {
SIGNAL_TYPE_NONE = 0L, /* no signal */
SIGNAL_TYPE_DVI_SINGLE_LINK = (1 << 0),
{0x67, 0x22, 0x00}, /* 0E: VCLK157_5 */
{0x6A, 0x22, 0x00}, /* 0F: VCLK162 */
{0x4d, 0x4c, 0x80}, /* 10: VCLK154 */
- {0xa7, 0x78, 0x80}, /* 11: VCLK83.5 */
+ {0x68, 0x6f, 0x80}, /* 11: VCLK83.5 */
{0x28, 0x49, 0x80}, /* 12: VCLK106.5 */
{0x37, 0x49, 0x80}, /* 13: VCLK146.25 */
{0x1f, 0x45, 0x80}, /* 14: VCLK148.5 */
{0x67, 0x22, 0x00}, /* 0E: VCLK157_5 */
{0x6A, 0x22, 0x00}, /* 0F: VCLK162 */
{0x4d, 0x4c, 0x80}, /* 10: VCLK154 */
- {0xa7, 0x78, 0x80}, /* 11: VCLK83.5 */
+ {0x68, 0x6f, 0x80}, /* 11: VCLK83.5 */
{0x28, 0x49, 0x80}, /* 12: VCLK106.5 */
{0x37, 0x49, 0x80}, /* 13: VCLK146.25 */
{0x1f, 0x45, 0x80}, /* 14: VCLK148.5 */
if (!fb)
return -ENOENT;
+ /* Multi-planar framebuffers need getfb2. */
+ if (fb->format->num_planes > 1) {
+ ret = -EINVAL;
+ goto out;
+ }
+
r->height = fb->height;
r->width = fb->width;
r->depth = fb->format->depth;
ret = -ENODEV;
}
+out:
drm_framebuffer_put(fb);
return ret;
* used when ret from 2nd level batch buffer
*/
int saved_buf_addr_type;
+ bool is_ctx_wa;
struct cmd_info *info;
bb->accessing = true;
bb->bb_start_cmd_va = s->ip_va;
+ if ((s->buf_type == BATCH_BUFFER_INSTRUCTION) && (!s->is_ctx_wa))
+ bb->bb_offset = s->ip_va - s->rb_va;
+ else
+ bb->bb_offset = 0;
+
/*
* ip_va saves the virtual address of the shadow batch buffer, while
* ip_gma saves the graphics address of the original batch buffer.
s.ring_tail = gma_tail;
s.rb_va = workload->shadow_ring_buffer_va;
s.workload = workload;
+ s.is_ctx_wa = false;
if ((bypass_scan_mask & (1 << workload->ring_id)) ||
gma_head == gma_tail)
s.ring_tail = gma_tail;
s.rb_va = wa_ctx->indirect_ctx.shadow_va;
s.workload = workload;
+ s.is_ctx_wa = true;
if (!intel_gvt_ggtt_validate_range(s.vgpu, s.ring_start, s.ring_size)) {
ret = -EINVAL;
* performace for batch mmio read/write, so we need
* handle forcewake mannually.
*/
+ intel_runtime_pm_get(dev_priv);
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
switch_mmio(pre, next, ring_id);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
+ intel_runtime_pm_put(dev_priv);
}
/**
pdp_pair[i].val = pdp[7 - i];
}
+/*
+ * when populating shadow ctx from guest, we should not overrride oa related
+ * registers, so that they will not be overlapped by guest oa configs. Thus
+ * made it possible to capture oa data from host for both host and guests.
+ */
+static void sr_oa_regs(struct intel_vgpu_workload *workload,
+ u32 *reg_state, bool save)
+{
+ struct drm_i915_private *dev_priv = workload->vgpu->gvt->dev_priv;
+ u32 ctx_oactxctrl = dev_priv->perf.oa.ctx_oactxctrl_offset;
+ u32 ctx_flexeu0 = dev_priv->perf.oa.ctx_flexeu0_offset;
+ int i = 0;
+ u32 flex_mmio[] = {
+ i915_mmio_reg_offset(EU_PERF_CNTL0),
+ i915_mmio_reg_offset(EU_PERF_CNTL1),
+ i915_mmio_reg_offset(EU_PERF_CNTL2),
+ i915_mmio_reg_offset(EU_PERF_CNTL3),
+ i915_mmio_reg_offset(EU_PERF_CNTL4),
+ i915_mmio_reg_offset(EU_PERF_CNTL5),
+ i915_mmio_reg_offset(EU_PERF_CNTL6),
+ };
+
+ if (!workload || !reg_state || workload->ring_id != RCS)
+ return;
+
+ if (save) {
+ workload->oactxctrl = reg_state[ctx_oactxctrl + 1];
+
+ for (i = 0; i < ARRAY_SIZE(workload->flex_mmio); i++) {
+ u32 state_offset = ctx_flexeu0 + i * 2;
+
+ workload->flex_mmio[i] = reg_state[state_offset + 1];
+ }
+ } else {
+ reg_state[ctx_oactxctrl] =
+ i915_mmio_reg_offset(GEN8_OACTXCONTROL);
+ reg_state[ctx_oactxctrl + 1] = workload->oactxctrl;
+
+ for (i = 0; i < ARRAY_SIZE(workload->flex_mmio); i++) {
+ u32 state_offset = ctx_flexeu0 + i * 2;
+ u32 mmio = flex_mmio[i];
+
+ reg_state[state_offset] = mmio;
+ reg_state[state_offset + 1] = workload->flex_mmio[i];
+ }
+ }
+}
+
static int populate_shadow_context(struct intel_vgpu_workload *workload)
{
struct intel_vgpu *vgpu = workload->vgpu;
page = i915_gem_object_get_page(ctx_obj, LRC_STATE_PN);
shadow_ring_context = kmap(page);
+ sr_oa_regs(workload, (u32 *)shadow_ring_context, true);
#define COPY_REG(name) \
intel_gvt_hypervisor_read_gpa(vgpu, workload->ring_context_gpa \
+ RING_CTX_OFF(name.val), &shadow_ring_context->name.val, 4)
sizeof(*shadow_ring_context),
I915_GTT_PAGE_SIZE - sizeof(*shadow_ring_context));
+ sr_oa_regs(workload, (u32 *)shadow_ring_context, false);
kunmap(page);
return 0;
}
goto err;
}
+ /* For privilge batch buffer and not wa_ctx, the bb_start_cmd_va
+ * is only updated into ring_scan_buffer, not real ring address
+ * allocated in later copy_workload_to_ring_buffer. pls be noted
+ * shadow_ring_buffer_va is now pointed to real ring buffer va
+ * in copy_workload_to_ring_buffer.
+ */
+
+ if (bb->bb_offset)
+ bb->bb_start_cmd_va = workload->shadow_ring_buffer_va
+ + bb->bb_offset;
+
/* relocate shadow batch buffer */
bb->bb_start_cmd_va[1] = i915_ggtt_offset(bb->vma);
if (gmadr_bytes == 8)
bitmap_zero(s->shadow_ctx_desc_updated, I915_NUM_ENGINES);
- s->workloads = kmem_cache_create("gvt-g_vgpu_workload",
- sizeof(struct intel_vgpu_workload), 0,
- SLAB_HWCACHE_ALIGN,
- NULL);
+ s->workloads = kmem_cache_create_usercopy("gvt-g_vgpu_workload",
+ sizeof(struct intel_vgpu_workload), 0,
+ SLAB_HWCACHE_ALIGN,
+ offsetof(struct intel_vgpu_workload, rb_tail),
+ sizeof_field(struct intel_vgpu_workload, rb_tail),
+ NULL);
if (!s->workloads) {
ret = -ENOMEM;
/* shadow batch buffer */
struct list_head shadow_bb;
struct intel_shadow_wa_ctx wa_ctx;
+
+ /* oa registers */
+ u32 oactxctrl;
+ u32 flex_mmio[7];
};
struct intel_vgpu_shadow_bb {
u32 *bb_start_cmd_va;
unsigned int clflush;
bool accessing;
+ unsigned long bb_offset;
};
#define workload_q_head(vgpu, ring_id) \
dma_fence_put(shared[i]);
kfree(shared);
+ /*
+ * If both shared fences and an exclusive fence exist,
+ * then by construction the shared fences must be later
+ * than the exclusive fence. If we successfully wait for
+ * all the shared fences, we know that the exclusive fence
+ * must all be signaled. If all the shared fences are
+ * signaled, we can prune the array and recover the
+ * floating references on the fences/requests.
+ */
prune_fences = count && timeout >= 0;
} else {
excl = reservation_object_get_excl_rcu(resv);
}
- if (excl && timeout >= 0) {
+ if (excl && timeout >= 0)
timeout = i915_gem_object_wait_fence(excl, flags, timeout,
rps_client);
- prune_fences = timeout >= 0;
- }
dma_fence_put(excl);
- /* Oportunistically prune the fences iff we know they have *all* been
+ /*
+ * Opportunistically prune the fences iff we know they have *all* been
* signaled and that the reservation object has not been changed (i.e.
* no new fences have been added).
*/
* rolling the global seqno forward (since this would complete requests
* for which we haven't set the fence error to EIO yet).
*/
- for_each_engine(engine, i915, id)
+ for_each_engine(engine, i915, id) {
+ i915_gem_reset_prepare_engine(engine);
engine->submit_request = nop_submit_request;
+ }
/*
* Make sure no one is running the old callback before we proceed with
intel_engine_init_global_seqno(engine,
intel_engine_last_submit(engine));
spin_unlock_irqrestore(&engine->timeline->lock, flags);
+
+ i915_gem_reset_finish_engine(engine);
}
set_bit(I915_WEDGED, &i915->gpu_error.flags);
*/
mutex_lock(&dev_priv->drm.struct_mutex);
dev_priv->perf.oa.exclusive_stream = NULL;
- mutex_unlock(&dev_priv->drm.struct_mutex);
-
dev_priv->perf.oa.ops.disable_metric_set(dev_priv);
+ mutex_unlock(&dev_priv->drm.struct_mutex);
free_oa_buffer(dev_priv);
* Note: it's only the RCS/Render context that has any OA state.
*/
static int gen8_configure_all_contexts(struct drm_i915_private *dev_priv,
- const struct i915_oa_config *oa_config,
- bool interruptible)
+ const struct i915_oa_config *oa_config)
{
struct i915_gem_context *ctx;
int ret;
unsigned int wait_flags = I915_WAIT_LOCKED;
- if (interruptible) {
- ret = i915_mutex_lock_interruptible(&dev_priv->drm);
- if (ret)
- return ret;
-
- wait_flags |= I915_WAIT_INTERRUPTIBLE;
- } else {
- mutex_lock(&dev_priv->drm.struct_mutex);
- }
+ lockdep_assert_held(&dev_priv->drm.struct_mutex);
/* Switch away from any user context. */
ret = gen8_switch_to_updated_kernel_context(dev_priv, oa_config);
}
out:
- mutex_unlock(&dev_priv->drm.struct_mutex);
-
return ret;
}
* to make sure all slices/subslices are ON before writing to NOA
* registers.
*/
- ret = gen8_configure_all_contexts(dev_priv, oa_config, true);
+ ret = gen8_configure_all_contexts(dev_priv, oa_config);
if (ret)
return ret;
static void gen8_disable_metric_set(struct drm_i915_private *dev_priv)
{
/* Reset all contexts' slices/subslices configurations. */
- gen8_configure_all_contexts(dev_priv, NULL, false);
+ gen8_configure_all_contexts(dev_priv, NULL);
I915_WRITE(GDT_CHICKEN_BITS, (I915_READ(GDT_CHICKEN_BITS) &
~GT_NOA_ENABLE));
static void gen10_disable_metric_set(struct drm_i915_private *dev_priv)
{
/* Reset all contexts' slices/subslices configurations. */
- gen8_configure_all_contexts(dev_priv, NULL, false);
+ gen8_configure_all_contexts(dev_priv, NULL);
/* Make sure we disable noa to save power. */
I915_WRITE(RPM_CONFIG1,
if (ret)
goto err_oa_buf_alloc;
+ ret = i915_mutex_lock_interruptible(&dev_priv->drm);
+ if (ret)
+ goto err_lock;
+
ret = dev_priv->perf.oa.ops.enable_metric_set(dev_priv,
stream->oa_config);
if (ret)
stream->ops = &i915_oa_stream_ops;
- /* Lock device for exclusive_stream access late because
- * enable_metric_set() might lock as well on gen8+.
- */
- ret = i915_mutex_lock_interruptible(&dev_priv->drm);
- if (ret)
- goto err_lock;
-
dev_priv->perf.oa.exclusive_stream = stream;
mutex_unlock(&dev_priv->drm.struct_mutex);
return 0;
-err_lock:
+err_enable:
dev_priv->perf.oa.ops.disable_metric_set(dev_priv);
+ mutex_unlock(&dev_priv->drm.struct_mutex);
-err_enable:
+err_lock:
free_oa_buffer(dev_priv);
err_oa_buf_alloc:
{
struct drm_i915_private *dev_priv = kdev_minor_to_i915(kdev);
struct intel_rps *rps = &dev_priv->gt_pm.rps;
- u32 val;
+ bool boost = false;
ssize_t ret;
+ u32 val;
ret = kstrtou32(buf, 0, &val);
if (ret)
return -EINVAL;
mutex_lock(&dev_priv->pcu_lock);
- rps->boost_freq = val;
+ if (val != rps->boost_freq) {
+ rps->boost_freq = val;
+ boost = atomic_read(&rps->num_waiters);
+ }
mutex_unlock(&dev_priv->pcu_lock);
+ if (boost)
+ schedule_work(&rps->work);
return count;
}
intel_prepare_dp_ddi_buffers(encoder, crtc_state);
intel_ddi_init_dp_buf_reg(encoder);
- if (!is_mst)
- intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_ON);
+ intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_ON);
intel_dp_start_link_train(intel_dp);
if (port != PORT_A || INTEL_GEN(dev_priv) >= 9)
intel_dp_stop_link_train(intel_dp);
struct drm_i915_private *dev_priv = to_i915(encoder->base.dev);
struct intel_digital_port *dig_port = enc_to_dig_port(&encoder->base);
struct intel_dp *intel_dp = &dig_port->dp;
- bool is_mst = intel_crtc_has_type(old_crtc_state, INTEL_OUTPUT_DP_MST);
/*
* Power down sink before disabling the port, otherwise we end
* up getting interrupts from the sink on detecting link loss.
*/
- if (!is_mst)
- intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);
+ intel_dp_sink_dpms(intel_dp, DRM_MODE_DPMS_OFF);
intel_disable_ddi_buf(encoder);
bxt_power_sequencer_idx(struct intel_dp *intel_dp)
{
struct drm_i915_private *dev_priv = to_i915(intel_dp_to_dev(intel_dp));
+ int backlight_controller = dev_priv->vbt.backlight.controller;
lockdep_assert_held(&dev_priv->pps_mutex);
/* We should never land here with regular DP ports */
WARN_ON(!intel_dp_is_edp(intel_dp));
- /*
- * TODO: BXT has 2 PPS instances. The correct port->PPS instance
- * mapping needs to be retrieved from VBT, for now just hard-code to
- * use instance #0 always.
- */
if (!intel_dp->pps_reset)
- return 0;
+ return backlight_controller;
intel_dp->pps_reset = false;
*/
intel_dp_init_panel_power_sequencer_registers(intel_dp, false);
- return 0;
+ return backlight_controller;
}
typedef bool (*vlv_pipe_check)(struct drm_i915_private *dev_priv,
*/
tmp = I915_READ_CTL(engine);
if (tmp & RING_WAIT) {
- i915_handle_error(dev_priv, 0,
+ i915_handle_error(dev_priv, BIT(engine->id),
"Kicking stuck wait on %s",
engine->name);
I915_WRITE_CTL(engine, tmp);
default:
return ENGINE_DEAD;
case 1:
- i915_handle_error(dev_priv, 0,
+ i915_handle_error(dev_priv, ALL_ENGINES,
"Kicking stuck semaphore on %s",
engine->name);
I915_WRITE_CTL(engine, tmp);
struct rb_node *rb;
unsigned long flags;
+ GEM_TRACE("%s\n", engine->name);
+
spin_lock_irqsave(&engine->timeline->lock, flags);
/* Cancel the requests on the HW and clear the ELSP tracker. */
*/
clear_bit(ENGINE_IRQ_EXECLIST, &engine->irq_posted);
+ /* Mark all CS interrupts as complete */
+ execlists->active = 0;
+
spin_unlock_irqrestore(&engine->timeline->lock, flags);
}
struct drm_crtc_state *old_crtc_state)
{
drm_crtc_vblank_on(crtc);
+}
+static void ipu_crtc_atomic_flush(struct drm_crtc *crtc,
+ struct drm_crtc_state *old_crtc_state)
+{
spin_lock_irq(&crtc->dev->event_lock);
if (crtc->state->event) {
WARN_ON(drm_crtc_vblank_get(crtc));
.mode_set_nofb = ipu_crtc_mode_set_nofb,
.atomic_check = ipu_crtc_atomic_check,
.atomic_begin = ipu_crtc_atomic_begin,
+ .atomic_flush = ipu_crtc_atomic_flush,
.atomic_disable = ipu_crtc_atomic_disable,
.atomic_enable = ipu_crtc_atomic_enable,
};
#include <drm/drm_plane_helper.h>
#include "video/imx-ipu-v3.h"
+#include "imx-drm.h"
#include "ipuv3-plane.h"
struct ipu_plane_state {
kfree(ipu_plane);
}
-void ipu_plane_state_reset(struct drm_plane *plane)
+static void ipu_plane_state_reset(struct drm_plane *plane)
{
struct ipu_plane_state *ipu_state;
plane->state = &ipu_state->base;
}
-struct drm_plane_state *ipu_plane_duplicate_state(struct drm_plane *plane)
+static struct drm_plane_state *
+ipu_plane_duplicate_state(struct drm_plane *plane)
{
struct ipu_plane_state *state;
return &state->base;
}
-void ipu_plane_destroy_state(struct drm_plane *plane,
- struct drm_plane_state *state)
+static void ipu_plane_destroy_state(struct drm_plane *plane,
+ struct drm_plane_state *state)
{
struct ipu_plane_state *ipu_state = to_ipu_plane_state(state);
struct nouveau_encoder *nv_encoder = bl_get_data(bd);
struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev);
struct nvif_object *device = &drm->client.device.object;
- int or = nv_encoder->or;
+ int or = ffs(nv_encoder->dcb->or) - 1;
u32 div = 1025;
u32 val;
struct nouveau_encoder *nv_encoder = bl_get_data(bd);
struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev);
struct nvif_object *device = &drm->client.device.object;
- int or = nv_encoder->or;
+ int or = ffs(nv_encoder->dcb->or) - 1;
u32 div = 1025;
u32 val = (bd->props.brightness * div) / 100;
struct nouveau_encoder *nv_encoder = bl_get_data(bd);
struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev);
struct nvif_object *device = &drm->client.device.object;
- int or = nv_encoder->or;
+ int or = ffs(nv_encoder->dcb->or) - 1;
u32 div, val;
div = nvif_rd32(device, NV50_PDISP_SOR_PWM_DIV(or));
struct nouveau_encoder *nv_encoder = bl_get_data(bd);
struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev);
struct nvif_object *device = &drm->client.device.object;
- int or = nv_encoder->or;
+ int or = ffs(nv_encoder->dcb->or) - 1;
u32 div, val;
div = nvif_rd32(device, NV50_PDISP_SOR_PWM_DIV(or));
return -ENODEV;
}
- if (!nvif_rd32(device, NV50_PDISP_SOR_PWM_CTL(nv_encoder->or)))
+ if (!nvif_rd32(device, NV50_PDISP_SOR_PWM_CTL(ffs(nv_encoder->dcb->or) - 1)))
return 0;
if (drm->client.device.info.chipset <= 0xa0 ||
struct nvif_device *device = &drm->client.device;
struct drm_connector *connector;
+ INIT_LIST_HEAD(&drm->bl_connectors);
+
if (apple_gmux_present()) {
NV_INFO(drm, "Apple GMUX detected: not registering Nouveau backlight interface\n");
return 0;
}
- INIT_LIST_HEAD(&drm->bl_connectors);
-
list_for_each_entry(connector, &dev->mode_config.connector_list, head) {
if (connector->connector_type != DRM_MODE_CONNECTOR_LVDS &&
connector->connector_type != DRM_MODE_CONNECTOR_eDP)
tail = this->addr + this->size;
if (vmm->func->page_block && next && next->page != p)
- tail = ALIGN_DOWN(addr, vmm->func->page_block);
+ tail = ALIGN_DOWN(tail, vmm->func->page_block);
if (addr <= tail && tail - addr >= size) {
rb_erase(&this->tree, &vmm->free);
case CHIP_KAVERI:
rdev->config.cik.max_shader_engines = 1;
rdev->config.cik.max_tile_pipes = 4;
- if ((rdev->pdev->device == 0x1304) ||
- (rdev->pdev->device == 0x1305) ||
- (rdev->pdev->device == 0x130C) ||
- (rdev->pdev->device == 0x130F) ||
- (rdev->pdev->device == 0x1310) ||
- (rdev->pdev->device == 0x1311) ||
- (rdev->pdev->device == 0x131C)) {
- rdev->config.cik.max_cu_per_sh = 8;
- rdev->config.cik.max_backends_per_se = 2;
- } else if ((rdev->pdev->device == 0x1309) ||
- (rdev->pdev->device == 0x130A) ||
- (rdev->pdev->device == 0x130D) ||
- (rdev->pdev->device == 0x1313) ||
- (rdev->pdev->device == 0x131D)) {
- rdev->config.cik.max_cu_per_sh = 6;
- rdev->config.cik.max_backends_per_se = 2;
- } else if ((rdev->pdev->device == 0x1306) ||
- (rdev->pdev->device == 0x1307) ||
- (rdev->pdev->device == 0x130B) ||
- (rdev->pdev->device == 0x130E) ||
- (rdev->pdev->device == 0x1315) ||
- (rdev->pdev->device == 0x1318) ||
- (rdev->pdev->device == 0x131B)) {
- rdev->config.cik.max_cu_per_sh = 4;
- rdev->config.cik.max_backends_per_se = 1;
- } else {
- rdev->config.cik.max_cu_per_sh = 3;
- rdev->config.cik.max_backends_per_se = 1;
- }
+ rdev->config.cik.max_cu_per_sh = 8;
+ rdev->config.cik.max_backends_per_se = 2;
rdev->config.cik.max_sh_per_se = 1;
rdev->config.cik.max_texture_channel_caches = 4;
rdev->config.cik.max_gprs = 256;
/* don't do anything if sink is not display port, i.e.,
* passive dp->(dvi|hdmi) adaptor
*/
- if (dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT) {
- int saved_dpms = connector->dpms;
- /* Only turn off the display if it's physically disconnected */
- if (!radeon_hpd_sense(rdev, radeon_connector->hpd.hpd)) {
- drm_helper_connector_dpms(connector, DRM_MODE_DPMS_OFF);
- } else if (radeon_dp_needs_link_train(radeon_connector)) {
- /* Don't try to start link training before we
- * have the dpcd */
- if (!radeon_dp_getdpcd(radeon_connector))
- return;
-
- /* set it to OFF so that drm_helper_connector_dpms()
- * won't return immediately since the current state
- * is ON at this point.
- */
- connector->dpms = DRM_MODE_DPMS_OFF;
- drm_helper_connector_dpms(connector, DRM_MODE_DPMS_ON);
- }
- connector->dpms = saved_dpms;
+ if (dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT &&
+ radeon_hpd_sense(rdev, radeon_connector->hpd.hpd) &&
+ radeon_dp_needs_link_train(radeon_connector)) {
+ /* Don't start link training before we have the DPCD */
+ if (!radeon_dp_getdpcd(radeon_connector))
+ return;
+
+ /* Turn the connector off and back on immediately, which
+ * will trigger link training
+ */
+ drm_helper_connector_dpms(connector, DRM_MODE_DPMS_OFF);
+ drm_helper_connector_dpms(connector, DRM_MODE_DPMS_ON);
}
}
}
struct radeon_bo *robj = gem_to_radeon_bo(gobj);
if (robj) {
- if (robj->gem_base.import_attach)
- drm_prime_gem_destroy(&robj->gem_base, robj->tbo.sg);
radeon_mn_unregister(robj);
radeon_bo_unref(&robj);
}
mutex_unlock(&bo->rdev->gem.mutex);
radeon_bo_clear_surface_reg(bo);
WARN_ON_ONCE(!list_empty(&bo->va));
+ if (bo->gem_base.import_attach)
+ drm_prime_gem_destroy(&bo->gem_base, bo->tbo.sg);
drm_gem_object_release(&bo->gem_base);
kfree(bo);
}
DRM_DEBUG_DRIVER("Disabling the CRTC\n");
+ drm_crtc_vblank_off(crtc);
+
sun4i_tcon_set_status(scrtc->tcon, encoder, false);
if (crtc->state->event && !crtc->state->active) {
DRM_DEBUG_DRIVER("Enabling the CRTC\n");
sun4i_tcon_set_status(scrtc->tcon, encoder, true);
+
+ drm_crtc_vblank_on(crtc);
}
static void sun4i_crtc_mode_set_nofb(struct drm_crtc *crtc)
static int sun4i_dclk_set_phase(struct clk_hw *hw, int degrees)
{
struct sun4i_dclk *dclk = hw_to_dclk(hw);
+ u32 val = degrees / 120;
+
+ val <<= 28;
regmap_update_bits(dclk->regmap, SUN4I_TCON0_IO_POL_REG,
GENMASK(29, 28),
- degrees / 120);
+ val);
return 0;
}
/* drm_vblank_init calls kcalloc, which can fail */
ret = drm_vblank_init(drm, drm->mode_config.num_crtc);
if (ret)
- goto free_mem_region;
+ goto cleanup_mode_config;
drm->irq_enabled = true;
sun4i_framebuffer_free(drm);
cleanup_mode_config:
drm_mode_config_cleanup(drm);
-free_mem_region:
of_reserved_mem_device_release(dev);
free_drm:
drm_dev_unref(drm);
&sun4i_hdmi_regmap_config);
if (IS_ERR(hdmi->regmap)) {
dev_err(dev, "Couldn't create HDMI encoder regmap\n");
- return PTR_ERR(hdmi->regmap);
+ ret = PTR_ERR(hdmi->regmap);
+ goto err_disable_mod_clk;
}
ret = sun4i_tmds_create(hdmi);
hdmi->ddc_parent_clk = devm_clk_get(dev, "ddc");
if (IS_ERR(hdmi->ddc_parent_clk)) {
dev_err(dev, "Couldn't get the HDMI DDC clock\n");
- return PTR_ERR(hdmi->ddc_parent_clk);
+ ret = PTR_ERR(hdmi->ddc_parent_clk);
+ goto err_disable_mod_clk;
}
} else {
hdmi->ddc_parent_clk = hdmi->tmds_clk;
DRM_DEBUG_DRIVER("Vertical parameters OK\n");
+ tcon->dclk_min_div = 6;
+ tcon->dclk_max_div = 127;
rounded_rate = clk_round_rate(tcon->dclk, rate);
if (rounded_rate < rate)
return MODE_CLOCK_LOW;
return;
}
- if (enabled)
+ if (enabled) {
clk_prepare_enable(clk);
- else
+ clk_rate_exclusive_get(clk);
+ } else {
+ clk_rate_exclusive_put(clk);
clk_disable_unprepare(clk);
+ }
}
static void sun4i_tcon_lvds_set_status(struct sun4i_tcon *tcon,
const struct drm_display_mode *mode)
{
/* Configure the dot clock */
- clk_set_rate_exclusive(tcon->dclk, mode->crtc_clock * 1000);
+ clk_set_rate(tcon->dclk, mode->crtc_clock * 1000);
/* Set the resolution */
regmap_write(tcon->regs, SUN4I_TCON0_BASIC0_REG,
WARN_ON(!tcon->quirks->has_channel_1);
/* Configure the dot clock */
- clk_set_rate_exclusive(tcon->sclk1, mode->crtc_clock * 1000);
+ clk_set_rate(tcon->sclk1, mode->crtc_clock * 1000);
/* Adjust clock delay */
clk_delay = sun4i_tcon_get_clk_delay(mode, 1);
return ret;
}
- /*
- * This can only be made optional since we've had DT nodes
- * without the LVDS reset properties.
- *
- * If the property is missing, just disable LVDS, and print a
- * warning.
- */
- tcon->lvds_rst = devm_reset_control_get_optional(dev, "lvds");
- if (IS_ERR(tcon->lvds_rst)) {
- dev_err(dev, "Couldn't get our reset line\n");
- return PTR_ERR(tcon->lvds_rst);
- } else if (tcon->lvds_rst) {
- has_lvds_rst = true;
- reset_control_reset(tcon->lvds_rst);
- } else {
- has_lvds_rst = false;
- }
+ if (tcon->quirks->supports_lvds) {
+ /*
+ * This can only be made optional since we've had DT
+ * nodes without the LVDS reset properties.
+ *
+ * If the property is missing, just disable LVDS, and
+ * print a warning.
+ */
+ tcon->lvds_rst = devm_reset_control_get_optional(dev, "lvds");
+ if (IS_ERR(tcon->lvds_rst)) {
+ dev_err(dev, "Couldn't get our reset line\n");
+ return PTR_ERR(tcon->lvds_rst);
+ } else if (tcon->lvds_rst) {
+ has_lvds_rst = true;
+ reset_control_reset(tcon->lvds_rst);
+ } else {
+ has_lvds_rst = false;
+ }
- /*
- * This can only be made optional since we've had DT nodes
- * without the LVDS reset properties.
- *
- * If the property is missing, just disable LVDS, and print a
- * warning.
- */
- if (tcon->quirks->has_lvds_alt) {
- tcon->lvds_pll = devm_clk_get(dev, "lvds-alt");
- if (IS_ERR(tcon->lvds_pll)) {
- if (PTR_ERR(tcon->lvds_pll) == -ENOENT) {
- has_lvds_alt = false;
+ /*
+ * This can only be made optional since we've had DT
+ * nodes without the LVDS reset properties.
+ *
+ * If the property is missing, just disable LVDS, and
+ * print a warning.
+ */
+ if (tcon->quirks->has_lvds_alt) {
+ tcon->lvds_pll = devm_clk_get(dev, "lvds-alt");
+ if (IS_ERR(tcon->lvds_pll)) {
+ if (PTR_ERR(tcon->lvds_pll) == -ENOENT) {
+ has_lvds_alt = false;
+ } else {
+ dev_err(dev, "Couldn't get the LVDS PLL\n");
+ return PTR_ERR(tcon->lvds_pll);
+ }
} else {
- dev_err(dev, "Couldn't get the LVDS PLL\n");
- return PTR_ERR(tcon->lvds_pll);
+ has_lvds_alt = true;
}
- } else {
- has_lvds_alt = true;
}
- }
- if (!has_lvds_rst || (tcon->quirks->has_lvds_alt && !has_lvds_alt)) {
- dev_warn(dev,
- "Missing LVDS properties, Please upgrade your DT\n");
- dev_warn(dev, "LVDS output disabled\n");
- can_lvds = false;
+ if (!has_lvds_rst ||
+ (tcon->quirks->has_lvds_alt && !has_lvds_alt)) {
+ dev_warn(dev, "Missing LVDS properties, Please upgrade your DT\n");
+ dev_warn(dev, "LVDS output disabled\n");
+ can_lvds = false;
+ } else {
+ can_lvds = true;
+ }
} else {
- can_lvds = true;
+ can_lvds = false;
}
ret = sun4i_tcon_init_clocks(dev, tcon);
};
static const struct sun4i_tcon_quirks sun8i_a83t_lcd_quirks = {
- /* nothing is supported */
+ .supports_lvds = true,
};
static const struct sun4i_tcon_quirks sun8i_v3s_quirks = {
bool has_channel_1; /* a33 does not have channel 1 */
bool has_lvds_alt; /* Does the LVDS clock have a parent other than the TCON clock? */
bool needs_de_be_mux; /* sun6i needs mux to select backend */
+ bool supports_lvds; /* Does the TCON support an LVDS output? */
/* callback to handle tcon muxing options */
int (*set_mux)(struct sun4i_tcon *, const struct drm_encoder *);
if (!IS_ERR(primary))
drm_plane_cleanup(primary);
- if (group && tegra->domain) {
- iommu_detach_group(tegra->domain, group);
+ if (group && dc->domain) {
+ if (group == tegra->group) {
+ iommu_detach_group(dc->domain, group);
+ tegra->group = NULL;
+ }
+
dc->domain = NULL;
}
static int tegra_dc_exit(struct host1x_client *client)
{
+ struct drm_device *drm = dev_get_drvdata(client->parent);
struct iommu_group *group = iommu_group_get(client->dev);
struct tegra_dc *dc = host1x_client_to_dc(client);
+ struct tegra_drm *tegra = drm->dev_private;
int err;
devm_free_irq(dc->dev, dc->irq, dc);
}
if (group && dc->domain) {
- iommu_detach_group(dc->domain, group);
+ if (group == tegra->group) {
+ iommu_detach_group(dc->domain, group);
+ tegra->group = NULL;
+ }
+
dc->domain = NULL;
}
drm_kms_helper_poll_fini(drm);
tegra_drm_fb_exit(drm);
+ drm_atomic_helper_shutdown(drm);
drm_mode_config_cleanup(drm);
err = host1x_device_exit(device);
struct tegra_dsi *dsi = host1x_client_to_dsi(client);
tegra_output_exit(&dsi->output);
- regulator_disable(dsi->vdd);
return 0;
}
case WIN_COLOR_DEPTH_B8G8R8X8:
*alpha = WIN_COLOR_DEPTH_B8G8R8A8;
return 0;
+
+ case WIN_COLOR_DEPTH_B5G6R5:
+ *alpha = opaque;
+ return 0;
}
return -EINVAL;
unsigned int zpos[2];
unsigned int i;
- for (i = 0; i < 3; i++)
- state->dependent[i] = false;
-
for (i = 0; i < 2; i++)
zpos[i] = 0;
index = tegra_plane_get_overlap_index(tegra, p);
+ state->dependent[index] = false;
+
/*
* If any of the other planes is on top of this plane and uses
* a format with an alpha component, mark this plane as being
{
unsigned long start = vma->vm_start;
unsigned long size = vma->vm_end - vma->vm_start;
- unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
+ unsigned long offset;
unsigned long page, pos;
- if (offset + size > info->fix.smem_len)
+ if (vma->vm_pgoff > (~0UL >> PAGE_SHIFT))
+ return -EINVAL;
+
+ offset = vma->vm_pgoff << PAGE_SHIFT;
+
+ if (offset > info->fix.smem_len || size > info->fix.smem_len - offset)
return -EINVAL;
pos = (unsigned long)info->fix.smem_start + offset;
*/
void vmw_svga_disable(struct vmw_private *dev_priv)
{
+ /*
+ * Disabling SVGA will turn off device modesetting capabilities, so
+ * notify KMS about that so that it doesn't cache atomic state that
+ * isn't valid anymore, for example crtcs turned on.
+ * Strictly we'd want to do this under the SVGA lock (or an SVGA mutex),
+ * but vmw_kms_lost_device() takes the reservation sem and thus we'll
+ * end up with lock order reversal. Thus, a master may actually perform
+ * a new modeset just after we call vmw_kms_lost_device() and race with
+ * vmw_svga_disable(), but that should at worst cause atomic KMS state
+ * to be inconsistent with the device, causing modesetting problems.
+ *
+ */
+ vmw_kms_lost_device(dev_priv->dev);
ttm_write_lock(&dev_priv->reservation_sem, false);
spin_lock(&dev_priv->svga_lock);
if (dev_priv->bdev.man[TTM_PL_VRAM].use_type) {
int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
void vmw_kms_legacy_hotspot_clear(struct vmw_private *dev_priv);
+void vmw_kms_lost_device(struct drm_device *dev);
int vmw_dumb_create(struct drm_file *file_priv,
struct drm_device *dev,
#include <drm/drm_atomic_helper.h>
#include <drm/drm_rect.h>
-
/* Might need a hrtimer here? */
#define VMWGFX_PRESENT_RATE ((HZ / 60 > 0) ? HZ / 60 : 1)
* Helper to be used if an error forces the caller to undo the actions of
* vmw_kms_helper_resource_prepare.
*/
-void vmw_kms_helper_resource_revert(struct vmw_resource *res)
+void vmw_kms_helper_resource_revert(struct vmw_validation_ctx *ctx)
{
- vmw_kms_helper_buffer_revert(res->backup);
+ struct vmw_resource *res = ctx->res;
+
+ vmw_kms_helper_buffer_revert(ctx->buf);
+ vmw_dmabuf_unreference(&ctx->buf);
vmw_resource_unreserve(res, false, NULL, 0);
mutex_unlock(&res->dev_priv->cmdbuf_mutex);
}
* interrupted by a signal.
*/
int vmw_kms_helper_resource_prepare(struct vmw_resource *res,
- bool interruptible)
+ bool interruptible,
+ struct vmw_validation_ctx *ctx)
{
int ret = 0;
+ ctx->buf = NULL;
+ ctx->res = res;
+
if (interruptible)
ret = mutex_lock_interruptible(&res->dev_priv->cmdbuf_mutex);
else
res->dev_priv->has_mob);
if (ret)
goto out_unreserve;
+
+ ctx->buf = vmw_dmabuf_reference(res->backup);
}
ret = vmw_resource_validate(res);
if (ret)
return 0;
out_revert:
- vmw_kms_helper_buffer_revert(res->backup);
+ vmw_kms_helper_buffer_revert(ctx->buf);
out_unreserve:
vmw_resource_unreserve(res, false, NULL, 0);
out_unlock:
* @out_fence: Optional pointer to a fence pointer. If non-NULL, a
* ref-counted fence pointer is returned here.
*/
-void vmw_kms_helper_resource_finish(struct vmw_resource *res,
- struct vmw_fence_obj **out_fence)
+void vmw_kms_helper_resource_finish(struct vmw_validation_ctx *ctx,
+ struct vmw_fence_obj **out_fence)
{
- if (res->backup || out_fence)
- vmw_kms_helper_buffer_finish(res->dev_priv, NULL, res->backup,
+ struct vmw_resource *res = ctx->res;
+
+ if (ctx->buf || out_fence)
+ vmw_kms_helper_buffer_finish(res->dev_priv, NULL, ctx->buf,
out_fence, NULL);
vmw_resource_unreserve(res, false, NULL, 0);
return drm_atomic_helper_set_config(set, ctx);
}
+
+
+/**
+ * vmw_kms_lost_device - Notify kms that modesetting capabilities will be lost
+ *
+ * @dev: Pointer to the drm device
+ */
+void vmw_kms_lost_device(struct drm_device *dev)
+{
+ drm_atomic_helper_shutdown(dev);
+}
int set_gui_y;
};
+struct vmw_validation_ctx {
+ struct vmw_resource *res;
+ struct vmw_dma_buffer *buf;
+};
+
#define vmw_crtc_to_du(x) \
container_of(x, struct vmw_display_unit, crtc)
#define vmw_connector_to_du(x) \
struct drm_vmw_fence_rep __user *
user_fence_rep);
int vmw_kms_helper_resource_prepare(struct vmw_resource *res,
- bool interruptible);
-void vmw_kms_helper_resource_revert(struct vmw_resource *res);
-void vmw_kms_helper_resource_finish(struct vmw_resource *res,
+ bool interruptible,
+ struct vmw_validation_ctx *ctx);
+void vmw_kms_helper_resource_revert(struct vmw_validation_ctx *ctx);
+void vmw_kms_helper_resource_finish(struct vmw_validation_ctx *ctx,
struct vmw_fence_obj **out_fence);
int vmw_kms_readback(struct vmw_private *dev_priv,
struct drm_file *file_priv,
int vmw_kms_set_config(struct drm_mode_set *set,
struct drm_modeset_acquire_ctx *ctx);
-
#endif
struct vmw_framebuffer_surface *vfbs =
container_of(framebuffer, typeof(*vfbs), base);
struct vmw_kms_sou_surface_dirty sdirty;
+ struct vmw_validation_ctx ctx;
int ret;
if (!srf)
srf = &vfbs->surface->res;
- ret = vmw_kms_helper_resource_prepare(srf, true);
+ ret = vmw_kms_helper_resource_prepare(srf, true, &ctx);
if (ret)
return ret;
ret = vmw_kms_helper_dirty(dev_priv, framebuffer, clips, vclips,
dest_x, dest_y, num_clips, inc,
&sdirty.base);
- vmw_kms_helper_resource_finish(srf, out_fence);
+ vmw_kms_helper_resource_finish(&ctx, out_fence);
return ret;
}
struct vmw_framebuffer_surface *vfbs =
container_of(framebuffer, typeof(*vfbs), base);
struct vmw_stdu_dirty sdirty;
+ struct vmw_validation_ctx ctx;
int ret;
if (!srf)
srf = &vfbs->surface->res;
- ret = vmw_kms_helper_resource_prepare(srf, true);
+ ret = vmw_kms_helper_resource_prepare(srf, true, &ctx);
if (ret)
return ret;
dest_x, dest_y, num_clips, inc,
&sdirty.base);
out_finish:
- vmw_kms_helper_resource_finish(srf, out_fence);
+ vmw_kms_helper_resource_finish(&ctx, out_fence);
return ret;
}
{
int prg_chan = ipu_prg_ipu_to_prg_chan(ipu_chan->num);
struct ipu_prg *prg = ipu_chan->ipu->prg_priv;
- struct ipu_prg_channel *chan = &prg->chan[prg_chan];
+ struct ipu_prg_channel *chan;
u32 val;
- if (!chan->enabled || prg_chan < 0)
+ if (prg_chan < 0)
+ return;
+
+ chan = &prg->chan[prg_chan];
+ if (!chan->enabled)
return;
pm_runtime_get_sync(prg->dev);
{
int prg_chan = ipu_prg_ipu_to_prg_chan(ipu_chan->num);
struct ipu_prg *prg = ipu_chan->ipu->prg_priv;
- struct ipu_prg_channel *chan = &prg->chan[prg_chan];
+ struct ipu_prg_channel *chan;
u32 val;
int ret;
if (prg_chan < 0)
return prg_chan;
+ chan = &prg->chan[prg_chan];
+
if (chan->enabled) {
ipu_pre_update(prg->pres[chan->used_pre], *eba);
return 0;
dst_release(dst);
}
- if (ndev->flags & IFF_LOOPBACK) {
- ret = rdma_translate_ip(dst_in, addr);
- /*
- * Put the loopback device and get the translated
- * device instead.
- */
+ if (ndev) {
+ if (ndev->flags & IFF_LOOPBACK)
+ ret = rdma_translate_ip(dst_in, addr);
+ else
+ addr->bound_dev_if = ndev->ifindex;
dev_put(ndev);
- ndev = dev_get_by_index(addr->net, addr->bound_dev_if);
- } else {
- addr->bound_dev_if = ndev->ifindex;
}
- dev_put(ndev);
return ret;
}
continue;
/* different dest port -> unique */
- if (!cma_any_port(cur_daddr) &&
+ if (!cma_any_port(daddr) &&
+ !cma_any_port(cur_daddr) &&
(dport != cur_dport))
continue;
continue;
/* different dst address -> unique */
- if (!cma_any_addr(cur_daddr) &&
+ if (!cma_any_addr(daddr) &&
+ !cma_any_addr(cur_daddr) &&
cma_addr_cmp(daddr, cur_daddr))
continue;
}
#endif
}
+ daddr = cma_dst_addr(id_priv);
+ daddr->sa_family = addr->sa_family;
+
ret = cma_get_port(id_priv);
if (ret)
goto err2;
- daddr = cma_dst_addr(id_priv);
- daddr->sa_family = addr->sa_family;
-
return 0;
err2:
if (id_priv->cma_dev)
struct cma_multicast *mc;
int ret;
+ if (!id->device)
+ return -EINVAL;
+
id_priv = container_of(id, struct rdma_id_private, id);
if (!cma_comp(id_priv, RDMA_CM_ADDR_BOUND) &&
!cma_comp(id_priv, RDMA_CM_ADDR_RESOLVED))
.exit = cma_exit_net,
.id = &cma_pernet_id,
.size = sizeof(struct cma_pernet),
- .async = true,
};
static int __init cma_init(void)
/* # of WCs to poll for with a single call to ib_poll_cq */
#define IB_POLL_BATCH 16
+#define IB_POLL_BATCH_DIRECT 8
/* # of WCs to iterate over before yielding */
#define IB_POLL_BUDGET_IRQ 256
#define IB_POLL_FLAGS \
(IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)
-static int __ib_process_cq(struct ib_cq *cq, int budget, struct ib_wc *poll_wc)
+static int __ib_process_cq(struct ib_cq *cq, int budget, struct ib_wc *wcs,
+ int batch)
{
int i, n, completed = 0;
- struct ib_wc *wcs = poll_wc ? : cq->wc;
/*
* budget might be (-1) if the caller does not
* want to bound this call, thus we need unsigned
* minimum here.
*/
- while ((n = ib_poll_cq(cq, min_t(u32, IB_POLL_BATCH,
- budget - completed), wcs)) > 0) {
+ while ((n = ib_poll_cq(cq, min_t(u32, batch,
+ budget - completed), wcs)) > 0) {
for (i = 0; i < n; i++) {
struct ib_wc *wc = &wcs[i];
completed += n;
- if (n != IB_POLL_BATCH ||
- (budget != -1 && completed >= budget))
+ if (n != batch || (budget != -1 && completed >= budget))
break;
}
*/
int ib_process_cq_direct(struct ib_cq *cq, int budget)
{
- struct ib_wc wcs[IB_POLL_BATCH];
+ struct ib_wc wcs[IB_POLL_BATCH_DIRECT];
- return __ib_process_cq(cq, budget, wcs);
+ return __ib_process_cq(cq, budget, wcs, IB_POLL_BATCH_DIRECT);
}
EXPORT_SYMBOL(ib_process_cq_direct);
struct ib_cq *cq = container_of(iop, struct ib_cq, iop);
int completed;
- completed = __ib_process_cq(cq, budget, NULL);
+ completed = __ib_process_cq(cq, budget, cq->wc, IB_POLL_BATCH);
if (completed < budget) {
irq_poll_complete(&cq->iop);
if (ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0)
struct ib_cq *cq = container_of(work, struct ib_cq, work);
int completed;
- completed = __ib_process_cq(cq, IB_POLL_BUDGET_WORKQUEUE, NULL);
+ completed = __ib_process_cq(cq, IB_POLL_BUDGET_WORKQUEUE, cq->wc,
+ IB_POLL_BATCH);
if (completed >= IB_POLL_BUDGET_WORKQUEUE ||
ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0)
queue_work(ib_comp_wq, &cq->work);
ret = device->query_device(device, &device->attrs, &uhw);
if (ret) {
pr_warn("Couldn't query the device attributes\n");
- goto cache_cleanup;
+ goto cg_cleanup;
}
ret = ib_device_register_sysfs(device, port_callback);
if (ret) {
pr_warn("Couldn't register device %s with driver model\n",
device->name);
- goto cache_cleanup;
+ goto cg_cleanup;
}
device->reg_state = IB_DEV_REGISTERED;
mutex_unlock(&device_mutex);
return 0;
+cg_cleanup:
+ ib_device_unregister_rdmacg(device);
cache_cleanup:
ib_cache_cleanup_one(device);
ib_cache_release_one(device);
* our feet
*/
rtnl_lock();
+ down_read(&net_rwsem);
for_each_net(net)
for_each_netdev(net, ndev)
if (is_eth_port_of_netdev(ib_dev, port, rdma_ndev, ndev))
add_netdev_ips(ib_dev, port, rdma_ndev, ndev);
+ up_read(&net_rwsem);
rtnl_unlock();
}
resolved_dev = dev_get_by_index(dev_addr.net,
dev_addr.bound_dev_if);
- if (resolved_dev->flags & IFF_LOOPBACK) {
- dev_put(resolved_dev);
- resolved_dev = idev;
- dev_hold(resolved_dev);
+ if (!resolved_dev) {
+ dev_put(idev);
+ return -ENODEV;
}
ndev = ib_get_ndev_from_path(rec);
rcu_read_lock();
ctx = idr_find(&ctx_idr, id);
if (!ctx)
ctx = ERR_PTR(-ENOENT);
- else if (ctx->file != file)
+ else if (ctx->file != file || !ctx->cm_id)
ctx = ERR_PTR(-EINVAL);
return ctx;
}
struct rdma_ucm_create_id cmd;
struct rdma_ucm_create_id_resp resp;
struct ucma_context *ctx;
+ struct rdma_cm_id *cm_id;
enum ib_qp_type qp_type;
int ret;
return -ENOMEM;
ctx->uid = cmd.uid;
- ctx->cm_id = rdma_create_id(current->nsproxy->net_ns,
- ucma_event_handler, ctx, cmd.ps, qp_type);
- if (IS_ERR(ctx->cm_id)) {
- ret = PTR_ERR(ctx->cm_id);
+ cm_id = rdma_create_id(current->nsproxy->net_ns,
+ ucma_event_handler, ctx, cmd.ps, qp_type);
+ if (IS_ERR(cm_id)) {
+ ret = PTR_ERR(cm_id);
goto err1;
}
ret = -EFAULT;
goto err2;
}
+
+ ctx->cm_id = cm_id;
return 0;
err2:
- rdma_destroy_id(ctx->cm_id);
+ rdma_destroy_id(cm_id);
err1:
mutex_lock(&mut);
idr_remove(&ctx_idr, ctx->id);
mutex_unlock(&mut);
+ mutex_lock(&file->mut);
+ list_del(&ctx->list);
+ mutex_unlock(&file->mut);
kfree(ctx);
return ret;
}
int in_len, int out_len)
{
struct rdma_ucm_resolve_ip cmd;
+ struct sockaddr *src, *dst;
struct ucma_context *ctx;
int ret;
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
return -EFAULT;
+ src = (struct sockaddr *) &cmd.src_addr;
+ dst = (struct sockaddr *) &cmd.dst_addr;
+ if (!rdma_addr_size(src) || !rdma_addr_size(dst))
+ return -EINVAL;
+
ctx = ucma_get_ctx(file, cmd.id);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
- ret = rdma_resolve_addr(ctx->cm_id, (struct sockaddr *) &cmd.src_addr,
- (struct sockaddr *) &cmd.dst_addr,
- cmd.timeout_ms);
+ ret = rdma_resolve_addr(ctx->cm_id, src, dst, cmd.timeout_ms);
ucma_put_ctx(ctx);
return ret;
}
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
return -EFAULT;
+ if (cmd.qp_state > IB_QPS_ERR)
+ return -EINVAL;
+
ctx = ucma_get_ctx(file, cmd.id);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
+ if (unlikely(cmd.optval > KMALLOC_MAX_SIZE))
+ return -EINVAL;
+
optval = memdup_user((void __user *) (unsigned long) cmd.optval,
cmd.optlen);
if (IS_ERR(optval)) {
return -ENOSPC;
addr = (struct sockaddr *) &cmd->addr;
- if (!cmd->addr_size || (cmd->addr_size != rdma_addr_size(addr)))
+ if (cmd->addr_size != rdma_addr_size(addr))
return -EINVAL;
if (cmd->join_flags == RDMA_MC_JOIN_FLAG_FULLMEMBER)
join_cmd.uid = cmd.uid;
join_cmd.id = cmd.id;
join_cmd.addr_size = rdma_addr_size((struct sockaddr *) &cmd.addr);
+ if (!join_cmd.addr_size)
+ return -EINVAL;
+
join_cmd.join_flags = RDMA_MC_JOIN_FLAG_FULLMEMBER;
memcpy(&join_cmd.addr, &cmd.addr, join_cmd.addr_size);
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
return -EFAULT;
+ if (!rdma_addr_size((struct sockaddr *)&cmd.addr))
+ return -EINVAL;
+
return ucma_process_join(file, &cmd, out_len);
}
#define BNXT_RE_PAGE_SIZE_8M BIT(BNXT_RE_PAGE_SHIFT_8M)
#define BNXT_RE_PAGE_SIZE_1G BIT(BNXT_RE_PAGE_SHIFT_1G)
-#define BNXT_RE_MAX_MR_SIZE_LOW BIT(BNXT_RE_PAGE_SHIFT_1G)
-#define BNXT_RE_MAX_MR_SIZE_HIGH BIT(39)
+#define BNXT_RE_MAX_MR_SIZE_LOW BIT_ULL(BNXT_RE_PAGE_SHIFT_1G)
+#define BNXT_RE_MAX_MR_SIZE_HIGH BIT_ULL(39)
#define BNXT_RE_MAX_MR_SIZE BNXT_RE_MAX_MR_SIZE_HIGH
#define BNXT_RE_MAX_QPC_COUNT (64 * 1024)
return 0;
}
-static unsigned long bnxt_re_lock_cqs(struct bnxt_re_qp *qp)
+unsigned long bnxt_re_lock_cqs(struct bnxt_re_qp *qp)
__acquires(&qp->scq->cq_lock) __acquires(&qp->rcq->cq_lock)
{
unsigned long flags;
return flags;
}
-static void bnxt_re_unlock_cqs(struct bnxt_re_qp *qp,
- unsigned long flags)
+void bnxt_re_unlock_cqs(struct bnxt_re_qp *qp,
+ unsigned long flags)
__releases(&qp->scq->cq_lock) __releases(&qp->rcq->cq_lock)
{
if (qp->rcq != qp->scq)
int status;
union ib_gid sgid;
struct ib_gid_attr sgid_attr;
+ unsigned int flags;
u8 nw_type;
qp->qplib_qp.modify_flags = 0;
dev_dbg(rdev_to_dev(rdev),
"Move QP = %p to flush list\n",
qp);
+ flags = bnxt_re_lock_cqs(qp);
bnxt_qplib_add_flush_qp(&qp->qplib_qp);
+ bnxt_re_unlock_cqs(qp, flags);
}
if (!qp->sumem &&
qp->qplib_qp.state == CMDQ_MODIFY_QP_NEW_STATE_RESET) {
dev_dbg(rdev_to_dev(rdev),
"Move QP = %p out of flush list\n",
qp);
+ flags = bnxt_re_lock_cqs(qp);
bnxt_qplib_clean_qp(&qp->qplib_qp);
+ bnxt_re_unlock_cqs(qp, flags);
}
}
if (qp_attr_mask & IB_QP_EN_SQD_ASYNC_NOTIFY) {
wqe->type = BNXT_QPLIB_SWQE_TYPE_LOCAL_INV;
wqe->local_inv.inv_l_key = wr->ex.invalidate_rkey;
+ /* Need unconditional fence for local invalidate
+ * opcode to work as expected.
+ */
+ wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_UC_FENCE;
+
if (wr->send_flags & IB_SEND_SIGNALED)
wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_SIGNAL_COMP;
- if (wr->send_flags & IB_SEND_FENCE)
- wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_UC_FENCE;
if (wr->send_flags & IB_SEND_SOLICITED)
wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_SOLICIT_EVENT;
wqe->frmr.levels = qplib_frpl->hwq.level + 1;
wqe->type = BNXT_QPLIB_SWQE_TYPE_REG_MR;
- if (wr->wr.send_flags & IB_SEND_FENCE)
- wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_UC_FENCE;
+ /* Need unconditional fence for reg_mr
+ * opcode to function as expected.
+ */
+
+ wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_UC_FENCE;
+
if (wr->wr.send_flags & IB_SEND_SIGNALED)
wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_SIGNAL_COMP;
int umem_pgs, page_shift, rc;
if (length > BNXT_RE_MAX_MR_SIZE) {
- dev_err(rdev_to_dev(rdev), "MR Size: %lld > Max supported:%ld\n",
+ dev_err(rdev_to_dev(rdev), "MR Size: %lld > Max supported:%lld\n",
length, BNXT_RE_MAX_MR_SIZE);
return ERR_PTR(-ENOMEM);
}
struct ib_udata *udata);
int bnxt_re_dealloc_ucontext(struct ib_ucontext *context);
int bnxt_re_mmap(struct ib_ucontext *context, struct vm_area_struct *vma);
+
+unsigned long bnxt_re_lock_cqs(struct bnxt_re_qp *qp);
+void bnxt_re_unlock_cqs(struct bnxt_re_qp *qp, unsigned long flags);
#endif /* __BNXT_RE_IB_VERBS_H__ */
struct bnxt_re_qp *qp)
{
struct ib_event event;
+ unsigned int flags;
+
+ if (qp->qplib_qp.state == CMDQ_MODIFY_QP_NEW_STATE_ERR) {
+ flags = bnxt_re_lock_cqs(qp);
+ bnxt_qplib_add_flush_qp(&qp->qplib_qp);
+ bnxt_re_unlock_cqs(qp, flags);
+ }
memset(&event, 0, sizeof(event));
if (qp->qplib_qp.srq) {
switch (re_work->event) {
case NETDEV_REGISTER:
rc = bnxt_re_ib_reg(rdev);
- if (rc)
+ if (rc) {
dev_err(rdev_to_dev(rdev),
"Failed to register with IB: %#x", rc);
+ bnxt_re_remove_one(rdev);
+ bnxt_re_dev_unreg(rdev);
+ }
break;
case NETDEV_UP:
bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1,
}
}
-void bnxt_qplib_acquire_cq_locks(struct bnxt_qplib_qp *qp,
- unsigned long *flags)
- __acquires(&qp->scq->hwq.lock) __acquires(&qp->rcq->hwq.lock)
+static void bnxt_qplib_acquire_cq_flush_locks(struct bnxt_qplib_qp *qp,
+ unsigned long *flags)
+ __acquires(&qp->scq->flush_lock) __acquires(&qp->rcq->flush_lock)
{
- spin_lock_irqsave(&qp->scq->hwq.lock, *flags);
+ spin_lock_irqsave(&qp->scq->flush_lock, *flags);
if (qp->scq == qp->rcq)
- __acquire(&qp->rcq->hwq.lock);
+ __acquire(&qp->rcq->flush_lock);
else
- spin_lock(&qp->rcq->hwq.lock);
+ spin_lock(&qp->rcq->flush_lock);
}
-void bnxt_qplib_release_cq_locks(struct bnxt_qplib_qp *qp,
- unsigned long *flags)
- __releases(&qp->scq->hwq.lock) __releases(&qp->rcq->hwq.lock)
+static void bnxt_qplib_release_cq_flush_locks(struct bnxt_qplib_qp *qp,
+ unsigned long *flags)
+ __releases(&qp->scq->flush_lock) __releases(&qp->rcq->flush_lock)
{
if (qp->scq == qp->rcq)
- __release(&qp->rcq->hwq.lock);
+ __release(&qp->rcq->flush_lock);
else
- spin_unlock(&qp->rcq->hwq.lock);
- spin_unlock_irqrestore(&qp->scq->hwq.lock, *flags);
-}
-
-static struct bnxt_qplib_cq *bnxt_qplib_find_buddy_cq(struct bnxt_qplib_qp *qp,
- struct bnxt_qplib_cq *cq)
-{
- struct bnxt_qplib_cq *buddy_cq = NULL;
-
- if (qp->scq == qp->rcq)
- buddy_cq = NULL;
- else if (qp->scq == cq)
- buddy_cq = qp->rcq;
- else
- buddy_cq = qp->scq;
- return buddy_cq;
-}
-
-static void bnxt_qplib_lock_buddy_cq(struct bnxt_qplib_qp *qp,
- struct bnxt_qplib_cq *cq)
- __acquires(&buddy_cq->hwq.lock)
-{
- struct bnxt_qplib_cq *buddy_cq = NULL;
-
- buddy_cq = bnxt_qplib_find_buddy_cq(qp, cq);
- if (!buddy_cq)
- __acquire(&cq->hwq.lock);
- else
- spin_lock(&buddy_cq->hwq.lock);
-}
-
-static void bnxt_qplib_unlock_buddy_cq(struct bnxt_qplib_qp *qp,
- struct bnxt_qplib_cq *cq)
- __releases(&buddy_cq->hwq.lock)
-{
- struct bnxt_qplib_cq *buddy_cq = NULL;
-
- buddy_cq = bnxt_qplib_find_buddy_cq(qp, cq);
- if (!buddy_cq)
- __release(&cq->hwq.lock);
- else
- spin_unlock(&buddy_cq->hwq.lock);
+ spin_unlock(&qp->rcq->flush_lock);
+ spin_unlock_irqrestore(&qp->scq->flush_lock, *flags);
}
void bnxt_qplib_add_flush_qp(struct bnxt_qplib_qp *qp)
{
unsigned long flags;
- bnxt_qplib_acquire_cq_locks(qp, &flags);
+ bnxt_qplib_acquire_cq_flush_locks(qp, &flags);
__bnxt_qplib_add_flush_qp(qp);
- bnxt_qplib_release_cq_locks(qp, &flags);
+ bnxt_qplib_release_cq_flush_locks(qp, &flags);
}
static void __bnxt_qplib_del_flush_qp(struct bnxt_qplib_qp *qp)
{
unsigned long flags;
- bnxt_qplib_acquire_cq_locks(qp, &flags);
+ bnxt_qplib_acquire_cq_flush_locks(qp, &flags);
__clean_cq(qp->scq, (u64)(unsigned long)qp);
qp->sq.hwq.prod = 0;
qp->sq.hwq.cons = 0;
qp->rq.hwq.cons = 0;
__bnxt_qplib_del_flush_qp(qp);
- bnxt_qplib_release_cq_locks(qp, &flags);
+ bnxt_qplib_release_cq_flush_locks(qp, &flags);
}
static void bnxt_qpn_cqn_sched_task(struct work_struct *work)
u32 sw_cons, raw_cons;
u16 type;
int budget = nq->budget;
- u64 q_handle;
+ uintptr_t q_handle;
/* Service the NQ until empty */
raw_cons = hwq->cons;
/* Configure the request */
req.dpi = cpu_to_le32(srq->dpi->dpi);
- req.srq_handle = cpu_to_le64(srq);
+ req.srq_handle = cpu_to_le64((uintptr_t)srq);
req.srq_size = cpu_to_le16((u16)srq->hwq.max_elements);
pbl = &srq->hwq.pbl[PBL_LVL_0];
/* Must block new posting of SQ and RQ */
qp->state = CMDQ_MODIFY_QP_NEW_STATE_ERR;
bnxt_qplib_cancel_phantom_processing(qp);
-
- /* Add qp to flush list of the CQ */
- __bnxt_qplib_add_flush_qp(qp);
}
/* Note: SQE is valid from sw_sq_cons up to cqe_sq_cons (exclusive)
sw_sq_cons, cqe->wr_id, cqe->status);
cqe++;
(*budget)--;
- bnxt_qplib_lock_buddy_cq(qp, cq);
bnxt_qplib_mark_qp_error(qp);
- bnxt_qplib_unlock_buddy_cq(qp, cq);
+ /* Add qp to flush list of the CQ */
+ bnxt_qplib_add_flush_qp(qp);
} else {
if (swq->flags & SQ_SEND_FLAGS_SIGNAL_COMP) {
/* Before we complete, do WA 9060 */
if (hwcqe->status != CQ_RES_RC_STATUS_OK) {
qp->state = CMDQ_MODIFY_QP_NEW_STATE_ERR;
/* Add qp to flush list of the CQ */
- bnxt_qplib_lock_buddy_cq(qp, cq);
- __bnxt_qplib_add_flush_qp(qp);
- bnxt_qplib_unlock_buddy_cq(qp, cq);
+ bnxt_qplib_add_flush_qp(qp);
}
}
if (hwcqe->status != CQ_RES_RC_STATUS_OK) {
qp->state = CMDQ_MODIFY_QP_NEW_STATE_ERR;
/* Add qp to flush list of the CQ */
- bnxt_qplib_lock_buddy_cq(qp, cq);
- __bnxt_qplib_add_flush_qp(qp);
- bnxt_qplib_unlock_buddy_cq(qp, cq);
+ bnxt_qplib_add_flush_qp(qp);
}
}
done:
bool bnxt_qplib_is_cq_empty(struct bnxt_qplib_cq *cq)
{
struct cq_base *hw_cqe, **hw_cqe_ptr;
- unsigned long flags;
u32 sw_cons, raw_cons;
bool rc = true;
- spin_lock_irqsave(&cq->hwq.lock, flags);
raw_cons = cq->hwq.cons;
sw_cons = HWQ_CMP(raw_cons, &cq->hwq);
hw_cqe_ptr = (struct cq_base **)cq->hwq.pbl_ptr;
/* Check for Valid bit. If the CQE is valid, return false */
rc = !CQE_CMP_VALID(hw_cqe, raw_cons, cq->hwq.max_elements);
- spin_unlock_irqrestore(&cq->hwq.lock, flags);
return rc;
}
if (hwcqe->status != CQ_RES_RC_STATUS_OK) {
qp->state = CMDQ_MODIFY_QP_NEW_STATE_ERR;
/* Add qp to flush list of the CQ */
- bnxt_qplib_lock_buddy_cq(qp, cq);
- __bnxt_qplib_add_flush_qp(qp);
- bnxt_qplib_unlock_buddy_cq(qp, cq);
+ bnxt_qplib_add_flush_qp(qp);
}
}
*/
/* Add qp to flush list of the CQ */
- bnxt_qplib_lock_buddy_cq(qp, cq);
- __bnxt_qplib_add_flush_qp(qp);
- bnxt_qplib_unlock_buddy_cq(qp, cq);
+ bnxt_qplib_add_flush_qp(qp);
done:
return rc;
}
u32 budget = num_cqes;
unsigned long flags;
- spin_lock_irqsave(&cq->hwq.lock, flags);
+ spin_lock_irqsave(&cq->flush_lock, flags);
list_for_each_entry(qp, &cq->sqf_head, sq_flush) {
dev_dbg(&cq->hwq.pdev->dev,
"QPLIB: FP: Flushing SQ QP= %p",
qp);
__flush_rq(&qp->rq, qp, &cqe, &budget);
}
- spin_unlock_irqrestore(&cq->hwq.lock, flags);
+ spin_unlock_irqrestore(&cq->flush_lock, flags);
return num_cqes - budget;
}
int num_cqes, struct bnxt_qplib_qp **lib_qp)
{
struct cq_base *hw_cqe, **hw_cqe_ptr;
- unsigned long flags;
u32 sw_cons, raw_cons;
int budget, rc = 0;
- spin_lock_irqsave(&cq->hwq.lock, flags);
raw_cons = cq->hwq.cons;
budget = num_cqes;
bnxt_qplib_arm_cq(cq, DBR_DBR_TYPE_CQ);
}
exit:
- spin_unlock_irqrestore(&cq->hwq.lock, flags);
return num_cqes - budget;
}
void bnxt_qplib_req_notify_cq(struct bnxt_qplib_cq *cq, u32 arm_type)
{
- unsigned long flags;
-
- spin_lock_irqsave(&cq->hwq.lock, flags);
if (arm_type)
bnxt_qplib_arm_cq(cq, arm_type);
/* Using cq->arm_state variable to track whether to issue cq handler */
atomic_set(&cq->arm_state, 1);
- spin_unlock_irqrestore(&cq->hwq.lock, flags);
}
void bnxt_qplib_flush_cqn_wq(struct bnxt_qplib_qp *qp)
struct list_head sqf_head, rqf_head;
atomic_t arm_state;
spinlock_t compl_lock; /* synch CQ handlers */
+/* Locking Notes:
+ * QP can move to error state from modify_qp, async error event or error
+ * CQE as part of poll_cq. When QP is moved to error state, it gets added
+ * to two flush lists, one each for SQ and RQ.
+ * Each flush list is protected by qplib_cq->flush_lock. Both scq and rcq
+ * flush_locks should be acquired when QP is moved to error. The control path
+ * operations(modify_qp and async error events) are synchronized with poll_cq
+ * using upper level CQ locks (bnxt_re_cq->cq_lock) of both SCQ and RCQ.
+ * The qplib_cq->flush_lock is required to synchronize two instances of poll_cq
+ * of the same QP while manipulating the flush list.
+ */
+ spinlock_t flush_lock; /* QP flush management */
};
#define BNXT_QPLIB_MAX_IRRQE_ENTRY_SIZE sizeof(struct xrrq_irrq)
err_event->res_err_state_reason);
if (!qp)
break;
- bnxt_qplib_acquire_cq_locks(qp, &flags);
bnxt_qplib_mark_qp_error(qp);
- bnxt_qplib_release_cq_locks(qp, &flags);
+ rcfw->aeq_handler(rcfw, qp_event, qp);
break;
default:
/* Command Response */
int rc;
RCFW_CMD_PREP(req, INITIALIZE_FW, cmd_flags);
-
+ /* Supply (log-base-2-of-host-page-size - base-page-shift)
+ * to bono to adjust the doorbell page sizes.
+ */
+ req.log2_dbr_pg_size = cpu_to_le16(PAGE_SHIFT -
+ RCFW_DBR_BASE_PAGE_SHIFT);
/*
* VFs need not setup the HW context area, PF
* shall setup this area for VF. Skipping the
#define RCFW_COMM_SIZE 0x104
#define RCFW_DBR_PCI_BAR_REGION 2
+#define RCFW_DBR_BASE_PAGE_SHIFT 12
#define RCFW_CMD_PREP(req, CMD, cmd_flags) \
do { \
attr->max_pkey = le32_to_cpu(sb->max_pkeys);
attr->max_inline_data = le32_to_cpu(sb->max_inline_data);
- attr->l2_db_size = (sb->l2_db_space_size + 1) * PAGE_SIZE;
+ attr->l2_db_size = (sb->l2_db_space_size + 1) *
+ (0x01 << RCFW_DBR_BASE_PAGE_SHIFT);
attr->max_sgid = le32_to_cpu(sb->max_gid);
bnxt_qplib_query_version(rcfw, attr->fw_ver);
#define CMDQ_INITIALIZE_FW_TIM_PG_SIZE_PG_2M (0x3UL << 4)
#define CMDQ_INITIALIZE_FW_TIM_PG_SIZE_PG_8M (0x4UL << 4)
#define CMDQ_INITIALIZE_FW_TIM_PG_SIZE_PG_1G (0x5UL << 4)
- __le16 reserved16;
+ /* This value is (log-base-2-of-DBR-page-size - 12).
+ * 0 for 4KB. HW supported values are enumerated below.
+ */
+ __le16 log2_dbr_pg_size;
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_MASK 0xfUL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_SFT 0
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_4K 0x0UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_8K 0x1UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_16K 0x2UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_32K 0x3UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_64K 0x4UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_128K 0x5UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_256K 0x6UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_512K 0x7UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_1M 0x8UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_2M 0x9UL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_4M 0xaUL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_8M 0xbUL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_16M 0xcUL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_32M 0xdUL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_64M 0xeUL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_128M 0xfUL
+ #define CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_LAST \
+ CMDQ_INITIALIZE_FW_LOG2_DBR_PG_SIZE_PG_128M
__le64 qpc_page_dir;
__le64 mrw_page_dir;
__le64 srq_page_dir;
wc->dlid_path_bits = 0;
if (is_eth) {
+ wc->slid = 0;
wc->vlan_id = be16_to_cpu(hdr->tun.sl_vid);
memcpy(&(wc->smac[0]), (char *)&hdr->tun.mac_31_0, 4);
memcpy(&(wc->smac[4]), (char *)&hdr->tun.slid_mac_47_32, 2);
}
}
- wc->slid = be16_to_cpu(cqe->rlid);
g_mlpath_rqpn = be32_to_cpu(cqe->g_mlpath_rqpn);
wc->src_qp = g_mlpath_rqpn & 0xffffff;
wc->dlid_path_bits = (g_mlpath_rqpn >> 24) & 0x7f;
wc->wc_flags |= mlx4_ib_ipoib_csum_ok(cqe->status,
cqe->checksum) ? IB_WC_IP_CSUM_OK : 0;
if (is_eth) {
+ wc->slid = 0;
wc->sl = be16_to_cpu(cqe->sl_vid) >> 13;
if (be32_to_cpu(cqe->vlan_my_qpn) &
MLX4_CQE_CVLAN_PRESENT_MASK) {
memcpy(wc->smac, cqe->smac, ETH_ALEN);
wc->wc_flags |= (IB_WC_WITH_VLAN | IB_WC_WITH_SMAC);
} else {
+ wc->slid = be16_to_cpu(cqe->rlid);
wc->sl = be16_to_cpu(cqe->sl_vid) >> 12;
wc->vlan_id = 0xffff;
}
gid_tbl[i].version = 2;
if (!ipv6_addr_v4mapped((struct in6_addr *)&gids[i].gid))
gid_tbl[i].type = 1;
- else
- memset(&gid_tbl[i].gid, 0, 12);
}
}
if (!gids) {
ret = -ENOMEM;
} else {
- for (i = 0; i < MLX4_MAX_PORT_GIDS; i++)
- memcpy(&gids[i].gid, &port_gid_table->gids[i].gid, sizeof(union ib_gid));
+ for (i = 0; i < MLX4_MAX_PORT_GIDS; i++) {
+ memcpy(&gids[i].gid,
+ &port_gid_table->gids[i].gid,
+ sizeof(union ib_gid));
+ gids[i].gid_type =
+ port_gid_table->gids[i].gid_type;
+ }
}
}
spin_unlock_bh(&iboe->lock);
wc->ex.invalidate_rkey = be32_to_cpu(cqe->imm_inval_pkey);
break;
}
- wc->slid = be16_to_cpu(cqe->slid);
wc->src_qp = be32_to_cpu(cqe->flags_rqpn) & 0xffffff;
wc->dlid_path_bits = cqe->ml_path;
g = (be32_to_cpu(cqe->flags_rqpn) >> 28) & 3;
}
if (ll != IB_LINK_LAYER_ETHERNET) {
+ wc->slid = be16_to_cpu(cqe->slid);
wc->sl = (be32_to_cpu(cqe->flags_rqpn) >> 24) & 0xf;
return;
}
+ wc->slid = 0;
vlan_present = cqe->l4_l3_hdr_type & 0x1;
roce_packet_type = (be32_to_cpu(cqe->flags_rqpn) >> 24) & 0x3;
if (vlan_present) {
static void dump_cqe(struct mlx5_ib_dev *dev, struct mlx5_err_cqe *cqe)
{
- __be32 *p = (__be32 *)cqe;
- int i;
-
mlx5_ib_warn(dev, "dump error cqe\n");
- for (i = 0; i < sizeof(*cqe) / 16; i++, p += 4)
- pr_info("%08x %08x %08x %08x\n", be32_to_cpu(p[0]),
- be32_to_cpu(p[1]), be32_to_cpu(p[2]),
- be32_to_cpu(p[3]));
+ mlx5_dump_err_cqe(dev->mdev, cqe);
}
static void mlx5_handle_error_cqe(struct mlx5_ib_dev *dev,
if (ucmd.reserved0 || ucmd.reserved1)
return -EINVAL;
- umem = ib_umem_get(context, ucmd.buf_addr, entries * ucmd.cqe_size,
+ /* check multiplication overflow */
+ if (ucmd.cqe_size && SIZE_MAX / ucmd.cqe_size <= entries - 1)
+ return -EINVAL;
+
+ umem = ib_umem_get(context, ucmd.buf_addr,
+ (size_t)ucmd.cqe_size * entries,
IB_ACCESS_LOCAL_WRITE, 1);
if (IS_ERR(umem)) {
err = PTR_ERR(umem);
STAGE_CREATE(MLX5_IB_STAGE_BFREG,
mlx5_ib_stage_bfrag_init,
mlx5_ib_stage_bfrag_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_PRE_IB_REG_UMR,
+ NULL,
+ mlx5_ib_stage_pre_ib_reg_umr_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
mlx5_ib_stage_ib_reg_init,
mlx5_ib_stage_ib_reg_cleanup),
- STAGE_CREATE(MLX5_IB_STAGE_UMR_RESOURCES,
- mlx5_ib_stage_umr_res_init,
- mlx5_ib_stage_umr_res_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_POST_IB_REG_UMR,
+ mlx5_ib_stage_post_ib_reg_umr_init,
+ NULL),
STAGE_CREATE(MLX5_IB_STAGE_CLASS_ATTR,
mlx5_ib_stage_class_attr_init,
NULL),
struct mlx5_ib_multiport_info *mpi;
struct mlx5_ib_port *port;
+ if (!mlx5_core_mp_enabled(ibdev->mdev) ||
+ ll != IB_LINK_LAYER_ETHERNET) {
+ if (native_port_num)
+ *native_port_num = ib_port_num;
+ return ibdev->mdev;
+ }
+
if (native_port_num)
*native_port_num = 1;
- if (!mlx5_core_mp_enabled(ibdev->mdev) || ll != IB_LINK_LAYER_ETHERNET)
- return ibdev->mdev;
-
port = &ibdev->port[ib_port_num - 1];
if (!port)
return NULL;
struct mlx5_ib_dev *ibdev;
struct ib_event ibev;
bool fatal = false;
- u8 port = 0;
+ u8 port = (u8)work->param;
if (mlx5_core_is_mp_slave(work->dev)) {
ibdev = mlx5_ib_get_ibdev_from_mpi(work->context);
case MLX5_DEV_EVENT_PORT_UP:
case MLX5_DEV_EVENT_PORT_DOWN:
case MLX5_DEV_EVENT_PORT_INITIALIZED:
- port = (u8)work->param;
-
/* In RoCE, port up/down events are handled in
* mlx5_netdev_event().
*/
case MLX5_DEV_EVENT_LID_CHANGE:
ibev.event = IB_EVENT_LID_CHANGE;
- port = (u8)work->param;
break;
case MLX5_DEV_EVENT_PKEY_CHANGE:
ibev.event = IB_EVENT_PKEY_CHANGE;
- port = (u8)work->param;
-
schedule_work(&ibdev->devr.ports[port - 1].pkey_change_work);
break;
case MLX5_DEV_EVENT_GUID_CHANGE:
ibev.event = IB_EVENT_GID_CHANGE;
- port = (u8)work->param;
break;
case MLX5_DEV_EVENT_CLIENT_REREG:
ibev.event = IB_EVENT_CLIENT_REREGISTER;
- port = (u8)work->param;
break;
case MLX5_DEV_EVENT_DELAY_DROP_TIMEOUT:
schedule_work(&ibdev->delay_drop.delay_drop_work);
ibev.device = &ibdev->ib_dev;
ibev.element.port_num = port;
- if (port < 1 || port > ibdev->num_ports) {
+ if (!rdma_is_port_valid(&ibdev->ib_dev, port)) {
mlx5_ib_warn(ibdev, "warning: event on port %d\n", port);
goto out;
}
return ib_register_device(&dev->ib_dev, NULL);
}
-void mlx5_ib_stage_ib_reg_cleanup(struct mlx5_ib_dev *dev)
+void mlx5_ib_stage_pre_ib_reg_umr_cleanup(struct mlx5_ib_dev *dev)
{
- ib_unregister_device(&dev->ib_dev);
+ destroy_umrc_res(dev);
}
-int mlx5_ib_stage_umr_res_init(struct mlx5_ib_dev *dev)
+void mlx5_ib_stage_ib_reg_cleanup(struct mlx5_ib_dev *dev)
{
- return create_umr_res(dev);
+ ib_unregister_device(&dev->ib_dev);
}
-void mlx5_ib_stage_umr_res_cleanup(struct mlx5_ib_dev *dev)
+int mlx5_ib_stage_post_ib_reg_umr_init(struct mlx5_ib_dev *dev)
{
- destroy_umrc_res(dev);
+ return create_umr_res(dev);
}
static int mlx5_ib_stage_delay_drop_init(struct mlx5_ib_dev *dev)
STAGE_CREATE(MLX5_IB_STAGE_BFREG,
mlx5_ib_stage_bfrag_init,
mlx5_ib_stage_bfrag_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_PRE_IB_REG_UMR,
+ NULL,
+ mlx5_ib_stage_pre_ib_reg_umr_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
mlx5_ib_stage_ib_reg_init,
mlx5_ib_stage_ib_reg_cleanup),
- STAGE_CREATE(MLX5_IB_STAGE_UMR_RESOURCES,
- mlx5_ib_stage_umr_res_init,
- mlx5_ib_stage_umr_res_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_POST_IB_REG_UMR,
+ mlx5_ib_stage_post_ib_reg_umr_init,
+ NULL),
STAGE_CREATE(MLX5_IB_STAGE_DELAY_DROP,
mlx5_ib_stage_delay_drop_init,
mlx5_ib_stage_delay_drop_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_BFREG,
mlx5_ib_stage_bfrag_init,
mlx5_ib_stage_bfrag_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_PRE_IB_REG_UMR,
+ NULL,
+ mlx5_ib_stage_pre_ib_reg_umr_cleanup),
STAGE_CREATE(MLX5_IB_STAGE_IB_REG,
mlx5_ib_stage_ib_reg_init,
mlx5_ib_stage_ib_reg_cleanup),
- STAGE_CREATE(MLX5_IB_STAGE_UMR_RESOURCES,
- mlx5_ib_stage_umr_res_init,
- mlx5_ib_stage_umr_res_cleanup),
+ STAGE_CREATE(MLX5_IB_STAGE_POST_IB_REG_UMR,
+ mlx5_ib_stage_post_ib_reg_umr_init,
+ NULL),
STAGE_CREATE(MLX5_IB_STAGE_CLASS_ATTR,
mlx5_ib_stage_class_attr_init,
NULL),
MLX5_IB_STAGE_CONG_DEBUGFS,
MLX5_IB_STAGE_UAR,
MLX5_IB_STAGE_BFREG,
+ MLX5_IB_STAGE_PRE_IB_REG_UMR,
MLX5_IB_STAGE_IB_REG,
- MLX5_IB_STAGE_UMR_RESOURCES,
+ MLX5_IB_STAGE_POST_IB_REG_UMR,
MLX5_IB_STAGE_DELAY_DROP,
MLX5_IB_STAGE_CLASS_ATTR,
MLX5_IB_STAGE_REP_REG,
void mlx5_ib_stage_counters_cleanup(struct mlx5_ib_dev *dev);
int mlx5_ib_stage_bfrag_init(struct mlx5_ib_dev *dev);
void mlx5_ib_stage_bfrag_cleanup(struct mlx5_ib_dev *dev);
+void mlx5_ib_stage_pre_ib_reg_umr_cleanup(struct mlx5_ib_dev *dev);
int mlx5_ib_stage_ib_reg_init(struct mlx5_ib_dev *dev);
void mlx5_ib_stage_ib_reg_cleanup(struct mlx5_ib_dev *dev);
-int mlx5_ib_stage_umr_res_init(struct mlx5_ib_dev *dev);
-void mlx5_ib_stage_umr_res_cleanup(struct mlx5_ib_dev *dev);
+int mlx5_ib_stage_post_ib_reg_umr_init(struct mlx5_ib_dev *dev);
int mlx5_ib_stage_class_attr_init(struct mlx5_ib_dev *dev);
void __mlx5_ib_remove(struct mlx5_ib_dev *dev,
const struct mlx5_ib_profile *profile,
*umem = ib_umem_get(pd->uobject->context, start, length,
access_flags, 0);
err = PTR_ERR_OR_ZERO(*umem);
- if (err < 0) {
+ if (err) {
+ *umem = NULL;
mlx5_ib_err(dev, "umem get failed (%d)\n", err);
return err;
}
if (err) {
mlx5_ib_warn(dev, "Failed to rereg UMR\n");
ib_umem_release(mr->umem);
+ mr->umem = NULL;
clean_mr(dev, mr);
return err;
}
u32 key = mr->mmkey.key;
err = destroy_mkey(dev, mr);
- kfree(mr);
if (err) {
mlx5_ib_warn(dev, "failed to destroy mkey 0x%x (%d)\n",
key, err);
return err;
}
- } else {
- mlx5_mr_cache_free(dev, mr);
}
return 0;
atomic_sub(npages, &dev->mdev->priv.reg_pages);
}
+ if (!mr->allocated_from_cache)
+ kfree(mr);
+ else
+ mlx5_mr_cache_free(dev, mr);
+
return 0;
}
mr->ibmr.iova = sg_dma_address(sg) + sg_offset;
mr->ibmr.length = 0;
- mr->ndescs = sg_nents;
for_each_sg(sgl, sg, sg_nents, i) {
if (unlikely(i >= mr->max_descs))
sg_offset = 0;
}
+ mr->ndescs = i;
if (sg_offset_p)
*sg_offset_p = sg_offset;
ib_umem_release(sq->ubuffer.umem);
}
-static int get_rq_pas_size(void *qpc)
+static size_t get_rq_pas_size(void *qpc)
{
u32 log_page_size = MLX5_GET(qpc, qpc, log_page_size) + 12;
u32 log_rq_stride = MLX5_GET(qpc, qpc, log_rq_stride);
}
static int create_raw_packet_qp_rq(struct mlx5_ib_dev *dev,
- struct mlx5_ib_rq *rq, void *qpin)
+ struct mlx5_ib_rq *rq, void *qpin,
+ size_t qpinlen)
{
struct mlx5_ib_qp *mqp = rq->base.container_mibqp;
__be64 *pas;
void *rqc;
void *wq;
void *qpc = MLX5_ADDR_OF(create_qp_in, qpin, qpc);
- int inlen;
+ size_t rq_pas_size = get_rq_pas_size(qpc);
+ size_t inlen;
int err;
- u32 rq_pas_size = get_rq_pas_size(qpc);
+
+ if (qpinlen < rq_pas_size + MLX5_BYTE_OFF(create_qp_in, pas))
+ return -EINVAL;
inlen = MLX5_ST_SZ_BYTES(create_rq_in) + rq_pas_size;
in = kvzalloc(inlen, GFP_KERNEL);
}
static int create_raw_packet_qp(struct mlx5_ib_dev *dev, struct mlx5_ib_qp *qp,
- u32 *in,
+ u32 *in, size_t inlen,
struct ib_pd *pd)
{
struct mlx5_ib_raw_packet_qp *raw_packet_qp = &qp->raw_packet_qp;
rq->flags |= MLX5_IB_RQ_CVLAN_STRIPPING;
if (qp->flags & MLX5_IB_QP_PCI_WRITE_END_PADDING)
rq->flags |= MLX5_IB_RQ_PCI_WRITE_END_PADDING;
- err = create_raw_packet_qp_rq(dev, rq, in);
+ err = create_raw_packet_qp_rq(dev, rq, in, inlen);
if (err)
goto err_destroy_sq;
u32 uidx = MLX5_IB_DEFAULT_UIDX;
struct mlx5_ib_create_qp ucmd;
struct mlx5_ib_qp_base *base;
+ int mlx5_st;
void *qpc;
u32 *in;
int err;
spin_lock_init(&qp->sq.lock);
spin_lock_init(&qp->rq.lock);
+ mlx5_st = to_mlx5_st(init_attr->qp_type);
+ if (mlx5_st < 0)
+ return -EINVAL;
+
if (init_attr->rwq_ind_tbl) {
if (!udata)
return -ENOSYS;
qpc = MLX5_ADDR_OF(create_qp_in, in, qpc);
- MLX5_SET(qpc, qpc, st, to_mlx5_st(init_attr->qp_type));
+ MLX5_SET(qpc, qpc, st, mlx5_st);
MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED);
if (init_attr->qp_type != MLX5_IB_QPT_REG_UMR)
}
}
+ if (inlen < 0) {
+ err = -EINVAL;
+ goto err;
+ }
+
if (init_attr->qp_type == IB_QPT_RAW_PACKET ||
qp->flags & MLX5_IB_QP_UNDERLAY) {
qp->raw_packet_qp.sq.ubuffer.buf_addr = ucmd.sq_buf_addr;
raw_packet_qp_copy_info(qp, &qp->raw_packet_qp);
- err = create_raw_packet_qp(dev, qp, in, pd);
+ err = create_raw_packet_qp(dev, qp, in, inlen, pd);
} else {
err = mlx5_core_create_qp(dev->mdev, &base->mqp, in, inlen);
}
goto out;
if (mlx5_cur >= MLX5_QP_NUM_STATE || mlx5_new >= MLX5_QP_NUM_STATE ||
- !optab[mlx5_cur][mlx5_new])
+ !optab[mlx5_cur][mlx5_new]) {
+ err = -EINVAL;
goto out;
+ }
op = optab[mlx5_cur][mlx5_new];
optpar = ib_mask_to_mlx5_opt(attr_mask);
struct mlx5_ib_sq *sq,
u8 *sq_state)
{
- void *out;
- void *sqc;
- int inlen;
int err;
- inlen = MLX5_ST_SZ_BYTES(query_sq_out);
- out = kvzalloc(inlen, GFP_KERNEL);
- if (!out)
- return -ENOMEM;
-
- err = mlx5_core_query_sq(dev->mdev, sq->base.mqp.qpn, out);
+ err = mlx5_core_query_sq_state(dev->mdev, sq->base.mqp.qpn, sq_state);
if (err)
goto out;
-
- sqc = MLX5_ADDR_OF(query_sq_out, out, sq_context);
- *sq_state = MLX5_GET(sqc, sqc, state);
sq->state = *sq_state;
out:
- kvfree(out);
return err;
}
{
struct mlx5_ib_dev *dev = to_mdev(pd->device);
struct mlx5_ib_srq *srq;
- int desc_size;
- int buf_size;
+ size_t desc_size;
+ size_t buf_size;
int err;
struct mlx5_srq_attr in = {0};
__u32 max_srq_wqes = 1 << MLX5_CAP_GEN(dev->mdev, log_max_srq_sz);
desc_size = sizeof(struct mlx5_wqe_srq_next_seg) +
srq->msrq.max_gs * sizeof(struct mlx5_wqe_data_seg);
+ if (desc_size == 0 || srq->msrq.max_gs > desc_size)
+ return ERR_PTR(-EINVAL);
desc_size = roundup_pow_of_two(desc_size);
- desc_size = max_t(int, 32, desc_size);
+ desc_size = max_t(size_t, 32, desc_size);
+ if (desc_size < sizeof(struct mlx5_wqe_srq_next_seg))
+ return ERR_PTR(-EINVAL);
srq->msrq.max_avail_gather = (desc_size - sizeof(struct mlx5_wqe_srq_next_seg)) /
sizeof(struct mlx5_wqe_data_seg);
srq->msrq.wqe_shift = ilog2(desc_size);
buf_size = srq->msrq.max * desc_size;
- mlx5_ib_dbg(dev, "desc_size 0x%x, req wr 0x%x, srq size 0x%x, max_gs 0x%x, max_avail_gather 0x%x\n",
- desc_size, init_attr->attr.max_wr, srq->msrq.max, srq->msrq.max_gs,
- srq->msrq.max_avail_gather);
+ if (buf_size < desc_size)
+ return ERR_PTR(-EINVAL);
in.type = init_attr->srq_type;
if (pd->uobject)
dev_hold(qdev->ndev);
/* The HW vendor's device driver must guarantee
- * that this function returns NULL before the net device reaches
- * NETDEV_UNREGISTER_FINAL state.
+ * that this function returns NULL before the net device has finished
+ * NETDEV_UNREGISTER state.
*/
return qdev->ndev;
}
__le32 imm_data_or_inv_r_Key;
__le32 length;
__le32 imm_data_hi;
- __le16 rq_cons;
+ __le16 rq_cons_or_srq_id;
u8 flags;
#define RDMA_CQE_RESPONDER_TOGGLE_BIT_MASK 0x1
#define RDMA_CQE_RESPONDER_TOGGLE_BIT_SHIFT 0
RDMA_CQE_REQ_STS_RNR_NAK_RETRY_CNT_ERR,
RDMA_CQE_REQ_STS_TRANSPORT_RETRY_CNT_ERR,
RDMA_CQE_REQ_STS_WORK_REQUEST_FLUSHED_ERR,
+ RDMA_CQE_REQ_STS_XRC_VOILATION_ERR,
MAX_RDMA_CQE_REQUESTER_STATUS_ENUM
};
RDMA_CQE_TYPE_REQUESTER,
RDMA_CQE_TYPE_RESPONDER_RQ,
RDMA_CQE_TYPE_RESPONDER_SRQ,
+ RDMA_CQE_TYPE_RESPONDER_XRC_SRQ,
RDMA_CQE_TYPE_INVALID,
MAX_RDMA_CQE_TYPE
};
}
return -EINVAL;
}
- neigh = dst_neigh_lookup(dst, &dst_in);
-
+ neigh = dst_neigh_lookup(dst, &fl6.daddr);
if (neigh) {
rcu_read_lock();
if (neigh->nud_state & NUD_VALID) {
qp = idr_find(&dev->qpidr, conn_param->qpn);
- laddr = (struct sockaddr_in *)&cm_id->local_addr;
- raddr = (struct sockaddr_in *)&cm_id->remote_addr;
- laddr6 = (struct sockaddr_in6 *)&cm_id->local_addr;
- raddr6 = (struct sockaddr_in6 *)&cm_id->remote_addr;
+ laddr = (struct sockaddr_in *)&cm_id->m_local_addr;
+ raddr = (struct sockaddr_in *)&cm_id->m_remote_addr;
+ laddr6 = (struct sockaddr_in6 *)&cm_id->m_local_addr;
+ raddr6 = (struct sockaddr_in6 *)&cm_id->m_remote_addr;
+
+ DP_DEBUG(dev, QEDR_MSG_IWARP, "MAPPED %d %d\n",
+ ntohs(((struct sockaddr_in *)&cm_id->remote_addr)->sin_port),
+ ntohs(raddr->sin_port));
DP_DEBUG(dev, QEDR_MSG_IWARP,
"Connect source address: %pISpc, remote address: %pISpc\n",
int rc;
int i;
- laddr = (struct sockaddr_in *)&cm_id->local_addr;
- laddr6 = (struct sockaddr_in6 *)&cm_id->local_addr;
+ laddr = (struct sockaddr_in *)&cm_id->m_local_addr;
+ laddr6 = (struct sockaddr_in6 *)&cm_id->m_local_addr;
DP_DEBUG(dev, QEDR_MSG_IWARP,
"Create Listener address: %pISpc\n", &cm_id->local_addr);
switch (wr->opcode) {
case IB_WR_SEND_WITH_IMM:
+ if (unlikely(rdma_protocol_iwarp(&dev->ibdev, 1))) {
+ rc = -EINVAL;
+ *bad_wr = wr;
+ break;
+ }
wqe->req_type = RDMA_SQ_REQ_TYPE_SEND_WITH_IMM;
swqe = (struct rdma_sq_send_wqe_1st *)wqe;
swqe->wqe_size = 2;
break;
case IB_WR_RDMA_WRITE_WITH_IMM:
+ if (unlikely(rdma_protocol_iwarp(&dev->ibdev, 1))) {
+ rc = -EINVAL;
+ *bad_wr = wr;
+ break;
+ }
wqe->req_type = RDMA_SQ_REQ_TYPE_RDMA_WR_WITH_IMM;
rwqe = (struct rdma_sq_rdma_wqe_1st *)wqe;
static void try_consume_resp_cqe(struct qedr_cq *cq, struct qedr_qp *qp,
struct rdma_cqe_responder *resp, int *update)
{
- if (le16_to_cpu(resp->rq_cons) == qp->rq.wqe_cons) {
+ if (le16_to_cpu(resp->rq_cons_or_srq_id) == qp->rq.wqe_cons) {
consume_cqe(cq);
*update |= 1;
}
if (resp->status == RDMA_CQE_RESP_STS_WORK_REQUEST_FLUSHED_ERR) {
cnt = process_resp_flush(qp, cq, num_entries, wc,
- resp->rq_cons);
+ resp->rq_cons_or_srq_id);
try_consume_resp_cqe(cq, qp, resp, update);
} else {
cnt = process_resp_one(dev, qp, cq, wc, resp);
{
struct qedr_dev *dev = get_qedr_dev(ibcq->device);
struct qedr_cq *cq = get_qedr_cq(ibcq);
- union rdma_cqe *cqe = cq->latest_cqe;
+ union rdma_cqe *cqe;
u32 old_cons, new_cons;
unsigned long flags;
int update = 0;
return qedr_gsi_poll_cq(ibcq, num_entries, wc);
spin_lock_irqsave(&cq->cq_lock, flags);
+ cqe = cq->latest_cqe;
old_cons = qed_chain_get_cons_idx_u32(&cq->pbl);
while (num_entries && is_valid_cqe(cq, cqe)) {
struct qedr_qp *qp;
}
/* Start of netdev section */
-static inline const char *usnic_ib_netdev_event_to_string(unsigned long event)
-{
- const char *event2str[] = {"NETDEV_NONE", "NETDEV_UP", "NETDEV_DOWN",
- "NETDEV_REBOOT", "NETDEV_CHANGE",
- "NETDEV_REGISTER", "NETDEV_UNREGISTER", "NETDEV_CHANGEMTU",
- "NETDEV_CHANGEADDR", "NETDEV_GOING_DOWN", "NETDEV_FEAT_CHANGE",
- "NETDEV_BONDING_FAILOVER", "NETDEV_PRE_UP",
- "NETDEV_PRE_TYPE_CHANGE", "NETDEV_POST_TYPE_CHANGE",
- "NETDEV_POST_INT", "NETDEV_UNREGISTER_FINAL", "NETDEV_RELEASE",
- "NETDEV_NOTIFY_PEERS", "NETDEV_JOIN"
- };
-
- if (event >= ARRAY_SIZE(event2str))
- return "UNKNOWN_NETDEV_EVENT";
- else
- return event2str[event];
-}
-
static void usnic_ib_qp_grp_modify_active_to_err(struct usnic_ib_dev *us_ibdev)
{
struct usnic_ib_ucontext *ctx;
ib_dispatch_event(&ib_event);
} else {
usnic_dbg("Ignoring %s on %s\n",
- usnic_ib_netdev_event_to_string(event),
+ netdev_cmd_to_name(event),
us_ibdev->ib_dev.name);
}
break;
break;
default:
usnic_dbg("Ignoring event %s on %s",
- usnic_ib_netdev_event_to_string(event),
+ netdev_cmd_to_name(event),
us_ibdev->ib_dev.name);
}
mutex_unlock(&us_ibdev->usdev_lock);
switch (event) {
case NETDEV_DOWN:
usnic_info("%s via ip notifiers",
- usnic_ib_netdev_event_to_string(event));
+ netdev_cmd_to_name(event));
usnic_fwd_del_ipaddr(us_ibdev->ufdev);
usnic_ib_qp_grp_modify_active_to_err(us_ibdev);
ib_event.event = IB_EVENT_GID_CHANGE;
case NETDEV_UP:
usnic_fwd_add_ipaddr(us_ibdev->ufdev, ifa->ifa_address);
usnic_info("%s via ip notifiers: ip %pI4",
- usnic_ib_netdev_event_to_string(event),
+ netdev_cmd_to_name(event),
&us_ibdev->ufdev->inaddr);
ib_event.event = IB_EVENT_GID_CHANGE;
ib_event.device = &us_ibdev->ib_dev;
break;
default:
usnic_info("Ignoring event %s on %s",
- usnic_ib_netdev_event_to_string(event),
+ netdev_cmd_to_name(event),
us_ibdev->ib_dev.name);
}
mutex_unlock(&us_ibdev->usdev_lock);
unsigned long timeout;
struct rvt_dev_info *rdi = ib_to_rvt(mr->pd->device);
- if (percpu_ref_is_zero(&mr->refcount))
- return 0;
- /* avoid dma mr */
- if (mr->lkey)
+ if (mr->lkey) {
+ /* avoid dma mr */
rvt_dereg_clean_qps(mr);
+ /* @mr was indexed on rcu protected @lkey_table */
+ synchronize_rcu();
+ }
+
timeout = wait_for_completion_timeout(&mr->comp, 5 * HZ);
if (!timeout) {
rvt_pr_err(rdi,
{
struct matrix_keypad *keypad = input_get_drvdata(dev);
+ spin_lock_irq(&keypad->lock);
keypad->stopped = true;
- mb();
+ spin_unlock_irq(&keypad->lock);
+
flush_work(&keypad->work.work);
/*
* matrix_keypad_scan() will leave IRQs enabled;
"LEN0046", /* X250 */
"LEN004a", /* W541 */
"LEN200f", /* T450s */
- "LEN2018", /* T460p */
NULL
};
-/*
- * Copyright (C) 2012 Samsung Electronics Co.Ltd
- * Author: Joonyoung Shim <jy0922.shim@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
+// SPDX-License-Identifier: GPL-2.0
+// Melfas MMS114/MMS152 touchscreen device driver
+//
+// Copyright (c) 2012 Samsung Electronics Co., Ltd.
+// Author: Joonyoung Shim <jy0922.shim@samsung.com>
#include <linux/module.h>
#include <linux/delay.h>
/* Module information */
MODULE_AUTHOR("Joonyoung Shim <jy0922.shim@samsung.com>");
MODULE_DESCRIPTION("MELFAS mms114 Touchscreen driver");
-MODULE_LICENSE("GPL");
+MODULE_LICENSE("GPL v2");
* This gives us (((1UL << id_bits) - 8192) >> 5) possible allocations.
*/
#define IRQS_PER_CHUNK_SHIFT 5
-#define IRQS_PER_CHUNK (1 << IRQS_PER_CHUNK_SHIFT)
+#define IRQS_PER_CHUNK (1UL << IRQS_PER_CHUNK_SHIFT)
#define ITS_MAX_LPI_NRBITS 16 /* 64K LPIs */
static unsigned long *lpi_bitmap;
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
/*
- * At least one bit of EventID is being used, hence a minimum
- * of two entries. No, the architecture doesn't let you
- * express an ITT with a single entry.
+ * We allocate at least one chunk worth of LPIs bet device,
+ * and thus that many ITEs. The device may require less though.
*/
- nr_ites = max(2UL, roundup_pow_of_two(nvecs));
+ nr_ites = max(IRQS_PER_CHUNK, roundup_pow_of_two(nvecs));
sz = nr_ites * its->ite_size;
sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
itt = kzalloc(sz, GFP_KERNEL);
static void its_vpe_schedule(struct its_vpe *vpe)
{
- void * __iomem vlpi_base = gic_data_rdist_vlpi_base();
+ void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
u64 val;
/* Schedule the VPE */
static void its_vpe_deschedule(struct its_vpe *vpe)
{
- void * __iomem vlpi_base = gic_data_rdist_vlpi_base();
+ void __iomem *vlpi_base = gic_data_rdist_vlpi_base();
u32 count = 1000000; /* 1s! */
bool clean;
u64 val;
static struct gpcv2_irqchip_data *imx_gpcv2_instance;
-/*
- * Interface for the low level wakeup code.
- */
-u32 imx_gpcv2_get_wakeup_source(u32 **sources)
-{
- if (!imx_gpcv2_instance)
- return 0;
-
- if (sources)
- *sources = imx_gpcv2_instance->wakeup_sources;
-
- return IMR_NUM;
-}
-
static int gpcv2_wakeup_source_save(void)
{
struct gpcv2_irqchip_data *cd;
uint32_t rtime = cpu_to_le32(get_seconds());
struct uuid_entry *u;
char buf[BDEVNAME_SIZE];
+ struct cached_dev *exist_dc, *t;
bdevname(dc->bdev, buf);
return -EINVAL;
}
+ /* Check whether already attached */
+ list_for_each_entry_safe(exist_dc, t, &c->cached_devs, list) {
+ if (!memcmp(dc->sb.uuid, exist_dc->sb.uuid, 16)) {
+ pr_err("Tried to attach %s but duplicate UUID already attached",
+ buf);
+
+ return -EINVAL;
+ }
+ }
+
u = uuid_find(c, dc->sb.uuid);
if (u &&
return;
err:
- pr_notice("error opening %s: %s", bdevname(bdev, name), err);
+ pr_notice("error %s: %s", bdevname(bdev, name), err);
bcache_device_stop(&dc->disk);
}
const char *err = NULL; /* must be set for any error case */
int ret = 0;
+ bdevname(bdev, name);
+
memcpy(&ca->sb, sb, sizeof(struct cache_sb));
ca->bdev = bdev;
ca->bdev->bd_holder = ca;
bio_first_bvec_all(&ca->sb_bio)->bv_page = sb_page;
get_page(sb_page);
- if (blk_queue_discard(bdev_get_queue(ca->bdev)))
+ if (blk_queue_discard(bdev_get_queue(bdev)))
ca->discard = CACHE_DISCARD(&ca->sb);
ret = cache_alloc(ca);
if (ret != 0) {
+ blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
if (ret == -ENOMEM)
err = "cache_alloc(): -ENOMEM";
else
goto out;
}
- pr_info("registered cache device %s", bdevname(bdev, name));
+ pr_info("registered cache device %s", name);
out:
kobject_put(&ca->kobj);
err:
if (err)
- pr_notice("error opening %s: %s", bdevname(bdev, name), err);
+ pr_notice("error %s: %s", name, err);
return ret;
}
if (err)
goto err_close;
+ err = "failed to register device";
if (SB_IS_BDEV(sb)) {
struct cached_dev *dc = kzalloc(sizeof(*dc), GFP_KERNEL);
if (!dc)
goto err_close;
if (register_cache(sb, sb_page, bdev, ca) != 0)
- goto err_close;
+ goto err;
}
out:
if (sb_page)
err_close:
blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
err:
- pr_info("error opening %s: %s", path, err);
+ pr_info("error %s: %s", path, err);
ret = -EINVAL;
goto out;
}
static void *alloc_buffer_data(struct dm_bufio_client *c, gfp_t gfp_mask,
enum data_mode *data_mode)
{
- unsigned noio_flag;
- void *ptr;
-
if (c->block_size <= DM_BUFIO_BLOCK_SIZE_SLAB_LIMIT) {
*data_mode = DATA_MODE_SLAB;
return kmem_cache_alloc(DM_BUFIO_CACHE(c), gfp_mask);
* all allocations done by this process (including pagetables) are done
* as if GFP_NOIO was specified.
*/
+ if (gfp_mask & __GFP_NORETRY) {
+ unsigned noio_flag = memalloc_noio_save();
+ void *ptr = __vmalloc(c->block_size, gfp_mask, PAGE_KERNEL);
- if (gfp_mask & __GFP_NORETRY)
- noio_flag = memalloc_noio_save();
-
- ptr = __vmalloc(c->block_size, gfp_mask, PAGE_KERNEL);
-
- if (gfp_mask & __GFP_NORETRY)
memalloc_noio_restore(noio_flag);
+ return ptr;
+ }
- return ptr;
+ return __vmalloc(c->block_size, gfp_mask, PAGE_KERNEL);
}
/*
else
m->queue_mode = DM_TYPE_REQUEST_BASED;
- } else if (m->queue_mode == DM_TYPE_BIO_BASED ||
- m->queue_mode == DM_TYPE_NVME_BIO_BASED) {
+ } else if (m->queue_mode == DM_TYPE_BIO_BASED) {
INIT_WORK(&m->process_queued_bios, process_queued_bios);
-
- if (m->queue_mode == DM_TYPE_BIO_BASED) {
- /*
- * bio-based doesn't support any direct scsi_dh management;
- * it just discovers if a scsi_dh is attached.
- */
- set_bit(MPATHF_RETAIN_ATTACHED_HW_HANDLER, &m->flags);
- }
- }
-
- if (m->queue_mode != DM_TYPE_NVME_BIO_BASED) {
- set_bit(MPATHF_QUEUE_IO, &m->flags);
- atomic_set(&m->pg_init_in_progress, 0);
- atomic_set(&m->pg_init_count, 0);
- m->pg_init_delay_msecs = DM_PG_INIT_DELAY_DEFAULT;
- init_waitqueue_head(&m->pg_init_wait);
+ /*
+ * bio-based doesn't support any direct scsi_dh management;
+ * it just discovers if a scsi_dh is attached.
+ */
+ set_bit(MPATHF_RETAIN_ATTACHED_HW_HANDLER, &m->flags);
}
dm_table_set_type(ti->table, m->queue_mode);
+ /*
+ * Init fields that are only used when a scsi_dh is attached
+ * - must do this unconditionally (really doesn't hurt non-SCSI uses)
+ */
+ set_bit(MPATHF_QUEUE_IO, &m->flags);
+ atomic_set(&m->pg_init_in_progress, 0);
+ atomic_set(&m->pg_init_count, 0);
+ m->pg_init_delay_msecs = DM_PG_INIT_DELAY_DEFAULT;
+ init_waitqueue_head(&m->pg_init_wait);
+
return 0;
}
{
m->current_pg = pg;
- if (m->queue_mode == DM_TYPE_NVME_BIO_BASED)
- return;
-
/* Must we initialise the PG first, and queue I/O till it's ready? */
if (m->hw_handler_name) {
set_bit(MPATHF_PG_INIT_REQUIRED, &m->flags);
unsigned bypassed = 1;
if (!atomic_read(&m->nr_valid_paths)) {
- if (m->queue_mode != DM_TYPE_NVME_BIO_BASED)
- clear_bit(MPATHF_QUEUE_IO, &m->flags);
+ clear_bit(MPATHF_QUEUE_IO, &m->flags);
goto failed;
}
return pgpath;
}
-static struct pgpath *__map_bio_nvme(struct multipath *m, struct bio *bio)
+static struct pgpath *__map_bio_fast(struct multipath *m, struct bio *bio)
{
struct pgpath *pgpath;
unsigned long flags;
{
struct pgpath *pgpath;
- if (m->queue_mode == DM_TYPE_NVME_BIO_BASED)
- pgpath = __map_bio_nvme(m, bio);
+ if (!m->hw_handler_name)
+ pgpath = __map_bio_fast(m, bio);
else
pgpath = __map_bio(m, bio);
{
if (m->queue_mode == DM_TYPE_MQ_REQUEST_BASED)
dm_mq_kick_requeue_list(dm_table_get_md(m->ti->table));
- else if (m->queue_mode == DM_TYPE_BIO_BASED ||
- m->queue_mode == DM_TYPE_NVME_BIO_BASED)
+ else if (m->queue_mode == DM_TYPE_BIO_BASED)
queue_work(kmultipathd, &m->process_queued_bios);
}
return 0;
}
-static int setup_scsi_dh(struct block_device *bdev, struct multipath *m, char **error)
+static int setup_scsi_dh(struct block_device *bdev, struct multipath *m,
+ const char *attached_handler_name, char **error)
{
struct request_queue *q = bdev_get_queue(bdev);
- const char *attached_handler_name;
int r;
if (test_bit(MPATHF_RETAIN_ATTACHED_HW_HANDLER, &m->flags)) {
retain:
- attached_handler_name = scsi_dh_attached_handler_name(q, GFP_KERNEL);
if (attached_handler_name) {
/*
* Clear any hw_handler_params associated with a
int r;
struct pgpath *p;
struct multipath *m = ti->private;
+ struct request_queue *q;
+ const char *attached_handler_name;
/* we need at least a path arg */
if (as->argc < 1) {
goto bad;
}
- if (m->queue_mode != DM_TYPE_NVME_BIO_BASED) {
+ q = bdev_get_queue(p->path.dev->bdev);
+ attached_handler_name = scsi_dh_attached_handler_name(q, GFP_KERNEL);
+ if (attached_handler_name) {
INIT_DELAYED_WORK(&p->activate_path, activate_path_work);
- r = setup_scsi_dh(p->path.dev->bdev, m, &ti->error);
+ r = setup_scsi_dh(p->path.dev->bdev, m, attached_handler_name, &ti->error);
if (r) {
dm_put_device(ti, p->path.dev);
goto bad;
if (!hw_argc)
return 0;
- if (m->queue_mode == DM_TYPE_BIO_BASED ||
- m->queue_mode == DM_TYPE_NVME_BIO_BASED) {
+ if (m->queue_mode == DM_TYPE_BIO_BASED) {
dm_consume_args(as, hw_argc);
DMERR("bio-based multipath doesn't allow hardware handler args");
return 0;
if (!strcasecmp(queue_mode_name, "bio"))
m->queue_mode = DM_TYPE_BIO_BASED;
- else if (!strcasecmp(queue_mode_name, "nvme"))
- m->queue_mode = DM_TYPE_NVME_BIO_BASED;
else if (!strcasecmp(queue_mode_name, "rq"))
m->queue_mode = DM_TYPE_REQUEST_BASED;
else if (!strcasecmp(queue_mode_name, "mq"))
ti->num_discard_bios = 1;
ti->num_write_same_bios = 1;
ti->num_write_zeroes_bios = 1;
- if (m->queue_mode == DM_TYPE_BIO_BASED || m->queue_mode == DM_TYPE_NVME_BIO_BASED)
+ if (m->queue_mode == DM_TYPE_BIO_BASED)
ti->per_io_data_size = multipath_per_bio_data_size();
else
ti->per_io_data_size = sizeof(struct dm_mpath_io);
case DM_TYPE_BIO_BASED:
DMEMIT("queue_mode bio ");
break;
- case DM_TYPE_NVME_BIO_BASED:
- DMEMIT("queue_mode nvme ");
- break;
case DM_TYPE_MQ_REQUEST_BASED:
DMEMIT("queue_mode mq ");
break;
*---------------------------------------------------------------*/
static struct target_type multipath_target = {
.name = "multipath",
- .version = {1, 12, 0},
- .features = DM_TARGET_SINGLETON | DM_TARGET_IMMUTABLE,
+ .version = {1, 13, 0},
+ .features = DM_TARGET_SINGLETON | DM_TARGET_IMMUTABLE |
+ DM_TARGET_PASSES_INTEGRITY,
.module = THIS_MODULE,
.ctr = multipath_ctr,
.dtr = multipath_dtr,
set_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags);
} else {
- if (test_bit(MD_RECOVERY_NEEDED, &recovery) ||
- test_bit(MD_RECOVERY_RESHAPE, &recovery) ||
- test_bit(MD_RECOVERY_RUNNING, &recovery))
+ if (!test_bit(MD_RECOVERY_INTR, &recovery) &&
+ (test_bit(MD_RECOVERY_NEEDED, &recovery) ||
+ test_bit(MD_RECOVERY_RESHAPE, &recovery) ||
+ test_bit(MD_RECOVERY_RUNNING, &recovery)))
r = mddev->curr_resync_completed;
else
r = mddev->recovery_cp;
if (t->type != DM_TYPE_NONE) {
/* target already set the table's type */
- if (t->type == DM_TYPE_BIO_BASED)
- return 0;
- else if (t->type == DM_TYPE_NVME_BIO_BASED) {
- if (!dm_table_does_not_support_partial_completion(t)) {
- DMERR("nvme bio-based is only possible with devices"
- " that don't support partial completion");
- return -EINVAL;
- }
- /* Fallthru, also verify all devices are blk-mq */
+ if (t->type == DM_TYPE_BIO_BASED) {
+ /* possibly upgrade to a variant of bio-based */
+ goto verify_bio_based;
}
BUG_ON(t->type == DM_TYPE_DAX_BIO_BASED);
+ BUG_ON(t->type == DM_TYPE_NVME_BIO_BASED);
goto verify_rq_based;
}
}
if (bio_based) {
+verify_bio_based:
/* We must use this table as bio-based */
t->type = DM_TYPE_BIO_BASED;
if (dm_table_supports_dax(t) ||
char b[BDEVNAME_SIZE];
/* For now, NVMe devices are the only devices of this class */
- return (strncmp(bdevname(dev->bdev, b), "nvme", 3) == 0);
+ return (strncmp(bdevname(dev->bdev, b), "nvme", 4) == 0);
}
static bool dm_table_does_not_support_partial_completion(struct dm_table *t)
return dm_get_geometry(md, geo);
}
-static int dm_grab_bdev_for_ioctl(struct mapped_device *md,
- struct block_device **bdev,
- fmode_t *mode)
+static char *_dm_claim_ptr = "I belong to device-mapper";
+
+static int dm_get_bdev_for_ioctl(struct mapped_device *md,
+ struct block_device **bdev,
+ fmode_t *mode)
{
struct dm_target *tgt;
struct dm_table *map;
goto out;
bdgrab(*bdev);
+ r = blkdev_get(*bdev, *mode, _dm_claim_ptr);
+ if (r < 0)
+ goto out;
+
dm_put_live_table(md, srcu_idx);
return r;
struct mapped_device *md = bdev->bd_disk->private_data;
int r;
- r = dm_grab_bdev_for_ioctl(md, &bdev, &mode);
+ r = dm_get_bdev_for_ioctl(md, &bdev, &mode);
if (r < 0)
return r;
r = __blkdev_driver_ioctl(bdev, mode, cmd, arg);
out:
- bdput(bdev);
+ blkdev_put(bdev, mode);
return r;
}
static int open_table_device(struct table_device *td, dev_t dev,
struct mapped_device *md)
{
- static char *_claim_ptr = "I belong to device-mapper";
struct block_device *bdev;
int r;
BUG_ON(td->dm_dev.bdev);
- bdev = blkdev_get_by_dev(dev, td->dm_dev.mode | FMODE_EXCL, _claim_ptr);
+ bdev = blkdev_get_by_dev(dev, td->dm_dev.mode | FMODE_EXCL, _dm_claim_ptr);
if (IS_ERR(bdev))
return PTR_ERR(bdev);
fmode_t mode;
int r;
- r = dm_grab_bdev_for_ioctl(md, &bdev, &mode);
+ r = dm_get_bdev_for_ioctl(md, &bdev, &mode);
if (r < 0)
return r;
else
r = -EOPNOTSUPP;
- bdput(bdev);
+ blkdev_put(bdev, mode);
return r;
}
fmode_t mode;
int r;
- r = dm_grab_bdev_for_ioctl(md, &bdev, &mode);
+ r = dm_get_bdev_for_ioctl(md, &bdev, &mode);
if (r < 0)
return r;
else
r = -EOPNOTSUPP;
- bdput(bdev);
+ blkdev_put(bdev, mode);
return r;
}
fmode_t mode;
int r;
- r = dm_grab_bdev_for_ioctl(md, &bdev, &mode);
+ r = dm_get_bdev_for_ioctl(md, &bdev, &mode);
if (r < 0)
return r;
else
r = -EOPNOTSUPP;
- bdput(bdev);
+ blkdev_put(bdev, mode);
return r;
}
fmode_t mode;
int r;
- r = dm_grab_bdev_for_ioctl(md, &bdev, &mode);
+ r = dm_get_bdev_for_ioctl(md, &bdev, &mode);
if (r < 0)
return r;
else
r = -EOPNOTSUPP;
- bdput(bdev);
+ blkdev_put(bdev, mode);
return r;
}
return rc;
}
+static long afu_ioctl_get_metadata(struct ocxl_context *ctx,
+ struct ocxl_ioctl_metadata __user *uarg)
+{
+ struct ocxl_ioctl_metadata arg;
+
+ memset(&arg, 0, sizeof(arg));
+
+ arg.version = 0;
+
+ arg.afu_version_major = ctx->afu->config.version_major;
+ arg.afu_version_minor = ctx->afu->config.version_minor;
+ arg.pasid = ctx->pasid;
+ arg.pp_mmio_size = ctx->afu->config.pp_mmio_stride;
+ arg.global_mmio_size = ctx->afu->config.global_mmio_size;
+
+ if (copy_to_user(uarg, &arg, sizeof(arg)))
+ return -EFAULT;
+
+ return 0;
+}
+
#define CMD_STR(x) (x == OCXL_IOCTL_ATTACH ? "ATTACH" : \
x == OCXL_IOCTL_IRQ_ALLOC ? "IRQ_ALLOC" : \
x == OCXL_IOCTL_IRQ_FREE ? "IRQ_FREE" : \
x == OCXL_IOCTL_IRQ_SET_FD ? "IRQ_SET_FD" : \
+ x == OCXL_IOCTL_GET_METADATA ? "GET_METADATA" : \
"UNKNOWN")
static long afu_ioctl(struct file *file, unsigned int cmd,
irq_fd.eventfd);
break;
+ case OCXL_IOCTL_GET_METADATA:
+ rc = afu_ioctl_get_metadata(ctx,
+ (struct ocxl_ioctl_metadata __user *) args);
+ break;
+
default:
rc = -EINVAL;
}
#define MMC_BLK_TIMEOUT_MS (10 * 1000)
#define MMC_SANITIZE_REQ_TIMEOUT 240000
#define MMC_EXTRACT_INDEX_FROM_ARG(x) ((x & 0x00FF0000) >> 16)
+#define MMC_EXTRACT_VALUE_FROM_ARG(x) ((x & 0x0000FF00) >> 8)
#define mmc_req_rel_wr(req) ((req->cmd_flags & REQ_FUA) && \
(rq_data_dir(req) == WRITE))
return data.error;
}
+ /*
+ * Make sure the cache of the PARTITION_CONFIG register and
+ * PARTITION_ACCESS bits is updated in case the ioctl ext_csd write
+ * changed it successfully.
+ */
+ if ((MMC_EXTRACT_INDEX_FROM_ARG(cmd.arg) == EXT_CSD_PART_CONFIG) &&
+ (cmd.opcode == MMC_SWITCH)) {
+ struct mmc_blk_data *main_md = dev_get_drvdata(&card->dev);
+ u8 value = MMC_EXTRACT_VALUE_FROM_ARG(cmd.arg);
+
+ /*
+ * Update cache so the next mmc_blk_part_switch call operates
+ * on up-to-date data.
+ */
+ card->ext_csd.part_config = value;
+ main_md->part_curr = value & EXT_CSD_PART_CONFIG_ACC_MASK;
+ }
+
/*
* According to the SD specs, some commands require a delay after
* issuing the command.
#define CID_MANFID_APACER 0x27
#define CID_MANFID_KINGSTON 0x70
#define CID_MANFID_HYNIX 0x90
+#define CID_MANFID_NUMONYX 0xFE
#define END_FIXUP { NULL }
*/
MMC_FIXUP_EXT_CSD_REV(CID_NAME_ANY, CID_MANFID_HYNIX,
0x014a, add_quirk, MMC_QUIRK_BROKEN_HPI, 5),
+ /*
+ * Certain Micron (Numonyx) eMMC 4.5 cards might get broken when HPI
+ * feature is used so disable the HPI feature for such buggy cards.
+ */
+ MMC_FIXUP_EXT_CSD_REV(CID_NAME_ANY, CID_MANFID_NUMONYX,
+ 0x014e, add_quirk, MMC_QUIRK_BROKEN_HPI, 6),
END_FIXUP
};
static int dw_mci_exynos_runtime_resume(struct device *dev)
{
struct dw_mci *host = dev_get_drvdata(dev);
+ int ret;
+
+ ret = dw_mci_runtime_resume(dev);
+ if (ret)
+ return ret;
dw_mci_exynos_config_smu(host);
- return dw_mci_runtime_resume(dev);
+
+ return ret;
}
/**
cto_div = (mci_readl(host, CLKDIV) & 0xff) * 2;
if (cto_div == 0)
cto_div = 1;
- cto_ms = DIV_ROUND_UP(MSEC_PER_SEC * cto_clks * cto_div, host->bus_hz);
+
+ cto_ms = DIV_ROUND_UP_ULL((u64)MSEC_PER_SEC * cto_clks * cto_div,
+ host->bus_hz);
/* add a bit spare time */
cto_ms += 10;
(sizeof(struct idmac_desc_64addr) *
(i + 1))) >> 32;
/* Initialize reserved and buffer size fields to "0" */
+ p->des0 = 0;
p->des1 = 0;
p->des2 = 0;
p->des3 = 0;
i++, p++) {
p->des3 = cpu_to_le32(host->sg_dma +
(sizeof(struct idmac_desc) * (i + 1)));
+ p->des0 = 0;
p->des1 = 0;
}
}
if (host->use_dma == TRANS_MODE_IDMAC)
- /* It is also recommended that we reset and reprogram idmac */
- dw_mci_idmac_reset(host);
+ /* It is also required that we reinit idmac */
+ dw_mci_idmac_init(host);
ret = true;
drto_div = (mci_readl(host, CLKDIV) & 0xff) * 2;
if (drto_div == 0)
drto_div = 1;
- drto_ms = DIV_ROUND_UP(MSEC_PER_SEC * drto_clks * drto_div,
- host->bus_hz);
+
+ drto_ms = DIV_ROUND_UP_ULL((u64)MSEC_PER_SEC * drto_clks * drto_div,
+ host->bus_hz);
/* add a bit spare time */
drto_ms += 10;
host->hw_name = "ACPI";
host->ops = &sdhci_acpi_ops_dflt;
host->irq = platform_get_irq(pdev, 0);
- if (host->irq <= 0) {
+ if (host->irq < 0) {
err = -EINVAL;
goto err_free;
}
config NETDEVSIM
tristate "Simulated networking device"
depends on DEBUG_FS
+ depends on MAY_USE_DEVLINK
help
This driver is a developer testing tool and software model that can
be used to test various control path networking APIs, especially
.exit = bond_net_exit,
.id = &bond_net_id,
.size = sizeof(struct bond_net),
- .async = true,
};
static int __init bonding_init(void)
if (bn->proc_dir) {
bond->proc_entry = proc_create_data(bond_dev->name,
- S_IRUGO, bn->proc_dir,
+ 0444, bn->proc_dir,
&bond_info_fops, bond);
if (bond->proc_entry == NULL)
netdev_warn(bond_dev, "Cannot create /proc/net/%s/%s\n",
static const struct class_attribute class_attr_bonding_masters = {
.attr = {
.name = "bonding_masters",
- .mode = S_IWUSR | S_IRUGO,
+ .mode = 0644,
},
.show = bonding_show_bonds,
.store = bonding_store_bonds,
return res;
}
-static DEVICE_ATTR(slaves, S_IRUGO | S_IWUSR, bonding_show_slaves,
+static DEVICE_ATTR(slaves, 0644, bonding_show_slaves,
bonding_sysfs_store_option);
/* Show the bonding mode. */
return sprintf(buf, "%s %d\n", val->string, BOND_MODE(bond));
}
-static DEVICE_ATTR(mode, S_IRUGO | S_IWUSR,
- bonding_show_mode, bonding_sysfs_store_option);
+static DEVICE_ATTR(mode, 0644, bonding_show_mode, bonding_sysfs_store_option);
/* Show the bonding transmit hash method. */
static ssize_t bonding_show_xmit_hash(struct device *d,
return sprintf(buf, "%s %d\n", val->string, bond->params.xmit_policy);
}
-static DEVICE_ATTR(xmit_hash_policy, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(xmit_hash_policy, 0644,
bonding_show_xmit_hash, bonding_sysfs_store_option);
/* Show arp_validate. */
return sprintf(buf, "%s %d\n", val->string, bond->params.arp_validate);
}
-static DEVICE_ATTR(arp_validate, S_IRUGO | S_IWUSR, bonding_show_arp_validate,
+static DEVICE_ATTR(arp_validate, 0644, bonding_show_arp_validate,
bonding_sysfs_store_option);
/* Show arp_all_targets. */
return sprintf(buf, "%s %d\n",
val->string, bond->params.arp_all_targets);
}
-static DEVICE_ATTR(arp_all_targets, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(arp_all_targets, 0644,
bonding_show_arp_all_targets, bonding_sysfs_store_option);
/* Show fail_over_mac. */
return sprintf(buf, "%s %d\n", val->string, bond->params.fail_over_mac);
}
-static DEVICE_ATTR(fail_over_mac, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(fail_over_mac, 0644,
bonding_show_fail_over_mac, bonding_sysfs_store_option);
/* Show the arp timer interval. */
return sprintf(buf, "%d\n", bond->params.arp_interval);
}
-static DEVICE_ATTR(arp_interval, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(arp_interval, 0644,
bonding_show_arp_interval, bonding_sysfs_store_option);
/* Show the arp targets. */
return res;
}
-static DEVICE_ATTR(arp_ip_target, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(arp_ip_target, 0644,
bonding_show_arp_targets, bonding_sysfs_store_option);
/* Show the up and down delays. */
return sprintf(buf, "%d\n", bond->params.downdelay * bond->params.miimon);
}
-static DEVICE_ATTR(downdelay, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(downdelay, 0644,
bonding_show_downdelay, bonding_sysfs_store_option);
static ssize_t bonding_show_updelay(struct device *d,
return sprintf(buf, "%d\n", bond->params.updelay * bond->params.miimon);
}
-static DEVICE_ATTR(updelay, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(updelay, 0644,
bonding_show_updelay, bonding_sysfs_store_option);
/* Show the LACP interval. */
return sprintf(buf, "%s %d\n", val->string, bond->params.lacp_fast);
}
-static DEVICE_ATTR(lacp_rate, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(lacp_rate, 0644,
bonding_show_lacp, bonding_sysfs_store_option);
static ssize_t bonding_show_min_links(struct device *d,
return sprintf(buf, "%u\n", bond->params.min_links);
}
-static DEVICE_ATTR(min_links, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(min_links, 0644,
bonding_show_min_links, bonding_sysfs_store_option);
static ssize_t bonding_show_ad_select(struct device *d,
return sprintf(buf, "%s %d\n", val->string, bond->params.ad_select);
}
-static DEVICE_ATTR(ad_select, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(ad_select, 0644,
bonding_show_ad_select, bonding_sysfs_store_option);
/* Show the number of peer notifications to send after a failover event. */
struct bonding *bond = to_bond(d);
return sprintf(buf, "%d\n", bond->params.num_peer_notif);
}
-static DEVICE_ATTR(num_grat_arp, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(num_grat_arp, 0644,
bonding_show_num_peer_notif, bonding_sysfs_store_option);
-static DEVICE_ATTR(num_unsol_na, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(num_unsol_na, 0644,
bonding_show_num_peer_notif, bonding_sysfs_store_option);
/* Show the MII monitor interval. */
return sprintf(buf, "%d\n", bond->params.miimon);
}
-static DEVICE_ATTR(miimon, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(miimon, 0644,
bonding_show_miimon, bonding_sysfs_store_option);
/* Show the primary slave. */
return count;
}
-static DEVICE_ATTR(primary, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(primary, 0644,
bonding_show_primary, bonding_sysfs_store_option);
/* Show the primary_reselect flag. */
return sprintf(buf, "%s %d\n",
val->string, bond->params.primary_reselect);
}
-static DEVICE_ATTR(primary_reselect, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(primary_reselect, 0644,
bonding_show_primary_reselect, bonding_sysfs_store_option);
/* Show the use_carrier flag. */
return sprintf(buf, "%d\n", bond->params.use_carrier);
}
-static DEVICE_ATTR(use_carrier, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(use_carrier, 0644,
bonding_show_carrier, bonding_sysfs_store_option);
return count;
}
-static DEVICE_ATTR(active_slave, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(active_slave, 0644,
bonding_show_active_slave, bonding_sysfs_store_option);
/* Show link status of the bond interface. */
return sprintf(buf, "%s\n", active ? "up" : "down");
}
-static DEVICE_ATTR(mii_status, S_IRUGO, bonding_show_mii_status, NULL);
+static DEVICE_ATTR(mii_status, 0444, bonding_show_mii_status, NULL);
/* Show current 802.3ad aggregator ID. */
static ssize_t bonding_show_ad_aggregator(struct device *d,
return count;
}
-static DEVICE_ATTR(ad_aggregator, S_IRUGO, bonding_show_ad_aggregator, NULL);
+static DEVICE_ATTR(ad_aggregator, 0444, bonding_show_ad_aggregator, NULL);
/* Show number of active 802.3ad ports. */
return count;
}
-static DEVICE_ATTR(ad_num_ports, S_IRUGO, bonding_show_ad_num_ports, NULL);
+static DEVICE_ATTR(ad_num_ports, 0444, bonding_show_ad_num_ports, NULL);
/* Show current 802.3ad actor key. */
return count;
}
-static DEVICE_ATTR(ad_actor_key, S_IRUGO, bonding_show_ad_actor_key, NULL);
+static DEVICE_ATTR(ad_actor_key, 0444, bonding_show_ad_actor_key, NULL);
/* Show current 802.3ad partner key. */
return count;
}
-static DEVICE_ATTR(ad_partner_key, S_IRUGO, bonding_show_ad_partner_key, NULL);
+static DEVICE_ATTR(ad_partner_key, 0444, bonding_show_ad_partner_key, NULL);
/* Show current 802.3ad partner mac. */
return count;
}
-static DEVICE_ATTR(ad_partner_mac, S_IRUGO, bonding_show_ad_partner_mac, NULL);
+static DEVICE_ATTR(ad_partner_mac, 0444, bonding_show_ad_partner_mac, NULL);
/* Show the queue_ids of the slaves in the current bond. */
static ssize_t bonding_show_queue_id(struct device *d,
return res;
}
-static DEVICE_ATTR(queue_id, S_IRUGO | S_IWUSR, bonding_show_queue_id,
+static DEVICE_ATTR(queue_id, 0644, bonding_show_queue_id,
bonding_sysfs_store_option);
return sprintf(buf, "%d\n", bond->params.all_slaves_active);
}
-static DEVICE_ATTR(all_slaves_active, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(all_slaves_active, 0644,
bonding_show_slaves_active, bonding_sysfs_store_option);
/* Show the number of IGMP membership reports to send on link failure */
return sprintf(buf, "%d\n", bond->params.resend_igmp);
}
-static DEVICE_ATTR(resend_igmp, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(resend_igmp, 0644,
bonding_show_resend_igmp, bonding_sysfs_store_option);
return sprintf(buf, "%d\n", bond->params.lp_interval);
}
-static DEVICE_ATTR(lp_interval, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(lp_interval, 0644,
bonding_show_lp_interval, bonding_sysfs_store_option);
static ssize_t bonding_show_tlb_dynamic_lb(struct device *d,
struct bonding *bond = to_bond(d);
return sprintf(buf, "%d\n", bond->params.tlb_dynamic_lb);
}
-static DEVICE_ATTR(tlb_dynamic_lb, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(tlb_dynamic_lb, 0644,
bonding_show_tlb_dynamic_lb, bonding_sysfs_store_option);
static ssize_t bonding_show_packets_per_slave(struct device *d,
return sprintf(buf, "%u\n", packets_per_slave);
}
-static DEVICE_ATTR(packets_per_slave, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(packets_per_slave, 0644,
bonding_show_packets_per_slave, bonding_sysfs_store_option);
static ssize_t bonding_show_ad_actor_sys_prio(struct device *d,
return 0;
}
-static DEVICE_ATTR(ad_actor_sys_prio, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(ad_actor_sys_prio, 0644,
bonding_show_ad_actor_sys_prio, bonding_sysfs_store_option);
static ssize_t bonding_show_ad_actor_system(struct device *d,
return 0;
}
-static DEVICE_ATTR(ad_actor_system, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(ad_actor_system, 0644,
bonding_show_ad_actor_system, bonding_sysfs_store_option);
static ssize_t bonding_show_ad_user_port_key(struct device *d,
return 0;
}
-static DEVICE_ATTR(ad_user_port_key, S_IRUGO | S_IWUSR,
+static DEVICE_ATTR(ad_user_port_key, 0644,
bonding_show_ad_user_port_key, bonding_sysfs_store_option);
static struct attribute *per_bond_attrs[] = {
.mode = _mode }, \
.show = _show, \
};
-#define SLAVE_ATTR_RO(_name) \
- SLAVE_ATTR(_name, S_IRUGO, _name##_show)
+#define SLAVE_ATTR_RO(_name) \
+ SLAVE_ATTR(_name, 0444, _name##_show)
static ssize_t state_show(struct slave *slave, char *buf)
{
static LIST_HEAD(ser_release_list);
static bool ser_loop;
-module_param(ser_loop, bool, S_IRUGO);
+module_param(ser_loop, bool, 0444);
MODULE_PARM_DESC(ser_loop, "Run in simulated loopback mode.");
static bool ser_use_stx = true;
-module_param(ser_use_stx, bool, S_IRUGO);
+module_param(ser_use_stx, bool, 0444);
MODULE_PARM_DESC(ser_use_stx, "STX enabled or not.");
static bool ser_use_fcs = true;
-module_param(ser_use_fcs, bool, S_IRUGO);
+module_param(ser_use_fcs, bool, 0444);
MODULE_PARM_DESC(ser_use_fcs, "FCS enabled or not.");
static int ser_write_chunk = MAX_WRITE_CHUNK;
-module_param(ser_write_chunk, int, S_IRUGO);
+module_param(ser_write_chunk, int, 0444);
MODULE_PARM_DESC(ser_write_chunk, "Maximum size of data written to UART.");
ser->debugfs_tty_dir =
debugfs_create_dir(tty->name, debugfsdir);
if (!IS_ERR(ser->debugfs_tty_dir)) {
- debugfs_create_blob("last_tx_msg", S_IRUSR,
- ser->debugfs_tty_dir,
- &ser->tx_blob);
+ debugfs_create_blob("last_tx_msg", 0400,
+ ser->debugfs_tty_dir,
+ &ser->tx_blob);
- debugfs_create_blob("last_rx_msg", S_IRUSR,
- ser->debugfs_tty_dir,
- &ser->rx_blob);
+ debugfs_create_blob("last_rx_msg", 0400,
+ ser->debugfs_tty_dir,
+ &ser->rx_blob);
- debugfs_create_x32("ser_state", S_IRUSR,
- ser->debugfs_tty_dir,
- (u32 *)&ser->state);
+ debugfs_create_x32("ser_state", 0400,
+ ser->debugfs_tty_dir,
+ (u32 *)&ser->state);
- debugfs_create_x8("tty_status", S_IRUSR,
- ser->debugfs_tty_dir,
- &ser->tty_status);
+ debugfs_create_x8("tty_status", 0400,
+ ser->debugfs_tty_dir,
+ &ser->tty_status);
}
ser->tx_blob.data = ser->tx_data;
#define PAD_POW2(x, pow) ((((x)&((pow)-1))==0) ? 0 : (((pow)-((x)&((pow)-1)))))
static bool spi_loop;
-module_param(spi_loop, bool, S_IRUGO);
+module_param(spi_loop, bool, 0444);
MODULE_PARM_DESC(spi_loop, "SPI running in loopback mode.");
/* SPI frame alignment. */
-module_param(spi_frm_align, int, S_IRUGO);
+module_param(spi_frm_align, int, 0444);
MODULE_PARM_DESC(spi_frm_align, "SPI frame alignment.");
/*
* SPI padding options.
* Warning: must be a base of 2 (& operation used) and can not be zero !
*/
-module_param(spi_up_head_align, int, S_IRUGO);
+module_param(spi_up_head_align, int, 0444);
MODULE_PARM_DESC(spi_up_head_align, "SPI uplink head alignment.");
-module_param(spi_up_tail_align, int, S_IRUGO);
+module_param(spi_up_tail_align, int, 0444);
MODULE_PARM_DESC(spi_up_tail_align, "SPI uplink tail alignment.");
-module_param(spi_down_head_align, int, S_IRUGO);
+module_param(spi_down_head_align, int, 0444);
MODULE_PARM_DESC(spi_down_head_align, "SPI downlink head alignment.");
-module_param(spi_down_tail_align, int, S_IRUGO);
+module_param(spi_down_tail_align, int, 0444);
MODULE_PARM_DESC(spi_down_tail_align, "SPI downlink tail alignment.");
#ifdef CONFIG_ARM
static inline void dev_debugfs_add(struct cfspi *cfspi)
{
cfspi->dbgfs_dir = debugfs_create_dir(cfspi->pdev->name, dbgfs_root);
- cfspi->dbgfs_state = debugfs_create_file("state", S_IRUGO,
+ cfspi->dbgfs_state = debugfs_create_file("state", 0444,
cfspi->dbgfs_dir, cfspi,
&dbgfs_state_fops);
- cfspi->dbgfs_frame = debugfs_create_file("frame", S_IRUGO,
+ cfspi->dbgfs_frame = debugfs_create_file("frame", 0444,
cfspi->dbgfs_dir, cfspi,
&dbgfs_frame_fops);
}
if (IS_ERR(cfv->debugfs))
return;
- debugfs_create_u32("rx-napi-complete", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("rx-napi-complete", 0400, cfv->debugfs,
&cfv->stats.rx_napi_complete);
- debugfs_create_u32("rx-napi-resched", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("rx-napi-resched", 0400, cfv->debugfs,
&cfv->stats.rx_napi_resched);
- debugfs_create_u32("rx-nomem", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("rx-nomem", 0400, cfv->debugfs,
&cfv->stats.rx_nomem);
- debugfs_create_u32("rx-kicks", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("rx-kicks", 0400, cfv->debugfs,
&cfv->stats.rx_kicks);
- debugfs_create_u32("tx-full-ring", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("tx-full-ring", 0400, cfv->debugfs,
&cfv->stats.tx_full_ring);
- debugfs_create_u32("tx-no-mem", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("tx-no-mem", 0400, cfv->debugfs,
&cfv->stats.tx_no_mem);
- debugfs_create_u32("tx-kicks", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("tx-kicks", 0400, cfv->debugfs,
&cfv->stats.tx_kicks);
- debugfs_create_u32("tx-flow-on", S_IRUSR, cfv->debugfs,
+ debugfs_create_u32("tx-flow-on", 0400, cfv->debugfs,
&cfv->stats.tx_flow_on);
}
return ret;
}
-static DEVICE_ATTR(mb0_id, S_IWUSR | S_IRUGO,
- at91_sysfs_show_mb0_id, at91_sysfs_set_mb0_id);
+static DEVICE_ATTR(mb0_id, 0644, at91_sysfs_show_mb0_id, at91_sysfs_set_mb0_id);
static struct attribute *at91_sysfs_attrs[] = {
&dev_attr_mb0_id.attr,
* otherwise 11 bit SFF messages.
*/
static int msgobj15_eff;
-module_param(msgobj15_eff, int, S_IRUGO);
+module_param(msgobj15_eff, int, 0444);
MODULE_PARM_DESC(msgobj15_eff, "Extended 29-bit frames for message object 15 "
"(default: 11-bit standard frames)");
static int i82527_compat;
-module_param(i82527_compat, int, S_IRUGO);
+module_param(i82527_compat, int, 0444);
MODULE_PARM_DESC(i82527_compat, "Strict Intel 82527 comptibility mode "
"without using additional functions");
return 0;
}
-static netdev_tx_t cc770_start_xmit(struct sk_buff *skb, struct net_device *dev)
+static void cc770_tx(struct net_device *dev, int mo)
{
struct cc770_priv *priv = netdev_priv(dev);
- struct net_device_stats *stats = &dev->stats;
- struct can_frame *cf = (struct can_frame *)skb->data;
- unsigned int mo = obj2msgobj(CC770_OBJ_TX);
+ struct can_frame *cf = (struct can_frame *)priv->tx_skb->data;
u8 dlc, rtr;
u32 id;
int i;
- if (can_dropped_invalid_skb(dev, skb))
- return NETDEV_TX_OK;
-
- if ((cc770_read_reg(priv,
- msgobj[mo].ctrl1) & TXRQST_UNC) == TXRQST_SET) {
- netdev_err(dev, "TX register is still occupied!\n");
- return NETDEV_TX_BUSY;
- }
-
- netif_stop_queue(dev);
-
dlc = cf->can_dlc;
id = cf->can_id;
- if (cf->can_id & CAN_RTR_FLAG)
- rtr = 0;
- else
- rtr = MSGCFG_DIR;
+ rtr = cf->can_id & CAN_RTR_FLAG ? 0 : MSGCFG_DIR;
+
+ cc770_write_reg(priv, msgobj[mo].ctrl0,
+ MSGVAL_RES | TXIE_RES | RXIE_RES | INTPND_RES);
cc770_write_reg(priv, msgobj[mo].ctrl1,
RMTPND_RES | TXRQST_RES | CPUUPD_SET | NEWDAT_RES);
- cc770_write_reg(priv, msgobj[mo].ctrl0,
- MSGVAL_SET | TXIE_SET | RXIE_RES | INTPND_RES);
+
if (id & CAN_EFF_FLAG) {
id &= CAN_EFF_MASK;
cc770_write_reg(priv, msgobj[mo].config,
for (i = 0; i < dlc; i++)
cc770_write_reg(priv, msgobj[mo].data[i], cf->data[i]);
- /* Store echo skb before starting the transfer */
- can_put_echo_skb(skb, dev, 0);
-
cc770_write_reg(priv, msgobj[mo].ctrl1,
- RMTPND_RES | TXRQST_SET | CPUUPD_RES | NEWDAT_UNC);
+ RMTPND_UNC | TXRQST_SET | CPUUPD_RES | NEWDAT_UNC);
+ cc770_write_reg(priv, msgobj[mo].ctrl0,
+ MSGVAL_SET | TXIE_SET | RXIE_SET | INTPND_UNC);
+}
- stats->tx_bytes += dlc;
+static netdev_tx_t cc770_start_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ struct cc770_priv *priv = netdev_priv(dev);
+ unsigned int mo = obj2msgobj(CC770_OBJ_TX);
+ if (can_dropped_invalid_skb(dev, skb))
+ return NETDEV_TX_OK;
- /*
- * HM: We had some cases of repeated IRQs so make sure the
- * INT is acknowledged I know it's already further up, but
- * doing again fixed the issue
- */
- cc770_write_reg(priv, msgobj[mo].ctrl0,
- MSGVAL_UNC | TXIE_UNC | RXIE_UNC | INTPND_RES);
+ netif_stop_queue(dev);
+
+ if ((cc770_read_reg(priv,
+ msgobj[mo].ctrl1) & TXRQST_UNC) == TXRQST_SET) {
+ netdev_err(dev, "TX register is still occupied!\n");
+ return NETDEV_TX_BUSY;
+ }
+
+ priv->tx_skb = skb;
+ cc770_tx(dev, mo);
return NETDEV_TX_OK;
}
struct cc770_priv *priv = netdev_priv(dev);
struct net_device_stats *stats = &dev->stats;
unsigned int mo = obj2msgobj(o);
+ struct can_frame *cf;
+ u8 ctrl1;
+
+ ctrl1 = cc770_read_reg(priv, msgobj[mo].ctrl1);
- /* Nothing more to send, switch off interrupts */
cc770_write_reg(priv, msgobj[mo].ctrl0,
MSGVAL_RES | TXIE_RES | RXIE_RES | INTPND_RES);
- /*
- * We had some cases of repeated IRQ so make sure the
- * INT is acknowledged
+ cc770_write_reg(priv, msgobj[mo].ctrl1,
+ RMTPND_RES | TXRQST_RES | MSGLST_RES | NEWDAT_RES);
+
+ if (unlikely(!priv->tx_skb)) {
+ netdev_err(dev, "missing tx skb in tx interrupt\n");
+ return;
+ }
+
+ if (unlikely(ctrl1 & MSGLST_SET)) {
+ stats->rx_over_errors++;
+ stats->rx_errors++;
+ }
+
+ /* When the CC770 is sending an RTR message and it receives a regular
+ * message that matches the id of the RTR message, it will overwrite the
+ * outgoing message in the TX register. When this happens we must
+ * process the received message and try to transmit the outgoing skb
+ * again.
*/
- cc770_write_reg(priv, msgobj[mo].ctrl0,
- MSGVAL_UNC | TXIE_UNC | RXIE_UNC | INTPND_RES);
+ if (unlikely(ctrl1 & NEWDAT_SET)) {
+ cc770_rx(dev, mo, ctrl1);
+ cc770_tx(dev, mo);
+ return;
+ }
+ cf = (struct can_frame *)priv->tx_skb->data;
+ stats->tx_bytes += cf->can_dlc;
stats->tx_packets++;
+
+ can_put_echo_skb(priv->tx_skb, dev, 0);
can_get_echo_skb(dev, 0);
+ priv->tx_skb = NULL;
+
netif_wake_queue(dev);
}
priv->can.do_set_bittiming = cc770_set_bittiming;
priv->can.do_set_mode = cc770_set_mode;
priv->can.ctrlmode_supported = CAN_CTRLMODE_3_SAMPLES;
+ priv->tx_skb = NULL;
memcpy(priv->obj_flags, cc770_obj_flags, sizeof(cc770_obj_flags));
u8 cpu_interface; /* CPU interface register */
u8 clkout; /* Clock out register */
u8 bus_config; /* Bus conffiguration register */
+
+ struct sk_buff *tx_skb;
};
struct net_device *alloc_cc770dev(int sizeof_priv);
static u8 bcr[MAXDEV] = {[0 ... (MAXDEV - 1)] = 0xff};
static int indirect[MAXDEV] = {[0 ... (MAXDEV - 1)] = -1};
-module_param_hw_array(port, ulong, ioport, NULL, S_IRUGO);
+module_param_hw_array(port, ulong, ioport, NULL, 0444);
MODULE_PARM_DESC(port, "I/O port number");
-module_param_hw_array(mem, ulong, iomem, NULL, S_IRUGO);
+module_param_hw_array(mem, ulong, iomem, NULL, 0444);
MODULE_PARM_DESC(mem, "I/O memory address");
-module_param_hw_array(indirect, int, ioport, NULL, S_IRUGO);
+module_param_hw_array(indirect, int, ioport, NULL, 0444);
MODULE_PARM_DESC(indirect, "Indirect access via address and data port");
-module_param_hw_array(irq, int, irq, NULL, S_IRUGO);
+module_param_hw_array(irq, int, irq, NULL, 0444);
MODULE_PARM_DESC(irq, "IRQ number");
-module_param_array(clk, int, NULL, S_IRUGO);
+module_param_array(clk, int, NULL, 0444);
MODULE_PARM_DESC(clk, "External oscillator clock frequency "
"(default=16000000 [16 MHz])");
-module_param_array(cir, byte, NULL, S_IRUGO);
+module_param_array(cir, byte, NULL, 0444);
MODULE_PARM_DESC(cir, "CPU interface register (default=0x40 [DSC])");
-module_param_array(cor, byte, NULL, S_IRUGO);
+module_param_array(cor, byte, NULL, 0444);
MODULE_PARM_DESC(cor, "Clockout register (default=0x00)");
-module_param_array(bcr, byte, NULL, S_IRUGO);
+module_param_array(bcr, byte, NULL, 0444);
MODULE_PARM_DESC(bcr, "Bus configuration register (default=0x40 [CBY])");
#define CC770_IOSIZE 0x20
} \
} \
module_param_named(name, grcan_module_config.name, \
- mtype, S_IRUGO); \
+ mtype, 0444); \
MODULE_PARM_DESC(name, desc)
#define GRCAN_CONFIG_ATTR(name, desc) \
struct grcan_priv *priv = netdev_priv(dev); \
return sprintf(buf, "%d\n", priv->config.name); \
} \
- static DEVICE_ATTR(name, S_IRUGO | S_IWUSR, \
+ static DEVICE_ATTR(name, 0644, \
grcan_show_##name, \
grcan_store_##name); \
GRCAN_MODULE_PARAM(name, ushort, GRCAN_NOT_BOOL, desc)
#define IFI_CANFD_STCMD_ERROR_ACTIVE BIT(2)
#define IFI_CANFD_STCMD_ERROR_PASSIVE BIT(3)
#define IFI_CANFD_STCMD_BUSOFF BIT(4)
+#define IFI_CANFD_STCMD_ERROR_WARNING BIT(5)
#define IFI_CANFD_STCMD_BUSMONITOR BIT(16)
#define IFI_CANFD_STCMD_LOOPBACK BIT(18)
#define IFI_CANFD_STCMD_DISABLE_CANFD BIT(24)
#define IFI_CANFD_TXSTCMD_OVERFLOW BIT(13)
#define IFI_CANFD_INTERRUPT 0xc
+#define IFI_CANFD_INTERRUPT_ERROR_BUSOFF BIT(0)
#define IFI_CANFD_INTERRUPT_ERROR_WARNING BIT(1)
+#define IFI_CANFD_INTERRUPT_ERROR_STATE_CHG BIT(2)
+#define IFI_CANFD_INTERRUPT_ERROR_REC_TEC_INC BIT(3)
#define IFI_CANFD_INTERRUPT_ERROR_COUNTER BIT(10)
#define IFI_CANFD_INTERRUPT_TXFIFO_EMPTY BIT(16)
#define IFI_CANFD_INTERRUPT_TXFIFO_REMOVE BIT(22)
#define IFI_CANFD_INTERRUPT_SET_IRQ ((u32)BIT(31))
#define IFI_CANFD_IRQMASK 0x10
+#define IFI_CANFD_IRQMASK_ERROR_BUSOFF BIT(0)
+#define IFI_CANFD_IRQMASK_ERROR_WARNING BIT(1)
+#define IFI_CANFD_IRQMASK_ERROR_STATE_CHG BIT(2)
+#define IFI_CANFD_IRQMASK_ERROR_REC_TEC_INC BIT(3)
#define IFI_CANFD_IRQMASK_SET_ERR BIT(7)
#define IFI_CANFD_IRQMASK_SET_TS BIT(15)
#define IFI_CANFD_IRQMASK_TXFIFO_EMPTY BIT(16)
#define IFI_CANFD_SYSCLOCK 0x50
#define IFI_CANFD_VER 0x54
+#define IFI_CANFD_VER_REV_MASK 0xff
+#define IFI_CANFD_VER_REV_MIN_SUPPORTED 0x15
#define IFI_CANFD_IP_ID 0x58
#define IFI_CANFD_IP_ID_VALUE 0xD073CAFD
if (enable) {
enirq = IFI_CANFD_IRQMASK_TXFIFO_EMPTY |
- IFI_CANFD_IRQMASK_RXFIFO_NEMPTY;
+ IFI_CANFD_IRQMASK_RXFIFO_NEMPTY |
+ IFI_CANFD_IRQMASK_ERROR_STATE_CHG |
+ IFI_CANFD_IRQMASK_ERROR_WARNING |
+ IFI_CANFD_IRQMASK_ERROR_BUSOFF;
if (priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING)
enirq |= IFI_CANFD_INTERRUPT_ERROR_COUNTER;
}
return 1;
}
-static int ifi_canfd_handle_lec_err(struct net_device *ndev, const u32 errctr)
+static int ifi_canfd_handle_lec_err(struct net_device *ndev)
{
struct ifi_canfd_priv *priv = netdev_priv(ndev);
struct net_device_stats *stats = &ndev->stats;
struct can_frame *cf;
struct sk_buff *skb;
+ u32 errctr = readl(priv->base + IFI_CANFD_ERROR_CTR);
const u32 errmask = IFI_CANFD_ERROR_CTR_OVERLOAD_FIRST |
IFI_CANFD_ERROR_CTR_ACK_ERROR_FIRST |
IFI_CANFD_ERROR_CTR_BIT0_ERROR_FIRST |
switch (new_state) {
case CAN_STATE_ERROR_ACTIVE:
+ /* error active state */
+ priv->can.can_stats.error_warning++;
+ priv->can.state = CAN_STATE_ERROR_ACTIVE;
+ break;
+ case CAN_STATE_ERROR_WARNING:
/* error warning state */
priv->can.can_stats.error_warning++;
priv->can.state = CAN_STATE_ERROR_WARNING;
ifi_canfd_get_berr_counter(ndev, &bec);
switch (new_state) {
- case CAN_STATE_ERROR_ACTIVE:
+ case CAN_STATE_ERROR_WARNING:
/* error warning state */
cf->can_id |= CAN_ERR_CRTL;
cf->data[1] = (bec.txerr > bec.rxerr) ?
return 1;
}
-static int ifi_canfd_handle_state_errors(struct net_device *ndev, u32 stcmd)
+static int ifi_canfd_handle_state_errors(struct net_device *ndev)
{
struct ifi_canfd_priv *priv = netdev_priv(ndev);
+ u32 stcmd = readl(priv->base + IFI_CANFD_STCMD);
int work_done = 0;
- u32 isr;
- /*
- * The ErrWarn condition is a little special, since the bit is
- * located in the INTERRUPT register instead of STCMD register.
- */
- isr = readl(priv->base + IFI_CANFD_INTERRUPT);
- if ((isr & IFI_CANFD_INTERRUPT_ERROR_WARNING) &&
+ if ((stcmd & IFI_CANFD_STCMD_ERROR_ACTIVE) &&
+ (priv->can.state != CAN_STATE_ERROR_ACTIVE)) {
+ netdev_dbg(ndev, "Error, entered active state\n");
+ work_done += ifi_canfd_handle_state_change(ndev,
+ CAN_STATE_ERROR_ACTIVE);
+ }
+
+ if ((stcmd & IFI_CANFD_STCMD_ERROR_WARNING) &&
(priv->can.state != CAN_STATE_ERROR_WARNING)) {
- /* Clear the interrupt */
- writel(IFI_CANFD_INTERRUPT_ERROR_WARNING,
- priv->base + IFI_CANFD_INTERRUPT);
netdev_dbg(ndev, "Error, entered warning state\n");
work_done += ifi_canfd_handle_state_change(ndev,
CAN_STATE_ERROR_WARNING);
{
struct net_device *ndev = napi->dev;
struct ifi_canfd_priv *priv = netdev_priv(ndev);
- const u32 stcmd_state_mask = IFI_CANFD_STCMD_ERROR_PASSIVE |
- IFI_CANFD_STCMD_BUSOFF;
- int work_done = 0;
-
- u32 stcmd = readl(priv->base + IFI_CANFD_STCMD);
u32 rxstcmd = readl(priv->base + IFI_CANFD_RXSTCMD);
- u32 errctr = readl(priv->base + IFI_CANFD_ERROR_CTR);
+ int work_done = 0;
/* Handle bus state changes */
- if ((stcmd & stcmd_state_mask) ||
- ((stcmd & IFI_CANFD_STCMD_ERROR_ACTIVE) == 0))
- work_done += ifi_canfd_handle_state_errors(ndev, stcmd);
+ work_done += ifi_canfd_handle_state_errors(ndev);
/* Handle lost messages on RX */
if (rxstcmd & IFI_CANFD_RXSTCMD_OVERFLOW)
/* Handle lec errors on the bus */
if (priv->can.ctrlmode & CAN_CTRLMODE_BERR_REPORTING)
- work_done += ifi_canfd_handle_lec_err(ndev, errctr);
+ work_done += ifi_canfd_handle_lec_err(ndev);
/* Handle normal messages on RX */
if (!(rxstcmd & IFI_CANFD_RXSTCMD_EMPTY))
struct net_device_stats *stats = &ndev->stats;
const u32 rx_irq_mask = IFI_CANFD_INTERRUPT_RXFIFO_NEMPTY |
IFI_CANFD_INTERRUPT_RXFIFO_NEMPTY_PER |
+ IFI_CANFD_INTERRUPT_ERROR_COUNTER |
+ IFI_CANFD_INTERRUPT_ERROR_STATE_CHG |
IFI_CANFD_INTERRUPT_ERROR_WARNING |
- IFI_CANFD_INTERRUPT_ERROR_COUNTER;
+ IFI_CANFD_INTERRUPT_ERROR_BUSOFF;
const u32 tx_irq_mask = IFI_CANFD_INTERRUPT_TXFIFO_EMPTY |
IFI_CANFD_INTERRUPT_TXFIFO_REMOVE;
- const u32 clr_irq_mask = ~((u32)(IFI_CANFD_INTERRUPT_SET_IRQ |
- IFI_CANFD_INTERRUPT_ERROR_WARNING));
+ const u32 clr_irq_mask = ~((u32)IFI_CANFD_INTERRUPT_SET_IRQ);
u32 isr;
isr = readl(priv->base + IFI_CANFD_INTERRUPT);
struct resource *res;
void __iomem *addr;
int irq, ret;
- u32 id;
+ u32 id, rev;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
addr = devm_ioremap_resource(dev, res);
return -EINVAL;
}
+ rev = readl(addr + IFI_CANFD_VER) & IFI_CANFD_VER_REV_MASK;
+ if (rev < IFI_CANFD_VER_REV_MIN_SUPPORTED) {
+ dev_err(dev, "This block is too old (rev %i), minimum supported is rev %i\n",
+ rev, IFI_CANFD_VER_REV_MIN_SUPPORTED);
+ return -EINVAL;
+ }
+
ndev = alloc_candev(sizeof(*priv), 1);
if (!ndev)
return -ENOMEM;
return scnprintf(buf, PAGE_SIZE, "%s\n", mod->fwinfo);
}
-static DEVICE_ATTR(termination, S_IWUSR | S_IRUGO, ican3_sysfs_show_term,
- ican3_sysfs_set_term);
-static DEVICE_ATTR(fwinfo, S_IRUSR | S_IRUGO, ican3_sysfs_show_fwinfo, NULL);
+static DEVICE_ATTR(termination, 0644, ican3_sysfs_show_term,
+ ican3_sysfs_set_term);
+static DEVICE_ATTR(fwinfo, 0444, ican3_sysfs_show_fwinfo, NULL);
static struct attribute *ican3_sysfs_attrs[] = {
&dev_attr_termination.attr,
#include <linux/pm_runtime.h>
#include <linux/iopoll.h>
#include <linux/can/dev.h>
+#include <linux/pinctrl/consumer.h>
/* napi related */
#define M_CAN_NAPI_WEIGHT 64
/* Rx FIFO 0/1 Configuration (RXF0C/RXF1C) */
#define RXFC_FWM_SHIFT 24
-#define RXFC_FWM_MASK (0x7f < RXFC_FWM_SHIFT)
+#define RXFC_FWM_MASK (0x7f << RXFC_FWM_SHIFT)
#define RXFC_FS_SHIFT 16
#define RXFC_FS_MASK (0x7f << RXFC_FS_SHIFT)
m_can_clk_stop(priv);
}
+ pinctrl_pm_select_sleep_state(dev);
+
priv->can.state = CAN_STATE_SLEEPING;
return 0;
struct net_device *ndev = dev_get_drvdata(dev);
struct m_can_priv *priv = netdev_priv(ndev);
+ pinctrl_pm_select_default_state(dev);
+
m_can_init_ram(priv);
priv->can.state = CAN_STATE_ERROR_ACTIVE;
spin_lock_irqsave(&priv->echo_lock, flags);
can_get_echo_skb(priv->ndev, msg->client);
- spin_unlock_irqrestore(&priv->echo_lock, flags);
/* count bytes of the echo instead of skb */
stats->tx_bytes += cf_len;
/* restart tx queue (a slot is free) */
netif_wake_queue(priv->ndev);
+ spin_unlock_irqrestore(&priv->echo_lock, flags);
return 0;
}
/* this STATUS is the CNF of the RX_BARRIER: Tx path can be setup */
if (pucan_status_is_rx_barrier(msg)) {
- unsigned long flags;
if (priv->enable_tx_path) {
int err = priv->enable_tx_path(priv);
return err;
}
- /* restart network queue only if echo skb array is free */
- spin_lock_irqsave(&priv->echo_lock, flags);
-
- if (!priv->can.echo_skb[priv->echo_idx]) {
- spin_unlock_irqrestore(&priv->echo_lock, flags);
-
- netif_wake_queue(ndev);
- } else {
- spin_unlock_irqrestore(&priv->echo_lock, flags);
- }
+ /* start network queue (echo_skb array is empty) */
+ netif_start_queue(ndev);
return 0;
}
*/
should_stop_tx_queue = !!(priv->can.echo_skb[priv->echo_idx]);
- spin_unlock_irqrestore(&priv->echo_lock, flags);
-
- /* write the skb on the interface */
- priv->write_tx_msg(priv, msg);
-
/* stop network tx queue if not enough room to save one more msg too */
if (priv->can.ctrlmode & CAN_CTRLMODE_FD)
should_stop_tx_queue |= (room_left <
if (should_stop_tx_queue)
netif_stop_queue(ndev);
+ spin_unlock_irqrestore(&priv->echo_lock, flags);
+
+ /* write the skb on the interface */
+ priv->write_tx_msg(priv, msg);
+
return NETDEV_TX_OK;
}
priv->tx_pages_free++;
spin_unlock_irqrestore(&priv->tx_lock, flags);
- /* wake producer up */
- netif_wake_queue(priv->ucan.ndev);
+ /* wake producer up (only if enough room in echo_skb array) */
+ spin_lock_irqsave(&priv->ucan.echo_lock, flags);
+ if (!priv->ucan.can.echo_skb[priv->ucan.echo_idx])
+ netif_wake_queue(priv->ucan.ndev);
+
+ spin_unlock_irqrestore(&priv->ucan.echo_lock, flags);
}
/* re-enable Rx DMA transfer for this CAN */
static int indirect[MAXDEV] = {[0 ... (MAXDEV - 1)] = -1};
static spinlock_t indirect_lock[MAXDEV]; /* lock for indirect access mode */
-module_param_hw_array(port, ulong, ioport, NULL, S_IRUGO);
+module_param_hw_array(port, ulong, ioport, NULL, 0444);
MODULE_PARM_DESC(port, "I/O port number");
-module_param_hw_array(mem, ulong, iomem, NULL, S_IRUGO);
+module_param_hw_array(mem, ulong, iomem, NULL, 0444);
MODULE_PARM_DESC(mem, "I/O memory address");
-module_param_hw_array(indirect, int, ioport, NULL, S_IRUGO);
+module_param_hw_array(indirect, int, ioport, NULL, 0444);
MODULE_PARM_DESC(indirect, "Indirect access via address and data port");
-module_param_hw_array(irq, int, irq, NULL, S_IRUGO);
+module_param_hw_array(irq, int, irq, NULL, 0444);
MODULE_PARM_DESC(irq, "IRQ number");
-module_param_array(clk, int, NULL, S_IRUGO);
+module_param_array(clk, int, NULL, 0444);
MODULE_PARM_DESC(clk, "External oscillator clock frequency "
"(default=16000000 [16 MHz])");
-module_param_array(cdr, byte, NULL, S_IRUGO);
+module_param_array(cdr, byte, NULL, 0444);
MODULE_PARM_DESC(cdr, "Clock divider register "
"(default=0x48 [CDR_CBP | CDR_CLK_OFF])");
-module_param_array(ocr, byte, NULL, S_IRUGO);
+module_param_array(ocr, byte, NULL, 0444);
MODULE_PARM_DESC(ocr, "Output control register "
"(default=0x18 [OCR_TX0_PUSHPULL])");
return count;
}
-static const DEVICE_ATTR(chip, S_IRUGO, show_chip, NULL);
-static const DEVICE_ATTR(output, S_IRUGO | S_IWUSR, show_output, store_output);
+static const DEVICE_ATTR(chip, 0444, show_chip, NULL);
+static const DEVICE_ATTR(output, 0644, show_output, store_output);
static const struct attribute *const netdev_sysfs_attrs[] = {
&dev_attr_chip.attr,
#define DEVICE_NAME "mcp251x"
static int mcp251x_enable_dma; /* Enable SPI DMA. Default: 0 (Off) */
-module_param(mcp251x_enable_dma, int, S_IRUGO);
+module_param(mcp251x_enable_dma, int, 0444);
MODULE_PARM_DESC(mcp251x_enable_dma, "Enable SPI DMA. Default: 0 (Off)");
static const struct can_bittiming_const mcp251x_bittiming_const = {
(dev->version >> 8) & 0xf,
dev->version & 0xff);
}
-static DEVICE_ATTR(firmware, S_IRUGO, show_firmware, NULL);
+static DEVICE_ATTR(firmware, 0444, show_firmware, NULL);
static ssize_t show_hardware(struct device *d,
struct device_attribute *attr, char *buf)
(dev->version >> 24) & 0xf,
(dev->version >> 16) & 0xff);
}
-static DEVICE_ATTR(hardware, S_IRUGO, show_hardware, NULL);
+static DEVICE_ATTR(hardware, 0444, show_hardware, NULL);
static ssize_t show_nets(struct device *d,
struct device_attribute *attr, char *buf)
return sprintf(buf, "%d", dev->net_count);
}
-static DEVICE_ATTR(nets, S_IRUGO, show_nets, NULL);
+static DEVICE_ATTR(nets, 0444, show_nets, NULL);
static int esd_usb2_send_msg(struct esd_usb2 *dev, struct esd_usb2_msg *msg)
{
*/
static bool echo; /* echo testing. Default: 0 (Off) */
-module_param(echo, bool, S_IRUGO);
+module_param(echo, bool, 0444);
MODULE_PARM_DESC(echo, "Echo sent frames (for testing). Default: 0 (Off)");
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_NET_DSA_BCM_SF2) += bcm-sf2.o
bcm-sf2-objs := bcm_sf2.o bcm_sf2_cfp.o
-obj-$(CONFIG_NET_DSA_LOOP) += dsa_loop.o dsa_loop_bdinfo.o
+obj-$(CONFIG_NET_DSA_LOOP) += dsa_loop.o
+ifdef CONFIG_NET_DSA_LOOP
+obj-$(CONFIG_FIXED_PHY) += dsa_loop_bdinfo.o
+endif
obj-$(CONFIG_NET_DSA_MT7530) += mt7530.o
obj-$(CONFIG_NET_DSA_MV88E6060) += mv88e6060.o
obj-$(CONFIG_NET_DSA_QCA8K) += qca8k.o
unsigned int i;
for (i = 0; i < mib_size; i++)
- memcpy(data + i * ETH_GSTRING_LEN,
- mibs[i].name, ETH_GSTRING_LEN);
+ strlcpy(data + i * ETH_GSTRING_LEN,
+ mibs[i].name, ETH_GSTRING_LEN);
}
EXPORT_SYMBOL(b53_get_strings);
err = request_threaded_irq(chip->irq, NULL,
mv88e6xxx_g1_irq_thread_fn,
- IRQF_ONESHOT | IRQF_TRIGGER_FALLING,
+ IRQF_ONESHOT,
dev_name(chip->dev), chip);
if (err)
mv88e6xxx_g1_irq_free_common(chip);
static void mv88e6xxx_irq_poll_free(struct mv88e6xxx_chip *chip)
{
+ mv88e6xxx_g1_irq_free_common(chip);
+
kthread_cancel_delayed_work_sync(&chip->irq_poll_work);
kthread_destroy_worker(chip->kworker);
}
STATS_TYPE_BANK0 | STATS_TYPE_BANK1);
}
+static const uint8_t *mv88e6xxx_atu_vtu_stats_strings[] = {
+ "atu_member_violation",
+ "atu_miss_violation",
+ "atu_full_violation",
+ "vtu_member_violation",
+ "vtu_miss_violation",
+};
+
+static void mv88e6xxx_atu_vtu_get_strings(uint8_t *data)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(mv88e6xxx_atu_vtu_stats_strings); i++)
+ strlcpy(data + i * ETH_GSTRING_LEN,
+ mv88e6xxx_atu_vtu_stats_strings[i],
+ ETH_GSTRING_LEN);
+}
+
static void mv88e6xxx_get_strings(struct dsa_switch *ds, int port,
uint8_t *data)
{
if (chip->info->ops->serdes_get_strings) {
data += count * ETH_GSTRING_LEN;
- chip->info->ops->serdes_get_strings(chip, port, data);
+ count = chip->info->ops->serdes_get_strings(chip, port, data);
}
+ data += count * ETH_GSTRING_LEN;
+ mv88e6xxx_atu_vtu_get_strings(data);
+
mutex_unlock(&chip->reg_lock);
}
if (chip->info->ops->serdes_get_sset_count)
serdes_count = chip->info->ops->serdes_get_sset_count(chip,
port);
- if (serdes_count < 0)
+ if (serdes_count < 0) {
count = serdes_count;
- else
- count += serdes_count;
+ goto out;
+ }
+ count += serdes_count;
+ count += ARRAY_SIZE(mv88e6xxx_atu_vtu_stats_strings);
+
out:
mutex_unlock(&chip->reg_lock);
0);
}
+static void mv88e6xxx_atu_vtu_get_stats(struct mv88e6xxx_chip *chip, int port,
+ uint64_t *data)
+{
+ *data++ = chip->ports[port].atu_member_violation;
+ *data++ = chip->ports[port].atu_miss_violation;
+ *data++ = chip->ports[port].atu_full_violation;
+ *data++ = chip->ports[port].vtu_member_violation;
+ *data++ = chip->ports[port].vtu_miss_violation;
+}
+
static void mv88e6xxx_get_stats(struct mv88e6xxx_chip *chip, int port,
uint64_t *data)
{
if (chip->info->ops->stats_get_stats)
count = chip->info->ops->stats_get_stats(chip, port, data);
+ mutex_lock(&chip->reg_lock);
if (chip->info->ops->serdes_get_stats) {
data += count;
- mutex_lock(&chip->reg_lock);
- chip->info->ops->serdes_get_stats(chip, port, data);
- mutex_unlock(&chip->reg_lock);
+ count = chip->info->ops->serdes_get_stats(chip, port, data);
}
+ data += count;
+ mv88e6xxx_atu_vtu_get_stats(chip, port, data);
+ mutex_unlock(&chip->reg_lock);
}
static void mv88e6xxx_get_ethtool_stats(struct dsa_switch *ds, int port,
struct mv88e6xxx_port {
u64 serdes_stats[2];
+ u64 atu_member_violation;
+ u64 atu_miss_violation;
+ u64 atu_full_violation;
+ u64 vtu_member_violation;
+ u64 vtu_miss_violation;
};
struct mv88e6xxx_chip {
/* Statistics from the SERDES interface */
int (*serdes_get_sset_count)(struct mv88e6xxx_chip *chip, int port);
- void (*serdes_get_strings)(struct mv88e6xxx_chip *chip, int port,
- uint8_t *data);
- void (*serdes_get_stats)(struct mv88e6xxx_chip *chip, int port,
- uint64_t *data);
+ int (*serdes_get_strings)(struct mv88e6xxx_chip *chip, int port,
+ uint8_t *data);
+ int (*serdes_get_stats)(struct mv88e6xxx_chip *chip, int port,
+ uint64_t *data);
/* VLAN Translation Unit operations */
int (*vtu_getnext)(struct mv88e6xxx_chip *chip,
if (err)
goto out;
- mutex_unlock(&chip->reg_lock);
-
if (val & MV88E6XXX_G1_ATU_OP_AGE_OUT_VIOLATION) {
dev_err_ratelimited(chip->dev,
"ATU age out violation for %pM\n",
dev_err_ratelimited(chip->dev,
"ATU member violation for %pM portvec %x\n",
entry.mac, entry.portvec);
+ chip->ports[entry.portvec].atu_member_violation++;
}
- if (val & MV88E6XXX_G1_ATU_OP_MEMBER_VIOLATION)
+ if (val & MV88E6XXX_G1_ATU_OP_MEMBER_VIOLATION) {
dev_err_ratelimited(chip->dev,
"ATU miss violation for %pM portvec %x\n",
entry.mac, entry.portvec);
+ chip->ports[entry.portvec].atu_miss_violation++;
+ }
- if (val & MV88E6XXX_G1_ATU_OP_FULL_VIOLATION)
+ if (val & MV88E6XXX_G1_ATU_OP_FULL_VIOLATION) {
dev_err_ratelimited(chip->dev,
"ATU full violation for %pM portvec %x\n",
entry.mac, entry.portvec);
+ chip->ports[entry.portvec].atu_full_violation++;
+ }
+ mutex_unlock(&chip->reg_lock);
return IRQ_HANDLED;
if (err)
goto out;
- mutex_unlock(&chip->reg_lock);
-
spid = val & MV88E6XXX_G1_VTU_OP_SPID_MASK;
if (val & MV88E6XXX_G1_VTU_OP_MEMBER_VIOLATION) {
dev_err_ratelimited(chip->dev, "VTU member violation for vid %d, source port %d\n",
entry.vid, spid);
+ chip->ports[spid].vtu_member_violation++;
}
- if (val & MV88E6XXX_G1_VTU_OP_MISS_VIOLATION)
- dev_err_ratelimited(chip->dev, "VTU miss violation for vid %d, source port %d\n",
+ if (val & MV88E6XXX_G1_VTU_OP_MISS_VIOLATION) {
+ dev_dbg_ratelimited(chip->dev, "VTU miss violation for vid %d, source port %d\n",
entry.vid, spid);
+ chip->ports[spid].vtu_miss_violation++;
+ }
+
+ mutex_unlock(&chip->reg_lock);
return IRQ_HANDLED;
return 0;
}
-void mv88e6352_serdes_get_strings(struct mv88e6xxx_chip *chip,
- int port, uint8_t *data)
+int mv88e6352_serdes_get_strings(struct mv88e6xxx_chip *chip,
+ int port, uint8_t *data)
{
struct mv88e6352_serdes_hw_stat *stat;
int i;
if (!mv88e6352_port_has_serdes(chip, port))
- return;
+ return 0;
for (i = 0; i < ARRAY_SIZE(mv88e6352_serdes_hw_stats); i++) {
stat = &mv88e6352_serdes_hw_stats[i];
memcpy(data + i * ETH_GSTRING_LEN, stat->string,
ETH_GSTRING_LEN);
}
+ return ARRAY_SIZE(mv88e6352_serdes_hw_stats);
}
static uint64_t mv88e6352_serdes_get_stat(struct mv88e6xxx_chip *chip,
return val;
}
-void mv88e6352_serdes_get_stats(struct mv88e6xxx_chip *chip, int port,
- uint64_t *data)
+int mv88e6352_serdes_get_stats(struct mv88e6xxx_chip *chip, int port,
+ uint64_t *data)
{
struct mv88e6xxx_port *mv88e6xxx_port = &chip->ports[port];
struct mv88e6352_serdes_hw_stat *stat;
int i;
if (!mv88e6352_port_has_serdes(chip, port))
- return;
+ return 0;
BUILD_BUG_ON(ARRAY_SIZE(mv88e6352_serdes_hw_stats) >
ARRAY_SIZE(mv88e6xxx_port->serdes_stats));
mv88e6xxx_port->serdes_stats[i] += value;
data[i] = mv88e6xxx_port->serdes_stats[i];
}
+
+ return ARRAY_SIZE(mv88e6352_serdes_hw_stats);
}
/* Set the power on/off for 10GBASE-R and 10GBASE-X4/X2 */
int mv88e6352_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on);
int mv88e6390_serdes_power(struct mv88e6xxx_chip *chip, int port, bool on);
int mv88e6352_serdes_get_sset_count(struct mv88e6xxx_chip *chip, int port);
-void mv88e6352_serdes_get_strings(struct mv88e6xxx_chip *chip,
- int port, uint8_t *data);
-void mv88e6352_serdes_get_stats(struct mv88e6xxx_chip *chip, int port,
- uint64_t *data);
+int mv88e6352_serdes_get_strings(struct mv88e6xxx_chip *chip,
+ int port, uint8_t *data);
+int mv88e6352_serdes_get_stats(struct mv88e6xxx_chip *chip, int port,
+ uint64_t *data);
#endif
#
config NET_VENDOR_8390
- bool "National Semi-conductor 8390 devices"
+ bool "National Semiconductor 8390 devices"
default y
depends on NET_VENDOR_NATSEMI
---help---
static int apne_owned; /* signal if card already owned */
static u32 apne_msg_enable;
-module_param_named(msg_enable, apne_msg_enable, uint, (S_IRUSR|S_IRGRP|S_IROTH));
+module_param_named(msg_enable, apne_msg_enable, uint, 0444);
MODULE_PARM_DESC(msg_enable, "Debug message level (see linux/netdevice.h for bitmap)");
struct net_device * __init apne_probe(int unit)
static unsigned version_printed;
static u32 msg_enable;
-module_param(msg_enable, uint, (S_IRUSR|S_IRGRP|S_IROTH));
+module_param(msg_enable, uint, 0444);
MODULE_PARM_DESC(msg_enable, "Debug message level (see linux/netdevice.h for bitmap)");
/*
module_param_hw_array(io, int, ioport, NULL, 0);
module_param_hw_array(irq, int, irq, NULL, 0);
module_param_array(bad, int, NULL, 0);
-module_param_named(msg_enable, ne_msg_enable, uint, (S_IRUSR|S_IRGRP|S_IROTH));
+module_param_named(msg_enable, ne_msg_enable, uint, 0444);
MODULE_PARM_DESC(io, "I/O base address(es),required");
MODULE_PARM_DESC(irq, "IRQ number(s)");
MODULE_PARM_DESC(bad, "Accept card(s) with bad signatures");
MODULE_DESCRIPTION("PCI NE2000 clone driver");
MODULE_LICENSE("GPL");
-module_param_named(msg_enable, ne2k_msg_enable, uint, (S_IRUSR|S_IRGRP|S_IROTH));
+module_param_named(msg_enable, ne2k_msg_enable, uint, 0444);
module_param_array(options, int, NULL, 0);
module_param_array(full_duplex, int, NULL, 0);
MODULE_PARM_DESC(msg_enable, "Debug message level (see linux/netdevice.h for bitmap)");
module_param_hw_array(io, int, ioport, NULL, 0);
module_param_hw_array(irq, int, irq, NULL, 0);
-module_param_named(msg_enable, ultra_msg_enable, uint, (S_IRUSR|S_IRGRP|S_IROTH));
+module_param_named(msg_enable, ultra_msg_enable, uint, 0444);
MODULE_PARM_DESC(io, "I/O base address(es)");
MODULE_PARM_DESC(irq, "IRQ number(s) (assigned)");
MODULE_PARM_DESC(msg_enable, "Debug message level (see linux/netdevice.h for bitmap)");
static u32 stnic_msg_enable;
-module_param_named(msg_enable, stnic_msg_enable, uint, (S_IRUSR|S_IRGRP|S_IROTH));
+module_param_named(msg_enable, stnic_msg_enable, uint, 0444);
MODULE_PARM_DESC(msg_enable, "Debug message level (see linux/netdevice.h for bitmap)");
/* SH7750 specific read/write io. */
module_param_hw_array(irq, int, irq, NULL, 0);
module_param_hw_array(mem, int, iomem, NULL, 0);
module_param_hw_array(mem_end, int, iomem, NULL, 0);
-module_param_named(msg_enable, wd_msg_enable, uint, (S_IRUSR|S_IRGRP|S_IROTH));
+module_param_named(msg_enable, wd_msg_enable, uint, 0444);
MODULE_PARM_DESC(io, "I/O base address(es)");
MODULE_PARM_DESC(irq, "IRQ number(s) (ignored for PureData boards)");
MODULE_PARM_DESC(mem, "memory base address(es)(ignored for PureData boards)");
source "drivers/net/ethernet/natsemi/Kconfig"
source "drivers/net/ethernet/netronome/Kconfig"
+source "drivers/net/ethernet/ni/Kconfig"
source "drivers/net/ethernet/8390/Kconfig"
config NET_NETX
obj-$(CONFIG_FEALNX) += fealnx.o
obj-$(CONFIG_NET_VENDOR_NATSEMI) += natsemi/
obj-$(CONFIG_NET_VENDOR_NETRONOME) += netronome/
+obj-$(CONFIG_NET_VENDOR_NI) += ni/
obj-$(CONFIG_NET_NETX) += netx-eth.o
obj-$(CONFIG_NET_VENDOR_NUVOTON) += nuvoton/
obj-$(CONFIG_NET_VENDOR_NVIDIA) += nvidia/
static atomic_t instance_count = ATOMIC_INIT(~0);
/* Module parameters */
static int debug = -1;
-module_param(debug, int, S_IRUGO | S_IWUSR);
+module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, "Message Level (-1: default, 0: no output, 16: all)");
static const u32 default_msg_level = (NETIF_MSG_DRV | NETIF_MSG_PROBE |
#define RX_DESCRIPTORS 64
static int dma_rx_num = RX_DESCRIPTORS;
-module_param(dma_rx_num, int, S_IRUGO | S_IWUSR);
+module_param(dma_rx_num, int, 0644);
MODULE_PARM_DESC(dma_rx_num, "Number of descriptors in the RX list");
#define TX_DESCRIPTORS 64
static int dma_tx_num = TX_DESCRIPTORS;
-module_param(dma_tx_num, int, S_IRUGO | S_IWUSR);
+module_param(dma_tx_num, int, 0644);
MODULE_PARM_DESC(dma_tx_num, "Number of descriptors in the TX list");
*/
wmb();
- writel(mmio_read_reg, ena_dev->reg_bar + ENA_REGS_MMIO_REG_READ_OFF);
+ writel_relaxed(mmio_read_reg,
+ ena_dev->reg_bar + ENA_REGS_MMIO_REG_READ_OFF);
+ mmiowb();
for (i = 0; i < timeout; i++) {
if (read_resp->req_id == mmio_read->seq_num)
break;
/* write the aenq doorbell after all AENQ descriptors were read */
mb();
- writel((u32)aenq->head, dev->reg_bar + ENA_REGS_AENQ_HEAD_DB_OFF);
+ writel_relaxed((u32)aenq->head,
+ dev->reg_bar + ENA_REGS_AENQ_HEAD_DB_OFF);
+ mmiowb();
}
int ena_com_dev_reset(struct ena_com_dev *ena_dev,
return io_sq->q_depth - 1 - cnt;
}
-static inline int ena_com_write_sq_doorbell(struct ena_com_io_sq *io_sq)
+static inline int ena_com_write_sq_doorbell(struct ena_com_io_sq *io_sq,
+ bool relaxed)
{
u16 tail;
pr_debug("write submission queue doorbell for queue: %d tail: %d\n",
io_sq->qid, tail);
- writel(tail, io_sq->db_addr);
+ if (relaxed)
+ writel_relaxed(tail, io_sq->db_addr);
+ else
+ writel(tail, io_sq->db_addr);
return 0;
}
* issue a doorbell
*/
wmb();
- ena_com_write_sq_doorbell(rx_ring->ena_com_io_sq);
+ ena_com_write_sq_doorbell(rx_ring->ena_com_io_sq, true);
+ mmiowb();
}
rx_ring->next_to_use = next_to_use;
if (netif_xmit_stopped(txq) || !skb->xmit_more) {
/* trigger the dma engine */
- ena_com_write_sq_doorbell(tx_ring->ena_com_io_sq);
+ ena_com_write_sq_doorbell(tx_ring->ena_com_io_sq, false);
u64_stats_update_begin(&tx_ring->syncp);
tx_ring->tx_stats.doorbells++;
u64_stats_update_end(&tx_ring->syncp);
#ifdef CONFIG_AMD_XGBE_HAVE_ECC
/* Only expose the ECC parameters if supported */
-module_param(ecc_sec_info_threshold, uint, S_IWUSR | S_IRUGO);
+module_param(ecc_sec_info_threshold, uint, 0644);
MODULE_PARM_DESC(ecc_sec_info_threshold,
" ECC corrected error informational threshold setting");
-module_param(ecc_sec_warn_threshold, uint, S_IWUSR | S_IRUGO);
+module_param(ecc_sec_warn_threshold, uint, 0644);
MODULE_PARM_DESC(ecc_sec_warn_threshold,
" ECC corrected error warning threshold setting");
-module_param(ecc_sec_period, uint, S_IWUSR | S_IRUGO);
+module_param(ecc_sec_period, uint, 0644);
MODULE_PARM_DESC(ecc_sec_period, " ECC corrected error period (in seconds)");
-module_param(ecc_ded_threshold, uint, S_IWUSR | S_IRUGO);
+module_param(ecc_ded_threshold, uint, 0644);
MODULE_PARM_DESC(ecc_ded_threshold, " ECC detected error threshold setting");
-module_param(ecc_ded_period, uint, S_IWUSR | S_IRUGO);
+module_param(ecc_ded_period, uint, 0644);
MODULE_PARM_DESC(ecc_ded_period, " ECC detected error period (in seconds)");
#endif
MODULE_DESCRIPTION(XGBE_DRV_DESC);
static int debug = -1;
-module_param(debug, int, S_IWUSR | S_IRUGO);
+module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, " Network interface message level setting");
static const u32 default_msg_level = (NETIF_MSG_LINK | NETIF_MSG_IFDOWN |
#define AQ_CFG_TX_FRAME_MAX (16U * 1024U)
#define AQ_CFG_RX_FRAME_MAX (4U * 1024U)
+#define AQ_CFG_TX_CLEAN_BUDGET 256U
+
/* LRO */
#define AQ_CFG_IS_LRO_DEF 1U
self->ndev->hw_features |= aq_hw_caps->hw_features;
self->ndev->features = aq_hw_caps->hw_features;
self->ndev->priv_flags = aq_hw_caps->hw_priv_flags;
+ self->ndev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+
self->ndev->mtu = aq_nic_cfg->mtu - ETH_HLEN;
self->ndev->max_mtu = aq_hw_caps->mtu - ETH_FCS_LEN - ETH_HLEN;
out:
return err;
}
+
+void aq_nic_shutdown(struct aq_nic_s *self)
+{
+ int err = 0;
+
+ if (!self->ndev)
+ return;
+
+ rtnl_lock();
+
+ netif_device_detach(self->ndev);
+
+ err = aq_nic_stop(self);
+ if (err < 0)
+ goto err_exit;
+ aq_nic_deinit(self);
+
+err_exit:
+ rtnl_unlock();
+}
\ No newline at end of file
u32 aq_nic_get_fw_version(struct aq_nic_s *self);
int aq_nic_change_pm_state(struct aq_nic_s *self, pm_message_t *pm_msg);
int aq_nic_update_interrupt_moderation_settings(struct aq_nic_s *self);
+void aq_nic_shutdown(struct aq_nic_s *self);
#endif /* AQ_NIC_H */
pci_disable_device(pdev);
}
+static void aq_pci_shutdown(struct pci_dev *pdev)
+{
+ struct aq_nic_s *self = pci_get_drvdata(pdev);
+
+ aq_nic_shutdown(self);
+
+ pci_disable_device(pdev);
+
+ if (system_state == SYSTEM_POWER_OFF) {
+ pci_wake_from_d3(pdev, false);
+ pci_set_power_state(pdev, PCI_D3hot);
+ }
+}
+
static int aq_pci_suspend(struct pci_dev *pdev, pm_message_t pm_msg)
{
struct aq_nic_s *self = pci_get_drvdata(pdev);
.remove = aq_pci_remove,
.suspend = aq_pci_suspend,
.resume = aq_pci_resume,
+ .shutdown = aq_pci_shutdown,
};
module_pci_driver(aq_pci_ops);
netif_stop_subqueue(ndev, ring->idx);
}
-void aq_ring_tx_clean(struct aq_ring_s *self)
+bool aq_ring_tx_clean(struct aq_ring_s *self)
{
struct device *dev = aq_nic_get_dev(self->aq_nic);
+ unsigned int budget = AQ_CFG_TX_CLEAN_BUDGET;
- for (; self->sw_head != self->hw_head;
+ for (; self->sw_head != self->hw_head && budget--;
self->sw_head = aq_ring_next_dx(self, self->sw_head)) {
struct aq_ring_buff_s *buff = &self->buff_ring[self->sw_head];
buff->pa = 0U;
buff->eop_index = 0xffffU;
}
+
+ return !!budget;
}
#define AQ_SKB_ALIGN SKB_DATA_ALIGN(sizeof(struct skb_shared_info))
void aq_ring_update_queue_state(struct aq_ring_s *ring);
void aq_ring_queue_wake(struct aq_ring_s *ring);
void aq_ring_queue_stop(struct aq_ring_s *ring);
-void aq_ring_tx_clean(struct aq_ring_s *self);
+bool aq_ring_tx_clean(struct aq_ring_s *self);
int aq_ring_rx_clean(struct aq_ring_s *self,
struct napi_struct *napi,
int *work_done,
static int aq_vec_poll(struct napi_struct *napi, int budget)
{
struct aq_vec_s *self = container_of(napi, struct aq_vec_s, napi);
+ unsigned int sw_tail_old = 0U;
struct aq_ring_s *ring = NULL;
+ bool was_tx_cleaned = true;
+ unsigned int i = 0U;
int work_done = 0;
int err = 0;
- unsigned int i = 0U;
- unsigned int sw_tail_old = 0U;
- bool was_tx_cleaned = false;
if (!self) {
err = -EINVAL;
if (ring[AQ_VEC_TX_ID].sw_head !=
ring[AQ_VEC_TX_ID].hw_head) {
- aq_ring_tx_clean(&ring[AQ_VEC_TX_ID]);
+ was_tx_cleaned = aq_ring_tx_clean(&ring[AQ_VEC_TX_ID]);
aq_ring_update_queue_state(&ring[AQ_VEC_TX_ID]);
- was_tx_cleaned = true;
}
err = self->aq_hw_ops->hw_ring_rx_receive(self->aq_hw,
}
}
- if (was_tx_cleaned)
+ if (!was_tx_cleaned)
work_done = budget;
if (work_done < budget) {
#define HW_ATL_UCP_0X370_REG 0x0370U
+#define HW_ATL_MIF_CMD 0x0200U
+#define HW_ATL_MIF_ADDR 0x0208U
+#define HW_ATL_MIF_VAL 0x020CU
+
#define HW_ATL_FW_SM_RAM 0x2U
#define HW_ATL_MPI_FW_VERSION 0x18
#define HW_ATL_MPI_CONTROL_ADR 0x0368U
static int hw_atl_utils_soft_reset_flb(struct aq_hw_s *self)
{
+ u32 gsr, val;
int k = 0;
- u32 gsr;
aq_hw_write_reg(self, 0x404, 0x40e1);
AQ_HW_SLEEP(50);
/* Cleanup SPI */
- aq_hw_write_reg(self, 0x534, 0xA0);
- aq_hw_write_reg(self, 0x100, 0x9F);
- aq_hw_write_reg(self, 0x100, 0x809F);
+ val = aq_hw_read_reg(self, 0x53C);
+ aq_hw_write_reg(self, 0x53C, val | 0x10);
gsr = aq_hw_read_reg(self, HW_ATL_GLB_SOFT_RES_ADR);
aq_hw_write_reg(self, HW_ATL_GLB_SOFT_RES_ADR, (gsr & 0xBFFF) | 0x8000);
aq_hw_write_reg(self, 0x404, 0x80e0);
aq_hw_write_reg(self, 0x32a8, 0x0);
aq_hw_write_reg(self, 0x520, 0x1);
+
+ /* Reset SPI again because of possible interrupted SPI burst */
+ val = aq_hw_read_reg(self, 0x53C);
+ aq_hw_write_reg(self, 0x53C, val | 0x10);
AQ_HW_SLEEP(10);
+ /* Clear SPI reset state */
+ aq_hw_write_reg(self, 0x53C, val & ~0x10);
+
aq_hw_write_reg(self, 0x404, 0x180e0);
for (k = 0; k < 1000; k++) {
aq_pr_err("FW kickstart failed\n");
return -EIO;
}
+ /* Old FW requires fixed delay after init */
+ AQ_HW_SLEEP(15);
return 0;
}
static int hw_atl_utils_soft_reset_rbl(struct aq_hw_s *self)
{
- u32 gsr, rbl_status;
+ u32 gsr, val, rbl_status;
int k;
aq_hw_write_reg(self, 0x404, 0x40e1);
/* Alter RBL status */
aq_hw_write_reg(self, 0x388, 0xDEAD);
+ /* Cleanup SPI */
+ val = aq_hw_read_reg(self, 0x53C);
+ aq_hw_write_reg(self, 0x53C, val | 0x10);
+
/* Global software reset*/
hw_atl_rx_rx_reg_res_dis_set(self, 0U);
hw_atl_tx_tx_reg_res_dis_set(self, 0U);
aq_pr_err("FW kickstart failed\n");
return -EIO;
}
+ /* Old FW requires fixed delay after init */
+ AQ_HW_SLEEP(15);
return 0;
}
}
}
- aq_hw_write_reg(self, 0x00000208U, a);
-
- for (++cnt; --cnt;) {
- u32 i = 0U;
+ aq_hw_write_reg(self, HW_ATL_MIF_ADDR, a);
- aq_hw_write_reg(self, 0x00000200U, 0x00008000U);
+ for (++cnt; --cnt && !err;) {
+ aq_hw_write_reg(self, HW_ATL_MIF_CMD, 0x00008000U);
- for (i = 1024U;
- (0x100U & aq_hw_read_reg(self, 0x00000200U)) && --i;) {
- }
+ if (IS_CHIP_FEATURE(REVISION_B1))
+ AQ_HW_WAIT_FOR(a != aq_hw_read_reg(self,
+ HW_ATL_MIF_ADDR),
+ 1, 1000U);
+ else
+ AQ_HW_WAIT_FOR(!(0x100 & aq_hw_read_reg(self,
+ HW_ATL_MIF_CMD)),
+ 1, 1000U);
- *(p++) = aq_hw_read_reg(self, 0x0000020CU);
+ *(p++) = aq_hw_read_reg(self, HW_ATL_MIF_VAL);
+ a += 4;
}
hw_atl_reg_glb_cpu_sem_set(self, 1U, HW_ATL_FW_SM_RAM);
err_exit:;
}
-int hw_atl_utils_mpi_set_speed(struct aq_hw_s *self, u32 speed)
+static int hw_atl_utils_mpi_set_speed(struct aq_hw_s *self, u32 speed)
{
u32 val = aq_hw_read_reg(self, HW_ATL_MPI_CONTROL_ADR);
u32 val = hw_atl_reg_glb_mif_id_get(self);
u32 mif_rev = val & 0xFFU;
- if ((3U & mif_rev) == 1U) {
- chip_features |=
- HAL_ATLANTIC_UTILS_CHIP_REVISION_A0 |
+ if ((0xFU & mif_rev) == 1U) {
+ chip_features |= HAL_ATLANTIC_UTILS_CHIP_REVISION_A0 |
HAL_ATLANTIC_UTILS_CHIP_MPI_AQ |
HAL_ATLANTIC_UTILS_CHIP_MIPS;
- } else if ((3U & mif_rev) == 2U) {
- chip_features |=
- HAL_ATLANTIC_UTILS_CHIP_REVISION_B0 |
+ } else if ((0xFU & mif_rev) == 2U) {
+ chip_features |= HAL_ATLANTIC_UTILS_CHIP_REVISION_B0 |
+ HAL_ATLANTIC_UTILS_CHIP_MPI_AQ |
+ HAL_ATLANTIC_UTILS_CHIP_MIPS |
+ HAL_ATLANTIC_UTILS_CHIP_TPO2 |
+ HAL_ATLANTIC_UTILS_CHIP_RPF2;
+ } else if ((0xFU & mif_rev) == 0xAU) {
+ chip_features |= HAL_ATLANTIC_UTILS_CHIP_REVISION_B1 |
HAL_ATLANTIC_UTILS_CHIP_MPI_AQ |
HAL_ATLANTIC_UTILS_CHIP_MIPS |
HAL_ATLANTIC_UTILS_CHIP_TPO2 |
#define HAL_ATLANTIC_UTILS_CHIP_MPI_AQ 0x00000010U
#define HAL_ATLANTIC_UTILS_CHIP_REVISION_A0 0x01000000U
#define HAL_ATLANTIC_UTILS_CHIP_REVISION_B0 0x02000000U
+#define HAL_ATLANTIC_UTILS_CHIP_REVISION_B1 0x04000000U
#define IS_CHIP_FEATURE(_F_) (HAL_ATLANTIC_UTILS_CHIP_##_F_ & \
self->chip_features)
#define NIC_MAJOR_DRIVER_VERSION 2
#define NIC_MINOR_DRIVER_VERSION 0
#define NIC_BUILD_DRIVER_VERSION 2
-#define NIC_REVISION_DRIVER_VERSION 0
+#define NIC_REVISION_DRIVER_VERSION 1
#define AQ_CFG_DRV_VERSION_SUFFIX "-kern"
/* Optional regulator for PHY */
priv->regulator = devm_regulator_get_optional(dev, "phy");
if (IS_ERR(priv->regulator)) {
- if (PTR_ERR(priv->regulator) == -EPROBE_DEFER)
- return -EPROBE_DEFER;
+ if (PTR_ERR(priv->regulator) == -EPROBE_DEFER) {
+ err = -EPROBE_DEFER;
+ goto out_clk_disable;
+ }
dev_err(dev, "no regulator found\n");
priv->regulator = NULL;
}
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
+#include <linux/net_dim.h>
#include <linux/etherdevice.h>
#include <linux/platform_device.h>
#include <linux/of.h>
return 0;
}
+static void bcm_sysport_set_rx_coalesce(struct bcm_sysport_priv *priv)
+{
+ u32 reg;
+
+ reg = rdma_readl(priv, RDMA_MBDONE_INTR);
+ reg &= ~(RDMA_INTR_THRESH_MASK |
+ RDMA_TIMEOUT_MASK << RDMA_TIMEOUT_SHIFT);
+ reg |= priv->dim.coal_pkts;
+ reg |= DIV_ROUND_UP(priv->dim.coal_usecs * 1000, 8192) <<
+ RDMA_TIMEOUT_SHIFT;
+ rdma_writel(priv, reg, RDMA_MBDONE_INTR);
+}
+
+static void bcm_sysport_set_tx_coalesce(struct bcm_sysport_tx_ring *ring)
+{
+ struct bcm_sysport_priv *priv = ring->priv;
+ u32 reg;
+
+ reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(ring->index));
+ reg &= ~(RING_INTR_THRESH_MASK |
+ RING_TIMEOUT_MASK << RING_TIMEOUT_SHIFT);
+ reg |= ring->dim.coal_pkts;
+ reg |= DIV_ROUND_UP(ring->dim.coal_usecs * 1000, 8192) <<
+ RING_TIMEOUT_SHIFT;
+ tdma_writel(priv, reg, TDMA_DESC_RING_INTR_CONTROL(ring->index));
+}
+
static int bcm_sysport_get_coalesce(struct net_device *dev,
struct ethtool_coalesce *ec)
{
struct bcm_sysport_priv *priv = netdev_priv(dev);
+ struct bcm_sysport_tx_ring *ring;
+ unsigned int i;
u32 reg;
reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(0));
ec->tx_coalesce_usecs = (reg >> RING_TIMEOUT_SHIFT) * 8192 / 1000;
ec->tx_max_coalesced_frames = reg & RING_INTR_THRESH_MASK;
+ for (i = 0; i < dev->num_tx_queues; i++) {
+ ring = &priv->tx_rings[i];
+ ec->use_adaptive_tx_coalesce |= ring->dim.use_dim;
+ }
reg = rdma_readl(priv, RDMA_MBDONE_INTR);
ec->rx_coalesce_usecs = (reg >> RDMA_TIMEOUT_SHIFT) * 8192 / 1000;
ec->rx_max_coalesced_frames = reg & RDMA_INTR_THRESH_MASK;
+ ec->use_adaptive_rx_coalesce = priv->dim.use_dim;
return 0;
}
struct ethtool_coalesce *ec)
{
struct bcm_sysport_priv *priv = netdev_priv(dev);
+ struct bcm_sysport_tx_ring *ring;
unsigned int i;
- u32 reg;
/* Base system clock is 125Mhz, DMA timeout is this reference clock
* divided by 1024, which yield roughly 8.192 us, our maximum value has
return -EINVAL;
for (i = 0; i < dev->num_tx_queues; i++) {
- reg = tdma_readl(priv, TDMA_DESC_RING_INTR_CONTROL(i));
- reg &= ~(RING_INTR_THRESH_MASK |
- RING_TIMEOUT_MASK << RING_TIMEOUT_SHIFT);
- reg |= ec->tx_max_coalesced_frames;
- reg |= DIV_ROUND_UP(ec->tx_coalesce_usecs * 1000, 8192) <<
- RING_TIMEOUT_SHIFT;
- tdma_writel(priv, reg, TDMA_DESC_RING_INTR_CONTROL(i));
+ ring = &priv->tx_rings[i];
+ ring->dim.coal_pkts = ec->tx_max_coalesced_frames;
+ ring->dim.coal_usecs = ec->tx_coalesce_usecs;
+ if (!ec->use_adaptive_tx_coalesce && ring->dim.use_dim) {
+ ring->dim.coal_pkts = 1;
+ ring->dim.coal_usecs = 0;
+ }
+ ring->dim.use_dim = ec->use_adaptive_tx_coalesce;
+ bcm_sysport_set_tx_coalesce(ring);
}
- reg = rdma_readl(priv, RDMA_MBDONE_INTR);
- reg &= ~(RDMA_INTR_THRESH_MASK |
- RDMA_TIMEOUT_MASK << RDMA_TIMEOUT_SHIFT);
- reg |= ec->rx_max_coalesced_frames;
- reg |= DIV_ROUND_UP(ec->rx_coalesce_usecs * 1000, 8192) <<
- RDMA_TIMEOUT_SHIFT;
- rdma_writel(priv, reg, RDMA_MBDONE_INTR);
+ priv->dim.coal_usecs = ec->rx_coalesce_usecs;
+ priv->dim.coal_pkts = ec->rx_max_coalesced_frames;
+
+ if (!ec->use_adaptive_rx_coalesce && priv->dim.use_dim) {
+ priv->dim.coal_pkts = 1;
+ priv->dim.coal_usecs = 0;
+ }
+ priv->dim.use_dim = ec->use_adaptive_rx_coalesce;
+ bcm_sysport_set_rx_coalesce(priv);
return 0;
}
struct bcm_sysport_stats64 *stats64 = &priv->stats64;
struct net_device *ndev = priv->netdev;
unsigned int processed = 0, to_process;
+ unsigned int processed_bytes = 0;
struct bcm_sysport_cb *cb;
struct sk_buff *skb;
unsigned int p_index;
*/
skb_pull(skb, sizeof(*rsb) + 2);
len -= (sizeof(*rsb) + 2);
+ processed_bytes += len;
/* UniMAC may forward CRC */
if (priv->crc_fwd) {
priv->rx_read_ptr = 0;
}
+ priv->dim.packets = processed;
+ priv->dim.bytes = processed_bytes;
+
return processed;
}
static unsigned int __bcm_sysport_tx_reclaim(struct bcm_sysport_priv *priv,
struct bcm_sysport_tx_ring *ring)
{
- unsigned int c_index, last_c_index, last_tx_cn, num_tx_cbs;
unsigned int pkts_compl = 0, bytes_compl = 0;
struct net_device *ndev = priv->netdev;
+ unsigned int txbds_processed = 0;
struct bcm_sysport_cb *cb;
+ unsigned int txbds_ready;
+ unsigned int c_index;
u32 hw_ind;
/* Clear status before servicing to reduce spurious interrupts */
/* Compute how many descriptors have been processed since last call */
hw_ind = tdma_readl(priv, TDMA_DESC_RING_PROD_CONS_INDEX(ring->index));
c_index = (hw_ind >> RING_CONS_INDEX_SHIFT) & RING_CONS_INDEX_MASK;
- ring->p_index = (hw_ind & RING_PROD_INDEX_MASK);
-
- last_c_index = ring->c_index;
- num_tx_cbs = ring->size;
-
- c_index &= (num_tx_cbs - 1);
-
- if (c_index >= last_c_index)
- last_tx_cn = c_index - last_c_index;
- else
- last_tx_cn = num_tx_cbs - last_c_index + c_index;
+ txbds_ready = (c_index - ring->c_index) & RING_CONS_INDEX_MASK;
netif_dbg(priv, tx_done, ndev,
- "ring=%d c_index=%d last_tx_cn=%d last_c_index=%d\n",
- ring->index, c_index, last_tx_cn, last_c_index);
+ "ring=%d old_c_index=%u c_index=%u txbds_ready=%u\n",
+ ring->index, ring->c_index, c_index, txbds_ready);
- while (last_tx_cn-- > 0) {
- cb = ring->cbs + last_c_index;
+ while (txbds_processed < txbds_ready) {
+ cb = &ring->cbs[ring->clean_index];
bcm_sysport_tx_reclaim_one(ring, cb, &bytes_compl, &pkts_compl);
ring->desc_count++;
- last_c_index++;
- last_c_index &= (num_tx_cbs - 1);
+ txbds_processed++;
+
+ if (likely(ring->clean_index < ring->size - 1))
+ ring->clean_index++;
+ else
+ ring->clean_index = 0;
}
u64_stats_update_begin(&priv->syncp);
ring->packets += pkts_compl;
ring->bytes += bytes_compl;
u64_stats_update_end(&priv->syncp);
+ ring->dim.packets = pkts_compl;
+ ring->dim.bytes = bytes_compl;
ring->c_index = c_index;
{
struct bcm_sysport_tx_ring *ring =
container_of(napi, struct bcm_sysport_tx_ring, napi);
+ struct net_dim_sample dim_sample;
unsigned int work_done = 0;
work_done = bcm_sysport_tx_reclaim(ring->priv, ring);
return 0;
}
+ if (ring->dim.use_dim) {
+ net_dim_sample(ring->dim.event_ctr, ring->dim.packets,
+ ring->dim.bytes, &dim_sample);
+ net_dim(&ring->dim.dim, dim_sample);
+ }
+
return budget;
}
{
struct bcm_sysport_priv *priv =
container_of(napi, struct bcm_sysport_priv, napi);
+ struct net_dim_sample dim_sample;
unsigned int work_done = 0;
work_done = bcm_sysport_desc_rx(priv, budget);
intrl2_0_mask_clear(priv, INTRL2_0_RDMA_MBDONE);
}
+ if (priv->dim.use_dim) {
+ net_dim_sample(priv->dim.event_ctr, priv->dim.packets,
+ priv->dim.bytes, &dim_sample);
+ net_dim(&priv->dim.dim, dim_sample);
+ }
+
return work_done;
}
netif_dbg(priv, wol, priv->netdev, "resumed from WOL\n");
}
+static void bcm_sysport_dim_work(struct work_struct *work)
+{
+ struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct bcm_sysport_net_dim *ndim =
+ container_of(dim, struct bcm_sysport_net_dim, dim);
+ struct bcm_sysport_priv *priv =
+ container_of(ndim, struct bcm_sysport_priv, dim);
+ struct net_dim_cq_moder cur_profile =
+ net_dim_get_profile(dim->mode, dim->profile_ix);
+
+ priv->dim.coal_usecs = cur_profile.usec;
+ priv->dim.coal_pkts = cur_profile.pkts;
+
+ bcm_sysport_set_rx_coalesce(priv);
+ dim->state = NET_DIM_START_MEASURE;
+}
+
+static void bcm_sysport_dim_tx_work(struct work_struct *work)
+{
+ struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct bcm_sysport_net_dim *ndim =
+ container_of(dim, struct bcm_sysport_net_dim, dim);
+ struct bcm_sysport_tx_ring *ring =
+ container_of(ndim, struct bcm_sysport_tx_ring, dim);
+ struct net_dim_cq_moder cur_profile =
+ net_dim_get_profile(dim->mode, dim->profile_ix);
+
+ ring->dim.coal_usecs = cur_profile.usec;
+ ring->dim.coal_pkts = cur_profile.pkts;
+
+ bcm_sysport_set_tx_coalesce(ring);
+ dim->state = NET_DIM_START_MEASURE;
+}
+
/* RX and misc interrupt routine */
static irqreturn_t bcm_sysport_rx_isr(int irq, void *dev_id)
{
}
if (priv->irq0_stat & INTRL2_0_RDMA_MBDONE) {
+ priv->dim.event_ctr++;
if (likely(napi_schedule_prep(&priv->napi))) {
/* disable RX interrupts */
intrl2_0_mask_set(priv, INTRL2_0_RDMA_MBDONE);
continue;
txr = &priv->tx_rings[ring];
+ txr->dim.event_ctr++;
if (likely(napi_schedule_prep(&txr->napi))) {
intrl2_0_mask_set(priv, ring_bit);
continue;
txr = &priv->tx_rings[ring];
+ txr->dim.event_ctr++;
if (likely(napi_schedule_prep(&txr->napi))) {
intrl2_1_mask_set(priv, BIT(ring));
phy_print_status(phydev);
}
+static void bcm_sysport_init_dim(struct bcm_sysport_net_dim *dim,
+ void (*cb)(struct work_struct *work))
+{
+ INIT_WORK(&dim->dim.work, cb);
+ dim->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ dim->event_ctr = 0;
+ dim->packets = 0;
+ dim->bytes = 0;
+}
+
static int bcm_sysport_init_tx_ring(struct bcm_sysport_priv *priv,
unsigned int index)
{
netif_tx_napi_add(priv->netdev, &ring->napi, bcm_sysport_tx_poll, 64);
ring->index = index;
ring->size = size;
+ ring->clean_index = 0;
ring->alloc_size = ring->size;
ring->desc_cpu = p;
ring->desc_count = ring->size;
reg |= (1 << index);
tdma_writel(priv, reg, TDMA_TIER1_ARB_0_QUEUE_EN);
+ bcm_sysport_init_dim(&ring->dim, bcm_sysport_dim_tx_work);
napi_enable(&ring->napi);
netif_dbg(priv, hw, priv->netdev,
return;
napi_disable(&ring->napi);
+ cancel_work_sync(&ring->dim.dim.work);
netif_napi_del(&ring->napi);
bcm_sysport_tx_clean(priv, ring);
struct bcm_sysport_priv *priv = netdev_priv(dev);
/* Enable NAPI */
+ bcm_sysport_init_dim(&priv->dim, bcm_sysport_dim_work);
napi_enable(&priv->napi);
/* Enable RX interrupt and TX ring full interrupt */
/* stop all software from updating hardware */
netif_tx_stop_all_queues(dev);
napi_disable(&priv->napi);
+ cancel_work_sync(&priv->dim.dim.work);
phy_stop(dev->phydev);
/* mask all interrupts */
#define __BCM_SYSPORT_H
#include <linux/if_vlan.h>
+#include <linux/net_dim.h>
/* Receive/transmit descriptor format */
#define DESC_ADDR_HI_STATUS_LEN 0x00
unsigned int num_rx_desc_words;
};
+struct bcm_sysport_net_dim {
+ u16 use_dim;
+ u16 event_ctr;
+ unsigned long packets;
+ unsigned long bytes;
+ u32 coal_usecs;
+ u32 coal_pkts;
+ struct net_dim dim;
+};
+
/* Software view of the TX ring */
struct bcm_sysport_tx_ring {
spinlock_t lock; /* Ring lock for tx reclaim/xmit */
unsigned int desc_count; /* Number of descriptors */
unsigned int curr_desc; /* Current descriptor */
unsigned int c_index; /* Last consumer index */
- unsigned int p_index; /* Current producer index */
+ unsigned int clean_index; /* Current clean index */
struct bcm_sysport_cb *cbs; /* Transmit control blocks */
struct dma_desc *desc_cpu; /* CPU view of the descriptor */
struct bcm_sysport_priv *priv; /* private context backpointer */
unsigned long packets; /* packets statistics */
unsigned long bytes; /* bytes statistics */
+ struct bcm_sysport_net_dim dim; /* Net DIM context */
unsigned int switch_queue; /* switch port queue number */
unsigned int switch_port; /* switch port queue number */
bool inspect; /* inspect switch port and queue */
unsigned int rx_read_ptr;
unsigned int rx_c_index;
+ struct bcm_sysport_net_dim dim;
+
/* PHY device */
struct device_node *phy_dn;
phy_interface_t phy_interface;
bgmac_chip_init(bgmac);
err = request_irq(bgmac->irq, bgmac_interrupt, IRQF_SHARED,
- KBUILD_MODNAME, net_dev);
+ net_dev->name, net_dev);
if (err < 0) {
dev_err(bgmac->dev, "IRQ request error: %d!\n", err);
bgmac_dma_cleanup(bgmac);
struct net_device *net_dev = bgmac->net_dev;
int err;
+ bgmac_chip_intrs_off(bgmac);
+
net_dev->irq = bgmac->irq;
SET_NETDEV_DEV(net_dev, bgmac->dev);
dev_set_drvdata(bgmac->dev, bgmac);
static int disable_msi = 0;
-module_param(disable_msi, int, S_IRUGO);
+module_param(disable_msi, int, 0444);
MODULE_PARM_DESC(disable_msi, "Disable Message Signaled Interrupt (MSI)");
typedef enum {
#define REG_RD8(bp, offset) readb(REG_ADDR(bp, offset))
#define REG_RD16(bp, offset) readw(REG_ADDR(bp, offset))
+#define REG_WR_RELAXED(bp, offset, val) \
+ writel_relaxed((u32)val, REG_ADDR(bp, offset))
+
+#define REG_WR16_RELAXED(bp, offset, val) \
+ writew_relaxed((u16)val, REG_ADDR(bp, offset))
+
#define REG_WR(bp, offset, val) writel((u32)val, REG_ADDR(bp, offset))
#define REG_WR8(bp, offset, val) writeb((u8)val, REG_ADDR(bp, offset))
#define REG_WR16(bp, offset, val) writew((u16)val, REG_ADDR(bp, offset))
#if (BNX2X_DB_SHIFT < BNX2X_DB_MIN_SHIFT)
#error "Min DB doorbell stride is 8"
#endif
-#define DOORBELL(bp, cid, val) \
- do { \
- writel((u32)(val), bp->doorbells + (bp->db_size * (cid))); \
- } while (0)
+#define DOORBELL_RELAXED(bp, cid, val) \
+ writel_relaxed((u32)(val), (bp)->doorbells + ((bp)->db_size * (cid)))
/* TX CSUM helpers */
#define SKB_CS_OFF(skb) (offsetof(struct tcphdr, check) - \
wmb();
txdata->tx_db.data.prod += nbd;
- barrier();
+ /* make sure descriptor update is observed by HW */
+ wmb();
- DOORBELL(bp, txdata->cid, txdata->tx_db.raw);
+ DOORBELL_RELAXED(bp, txdata->cid, txdata->tx_db.raw);
mmiowb();
wmb();
for (i = 0; i < sizeof(rx_prods)/4; i++)
- REG_WR(bp, fp->ustorm_rx_prods_offset + i*4,
- ((u32 *)&rx_prods)[i]);
+ REG_WR_RELAXED(bp, fp->ustorm_rx_prods_offset + i * 4,
+ ((u32 *)&rx_prods)[i]);
mmiowb(); /* keep prod updates ordered */
wmb();
txdata->tx_db.data.prod += 2;
- barrier();
- DOORBELL(bp, txdata->cid, txdata->tx_db.raw);
+ /* make sure descriptor update is observed by the HW */
+ wmb();
+ DOORBELL_RELAXED(bp, txdata->cid, txdata->tx_db.raw);
mmiowb();
barrier();
MODULE_FIRMWARE(FW_FILE_NAME_E2);
int bnx2x_num_queues;
-module_param_named(num_queues, bnx2x_num_queues, int, S_IRUGO);
+module_param_named(num_queues, bnx2x_num_queues, int, 0444);
MODULE_PARM_DESC(num_queues,
" Set number of queues (default is as a number of CPUs)");
static int disable_tpa;
-module_param(disable_tpa, int, S_IRUGO);
+module_param(disable_tpa, int, 0444);
MODULE_PARM_DESC(disable_tpa, " Disable the TPA (LRO) feature");
static int int_mode;
-module_param(int_mode, int, S_IRUGO);
+module_param(int_mode, int, 0444);
MODULE_PARM_DESC(int_mode, " Force interrupt mode other than MSI-X "
"(1 INT#x; 2 MSI)");
static int dropless_fc;
-module_param(dropless_fc, int, S_IRUGO);
+module_param(dropless_fc, int, 0444);
MODULE_PARM_DESC(dropless_fc, " Pause on exhausted host ring");
static int mrrs = -1;
-module_param(mrrs, int, S_IRUGO);
+module_param(mrrs, int, 0444);
MODULE_PARM_DESC(mrrs, " Force Max Read Req Size (0..3) (for debug)");
static int debug;
-module_param(debug, int, S_IRUGO);
+module_param(debug, int, 0444);
MODULE_PARM_DESC(debug, " Default debug msglevel");
static struct workqueue_struct *bnx2x_wq;
*/
mb();
- REG_WR16(bp, BAR_XSTRORM_INTMEM + XSTORM_SPQ_PROD_OFFSET(func),
- bp->spq_prod_idx);
+ REG_WR16_RELAXED(bp, BAR_XSTRORM_INTMEM + XSTORM_SPQ_PROD_OFFSET(func),
+ bp->spq_prod_idx);
mmiowb();
}
bp->ptp_clock = ptp_clock_register(&bp->ptp_clock_info, &bp->pdev->dev);
if (IS_ERR(bp->ptp_clock)) {
bp->ptp_clock = NULL;
- BNX2X_ERR("PTP clock registeration failed\n");
+ BNX2X_ERR("PTP clock registration failed\n");
}
}
wmb();
/* Trigger the PF FW */
- writeb(1, &zone_data->trigger.vf_pf_channel.addr_valid);
+ writeb_relaxed(1, &zone_data->trigger.vf_pf_channel.addr_valid);
+
+ mmiowb();
/* Wait for PF to complete */
while ((tout >= 0) && (!*done)) {
(skb->dev->features & NETIF_F_HW_VLAN_CTAG_RX)) {
u16 vlan_proto = tpa_info->metadata >>
RX_CMP_FLAGS2_METADATA_TPID_SFT;
- u16 vtag = tpa_info->metadata & RX_CMP_FLAGS2_METADATA_VID_MASK;
+ u16 vtag = tpa_info->metadata & RX_CMP_FLAGS2_METADATA_TCI_MASK;
__vlan_hwaccel_put_tag(skb, htons(vlan_proto), vtag);
}
cpu_to_le32(RX_CMP_FLAGS2_META_FORMAT_VLAN)) &&
(skb->dev->features & NETIF_F_HW_VLAN_CTAG_RX)) {
u32 meta_data = le32_to_cpu(rxcmp1->rx_cmp_meta_data);
- u16 vtag = meta_data & RX_CMP_FLAGS2_METADATA_VID_MASK;
+ u16 vtag = meta_data & RX_CMP_FLAGS2_METADATA_TCI_MASK;
u16 vlan_proto = meta_data >> RX_CMP_FLAGS2_METADATA_TPID_SFT;
__vlan_hwaccel_put_tag(skb, htons(vlan_proto), vtag);
/* Sync BD data before updating doorbell */
wmb();
- bnxt_db_write(bp, db, DB_KEY_TX | prod);
+ bnxt_db_write_relaxed(bp, db, DB_KEY_TX | prod);
}
cpr->cp_raw_cons = raw_cons;
struct bnxt_vnic_info *vnic = &bp->vnic_info[vnic_id];
struct hwrm_vnic_tpa_cfg_input req = {0};
+ if (vnic->fw_vnic_id == INVALID_HW_RING_ID)
+ return 0;
+
bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_VNIC_TPA_CFG, -1, -1);
if (tpa_flags) {
return rc;
}
-static int
-bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
- int ring_grps, int cp_rings, int vnics)
+static void
+__bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, struct hwrm_func_cfg_input *req,
+ int tx_rings, int rx_rings, int ring_grps,
+ int cp_rings, int vnics)
{
- struct hwrm_func_cfg_input req = {0};
u32 enables = 0;
- int rc;
- bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_CFG, -1, -1);
- req.fid = cpu_to_le16(0xffff);
+ bnxt_hwrm_cmd_hdr_init(bp, req, HWRM_FUNC_CFG, -1, -1);
+ req->fid = cpu_to_le16(0xffff);
enables |= tx_rings ? FUNC_CFG_REQ_ENABLES_NUM_TX_RINGS : 0;
- req.num_tx_rings = cpu_to_le16(tx_rings);
+ req->num_tx_rings = cpu_to_le16(tx_rings);
if (bp->flags & BNXT_FLAG_NEW_RM) {
enables |= rx_rings ? FUNC_CFG_REQ_ENABLES_NUM_RX_RINGS : 0;
enables |= cp_rings ? FUNC_CFG_REQ_ENABLES_NUM_CMPL_RINGS |
FUNC_CFG_REQ_ENABLES_NUM_HW_RING_GRPS : 0;
enables |= vnics ? FUNC_VF_CFG_REQ_ENABLES_NUM_VNICS : 0;
- req.num_rx_rings = cpu_to_le16(rx_rings);
- req.num_hw_ring_grps = cpu_to_le16(ring_grps);
- req.num_cmpl_rings = cpu_to_le16(cp_rings);
- req.num_stat_ctxs = req.num_cmpl_rings;
- req.num_vnics = cpu_to_le16(vnics);
+ req->num_rx_rings = cpu_to_le16(rx_rings);
+ req->num_hw_ring_grps = cpu_to_le16(ring_grps);
+ req->num_cmpl_rings = cpu_to_le16(cp_rings);
+ req->num_stat_ctxs = req->num_cmpl_rings;
+ req->num_vnics = cpu_to_le16(vnics);
}
- if (!enables)
+ req->enables = cpu_to_le32(enables);
+}
+
+static void
+__bnxt_hwrm_reserve_vf_rings(struct bnxt *bp,
+ struct hwrm_func_vf_cfg_input *req, int tx_rings,
+ int rx_rings, int ring_grps, int cp_rings,
+ int vnics)
+{
+ u32 enables = 0;
+
+ bnxt_hwrm_cmd_hdr_init(bp, req, HWRM_FUNC_VF_CFG, -1, -1);
+ enables |= tx_rings ? FUNC_VF_CFG_REQ_ENABLES_NUM_TX_RINGS : 0;
+ enables |= rx_rings ? FUNC_VF_CFG_REQ_ENABLES_NUM_RX_RINGS : 0;
+ enables |= cp_rings ? FUNC_VF_CFG_REQ_ENABLES_NUM_CMPL_RINGS |
+ FUNC_VF_CFG_REQ_ENABLES_NUM_STAT_CTXS : 0;
+ enables |= ring_grps ? FUNC_VF_CFG_REQ_ENABLES_NUM_HW_RING_GRPS : 0;
+ enables |= vnics ? FUNC_VF_CFG_REQ_ENABLES_NUM_VNICS : 0;
+
+ req->num_tx_rings = cpu_to_le16(tx_rings);
+ req->num_rx_rings = cpu_to_le16(rx_rings);
+ req->num_hw_ring_grps = cpu_to_le16(ring_grps);
+ req->num_cmpl_rings = cpu_to_le16(cp_rings);
+ req->num_stat_ctxs = req->num_cmpl_rings;
+ req->num_vnics = cpu_to_le16(vnics);
+
+ req->enables = cpu_to_le32(enables);
+}
+
+static int
+bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
+ int ring_grps, int cp_rings, int vnics)
+{
+ struct hwrm_func_cfg_input req = {0};
+ int rc;
+
+ __bnxt_hwrm_reserve_pf_rings(bp, &req, tx_rings, rx_rings, ring_grps,
+ cp_rings, vnics);
+ if (!req.enables)
return 0;
- req.enables = cpu_to_le32(enables);
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
return -ENOMEM;
int ring_grps, int cp_rings, int vnics)
{
struct hwrm_func_vf_cfg_input req = {0};
- u32 enables = 0;
int rc;
if (!(bp->flags & BNXT_FLAG_NEW_RM)) {
return 0;
}
- bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_VF_CFG, -1, -1);
- enables |= tx_rings ? FUNC_VF_CFG_REQ_ENABLES_NUM_TX_RINGS : 0;
- enables |= rx_rings ? FUNC_VF_CFG_REQ_ENABLES_NUM_RX_RINGS : 0;
- enables |= cp_rings ? FUNC_VF_CFG_REQ_ENABLES_NUM_CMPL_RINGS |
- FUNC_VF_CFG_REQ_ENABLES_NUM_STAT_CTXS : 0;
- enables |= ring_grps ? FUNC_VF_CFG_REQ_ENABLES_NUM_HW_RING_GRPS : 0;
- enables |= vnics ? FUNC_VF_CFG_REQ_ENABLES_NUM_VNICS : 0;
-
- req.num_tx_rings = cpu_to_le16(tx_rings);
- req.num_rx_rings = cpu_to_le16(rx_rings);
- req.num_hw_ring_grps = cpu_to_le16(ring_grps);
- req.num_cmpl_rings = cpu_to_le16(cp_rings);
- req.num_stat_ctxs = req.num_cmpl_rings;
- req.num_vnics = cpu_to_le16(vnics);
-
- req.enables = cpu_to_le32(enables);
+ __bnxt_hwrm_reserve_vf_rings(bp, &req, tx_rings, rx_rings, ring_grps,
+ cp_rings, vnics);
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
return -ENOMEM;
}
static int bnxt_hwrm_check_vf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
- int ring_grps, int cp_rings)
+ int ring_grps, int cp_rings, int vnics)
{
struct hwrm_func_vf_cfg_input req = {0};
- u32 flags, enables;
+ u32 flags;
int rc;
if (!(bp->flags & BNXT_FLAG_NEW_RM))
return 0;
- bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_VF_CFG, -1, -1);
+ __bnxt_hwrm_reserve_vf_rings(bp, &req, tx_rings, rx_rings, ring_grps,
+ cp_rings, vnics);
flags = FUNC_VF_CFG_REQ_FLAGS_TX_ASSETS_TEST |
FUNC_VF_CFG_REQ_FLAGS_RX_ASSETS_TEST |
FUNC_VF_CFG_REQ_FLAGS_CMPL_ASSETS_TEST |
FUNC_VF_CFG_REQ_FLAGS_RING_GRP_ASSETS_TEST |
FUNC_VF_CFG_REQ_FLAGS_STAT_CTX_ASSETS_TEST |
FUNC_VF_CFG_REQ_FLAGS_VNIC_ASSETS_TEST;
- enables = FUNC_VF_CFG_REQ_ENABLES_NUM_TX_RINGS |
- FUNC_VF_CFG_REQ_ENABLES_NUM_RX_RINGS |
- FUNC_VF_CFG_REQ_ENABLES_NUM_CMPL_RINGS |
- FUNC_VF_CFG_REQ_ENABLES_NUM_HW_RING_GRPS |
- FUNC_VF_CFG_REQ_ENABLES_NUM_STAT_CTXS |
- FUNC_VF_CFG_REQ_ENABLES_NUM_VNICS;
req.flags = cpu_to_le32(flags);
- req.enables = cpu_to_le32(enables);
- req.num_tx_rings = cpu_to_le16(tx_rings);
- req.num_rx_rings = cpu_to_le16(rx_rings);
- req.num_cmpl_rings = cpu_to_le16(cp_rings);
- req.num_hw_ring_grps = cpu_to_le16(ring_grps);
- req.num_stat_ctxs = cpu_to_le16(cp_rings);
- req.num_vnics = cpu_to_le16(1);
- if (bp->flags & BNXT_FLAG_RFS)
- req.num_vnics = cpu_to_le16(rx_rings + 1);
rc = hwrm_send_message_silent(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
return -ENOMEM;
}
static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, int tx_rings, int rx_rings,
- int ring_grps, int cp_rings)
+ int ring_grps, int cp_rings, int vnics)
{
struct hwrm_func_cfg_input req = {0};
- u32 flags, enables;
+ u32 flags;
int rc;
- bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_CFG, -1, -1);
- req.fid = cpu_to_le16(0xffff);
+ __bnxt_hwrm_reserve_pf_rings(bp, &req, tx_rings, rx_rings, ring_grps,
+ cp_rings, vnics);
flags = FUNC_CFG_REQ_FLAGS_TX_ASSETS_TEST;
- enables = FUNC_CFG_REQ_ENABLES_NUM_TX_RINGS;
- req.num_tx_rings = cpu_to_le16(tx_rings);
- if (bp->flags & BNXT_FLAG_NEW_RM) {
+ if (bp->flags & BNXT_FLAG_NEW_RM)
flags |= FUNC_CFG_REQ_FLAGS_RX_ASSETS_TEST |
FUNC_CFG_REQ_FLAGS_CMPL_ASSETS_TEST |
FUNC_CFG_REQ_FLAGS_RING_GRP_ASSETS_TEST |
FUNC_CFG_REQ_FLAGS_STAT_CTX_ASSETS_TEST |
FUNC_CFG_REQ_FLAGS_VNIC_ASSETS_TEST;
- enables |= FUNC_CFG_REQ_ENABLES_NUM_RX_RINGS |
- FUNC_CFG_REQ_ENABLES_NUM_CMPL_RINGS |
- FUNC_CFG_REQ_ENABLES_NUM_HW_RING_GRPS |
- FUNC_CFG_REQ_ENABLES_NUM_STAT_CTXS |
- FUNC_CFG_REQ_ENABLES_NUM_VNICS;
- req.num_rx_rings = cpu_to_le16(rx_rings);
- req.num_cmpl_rings = cpu_to_le16(cp_rings);
- req.num_hw_ring_grps = cpu_to_le16(ring_grps);
- req.num_stat_ctxs = cpu_to_le16(cp_rings);
- req.num_vnics = cpu_to_le16(1);
- if (bp->flags & BNXT_FLAG_RFS)
- req.num_vnics = cpu_to_le16(rx_rings + 1);
- }
+
req.flags = cpu_to_le32(flags);
- req.enables = cpu_to_le32(enables);
rc = hwrm_send_message_silent(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
return -ENOMEM;
}
static int bnxt_hwrm_check_rings(struct bnxt *bp, int tx_rings, int rx_rings,
- int ring_grps, int cp_rings)
+ int ring_grps, int cp_rings, int vnics)
{
if (bp->hwrm_spec_code < 0x10801)
return 0;
if (BNXT_PF(bp))
return bnxt_hwrm_check_pf_rings(bp, tx_rings, rx_rings,
- ring_grps, cp_rings);
+ ring_grps, cp_rings, vnics);
return bnxt_hwrm_check_vf_rings(bp, tx_rings, rx_rings, ring_grps,
- cp_rings);
+ cp_rings, vnics);
}
static void bnxt_hwrm_set_coal_params(struct bnxt_coal *hw_coal,
if (rc)
goto msix_setup_exit;
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
bp->cp_nr_rings = (min == 1) ?
max_t(int, bp->tx_nr_rings, bp->rx_nr_rings) :
bp->tx_nr_rings + bp->rx_nr_rings;
bp->rx_nr_rings = 1;
bp->tx_nr_rings = 1;
bp->cp_nr_rings = 1;
- bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
bp->flags |= BNXT_FLAG_SHARED_RINGS;
bp->irq_tbl[0].vector = bp->pdev->irq;
return 0;
int max_rx, max_tx, tx_sets = 1;
int tx_rings_needed;
int rx_rings = rx;
- int cp, rc;
+ int cp, vnics, rc;
if (tcs)
tx_sets = tcs;
if (max_tx < tx_rings_needed)
return -ENOMEM;
+ vnics = 1;
+ if (bp->flags & BNXT_FLAG_RFS)
+ vnics += rx_rings;
+
if (bp->flags & BNXT_FLAG_AGG_RINGS)
rx_rings <<= 1;
cp = sh ? max_t(int, tx_rings_needed, rx) : tx_rings_needed + rx;
- return bnxt_hwrm_check_rings(bp, tx_rings_needed, rx_rings, rx, cp);
+ return bnxt_hwrm_check_rings(bp, tx_rings_needed, rx_rings, rx, cp,
+ vnics);
}
static void bnxt_unmap_bars(struct bnxt *bp, struct pci_dev *pdev)
return 0;
bnxt_hwrm_func_qcaps(bp);
- __bnxt_close_nic(bp, true, false);
+
+ if (netif_running(bp->dev))
+ __bnxt_close_nic(bp, true, false);
+
bnxt_clear_int_mode(bp);
rc = bnxt_init_int_mode(bp);
- if (rc)
- dev_close(bp->dev);
- else
- rc = bnxt_open_nic(bp, true, false);
+
+ if (netif_running(bp->dev)) {
+ if (rc)
+ dev_close(bp->dev);
+ else
+ rc = bnxt_open_nic(bp, true, false);
+ }
+
return rc;
}
if (rc)
goto init_err_pci_clean;
+ /* No TC has been set yet and rings may have been trimmed due to
+ * limited MSIX, so we re-initialize the TX rings per TC.
+ */
+ bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
+
bnxt_get_wol_settings(bp);
if (bp->flags & BNXT_FLAG_WOL_CAP)
device_set_wakeup_enable(&pdev->dev, bp->wol);
#define RX_CMP_FLAGS2_T_L4_CS_CALC (0x1 << 3)
#define RX_CMP_FLAGS2_META_FORMAT_VLAN (0x1 << 4)
__le32 rx_cmp_meta_data;
+ #define RX_CMP_FLAGS2_METADATA_TCI_MASK 0xffff
#define RX_CMP_FLAGS2_METADATA_VID_MASK 0xfff
#define RX_CMP_FLAGS2_METADATA_TPID_MASK 0xffff0000
#define RX_CMP_FLAGS2_METADATA_TPID_SFT 16
((txr->tx_prod - txr->tx_cons) & bp->tx_ring_mask);
}
+/* For TX and RX ring doorbells with no ordering guarantee*/
+static inline void bnxt_db_write_relaxed(struct bnxt *bp, void __iomem *db,
+ u32 val)
+{
+ writel_relaxed(val, db);
+ if (bp->flags & BNXT_FLAG_DOUBLE_DB)
+ writel_relaxed(val, db);
+}
+
/* For TX and RX ring doorbells */
static inline void bnxt_db_write(struct bnxt *bp, void __iomem *db, u32 val)
{
if (rc)
netdev_info(bp->dev, "Error: %s: flow_handle=0x%x rc=%d",
__func__, flow_handle, rc);
+
+ if (rc)
+ rc = -EIO;
return rc;
}
req.action_flags = cpu_to_le16(action_flags);
mutex_lock(&bp->hwrm_cmd_lock);
-
rc = _hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (!rc)
*flow_handle = resp->flow_handle;
-
mutex_unlock(&bp->hwrm_cmd_lock);
+ if (rc == HWRM_ERR_CODE_RESOURCE_ALLOC_ERROR)
+ rc = -ENOSPC;
+ else if (rc)
+ rc = -EIO;
return rc;
}
netdev_info(bp->dev, "%s: Error rc=%d", __func__, rc);
mutex_unlock(&bp->hwrm_cmd_lock);
+ if (rc)
+ rc = -EIO;
return rc;
}
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
netdev_info(bp->dev, "%s: Error rc=%d", __func__, rc);
+
+ if (rc)
+ rc = -EIO;
return rc;
}
netdev_info(bp->dev, "%s: Error rc=%d", __func__, rc);
mutex_unlock(&bp->hwrm_cmd_lock);
+ if (rc)
+ rc = -EIO;
return rc;
}
rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT);
if (rc)
netdev_info(bp->dev, "%s: Error rc=%d", __func__, rc);
+
+ if (rc)
+ rc = -EIO;
return rc;
}
flow_node = rhashtable_lookup_fast(&tc_info->flow_table,
&tc_flow_cmd->cookie,
tc_info->flow_ht_params);
- if (!flow_node) {
- netdev_info(bp->dev, "ERROR: no flow_node for cookie %lx",
- tc_flow_cmd->cookie);
+ if (!flow_node)
return -EINVAL;
- }
return __bnxt_tc_del_flow(bp, flow_node);
}
flow_node = rhashtable_lookup_fast(&tc_info->flow_table,
&tc_flow_cmd->cookie,
tc_info->flow_ht_params);
- if (!flow_node) {
- netdev_info(bp->dev, "Error: no flow_node for cookie %lx",
- tc_flow_cmd->cookie);
+ if (!flow_node)
return -1;
- }
flow = &flow_node->flow;
curr_stats = &flow->stats;
} else {
netdev_info(bp->dev, "error rc=%d", rc);
}
-
mutex_unlock(&bp->hwrm_cmd_lock);
+
+ if (rc)
+ rc = -EIO;
return rc;
}
struct ethtool_coalesce *ec)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
+ struct bcmgenet_rx_ring *ring;
+ unsigned int i;
ec->tx_max_coalesced_frames =
bcmgenet_tdma_ring_readl(priv, DESC_INDEX,
ec->rx_coalesce_usecs =
bcmgenet_rdma_readl(priv, DMA_RING16_TIMEOUT) * 8192 / 1000;
+ for (i = 0; i < priv->hw_params->rx_queues; i++) {
+ ring = &priv->rx_rings[i];
+ ec->use_adaptive_rx_coalesce |= ring->dim.use_dim;
+ }
+ ring = &priv->rx_rings[DESC_INDEX];
+ ec->use_adaptive_rx_coalesce |= ring->dim.use_dim;
+
return 0;
}
+static void bcmgenet_set_rx_coalesce(struct bcmgenet_rx_ring *ring)
+{
+ struct bcmgenet_priv *priv = ring->priv;
+ unsigned int i = ring->index;
+ u32 reg;
+
+ bcmgenet_rdma_ring_writel(priv, i, ring->dim.coal_pkts,
+ DMA_MBUF_DONE_THRESH);
+
+ reg = bcmgenet_rdma_readl(priv, DMA_RING0_TIMEOUT + i);
+ reg &= ~DMA_TIMEOUT_MASK;
+ reg |= DIV_ROUND_UP(ring->dim.coal_usecs * 1000, 8192);
+ bcmgenet_rdma_writel(priv, reg, DMA_RING0_TIMEOUT + i);
+}
+
static int bcmgenet_set_coalesce(struct net_device *dev,
struct ethtool_coalesce *ec)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
+ struct bcmgenet_rx_ring *ring;
unsigned int i;
- u32 reg;
/* Base system clock is 125Mhz, DMA timeout is this reference clock
* divided by 1024, which yields roughly 8.192us, our maximum value
* transmitted, or when the ring is empty.
*/
if (ec->tx_coalesce_usecs || ec->tx_coalesce_usecs_high ||
- ec->tx_coalesce_usecs_irq || ec->tx_coalesce_usecs_low)
+ ec->tx_coalesce_usecs_irq || ec->tx_coalesce_usecs_low ||
+ ec->use_adaptive_tx_coalesce)
return -EOPNOTSUPP;
/* Program all TX queues with the same values, as there is no
DMA_MBUF_DONE_THRESH);
for (i = 0; i < priv->hw_params->rx_queues; i++) {
- bcmgenet_rdma_ring_writel(priv, i,
- ec->rx_max_coalesced_frames,
- DMA_MBUF_DONE_THRESH);
-
- reg = bcmgenet_rdma_readl(priv, DMA_RING0_TIMEOUT + i);
- reg &= ~DMA_TIMEOUT_MASK;
- reg |= DIV_ROUND_UP(ec->rx_coalesce_usecs * 1000, 8192);
- bcmgenet_rdma_writel(priv, reg, DMA_RING0_TIMEOUT + i);
+ ring = &priv->rx_rings[i];
+ ring->dim.coal_usecs = ec->rx_coalesce_usecs;
+ ring->dim.coal_pkts = ec->rx_max_coalesced_frames;
+ if (!ec->use_adaptive_rx_coalesce && ring->dim.use_dim) {
+ ring->dim.coal_pkts = 1;
+ ring->dim.coal_usecs = 0;
+ }
+ ring->dim.use_dim = ec->use_adaptive_rx_coalesce;
+ bcmgenet_set_rx_coalesce(ring);
}
- bcmgenet_rdma_ring_writel(priv, DESC_INDEX,
- ec->rx_max_coalesced_frames,
- DMA_MBUF_DONE_THRESH);
-
- reg = bcmgenet_rdma_readl(priv, DMA_RING16_TIMEOUT);
- reg &= ~DMA_TIMEOUT_MASK;
- reg |= DIV_ROUND_UP(ec->rx_coalesce_usecs * 1000, 8192);
- bcmgenet_rdma_writel(priv, reg, DMA_RING16_TIMEOUT);
+ ring = &priv->rx_rings[DESC_INDEX];
+ ring->dim.coal_usecs = ec->rx_coalesce_usecs;
+ ring->dim.coal_pkts = ec->rx_max_coalesced_frames;
+ if (!ec->use_adaptive_rx_coalesce && ring->dim.use_dim) {
+ ring->dim.coal_pkts = 1;
+ ring->dim.coal_usecs = 0;
+ }
+ ring->dim.use_dim = ec->use_adaptive_rx_coalesce;
+ bcmgenet_set_rx_coalesce(ring);
return 0;
}
dma_unmap_addr_set(cb, dma_addr, 0);
}
- return 0;
+ return NULL;
}
/* Simple helper to free a receive control block's resources */
unsigned long dma_flag;
int len;
unsigned int rxpktprocessed = 0, rxpkttoprocess;
+ unsigned int bytes_processed = 0;
unsigned int p_index, mask;
unsigned int discards;
unsigned int chksum_ok = 0;
len -= ETH_FCS_LEN;
}
+ bytes_processed += len;
+
/*Finish setting up the received SKB and send it to the kernel*/
skb->protocol = eth_type_trans(skb, priv->dev);
ring->packets++;
bcmgenet_rdma_ring_writel(priv, ring->index, ring->c_index, RDMA_CONS_INDEX);
}
+ ring->dim.bytes = bytes_processed;
+ ring->dim.packets = rxpktprocessed;
+
return rxpktprocessed;
}
{
struct bcmgenet_rx_ring *ring = container_of(napi,
struct bcmgenet_rx_ring, napi);
+ struct net_dim_sample dim_sample;
unsigned int work_done;
work_done = bcmgenet_desc_rx(ring, budget);
ring->int_enable(ring);
}
+ if (ring->dim.use_dim) {
+ net_dim_sample(ring->dim.event_ctr, ring->dim.packets,
+ ring->dim.bytes, &dim_sample);
+ net_dim(&ring->dim.dim, dim_sample);
+ }
+
return work_done;
}
+static void bcmgenet_dim_work(struct work_struct *work)
+{
+ struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct bcmgenet_net_dim *ndim =
+ container_of(dim, struct bcmgenet_net_dim, dim);
+ struct bcmgenet_rx_ring *ring =
+ container_of(ndim, struct bcmgenet_rx_ring, dim);
+ struct net_dim_cq_moder cur_profile =
+ net_dim_get_profile(dim->mode, dim->profile_ix);
+
+ ring->dim.coal_usecs = cur_profile.usec;
+ ring->dim.coal_pkts = cur_profile.pkts;
+
+ bcmgenet_set_rx_coalesce(ring);
+ dim->state = NET_DIM_START_MEASURE;
+}
+
/* Assign skb to RX DMA descriptor. */
static int bcmgenet_alloc_rx_buffers(struct bcmgenet_priv *priv,
struct bcmgenet_rx_ring *ring)
dev_dbg(kdev, "done init umac\n");
}
+static void bcmgenet_init_dim(struct bcmgenet_net_dim *dim,
+ void (*cb)(struct work_struct *work))
+{
+ INIT_WORK(&dim->dim.work, cb);
+ dim->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ dim->event_ctr = 0;
+ dim->packets = 0;
+ dim->bytes = 0;
+}
+
/* Initialize a Tx ring along with corresponding hardware registers */
static void bcmgenet_init_tx_ring(struct bcmgenet_priv *priv,
unsigned int index, unsigned int size,
if (ret)
return ret;
+ bcmgenet_init_dim(&ring->dim, bcmgenet_dim_work);
+
/* Initialize Rx NAPI */
netif_napi_add(priv->dev, &ring->napi, bcmgenet_rx_poll,
NAPI_POLL_WEIGHT);
for (i = 0; i < priv->hw_params->rx_queues; ++i) {
ring = &priv->rx_rings[i];
napi_disable(&ring->napi);
+ cancel_work_sync(&ring->dim.dim.work);
}
ring = &priv->rx_rings[DESC_INDEX];
napi_disable(&ring->napi);
+ cancel_work_sync(&ring->dim.dim.work);
}
static void bcmgenet_fini_rx_napi(struct bcmgenet_priv *priv)
continue;
rx_ring = &priv->rx_rings[index];
+ rx_ring->dim.event_ctr++;
if (likely(napi_schedule_prep(&rx_ring->napi))) {
rx_ring->int_disable(rx_ring);
if (status & UMAC_IRQ_RXDMA_DONE) {
rx_ring = &priv->rx_rings[DESC_INDEX];
+ rx_ring->dim.event_ctr++;
if (likely(napi_schedule_prep(&rx_ring->napi))) {
rx_ring->int_disable(rx_ring);
#include <linux/mii.h>
#include <linux/if_vlan.h>
#include <linux/phy.h>
+#include <linux/net_dim.h>
/* total number of Buffer Descriptors, same for Rx/Tx */
#define TOTAL_DESC 256
struct bcmgenet_priv *priv;
};
+struct bcmgenet_net_dim {
+ u16 use_dim;
+ u16 event_ctr;
+ unsigned long packets;
+ unsigned long bytes;
+ u32 coal_usecs;
+ u32 coal_pkts;
+ struct net_dim dim;
+};
+
struct bcmgenet_rx_ring {
struct napi_struct napi; /* Rx NAPI struct */
unsigned long bytes;
unsigned int cb_ptr; /* Rx ring initial CB ptr */
unsigned int end_ptr; /* Rx ring end CB ptr */
unsigned int old_discards;
+ struct bcmgenet_net_dim dim;
void (*int_enable)(struct bcmgenet_rx_ring *);
void (*int_disable)(struct bcmgenet_rx_ring *);
struct bcmgenet_priv *priv;
/* 1 normal messages, 0 quiet .. 7 verbose. */
static int debug = 1;
-module_param(debug, int, S_IRUGO);
+module_param(debug, int, 0444);
MODULE_PARM_DESC(debug, "Debug messages");
#ifdef CONFIG_SBMAC_COALESCE
static int int_pktcnt_tx = 255;
-module_param(int_pktcnt_tx, int, S_IRUGO);
+module_param(int_pktcnt_tx, int, 0444);
MODULE_PARM_DESC(int_pktcnt_tx, "TX packet count");
static int int_timeout_tx = 255;
-module_param(int_timeout_tx, int, S_IRUGO);
+module_param(int_timeout_tx, int, 0444);
MODULE_PARM_DESC(int_timeout_tx, "TX timeout value");
static int int_pktcnt_rx = 64;
-module_param(int_pktcnt_rx, int, S_IRUGO);
+module_param(int_pktcnt_rx, int, 0444);
MODULE_PARM_DESC(int_pktcnt_rx, "RX packet count");
static int int_timeout_rx = 64;
-module_param(int_timeout_rx, int, S_IRUGO);
+module_param(int_timeout_rx, int, 0444);
MODULE_PARM_DESC(int_timeout_rx, "RX timeout value");
#endif
tg3_ape_unlock(tp, TG3_APE_LOCK_MEM);
- usleep_range(10, 20);
+ udelay(10);
timeout_us -= (timeout_us > 10) ? 10 : timeout_us;
}
}
-static SENSOR_DEVICE_ATTR(temp1_input, S_IRUGO, tg3_show_temp, NULL,
+static SENSOR_DEVICE_ATTR(temp1_input, 0444, tg3_show_temp, NULL,
TG3_TEMP_SENSOR_OFFSET);
-static SENSOR_DEVICE_ATTR(temp1_crit, S_IRUGO, tg3_show_temp, NULL,
+static SENSOR_DEVICE_ATTR(temp1_crit, 0444, tg3_show_temp, NULL,
TG3_TEMP_CAUTION_OFFSET);
-static SENSOR_DEVICE_ATTR(temp1_max, S_IRUGO, tg3_show_temp, NULL,
+static SENSOR_DEVICE_ATTR(temp1_max, 0444, tg3_show_temp, NULL,
TG3_TEMP_MAX_OFFSET);
static struct attribute *tg3_attrs[] = {
MODULE_PARM_DESC(bnad_ioc_auto_recover, "Enable / Disable auto recovery");
static uint bna_debugfs_enable = 1;
-module_param(bna_debugfs_enable, uint, S_IRUGO | S_IWUSR);
+module_param(bna_debugfs_enable, uint, 0644);
MODULE_PARM_DESC(bna_debugfs_enable, "Enables debugfs feature, default=1,"
" Range[false:0|true:1]");
};
static const struct bnad_debugfs_entry bnad_debugfs_files[] = {
- { "fwtrc", S_IFREG|S_IRUGO, &bnad_debugfs_op_fwtrc, },
- { "fwsave", S_IFREG|S_IRUGO, &bnad_debugfs_op_fwsave, },
- { "regrd", S_IFREG|S_IRUGO|S_IWUSR, &bnad_debugfs_op_regrd, },
- { "regwr", S_IFREG|S_IWUSR, &bnad_debugfs_op_regwr, },
- { "drvinfo", S_IFREG|S_IRUGO, &bnad_debugfs_op_drvinfo, },
+ { "fwtrc", S_IFREG | 0444, &bnad_debugfs_op_fwtrc, },
+ { "fwsave", S_IFREG | 0444, &bnad_debugfs_op_fwsave, },
+ { "regrd", S_IFREG | 0644, &bnad_debugfs_op_regrd, },
+ { "regwr", S_IFREG | 0200, &bnad_debugfs_op_regwr, },
+ { "drvinfo", S_IFREG | 0444, &bnad_debugfs_op_drvinfo, },
};
static struct dentry *bna_debugfs_root;
.remove = cavium_ptp_remove,
};
-static int __init cavium_ptp_init_module(void)
-{
- return pci_register_driver(&cavium_ptp_driver);
-}
-
-static void __exit cavium_ptp_cleanup_module(void)
-{
- pci_unregister_driver(&cavium_ptp_driver);
-}
-
-module_init(cavium_ptp_init_module);
-module_exit(cavium_ptp_cleanup_module);
+module_pci_driver(cavium_ptp_driver);
MODULE_DESCRIPTION(DRV_NAME);
MODULE_AUTHOR("Cavium Networks <support@cavium.com>");
return;
lio = GET_LIO(netdev);
- if (netif_is_multiqueue(netdev)) {
- if (__netif_subqueue_stopped(netdev, iq->q_index) &&
- lio->linfo.link.s.link_up &&
- (!octnet_iq_is_full(oct, iq_num))) {
- netif_wake_subqueue(netdev, iq->q_index);
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq_num,
- tx_restart, 1);
- }
- } else if (netif_queue_stopped(netdev) &&
- lio->linfo.link.s.link_up &&
- (!octnet_iq_is_full(oct, lio->txq))) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
+ if (__netif_subqueue_stopped(netdev, iq->q_index) &&
+ lio->linfo.link.s.link_up &&
+ (!octnet_iq_is_full(oct, iq_num))) {
+ netif_wake_subqueue(netdev, iq->q_index);
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq_num,
tx_restart, 1);
- netif_wake_queue(netdev);
}
}
*/
#define LIO_SYNC_OCTEON_TIME_INTERVAL_MS 60000
+struct lio_trusted_vf_ctx {
+ struct completion complete;
+ int status;
+};
+
struct liquidio_rx_ctl_context {
int octeon_id;
pci_unregister_driver(&liquidio_pci_driver);
}
-/**
- * \brief Stop Tx queues
- * @param netdev network device
- */
-static inline void txqs_stop(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_stop_subqueue(netdev, i);
- } else {
- netif_stop_queue(netdev);
- }
-}
-
-/**
- * \brief Start Tx queues
- * @param netdev network device
- */
-static inline void txqs_start(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_start_subqueue(netdev, i);
- } else {
- netif_start_queue(netdev);
- }
-}
-
-/**
- * \brief Wake Tx queues
- * @param netdev network device
- */
-static inline void txqs_wake(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++) {
- int qno = lio->linfo.txpciq[i %
- lio->oct_dev->num_iqs].s.q_no;
-
- if (__netif_subqueue_stopped(netdev, i)) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
- tx_restart, 1);
- netif_wake_subqueue(netdev, i);
- }
- }
- } else {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
- tx_restart, 1);
- netif_wake_queue(netdev);
- }
-}
-
-/**
- * \brief Stop Tx queue
- * @param netdev network device
- */
-static void stop_txq(struct net_device *netdev)
-{
- txqs_stop(netdev);
-}
-
-/**
- * \brief Start Tx queue
- * @param netdev network device
- */
-static void start_txq(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (lio->linfo.link.s.link_up) {
- txqs_start(netdev);
- return;
- }
-}
-
-/**
- * \brief Wake a queue
- * @param netdev network device
- * @param q which queue to wake
- */
-static inline void wake_q(struct net_device *netdev, int q)
-{
- if (netif_is_multiqueue(netdev))
- netif_wake_subqueue(netdev, q);
- else
- netif_wake_queue(netdev);
-}
-
-/**
- * \brief Stop a queue
- * @param netdev network device
- * @param q which queue to stop
- */
-static inline void stop_q(struct net_device *netdev, int q)
-{
- if (netif_is_multiqueue(netdev))
- netif_stop_subqueue(netdev, q);
- else
- netif_stop_queue(netdev);
-}
-
/**
* \brief Check Tx queue status, and take appropriate action
* @param lio per-network private data
*/
static inline int check_txq_status(struct lio *lio)
{
+ int numqs = lio->netdev->num_tx_queues;
int ret_val = 0;
+ int q, iq;
- if (netif_is_multiqueue(lio->netdev)) {
- int numqs = lio->netdev->num_tx_queues;
- int q, iq = 0;
-
- /* check each sub-queue state */
- for (q = 0; q < numqs; q++) {
- iq = lio->linfo.txpciq[q %
- lio->oct_dev->num_iqs].s.q_no;
- if (octnet_iq_is_full(lio->oct_dev, iq))
- continue;
- if (__netif_subqueue_stopped(lio->netdev, q)) {
- wake_q(lio->netdev, q);
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq,
- tx_restart, 1);
- ret_val++;
- }
+ /* check each sub-queue state */
+ for (q = 0; q < numqs; q++) {
+ iq = lio->linfo.txpciq[q %
+ lio->oct_dev->num_iqs].s.q_no;
+ if (octnet_iq_is_full(lio->oct_dev, iq))
+ continue;
+ if (__netif_subqueue_stopped(lio->netdev, q)) {
+ netif_wake_subqueue(lio->netdev, q);
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq,
+ tx_restart, 1);
+ ret_val++;
}
- } else {
- if (octnet_iq_is_full(lio->oct_dev, lio->txq))
- return 0;
- wake_q(lio->netdev, lio->txq);
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
- tx_restart, 1);
- ret_val = 1;
}
+
return ret_val;
}
if (lio->linfo.link.s.link_up) {
dev_dbg(&oct->pci_dev->dev, "%s: link_up", __func__);
netif_carrier_on(netdev);
- txqs_wake(netdev);
+ wake_txqs(netdev);
} else {
dev_dbg(&oct->pci_dev->dev, "%s: link_off", __func__);
netif_carrier_off(netdev);
- stop_txq(netdev);
+ stop_txqs(netdev);
}
if (lio->linfo.link.s.mtu != current_max_mtu) {
netif_info(lio, probe, lio->netdev, "Max MTU changed from %d to %d\n",
return 0;
}
-static inline int skb_iq(struct lio *lio, struct sk_buff *skb)
-{
- int q = 0;
-
- if (netif_is_multiqueue(lio->netdev))
- q = skb->queue_mapping % lio->linfo.num_txpciq;
-
- return q;
-}
-
-/**
- * \brief Check Tx queue state for a given network buffer
- * @param lio per-network private data
- * @param skb network buffer
- */
-static inline int check_txq_state(struct lio *lio, struct sk_buff *skb)
-{
- int q = 0, iq = 0;
-
- if (netif_is_multiqueue(lio->netdev)) {
- q = skb->queue_mapping;
- iq = lio->linfo.txpciq[(q % lio->oct_dev->num_iqs)].s.q_no;
- } else {
- iq = lio->txq;
- q = iq;
- }
-
- if (octnet_iq_is_full(lio->oct_dev, iq))
- return 0;
-
- if (__netif_subqueue_stopped(lio->netdev, q)) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq, tx_restart, 1);
- wake_q(lio->netdev, q);
- }
- return 1;
-}
-
/**
* \brief Unmap and free network buffer
* @param buf buffer
dma_unmap_single(&lio->oct_dev->pci_dev->dev, finfo->dptr, skb->len,
DMA_TO_DEVICE);
- check_txq_state(lio, skb);
-
tx_buffer_free(skb);
}
list_add_tail(&g->list, &lio->glist[iq]);
spin_unlock(&lio->glist_lock[iq]);
- check_txq_state(lio, skb); /* mq support: sub-queue state check */
-
tx_buffer_free(skb);
}
spin_unlock(&lio->glist_lock[iq]);
/* Don't free the skb yet */
-
- check_txq_state(lio, skb);
}
/**
return -1;
}
- start_txq(netdev);
+ start_txqs(netdev);
/* tell Octeon to start forwarding packets to host */
send_rx_ctrl_cmd(lio, 1);
lio = GET_LIO(netdev);
oct = lio->oct_dev;
- if (netif_is_multiqueue(netdev)) {
- q_idx = skb->queue_mapping;
- q_idx = (q_idx % (lio->linfo.num_txpciq));
- tag = q_idx;
- iq_no = lio->linfo.txpciq[q_idx].s.q_no;
- } else {
- iq_no = lio->txq;
- }
+ q_idx = skb_iq(lio, skb);
+ tag = q_idx;
+ iq_no = lio->linfo.txpciq[q_idx].s.q_no;
stats = &oct->instr_queue[iq_no]->stats;
ndata.q_no = iq_no;
- if (netif_is_multiqueue(netdev)) {
- if (octnet_iq_is_full(oct, ndata.q_no)) {
- /* defer sending if queue is full */
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- ndata.q_no);
- stats->tx_iq_busy++;
- return NETDEV_TX_BUSY;
- }
- } else {
- if (octnet_iq_is_full(oct, lio->txq)) {
- /* defer sending if queue is full */
- stats->tx_iq_busy++;
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- lio->txq);
- return NETDEV_TX_BUSY;
- }
+ if (octnet_iq_is_full(oct, ndata.q_no)) {
+ /* defer sending if queue is full */
+ netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
+ ndata.q_no);
+ stats->tx_iq_busy++;
+ return NETDEV_TX_BUSY;
}
+
/* pr_info(" XMIT - valid Qs: %d, 1st Q no: %d, cpu: %d, q_no:%d\n",
* lio->linfo.num_txpciq, lio->txq, cpu, ndata.q_no);
*/
netif_info(lio, tx_queued, lio->netdev, "Transmit queued successfully\n");
if (status == IQ_SEND_STOP)
- stop_q(netdev, q_idx);
+ netif_stop_subqueue(netdev, q_idx);
netif_trans_update(netdev);
"Transmit timeout tx_dropped:%ld, waking up queues now!!\n",
netdev->stats.tx_dropped);
netif_trans_update(netdev);
- txqs_wake(netdev);
+ wake_txqs(netdev);
}
static int liquidio_vlan_rx_add_vid(struct net_device *netdev,
ether_addr_copy(&ivi->mac[0], macaddr);
ivi->vlan = oct->sriov_info.vf_vlantci[vfidx] & VLAN_VID_MASK;
ivi->qos = oct->sriov_info.vf_vlantci[vfidx] >> VLAN_PRIO_SHIFT;
+ if (oct->sriov_info.trusted_vf.active &&
+ oct->sriov_info.trusted_vf.id == vfidx)
+ ivi->trusted = true;
+ else
+ ivi->trusted = false;
ivi->linkstate = oct->sriov_info.vf_linkstate[vfidx];
return 0;
}
+static void trusted_vf_callback(struct octeon_device *oct_dev,
+ u32 status, void *ptr)
+{
+ struct octeon_soft_command *sc = (struct octeon_soft_command *)ptr;
+ struct lio_trusted_vf_ctx *ctx;
+
+ ctx = (struct lio_trusted_vf_ctx *)sc->ctxptr;
+ ctx->status = status;
+
+ complete(&ctx->complete);
+}
+
+static int liquidio_send_vf_trust_cmd(struct lio *lio, int vfidx, bool trusted)
+{
+ struct octeon_device *oct = lio->oct_dev;
+ struct lio_trusted_vf_ctx *ctx;
+ struct octeon_soft_command *sc;
+ int ctx_size, retval;
+
+ ctx_size = sizeof(struct lio_trusted_vf_ctx);
+ sc = octeon_alloc_soft_command(oct, 0, 0, ctx_size);
+
+ ctx = (struct lio_trusted_vf_ctx *)sc->ctxptr;
+ init_completion(&ctx->complete);
+
+ sc->iq_no = lio->linfo.txpciq[0].s.q_no;
+
+ /* vfidx is 0 based, but vf_num (param1) is 1 based */
+ octeon_prepare_soft_command(oct, sc, OPCODE_NIC,
+ OPCODE_NIC_SET_TRUSTED_VF, 0, vfidx + 1,
+ trusted);
+
+ sc->callback = trusted_vf_callback;
+ sc->callback_arg = sc;
+ sc->wait_time = 1000;
+
+ retval = octeon_send_soft_command(oct, sc);
+ if (retval == IQ_SEND_FAILED) {
+ retval = -1;
+ } else {
+ /* Wait for response or timeout */
+ if (wait_for_completion_timeout(&ctx->complete,
+ msecs_to_jiffies(2000)))
+ retval = ctx->status;
+ else
+ retval = -1;
+ }
+
+ octeon_free_soft_command(oct, sc);
+
+ return retval;
+}
+
+static int liquidio_set_vf_trust(struct net_device *netdev, int vfidx,
+ bool setting)
+{
+ struct lio *lio = GET_LIO(netdev);
+ struct octeon_device *oct = lio->oct_dev;
+
+ if (strcmp(oct->fw_info.liquidio_firmware_version, "1.7.1") < 0) {
+ /* trusted vf is not supported by firmware older than 1.7.1 */
+ return -EOPNOTSUPP;
+ }
+
+ if (vfidx < 0 || vfidx >= oct->sriov_info.num_vfs_alloced) {
+ netif_info(lio, drv, lio->netdev, "Invalid vfidx %d\n", vfidx);
+ return -EINVAL;
+ }
+
+ if (setting) {
+ /* Set */
+
+ if (oct->sriov_info.trusted_vf.active &&
+ oct->sriov_info.trusted_vf.id == vfidx)
+ return 0;
+
+ if (oct->sriov_info.trusted_vf.active) {
+ netif_info(lio, drv, lio->netdev, "More than one trusted VF is not allowed\n");
+ return -EPERM;
+ }
+ } else {
+ /* Clear */
+
+ if (!oct->sriov_info.trusted_vf.active)
+ return 0;
+ }
+
+ if (!liquidio_send_vf_trust_cmd(lio, vfidx, setting)) {
+ if (setting) {
+ oct->sriov_info.trusted_vf.id = vfidx;
+ oct->sriov_info.trusted_vf.active = true;
+ } else {
+ oct->sriov_info.trusted_vf.active = false;
+ }
+
+ netif_info(lio, drv, lio->netdev, "VF %u is %strusted\n", vfidx,
+ setting ? "" : "not ");
+ } else {
+ netif_info(lio, drv, lio->netdev, "Failed to set VF trusted\n");
+ return -1;
+ }
+
+ return 0;
+}
+
static int liquidio_set_vf_link_state(struct net_device *netdev, int vfidx,
int linkstate)
{
.ndo_set_vf_mac = liquidio_set_vf_mac,
.ndo_set_vf_vlan = liquidio_set_vf_vlan,
.ndo_get_vf_config = liquidio_get_vf_config,
+ .ndo_set_vf_trust = liquidio_set_vf_trust,
.ndo_set_vf_link_state = liquidio_set_vf_link_state,
};
.err_handler = &liquidio_vf_err_handler, /* For AER */
};
-/**
- * \brief Stop Tx queues
- * @param netdev network device
- */
-static void txqs_stop(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_stop_subqueue(netdev, i);
- } else {
- netif_stop_queue(netdev);
- }
-}
-
-/**
- * \brief Start Tx queues
- * @param netdev network device
- */
-static void txqs_start(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_start_subqueue(netdev, i);
- } else {
- netif_start_queue(netdev);
- }
-}
-
-/**
- * \brief Wake Tx queues
- * @param netdev network device
- */
-static void txqs_wake(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++) {
- int qno = lio->linfo.txpciq[i % lio->oct_dev->num_iqs]
- .s.q_no;
- if (__netif_subqueue_stopped(netdev, i)) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
- tx_restart, 1);
- netif_wake_subqueue(netdev, i);
- }
- }
- } else {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
- tx_restart, 1);
- netif_wake_queue(netdev);
- }
-}
-
-/**
- * \brief Start Tx queue
- * @param netdev network device
- */
-static void start_txq(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (lio->linfo.link.s.link_up) {
- txqs_start(netdev);
- return;
- }
-}
-
-/**
- * \brief Wake a queue
- * @param netdev network device
- * @param q which queue to wake
- */
-static void wake_q(struct net_device *netdev, int q)
-{
- if (netif_is_multiqueue(netdev))
- netif_wake_subqueue(netdev, q);
- else
- netif_wake_queue(netdev);
-}
-
-/**
- * \brief Stop a queue
- * @param netdev network device
- * @param q which queue to stop
- */
-static void stop_q(struct net_device *netdev, int q)
-{
- if (netif_is_multiqueue(netdev))
- netif_stop_subqueue(netdev, q);
- else
- netif_stop_queue(netdev);
-}
-
/**
* Remove the node at the head of the list. The list would be empty at
* the end of this call if there are no more nodes in the list.
if (lio->linfo.link.s.link_up) {
netif_carrier_on(netdev);
- txqs_wake(netdev);
+ wake_txqs(netdev);
} else {
netif_carrier_off(netdev);
- txqs_stop(netdev);
+ stop_txqs(netdev);
}
if (lio->linfo.link.s.mtu != current_max_mtu) {
return 0;
}
-static int skb_iq(struct lio *lio, struct sk_buff *skb)
-{
- int q = 0;
-
- if (netif_is_multiqueue(lio->netdev))
- q = skb->queue_mapping % lio->linfo.num_txpciq;
-
- return q;
-}
-
-/**
- * \brief Check Tx queue state for a given network buffer
- * @param lio per-network private data
- * @param skb network buffer
- */
-static int check_txq_state(struct lio *lio, struct sk_buff *skb)
-{
- int q = 0, iq = 0;
-
- if (netif_is_multiqueue(lio->netdev)) {
- q = skb->queue_mapping;
- iq = lio->linfo.txpciq[q % lio->oct_dev->num_iqs].s.q_no;
- } else {
- iq = lio->txq;
- q = iq;
- }
-
- if (octnet_iq_is_full(lio->oct_dev, iq))
- return 0;
-
- if (__netif_subqueue_stopped(lio->netdev, q)) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq, tx_restart, 1);
- wake_q(lio->netdev, q);
- }
-
- return 1;
-}
-
/**
* \brief Unmap and free network buffer
* @param buf buffer
dma_unmap_single(&lio->oct_dev->pci_dev->dev, finfo->dptr, skb->len,
DMA_TO_DEVICE);
- check_txq_state(lio, skb);
-
tx_buffer_free(skb);
}
list_add_tail(&g->list, &lio->glist[iq]);
spin_unlock(&lio->glist_lock[iq]);
- check_txq_state(lio, skb); /* mq support: sub-queue state check */
-
tx_buffer_free(skb);
}
spin_unlock(&lio->glist_lock[iq]);
/* Don't free the skb yet */
-
- check_txq_state(lio, skb);
}
/**
lio->intf_open = 1;
netif_info(lio, ifup, lio->netdev, "Interface Open, ready for traffic\n");
- start_txq(netdev);
+ start_txqs(netdev);
/* tell Octeon to start forwarding packets to host */
send_rx_ctrl_cmd(lio, 1);
ifstate_reset(lio, LIO_IFSTATE_RUNNING);
- txqs_stop(netdev);
+ stop_txqs(netdev);
dev_info(&oct->pci_dev->dev, "%s interface is stopped\n", netdev->name);
lio = GET_LIO(netdev);
oct = lio->oct_dev;
- if (netif_is_multiqueue(netdev)) {
- q_idx = skb->queue_mapping;
- q_idx = (q_idx % (lio->linfo.num_txpciq));
- tag = q_idx;
- iq_no = lio->linfo.txpciq[q_idx].s.q_no;
- } else {
- iq_no = lio->txq;
- }
+ q_idx = skb_iq(lio, skb);
+ tag = q_idx;
+ iq_no = lio->linfo.txpciq[q_idx].s.q_no;
stats = &oct->instr_queue[iq_no]->stats;
ndata.q_no = iq_no;
- if (netif_is_multiqueue(netdev)) {
- if (octnet_iq_is_full(oct, ndata.q_no)) {
- /* defer sending if queue is full */
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- ndata.q_no);
- stats->tx_iq_busy++;
- return NETDEV_TX_BUSY;
- }
- } else {
- if (octnet_iq_is_full(oct, lio->txq)) {
- /* defer sending if queue is full */
- stats->tx_iq_busy++;
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- ndata.q_no);
- return NETDEV_TX_BUSY;
- }
+ if (octnet_iq_is_full(oct, ndata.q_no)) {
+ /* defer sending if queue is full */
+ netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
+ ndata.q_no);
+ stats->tx_iq_busy++;
+ return NETDEV_TX_BUSY;
}
ndata.datasize = skb->len;
if (status == IQ_SEND_STOP) {
dev_err(&oct->pci_dev->dev, "Rcvd IQ_SEND_STOP signal; stopping IQ-%d\n",
iq_no);
- stop_q(netdev, q_idx);
+ netif_stop_subqueue(netdev, q_idx);
}
netif_trans_update(netdev);
"Transmit timeout tx_dropped:%ld, waking up queues now!!\n",
netdev->stats.tx_dropped);
netif_trans_update(netdev);
- txqs_wake(netdev);
+ wake_txqs(netdev);
}
static int
#define OPCODE_NIC_IF_CFG 0x09
#define OPCODE_NIC_VF_DRV_NOTICE 0x0A
#define OPCODE_NIC_INTRMOD_PARAMS 0x0B
+#define OPCODE_NIC_SET_TRUSTED_VF 0x13
#define OPCODE_NIC_SYNC_OCTEON_TIME 0x14
#define VF_DRV_LOADED 1
#define VF_DRV_REMOVED -1
u64 pkind:6;
u64 use_qpg:1;
u64 qpg:11;
- u64 reserved:30;
+ u64 reserved0:10;
+ u64 ctrl_qpg:11;
+ u64 reserved:9;
#else
- u64 reserved:30;
+ u64 reserved:9;
+ u64 ctrl_qpg:11;
+ u64 reserved0:10;
u64 qpg:11;
u64 use_qpg:1;
u64 pkind:6;
} s;
};
+struct lio_trusted_vf {
+ uint64_t active: 1;
+ uint64_t id : 8;
+ uint64_t reserved: 55;
+};
+
struct lio_time {
s64 sec; /* seconds */
s64 nsec; /* nanoseconds */
u32 sriov_enabled;
+ struct lio_trusted_vf trusted_vf;
+
/*lookup table that maps DPI ring number to VF pci_dev struct pointer*/
struct pci_dev *dpiring_to_vfpcidev_lut[MAX_POSSIBLE_VFS];
return 0;
}
+/**
+ * \brief Stop Tx queues
+ * @param netdev network device
+ */
+static inline void stop_txqs(struct net_device *netdev)
+{
+ int i;
+
+ for (i = 0; i < netdev->num_tx_queues; i++)
+ netif_stop_subqueue(netdev, i);
+}
+
+/**
+ * \brief Wake Tx queues
+ * @param netdev network device
+ */
+static inline void wake_txqs(struct net_device *netdev)
+{
+ struct lio *lio = GET_LIO(netdev);
+ int i, qno;
+
+ for (i = 0; i < netdev->num_tx_queues; i++) {
+ qno = lio->linfo.txpciq[i % lio->oct_dev->num_iqs].s.q_no;
+
+ if (__netif_subqueue_stopped(netdev, i)) {
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
+ tx_restart, 1);
+ netif_wake_subqueue(netdev, i);
+ }
+ }
+}
+
+/**
+ * \brief Start Tx queues
+ * @param netdev network device
+ */
+static inline void start_txqs(struct net_device *netdev)
+{
+ struct lio *lio = GET_LIO(netdev);
+ int i;
+
+ if (lio->linfo.link.s.link_up) {
+ for (i = 0; i < netdev->num_tx_queues; i++)
+ netif_start_subqueue(netdev, i);
+ }
+}
+
+static inline int skb_iq(struct lio *lio, struct sk_buff *skb)
+{
+ return skb->queue_mapping % lio->linfo.num_txpciq;
+}
+
#endif
pki_ih3->tag = LIO_CONTROL;
pki_ih3->tagtype = ATOMIC_TAG;
pki_ih3->qpg =
- oct->instr_queue[sc->iq_no]->txpciq.s.qpg;
+ oct->instr_queue[sc->iq_no]->txpciq.s.ctrl_qpg;
+
pki_ih3->pm = 0x7;
pki_ih3->sl = 8;
MODULE_PARM_DESC(debug, "Debug message level bitmap");
static int cpi_alg = CPI_ALG_NONE;
-module_param(cpi_alg, int, S_IRUGO);
+module_param(cpi_alg, int, 0444);
MODULE_PARM_DESC(cpi_alg,
"PFC algorithm (0=none, 1=VLAN, 2=VLAN16, 3=IP Diffserv)");
#define CXGB3_ATTR_R(name, val_expr) \
CXGB3_SHOW(name, val_expr) \
-static DEVICE_ATTR(name, S_IRUGO, show_##name, NULL)
+static DEVICE_ATTR(name, 0444, show_##name, NULL)
#define CXGB3_ATTR_RW(name, val_expr, store_method) \
CXGB3_SHOW(name, val_expr) \
-static DEVICE_ATTR(name, S_IRUGO | S_IWUSR, show_##name, store_method)
+static DEVICE_ATTR(name, 0644, show_##name, store_method)
CXGB3_ATTR_R(cam_size, t3_mc5_size(&adap->mc5));
CXGB3_ATTR_RW(nfilters, adap->params.mc5.nfilters, store_nfilters);
{ \
return tm_attr_store(d, buf, len, sched); \
} \
-static DEVICE_ATTR(name, S_IRUGO | S_IWUSR, show_##name, store_##name)
+static DEVICE_ATTR(name, 0644, show_##name, store_##name)
TM_ATTR(sched0, 0);
TM_ATTR(sched1, 1);
obj-$(CONFIG_CHELSIO_T4) += cxgb4.o
cxgb4-objs := cxgb4_main.o l2t.o smt.o t4_hw.o sge.o clip_tbl.o cxgb4_ethtool.o \
- cxgb4_uld.o sched.o cxgb4_filter.o cxgb4_tc_u32.o \
+ cxgb4_uld.o srq.o sched.o cxgb4_filter.o cxgb4_tc_u32.o \
cxgb4_ptp.o cxgb4_tc_flower.o cxgb4_cudbg.o \
cudbg_common.o cudbg_lib.o cudbg_zlib.o
cxgb4-$(CONFIG_CHELSIO_T4_DCB) += cxgb4_dcb.o
* used by the Port
*/
u8 mps_bg_map[MAX_NPORTS]; /* MPS Buffer Group Map */
+ bool write_w_imm_support; /* FW supports WRITE_WITH_IMMEDIATE */
+ bool write_cmpl_support; /* FW supports WRITE_CMPL */
};
/* State needed to monitor the forward progress of SGE Ingress DMA activities
/* HMA */
struct hma_data hma;
+
+ struct srq_data *srq;
};
/* Support for "sched-class" command to allow a TX Scheduling Class to be
static void add_debugfs_mem(struct adapter *adap, const char *name,
unsigned int idx, unsigned int size_mb)
{
- debugfs_create_file_size(name, S_IRUSR, adap->debugfs_root,
+ debugfs_create_file_size(name, 0400, adap->debugfs_root,
(void *)adap + idx, &mem_debugfs_fops,
size_mb << 20);
}
struct dentry *de;
static struct t4_debugfs_entry t4_debugfs_files[] = {
- { "cim_la", &cim_la_fops, S_IRUSR, 0 },
- { "cim_pif_la", &cim_pif_la_fops, S_IRUSR, 0 },
- { "cim_ma_la", &cim_ma_la_fops, S_IRUSR, 0 },
- { "cim_qcfg", &cim_qcfg_fops, S_IRUSR, 0 },
- { "clk", &clk_debugfs_fops, S_IRUSR, 0 },
- { "devlog", &devlog_fops, S_IRUSR, 0 },
- { "mboxlog", &mboxlog_fops, S_IRUSR, 0 },
- { "mbox0", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 0 },
- { "mbox1", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 1 },
- { "mbox2", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 2 },
- { "mbox3", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 3 },
- { "mbox4", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 4 },
- { "mbox5", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 5 },
- { "mbox6", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 6 },
- { "mbox7", &mbox_debugfs_fops, S_IRUSR | S_IWUSR, 7 },
- { "trace0", &mps_trc_debugfs_fops, S_IRUSR | S_IWUSR, 0 },
- { "trace1", &mps_trc_debugfs_fops, S_IRUSR | S_IWUSR, 1 },
- { "trace2", &mps_trc_debugfs_fops, S_IRUSR | S_IWUSR, 2 },
- { "trace3", &mps_trc_debugfs_fops, S_IRUSR | S_IWUSR, 3 },
- { "l2t", &t4_l2t_fops, S_IRUSR, 0},
- { "mps_tcam", &mps_tcam_debugfs_fops, S_IRUSR, 0 },
- { "rss", &rss_debugfs_fops, S_IRUSR, 0 },
- { "rss_config", &rss_config_debugfs_fops, S_IRUSR, 0 },
- { "rss_key", &rss_key_debugfs_fops, S_IRUSR, 0 },
- { "rss_pf_config", &rss_pf_config_debugfs_fops, S_IRUSR, 0 },
- { "rss_vf_config", &rss_vf_config_debugfs_fops, S_IRUSR, 0 },
- { "sge_qinfo", &sge_qinfo_debugfs_fops, S_IRUSR, 0 },
- { "ibq_tp0", &cim_ibq_fops, S_IRUSR, 0 },
- { "ibq_tp1", &cim_ibq_fops, S_IRUSR, 1 },
- { "ibq_ulp", &cim_ibq_fops, S_IRUSR, 2 },
- { "ibq_sge0", &cim_ibq_fops, S_IRUSR, 3 },
- { "ibq_sge1", &cim_ibq_fops, S_IRUSR, 4 },
- { "ibq_ncsi", &cim_ibq_fops, S_IRUSR, 5 },
- { "obq_ulp0", &cim_obq_fops, S_IRUSR, 0 },
- { "obq_ulp1", &cim_obq_fops, S_IRUSR, 1 },
- { "obq_ulp2", &cim_obq_fops, S_IRUSR, 2 },
- { "obq_ulp3", &cim_obq_fops, S_IRUSR, 3 },
- { "obq_sge", &cim_obq_fops, S_IRUSR, 4 },
- { "obq_ncsi", &cim_obq_fops, S_IRUSR, 5 },
- { "tp_la", &tp_la_fops, S_IRUSR, 0 },
- { "ulprx_la", &ulprx_la_fops, S_IRUSR, 0 },
- { "sensors", &sensors_debugfs_fops, S_IRUSR, 0 },
- { "pm_stats", &pm_stats_debugfs_fops, S_IRUSR, 0 },
- { "tx_rate", &tx_rate_debugfs_fops, S_IRUSR, 0 },
- { "cctrl", &cctrl_tbl_debugfs_fops, S_IRUSR, 0 },
+ { "cim_la", &cim_la_fops, 0400, 0 },
+ { "cim_pif_la", &cim_pif_la_fops, 0400, 0 },
+ { "cim_ma_la", &cim_ma_la_fops, 0400, 0 },
+ { "cim_qcfg", &cim_qcfg_fops, 0400, 0 },
+ { "clk", &clk_debugfs_fops, 0400, 0 },
+ { "devlog", &devlog_fops, 0400, 0 },
+ { "mboxlog", &mboxlog_fops, 0400, 0 },
+ { "mbox0", &mbox_debugfs_fops, 0600, 0 },
+ { "mbox1", &mbox_debugfs_fops, 0600, 1 },
+ { "mbox2", &mbox_debugfs_fops, 0600, 2 },
+ { "mbox3", &mbox_debugfs_fops, 0600, 3 },
+ { "mbox4", &mbox_debugfs_fops, 0600, 4 },
+ { "mbox5", &mbox_debugfs_fops, 0600, 5 },
+ { "mbox6", &mbox_debugfs_fops, 0600, 6 },
+ { "mbox7", &mbox_debugfs_fops, 0600, 7 },
+ { "trace0", &mps_trc_debugfs_fops, 0600, 0 },
+ { "trace1", &mps_trc_debugfs_fops, 0600, 1 },
+ { "trace2", &mps_trc_debugfs_fops, 0600, 2 },
+ { "trace3", &mps_trc_debugfs_fops, 0600, 3 },
+ { "l2t", &t4_l2t_fops, 0400, 0},
+ { "mps_tcam", &mps_tcam_debugfs_fops, 0400, 0 },
+ { "rss", &rss_debugfs_fops, 0400, 0 },
+ { "rss_config", &rss_config_debugfs_fops, 0400, 0 },
+ { "rss_key", &rss_key_debugfs_fops, 0400, 0 },
+ { "rss_pf_config", &rss_pf_config_debugfs_fops, 0400, 0 },
+ { "rss_vf_config", &rss_vf_config_debugfs_fops, 0400, 0 },
+ { "sge_qinfo", &sge_qinfo_debugfs_fops, 0400, 0 },
+ { "ibq_tp0", &cim_ibq_fops, 0400, 0 },
+ { "ibq_tp1", &cim_ibq_fops, 0400, 1 },
+ { "ibq_ulp", &cim_ibq_fops, 0400, 2 },
+ { "ibq_sge0", &cim_ibq_fops, 0400, 3 },
+ { "ibq_sge1", &cim_ibq_fops, 0400, 4 },
+ { "ibq_ncsi", &cim_ibq_fops, 0400, 5 },
+ { "obq_ulp0", &cim_obq_fops, 0400, 0 },
+ { "obq_ulp1", &cim_obq_fops, 0400, 1 },
+ { "obq_ulp2", &cim_obq_fops, 0400, 2 },
+ { "obq_ulp3", &cim_obq_fops, 0400, 3 },
+ { "obq_sge", &cim_obq_fops, 0400, 4 },
+ { "obq_ncsi", &cim_obq_fops, 0400, 5 },
+ { "tp_la", &tp_la_fops, 0400, 0 },
+ { "ulprx_la", &ulprx_la_fops, 0400, 0 },
+ { "sensors", &sensors_debugfs_fops, 0400, 0 },
+ { "pm_stats", &pm_stats_debugfs_fops, 0400, 0 },
+ { "tx_rate", &tx_rate_debugfs_fops, 0400, 0 },
+ { "cctrl", &cctrl_tbl_debugfs_fops, 0400, 0 },
#if IS_ENABLED(CONFIG_IPV6)
- { "clip_tbl", &clip_tbl_debugfs_fops, S_IRUSR, 0 },
+ { "clip_tbl", &clip_tbl_debugfs_fops, 0400, 0 },
#endif
- { "tids", &tid_info_debugfs_fops, S_IRUSR, 0},
- { "blocked_fl", &blocked_fl_fops, S_IRUSR | S_IWUSR, 0 },
- { "meminfo", &meminfo_fops, S_IRUSR, 0 },
- { "crypto", &chcr_stats_debugfs_fops, S_IRUSR, 0 },
+ { "tids", &tid_info_debugfs_fops, 0400, 0},
+ { "blocked_fl", &blocked_fl_fops, 0600, 0 },
+ { "meminfo", &meminfo_fops, 0400, 0 },
+ { "crypto", &chcr_stats_debugfs_fops, 0400, 0 },
};
/* Debug FS nodes common to all T5 and later adapters.
*/
static struct t4_debugfs_entry t5_debugfs_files[] = {
- { "obq_sge_rx_q0", &cim_obq_fops, S_IRUSR, 6 },
- { "obq_sge_rx_q1", &cim_obq_fops, S_IRUSR, 7 },
+ { "obq_sge_rx_q0", &cim_obq_fops, 0400, 6 },
+ { "obq_sge_rx_q1", &cim_obq_fops, 0400, 7 },
};
add_debugfs_files(adap,
}
}
- de = debugfs_create_file_size("flash", S_IRUSR, adap->debugfs_root, adap,
+ de = debugfs_create_file_size("flash", 0400, adap->debugfs_root, adap,
&flash_debugfs_fops, adap->params.sf_size);
- debugfs_create_bool("use_backdoor", S_IWUSR | S_IRUSR,
+ debugfs_create_bool("use_backdoor", 0600,
adap->debugfs_root, &adap->use_bd);
- debugfs_create_bool("trace_rss", S_IWUSR | S_IRUSR,
+ debugfs_create_bool("trace_rss", 0600,
adap->debugfs_root, &adap->trace_rss);
return 0;
#include "t4fw_api.h"
#include "t4fw_version.h"
#include "cxgb4_dcb.h"
+#include "srq.h"
#include "cxgb4_debugfs.h"
#include "clip_tbl.h"
#include "l2t.h"
const struct cpl_abort_rpl_rss *p = (void *)rsp;
hash_del_filter_rpl(q->adap, p);
+ } else if (opcode == CPL_SRQ_TABLE_RPL) {
+ const struct cpl_srq_table_rpl *p = (void *)rsp;
+
+ do_srq_table_rpl(q->adap, p);
} else
dev_err(q->adap->pdev_dev,
"unexpected CPL %#x on FW event queue\n", opcode);
err = t4_sge_alloc_rxq(adap, &s->fw_evtq, true, adap->port[0],
adap->msi_idx, NULL, fwevtq_handler, NULL, -1);
- if (err)
- t4_free_sge_resources(adap);
return err;
}
{
struct port_info *pi = netdev_priv(dev);
struct adapter *adap = pi->adapter;
+ struct vf_info *vfinfo;
if (vf >= adap->num_vfs)
return -EINVAL;
+ vfinfo = &adap->vfinfo[vf];
+
ivi->vf = vf;
- ivi->max_tx_rate = adap->vfinfo[vf].tx_rate;
+ ivi->max_tx_rate = vfinfo->tx_rate;
ivi->min_tx_rate = 0;
- ether_addr_copy(ivi->mac, adap->vfinfo[vf].vf_mac_addr);
+ ether_addr_copy(ivi->mac, vfinfo->vf_mac_addr);
+ ivi->vlan = vfinfo->vlan;
return 0;
}
adap->vres.pbl.start = val[4];
adap->vres.pbl.size = val[5] - val[4] + 1;
+ params[0] = FW_PARAM_PFVF(SRQ_START);
+ params[1] = FW_PARAM_PFVF(SRQ_END);
+ ret = t4_query_params(adap, adap->mbox, adap->pf, 0, 2,
+ params, val);
+ if (!ret) {
+ adap->vres.srq.start = val[0];
+ adap->vres.srq.size = val[1] - val[0] + 1;
+ }
+ if (adap->vres.srq.size) {
+ adap->srq = t4_init_srq(adap->vres.srq.size);
+ if (!adap->srq)
+ dev_warn(&adap->pdev->dev, "could not allocate SRQ, continuing\n");
+ }
+
params[0] = FW_PARAM_PFVF(SQRQ_START);
params[1] = FW_PARAM_PFVF(SQRQ_END);
params[2] = FW_PARAM_PFVF(CQ_START);
"max_ordird_qp %d max_ird_adapter %d\n",
adap->params.max_ordird_qp,
adap->params.max_ird_adapter);
+
+ /* Enable write_with_immediate if FW supports it */
+ params[0] = FW_PARAM_DEV(RDMA_WRITE_WITH_IMM);
+ ret = t4_query_params(adap, adap->mbox, adap->pf, 0, 1, params,
+ val);
+ adap->params.write_w_imm_support = (ret == 0 && val[0] != 0);
+
+ /* Enable write_cmpl if FW supports it */
+ params[0] = FW_PARAM_DEV(RI_WRITE_CMPL_WR);
+ ret = t4_query_params(adap, adap->mbox, adap->pf, 0, 1, params,
+ val);
+ adap->params.write_cmpl_support = (ret == 0 && val[0] != 0);
adap->num_ofld_uld += 2;
}
if (caps_cmd.iscsicaps) {
kvfree(adapter->smt);
kvfree(adapter->l2t);
+ kvfree(adapter->srq);
t4_cleanup_sched(adapter);
kvfree(adapter->tids.tid_tab);
cxgb4_cleanup_tc_flower(adapter);
/* Initialize the device structure. */
dev->netdev_ops = &cxgb4_mgmt_netdev_ops;
dev->ethtool_ops = &cxgb4_mgmt_ethtool_ops;
- dev->needs_free_netdev = true;
}
static int cxgb4_iov_configure(struct pci_dev *pdev, int num_vfs)
adapter->name = pci_name(pdev);
adapter->mbox = func;
adapter->pf = func;
+ adapter->params.chip = chip;
+ adapter->adap_idx = adap_idx;
adapter->msg_enable = DFLT_MSG_ENABLE;
adapter->mbox_log = kzalloc(sizeof(*adapter->mbox_log) +
(sizeof(struct mbox_cmd) *
if (err)
goto out_free_dev;
+ err = setup_fw_sge_queues(adapter);
+ if (err) {
+ dev_err(adapter->pdev_dev,
+ "FW sge queue allocation failed, err %d", err);
+ goto out_free_dev;
+ }
+
/*
* The card is now ready to go. If any errors occur during device
* registration we do not fail the whole card but rather proceed only
cxgb4_ptp_init(adapter);
print_adapter_info(adapter);
- setup_fw_sge_queues(adapter);
return 0;
out_free_dev:
+ t4_free_sge_resources(adapter);
free_some_resources(adapter);
if (adapter->flags & USING_MSIX)
free_msix_info(adapter);
lld->ulptx_memwrite_dsgl = adap->params.ulptx_memwrite_dsgl;
lld->nodeid = dev_to_node(adap->pdev_dev);
lld->fr_nsmr_tpte_wr_support = adap->params.fr_nsmr_tpte_wr_support;
+ lld->write_w_imm_support = adap->params.write_w_imm_support;
+ lld->write_cmpl_support = adap->params.write_cmpl_support;
}
static void uld_attach(struct adapter *adap, unsigned int uld)
struct cxgb4_range iscsi;
struct cxgb4_range stag;
struct cxgb4_range rq;
+ struct cxgb4_range srq;
struct cxgb4_range pbl;
struct cxgb4_range qp;
struct cxgb4_range cq;
void **iscsi_ppm; /* iscsi page pod manager */
int nodeid; /* device numa node id */
bool fr_nsmr_tpte_wr_support; /* FW supports FR_NSMR_TPTE_WR */
+ bool write_w_imm_support; /* FW supports WRITE_WITH_IMMEDIATE */
+ bool write_cmpl_support; /* FW supports WRITE_CMPL WR */
};
struct cxgb4_uld_info {
--- /dev/null
+/*
+ * This file is part of the Chelsio T6 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2017-2018 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include "cxgb4.h"
+#include "t4_msg.h"
+#include "srq.h"
+
+struct srq_data *t4_init_srq(int srq_size)
+{
+ struct srq_data *s;
+
+ s = kvzalloc(sizeof(*s), GFP_KERNEL);
+ if (!s)
+ return NULL;
+
+ s->srq_size = srq_size;
+ init_completion(&s->comp);
+ mutex_init(&s->lock);
+
+ return s;
+}
+
+/* cxgb4_get_srq_entry: read the SRQ table entry
+ * @dev: Pointer to the net_device
+ * @idx: Index to the srq
+ * @entryp: pointer to the srq entry
+ *
+ * Sends CPL_SRQ_TABLE_REQ message for the given index.
+ * Contents will be returned in CPL_SRQ_TABLE_RPL message.
+ *
+ * Returns zero if the read is successful, else a error
+ * number will be returned. Caller should not use the srq
+ * entry if the return value is non-zero.
+ *
+ *
+ */
+int cxgb4_get_srq_entry(struct net_device *dev,
+ int srq_idx, struct srq_entry *entryp)
+{
+ struct cpl_srq_table_req *req;
+ struct adapter *adap;
+ struct sk_buff *skb;
+ struct srq_data *s;
+ int rc = -ENODEV;
+
+ adap = netdev2adap(dev);
+ s = adap->srq;
+
+ if (!(adap->flags & FULL_INIT_DONE) || !s)
+ goto out;
+
+ skb = alloc_skb(sizeof(*req), GFP_KERNEL);
+ if (!skb)
+ return -ENOMEM;
+ req = (struct cpl_srq_table_req *)
+ __skb_put(skb, sizeof(*req));
+ memset(req, 0, sizeof(*req));
+ INIT_TP_WR(req, 0);
+ OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_SRQ_TABLE_REQ,
+ TID_TID_V(srq_idx) |
+ TID_QID_V(adap->sge.fw_evtq.abs_id)));
+ req->idx = srq_idx;
+
+ mutex_lock(&s->lock);
+
+ s->entryp = entryp;
+ t4_mgmt_tx(adap, skb);
+
+ rc = wait_for_completion_timeout(&s->comp, SRQ_WAIT_TO);
+ if (rc)
+ rc = 0;
+ else /* !rc means we timed out */
+ rc = -ETIMEDOUT;
+
+ WARN_ON_ONCE(entryp->idx != srq_idx);
+ mutex_unlock(&s->lock);
+out:
+ return rc;
+}
+EXPORT_SYMBOL(cxgb4_get_srq_entry);
+
+void do_srq_table_rpl(struct adapter *adap,
+ const struct cpl_srq_table_rpl *rpl)
+{
+ unsigned int idx = TID_TID_G(GET_TID(rpl));
+ struct srq_data *s = adap->srq;
+ struct srq_entry *e;
+
+ if (unlikely(rpl->status != CPL_CONTAINS_READ_RPL)) {
+ dev_err(adap->pdev_dev,
+ "Unexpected SRQ_TABLE_RPL status %u for entry %u\n",
+ rpl->status, idx);
+ goto out;
+ }
+
+ /* Store the read entry */
+ e = s->entryp;
+ e->valid = 1;
+ e->idx = idx;
+ e->pdid = SRQT_PDID_G(be64_to_cpu(rpl->rsvd_pdid));
+ e->qlen = SRQT_QLEN_G(be32_to_cpu(rpl->qlen_qbase));
+ e->qbase = SRQT_QBASE_G(be32_to_cpu(rpl->qlen_qbase));
+ e->cur_msn = be16_to_cpu(rpl->cur_msn);
+ e->max_msn = be16_to_cpu(rpl->max_msn);
+out:
+ complete(&s->comp);
+}
--- /dev/null
+/*
+ * This file is part of the Chelsio T6 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2017-2018 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __CXGB4_SRQ_H
+#define __CXGB4_SRQ_H
+
+struct adapter;
+struct cpl_srq_table_rpl;
+
+#define SRQ_WAIT_TO (HZ * 5)
+
+struct srq_entry {
+ u8 valid;
+ u8 idx;
+ u8 qlen;
+ u16 pdid;
+ u16 cur_msn;
+ u16 max_msn;
+ u32 qbase;
+};
+
+struct srq_data {
+ unsigned int srq_size;
+ struct srq_entry *entryp;
+ struct completion comp;
+ struct mutex lock; /* generic mutex for srq data */
+};
+
+struct srq_data *t4_init_srq(int srq_size);
+int cxgb4_get_srq_entry(struct net_device *dev,
+ int srq_idx, struct srq_entry *entryp);
+void do_srq_table_rpl(struct adapter *adap,
+ const struct cpl_srq_table_rpl *rpl);
+#endif /* __CXGB4_SRQ_H */
unsigned int fw_mdi = FW_PORT_CAP32_MDI_V(FW_PORT_CAP32_MDI_AUTO);
fw_port_cap32_t fw_fc, cc_fec, fw_fec, rcap;
- lc->link_ok = 0;
-
/* Convert driver coding of Pause Frame Flow Control settings into the
* Firmware's API.
*/
}
break;
}
+ case 0x9d: { /* ISSI -- Integrated Silicon Solution, Inc. */
+ /* This Density -> Size decoding table is taken from ISSI
+ * Data Sheets.
+ */
+ density = (flashid >> 16) & 0xff;
+ switch (density) {
+ case 0x16: /* 32 MB */
+ size = 1 << 25;
+ break;
+ case 0x17: /* 64MB */
+ size = 1 << 26;
+ break;
+ default:
+ dev_err(adap->pdev_dev, "ISSI Flash Part has bad size, ID = %#x, Density code = %#x\n",
+ flashid, density);
+ return -EINVAL;
+ }
+ break;
+ }
case 0xc2: { /* Macronix */
/* This Density -> Size decoding table is taken from Macronix
* Data Sheets.
CPL_L2T_WRITE_REQ = 0x12,
CPL_SMT_WRITE_REQ = 0x14,
CPL_TID_RELEASE = 0x1A,
+ CPL_SRQ_TABLE_REQ = 0x1C,
CPL_TX_DATA_ISO = 0x1F,
CPL_CLOSE_LISTSRV_RPL = 0x20,
CPL_FW4_MSG = 0xC0,
CPL_FW4_PLD = 0xC1,
CPL_FW4_ACK = 0xC3,
+ CPL_SRQ_TABLE_RPL = 0xCC,
CPL_RX_PHYS_DSGL = 0xD0,
CPL_ERR_KEEPALV_NEG_ADVICE = 37,
CPL_ERR_ABORT_FAILED = 42,
CPL_ERR_IWARP_FLM = 50,
+ CPL_CONTAINS_READ_RPL = 60,
+ CPL_CONTAINS_WRITE_RPL = 61,
};
enum {
/* partitioning of TID fields that also carry a queue id */
#define TID_TID_S 0
#define TID_TID_M 0x3fff
+#define TID_TID_V(x) ((x) << TID_TID_S)
#define TID_TID_G(x) (((x) >> TID_TID_S) & TID_TID_M)
#define TID_QID_S 14
u8 status;
};
+struct cpl_abort_req_rss6 {
+ WR_HDR;
+ union opcode_tid ot;
+ __u32 srqidx_status;
+};
+
+#define ABORT_RSS_STATUS_S 0
+#define ABORT_RSS_STATUS_M 0xff
+#define ABORT_RSS_STATUS_V(x) ((x) << ABORT_RSS_STATUS_S)
+#define ABORT_RSS_STATUS_G(x) (((x) >> ABORT_RSS_STATUS_S) & ABORT_RSS_STATUS_M)
+
+#define ABORT_RSS_SRQIDX_S 8
+#define ABORT_RSS_SRQIDX_M 0xffffff
+#define ABORT_RSS_SRQIDX_V(x) ((x) << ABORT_RSS_SRQIDX_S)
+#define ABORT_RSS_SRQIDX_G(x) (((x) >> ABORT_RSS_SRQIDX_S) & ABORT_RSS_SRQIDX_M)
+
struct cpl_abort_req {
WR_HDR;
union opcode_tid ot;
u8 status;
};
+struct cpl_abort_rpl_rss6 {
+ union opcode_tid ot;
+ __u32 srqidx_status;
+};
+
struct cpl_abort_rpl {
WR_HDR;
union opcode_tid ot;
X_CPL_RX_MPS_PKT_TYPE_QFC = 1 << 2,
X_CPL_RX_MPS_PKT_TYPE_PTP = 1 << 3
};
+
+struct cpl_srq_table_req {
+ WR_HDR;
+ union opcode_tid ot;
+ __u8 status;
+ __u8 rsvd[2];
+ __u8 idx;
+ __be64 rsvd_pdid;
+ __be32 qlen_qbase;
+ __be16 cur_msn;
+ __be16 max_msn;
+};
+
+struct cpl_srq_table_rpl {
+ union opcode_tid ot;
+ __u8 status;
+ __u8 rsvd[2];
+ __u8 idx;
+ __be64 rsvd_pdid;
+ __be32 qlen_qbase;
+ __be16 cur_msn;
+ __be16 max_msn;
+};
+
+/* cpl_srq_table_{req,rpl}.params fields */
+#define SRQT_QLEN_S 28
+#define SRQT_QLEN_M 0xF
+#define SRQT_QLEN_V(x) ((x) << SRQT_QLEN_S)
+#define SRQT_QLEN_G(x) (((x) >> SRQT_QLEN_S) & SRQT_QLEN_M)
+
+#define SRQT_QBASE_S 0
+#define SRQT_QBASE_M 0x3FFFFFF
+#define SRQT_QBASE_V(x) ((x) << SRQT_QBASE_S)
+#define SRQT_QBASE_G(x) (((x) >> SRQT_QBASE_S) & SRQT_QBASE_M)
+
+#define SRQT_PDID_S 0
+#define SRQT_PDID_M 0xFF
+#define SRQT_PDID_V(x) ((x) << SRQT_PDID_S)
+#define SRQT_PDID_G(x) (((x) >> SRQT_PDID_S) & SRQT_PDID_M)
+
+#define SRQT_IDX_S 0
+#define SRQT_IDX_M 0xF
+#define SRQT_IDX_V(x) ((x) << SRQT_IDX_S)
+#define SRQT_IDX_G(x) (((x) >> SRQT_IDX_S) & SRQT_IDX_M)
+
#endif /* __T4_MSG_H */
FW_RI_BIND_MW_WR = 0x18,
FW_RI_FR_NSMR_WR = 0x19,
FW_RI_FR_NSMR_TPTE_WR = 0x20,
+ FW_RI_RDMA_WRITE_CMPL_WR = 0x21,
FW_RI_INV_LSTAG_WR = 0x1a,
FW_ISCSI_TX_DATA_WR = 0x45,
FW_PTP_TX_PKT_WR = 0x46,
FW_PARAMS_PARAM_DEV_FILTER2_WR = 0x1D,
FW_PARAMS_PARAM_DEV_MPSBGMAP = 0x1E,
FW_PARAMS_PARAM_DEV_HMA_SIZE = 0x20,
+ FW_PARAMS_PARAM_DEV_RDMA_WRITE_WITH_IMM = 0x21,
+ FW_PARAMS_PARAM_DEV_RI_WRITE_CMPL_WR = 0x24,
};
/*
FW_PARAMS_PARAM_PFVF_SQRQ_END = 0x16,
FW_PARAMS_PARAM_PFVF_CQ_START = 0x17,
FW_PARAMS_PARAM_PFVF_CQ_END = 0x18,
+ FW_PARAMS_PARAM_PFVF_SRQ_START = 0x19,
+ FW_PARAMS_PARAM_PFVF_SRQ_END = 0x1A,
FW_PARAMS_PARAM_PFVF_SCHEDCLASS_ETH = 0x20,
FW_PARAMS_PARAM_PFVF_VIID = 0x24,
FW_PARAMS_PARAM_PFVF_CPMASK = 0x25,
};
static struct cxgb4vf_debugfs_entry debugfs_files[] = {
- { "mboxlog", S_IRUGO, &mboxlog_fops },
- { "sge_qinfo", S_IRUGO, &sge_qinfo_debugfs_fops },
- { "sge_qstats", S_IRUGO, &sge_qstats_proc_fops },
- { "resources", S_IRUGO, &resources_proc_fops },
- { "interfaces", S_IRUGO, &interfaces_proc_fops },
+ { "mboxlog", 0444, &mboxlog_fops },
+ { "sge_qinfo", 0444, &sge_qinfo_debugfs_fops },
+ { "sge_qstats", 0444, &sge_qstats_proc_fops },
+ { "resources", 0444, &resources_proc_fops },
+ { "interfaces", 0444, &interfaces_proc_fops },
};
/*
if (port->txq_dma_base & ~DMA_Q_BASE_MASK) {
dev_warn(geth->dev, "TX queue base it not aligned\n");
+ kfree(skb_tab);
return -ENOMEM;
}
};
module_pci_driver(pci_driver);
-module_param(polling_frequency, long, S_IRUGO);
+module_param(polling_frequency, long, 0444);
MODULE_PARM_DESC(polling_frequency, "Polling timer frequency in ns");
MODULE_LICENSE("GPL");
* Use sysfs method to enable/disable VFs.
*/
static unsigned int num_vfs;
-module_param(num_vfs, uint, S_IRUGO);
+module_param(num_vfs, uint, 0444);
MODULE_PARM_DESC(num_vfs, "Number of PCI VFs to initialize");
static ushort rx_frag_size = 2048;
-module_param(rx_frag_size, ushort, S_IRUGO);
+module_param(rx_frag_size, ushort, 0444);
MODULE_PARM_DESC(rx_frag_size, "Size of a fragment that holds rcvd data.");
/* Per-module error detection/recovery workq shared across all functions.
adapter->hwmon_info.be_on_die_temp * 1000);
}
-static SENSOR_DEVICE_ATTR(temp1_input, S_IRUGO,
+static SENSOR_DEVICE_ATTR(temp1_input, 0444,
be_hwmon_show_temp, NULL, 1);
static struct attribute *be_hwmon_attrs[] = {
}
if (unlikely(err < 0)) {
- percpu_stats->tx_errors++;
percpu_stats->tx_fifo_errors++;
return err;
}
vaddr = phys_to_virt(addr);
prefetch(vaddr + qm_fd_get_offset(fd));
- fd_format = qm_fd_get_format(fd);
/* The only FD types that we may receive are contig and S/G */
WARN_ON((fd_format != qm_fd_contig) && (fd_format != qm_fd_sg));
skb_len = skb->len;
- if (unlikely(netif_receive_skb(skb) == NET_RX_DROP))
+ if (unlikely(netif_receive_skb(skb) == NET_RX_DROP)) {
+ percpu_stats->rx_dropped++;
return qman_cb_dqrr_consume;
+ }
percpu_stats->rx_packets++;
percpu_stats->rx_bytes += skb_len;
struct device *dev;
int err;
- dev = &pdev->dev;
+ dev = pdev->dev.parent;
net_dev = dev_get_drvdata(dev);
priv = netdev_priv(net_dev);
/* gather congestion related counters */
cg_num = 0;
- cg_status = 0;
+ cg_status = false;
cg_time = jiffies_to_msecs(priv->cgr_data.congested_jiffies);
if (qman_query_cgr_congested(&priv->cgr_data.cgr, &cg_status) == 0) {
cg_num = priv->cgr_data.cgr_congested_count;
fec_enet_mii_remove(fep);
if (fep->reg_phy)
regulator_disable(fep->reg_phy);
+ pm_runtime_put(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
if (of_phy_is_fixed_link(np))
of_phy_deregister_fixed_link(np);
of_node_put(fep->phy_node);
tristate "FMan support"
depends on FSL_SOC || ARCH_LAYERSCAPE || COMPILE_TEST
select GENERIC_ALLOCATOR
- depends on HAS_DMA
select PHYLIB
default n
help
set_bucket(dtsec->regs, bucket, true);
/* Create element to be added to the driver hash table */
- hash_entry = kmalloc(sizeof(*hash_entry), GFP_KERNEL);
+ hash_entry = kmalloc(sizeof(*hash_entry), GFP_ATOMIC);
if (!hash_entry)
return -ENOMEM;
hash_entry->addr = addr;
}
pdev->dev.parent = priv->dev;
- set_dma_ops(&pdev->dev, get_dma_ops(priv->dev));
ret = platform_device_add_data(pdev, &data, sizeof(data));
if (ret)
static int hns_gmac_get_sset_count(int stringset)
{
- if (stringset == ETH_SS_STATS || stringset == ETH_SS_PRIV_FLAGS)
+ if (stringset == ETH_SS_STATS)
return ARRAY_SIZE(g_gmac_stats_string);
return 0;
int hns_ppe_get_sset_count(int stringset)
{
- if (stringset == ETH_SS_STATS || stringset == ETH_SS_PRIV_FLAGS)
+ if (stringset == ETH_SS_STATS)
return ETH_PPE_STATIC_NUM;
return 0;
}
*/
int hns_rcb_get_ring_sset_count(int stringset)
{
- if (stringset == ETH_SS_STATS || stringset == ETH_SS_PRIV_FLAGS)
+ if (stringset == ETH_SS_STATS)
return HNS_RING_STATIC_REG_NUM;
return 0;
cnt--;
return cnt;
- } else {
+ } else if (stringset == ETH_SS_STATS) {
return (HNS_NET_STATS_CNT + ops->get_sset_count(h, stringset));
+ } else {
+ return -EOPNOTSUPP;
}
}
enum HCLGE_MBX_OPCODE {
HCLGE_MBX_RESET = 0x01, /* (VF -> PF) assert reset */
+ HCLGE_MBX_ASSERTING_RESET, /* (PF -> VF) PF is asserting reset*/
HCLGE_MBX_SET_UNICAST, /* (VF -> PF) set UC addr */
HCLGE_MBX_SET_MULTICAST, /* (VF -> PF) set MC addr */
HCLGE_MBX_SET_VLAN, /* (VF -> PF) set VLAN */
u16 msg[8];
};
+/* used by VF to store the received Async responses from PF */
+struct hclgevf_mbx_arq_ring {
+#define HCLGE_MBX_MAX_ARQ_MSG_SIZE 8
+#define HCLGE_MBX_MAX_ARQ_MSG_NUM 1024
+ struct hclgevf_dev *hdev;
+ u32 head;
+ u32 tail;
+ u32 count;
+ u16 msg_q[HCLGE_MBX_MAX_ARQ_MSG_NUM][HCLGE_MBX_MAX_ARQ_MSG_SIZE];
+};
+
#define hclge_mbx_ring_ptr_move_crq(crq) \
(crq->next_to_use = (crq->next_to_use + 1) % crq->desc_num)
+#define hclge_mbx_tail_ptr_move_arq(arq) \
+ (arq.tail = (arq.tail + 1) % HCLGE_MBX_MAX_ARQ_MSG_SIZE)
+#define hclge_mbx_head_ptr_move_arq(arq) \
+ (arq.head = (arq.head + 1) % HCLGE_MBX_MAX_ARQ_MSG_SIZE)
#endif
};
enum hnae3_reset_type {
+ HNAE3_VF_RESET,
+ HNAE3_VF_FULL_RESET,
HNAE3_FUNC_RESET,
HNAE3_CORE_RESET,
HNAE3_GLOBAL_RESET,
int (*set_vf_vlan_filter)(struct hnae3_handle *handle, int vfid,
u16 vlan, u8 qos, __be16 proto);
int (*enable_hw_strip_rxvtag)(struct hnae3_handle *handle, bool enable);
- void (*reset_event)(struct hnae3_handle *handle,
- enum hnae3_reset_type reset);
+ void (*reset_event)(struct hnae3_handle *handle);
void (*get_channels)(struct hnae3_handle *handle,
struct ethtool_channels *ch);
void (*get_tqps_and_rss_info)(struct hnae3_handle *h,
u32 *flowctrl_adv);
int (*set_led_id)(struct hnae3_handle *handle,
enum ethtool_phys_id_state status);
+ void (*get_link_mode)(struct hnae3_handle *handle,
+ unsigned long *supported,
+ unsigned long *advertising);
+ void (*get_port_type)(struct hnae3_handle *handle, u8 *port_type);
};
struct hnae3_dcb_ops {
struct hnae3_ae_algo *ae_algo; /* the class who provides this handle */
u64 flags; /* Indicate the capabilities for this handle*/
+ unsigned long last_reset_time;
+ enum hnae3_reset_type reset_level;
+
union {
struct net_device *netdev; /* first member */
struct hnae3_knic_private_info kinfo;
/* Default: disable RL */
h->kinfo.int_rl_setting = 0;
+ tqp_vector->int_adapt_down = HNS3_INT_ADAPT_DOWN_START;
tqp_vector->rx_group.coal.flow_level = HNS3_FLOW_LOW;
tqp_vector->tx_group.coal.flow_level = HNS3_FLOW_LOW;
}
return ret;
}
- priv->last_reset_time = jiffies;
+ priv->ae_handle->last_reset_time = jiffies;
return 0;
}
{
/* Config bd buffer end */
hnae_set_field(*bdtp_fe_sc_vld_ra_ri, HNS3_TXD_BDTYPE_M,
- HNS3_TXD_BDTYPE_M, 0);
+ HNS3_TXD_BDTYPE_S, 0);
hnae_set_bit(*bdtp_fe_sc_vld_ra_ri, HNS3_TXD_FE_B, !!frag_end);
hnae_set_bit(*bdtp_fe_sc_vld_ra_ri, HNS3_TXD_VLD_B, 1);
hnae_set_field(*bdtp_fe_sc_vld_ra_ri, HNS3_TXD_SC_M, HNS3_TXD_SC_S, 0);
__be16 proto, u16 vid)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
+ struct hns3_nic_priv *priv = netdev_priv(netdev);
int ret = -EIO;
if (h->ae_algo->ops->set_vlan_filter)
ret = h->ae_algo->ops->set_vlan_filter(h, proto, vid, false);
+ if (!ret)
+ set_bit(vid, priv->active_vlans);
+
return ret;
}
__be16 proto, u16 vid)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
+ struct hns3_nic_priv *priv = netdev_priv(netdev);
int ret = -EIO;
if (h->ae_algo->ops->set_vlan_filter)
ret = h->ae_algo->ops->set_vlan_filter(h, proto, vid, true);
+ if (!ret)
+ clear_bit(vid, priv->active_vlans);
+
return ret;
}
+static void hns3_restore_vlan(struct net_device *netdev)
+{
+ struct hns3_nic_priv *priv = netdev_priv(netdev);
+ u16 vid;
+ int ret;
+
+ for_each_set_bit(vid, priv->active_vlans, VLAN_N_VID) {
+ ret = hns3_vlan_rx_add_vid(netdev, htons(ETH_P_8021Q), vid);
+ if (ret)
+ netdev_warn(netdev, "Restore vlan: %d filter, ret:%d\n",
+ vid, ret);
+ }
+}
+
static int hns3_ndo_set_vf_vlan(struct net_device *netdev, int vf, u16 vlan,
u8 qos, __be16 vlan_proto)
{
static void hns3_nic_net_timeout(struct net_device *ndev)
{
struct hns3_nic_priv *priv = netdev_priv(ndev);
- unsigned long last_reset_time = priv->last_reset_time;
struct hnae3_handle *h = priv->ae_handle;
if (!hns3_get_tx_timeo_queue_info(ndev))
priv->tx_timeout_count++;
- /* This timeout is far away enough from last timeout,
- * if timeout again,set the reset type to PF reset
- */
- if (time_after(jiffies, (last_reset_time + 20 * HZ)))
- priv->reset_level = HNAE3_FUNC_RESET;
-
- /* Don't do any new action before the next timeout */
- else if (time_before(jiffies, (last_reset_time + ndev->watchdog_timeo)))
+ if (time_before(jiffies, (h->last_reset_time + ndev->watchdog_timeo)))
return;
- priv->last_reset_time = jiffies;
-
+ /* request the reset */
if (h->ae_algo->ops->reset_event)
- h->ae_algo->ops->reset_event(h, priv->reset_level);
-
- priv->reset_level++;
- if (priv->reset_level > HNAE3_GLOBAL_RESET)
- priv->reset_level = HNAE3_GLOBAL_RESET;
+ h->ae_algo->ops->reset_event(h);
}
static const struct net_device_ops hns3_nic_netdev_ops = {
static bool hns3_get_new_int_gl(struct hns3_enet_ring_group *ring_group)
{
-#define HNS3_RX_ULTRA_PACKET_RATE 40000
+ struct hns3_enet_tqp_vector *tqp_vector =
+ ring_group->ring->tqp_vector;
enum hns3_flow_level_range new_flow_level;
- struct hns3_enet_tqp_vector *tqp_vector;
- int packets_per_secs;
- int bytes_per_usecs;
+ int packets_per_msecs;
+ int bytes_per_msecs;
+ u32 time_passed_ms;
u16 new_int_gl;
- int usecs;
- if (!ring_group->coal.int_gl)
+ if (!ring_group->coal.int_gl || !tqp_vector->last_jiffies)
return false;
if (ring_group->total_packets == 0) {
*/
new_flow_level = ring_group->coal.flow_level;
new_int_gl = ring_group->coal.int_gl;
- tqp_vector = ring_group->ring->tqp_vector;
- usecs = (ring_group->coal.int_gl << 1);
- bytes_per_usecs = ring_group->total_bytes / usecs;
- /* 1000000 microseconds */
- packets_per_secs = ring_group->total_packets * 1000000 / usecs;
+ time_passed_ms =
+ jiffies_to_msecs(jiffies - tqp_vector->last_jiffies);
+
+ if (!time_passed_ms)
+ return false;
+
+ do_div(ring_group->total_packets, time_passed_ms);
+ packets_per_msecs = ring_group->total_packets;
+
+ do_div(ring_group->total_bytes, time_passed_ms);
+ bytes_per_msecs = ring_group->total_bytes;
+
+#define HNS3_RX_LOW_BYTE_RATE 10000
+#define HNS3_RX_MID_BYTE_RATE 20000
switch (new_flow_level) {
case HNS3_FLOW_LOW:
- if (bytes_per_usecs > 10)
+ if (bytes_per_msecs > HNS3_RX_LOW_BYTE_RATE)
new_flow_level = HNS3_FLOW_MID;
break;
case HNS3_FLOW_MID:
- if (bytes_per_usecs > 20)
+ if (bytes_per_msecs > HNS3_RX_MID_BYTE_RATE)
new_flow_level = HNS3_FLOW_HIGH;
- else if (bytes_per_usecs <= 10)
+ else if (bytes_per_msecs <= HNS3_RX_LOW_BYTE_RATE)
new_flow_level = HNS3_FLOW_LOW;
break;
case HNS3_FLOW_HIGH:
case HNS3_FLOW_ULTRA:
default:
- if (bytes_per_usecs <= 20)
+ if (bytes_per_msecs <= HNS3_RX_MID_BYTE_RATE)
new_flow_level = HNS3_FLOW_MID;
break;
}
- if ((packets_per_secs > HNS3_RX_ULTRA_PACKET_RATE) &&
- (&tqp_vector->rx_group == ring_group))
+#define HNS3_RX_ULTRA_PACKET_RATE 40
+
+ if (packets_per_msecs > HNS3_RX_ULTRA_PACKET_RATE &&
+ &tqp_vector->rx_group == ring_group)
new_flow_level = HNS3_FLOW_ULTRA;
switch (new_flow_level) {
struct hns3_enet_ring_group *tx_group = &tqp_vector->tx_group;
bool rx_update, tx_update;
+ if (tqp_vector->int_adapt_down > 0) {
+ tqp_vector->int_adapt_down--;
+ return;
+ }
+
if (rx_group->coal.gl_adapt_enable) {
rx_update = hns3_get_new_int_gl(rx_group);
if (rx_update)
hns3_set_vector_coalesce_tx_gl(tqp_vector,
tx_group->coal.int_gl);
}
+
+ tqp_vector->last_jiffies = jiffies;
+ tqp_vector->int_adapt_down = HNS3_INT_ADAPT_DOWN_START;
}
static int hns3_nic_common_poll(struct napi_struct *napi, int budget)
priv->dev = &pdev->dev;
priv->netdev = netdev;
priv->ae_handle = handle;
- priv->last_reset_time = jiffies;
- priv->reset_level = HNAE3_FUNC_RESET;
+ priv->ae_handle->reset_level = HNAE3_NONE_RESET;
+ priv->ae_handle->last_reset_time = jiffies;
priv->tx_timeout_count = 0;
handle->kinfo.netdev = netdev;
static int hns3_reset_notify_up_enet(struct hnae3_handle *handle)
{
struct hnae3_knic_private_info *kinfo = &handle->kinfo;
- struct hns3_nic_priv *priv = netdev_priv(kinfo->netdev);
int ret = 0;
if (netif_running(kinfo->netdev)) {
"hns net up fail, ret=%d!\n", ret);
return ret;
}
-
- priv->last_reset_time = jiffies;
+ handle->last_reset_time = jiffies;
}
return ret;
struct hns3_nic_priv *priv = netdev_priv(netdev);
int ret;
- priv->reset_level = 1;
hns3_init_mac_addr(netdev);
hns3_nic_set_rx_mode(netdev);
hns3_recover_hw_addr(netdev);
+ /* Hardware table is only clear when pf resets */
+ if (!(handle->flags & HNAE3_SUPPORT_VF))
+ hns3_restore_vlan(netdev);
+
/* Carrier off reporting is important to ethtool even BEFORE open */
netif_carrier_off(netdev);
#ifndef __HNS3_ENET_H
#define __HNS3_ENET_H
+#include <linux/if_vlan.h>
+
#include "hnae3.h"
extern const char hns3_driver_version[];
#define HNS3_INT_RL_MAX 0x00EC
#define HNS3_INT_RL_ENABLE_MASK 0x40
+#define HNS3_INT_ADAPT_DOWN_START 100
+
struct hns3_enet_coalesce {
u16 int_gl;
u8 gl_adapt_enable;
/* when 0 should adjust interrupt coalesce parameter */
u8 int_adapt_down;
+ unsigned long last_jiffies;
} ____cacheline_internodealigned_in_smp;
enum hns3_udp_tnl_type {
/* The most recently read link state */
int link;
u64 tx_timeout_count;
- enum hnae3_reset_type reset_level;
- unsigned long last_reset_time;
unsigned long state;
struct notifier_block notifier_block;
/* Vxlan/Geneve information */
struct hns3_udp_tunnel udp_tnl[HNS3_UDP_TNL_MAX];
+ unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
};
union l3_hdr_info {
u32 ethtool_link_mode;
};
-static const struct hns3_link_mode_mapping hns3_lm_map[] = {
- {HNS3_LM_FIBRE_BIT, ETHTOOL_LINK_MODE_FIBRE_BIT},
- {HNS3_LM_AUTONEG_BIT, ETHTOOL_LINK_MODE_Autoneg_BIT},
- {HNS3_LM_TP_BIT, ETHTOOL_LINK_MODE_TP_BIT},
- {HNS3_LM_PAUSE_BIT, ETHTOOL_LINK_MODE_Pause_BIT},
- {HNS3_LM_BACKPLANE_BIT, ETHTOOL_LINK_MODE_Backplane_BIT},
- {HNS3_LM_10BASET_HALF_BIT, ETHTOOL_LINK_MODE_10baseT_Half_BIT},
- {HNS3_LM_10BASET_FULL_BIT, ETHTOOL_LINK_MODE_10baseT_Full_BIT},
- {HNS3_LM_100BASET_HALF_BIT, ETHTOOL_LINK_MODE_100baseT_Half_BIT},
- {HNS3_LM_100BASET_FULL_BIT, ETHTOOL_LINK_MODE_100baseT_Full_BIT},
- {HNS3_LM_1000BASET_FULL_BIT, ETHTOOL_LINK_MODE_1000baseT_Full_BIT},
-};
-
static int hns3_lp_setup(struct net_device *ndev, enum hnae3_loop loop)
{
struct hnae3_handle *h = hns3_get_handle(ndev);
dev_open(ndev);
}
-static void hns3_driv_to_eth_caps(u32 caps, struct ethtool_link_ksettings *cmd,
- bool is_advertised)
-{
- int i;
-
- for (i = 0; i < ARRAY_SIZE(hns3_lm_map); i++) {
- if (!(caps & hns3_lm_map[i].hns3_link_mode))
- continue;
-
- if (is_advertised)
- __set_bit(hns3_lm_map[i].ethtool_link_mode,
- cmd->link_modes.advertising);
- else
- __set_bit(hns3_lm_map[i].ethtool_link_mode,
- cmd->link_modes.supported);
- }
-}
-
static int hns3_get_sset_count(struct net_device *netdev, int stringset)
{
struct hnae3_handle *h = hns3_get_handle(netdev);
{
struct hnae3_handle *h = hns3_get_handle(netdev);
u32 flowctrl_adv = 0;
- u32 supported_caps;
- u32 advertised_caps;
- u8 media_type = HNAE3_MEDIA_TYPE_UNKNOWN;
u8 link_stat;
if (!h->ae_algo || !h->ae_algo->ops)
return -EOPNOTSUPP;
/* 1.auto_neg & speed & duplex from cmd */
- if (netdev->phydev)
+ if (netdev->phydev) {
phy_ethtool_ksettings_get(netdev->phydev, cmd);
- else if (h->ae_algo->ops->get_ksettings_an_result)
+
+ return 0;
+ }
+
+ if (h->ae_algo->ops->get_ksettings_an_result)
h->ae_algo->ops->get_ksettings_an_result(h,
&cmd->base.autoneg,
&cmd->base.speed,
cmd->base.duplex = DUPLEX_UNKNOWN;
}
- /* 2.media_type get from bios parameter block */
- if (h->ae_algo->ops->get_media_type) {
- h->ae_algo->ops->get_media_type(h, &media_type);
+ /* 2.get link mode and port type*/
+ if (h->ae_algo->ops->get_link_mode)
+ h->ae_algo->ops->get_link_mode(h,
+ cmd->link_modes.supported,
+ cmd->link_modes.advertising);
- switch (media_type) {
- case HNAE3_MEDIA_TYPE_FIBER:
- cmd->base.port = PORT_FIBRE;
- supported_caps = HNS3_LM_FIBRE_BIT |
- HNS3_LM_AUTONEG_BIT |
- HNS3_LM_PAUSE_BIT |
- HNS3_LM_1000BASET_FULL_BIT;
-
- advertised_caps = supported_caps;
- break;
- case HNAE3_MEDIA_TYPE_COPPER:
- cmd->base.port = PORT_TP;
- supported_caps = HNS3_LM_TP_BIT |
- HNS3_LM_AUTONEG_BIT |
- HNS3_LM_PAUSE_BIT |
- HNS3_LM_1000BASET_FULL_BIT |
- HNS3_LM_100BASET_FULL_BIT |
- HNS3_LM_100BASET_HALF_BIT |
- HNS3_LM_10BASET_FULL_BIT |
- HNS3_LM_10BASET_HALF_BIT;
- advertised_caps = supported_caps;
- break;
- case HNAE3_MEDIA_TYPE_BACKPLANE:
- cmd->base.port = PORT_NONE;
- supported_caps = HNS3_LM_BACKPLANE_BIT |
- HNS3_LM_PAUSE_BIT |
- HNS3_LM_AUTONEG_BIT |
- HNS3_LM_1000BASET_FULL_BIT |
- HNS3_LM_100BASET_FULL_BIT |
- HNS3_LM_100BASET_HALF_BIT |
- HNS3_LM_10BASET_FULL_BIT |
- HNS3_LM_10BASET_HALF_BIT;
-
- advertised_caps = supported_caps;
- break;
- case HNAE3_MEDIA_TYPE_UNKNOWN:
- default:
- cmd->base.port = PORT_OTHER;
- supported_caps = 0;
- advertised_caps = 0;
- break;
- }
-
- if (!cmd->base.autoneg)
- advertised_caps &= ~HNS3_LM_AUTONEG_BIT;
-
- advertised_caps &= ~HNS3_LM_PAUSE_BIT;
-
- /* now, map driver link modes to ethtool link modes */
- hns3_driv_to_eth_caps(supported_caps, cmd, false);
- hns3_driv_to_eth_caps(advertised_caps, cmd, true);
- }
+ cmd->base.port = PORT_NONE;
+ if (h->ae_algo->ops->get_port_type)
+ h->ae_algo->ops->get_port_type(h,
+ &cmd->base.port);
/* 3.mdix_ctrl&mdix get from phy reg */
if (h->ae_algo->ops->get_mdix_mode)
if (!h->ae_algo || !h->ae_algo->ops ||
!h->ae_algo->ops->get_rss_key_size)
- return -EOPNOTSUPP;
+ return 0;
return h->ae_algo->ops->get_rss_key_size(h);
}
if (!h->ae_algo || !h->ae_algo->ops ||
!h->ae_algo->ops->get_rss_indir_size)
- return -EOPNOTSUPP;
+ return 0;
return h->ae_algo->ops->get_rss_indir_size(h);
}
.get_channels = hns3_get_channels,
.get_coalesce = hns3_get_coalesce,
.set_coalesce = hns3_set_coalesce,
+ .get_link = hns3_get_link,
};
static const struct ethtool_ops hns3_ethtool_ops = {
#include <linux/types.h>
#include <linux/io.h>
-#define HCLGE_CMDQ_TX_TIMEOUT 1000
+#define HCLGE_CMDQ_TX_TIMEOUT 30000
struct hclge_dev;
struct hclge_desc {
#define HCLGE_CFG_DEFAULT_SPEED_M GENMASK(23, 16)
#define HCLGE_CFG_RSS_SIZE_S 24
#define HCLGE_CFG_RSS_SIZE_M GENMASK(31, 24)
+#define HCLGE_CFG_SPEED_ABILITY_S 0
+#define HCLGE_CFG_SPEED_ABILITY_M GENMASK(7, 0)
struct hclge_cfg_param_cmd {
__le32 offset;
{0, }
};
+MODULE_DEVICE_TABLE(pci, ae_algo_pci_tbl);
+
static const char hns3_nic_test_strs[][ETH_GSTRING_LEN] = {
"Mac Loopback test",
"Serdes Loopback test",
return 0;
}
+static void hclge_parse_fiber_link_mode(struct hclge_dev *hdev,
+ u8 speed_ability)
+{
+ unsigned long *supported = hdev->hw.mac.supported;
+
+ if (speed_ability & HCLGE_SUPPORT_1G_BIT)
+ set_bit(ETHTOOL_LINK_MODE_1000baseX_Full_BIT,
+ supported);
+
+ if (speed_ability & HCLGE_SUPPORT_10G_BIT)
+ set_bit(ETHTOOL_LINK_MODE_10000baseSR_Full_BIT,
+ supported);
+
+ if (speed_ability & HCLGE_SUPPORT_25G_BIT)
+ set_bit(ETHTOOL_LINK_MODE_25000baseSR_Full_BIT,
+ supported);
+
+ if (speed_ability & HCLGE_SUPPORT_50G_BIT)
+ set_bit(ETHTOOL_LINK_MODE_50000baseSR2_Full_BIT,
+ supported);
+
+ if (speed_ability & HCLGE_SUPPORT_100G_BIT)
+ set_bit(ETHTOOL_LINK_MODE_100000baseSR4_Full_BIT,
+ supported);
+
+ set_bit(ETHTOOL_LINK_MODE_FIBRE_BIT, supported);
+ set_bit(ETHTOOL_LINK_MODE_Pause_BIT, supported);
+}
+
+static void hclge_parse_link_mode(struct hclge_dev *hdev, u8 speed_ability)
+{
+ u8 media_type = hdev->hw.mac.media_type;
+
+ if (media_type != HNAE3_MEDIA_TYPE_FIBER)
+ return;
+
+ hclge_parse_fiber_link_mode(hdev, speed_ability);
+}
+
static void hclge_parse_cfg(struct hclge_cfg *cfg, struct hclge_desc *desc)
{
struct hclge_cfg_param_cmd *req;
req = (struct hclge_cfg_param_cmd *)desc[1].data;
cfg->numa_node_map = __le32_to_cpu(req->param[0]);
+
+ cfg->speed_ability = hnae_get_field(__le32_to_cpu(req->param[1]),
+ HCLGE_CFG_SPEED_ABILITY_M,
+ HCLGE_CFG_SPEED_ABILITY_S);
}
/* hclge_get_cfg: query the static parameter from flash
return ret;
}
+ hclge_parse_link_mode(hdev, cfg.speed_ability);
+
if ((hdev->tc_max > HNAE3_MAX_TC) ||
(hdev->tc_max < 1)) {
dev_warn(&hdev->pdev->dev, "TC num = %d.\n",
return 0;
}
-static int hclge_func_reset_cmd(struct hclge_dev *hdev, int func_id)
+int hclge_func_reset_cmd(struct hclge_dev *hdev, int func_id)
{
struct hclge_desc desc;
struct hclge_reset_cmd *req = (struct hclge_reset_cmd *)desc.data;
hclge_notify_client(hdev, HNAE3_UP_CLIENT);
}
-static void hclge_reset_event(struct hnae3_handle *handle,
- enum hnae3_reset_type reset)
+static void hclge_reset_event(struct hnae3_handle *handle)
{
struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_dev *hdev = vport->back;
- dev_info(&hdev->pdev->dev,
- "Receive reset event , reset_type is %d", reset);
+ /* check if this is a new reset request and we are not here just because
+ * last reset attempt did not succeed and watchdog hit us again. We will
+ * know this if last reset request did not occur very recently (watchdog
+ * timer = 5*HZ, let us check after sufficiently large time, say 4*5*Hz)
+ * In case of new request we reset the "reset level" to PF reset.
+ */
+ if (time_after(jiffies, (handle->last_reset_time + 4 * 5 * HZ)))
+ handle->reset_level = HNAE3_FUNC_RESET;
- switch (reset) {
- case HNAE3_FUNC_RESET:
- case HNAE3_CORE_RESET:
- case HNAE3_GLOBAL_RESET:
- /* request reset & schedule reset task */
- set_bit(reset, &hdev->reset_request);
- hclge_reset_task_schedule(hdev);
- break;
- default:
- dev_warn(&hdev->pdev->dev, "Unsupported reset event:%d", reset);
- break;
- }
+ dev_info(&hdev->pdev->dev, "received reset event , reset type is %d",
+ handle->reset_level);
+
+ /* request reset & schedule reset task */
+ set_bit(handle->reset_level, &hdev->reset_request);
+ hclge_reset_task_schedule(hdev);
+
+ if (handle->reset_level < HNAE3_GLOBAL_RESET)
+ handle->reset_level++;
+
+ handle->last_reset_time = jiffies;
}
static void hclge_reset_subtask(struct hclge_dev *hdev)
struct hclge_vport *vport = hdev->vport;
int i;
- netdev_rss_key_fill(vport->rss_hash_key, HCLGE_RSS_KEY_SIZE);
-
for (i = 0; i < hdev->num_vmdq_vport + 1; i++) {
vport[i].rss_tuple_sets.ipv4_tcp_en =
HCLGE_RSS_INPUT_TUPLE_OTHER;
HCLGE_RSS_INPUT_TUPLE_OTHER;
vport[i].rss_algo = HCLGE_RSS_HASH_ALGO_TOEPLITZ;
+
+ netdev_rss_key_fill(vport[i].rss_hash_key, HCLGE_RSS_KEY_SIZE);
}
hclge_rss_indir_init_cfg(hdev);
struct hclge_dev *hdev = vport->back;
int vector_id, ret;
+ if (test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state))
+ return 0;
+
vector_id = hclge_get_vector_index(hdev, vector);
if (vector_id < 0) {
dev_err(&handle->pdev->dev,
clear_bit(HCLGE_STATE_DOWN, &hdev->state);
mod_timer(&hdev->service_timer, jiffies + HZ);
+ /* reset tqp stats */
+ hclge_reset_tqp_stats(handle);
+
+ if (test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state))
+ return 0;
+
ret = hclge_mac_start_phy(hdev);
if (ret)
return ret;
- /* reset tqp stats */
- hclge_reset_tqp_stats(handle);
-
return 0;
}
struct hclge_dev *hdev = vport->back;
int i;
+ del_timer_sync(&hdev->service_timer);
+ cancel_work_sync(&hdev->service_task);
+
+ if (test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state))
+ return;
+
for (i = 0; i < vport->alloc_tqps; i++)
hclge_tqp_enable(hdev, i, 0, false);
/* reset tqp stats */
hclge_reset_tqp_stats(handle);
- del_timer_sync(&hdev->service_timer);
- cancel_work_sync(&hdev->service_task);
hclge_update_link_status(hdev);
}
return hclge_set_vlan_rx_offload_cfg(vport);
}
-static int hclge_set_mtu(struct hnae3_handle *handle, int new_mtu)
+static int hclge_set_mac_mtu(struct hclge_dev *hdev, int new_mtu)
{
- struct hclge_vport *vport = hclge_get_vport(handle);
struct hclge_config_max_frm_size_cmd *req;
- struct hclge_dev *hdev = vport->back;
struct hclge_desc desc;
int max_frm_size;
int ret;
return 0;
}
+static int hclge_set_mtu(struct hnae3_handle *handle, int new_mtu)
+{
+ struct hclge_vport *vport = hclge_get_vport(handle);
+ struct hclge_dev *hdev = vport->back;
+ int ret;
+
+ ret = hclge_set_mac_mtu(hdev, new_mtu);
+ if (ret) {
+ dev_err(&hdev->pdev->dev,
+ "Change mtu fail, ret =%d\n", ret);
+ return ret;
+ }
+
+ ret = hclge_buffer_alloc(hdev);
+ if (ret)
+ dev_err(&hdev->pdev->dev,
+ "Allocate buffer fail, ret =%d\n", ret);
+
+ return ret;
+}
+
static int hclge_send_reset_tqp_cmd(struct hclge_dev *hdev, u16 queue_id,
bool enable)
{
u16 queue_gid;
int ret;
+ if (test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state))
+ return;
+
queue_gid = hclge_covert_handle_qid_global(handle, queue_id);
ret = hclge_tqp_enable(hdev, queue_id, 0, false);
}
}
+void hclge_reset_vf_queue(struct hclge_vport *vport, u16 queue_id)
+{
+ struct hclge_dev *hdev = vport->back;
+ int reset_try_times = 0;
+ int reset_status;
+ u16 queue_gid;
+ int ret;
+
+ queue_gid = hclge_covert_handle_qid_global(&vport->nic, queue_id);
+
+ ret = hclge_send_reset_tqp_cmd(hdev, queue_gid, true);
+ if (ret) {
+ dev_warn(&hdev->pdev->dev,
+ "Send reset tqp cmd fail, ret = %d\n", ret);
+ return;
+ }
+
+ reset_try_times = 0;
+ while (reset_try_times++ < HCLGE_TQP_RESET_TRY_TIMES) {
+ /* Wait for tqp hw reset */
+ msleep(20);
+ reset_status = hclge_get_reset_status(hdev, queue_gid);
+ if (reset_status)
+ break;
+ }
+
+ if (reset_try_times >= HCLGE_TQP_RESET_TRY_TIMES) {
+ dev_warn(&hdev->pdev->dev, "Reset TQP fail\n");
+ return;
+ }
+
+ ret = hclge_send_reset_tqp_cmd(hdev, queue_gid, false);
+ if (ret)
+ dev_warn(&hdev->pdev->dev,
+ "Deassert the soft reset fail, ret = %d\n", ret);
+}
+
static u32 hclge_get_fw_version(struct hnae3_handle *handle)
{
struct hclge_vport *vport = hclge_get_vport(handle);
dev_err(&pdev->dev, "Mac init error, ret = %d\n", ret);
return ret;
}
- ret = hclge_buffer_alloc(hdev);
- if (ret) {
- dev_err(&pdev->dev, "Buffer allocate fail, ret =%d\n", ret);
- return ret;
- }
ret = hclge_config_tso(hdev, HCLGE_TSO_MSS_MIN, HCLGE_TSO_MSS_MAX);
if (ret) {
return ret;
}
- ret = hclge_buffer_alloc(hdev);
- if (ret) {
- dev_err(&pdev->dev, "Buffer allocate fail, ret =%d\n", ret);
- return ret;
- }
-
ret = hclge_config_tso(hdev, HCLGE_TSO_MSS_MIN, HCLGE_TSO_MSS_MAX);
if (ret) {
dev_err(&pdev->dev, "Enable tso fail, ret =%d\n", ret);
HCLGE_LED_NO_CHANGE);
}
+static void hclge_get_link_mode(struct hnae3_handle *handle,
+ unsigned long *supported,
+ unsigned long *advertising)
+{
+ unsigned int size = BITS_TO_LONGS(__ETHTOOL_LINK_MODE_MASK_NBITS);
+ struct hclge_vport *vport = hclge_get_vport(handle);
+ struct hclge_dev *hdev = vport->back;
+ unsigned int idx = 0;
+
+ for (; idx < size; idx++) {
+ supported[idx] = hdev->hw.mac.supported[idx];
+ advertising[idx] = hdev->hw.mac.advertising[idx];
+ }
+}
+
+static void hclge_get_port_type(struct hnae3_handle *handle,
+ u8 *port_type)
+{
+ struct hclge_vport *vport = hclge_get_vport(handle);
+ struct hclge_dev *hdev = vport->back;
+ u8 media_type = hdev->hw.mac.media_type;
+
+ switch (media_type) {
+ case HNAE3_MEDIA_TYPE_FIBER:
+ *port_type = PORT_FIBRE;
+ break;
+ case HNAE3_MEDIA_TYPE_COPPER:
+ *port_type = PORT_TP;
+ break;
+ case HNAE3_MEDIA_TYPE_UNKNOWN:
+ default:
+ *port_type = PORT_OTHER;
+ break;
+ }
+}
+
static const struct hnae3_ae_ops hclge_ops = {
.init_ae_dev = hclge_init_ae_dev,
.uninit_ae_dev = hclge_uninit_ae_dev,
.get_regs_len = hclge_get_regs_len,
.get_regs = hclge_get_regs,
.set_led_id = hclge_set_led_id,
+ .get_link_mode = hclge_get_link_mode,
+ .get_port_type = hclge_get_port_type,
};
static struct hnae3_ae_algo ae_algo = {
#define HCLGE_MAC_MIN_FRAME 64
#define HCLGE_MAC_MAX_FRAME 9728
+#define HCLGE_SUPPORT_1G_BIT BIT(0)
+#define HCLGE_SUPPORT_10G_BIT BIT(1)
+#define HCLGE_SUPPORT_25G_BIT BIT(2)
+#define HCLGE_SUPPORT_50G_BIT BIT(3)
+#define HCLGE_SUPPORT_100G_BIT BIT(4)
+
enum HCLGE_DEV_STATE {
HCLGE_STATE_REINITING,
HCLGE_STATE_DOWN,
struct phy_device *phydev;
struct mii_bus *mdio_bus;
phy_interface_t phy_if;
+ __ETHTOOL_DECLARE_LINK_MODE_MASK(supported);
+ __ETHTOOL_DECLARE_LINK_MODE_MASK(advertising);
};
struct hclge_hw {
u8 mac_addr[ETH_ALEN];
u8 default_speed;
u32 numa_node_map;
+ u8 speed_ability;
};
struct hclge_tm_info {
void hclge_mbx_handler(struct hclge_dev *hdev);
void hclge_reset_tqp(struct hnae3_handle *handle, u16 queue_id);
+void hclge_reset_vf_queue(struct hclge_vport *vport, u16 queue_id);
int hclge_cfg_flowctrl(struct hclge_dev *hdev);
+int hclge_func_reset_cmd(struct hclge_dev *hdev, int func_id);
#endif
return status;
}
+static int hclge_inform_reset_assert_to_vf(struct hclge_vport *vport)
+{
+ u8 msg_data[2];
+ u8 dest_vfid;
+
+ dest_vfid = (u8)vport->vport_id;
+
+ /* send this requested info to VF */
+ return hclge_send_mbx_msg(vport, msg_data, sizeof(u8),
+ HCLGE_MBX_ASSERTING_RESET, dest_vfid);
+}
+
static void hclge_free_vector_ring_chain(struct hnae3_ring_chain_node *head)
{
struct hnae3_ring_chain_node *chain_tmp, *chain;
{
struct hclge_dev *hdev = vport->back;
u16 link_status;
- u8 msg_data[2];
+ u8 msg_data[8];
u8 dest_vfid;
+ u16 duplex;
/* mac.link can only be 0 or 1 */
link_status = (u16)hdev->hw.mac.link;
+ duplex = hdev->hw.mac.duplex;
memcpy(&msg_data[0], &link_status, sizeof(u16));
+ memcpy(&msg_data[2], &hdev->hw.mac.speed, sizeof(u32));
+ memcpy(&msg_data[6], &duplex, sizeof(u16));
dest_vfid = mbx_req->mbx_src_vfid;
/* send this requested info to VF */
- return hclge_send_mbx_msg(vport, msg_data, sizeof(u8),
+ return hclge_send_mbx_msg(vport, msg_data, sizeof(msg_data),
HCLGE_MBX_LINK_STAT_CHANGE, dest_vfid);
}
-static void hclge_reset_vf_queue(struct hclge_vport *vport,
- struct hclge_mbx_vf_to_pf_cmd *mbx_req)
+static void hclge_mbx_reset_vf_queue(struct hclge_vport *vport,
+ struct hclge_mbx_vf_to_pf_cmd *mbx_req)
{
u16 queue_id;
memcpy(&queue_id, &mbx_req->msg[2], sizeof(queue_id));
- hclge_reset_tqp(&vport->nic, queue_id);
+ hclge_reset_vf_queue(vport, queue_id);
+
+ /* send response msg to VF after queue reset complete*/
+ hclge_gen_resp_to_vf(vport, mbx_req, 0, NULL, 0);
+}
+
+static void hclge_reset_vf(struct hclge_vport *vport,
+ struct hclge_mbx_vf_to_pf_cmd *mbx_req)
+{
+ struct hclge_dev *hdev = vport->back;
+ int ret;
+
+ dev_warn(&hdev->pdev->dev, "PF received VF reset request from VF %d!",
+ mbx_req->mbx_src_vfid);
+
+ /* Acknowledge VF that PF is now about to assert the reset for the VF.
+ * On receiving this message VF will get into pending state and will
+ * start polling for the hardware reset completion status.
+ */
+ ret = hclge_inform_reset_assert_to_vf(vport);
+ if (ret) {
+ dev_err(&hdev->pdev->dev,
+ "PF fail(%d) to inform VF(%d)of reset, reset failed!\n",
+ ret, vport->vport_id);
+ return;
+ }
+
+ dev_warn(&hdev->pdev->dev, "PF is now resetting VF %d.\n",
+ mbx_req->mbx_src_vfid);
+ /* reset this virtual function */
+ hclge_func_reset_cmd(hdev, mbx_req->mbx_src_vfid);
}
void hclge_mbx_handler(struct hclge_dev *hdev)
ret);
break;
case HCLGE_MBX_QUEUE_RESET:
- hclge_reset_vf_queue(vport, req);
+ hclge_mbx_reset_vf_queue(vport, req);
+ break;
+ case HCLGE_MBX_RESET:
+ hclge_reset_vf(vport, req);
break;
default:
dev_err(&hdev->pdev->dev,
struct hclge_desc desc;
int ret;
+ if (test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state))
+ return 0;
+
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_MDIO_CONFIG, false);
mdio_cmd = (struct hclge_mdio_cfg_cmd *)desc.data;
struct hclge_desc desc;
int ret;
+ if (test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state))
+ return 0;
+
hclge_cmd_setup_basic_desc(&desc, HCLGE_OPC_MDIO_CONFIG, true);
mdio_cmd = (struct hclge_mdio_cfg_cmd *)desc.data;
goto err_csq;
}
+ /* initialize the pointers of async rx queue of mailbox */
+ hdev->arq.hdev = hdev;
+ hdev->arq.head = 0;
+ hdev->arq.tail = 0;
+ hdev->arq.count = 0;
+
/* get firmware version */
ret = hclgevf_cmd_query_firmware_version(&hdev->hw, &version);
if (ret) {
#include <linux/types.h>
#include "hnae3.h"
-#define HCLGEVF_CMDQ_TX_TIMEOUT 200
+#define HCLGEVF_CMDQ_TX_TIMEOUT 30000
#define HCLGEVF_CMDQ_RX_INVLD_B 0
#define HCLGEVF_CMDQ_RX_OUTVLD_B 1
// Copyright (c) 2016-2017 Hisilicon Limited.
#include <linux/etherdevice.h>
+#include <net/rtnetlink.h>
#include "hclgevf_cmd.h"
#include "hclgevf_main.h"
#include "hclge_mbx.h"
#define HCLGEVF_NAME "hclgevf"
+static int hclgevf_init_hdev(struct hclgevf_dev *hdev);
+static void hclgevf_uninit_hdev(struct hclgevf_dev *hdev);
static struct hnae3_ae_algo ae_algovf;
static const struct pci_device_id ae_algovf_pci_tbl[] = {
{0, }
};
+MODULE_DEVICE_TABLE(pci, ae_algovf_pci_tbl);
+
static inline struct hclgevf_dev *hclgevf_ae_get_hdev(
struct hnae3_handle *handle)
{
struct hclgevf_tqp *tqp;
int i;
+ /* if this is on going reset then we need to re-allocate the TPQs
+ * since we cannot assume we would get same number of TPQs back from PF
+ */
+ if (hclgevf_dev_ongoing_reset(hdev))
+ devm_kfree(&hdev->pdev->dev, hdev->htqp);
+
hdev->htqp = devm_kcalloc(&hdev->pdev->dev, hdev->num_tqps,
sizeof(struct hclgevf_tqp), GFP_KERNEL);
if (!hdev->htqp)
new_tqps = kinfo->rss_size * kinfo->num_tc;
kinfo->num_tqps = min(new_tqps, hdev->num_tqps);
+ /* if this is on going reset then we need to re-allocate the hnae queues
+ * as well since number of TPQs from PF might have changed.
+ */
+ if (hclgevf_dev_ongoing_reset(hdev))
+ devm_kfree(&hdev->pdev->dev, kinfo->tqp);
+
kinfo->tqp = devm_kcalloc(&hdev->pdev->dev, kinfo->num_tqps,
sizeof(struct hnae3_queue *), GFP_KERNEL);
if (!kinfo->tqp)
{
struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
u8 msg_data[2];
+ int ret;
memcpy(&msg_data[0], &queue_id, sizeof(queue_id));
- hclgevf_send_mbx_msg(hdev, HCLGE_MBX_QUEUE_RESET, 0, msg_data, 2, false,
- NULL, 0);
+ /* disable vf queue before send queue reset msg to PF */
+ ret = hclgevf_tqp_enable(hdev, queue_id, 0, false);
+ if (ret)
+ return;
+
+ hclgevf_send_mbx_msg(hdev, HCLGE_MBX_QUEUE_RESET, 0, msg_data,
+ 2, true, NULL, 0);
+}
+
+static int hclgevf_notify_client(struct hclgevf_dev *hdev,
+ enum hnae3_reset_notify_type type)
+{
+ struct hnae3_client *client = hdev->nic_client;
+ struct hnae3_handle *handle = &hdev->nic;
+
+ if (!client->ops->reset_notify)
+ return -EOPNOTSUPP;
+
+ return client->ops->reset_notify(handle, type);
+}
+
+static int hclgevf_reset_wait(struct hclgevf_dev *hdev)
+{
+#define HCLGEVF_RESET_WAIT_MS 500
+#define HCLGEVF_RESET_WAIT_CNT 20
+ u32 val, cnt = 0;
+
+ /* wait to check the hardware reset completion status */
+ val = hclgevf_read_dev(&hdev->hw, HCLGEVF_FUN_RST_ING);
+ while (hnae_get_bit(val, HCLGEVF_FUN_RST_ING_B) &&
+ (cnt < HCLGEVF_RESET_WAIT_CNT)) {
+ msleep(HCLGEVF_RESET_WAIT_MS);
+ val = hclgevf_read_dev(&hdev->hw, HCLGEVF_FUN_RST_ING);
+ cnt++;
+ }
+
+ /* hardware completion status should be available by this time */
+ if (cnt >= HCLGEVF_RESET_WAIT_CNT) {
+ dev_warn(&hdev->pdev->dev,
+ "could'nt get reset done status from h/w, timeout!\n");
+ return -EBUSY;
+ }
+
+ /* we will wait a bit more to let reset of the stack to complete. This
+ * might happen in case reset assertion was made by PF. Yes, this also
+ * means we might end up waiting bit more even for VF reset.
+ */
+ msleep(5000);
+
+ return 0;
+}
+
+static int hclgevf_reset_stack(struct hclgevf_dev *hdev)
+{
+ int ret;
+
+ /* uninitialize the nic client */
+ hclgevf_notify_client(hdev, HNAE3_UNINIT_CLIENT);
+
+ /* re-initialize the hclge device */
+ ret = hclgevf_init_hdev(hdev);
+ if (ret) {
+ dev_err(&hdev->pdev->dev,
+ "hclge device re-init failed, VF is disabled!\n");
+ return ret;
+ }
+
+ /* bring up the nic client again */
+ hclgevf_notify_client(hdev, HNAE3_INIT_CLIENT);
+
+ return 0;
+}
+
+static int hclgevf_reset(struct hclgevf_dev *hdev)
+{
+ int ret;
+
+ rtnl_lock();
+
+ /* bring down the nic to stop any ongoing TX/RX */
+ hclgevf_notify_client(hdev, HNAE3_DOWN_CLIENT);
+
+ /* check if VF could successfully fetch the hardware reset completion
+ * status from the hardware
+ */
+ ret = hclgevf_reset_wait(hdev);
+ if (ret) {
+ /* can't do much in this situation, will disable VF */
+ dev_err(&hdev->pdev->dev,
+ "VF failed(=%d) to fetch H/W reset completion status\n",
+ ret);
+
+ dev_warn(&hdev->pdev->dev, "VF reset failed, disabling VF!\n");
+ hclgevf_notify_client(hdev, HNAE3_UNINIT_CLIENT);
+
+ rtnl_unlock();
+ return ret;
+ }
+
+ /* now, re-initialize the nic client and ae device*/
+ ret = hclgevf_reset_stack(hdev);
+ if (ret)
+ dev_err(&hdev->pdev->dev, "failed to reset VF stack\n");
+
+ /* bring up the nic to enable TX/RX again */
+ hclgevf_notify_client(hdev, HNAE3_UP_CLIENT);
+
+ rtnl_unlock();
+
+ return ret;
+}
+
+static int hclgevf_do_reset(struct hclgevf_dev *hdev)
+{
+ int status;
+ u8 respmsg;
+
+ status = hclgevf_send_mbx_msg(hdev, HCLGE_MBX_RESET, 0, NULL,
+ 0, false, &respmsg, sizeof(u8));
+ if (status)
+ dev_err(&hdev->pdev->dev,
+ "VF reset request to PF failed(=%d)\n", status);
+
+ return status;
+}
+
+static void hclgevf_reset_event(struct hnae3_handle *handle)
+{
+ struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
+
+ dev_info(&hdev->pdev->dev, "received reset request from VF enet\n");
+
+ handle->reset_level = HNAE3_VF_RESET;
+
+ /* reset of this VF requested */
+ set_bit(HCLGEVF_RESET_REQUESTED, &hdev->reset_state);
+ hclgevf_reset_task_schedule(hdev);
+
+ handle->last_reset_time = jiffies;
}
static u32 hclgevf_get_fw_version(struct hnae3_handle *handle)
hdev->num_msi_used += 1;
}
-static void hclgevf_mbx_task_schedule(struct hclgevf_dev *hdev)
+void hclgevf_reset_task_schedule(struct hclgevf_dev *hdev)
+{
+ if (!test_bit(HCLGEVF_STATE_RST_SERVICE_SCHED, &hdev->state) &&
+ !test_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) {
+ set_bit(HCLGEVF_STATE_RST_SERVICE_SCHED, &hdev->state);
+ schedule_work(&hdev->rst_service_task);
+ }
+}
+
+void hclgevf_mbx_task_schedule(struct hclgevf_dev *hdev)
{
- if (!test_and_set_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state))
+ if (!test_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state) &&
+ !test_bit(HCLGEVF_STATE_MBX_HANDLING, &hdev->state)) {
+ set_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state);
schedule_work(&hdev->mbx_service_task);
+ }
}
static void hclgevf_task_schedule(struct hclgevf_dev *hdev)
schedule_work(&hdev->service_task);
}
+static void hclgevf_deferred_task_schedule(struct hclgevf_dev *hdev)
+{
+ /* if we have any pending mailbox event then schedule the mbx task */
+ if (hdev->mbx_event_pending)
+ hclgevf_mbx_task_schedule(hdev);
+
+ if (test_bit(HCLGEVF_RESET_PENDING, &hdev->reset_state))
+ hclgevf_reset_task_schedule(hdev);
+}
+
static void hclgevf_service_timer(struct timer_list *t)
{
struct hclgevf_dev *hdev = from_timer(hdev, t, service_timer);
hclgevf_task_schedule(hdev);
}
+static void hclgevf_reset_service_task(struct work_struct *work)
+{
+ struct hclgevf_dev *hdev =
+ container_of(work, struct hclgevf_dev, rst_service_task);
+ int ret;
+
+ if (test_and_set_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state))
+ return;
+
+ clear_bit(HCLGEVF_STATE_RST_SERVICE_SCHED, &hdev->state);
+
+ if (test_and_clear_bit(HCLGEVF_RESET_PENDING,
+ &hdev->reset_state)) {
+ /* PF has initmated that it is about to reset the hardware.
+ * We now have to poll & check if harware has actually completed
+ * the reset sequence. On hardware reset completion, VF needs to
+ * reset the client and ae device.
+ */
+ hdev->reset_attempts = 0;
+
+ ret = hclgevf_reset(hdev);
+ if (ret)
+ dev_err(&hdev->pdev->dev, "VF stack reset failed.\n");
+ } else if (test_and_clear_bit(HCLGEVF_RESET_REQUESTED,
+ &hdev->reset_state)) {
+ /* we could be here when either of below happens:
+ * 1. reset was initiated due to watchdog timeout due to
+ * a. IMP was earlier reset and our TX got choked down and
+ * which resulted in watchdog reacting and inducing VF
+ * reset. This also means our cmdq would be unreliable.
+ * b. problem in TX due to other lower layer(example link
+ * layer not functioning properly etc.)
+ * 2. VF reset might have been initiated due to some config
+ * change.
+ *
+ * NOTE: Theres no clear way to detect above cases than to react
+ * to the response of PF for this reset request. PF will ack the
+ * 1b and 2. cases but we will not get any intimation about 1a
+ * from PF as cmdq would be in unreliable state i.e. mailbox
+ * communication between PF and VF would be broken.
+ */
+
+ /* if we are never geting into pending state it means either:
+ * 1. PF is not receiving our request which could be due to IMP
+ * reset
+ * 2. PF is screwed
+ * We cannot do much for 2. but to check first we can try reset
+ * our PCIe + stack and see if it alleviates the problem.
+ */
+ if (hdev->reset_attempts > 3) {
+ /* prepare for full reset of stack + pcie interface */
+ hdev->nic.reset_level = HNAE3_VF_FULL_RESET;
+
+ /* "defer" schedule the reset task again */
+ set_bit(HCLGEVF_RESET_PENDING, &hdev->reset_state);
+ } else {
+ hdev->reset_attempts++;
+
+ /* request PF for resetting this VF via mailbox */
+ ret = hclgevf_do_reset(hdev);
+ if (ret)
+ dev_warn(&hdev->pdev->dev,
+ "VF rst fail, stack will call\n");
+ }
+ }
+
+ clear_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state);
+}
+
static void hclgevf_mailbox_service_task(struct work_struct *work)
{
struct hclgevf_dev *hdev;
clear_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state);
- hclgevf_mbx_handler(hdev);
+ hclgevf_mbx_async_handler(hdev);
clear_bit(HCLGEVF_STATE_MBX_HANDLING, &hdev->state);
}
*/
hclgevf_request_link_info(hdev);
+ hclgevf_deferred_task_schedule(hdev);
+
clear_bit(HCLGEVF_STATE_SERVICE_SCHED, &hdev->state);
}
if (!hclgevf_check_event_cause(hdev, &clearval))
goto skip_sched;
- /* schedule the VF mailbox service task, if not already scheduled */
- hclgevf_mbx_task_schedule(hdev);
+ hclgevf_mbx_handler(hdev);
hclgevf_clear_event_cause(hdev, clearval);
return hclgevf_get_tc_info(hdev);
}
+static int hclgevf_alloc_hdev(struct hnae3_ae_dev *ae_dev)
+{
+ struct pci_dev *pdev = ae_dev->pdev;
+ struct hclgevf_dev *hdev = ae_dev->priv;
+
+ hdev = devm_kzalloc(&pdev->dev, sizeof(*hdev), GFP_KERNEL);
+ if (!hdev)
+ return -ENOMEM;
+
+ hdev->pdev = pdev;
+ hdev->ae_dev = ae_dev;
+ ae_dev->priv = hdev;
+
+ return 0;
+}
+
static int hclgevf_init_roce_base_info(struct hclgevf_dev *hdev)
{
struct hnae3_handle *roce = &hdev->roce;
static void hclgevf_state_init(struct hclgevf_dev *hdev)
{
+ /* if this is on going reset then skip this initialization */
+ if (hclgevf_dev_ongoing_reset(hdev))
+ return;
+
/* setup tasks for the MBX */
INIT_WORK(&hdev->mbx_service_task, hclgevf_mailbox_service_task);
clear_bit(HCLGEVF_STATE_MBX_SERVICE_SCHED, &hdev->state);
INIT_WORK(&hdev->service_task, hclgevf_service_task);
clear_bit(HCLGEVF_STATE_SERVICE_SCHED, &hdev->state);
+ INIT_WORK(&hdev->rst_service_task, hclgevf_reset_service_task);
+
mutex_init(&hdev->mbx_resp.mbx_mutex);
/* bring the device down */
cancel_work_sync(&hdev->service_task);
if (hdev->mbx_service_task.func)
cancel_work_sync(&hdev->mbx_service_task);
+ if (hdev->rst_service_task.func)
+ cancel_work_sync(&hdev->rst_service_task);
mutex_destroy(&hdev->mbx_resp.mbx_mutex);
}
int vectors;
int i;
+ /* if this is on going reset then skip this initialization */
+ if (hclgevf_dev_ongoing_reset(hdev))
+ return 0;
+
hdev->num_msi = HCLGEVF_MAX_VF_VECTOR_NUM;
vectors = pci_alloc_irq_vectors(pdev, 1, hdev->num_msi,
{
int ret = 0;
+ /* if this is on going reset then skip this initialization */
+ if (hclgevf_dev_ongoing_reset(hdev))
+ return 0;
+
hclgevf_get_misc_vector(hdev);
ret = request_irq(hdev->misc_vector.vector_irq, hclgevf_misc_irq_handle,
struct hclgevf_hw *hw;
int ret;
+ /* check if we need to skip initialization of pci. This will happen if
+ * device is undergoing VF reset. Otherwise, we would need to
+ * re-initialize pci interface again i.e. when device is not going
+ * through *any* reset or actually undergoing full reset.
+ */
+ if (hclgevf_dev_ongoing_reset(hdev))
+ return 0;
+
ret = pci_enable_device(pdev);
if (ret) {
dev_err(&pdev->dev, "failed to enable PCI device\n");
pci_set_drvdata(pdev, NULL);
}
-static int hclgevf_init_ae_dev(struct hnae3_ae_dev *ae_dev)
+static int hclgevf_init_hdev(struct hclgevf_dev *hdev)
{
- struct pci_dev *pdev = ae_dev->pdev;
- struct hclgevf_dev *hdev;
+ struct pci_dev *pdev = hdev->pdev;
int ret;
- hdev = devm_kzalloc(&pdev->dev, sizeof(*hdev), GFP_KERNEL);
- if (!hdev)
- return -ENOMEM;
-
- hdev->pdev = pdev;
- hdev->ae_dev = ae_dev;
- ae_dev->priv = hdev;
+ /* check if device is on-going full reset(i.e. pcie as well) */
+ if (hclgevf_dev_ongoing_full_reset(hdev)) {
+ dev_warn(&pdev->dev, "device is going full reset\n");
+ hclgevf_uninit_hdev(hdev);
+ }
ret = hclgevf_pci_init(hdev);
if (ret) {
return ret;
}
-static void hclgevf_uninit_ae_dev(struct hnae3_ae_dev *ae_dev)
+static void hclgevf_uninit_hdev(struct hclgevf_dev *hdev)
{
- struct hclgevf_dev *hdev = ae_dev->priv;
-
hclgevf_cmd_uninit(hdev);
hclgevf_misc_irq_uninit(hdev);
hclgevf_state_uninit(hdev);
hclgevf_uninit_msi(hdev);
hclgevf_pci_uninit(hdev);
+}
+
+static int hclgevf_init_ae_dev(struct hnae3_ae_dev *ae_dev)
+{
+ struct pci_dev *pdev = ae_dev->pdev;
+ int ret;
+
+ ret = hclgevf_alloc_hdev(ae_dev);
+ if (ret) {
+ dev_err(&pdev->dev, "hclge device allocation failed\n");
+ return ret;
+ }
+
+ ret = hclgevf_init_hdev(ae_dev->priv);
+ if (ret)
+ dev_err(&pdev->dev, "hclge device initialization failed\n");
+
+ return ret;
+}
+
+static void hclgevf_uninit_ae_dev(struct hnae3_ae_dev *ae_dev)
+{
+ struct hclgevf_dev *hdev = ae_dev->priv;
+
+ hclgevf_uninit_hdev(hdev);
ae_dev->priv = NULL;
}
*max_rss_size = hdev->rss_size_max;
}
+static int hclgevf_get_status(struct hnae3_handle *handle)
+{
+ struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
+
+ return hdev->hw.mac.link;
+}
+
+static void hclgevf_get_ksettings_an_result(struct hnae3_handle *handle,
+ u8 *auto_neg, u32 *speed,
+ u8 *duplex)
+{
+ struct hclgevf_dev *hdev = hclgevf_ae_get_hdev(handle);
+
+ if (speed)
+ *speed = hdev->hw.mac.speed;
+ if (duplex)
+ *duplex = hdev->hw.mac.duplex;
+ if (auto_neg)
+ *auto_neg = AUTONEG_DISABLE;
+}
+
+void hclgevf_update_speed_duplex(struct hclgevf_dev *hdev, u32 speed,
+ u8 duplex)
+{
+ hdev->hw.mac.speed = speed;
+ hdev->hw.mac.duplex = duplex;
+}
+
static const struct hnae3_ae_ops hclgevf_ops = {
.init_ae_dev = hclgevf_init_ae_dev,
.uninit_ae_dev = hclgevf_uninit_ae_dev,
.get_tc_size = hclgevf_get_tc_size,
.get_fw_version = hclgevf_get_fw_version,
.set_vlan_filter = hclgevf_set_vlan_filter,
+ .reset_event = hclgevf_reset_event,
.get_channels = hclgevf_get_channels,
.get_tqps_and_rss_info = hclgevf_get_tqps_and_rss_info,
+ .get_status = hclgevf_get_status,
+ .get_ksettings_an_result = hclgevf_get_ksettings_an_result,
};
static struct hnae3_ae_algo ae_algovf = {
#define HCLGEVF_VECTOR0_RX_CMDQ_INT_B 1
#define HCLGEVF_TQP_RESET_TRY_TIMES 10
+/* Reset related Registers */
+#define HCLGEVF_FUN_RST_ING 0x20C00
+#define HCLGEVF_FUN_RST_ING_B 0
#define HCLGEVF_RSS_IND_TBL_SIZE 512
#define HCLGEVF_RSS_SET_BITMAP_MSK 0xffff
HCLGEVF_STATE_DISABLED,
/* task states */
HCLGEVF_STATE_SERVICE_SCHED,
+ HCLGEVF_STATE_RST_SERVICE_SCHED,
+ HCLGEVF_STATE_RST_HANDLING,
HCLGEVF_STATE_MBX_SERVICE_SCHED,
HCLGEVF_STATE_MBX_HANDLING,
};
struct hclgevf_mac {
u8 mac_addr[ETH_ALEN];
int link;
+ u8 duplex;
+ u32 speed;
};
struct hclgevf_hw {
struct hclgevf_rss_cfg rss_cfg;
unsigned long state;
+#define HCLGEVF_RESET_REQUESTED 0
+#define HCLGEVF_RESET_PENDING 1
+ unsigned long reset_state; /* requested, pending */
+ u32 reset_attempts;
+
u32 fw_version;
u16 num_tqps; /* num task queue pairs of this PF */
int *vector_irq;
bool accept_mta_mc; /* whether to accept mta filter multicast */
+ bool mbx_event_pending;
struct hclgevf_mbx_resp_status mbx_resp; /* mailbox response */
+ struct hclgevf_mbx_arq_ring arq; /* mailbox async rx queue */
struct timer_list service_timer;
struct work_struct service_task;
+ struct work_struct rst_service_task;
struct work_struct mbx_service_task;
struct hclgevf_tqp *htqp;
u32 flag;
};
+static inline bool hclgevf_dev_ongoing_reset(struct hclgevf_dev *hdev)
+{
+ return (hdev &&
+ (test_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) &&
+ (hdev->nic.reset_level == HNAE3_VF_RESET));
+}
+
+static inline bool hclgevf_dev_ongoing_full_reset(struct hclgevf_dev *hdev)
+{
+ return (hdev &&
+ (test_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) &&
+ (hdev->nic.reset_level == HNAE3_VF_FULL_RESET));
+}
+
int hclgevf_send_mbx_msg(struct hclgevf_dev *hdev, u16 code, u16 subcode,
const u8 *msg_data, u8 msg_len, bool need_resp,
u8 *resp_data, u16 resp_len);
void hclgevf_mbx_handler(struct hclgevf_dev *hdev);
+void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev);
+
void hclgevf_update_link_status(struct hclgevf_dev *hdev, int link_state);
+void hclgevf_update_speed_duplex(struct hclgevf_dev *hdev, u32 speed,
+ u8 duplex);
+void hclgevf_reset_task_schedule(struct hclgevf_dev *hdev);
+void hclgevf_mbx_task_schedule(struct hclgevf_dev *hdev);
#endif
struct hclge_mbx_pf_to_vf_cmd *req;
struct hclgevf_cmq_ring *crq;
struct hclgevf_desc *desc;
- u16 link_status, flag;
+ u16 *msg_q;
+ u16 flag;
u8 *temp;
int i;
desc = &crq->desc[crq->next_to_use];
req = (struct hclge_mbx_pf_to_vf_cmd *)desc->data;
+ /* synchronous messages are time critical and need preferential
+ * treatment. Therefore, we need to acknowledge all the sync
+ * responses as quickly as possible so that waiting tasks do not
+ * timeout and simultaneously queue the async messages for later
+ * prcessing in context of mailbox task i.e. the slow path.
+ */
switch (req->msg[0]) {
case HCLGE_MBX_PF_VF_RESP:
if (resp->received_resp)
}
break;
case HCLGE_MBX_LINK_STAT_CHANGE:
- link_status = le16_to_cpu(req->msg[1]);
+ case HCLGE_MBX_ASSERTING_RESET:
+ /* set this mbx event as pending. This is required as we
+ * might loose interrupt event when mbx task is busy
+ * handling. This shall be cleared when mbx task just
+ * enters handling state.
+ */
+ hdev->mbx_event_pending = true;
- /* update upper layer with new link link status */
- hclgevf_update_link_status(hdev, link_status);
+ /* we will drop the async msg if we find ARQ as full
+ * and continue with next message
+ */
+ if (hdev->arq.count >= HCLGE_MBX_MAX_ARQ_MSG_NUM) {
+ dev_warn(&hdev->pdev->dev,
+ "Async Q full, dropping msg(%d)\n",
+ req->msg[1]);
+ break;
+ }
+
+ /* tail the async message in arq */
+ msg_q = hdev->arq.msg_q[hdev->arq.tail];
+ memcpy(&msg_q[0], req->msg, HCLGE_MBX_MAX_ARQ_MSG_SIZE);
+ hclge_mbx_tail_ptr_move_arq(hdev->arq);
+ hdev->arq.count++;
+
+ hclgevf_mbx_task_schedule(hdev);
break;
default:
hclgevf_write_dev(&hdev->hw, HCLGEVF_NIC_CRQ_HEAD_REG,
crq->next_to_use);
}
+
+void hclgevf_mbx_async_handler(struct hclgevf_dev *hdev)
+{
+ u16 link_status;
+ u16 *msg_q;
+ u8 duplex;
+ u32 speed;
+ u32 tail;
+
+ /* we can safely clear it now as we are at start of the async message
+ * processing
+ */
+ hdev->mbx_event_pending = false;
+
+ tail = hdev->arq.tail;
+
+ /* process all the async queue messages */
+ while (tail != hdev->arq.head) {
+ msg_q = hdev->arq.msg_q[hdev->arq.head];
+
+ switch (msg_q[0]) {
+ case HCLGE_MBX_LINK_STAT_CHANGE:
+ link_status = le16_to_cpu(msg_q[1]);
+ memcpy(&speed, &msg_q[2], sizeof(speed));
+ duplex = (u8)le16_to_cpu(msg_q[4]);
+
+ /* update upper layer with new link link status */
+ hclgevf_update_link_status(hdev, link_status);
+ hclgevf_update_speed_duplex(hdev, speed, duplex);
+
+ break;
+ case HCLGE_MBX_ASSERTING_RESET:
+ /* PF has asserted reset hence VF should go in pending
+ * state and poll for the hardware reset status till it
+ * has been completely reset. After this stack should
+ * eventually be re-initialized.
+ */
+ hdev->nic.reset_level = HNAE3_VF_RESET;
+ set_bit(HCLGEVF_RESET_PENDING, &hdev->reset_state);
+ hclgevf_reset_task_schedule(hdev);
+
+ break;
+ default:
+ dev_err(&hdev->pdev->dev,
+ "fetched unsupported(%d) message from arq\n",
+ msg_q[0]);
+ break;
+ }
+
+ hclge_mbx_head_ptr_move_arq(hdev->arq);
+ hdev->arq.count--;
+ msg_q = hdev->arq.msg_q[hdev->arq.head];
+ }
+}
return sprintf(buf, "%d", port->logical_port_id);
}
-static DEVICE_ATTR(log_port_id, S_IRUSR | S_IRGRP | S_IROTH, ehea_show_port_id,
- NULL);
+static DEVICE_ATTR(log_port_id, 0444, ehea_show_port_id, NULL);
static void logical_port_release(struct device *dev)
{
return (ssize_t) count;
}
-static DEVICE_ATTR(probe_port, S_IWUSR, NULL, ehea_probe_port);
-static DEVICE_ATTR(remove_port, S_IWUSR, NULL, ehea_remove_port);
+static DEVICE_ATTR(probe_port, 0200, NULL, ehea_probe_port);
+static DEVICE_ATTR(remove_port, 0200, NULL, ehea_remove_port);
static int ehea_create_device_sysfs(struct platform_device *dev)
{
MODULE_PARM_DESC(rx_flush, "Flush receive buffers before use");
static bool old_large_send __read_mostly;
-module_param(old_large_send, bool, S_IRUGO);
+module_param(old_large_send, bool, 0444);
MODULE_PARM_DESC(old_large_send,
"Use old large send method on firmware that supports the new method");
u64 tx_entries;
int i;
- if (!tx_pool && !tx_pool->tx_buff)
+ if (!tx_pool || !tx_pool->tx_buff)
return;
tx_entries = tx_pool->num_buffers;
will be called i40evf. MSI-X interrupt support is required
for this driver to work correctly.
+config ICE
+ tristate "Intel(R) Ethernet Connection E800 Series Support"
+ default n
+ depends on PCI_MSI
+ ---help---
+ This driver supports Intel(R) Ethernet Connection E800 Series of
+ devices. For more information on how to identify your adapter, go
+ to the Adapter & Driver ID Guide that can be located at:
+
+ <http://support.intel.com>
+
+ To compile this driver as a module, choose M here. The module
+ will be called ice.
+
config FM10K
tristate "Intel(R) FM10000 Ethernet Switch Host Interface Support"
default n
obj-$(CONFIG_IXGB) += ixgb/
obj-$(CONFIG_I40EVF) += i40evf/
obj-$(CONFIG_FM10K) += fm10k/
+obj-$(CONFIG_ICE) += ice/
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel PRO/100 Linux driver
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel PRO/1000 Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/1000 Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2006 Intel Corporation.
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
Intel PRO/1000 Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/1000 Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel PRO/1000 Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/1000 Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel PRO/1000 Linux driver
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel PRO/1000 Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
#define E1000_ICR_RXDMT0 0x00000010 /* Rx desc min. threshold (0) */
#define E1000_ICR_RXO 0x00000040 /* Receiver Overrun */
#define E1000_ICR_RXT0 0x00000080 /* Rx timer intr (ring 0) */
+#define E1000_ICR_MDAC 0x00000200 /* MDIO Access Complete */
+#define E1000_ICR_SRPD 0x00010000 /* Small Receive Packet Detected */
+#define E1000_ICR_ACK 0x00020000 /* Receive ACK Frame Detected */
+#define E1000_ICR_MNG 0x00040000 /* Manageability Event Detected */
#define E1000_ICR_ECCER 0x00400000 /* Uncorrectable ECC Error */
/* If this bit asserted, the driver should claim the interrupt */
#define E1000_ICR_INT_ASSERTED 0x80000000
#define E1000_ICR_RXQ1 0x00200000 /* Rx Queue 1 Interrupt */
#define E1000_ICR_TXQ0 0x00400000 /* Tx Queue 0 Interrupt */
#define E1000_ICR_TXQ1 0x00800000 /* Tx Queue 1 Interrupt */
-#define E1000_ICR_OTHER 0x01000000 /* Other Interrupts */
+#define E1000_ICR_OTHER 0x01000000 /* Other Interrupt */
/* PBA ECC Register */
#define E1000_PBA_ECC_COUNTER_MASK 0xFFF00000 /* ECC counter mask */
E1000_IMS_RXSEQ | \
E1000_IMS_LSC)
+/* These are all of the events related to the OTHER interrupt.
+ */
+#define IMS_OTHER_MASK ( \
+ E1000_IMS_LSC | \
+ E1000_IMS_RXO | \
+ E1000_IMS_MDAC | \
+ E1000_IMS_SRPD | \
+ E1000_IMS_ACK | \
+ E1000_IMS_MNG)
+
/* Interrupt Mask Set */
#define E1000_IMS_TXDW E1000_ICR_TXDW /* Transmit desc written back */
#define E1000_IMS_LSC E1000_ICR_LSC /* Link Status Change */
#define E1000_IMS_RXSEQ E1000_ICR_RXSEQ /* Rx sequence error */
#define E1000_IMS_RXDMT0 E1000_ICR_RXDMT0 /* Rx desc min. threshold */
+#define E1000_IMS_RXO E1000_ICR_RXO /* Receiver Overrun */
#define E1000_IMS_RXT0 E1000_ICR_RXT0 /* Rx timer intr */
+#define E1000_IMS_MDAC E1000_ICR_MDAC /* MDIO Access Complete */
+#define E1000_IMS_SRPD E1000_ICR_SRPD /* Small Receive Packet */
+#define E1000_IMS_ACK E1000_ICR_ACK /* Receive ACK Frame Detected */
+#define E1000_IMS_MNG E1000_ICR_MNG /* Manageability Event */
#define E1000_IMS_ECCER E1000_ICR_ECCER /* Uncorrectable ECC Error */
#define E1000_IMS_RXQ0 E1000_ICR_RXQ0 /* Rx Queue 0 Interrupt */
#define E1000_IMS_RXQ1 E1000_ICR_RXQ1 /* Rx Queue 1 Interrupt */
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
* Checks to see of the link status of the hardware has changed. If a
* change in link status has been detected, then we read the PHY registers
* to get the current speed/duplex if link exists.
- *
- * Returns a negative error code (-E1000_ERR_*) or 0 (link down) or 1 (link
- * up).
**/
static s32 e1000_check_for_copper_link_ich8lan(struct e1000_hw *hw)
{
* Change or Rx Sequence Error interrupt.
*/
if (!mac->get_link_status)
- return 1;
+ return 0;
+ mac->get_link_status = false;
/* First we want to see if the MII Status Register reports
* link. If so, then we want to get the current speed/duplex
*/
ret_val = e1000e_phy_has_link_generic(hw, 1, 0, &link);
if (ret_val)
- return ret_val;
+ goto out;
if (hw->mac.type == e1000_pchlan) {
ret_val = e1000_k1_gig_workaround_hv(hw, link);
if (ret_val)
- return ret_val;
+ goto out;
}
/* When connected at 10Mbps half-duplex, some parts are excessively
ret_val = hw->phy.ops.acquire(hw);
if (ret_val)
- return ret_val;
+ goto out;
if (hw->mac.type == e1000_pch2lan)
emi_addr = I82579_RX_CONFIG;
hw->phy.ops.release(hw);
if (ret_val)
- return ret_val;
+ goto out;
if (hw->mac.type >= e1000_pch_spt) {
u16 data;
if (speed == SPEED_1000) {
ret_val = hw->phy.ops.acquire(hw);
if (ret_val)
- return ret_val;
+ goto out;
ret_val = e1e_rphy_locked(hw,
PHY_REG(776, 20),
&data);
if (ret_val) {
hw->phy.ops.release(hw);
- return ret_val;
+ goto out;
}
ptr_gap = (data & (0x3FF << 2)) >> 2;
}
hw->phy.ops.release(hw);
if (ret_val)
- return ret_val;
+ goto out;
} else {
ret_val = hw->phy.ops.acquire(hw);
if (ret_val)
- return ret_val;
+ goto out;
ret_val = e1e_wphy_locked(hw,
PHY_REG(776, 20),
0xC023);
hw->phy.ops.release(hw);
if (ret_val)
- return ret_val;
+ goto out;
}
}
(hw->adapter->pdev->device == E1000_DEV_ID_PCH_I218_V3)) {
ret_val = e1000_k1_workaround_lpt_lp(hw, link);
if (ret_val)
- return ret_val;
+ goto out;
}
if (hw->mac.type >= e1000_pch_lpt) {
/* Set platform power management values for
*/
ret_val = e1000_platform_pm_pch_lpt(hw, link);
if (ret_val)
- return ret_val;
+ goto out;
}
/* Clear link partner's EEE ability */
}
if (!link)
- return 0; /* No link detected */
-
- mac->get_link_status = false;
+ goto out;
switch (hw->mac.type) {
case e1000_pch2lan:
* different link partner.
*/
ret_val = e1000e_config_fc_after_link_up(hw);
- if (ret_val) {
+ if (ret_val)
e_dbg("Error configuring flow control\n");
- return ret_val;
- }
- return 1;
+ return ret_val;
+
+out:
+ mac->get_link_status = true;
+ return ret_val;
}
static s32 e1000_get_variants_ich8lan(struct e1000_adapter *adapter)
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
* Checks to see of the link status of the hardware has changed. If a
* change in link status has been detected, then we read the PHY registers
* to get the current speed/duplex if link exists.
- *
- * Returns a negative error code (-E1000_ERR_*) or 0 (link down) or 1 (link
- * up).
**/
s32 e1000e_check_for_copper_link(struct e1000_hw *hw)
{
* Change or Rx Sequence Error interrupt.
*/
if (!mac->get_link_status)
- return 1;
+ return 0;
+ mac->get_link_status = false;
/* First we want to see if the MII Status Register reports
* link. If so, then we want to get the current speed/duplex
* of the PHY.
*/
ret_val = e1000e_phy_has_link_generic(hw, 1, 0, &link);
- if (ret_val)
- return ret_val;
-
- if (!link)
- return 0; /* No link detected */
-
- mac->get_link_status = false;
+ if (ret_val || !link)
+ goto out;
/* Check if there was DownShift, must be checked
* immediately after link-up
* different link partner.
*/
ret_val = e1000e_config_fc_after_link_up(hw);
- if (ret_val) {
+ if (ret_val)
e_dbg("Error configuring flow control\n");
- return ret_val;
- }
- return 1;
+ return ret_val;
+
+out:
+ mac->get_link_status = true;
+ return ret_val;
}
/**
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
struct net_device *netdev = data;
struct e1000_adapter *adapter = netdev_priv(netdev);
struct e1000_hw *hw = &adapter->hw;
- u32 icr;
- bool enable = true;
-
- icr = er32(ICR);
- if (icr & E1000_ICR_RXO) {
- ew32(ICR, E1000_ICR_RXO);
- enable = false;
- /* napi poll will re-enable Other, make sure it runs */
- if (napi_schedule_prep(&adapter->napi)) {
- adapter->total_rx_bytes = 0;
- adapter->total_rx_packets = 0;
- __napi_schedule(&adapter->napi);
- }
- }
+ u32 icr = er32(ICR);
+
+ if (icr & adapter->eiac_mask)
+ ew32(ICS, (icr & adapter->eiac_mask));
+
if (icr & E1000_ICR_LSC) {
- ew32(ICR, E1000_ICR_LSC);
hw->mac.get_link_status = true;
/* guard against interrupt when we're going down */
if (!test_bit(__E1000_DOWN, &adapter->state))
mod_timer(&adapter->watchdog_timer, jiffies + 1);
}
- if (enable && !test_bit(__E1000_DOWN, &adapter->state))
- ew32(IMS, E1000_IMS_OTHER);
+ if (!test_bit(__E1000_DOWN, &adapter->state))
+ ew32(IMS, E1000_IMS_OTHER | IMS_OTHER_MASK);
return IRQ_HANDLED;
}
hw->hw_addr + E1000_EITR_82574(vector));
else
writel(1, hw->hw_addr + E1000_EITR_82574(vector));
- adapter->eiac_mask |= E1000_IMS_OTHER;
/* Cause Tx interrupts on every write back */
ivar |= BIT(31);
if (adapter->msix_entries) {
ew32(EIAC_82574, adapter->eiac_mask & E1000_EIAC_MASK_82574);
- ew32(IMS, adapter->eiac_mask | E1000_IMS_LSC);
+ ew32(IMS, adapter->eiac_mask | E1000_IMS_OTHER |
+ IMS_OTHER_MASK);
} else if (hw->mac.type >= e1000_pch_lpt) {
ew32(IMS, IMS_ENABLE_MASK | E1000_IMS_ECCER);
} else {
{
struct pci_dev *pdev = adapter->pdev;
- ring->desc = dma_alloc_coherent(&pdev->dev, ring->size, &ring->dma,
- GFP_KERNEL);
+ ring->desc = dma_zalloc_coherent(&pdev->dev, ring->size, &ring->dma,
+ GFP_KERNEL);
if (!ring->desc)
return -ENOMEM;
napi_complete_done(napi, work_done);
if (!test_bit(__E1000_DOWN, &adapter->state)) {
if (adapter->msix_entries)
- ew32(IMS, adapter->rx_ring->ims_val |
- E1000_IMS_OTHER);
+ ew32(IMS, adapter->rx_ring->ims_val);
else
e1000_irq_enable(adapter);
}
case e1000_media_type_copper:
if (hw->mac.get_link_status) {
ret_val = hw->mac.ops.check_for_link(hw);
- link_active = ret_val > 0;
+ link_active = !hw->mac.get_link_status;
} else {
link_active = true;
}
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel PRO/1000 Linux driver
* Copyright(c) 1999 - 2015 Intel Corporation.
*
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel(R) Ethernet Switch Host Interface Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2017 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2018 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2017 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2017 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2017 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2017 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2018 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2018 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2018 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2017 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2018 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Ethernet Switch Host Interface Driver
* Copyright(c) 2013 - 2016 Intel Corporation.
*
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
__I40E_BAD_EEPROM,
__I40E_DOWN_REQUESTED,
__I40E_FD_FLUSH_REQUESTED,
+ __I40E_FD_ATR_AUTO_DISABLED,
+ __I40E_FD_SB_AUTO_DISABLED,
__I40E_RESET_FAILED,
__I40E_PORT_SUSPENDED,
__I40E_VF_DISABLE,
+ __I40E_MACVLAN_SYNC_PENDING,
+ __I40E_UDP_FILTER_SYNC_PENDING,
+ __I40E_TEMP_LINK_POLLING,
+ __I40E_CLIENT_SERVICE_REQUESTED,
+ __I40E_CLIENT_L2_CHANGE,
+ __I40E_CLIENT_RESET,
/* This must be last as it determines the size of the BITMAP */
__I40E_STATE_SIZE__,
};
#define I40E_HW_RESTART_AUTONEG BIT(18)
#define I40E_HW_STOPPABLE_FW_LLDP BIT(19)
- u64 flags;
-#define I40E_FLAG_RX_CSUM_ENABLED BIT_ULL(0)
-#define I40E_FLAG_MSI_ENABLED BIT_ULL(1)
-#define I40E_FLAG_MSIX_ENABLED BIT_ULL(2)
-#define I40E_FLAG_RSS_ENABLED BIT_ULL(3)
-#define I40E_FLAG_VMDQ_ENABLED BIT_ULL(4)
-#define I40E_FLAG_FILTER_SYNC BIT_ULL(5)
-#define I40E_FLAG_SRIOV_ENABLED BIT_ULL(6)
-#define I40E_FLAG_DCB_CAPABLE BIT_ULL(7)
-#define I40E_FLAG_DCB_ENABLED BIT_ULL(8)
-#define I40E_FLAG_FD_SB_ENABLED BIT_ULL(9)
-#define I40E_FLAG_FD_ATR_ENABLED BIT_ULL(10)
-#define I40E_FLAG_FD_SB_AUTO_DISABLED BIT_ULL(11)
-#define I40E_FLAG_FD_ATR_AUTO_DISABLED BIT_ULL(12)
-#define I40E_FLAG_MFP_ENABLED BIT_ULL(13)
-#define I40E_FLAG_UDP_FILTER_SYNC BIT_ULL(14)
-#define I40E_FLAG_HW_ATR_EVICT_ENABLED BIT_ULL(15)
-#define I40E_FLAG_VEB_MODE_ENABLED BIT_ULL(16)
-#define I40E_FLAG_VEB_STATS_ENABLED BIT_ULL(17)
-#define I40E_FLAG_LINK_POLLING_ENABLED BIT_ULL(18)
-#define I40E_FLAG_TRUE_PROMISC_SUPPORT BIT_ULL(19)
-#define I40E_FLAG_TEMP_LINK_POLLING BIT_ULL(20)
-#define I40E_FLAG_LEGACY_RX BIT_ULL(21)
-#define I40E_FLAG_PTP BIT_ULL(22)
-#define I40E_FLAG_IWARP_ENABLED BIT_ULL(23)
-#define I40E_FLAG_SERVICE_CLIENT_REQUESTED BIT_ULL(24)
-#define I40E_FLAG_CLIENT_L2_CHANGE BIT_ULL(25)
-#define I40E_FLAG_CLIENT_RESET BIT_ULL(26)
-#define I40E_FLAG_LINK_DOWN_ON_CLOSE_ENABLED BIT_ULL(27)
-#define I40E_FLAG_SOURCE_PRUNING_DISABLED BIT_ULL(28)
-#define I40E_FLAG_TC_MQPRIO BIT_ULL(29)
-#define I40E_FLAG_FD_SB_INACTIVE BIT_ULL(30)
-#define I40E_FLAG_FD_SB_TO_CLOUD_FILTER BIT_ULL(31)
-#define I40E_FLAG_DISABLE_FW_LLDP BIT_ULL(32)
+ u32 flags;
+#define I40E_FLAG_RX_CSUM_ENABLED BIT(0)
+#define I40E_FLAG_MSI_ENABLED BIT(1)
+#define I40E_FLAG_MSIX_ENABLED BIT(2)
+#define I40E_FLAG_RSS_ENABLED BIT(3)
+#define I40E_FLAG_VMDQ_ENABLED BIT(4)
+#define I40E_FLAG_SRIOV_ENABLED BIT(5)
+#define I40E_FLAG_DCB_CAPABLE BIT(6)
+#define I40E_FLAG_DCB_ENABLED BIT(7)
+#define I40E_FLAG_FD_SB_ENABLED BIT(8)
+#define I40E_FLAG_FD_ATR_ENABLED BIT(9)
+#define I40E_FLAG_MFP_ENABLED BIT(10)
+#define I40E_FLAG_HW_ATR_EVICT_ENABLED BIT(11)
+#define I40E_FLAG_VEB_MODE_ENABLED BIT(12)
+#define I40E_FLAG_VEB_STATS_ENABLED BIT(13)
+#define I40E_FLAG_LINK_POLLING_ENABLED BIT(14)
+#define I40E_FLAG_TRUE_PROMISC_SUPPORT BIT(15)
+#define I40E_FLAG_LEGACY_RX BIT(16)
+#define I40E_FLAG_PTP BIT(17)
+#define I40E_FLAG_IWARP_ENABLED BIT(18)
+#define I40E_FLAG_LINK_DOWN_ON_CLOSE_ENABLED BIT(19)
+#define I40E_FLAG_SOURCE_PRUNING_DISABLED BIT(20)
+#define I40E_FLAG_TC_MQPRIO BIT(21)
+#define I40E_FLAG_FD_SB_INACTIVE BIT(22)
+#define I40E_FLAG_FD_SB_TO_CLOUD_FILTER BIT(23)
+#define I40E_FLAG_DISABLE_FW_LLDP BIT(24)
struct i40e_client_instance *cinst;
bool stat_offsets_loaded;
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
struct i40e_vsi *vsi = pf->vsi[pf->lan_vsi];
int ret = 0;
- if (!(pf->flags & I40E_FLAG_SERVICE_CLIENT_REQUESTED))
+ if (!test_and_clear_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state))
return;
- pf->flags &= ~I40E_FLAG_SERVICE_CLIENT_REQUESTED;
cdev = pf->cinst;
/* If we're down or resetting, just bail */
* added, we can schedule a subtask to go initiate the clients if
* they can be launched at probe time.
*/
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
out:
pf = ldev->pf;
i40e_client_add_instance(pf);
/* Start the client subtask */
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
}
mutex_unlock(&i40e_device_mutex);
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
if (!(pf->flags & I40E_FLAG_FD_SB_ENABLED))
return -EOPNOTSUPP;
- if (pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED)
+ if (test_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state))
return -ENOSPC;
if (test_bit(__I40E_RESET_RECOVERY_PENDING, pf->state) ||
}
}
- /* Compare and exchange the new flags into place. If we failed, that
- * is if cmpxchg returns anything but the old value, this means that
- * something else has modified the flags variable since we copied it
- * originally. We'll just punt with an error and log something in the
- * message buffer.
- *
- * This is the point of no return for this function. We need to have
- * checked any discrepancies or misconfigurations and returned
- * EOPNOTSUPP before updating pf->flags here.
+ /* Now that we've checked to ensure that the new flags are valid, load
+ * them into place. Since we only modify flags either (a) during
+ * initialization or (b) while holding the RTNL lock, we don't need
+ * anything fancy here.
*/
- if (cmpxchg64(&pf->flags, orig_flags, new_flags) != orig_flags) {
- dev_warn(&pf->pdev->dev,
- "Unable to update pf->flags as it was modified by another thread...\n");
- return -EAGAIN;
- }
+ pf->flags = new_flags;
/* Process any additional changes needed as a result of flag changes.
* The changed_flags value reflects the list of bits that were
/* Flush current ATR settings if ATR was disabled */
if ((changed_flags & I40E_FLAG_FD_ATR_ENABLED) &&
!(pf->flags & I40E_FLAG_FD_ATR_ENABLED)) {
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
set_bit(__I40E_FD_FLUSH_REQUESTED, pf->state);
}
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
&osd->rx_lpi_count, &nsd->rx_lpi_count);
if (pf->flags & I40E_FLAG_FD_SB_ENABLED &&
- !(pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED))
+ !test_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state))
nsd->fd_sb_status = true;
else
nsd->fd_sb_status = false;
if (pf->flags & I40E_FLAG_FD_ATR_ENABLED &&
- !(pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED))
+ !test_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
nsd->fd_atr_status = true;
else
nsd->fd_atr_status = false;
hash_add(vsi->mac_filter_hash, &f->hlist, key);
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- vsi->back->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, vsi->back->state);
}
/* If we're asked to add a filter that has been marked for removal, it
}
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- vsi->back->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, vsi->state);
}
/**
/* check for other flag changes */
if (vsi->current_netdev_flags != vsi->netdev->flags) {
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- vsi->back->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, vsi->back->state);
}
}
{
int v;
- if (!pf || !(pf->flags & I40E_FLAG_FILTER_SYNC))
+ if (!pf)
+ return;
+ if (!test_and_clear_bit(__I40E_MACVLAN_SYNC_PENDING, pf->state))
return;
- pf->flags &= ~I40E_FLAG_FILTER_SYNC;
for (v = 0; v < pf->num_alloc_vsi; v++) {
if (pf->vsi[v] &&
if (ret) {
/* come back and try again later */
- pf->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING,
+ pf->state);
break;
}
}
netdev->mtu = new_mtu;
if (netif_running(netdev))
i40e_vsi_reinit_locked(vsi);
- pf->flags |= (I40E_FLAG_SERVICE_CLIENT_REQUESTED |
- I40E_FLAG_CLIENT_L2_CHANGE);
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
+ set_bit(__I40E_CLIENT_L2_CHANGE, pf->state);
return 0;
}
i40e_vsi_free_tx_resources(vsi);
i40e_vsi_free_rx_resources(vsi);
vsi->current_netdev_flags = 0;
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
if (test_bit(__I40E_RESET_RECOVERY_PENDING, pf->state))
- pf->flags |= I40E_FLAG_CLIENT_RESET;
+ set_bit(__I40E_CLIENT_RESET, pf->state);
}
/**
/* On the next run of the service_task, notify any clients of the new
* opened netdev
*/
- pf->flags |= I40E_FLAG_SERVICE_CLIENT_REQUESTED;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
return 0;
i40e_service_event_schedule(pf);
} else {
i40e_pf_unquiesce_all_vsi(pf);
- pf->flags |= (I40E_FLAG_SERVICE_CLIENT_REQUESTED |
- I40E_FLAG_CLIENT_L2_CHANGE);
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
+ set_bit(__I40E_CLIENT_L2_CHANGE, pf->state);
}
exit:
**/
static void i40e_reenable_fdir_sb(struct i40e_pf *pf)
{
- if (pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED) {
- pf->flags &= ~I40E_FLAG_FD_SB_AUTO_DISABLED;
+ if (test_and_clear_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state))
if ((pf->flags & I40E_FLAG_FD_SB_ENABLED) &&
(I40E_DEBUG_FD & pf->hw.debug_mask))
dev_info(&pf->pdev->dev, "FD Sideband/ntuple is being enabled since we have space in the table now\n");
- }
}
/**
**/
static void i40e_reenable_fdir_atr(struct i40e_pf *pf)
{
- if (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED) {
+ if (test_and_clear_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state)) {
/* ATR uses the same filtering logic as SB rules. It only
* functions properly if the input set mask is at the default
* settings. It is safe to restore the default input set
I40E_L3_SRC_MASK | I40E_L3_DST_MASK |
I40E_L4_SRC_MASK | I40E_L4_DST_MASK);
- pf->flags &= ~I40E_FLAG_FD_ATR_AUTO_DISABLED;
if ((pf->flags & I40E_FLAG_FD_ATR_ENABLED) &&
(I40E_DEBUG_FD & pf->hw.debug_mask))
dev_info(&pf->pdev->dev, "ATR is being enabled since we have space in the table and there are no conflicting ntuple rules\n");
}
pf->fd_flush_timestamp = jiffies;
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
/* flush all filters */
wr32(&pf->hw, I40E_PFQF_CTL_1,
I40E_PFQF_CTL_1_CLEARFDTABLE_MASK);
/* replay sideband filters */
i40e_fdir_filter_restore(pf->vsi[pf->lan_vsi]);
if (!disable_atr && !pf->fd_tcp4_filter_cnt)
- pf->flags &= ~I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ clear_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
clear_bit(__I40E_FD_FLUSH_REQUESTED, pf->state);
if (I40E_DEBUG_FD & pf->hw.debug_mask)
dev_info(&pf->pdev->dev, "FD Filter table flushed and FD-SB replayed.\n");
/* On success, disable temp link polling */
if (status == I40E_SUCCESS) {
- if (pf->flags & I40E_FLAG_TEMP_LINK_POLLING)
- pf->flags &= ~I40E_FLAG_TEMP_LINK_POLLING;
+ clear_bit(__I40E_TEMP_LINK_POLLING, pf->state);
} else {
/* Enable link polling temporarily until i40e_get_link_status
* returns I40E_SUCCESS
*/
- pf->flags |= I40E_FLAG_TEMP_LINK_POLLING;
+ set_bit(__I40E_TEMP_LINK_POLLING, pf->state);
dev_dbg(&pf->pdev->dev, "couldn't get link state, status: %d\n",
status);
return;
pf->service_timer_previous = jiffies;
if ((pf->flags & I40E_FLAG_LINK_POLLING_ENABLED) ||
- (pf->flags & I40E_FLAG_TEMP_LINK_POLLING))
+ test_bit(__I40E_TEMP_LINK_POLLING, pf->state))
i40e_link_event(pf);
/* Update the stats for active netdevs so the network stack
pf->pending_udp_bitmap |= BIT_ULL(i);
}
- pf->flags |= I40E_FLAG_UDP_FILTER_SYNC;
+ set_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state);
}
/**
u16 port;
int i;
- if (!(pf->flags & I40E_FLAG_UDP_FILTER_SYNC))
+ if (!test_and_clear_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state))
return;
- pf->flags &= ~I40E_FLAG_UDP_FILTER_SYNC;
-
for (i = 0; i < I40E_MAX_PF_UDP_OFFLOAD_PORTS; i++) {
if (pf->pending_udp_bitmap & BIT_ULL(i)) {
pf->pending_udp_bitmap &= ~BIT_ULL(i);
i40e_vc_process_vflr_event(pf);
i40e_watchdog_subtask(pf);
i40e_fdir_reinit_subtask(pf);
- if (pf->flags & I40E_FLAG_CLIENT_RESET) {
+ if (test_and_clear_bit(__I40E_CLIENT_RESET, pf->state)) {
/* Client subtask will reopen next time through. */
i40e_notify_client_of_netdev_close(pf->vsi[pf->lan_vsi], true);
- pf->flags &= ~I40E_FLAG_CLIENT_RESET;
} else {
i40e_client_subtask(pf);
- if (pf->flags & I40E_FLAG_CLIENT_L2_CHANGE) {
+ if (test_and_clear_bit(__I40E_CLIENT_L2_CHANGE,
+ pf->state))
i40e_notify_client_of_l2_param_changes(
pf->vsi[pf->lan_vsi]);
- pf->flags &= ~I40E_FLAG_CLIENT_L2_CHANGE;
- }
}
i40e_sync_filters_subtask(pf);
i40e_sync_udp_filters_subtask(pf);
need_reset = true;
i40e_fdir_filter_exit(pf);
}
- pf->flags &= ~(I40E_FLAG_FD_SB_ENABLED |
- I40E_FLAG_FD_SB_AUTO_DISABLED);
+ pf->flags &= ~I40E_FLAG_FD_SB_ENABLED;
+ clear_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state);
pf->flags |= I40E_FLAG_FD_SB_INACTIVE;
/* reset fd counters */
pf->fd_add_err = 0;
pf->fd_atr_cnt = 0;
/* if ATR was auto disabled it can be re-enabled. */
- if (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED) {
- pf->flags &= ~I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ if (test_and_clear_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
if ((pf->flags & I40E_FLAG_FD_ATR_ENABLED) &&
(I40E_DEBUG_FD & pf->hw.debug_mask))
dev_info(&pf->pdev->dev, "ATR re-enabled.\n");
- }
}
return need_reset;
}
/* New port: add it and mark its index in the bitmap */
pf->udp_ports[next_idx].port = port;
pf->pending_udp_bitmap |= BIT_ULL(next_idx);
- pf->flags |= I40E_FLAG_UDP_FILTER_SYNC;
+ set_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state);
}
/**
*/
pf->udp_ports[idx].port = 0;
pf->pending_udp_bitmap |= BIT_ULL(idx);
- pf->flags |= I40E_FLAG_UDP_FILTER_SYNC;
+ set_bit(__I40E_UDP_FILTER_SYNC_PENDING, pf->state);
return;
not_found:
.ndo_bridge_getlink = i40e_ndo_bridge_getlink,
.ndo_bridge_setlink = i40e_ndo_bridge_setlink,
.ndo_bpf = i40e_xdp,
+ .ndo_xdp_xmit = i40e_xdp_xmit,
+ .ndo_xdp_flush = i40e_xdp_flush,
};
/**
if (f_count) {
vsi->flags |= I40E_VSI_FLAG_FILTER_CHANGED;
- pf->flags |= I40E_FLAG_FILTER_SYNC;
+ set_bit(__I40E_MACVLAN_SYNC_PENDING, pf->state);
}
/* Update VSI BW information */
if (pf->wol_en && (pf->hw_features & I40E_HW_WOL_MC_MAGIC_PKT_WAKE))
i40e_enable_mc_magic_wake(pf);
- i40e_prep_for_reset(pf, false);
+ /* Since we're going to destroy queues during the
+ * i40e_clear_interrupt_scheme() we should hold the RTNL lock for this
+ * whole section
+ */
+ rtnl_lock();
+
+ i40e_prep_for_reset(pf, true);
wr32(hw, I40E_PFPM_APM, (pf->wol_en ? I40E_PFPM_APM_APME_MASK : 0));
wr32(hw, I40E_PFPM_WUFC, (pf->wol_en ? I40E_PFPM_WUFC_MAG_MASK : 0));
*/
i40e_clear_interrupt_scheme(pf);
+ rtnl_unlock();
+
return 0;
}
if (!test_bit(__I40E_SUSPENDED, pf->state))
return 0;
+ /* We need to hold the RTNL lock prior to restoring interrupt schemes,
+ * since we're going to be restoring queues
+ */
+ rtnl_lock();
+
/* We cleared the interrupt scheme when we suspended, so we need to
* restore it now to resume device functionality.
*/
}
clear_bit(__I40E_DOWN, pf->state);
- i40e_reset_and_rebuild(pf, false, false);
+ i40e_reset_and_rebuild(pf, false, true);
+
+ rtnl_unlock();
/* Clear suspended state last after everything is recovered */
clear_bit(__I40E_SUSPENDED, pf->state);
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel(R) 40-10 Gigabit Ethernet Connection Network Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
if ((pf->flags & I40E_FLAG_FD_ATR_ENABLED) &&
I40E_DEBUG_FD & pf->hw.debug_mask)
dev_info(&pf->pdev->dev, "Forcing ATR off, sideband rules for TCP/IPv4 flow being applied\n");
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
} else {
pf->fd_tcp4_filter_cnt--;
}
pf->fd_atr_cnt = i40e_get_current_atr_cnt(pf);
if ((rx_desc->wb.qword0.hi_dword.fd_id == 0) &&
- pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED) {
- pf->flags |= I40E_FLAG_FD_ATR_AUTO_DISABLED;
+ test_bit(__I40E_FD_SB_AUTO_DISABLED, pf->state)) {
+ /* These set_bit() calls aren't atomic with the
+ * test_bit() here, but worse case we potentially
+ * disable ATR and queue a flush right after SB
+ * support is re-enabled. That shouldn't cause an
+ * issue in practice
+ */
+ set_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state);
set_bit(__I40E_FD_FLUSH_REQUESTED, pf->state);
}
*/
if (fcnt_prog >= (fcnt_avail - I40E_FDIR_BUFFER_FULL_MARGIN)) {
if ((pf->flags & I40E_FLAG_FD_SB_ENABLED) &&
- !(pf->flags & I40E_FLAG_FD_SB_AUTO_DISABLED)) {
- pf->flags |= I40E_FLAG_FD_SB_AUTO_DISABLED;
+ !test_and_set_bit(__I40E_FD_SB_AUTO_DISABLED,
+ pf->state))
if (I40E_DEBUG_FD & pf->hw.debug_mask)
dev_warn(&pdev->dev, "FD filter space full, new ntuple rules will not be added\n");
- }
}
} else if (error == BIT(I40E_RX_PROG_STATUS_DESC_NO_FD_ENTRY_SHIFT)) {
if (I40E_DEBUG_FD & pf->hw.debug_mask)
bi->dma = dma;
bi->page = page;
bi->page_offset = i40e_rx_offset(rx_ring);
-
- /* initialize pagecnt_bias to 1 representing we fully own page */
- bi->pagecnt_bias = 1;
+ page_ref_add(page, USHRT_MAX - 1);
+ bi->pagecnt_bias = USHRT_MAX;
return true;
}
* the pagecnt_bias and page count so that we fully restock the
* number of references the driver holds.
*/
- if (unlikely(!pagecnt_bias)) {
- page_ref_add(page, USHRT_MAX);
+ if (unlikely(pagecnt_bias == 1)) {
+ page_ref_add(page, USHRT_MAX - 1);
rx_buffer->pagecnt_bias = USHRT_MAX;
}
static struct sk_buff *i40e_run_xdp(struct i40e_ring *rx_ring,
struct xdp_buff *xdp)
{
- int result = I40E_XDP_PASS;
+ int err, result = I40E_XDP_PASS;
struct i40e_ring *xdp_ring;
struct bpf_prog *xdp_prog;
u32 act;
xdp_ring = rx_ring->vsi->xdp_rings[rx_ring->queue_index];
result = i40e_xmit_xdp_ring(xdp, xdp_ring);
break;
+ case XDP_REDIRECT:
+ err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog);
+ result = !err ? I40E_XDP_TX : I40E_XDP_CONSUMED;
+ break;
default:
bpf_warn_invalid_xdp_action(act);
case XDP_ABORTED:
#endif
}
+static inline void i40e_xdp_ring_update_tail(struct i40e_ring *xdp_ring)
+{
+ /* Force memory writes to complete before letting h/w
+ * know there are new descriptors to fetch.
+ */
+ wmb();
+ writel_relaxed(xdp_ring->next_to_use, xdp_ring->tail);
+}
+
/**
* i40e_clean_rx_irq - Clean completed descriptors from Rx ring - bounce buf
* @rx_ring: rx descriptor ring to transact packets on
}
if (xdp_xmit) {
- struct i40e_ring *xdp_ring;
+ struct i40e_ring *xdp_ring =
+ rx_ring->vsi->xdp_rings[rx_ring->queue_index];
- xdp_ring = rx_ring->vsi->xdp_rings[rx_ring->queue_index];
-
- /* Force memory writes to complete before letting h/w
- * know there are new descriptors to fetch.
- */
- wmb();
-
- writel(xdp_ring->next_to_use, xdp_ring->tail);
+ i40e_xdp_ring_update_tail(xdp_ring);
+ xdp_do_flush_map();
}
rx_ring->skb = skb;
if (!(pf->flags & I40E_FLAG_FD_ATR_ENABLED))
return;
- if (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED)
+ if (test_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
return;
/* if sampling is disabled do nothing */
th = (struct tcphdr *)(hdr.network + hlen);
/* Due to lack of space, no more new filters can be programmed */
- if (th->syn && (pf->flags & I40E_FLAG_FD_ATR_AUTO_DISABLED))
+ if (th->syn && test_bit(__I40E_FD_ATR_AUTO_DISABLED, pf->state))
return;
if (pf->flags & I40E_FLAG_HW_ATR_EVICT_ENABLED) {
/* HW ATR eviction will take care of removing filters on FIN
return i40e_xmit_frame_ring(skb, tx_ring);
}
+
+/**
+ * i40e_xdp_xmit - Implements ndo_xdp_xmit
+ * @dev: netdev
+ * @xdp: XDP buffer
+ *
+ * Returns Zero if sent, else an error code
+ **/
+int i40e_xdp_xmit(struct net_device *dev, struct xdp_buff *xdp)
+{
+ struct i40e_netdev_priv *np = netdev_priv(dev);
+ unsigned int queue_index = smp_processor_id();
+ struct i40e_vsi *vsi = np->vsi;
+ int err;
+
+ if (test_bit(__I40E_VSI_DOWN, vsi->state))
+ return -ENETDOWN;
+
+ if (!i40e_enabled_xdp_vsi(vsi) || queue_index >= vsi->num_queue_pairs)
+ return -ENXIO;
+
+ err = i40e_xmit_xdp_ring(xdp, vsi->xdp_rings[queue_index]);
+ if (err != I40E_XDP_TX)
+ return -ENOSPC;
+
+ return 0;
+}
+
+/**
+ * i40e_xdp_flush - Implements ndo_xdp_flush
+ * @dev: netdev
+ **/
+void i40e_xdp_flush(struct net_device *dev)
+{
+ struct i40e_netdev_priv *np = netdev_priv(dev);
+ unsigned int queue_index = smp_processor_id();
+ struct i40e_vsi *vsi = np->vsi;
+
+ if (test_bit(__I40E_VSI_DOWN, vsi->state))
+ return;
+
+ if (!i40e_enabled_xdp_vsi(vsi) || queue_index >= vsi->num_queue_pairs)
+ return;
+
+ i40e_xdp_ring_update_tail(vsi->xdp_rings[queue_index]);
+}
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
void i40e_detect_recover_hung(struct i40e_vsi *vsi);
int __i40e_maybe_stop_tx(struct i40e_ring *tx_ring, int size);
bool __i40e_chk_linearize(struct sk_buff *skb);
+int i40e_xdp_xmit(struct net_device *dev, struct xdp_buff *xdp);
+void i40e_xdp_flush(struct net_device *dev);
/**
* i40e_get_head - Retrieve head from head writeback
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Driver
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel(R) 40-10 Gigabit Ethernet Virtual Function Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
*
* Intel Ethernet Controller XL710 Family Linux Virtual Function Driver
--- /dev/null
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2018, Intel Corporation.
+
+#
+# Makefile for the Intel(R) Ethernet Connection E800 Series Linux Driver
+#
+
+obj-$(CONFIG_ICE) += ice.o
+
+ice-y := ice_main.o \
+ ice_controlq.o \
+ ice_common.o \
+ ice_nvm.o \
+ ice_switch.o \
+ ice_sched.o \
+ ice_txrx.o \
+ ice_ethtool.o
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_H_
+#define _ICE_H_
+
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/compiler.h>
+#include <linux/etherdevice.h>
+#include <linux/skbuff.h>
+#include <linux/cpumask.h>
+#include <linux/rtnetlink.h>
+#include <linux/if_vlan.h>
+#include <linux/dma-mapping.h>
+#include <linux/pci.h>
+#include <linux/workqueue.h>
+#include <linux/aer.h>
+#include <linux/interrupt.h>
+#include <linux/ethtool.h>
+#include <linux/timer.h>
+#include <linux/delay.h>
+#include <linux/bitmap.h>
+#include <linux/log2.h>
+#include <linux/ip.h>
+#include <linux/ipv6.h>
+#include <linux/if_bridge.h>
+#include <net/ipv6.h>
+#include "ice_devids.h"
+#include "ice_type.h"
+#include "ice_txrx.h"
+#include "ice_switch.h"
+#include "ice_common.h"
+#include "ice_sched.h"
+
+extern const char ice_drv_ver[];
+#define ICE_BAR0 0
+#define ICE_DFLT_NUM_DESC 128
+#define ICE_MIN_NUM_DESC 8
+#define ICE_MAX_NUM_DESC 8160
+#define ICE_REQ_DESC_MULTIPLE 32
+#define ICE_DFLT_TRAFFIC_CLASS BIT(0)
+#define ICE_INT_NAME_STR_LEN (IFNAMSIZ + 16)
+#define ICE_ETHTOOL_FWVER_LEN 32
+#define ICE_AQ_LEN 64
+#define ICE_MIN_MSIX 2
+#define ICE_NO_VSI 0xffff
+#define ICE_MAX_VSI_ALLOC 130
+#define ICE_MAX_TXQS 2048
+#define ICE_MAX_RXQS 2048
+#define ICE_VSI_MAP_CONTIG 0
+#define ICE_VSI_MAP_SCATTER 1
+#define ICE_MAX_SCATTER_TXQS 16
+#define ICE_MAX_SCATTER_RXQS 16
+#define ICE_Q_WAIT_RETRY_LIMIT 10
+#define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
+#define ICE_MAX_LG_RSS_QS 256
+#define ICE_MAX_SMALL_RSS_QS 8
+#define ICE_RES_VALID_BIT 0x8000
+#define ICE_RES_MISC_VEC_ID (ICE_RES_VALID_BIT - 1)
+#define ICE_INVAL_Q_INDEX 0xffff
+
+#define ICE_VSIQF_HKEY_ARRAY_SIZE ((VSIQF_HKEY_MAX_INDEX + 1) * 4)
+
+#define ICE_DFLT_NETIF_M (NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_LINK)
+
+#define ICE_MAX_MTU (ICE_AQ_SET_MAC_FRAME_SIZE_MAX - \
+ ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN)
+
+#define ICE_UP_TABLE_TRANSLATE(val, i) \
+ (((val) << ICE_AQ_VSI_UP_TABLE_UP##i##_S) & \
+ ICE_AQ_VSI_UP_TABLE_UP##i##_M)
+
+#define ICE_TX_DESC(R, i) (&(((struct ice_tx_desc *)((R)->desc))[i]))
+#define ICE_RX_DESC(R, i) (&(((union ice_32b_rx_flex_desc *)((R)->desc))[i]))
+#define ICE_TX_CTX_DESC(R, i) (&(((struct ice_tx_ctx_desc *)((R)->desc))[i]))
+
+/* Macro for each VSI in a PF */
+#define ice_for_each_vsi(pf, i) \
+ for ((i) = 0; (i) < (pf)->num_alloc_vsi; (i)++)
+
+/* Macros for each tx/rx ring in a VSI */
+#define ice_for_each_txq(vsi, i) \
+ for ((i) = 0; (i) < (vsi)->num_txq; (i)++)
+
+#define ice_for_each_rxq(vsi, i) \
+ for ((i) = 0; (i) < (vsi)->num_rxq; (i)++)
+
+struct ice_tc_info {
+ u16 qoffset;
+ u16 qcount;
+};
+
+struct ice_tc_cfg {
+ u8 numtc; /* Total number of enabled TCs */
+ u8 ena_tc; /* TX map */
+ struct ice_tc_info tc_info[ICE_MAX_TRAFFIC_CLASS];
+};
+
+struct ice_res_tracker {
+ u16 num_entries;
+ u16 search_hint;
+ u16 list[1];
+};
+
+struct ice_sw {
+ struct ice_pf *pf;
+ u16 sw_id; /* switch ID for this switch */
+ u16 bridge_mode; /* VEB/VEPA/Port Virtualizer */
+};
+
+enum ice_state {
+ __ICE_DOWN,
+ __ICE_NEEDS_RESTART,
+ __ICE_RESET_RECOVERY_PENDING, /* set by driver when reset starts */
+ __ICE_PFR_REQ, /* set by driver and peers */
+ __ICE_CORER_REQ, /* set by driver and peers */
+ __ICE_GLOBR_REQ, /* set by driver and peers */
+ __ICE_CORER_RECV, /* set by OICR handler */
+ __ICE_GLOBR_RECV, /* set by OICR handler */
+ __ICE_EMPR_RECV, /* set by OICR handler */
+ __ICE_SUSPENDED, /* set on module remove path */
+ __ICE_RESET_FAILED, /* set by reset/rebuild */
+ __ICE_ADMINQ_EVENT_PENDING,
+ __ICE_FLTR_OVERFLOW_PROMISC,
+ __ICE_CFG_BUSY,
+ __ICE_SERVICE_SCHED,
+ __ICE_STATE_NBITS /* must be last */
+};
+
+enum ice_vsi_flags {
+ ICE_VSI_FLAG_UMAC_FLTR_CHANGED,
+ ICE_VSI_FLAG_MMAC_FLTR_CHANGED,
+ ICE_VSI_FLAG_VLAN_FLTR_CHANGED,
+ ICE_VSI_FLAG_PROMISC_CHANGED,
+ ICE_VSI_FLAG_NBITS /* must be last */
+};
+
+/* struct that defines a VSI, associated with a dev */
+struct ice_vsi {
+ struct net_device *netdev;
+ struct ice_sw *vsw; /* switch this VSI is on */
+ struct ice_pf *back; /* back pointer to PF */
+ struct ice_port_info *port_info; /* back pointer to port_info */
+ struct ice_ring **rx_rings; /* rx ring array */
+ struct ice_ring **tx_rings; /* tx ring array */
+ struct ice_q_vector **q_vectors; /* q_vector array */
+
+ irqreturn_t (*irq_handler)(int irq, void *data);
+
+ u64 tx_linearize;
+ DECLARE_BITMAP(state, __ICE_STATE_NBITS);
+ DECLARE_BITMAP(flags, ICE_VSI_FLAG_NBITS);
+ unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
+ unsigned int current_netdev_flags;
+ u32 tx_restart;
+ u32 tx_busy;
+ u32 rx_buf_failed;
+ u32 rx_page_failed;
+ int num_q_vectors;
+ int base_vector;
+ enum ice_vsi_type type;
+ u16 vsi_num; /* HW (absolute) index of this VSI */
+ u16 idx; /* software index in pf->vsi[] */
+
+ /* Interrupt thresholds */
+ u16 work_lmt;
+
+ /* RSS config */
+ u16 rss_table_size; /* HW RSS table size */
+ u16 rss_size; /* Allocated RSS queues */
+ u8 *rss_hkey_user; /* User configured hash keys */
+ u8 *rss_lut_user; /* User configured lookup table entries */
+ u8 rss_lut_type; /* used to configure Get/Set RSS LUT AQ call */
+
+ u16 max_frame;
+ u16 rx_buf_len;
+
+ struct ice_aqc_vsi_props info; /* VSI properties */
+
+ /* VSI stats */
+ struct rtnl_link_stats64 net_stats;
+ struct ice_eth_stats eth_stats;
+ struct ice_eth_stats eth_stats_prev;
+
+ struct list_head tmp_sync_list; /* MAC filters to be synced */
+ struct list_head tmp_unsync_list; /* MAC filters to be unsynced */
+
+ bool irqs_ready;
+ bool current_isup; /* Sync 'link up' logging */
+ bool stat_offsets_loaded;
+
+ /* queue information */
+ u8 tx_mapping_mode; /* ICE_MAP_MODE_[CONTIG|SCATTER] */
+ u8 rx_mapping_mode; /* ICE_MAP_MODE_[CONTIG|SCATTER] */
+ u16 txq_map[ICE_MAX_TXQS]; /* index in pf->avail_txqs */
+ u16 rxq_map[ICE_MAX_RXQS]; /* index in pf->avail_rxqs */
+ u16 alloc_txq; /* Allocated Tx queues */
+ u16 num_txq; /* Used Tx queues */
+ u16 alloc_rxq; /* Allocated Rx queues */
+ u16 num_rxq; /* Used Rx queues */
+ u16 num_desc;
+ struct ice_tc_cfg tc_cfg;
+} ____cacheline_internodealigned_in_smp;
+
+/* struct that defines an interrupt vector */
+struct ice_q_vector {
+ struct ice_vsi *vsi;
+ cpumask_t affinity_mask;
+ struct napi_struct napi;
+ struct ice_ring_container rx;
+ struct ice_ring_container tx;
+ struct irq_affinity_notify affinity_notify;
+ u16 v_idx; /* index in the vsi->q_vector array. */
+ u8 num_ring_tx; /* total number of tx rings in vector */
+ u8 num_ring_rx; /* total number of rx rings in vector */
+ char name[ICE_INT_NAME_STR_LEN];
+} ____cacheline_internodealigned_in_smp;
+
+enum ice_pf_flags {
+ ICE_FLAG_MSIX_ENA,
+ ICE_FLAG_FLTR_SYNC,
+ ICE_FLAG_RSS_ENA,
+ ICE_PF_FLAGS_NBITS /* must be last */
+};
+
+struct ice_pf {
+ struct pci_dev *pdev;
+ struct msix_entry *msix_entries;
+ struct ice_res_tracker *irq_tracker;
+ struct ice_vsi **vsi; /* VSIs created by the driver */
+ struct ice_sw *first_sw; /* first switch created by firmware */
+ DECLARE_BITMAP(state, __ICE_STATE_NBITS);
+ DECLARE_BITMAP(avail_txqs, ICE_MAX_TXQS);
+ DECLARE_BITMAP(avail_rxqs, ICE_MAX_RXQS);
+ DECLARE_BITMAP(flags, ICE_PF_FLAGS_NBITS);
+ unsigned long serv_tmr_period;
+ unsigned long serv_tmr_prev;
+ struct timer_list serv_tmr;
+ struct work_struct serv_task;
+ struct mutex avail_q_mutex; /* protects access to avail_[rx|tx]qs */
+ struct mutex sw_mutex; /* lock for protecting VSI alloc flow */
+ u32 msg_enable;
+ u32 hw_csum_rx_error;
+ u32 oicr_idx; /* Other interrupt cause vector index */
+ u32 num_lan_msix; /* Total MSIX vectors for base driver */
+ u32 num_avail_msix; /* remaining MSIX vectors left unclaimed */
+ u16 num_lan_tx; /* num lan tx queues setup */
+ u16 num_lan_rx; /* num lan rx queues setup */
+ u16 q_left_tx; /* remaining num tx queues left unclaimed */
+ u16 q_left_rx; /* remaining num rx queues left unclaimed */
+ u16 next_vsi; /* Next free slot in pf->vsi[] - 0-based! */
+ u16 num_alloc_vsi;
+ u16 corer_count; /* Core reset count */
+ u16 globr_count; /* Global reset count */
+ u16 empr_count; /* EMP reset count */
+ u16 pfr_count; /* PF reset count */
+
+ struct ice_hw_port_stats stats;
+ struct ice_hw_port_stats stats_prev;
+ struct ice_hw hw;
+ bool stat_prev_loaded; /* has previous stats been loaded */
+ char int_name[ICE_INT_NAME_STR_LEN];
+};
+
+struct ice_netdev_priv {
+ struct ice_vsi *vsi;
+};
+
+/**
+ * ice_irq_dynamic_ena - Enable default interrupt generation settings
+ * @hw: pointer to hw struct
+ * @vsi: pointer to vsi struct, can be NULL
+ * @q_vector: pointer to q_vector, can be NULL
+ */
+static inline void ice_irq_dynamic_ena(struct ice_hw *hw, struct ice_vsi *vsi,
+ struct ice_q_vector *q_vector)
+{
+ u32 vector = (vsi && q_vector) ? vsi->base_vector + q_vector->v_idx :
+ ((struct ice_pf *)hw->back)->oicr_idx;
+ int itr = ICE_ITR_NONE;
+ u32 val;
+
+ /* clear the PBA here, as this function is meant to clean out all
+ * previous interrupts and enable the interrupt
+ */
+ val = GLINT_DYN_CTL_INTENA_M | GLINT_DYN_CTL_CLEARPBA_M |
+ (itr << GLINT_DYN_CTL_ITR_INDX_S);
+ if (vsi)
+ if (test_bit(__ICE_DOWN, vsi->state))
+ return;
+ wr32(hw, GLINT_DYN_CTL(vector), val);
+}
+
+static inline void ice_vsi_set_tc_cfg(struct ice_vsi *vsi)
+{
+ vsi->tc_cfg.ena_tc = ICE_DFLT_TRAFFIC_CLASS;
+ vsi->tc_cfg.numtc = 1;
+}
+
+void ice_set_ethtool_ops(struct net_device *netdev);
+int ice_up(struct ice_vsi *vsi);
+int ice_down(struct ice_vsi *vsi);
+int ice_set_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);
+int ice_get_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);
+void ice_fill_rss_lut(u8 *lut, u16 rss_table_size, u16 rss_size);
+void ice_print_link_msg(struct ice_vsi *vsi, bool isup);
+
+#endif /* _ICE_H_ */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_ADMINQ_CMD_H_
+#define _ICE_ADMINQ_CMD_H_
+
+/* This header file defines the Admin Queue commands, error codes and
+ * descriptor format. It is shared between Firmware and Software.
+ */
+
+#define ICE_MAX_VSI 768
+#define ICE_AQC_TOPO_MAX_LEVEL_NUM 0x9
+#define ICE_AQ_SET_MAC_FRAME_SIZE_MAX 9728
+
+struct ice_aqc_generic {
+ __le32 param0;
+ __le32 param1;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Get version (direct 0x0001) */
+struct ice_aqc_get_ver {
+ __le32 rom_ver;
+ __le32 fw_build;
+ u8 fw_branch;
+ u8 fw_major;
+ u8 fw_minor;
+ u8 fw_patch;
+ u8 api_branch;
+ u8 api_major;
+ u8 api_minor;
+ u8 api_patch;
+};
+
+/* Queue Shutdown (direct 0x0003) */
+struct ice_aqc_q_shutdown {
+#define ICE_AQC_DRIVER_UNLOADING BIT(0)
+ __le32 driver_unloading;
+ u8 reserved[12];
+};
+
+/* Request resource ownership (direct 0x0008)
+ * Release resource ownership (direct 0x0009)
+ */
+struct ice_aqc_req_res {
+ __le16 res_id;
+#define ICE_AQC_RES_ID_NVM 1
+#define ICE_AQC_RES_ID_SDP 2
+#define ICE_AQC_RES_ID_CHNG_LOCK 3
+#define ICE_AQC_RES_ID_GLBL_LOCK 4
+ __le16 access_type;
+#define ICE_AQC_RES_ACCESS_READ 1
+#define ICE_AQC_RES_ACCESS_WRITE 2
+
+ /* Upon successful completion, FW writes this value and driver is
+ * expected to release resource before timeout. This value is provided
+ * in milliseconds.
+ */
+ __le32 timeout;
+#define ICE_AQ_RES_NVM_READ_DFLT_TIMEOUT_MS 3000
+#define ICE_AQ_RES_NVM_WRITE_DFLT_TIMEOUT_MS 180000
+#define ICE_AQ_RES_CHNG_LOCK_DFLT_TIMEOUT_MS 1000
+#define ICE_AQ_RES_GLBL_LOCK_DFLT_TIMEOUT_MS 3000
+ /* For SDP: pin id of the SDP */
+ __le32 res_number;
+ /* Status is only used for ICE_AQC_RES_ID_GLBL_LOCK */
+ __le16 status;
+#define ICE_AQ_RES_GLBL_SUCCESS 0
+#define ICE_AQ_RES_GLBL_IN_PROG 1
+#define ICE_AQ_RES_GLBL_DONE 2
+ u8 reserved[2];
+};
+
+/* Get function capabilities (indirect 0x000A)
+ * Get device capabilities (indirect 0x000B)
+ */
+struct ice_aqc_list_caps {
+ u8 cmd_flags;
+ u8 pf_index;
+ u8 reserved[2];
+ __le32 count;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Device/Function buffer entry, repeated per reported capability */
+struct ice_aqc_list_caps_elem {
+ __le16 cap;
+#define ICE_AQC_CAPS_VSI 0x0017
+#define ICE_AQC_CAPS_RSS 0x0040
+#define ICE_AQC_CAPS_RXQS 0x0041
+#define ICE_AQC_CAPS_TXQS 0x0042
+#define ICE_AQC_CAPS_MSIX 0x0043
+#define ICE_AQC_CAPS_MAX_MTU 0x0047
+
+ u8 major_ver;
+ u8 minor_ver;
+ /* Number of resources described by this capability */
+ __le32 number;
+ /* Only meaningful for some types of resources */
+ __le32 logical_id;
+ /* Only meaningful for some types of resources */
+ __le32 phys_id;
+ __le64 rsvd1;
+ __le64 rsvd2;
+};
+
+/* Manage MAC address, read command - indirect (0x0107)
+ * This struct is also used for the response
+ */
+struct ice_aqc_manage_mac_read {
+ __le16 flags; /* Zeroed by device driver */
+#define ICE_AQC_MAN_MAC_LAN_ADDR_VALID BIT(4)
+#define ICE_AQC_MAN_MAC_SAN_ADDR_VALID BIT(5)
+#define ICE_AQC_MAN_MAC_PORT_ADDR_VALID BIT(6)
+#define ICE_AQC_MAN_MAC_WOL_ADDR_VALID BIT(7)
+#define ICE_AQC_MAN_MAC_READ_S 4
+#define ICE_AQC_MAN_MAC_READ_M (0xF << ICE_AQC_MAN_MAC_READ_S)
+ u8 lport_num;
+ u8 lport_num_valid;
+#define ICE_AQC_MAN_MAC_PORT_NUM_IS_VALID BIT(0)
+ u8 num_addr; /* Used in response */
+ u8 reserved[3];
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Response buffer format for manage MAC read command */
+struct ice_aqc_manage_mac_read_resp {
+ u8 lport_num;
+ u8 addr_type;
+#define ICE_AQC_MAN_MAC_ADDR_TYPE_LAN 0
+#define ICE_AQC_MAN_MAC_ADDR_TYPE_WOL 1
+ u8 mac_addr[ETH_ALEN];
+};
+
+/* Manage MAC address, write command - direct (0x0108) */
+struct ice_aqc_manage_mac_write {
+ u8 port_num;
+ u8 flags;
+#define ICE_AQC_MAN_MAC_WR_MC_MAG_EN BIT(0)
+#define ICE_AQC_MAN_MAC_WR_WOL_LAA_PFR_KEEP BIT(1)
+#define ICE_AQC_MAN_MAC_WR_S 6
+#define ICE_AQC_MAN_MAC_WR_M (3 << ICE_AQC_MAN_MAC_WR_S)
+#define ICE_AQC_MAN_MAC_UPDATE_LAA 0
+#define ICE_AQC_MAN_MAC_UPDATE_LAA_WOL (BIT(0) << ICE_AQC_MAN_MAC_WR_S)
+ /* High 16 bits of MAC address in big endian order */
+ __be16 sah;
+ /* Low 32 bits of MAC address in big endian order */
+ __be32 sal;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Clear PXE Command and response (direct 0x0110) */
+struct ice_aqc_clear_pxe {
+ u8 rx_cnt;
+#define ICE_AQC_CLEAR_PXE_RX_CNT 0x2
+ u8 reserved[15];
+};
+
+/* Get switch configuration (0x0200) */
+struct ice_aqc_get_sw_cfg {
+ /* Reserved for command and copy of request flags for response */
+ __le16 flags;
+ /* First desc in case of command and next_elem in case of response
+ * In case of response, if it is not zero, means all the configuration
+ * was not returned and new command shall be sent with this value in
+ * the 'first desc' field
+ */
+ __le16 element;
+ /* Reserved for command, only used for response */
+ __le16 num_elems;
+ __le16 rsvd;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Each entry in the response buffer is of the following type: */
+struct ice_aqc_get_sw_cfg_resp_elem {
+ /* VSI/Port Number */
+ __le16 vsi_port_num;
+#define ICE_AQC_GET_SW_CONF_RESP_VSI_PORT_NUM_S 0
+#define ICE_AQC_GET_SW_CONF_RESP_VSI_PORT_NUM_M \
+ (0x3FF << ICE_AQC_GET_SW_CONF_RESP_VSI_PORT_NUM_S)
+#define ICE_AQC_GET_SW_CONF_RESP_TYPE_S 14
+#define ICE_AQC_GET_SW_CONF_RESP_TYPE_M (0x3 << ICE_AQC_GET_SW_CONF_RESP_TYPE_S)
+#define ICE_AQC_GET_SW_CONF_RESP_PHYS_PORT 0
+#define ICE_AQC_GET_SW_CONF_RESP_VIRT_PORT 1
+#define ICE_AQC_GET_SW_CONF_RESP_VSI 2
+
+ /* SWID VSI/Port belongs to */
+ __le16 swid;
+
+ /* Bit 14..0 : PF/VF number VSI belongs to
+ * Bit 15 : VF indication bit
+ */
+ __le16 pf_vf_num;
+#define ICE_AQC_GET_SW_CONF_RESP_FUNC_NUM_S 0
+#define ICE_AQC_GET_SW_CONF_RESP_FUNC_NUM_M \
+ (0x7FFF << ICE_AQC_GET_SW_CONF_RESP_FUNC_NUM_S)
+#define ICE_AQC_GET_SW_CONF_RESP_IS_VF BIT(15)
+};
+
+/* The response buffer is as follows. Note that the length of the
+ * elements array varies with the length of the command response.
+ */
+struct ice_aqc_get_sw_cfg_resp {
+ struct ice_aqc_get_sw_cfg_resp_elem elements[1];
+};
+
+/* These resource type defines are used for all switch resource
+ * commands where a resource type is required, such as:
+ * Get Resource Allocation command (indirect 0x0204)
+ * Allocate Resources command (indirect 0x0208)
+ * Free Resources command (indirect 0x0209)
+ * Get Allocated Resource Descriptors Command (indirect 0x020A)
+ */
+#define ICE_AQC_RES_TYPE_VSI_LIST_REP 0x03
+#define ICE_AQC_RES_TYPE_VSI_LIST_PRUNE 0x04
+
+/* Allocate Resources command (indirect 0x0208)
+ * Free Resources command (indirect 0x0209)
+ */
+struct ice_aqc_alloc_free_res_cmd {
+ __le16 num_entries; /* Number of Resource entries */
+ u8 reserved[6];
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Resource descriptor */
+struct ice_aqc_res_elem {
+ union {
+ __le16 sw_resp;
+ __le16 flu_resp;
+ } e;
+};
+
+/* Buffer for Allocate/Free Resources commands */
+struct ice_aqc_alloc_free_res_elem {
+ __le16 res_type; /* Types defined above cmd 0x0204 */
+#define ICE_AQC_RES_TYPE_SHARED_S 7
+#define ICE_AQC_RES_TYPE_SHARED_M (0x1 << ICE_AQC_RES_TYPE_SHARED_S)
+#define ICE_AQC_RES_TYPE_VSI_PRUNE_LIST_S 8
+#define ICE_AQC_RES_TYPE_VSI_PRUNE_LIST_M \
+ (0xF << ICE_AQC_RES_TYPE_VSI_PRUNE_LIST_S)
+ __le16 num_elems;
+ struct ice_aqc_res_elem elem[1];
+};
+
+/* Add VSI (indirect 0x0210)
+ * Update VSI (indirect 0x0211)
+ * Get VSI (indirect 0x0212)
+ * Free VSI (indirect 0x0213)
+ */
+struct ice_aqc_add_get_update_free_vsi {
+ __le16 vsi_num;
+#define ICE_AQ_VSI_NUM_S 0
+#define ICE_AQ_VSI_NUM_M (0x03FF << ICE_AQ_VSI_NUM_S)
+#define ICE_AQ_VSI_IS_VALID BIT(15)
+ __le16 cmd_flags;
+#define ICE_AQ_VSI_KEEP_ALLOC 0x1
+ u8 vf_id;
+ u8 reserved;
+ __le16 vsi_flags;
+#define ICE_AQ_VSI_TYPE_S 0
+#define ICE_AQ_VSI_TYPE_M (0x3 << ICE_AQ_VSI_TYPE_S)
+#define ICE_AQ_VSI_TYPE_VF 0x0
+#define ICE_AQ_VSI_TYPE_VMDQ2 0x1
+#define ICE_AQ_VSI_TYPE_PF 0x2
+#define ICE_AQ_VSI_TYPE_EMP_MNG 0x3
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Response descriptor for:
+ * Add VSI (indirect 0x0210)
+ * Update VSI (indirect 0x0211)
+ * Free VSI (indirect 0x0213)
+ */
+struct ice_aqc_add_update_free_vsi_resp {
+ __le16 vsi_num;
+ __le16 ext_status;
+ __le16 vsi_used;
+ __le16 vsi_free;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+struct ice_aqc_vsi_props {
+ __le16 valid_sections;
+#define ICE_AQ_VSI_PROP_SW_VALID BIT(0)
+#define ICE_AQ_VSI_PROP_SECURITY_VALID BIT(1)
+#define ICE_AQ_VSI_PROP_VLAN_VALID BIT(2)
+#define ICE_AQ_VSI_PROP_OUTER_TAG_VALID BIT(3)
+#define ICE_AQ_VSI_PROP_INGRESS_UP_VALID BIT(4)
+#define ICE_AQ_VSI_PROP_EGRESS_UP_VALID BIT(5)
+#define ICE_AQ_VSI_PROP_RXQ_MAP_VALID BIT(6)
+#define ICE_AQ_VSI_PROP_Q_OPT_VALID BIT(7)
+#define ICE_AQ_VSI_PROP_OUTER_UP_VALID BIT(8)
+#define ICE_AQ_VSI_PROP_FLOW_DIR_VALID BIT(11)
+#define ICE_AQ_VSI_PROP_PASID_VALID BIT(12)
+ /* switch section */
+ u8 sw_id;
+ u8 sw_flags;
+#define ICE_AQ_VSI_SW_FLAG_ALLOW_LB BIT(5)
+#define ICE_AQ_VSI_SW_FLAG_LOCAL_LB BIT(6)
+#define ICE_AQ_VSI_SW_FLAG_SRC_PRUNE BIT(7)
+ u8 sw_flags2;
+#define ICE_AQ_VSI_SW_FLAG_RX_PRUNE_EN_S 0
+#define ICE_AQ_VSI_SW_FLAG_RX_PRUNE_EN_M \
+ (0xF << ICE_AQ_VSI_SW_FLAG_RX_PRUNE_EN_S)
+#define ICE_AQ_VSI_SW_FLAG_RX_VLAN_PRUNE_ENA BIT(0)
+#define ICE_AQ_VSI_SW_FLAG_LAN_ENA BIT(4)
+ u8 veb_stat_id;
+#define ICE_AQ_VSI_SW_VEB_STAT_ID_S 0
+#define ICE_AQ_VSI_SW_VEB_STAT_ID_M (0x1F << ICE_AQ_VSI_SW_VEB_STAT_ID_S)
+#define ICE_AQ_VSI_SW_VEB_STAT_ID_VALID BIT(5)
+ /* security section */
+ u8 sec_flags;
+#define ICE_AQ_VSI_SEC_FLAG_ALLOW_DEST_OVRD BIT(0)
+#define ICE_AQ_VSI_SEC_FLAG_ENA_MAC_ANTI_SPOOF BIT(2)
+#define ICE_AQ_VSI_SEC_TX_PRUNE_ENA_S 4
+#define ICE_AQ_VSI_SEC_TX_PRUNE_ENA_M (0xF << ICE_AQ_VSI_SEC_TX_PRUNE_ENA_S)
+#define ICE_AQ_VSI_SEC_TX_VLAN_PRUNE_ENA BIT(0)
+ u8 sec_reserved;
+ /* VLAN section */
+ __le16 pvid; /* VLANS include priority bits */
+ u8 pvlan_reserved[2];
+ u8 port_vlan_flags;
+#define ICE_AQ_VSI_PVLAN_MODE_S 0
+#define ICE_AQ_VSI_PVLAN_MODE_M (0x3 << ICE_AQ_VSI_PVLAN_MODE_S)
+#define ICE_AQ_VSI_PVLAN_MODE_UNTAGGED 0x1
+#define ICE_AQ_VSI_PVLAN_MODE_TAGGED 0x2
+#define ICE_AQ_VSI_PVLAN_MODE_ALL 0x3
+#define ICE_AQ_VSI_PVLAN_INSERT_PVID BIT(2)
+#define ICE_AQ_VSI_PVLAN_EMOD_S 3
+#define ICE_AQ_VSI_PVLAN_EMOD_M (0x3 << ICE_AQ_VSI_PVLAN_EMOD_S)
+#define ICE_AQ_VSI_PVLAN_EMOD_STR_BOTH (0x0 << ICE_AQ_VSI_PVLAN_EMOD_S)
+#define ICE_AQ_VSI_PVLAN_EMOD_STR_UP (0x1 << ICE_AQ_VSI_PVLAN_EMOD_S)
+#define ICE_AQ_VSI_PVLAN_EMOD_STR (0x2 << ICE_AQ_VSI_PVLAN_EMOD_S)
+#define ICE_AQ_VSI_PVLAN_EMOD_NOTHING (0x3 << ICE_AQ_VSI_PVLAN_EMOD_S)
+ u8 pvlan_reserved2[3];
+ /* ingress egress up sections */
+ __le32 ingress_table; /* bitmap, 3 bits per up */
+#define ICE_AQ_VSI_UP_TABLE_UP0_S 0
+#define ICE_AQ_VSI_UP_TABLE_UP0_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP0_S)
+#define ICE_AQ_VSI_UP_TABLE_UP1_S 3
+#define ICE_AQ_VSI_UP_TABLE_UP1_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP1_S)
+#define ICE_AQ_VSI_UP_TABLE_UP2_S 6
+#define ICE_AQ_VSI_UP_TABLE_UP2_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP2_S)
+#define ICE_AQ_VSI_UP_TABLE_UP3_S 9
+#define ICE_AQ_VSI_UP_TABLE_UP3_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP3_S)
+#define ICE_AQ_VSI_UP_TABLE_UP4_S 12
+#define ICE_AQ_VSI_UP_TABLE_UP4_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP4_S)
+#define ICE_AQ_VSI_UP_TABLE_UP5_S 15
+#define ICE_AQ_VSI_UP_TABLE_UP5_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP5_S)
+#define ICE_AQ_VSI_UP_TABLE_UP6_S 18
+#define ICE_AQ_VSI_UP_TABLE_UP6_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP6_S)
+#define ICE_AQ_VSI_UP_TABLE_UP7_S 21
+#define ICE_AQ_VSI_UP_TABLE_UP7_M (0x7 << ICE_AQ_VSI_UP_TABLE_UP7_S)
+ __le32 egress_table; /* same defines as for ingress table */
+ /* outer tags section */
+ __le16 outer_tag;
+ u8 outer_tag_flags;
+#define ICE_AQ_VSI_OUTER_TAG_MODE_S 0
+#define ICE_AQ_VSI_OUTER_TAG_MODE_M (0x3 << ICE_AQ_VSI_OUTER_TAG_MODE_S)
+#define ICE_AQ_VSI_OUTER_TAG_NOTHING 0x0
+#define ICE_AQ_VSI_OUTER_TAG_REMOVE 0x1
+#define ICE_AQ_VSI_OUTER_TAG_COPY 0x2
+#define ICE_AQ_VSI_OUTER_TAG_TYPE_S 2
+#define ICE_AQ_VSI_OUTER_TAG_TYPE_M (0x3 << ICE_AQ_VSI_OUTER_TAG_TYPE_S)
+#define ICE_AQ_VSI_OUTER_TAG_NONE 0x0
+#define ICE_AQ_VSI_OUTER_TAG_STAG 0x1
+#define ICE_AQ_VSI_OUTER_TAG_VLAN_8100 0x2
+#define ICE_AQ_VSI_OUTER_TAG_VLAN_9100 0x3
+#define ICE_AQ_VSI_OUTER_TAG_INSERT BIT(4)
+#define ICE_AQ_VSI_OUTER_TAG_ACCEPT_HOST BIT(6)
+ u8 outer_tag_reserved;
+ /* queue mapping section */
+ __le16 mapping_flags;
+#define ICE_AQ_VSI_Q_MAP_CONTIG 0x0
+#define ICE_AQ_VSI_Q_MAP_NONCONTIG BIT(0)
+ __le16 q_mapping[16];
+#define ICE_AQ_VSI_Q_S 0
+#define ICE_AQ_VSI_Q_M (0x7FF << ICE_AQ_VSI_Q_S)
+ __le16 tc_mapping[8];
+#define ICE_AQ_VSI_TC_Q_OFFSET_S 0
+#define ICE_AQ_VSI_TC_Q_OFFSET_M (0x7FF << ICE_AQ_VSI_TC_Q_OFFSET_S)
+#define ICE_AQ_VSI_TC_Q_NUM_S 11
+#define ICE_AQ_VSI_TC_Q_NUM_M (0xF << ICE_AQ_VSI_TC_Q_NUM_S)
+ /* queueing option section */
+ u8 q_opt_rss;
+#define ICE_AQ_VSI_Q_OPT_RSS_LUT_S 0
+#define ICE_AQ_VSI_Q_OPT_RSS_LUT_M (0x3 << ICE_AQ_VSI_Q_OPT_RSS_LUT_S)
+#define ICE_AQ_VSI_Q_OPT_RSS_LUT_VSI 0x0
+#define ICE_AQ_VSI_Q_OPT_RSS_LUT_PF 0x2
+#define ICE_AQ_VSI_Q_OPT_RSS_LUT_GBL 0x3
+#define ICE_AQ_VSI_Q_OPT_RSS_GBL_LUT_S 2
+#define ICE_AQ_VSI_Q_OPT_RSS_GBL_LUT_M (0xF << ICE_AQ_VSI_Q_OPT_RSS_GBL_LUT_S)
+#define ICE_AQ_VSI_Q_OPT_RSS_HASH_S 6
+#define ICE_AQ_VSI_Q_OPT_RSS_HASH_M (0x3 << ICE_AQ_VSI_Q_OPT_RSS_HASH_S)
+#define ICE_AQ_VSI_Q_OPT_RSS_TPLZ (0x0 << ICE_AQ_VSI_Q_OPT_RSS_HASH_S)
+#define ICE_AQ_VSI_Q_OPT_RSS_SYM_TPLZ (0x1 << ICE_AQ_VSI_Q_OPT_RSS_HASH_S)
+#define ICE_AQ_VSI_Q_OPT_RSS_XOR (0x2 << ICE_AQ_VSI_Q_OPT_RSS_HASH_S)
+#define ICE_AQ_VSI_Q_OPT_RSS_JHASH (0x3 << ICE_AQ_VSI_Q_OPT_RSS_HASH_S)
+ u8 q_opt_tc;
+#define ICE_AQ_VSI_Q_OPT_TC_OVR_S 0
+#define ICE_AQ_VSI_Q_OPT_TC_OVR_M (0x1F << ICE_AQ_VSI_Q_OPT_TC_OVR_S)
+#define ICE_AQ_VSI_Q_OPT_PROF_TC_OVR BIT(7)
+ u8 q_opt_flags;
+#define ICE_AQ_VSI_Q_OPT_PE_FLTR_EN BIT(0)
+ u8 q_opt_reserved[3];
+ /* outer up section */
+ __le32 outer_up_table; /* same structure and defines as ingress tbl */
+ /* section 10 */
+ __le16 sect_10_reserved;
+ /* flow director section */
+ __le16 fd_options;
+#define ICE_AQ_VSI_FD_ENABLE BIT(0)
+#define ICE_AQ_VSI_FD_TX_AUTO_ENABLE BIT(1)
+#define ICE_AQ_VSI_FD_PROG_ENABLE BIT(3)
+ __le16 max_fd_fltr_dedicated;
+ __le16 max_fd_fltr_shared;
+ __le16 fd_def_q;
+#define ICE_AQ_VSI_FD_DEF_Q_S 0
+#define ICE_AQ_VSI_FD_DEF_Q_M (0x7FF << ICE_AQ_VSI_FD_DEF_Q_S)
+#define ICE_AQ_VSI_FD_DEF_GRP_S 12
+#define ICE_AQ_VSI_FD_DEF_GRP_M (0x7 << ICE_AQ_VSI_FD_DEF_GRP_S)
+ __le16 fd_report_opt;
+#define ICE_AQ_VSI_FD_REPORT_Q_S 0
+#define ICE_AQ_VSI_FD_REPORT_Q_M (0x7FF << ICE_AQ_VSI_FD_REPORT_Q_S)
+#define ICE_AQ_VSI_FD_DEF_PRIORITY_S 12
+#define ICE_AQ_VSI_FD_DEF_PRIORITY_M (0x7 << ICE_AQ_VSI_FD_DEF_PRIORITY_S)
+#define ICE_AQ_VSI_FD_DEF_DROP BIT(15)
+ /* PASID section */
+ __le32 pasid_id;
+#define ICE_AQ_VSI_PASID_ID_S 0
+#define ICE_AQ_VSI_PASID_ID_M (0xFFFFF << ICE_AQ_VSI_PASID_ID_S)
+#define ICE_AQ_VSI_PASID_ID_VALID BIT(31)
+ u8 reserved[24];
+};
+
+/* Add/Update/Remove/Get switch rules (indirect 0x02A0, 0x02A1, 0x02A2, 0x02A3)
+ */
+struct ice_aqc_sw_rules {
+ /* ops: add switch rules, referring the number of rules.
+ * ops: update switch rules, referring the number of filters
+ * ops: remove switch rules, referring the entry index.
+ * ops: get switch rules, referring to the number of filters.
+ */
+ __le16 num_rules_fltr_entry_index;
+ u8 reserved[6];
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Add/Update/Get/Remove lookup Rx/Tx command/response entry
+ * This structures describes the lookup rules and associated actions. "index"
+ * is returned as part of a response to a successful Add command, and can be
+ * used to identify the rule for Update/Get/Remove commands.
+ */
+struct ice_sw_rule_lkup_rx_tx {
+ __le16 recipe_id;
+#define ICE_SW_RECIPE_LOGICAL_PORT_FWD 10
+ /* Source port for LOOKUP_RX and source VSI in case of LOOKUP_TX */
+ __le16 src;
+ __le32 act;
+
+ /* Bit 0:1 - Action type */
+#define ICE_SINGLE_ACT_TYPE_S 0x00
+#define ICE_SINGLE_ACT_TYPE_M (0x3 << ICE_SINGLE_ACT_TYPE_S)
+
+ /* Bit 2 - Loop back enable
+ * Bit 3 - LAN enable
+ */
+#define ICE_SINGLE_ACT_LB_ENABLE BIT(2)
+#define ICE_SINGLE_ACT_LAN_ENABLE BIT(3)
+
+ /* Action type = 0 - Forward to VSI or VSI list */
+#define ICE_SINGLE_ACT_VSI_FORWARDING 0x0
+
+#define ICE_SINGLE_ACT_VSI_ID_S 4
+#define ICE_SINGLE_ACT_VSI_ID_M (0x3FF << ICE_SINGLE_ACT_VSI_ID_S)
+#define ICE_SINGLE_ACT_VSI_LIST_ID_S 4
+#define ICE_SINGLE_ACT_VSI_LIST_ID_M (0x3FF << ICE_SINGLE_ACT_VSI_LIST_ID_S)
+ /* This bit needs to be set if action is forward to VSI list */
+#define ICE_SINGLE_ACT_VSI_LIST BIT(14)
+#define ICE_SINGLE_ACT_VALID_BIT BIT(17)
+#define ICE_SINGLE_ACT_DROP BIT(18)
+
+ /* Action type = 1 - Forward to Queue of Queue group */
+#define ICE_SINGLE_ACT_TO_Q 0x1
+#define ICE_SINGLE_ACT_Q_INDEX_S 4
+#define ICE_SINGLE_ACT_Q_INDEX_M (0x7FF << ICE_SINGLE_ACT_Q_INDEX_S)
+#define ICE_SINGLE_ACT_Q_REGION_S 15
+#define ICE_SINGLE_ACT_Q_REGION_M (0x7 << ICE_SINGLE_ACT_Q_REGION_S)
+#define ICE_SINGLE_ACT_Q_PRIORITY BIT(18)
+
+ /* Action type = 2 - Prune */
+#define ICE_SINGLE_ACT_PRUNE 0x2
+#define ICE_SINGLE_ACT_EGRESS BIT(15)
+#define ICE_SINGLE_ACT_INGRESS BIT(16)
+#define ICE_SINGLE_ACT_PRUNET BIT(17)
+ /* Bit 18 should be set to 0 for this action */
+
+ /* Action type = 2 - Pointer */
+#define ICE_SINGLE_ACT_PTR 0x2
+#define ICE_SINGLE_ACT_PTR_VAL_S 4
+#define ICE_SINGLE_ACT_PTR_VAL_M (0x1FFF << ICE_SINGLE_ACT_PTR_VAL_S)
+ /* Bit 18 should be set to 1 */
+#define ICE_SINGLE_ACT_PTR_BIT BIT(18)
+
+ /* Action type = 3 - Other actions. Last two bits
+ * are other action identifier
+ */
+#define ICE_SINGLE_ACT_OTHER_ACTS 0x3
+#define ICE_SINGLE_OTHER_ACT_IDENTIFIER_S 17
+#define ICE_SINGLE_OTHER_ACT_IDENTIFIER_M \
+ (0x3 << \ ICE_SINGLE_OTHER_ACT_IDENTIFIER_S)
+
+ /* Bit 17:18 - Defines other actions */
+ /* Other action = 0 - Mirror VSI */
+#define ICE_SINGLE_OTHER_ACT_MIRROR 0
+#define ICE_SINGLE_ACT_MIRROR_VSI_ID_S 4
+#define ICE_SINGLE_ACT_MIRROR_VSI_ID_M \
+ (0x3FF << ICE_SINGLE_ACT_MIRROR_VSI_ID_S)
+
+ /* Other action = 3 - Set Stat count */
+#define ICE_SINGLE_OTHER_ACT_STAT_COUNT 3
+#define ICE_SINGLE_ACT_STAT_COUNT_INDEX_S 4
+#define ICE_SINGLE_ACT_STAT_COUNT_INDEX_M \
+ (0x7F << ICE_SINGLE_ACT_STAT_COUNT_INDEX_S)
+
+ __le16 index; /* The index of the rule in the lookup table */
+ /* Length and values of the header to be matched per recipe or
+ * lookup-type
+ */
+ __le16 hdr_len;
+ u8 hdr[1];
+} __packed;
+
+/* Add/Update/Remove large action command/response entry
+ * "index" is returned as part of a response to a successful Add command, and
+ * can be used to identify the action for Update/Get/Remove commands.
+ */
+struct ice_sw_rule_lg_act {
+ __le16 index; /* Index in large action table */
+ __le16 size;
+ __le32 act[1]; /* array of size for actions */
+ /* Max number of large actions */
+#define ICE_MAX_LG_ACT 4
+ /* Bit 0:1 - Action type */
+#define ICE_LG_ACT_TYPE_S 0
+#define ICE_LG_ACT_TYPE_M (0x7 << ICE_LG_ACT_TYPE_S)
+
+ /* Action type = 0 - Forward to VSI or VSI list */
+#define ICE_LG_ACT_VSI_FORWARDING 0
+#define ICE_LG_ACT_VSI_ID_S 3
+#define ICE_LG_ACT_VSI_ID_M (0x3FF << ICE_LG_ACT_VSI_ID_S)
+#define ICE_LG_ACT_VSI_LIST_ID_S 3
+#define ICE_LG_ACT_VSI_LIST_ID_M (0x3FF << ICE_LG_ACT_VSI_LIST_ID_S)
+ /* This bit needs to be set if action is forward to VSI list */
+#define ICE_LG_ACT_VSI_LIST BIT(13)
+
+#define ICE_LG_ACT_VALID_BIT BIT(16)
+
+ /* Action type = 1 - Forward to Queue of Queue group */
+#define ICE_LG_ACT_TO_Q 0x1
+#define ICE_LG_ACT_Q_INDEX_S 3
+#define ICE_LG_ACT_Q_INDEX_M (0x7FF << ICE_LG_ACT_Q_INDEX_S)
+#define ICE_LG_ACT_Q_REGION_S 14
+#define ICE_LG_ACT_Q_REGION_M (0x7 << ICE_LG_ACT_Q_REGION_S)
+#define ICE_LG_ACT_Q_PRIORITY_SET BIT(17)
+
+ /* Action type = 2 - Prune */
+#define ICE_LG_ACT_PRUNE 0x2
+#define ICE_LG_ACT_EGRESS BIT(14)
+#define ICE_LG_ACT_INGRESS BIT(15)
+#define ICE_LG_ACT_PRUNET BIT(16)
+
+ /* Action type = 3 - Mirror VSI */
+#define ICE_LG_OTHER_ACT_MIRROR 0x3
+#define ICE_LG_ACT_MIRROR_VSI_ID_S 3
+#define ICE_LG_ACT_MIRROR_VSI_ID_M (0x3FF << ICE_LG_ACT_MIRROR_VSI_ID_S)
+
+ /* Action type = 5 - Large Action */
+#define ICE_LG_ACT_GENERIC 0x5
+#define ICE_LG_ACT_GENERIC_VALUE_S 3
+#define ICE_LG_ACT_GENERIC_VALUE_M (0xFFFF << ICE_LG_ACT_GENERIC_VALUE_S)
+#define ICE_LG_ACT_GENERIC_OFFSET_S 19
+#define ICE_LG_ACT_GENERIC_OFFSET_M (0x7 << ICE_LG_ACT_GENERIC_OFFSET_S)
+#define ICE_LG_ACT_GENERIC_PRIORITY_S 22
+#define ICE_LG_ACT_GENERIC_PRIORITY_M (0x7 << ICE_LG_ACT_GENERIC_PRIORITY_S)
+
+ /* Action = 7 - Set Stat count */
+#define ICE_LG_ACT_STAT_COUNT 0x7
+#define ICE_LG_ACT_STAT_COUNT_S 3
+#define ICE_LG_ACT_STAT_COUNT_M (0x7F << ICE_LG_ACT_STAT_COUNT_S)
+};
+
+/* Add/Update/Remove VSI list command/response entry
+ * "index" is returned as part of a response to a successful Add command, and
+ * can be used to identify the VSI list for Update/Get/Remove commands.
+ */
+struct ice_sw_rule_vsi_list {
+ __le16 index; /* Index of VSI/Prune list */
+ __le16 number_vsi;
+ __le16 vsi[1]; /* Array of number_vsi VSI numbers */
+};
+
+/* Query VSI list command/response entry */
+struct ice_sw_rule_vsi_list_query {
+ __le16 index;
+ DECLARE_BITMAP(vsi_list, ICE_MAX_VSI);
+} __packed;
+
+/* Add switch rule response:
+ * Content of return buffer is same as the input buffer. The status field and
+ * LUT index are updated as part of the response
+ */
+struct ice_aqc_sw_rules_elem {
+ __le16 type; /* Switch rule type, one of T_... */
+#define ICE_AQC_SW_RULES_T_LKUP_RX 0x0
+#define ICE_AQC_SW_RULES_T_LKUP_TX 0x1
+#define ICE_AQC_SW_RULES_T_LG_ACT 0x2
+#define ICE_AQC_SW_RULES_T_VSI_LIST_SET 0x3
+#define ICE_AQC_SW_RULES_T_VSI_LIST_CLEAR 0x4
+#define ICE_AQC_SW_RULES_T_PRUNE_LIST_SET 0x5
+#define ICE_AQC_SW_RULES_T_PRUNE_LIST_CLEAR 0x6
+ __le16 status;
+ union {
+ struct ice_sw_rule_lkup_rx_tx lkup_tx_rx;
+ struct ice_sw_rule_lg_act lg_act;
+ struct ice_sw_rule_vsi_list vsi_list;
+ struct ice_sw_rule_vsi_list_query vsi_list_query;
+ } __packed pdata;
+};
+
+/* Get Default Topology (indirect 0x0400) */
+struct ice_aqc_get_topo {
+ u8 port_num;
+ u8 num_branches;
+ __le16 reserved1;
+ __le32 reserved2;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Update TSE (indirect 0x0403)
+ * Get TSE (indirect 0x0404)
+ */
+struct ice_aqc_get_cfg_elem {
+ __le16 num_elem_req; /* Used by commands */
+ __le16 num_elem_resp; /* Used by responses */
+ __le32 reserved;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* This is the buffer for:
+ * Suspend Nodes (indirect 0x0409)
+ * Resume Nodes (indirect 0x040A)
+ */
+struct ice_aqc_suspend_resume_elem {
+ __le32 teid[1];
+};
+
+/* Add TSE (indirect 0x0401)
+ * Delete TSE (indirect 0x040F)
+ * Move TSE (indirect 0x0408)
+ */
+struct ice_aqc_add_move_delete_elem {
+ __le16 num_grps_req;
+ __le16 num_grps_updated;
+ __le32 reserved;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+struct ice_aqc_elem_info_bw {
+ __le16 bw_profile_idx;
+ __le16 bw_alloc;
+};
+
+struct ice_aqc_txsched_elem {
+ u8 elem_type; /* Special field, reserved for some aq calls */
+#define ICE_AQC_ELEM_TYPE_UNDEFINED 0x0
+#define ICE_AQC_ELEM_TYPE_ROOT_PORT 0x1
+#define ICE_AQC_ELEM_TYPE_TC 0x2
+#define ICE_AQC_ELEM_TYPE_SE_GENERIC 0x3
+#define ICE_AQC_ELEM_TYPE_ENTRY_POINT 0x4
+#define ICE_AQC_ELEM_TYPE_LEAF 0x5
+#define ICE_AQC_ELEM_TYPE_SE_PADDED 0x6
+ u8 valid_sections;
+#define ICE_AQC_ELEM_VALID_GENERIC BIT(0)
+#define ICE_AQC_ELEM_VALID_CIR BIT(1)
+#define ICE_AQC_ELEM_VALID_EIR BIT(2)
+#define ICE_AQC_ELEM_VALID_SHARED BIT(3)
+ u8 generic;
+#define ICE_AQC_ELEM_GENERIC_MODE_M 0x1
+#define ICE_AQC_ELEM_GENERIC_PRIO_S 0x1
+#define ICE_AQC_ELEM_GENERIC_PRIO_M (0x7 << ICE_AQC_ELEM_GENERIC_PRIO_S)
+#define ICE_AQC_ELEM_GENERIC_SP_S 0x4
+#define ICE_AQC_ELEM_GENERIC_SP_M (0x1 << ICE_AQC_ELEM_GENERIC_SP_S)
+#define ICE_AQC_ELEM_GENERIC_ADJUST_VAL_S 0x5
+#define ICE_AQC_ELEM_GENERIC_ADJUST_VAL_M \
+ (0x3 << ICE_AQC_ELEM_GENERIC_ADJUST_VAL_S)
+ u8 flags; /* Special field, reserved for some aq calls */
+#define ICE_AQC_ELEM_FLAG_SUSPEND_M 0x1
+ struct ice_aqc_elem_info_bw cir_bw;
+ struct ice_aqc_elem_info_bw eir_bw;
+ __le16 srl_id;
+ __le16 reserved2;
+};
+
+struct ice_aqc_txsched_elem_data {
+ __le32 parent_teid;
+ __le32 node_teid;
+ struct ice_aqc_txsched_elem data;
+};
+
+struct ice_aqc_txsched_topo_grp_info_hdr {
+ __le32 parent_teid;
+ __le16 num_elems;
+ __le16 reserved2;
+};
+
+struct ice_aqc_add_elem {
+ struct ice_aqc_txsched_topo_grp_info_hdr hdr;
+ struct ice_aqc_txsched_elem_data generic[1];
+};
+
+struct ice_aqc_get_topo_elem {
+ struct ice_aqc_txsched_topo_grp_info_hdr hdr;
+ struct ice_aqc_txsched_elem_data
+ generic[ICE_AQC_TOPO_MAX_LEVEL_NUM];
+};
+
+struct ice_aqc_delete_elem {
+ struct ice_aqc_txsched_topo_grp_info_hdr hdr;
+ __le32 teid[1];
+};
+
+/* Query Scheduler Resource Allocation (indirect 0x0412)
+ * This indirect command retrieves the scheduler resources allocated by
+ * EMP Firmware to the given PF.
+ */
+struct ice_aqc_query_txsched_res {
+ u8 reserved[8];
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+struct ice_aqc_generic_sched_props {
+ __le16 phys_levels;
+ __le16 logical_levels;
+ u8 flattening_bitmap;
+ u8 max_device_cgds;
+ u8 max_pf_cgds;
+ u8 rsvd0;
+ __le16 rdma_qsets;
+ u8 rsvd1[22];
+};
+
+struct ice_aqc_layer_props {
+ u8 logical_layer;
+ u8 chunk_size;
+ __le16 max_device_nodes;
+ __le16 max_pf_nodes;
+ u8 rsvd0[2];
+ __le16 max_shared_rate_lmtr;
+ __le16 max_children;
+ __le16 max_cir_rl_profiles;
+ __le16 max_eir_rl_profiles;
+ __le16 max_srl_profiles;
+ u8 rsvd1[14];
+};
+
+struct ice_aqc_query_txsched_res_resp {
+ struct ice_aqc_generic_sched_props sched_props;
+ struct ice_aqc_layer_props layer_props[ICE_AQC_TOPO_MAX_LEVEL_NUM];
+};
+
+/* Get PHY capabilities (indirect 0x0600) */
+struct ice_aqc_get_phy_caps {
+ u8 lport_num;
+ u8 reserved;
+ __le16 param0;
+ /* 18.0 - Report qualified modules */
+#define ICE_AQC_GET_PHY_RQM BIT(0)
+ /* 18.1 - 18.2 : Report mode
+ * 00b - Report NVM capabilities
+ * 01b - Report topology capabilities
+ * 10b - Report SW configured
+ */
+#define ICE_AQC_REPORT_MODE_S 1
+#define ICE_AQC_REPORT_MODE_M (3 << ICE_AQC_REPORT_MODE_S)
+#define ICE_AQC_REPORT_NVM_CAP 0
+#define ICE_AQC_REPORT_TOPO_CAP BIT(1)
+#define ICE_AQC_REPORT_SW_CFG BIT(2)
+ __le32 reserved1;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* This is #define of PHY type (Extended):
+ * The first set of defines is for phy_type_low.
+ */
+#define ICE_PHY_TYPE_LOW_100BASE_TX BIT_ULL(0)
+#define ICE_PHY_TYPE_LOW_100M_SGMII BIT_ULL(1)
+#define ICE_PHY_TYPE_LOW_1000BASE_T BIT_ULL(2)
+#define ICE_PHY_TYPE_LOW_1000BASE_SX BIT_ULL(3)
+#define ICE_PHY_TYPE_LOW_1000BASE_LX BIT_ULL(4)
+#define ICE_PHY_TYPE_LOW_1000BASE_KX BIT_ULL(5)
+#define ICE_PHY_TYPE_LOW_1G_SGMII BIT_ULL(6)
+#define ICE_PHY_TYPE_LOW_2500BASE_T BIT_ULL(7)
+#define ICE_PHY_TYPE_LOW_2500BASE_X BIT_ULL(8)
+#define ICE_PHY_TYPE_LOW_2500BASE_KX BIT_ULL(9)
+#define ICE_PHY_TYPE_LOW_5GBASE_T BIT_ULL(10)
+#define ICE_PHY_TYPE_LOW_5GBASE_KR BIT_ULL(11)
+#define ICE_PHY_TYPE_LOW_10GBASE_T BIT_ULL(12)
+#define ICE_PHY_TYPE_LOW_10G_SFI_DA BIT_ULL(13)
+#define ICE_PHY_TYPE_LOW_10GBASE_SR BIT_ULL(14)
+#define ICE_PHY_TYPE_LOW_10GBASE_LR BIT_ULL(15)
+#define ICE_PHY_TYPE_LOW_10GBASE_KR_CR1 BIT_ULL(16)
+#define ICE_PHY_TYPE_LOW_10G_SFI_AOC_ACC BIT_ULL(17)
+#define ICE_PHY_TYPE_LOW_10G_SFI_C2C BIT_ULL(18)
+#define ICE_PHY_TYPE_LOW_25GBASE_T BIT_ULL(19)
+#define ICE_PHY_TYPE_LOW_25GBASE_CR BIT_ULL(20)
+#define ICE_PHY_TYPE_LOW_25GBASE_CR_S BIT_ULL(21)
+#define ICE_PHY_TYPE_LOW_25GBASE_CR1 BIT_ULL(22)
+#define ICE_PHY_TYPE_LOW_25GBASE_SR BIT_ULL(23)
+#define ICE_PHY_TYPE_LOW_25GBASE_LR BIT_ULL(24)
+#define ICE_PHY_TYPE_LOW_25GBASE_KR BIT_ULL(25)
+#define ICE_PHY_TYPE_LOW_25GBASE_KR_S BIT_ULL(26)
+#define ICE_PHY_TYPE_LOW_25GBASE_KR1 BIT_ULL(27)
+#define ICE_PHY_TYPE_LOW_25G_AUI_AOC_ACC BIT_ULL(28)
+#define ICE_PHY_TYPE_LOW_25G_AUI_C2C BIT_ULL(29)
+#define ICE_PHY_TYPE_LOW_40GBASE_CR4 BIT_ULL(30)
+#define ICE_PHY_TYPE_LOW_40GBASE_SR4 BIT_ULL(31)
+#define ICE_PHY_TYPE_LOW_40GBASE_LR4 BIT_ULL(32)
+#define ICE_PHY_TYPE_LOW_40GBASE_KR4 BIT_ULL(33)
+#define ICE_PHY_TYPE_LOW_40G_XLAUI_AOC_ACC BIT_ULL(34)
+#define ICE_PHY_TYPE_LOW_40G_XLAUI BIT_ULL(35)
+#define ICE_PHY_TYPE_LOW_MAX_INDEX 63
+
+struct ice_aqc_get_phy_caps_data {
+ __le64 phy_type_low; /* Use values from ICE_PHY_TYPE_LOW_* */
+ __le64 reserved;
+ u8 caps;
+#define ICE_AQC_PHY_EN_TX_LINK_PAUSE BIT(0)
+#define ICE_AQC_PHY_EN_RX_LINK_PAUSE BIT(1)
+#define ICE_AQC_PHY_LOW_POWER_MODE BIT(2)
+#define ICE_AQC_PHY_EN_LINK BIT(3)
+#define ICE_AQC_PHY_AN_MODE BIT(4)
+#define ICE_AQC_GET_PHY_EN_MOD_QUAL BIT(5)
+ u8 low_power_ctrl;
+#define ICE_AQC_PHY_EN_D3COLD_LOW_POWER_AUTONEG BIT(0)
+ __le16 eee_cap;
+#define ICE_AQC_PHY_EEE_EN_100BASE_TX BIT(0)
+#define ICE_AQC_PHY_EEE_EN_1000BASE_T BIT(1)
+#define ICE_AQC_PHY_EEE_EN_10GBASE_T BIT(2)
+#define ICE_AQC_PHY_EEE_EN_1000BASE_KX BIT(3)
+#define ICE_AQC_PHY_EEE_EN_10GBASE_KR BIT(4)
+#define ICE_AQC_PHY_EEE_EN_25GBASE_KR BIT(5)
+#define ICE_AQC_PHY_EEE_EN_40GBASE_KR4 BIT(6)
+ __le16 eeer_value;
+ u8 phy_id_oui[4]; /* PHY/Module ID connected on the port */
+ u8 link_fec_options;
+#define ICE_AQC_PHY_FEC_10G_KR_40G_KR4_EN BIT(0)
+#define ICE_AQC_PHY_FEC_10G_KR_40G_KR4_REQ BIT(1)
+#define ICE_AQC_PHY_FEC_25G_RS_528_REQ BIT(2)
+#define ICE_AQC_PHY_FEC_25G_KR_REQ BIT(3)
+#define ICE_AQC_PHY_FEC_25G_RS_544_REQ BIT(4)
+#define ICE_AQC_PHY_FEC_25G_RS_CLAUSE91_EN BIT(6)
+#define ICE_AQC_PHY_FEC_25G_KR_CLAUSE74_EN BIT(7)
+ u8 extended_compliance_code;
+#define ICE_MODULE_TYPE_TOTAL_BYTE 3
+ u8 module_type[ICE_MODULE_TYPE_TOTAL_BYTE];
+#define ICE_AQC_MOD_TYPE_BYTE0_SFP_PLUS 0xA0
+#define ICE_AQC_MOD_TYPE_BYTE0_QSFP_PLUS 0x80
+#define ICE_AQC_MOD_TYPE_BYTE1_SFP_PLUS_CU_PASSIVE BIT(0)
+#define ICE_AQC_MOD_TYPE_BYTE1_SFP_PLUS_CU_ACTIVE BIT(1)
+#define ICE_AQC_MOD_TYPE_BYTE1_10G_BASE_SR BIT(4)
+#define ICE_AQC_MOD_TYPE_BYTE1_10G_BASE_LR BIT(5)
+#define ICE_AQC_MOD_TYPE_BYTE1_10G_BASE_LRM BIT(6)
+#define ICE_AQC_MOD_TYPE_BYTE1_10G_BASE_ER BIT(7)
+#define ICE_AQC_MOD_TYPE_BYTE2_SFP_PLUS 0xA0
+#define ICE_AQC_MOD_TYPE_BYTE2_QSFP_PLUS 0x86
+ u8 qualified_module_count;
+#define ICE_AQC_QUAL_MOD_COUNT_MAX 16
+ struct {
+ u8 v_oui[3];
+ u8 rsvd1;
+ u8 v_part[16];
+ __le32 v_rev;
+ __le64 rsvd8;
+ } qual_modules[ICE_AQC_QUAL_MOD_COUNT_MAX];
+};
+
+/* Set PHY capabilities (direct 0x0601)
+ * NOTE: This command must be followed by setup link and restart auto-neg
+ */
+struct ice_aqc_set_phy_cfg {
+ u8 lport_num;
+ u8 reserved[7];
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Set PHY config command data structure */
+struct ice_aqc_set_phy_cfg_data {
+ __le64 phy_type_low; /* Use values from ICE_PHY_TYPE_LOW_* */
+ __le64 rsvd0;
+ u8 caps;
+#define ICE_AQ_PHY_ENA_TX_PAUSE_ABILITY BIT(0)
+#define ICE_AQ_PHY_ENA_RX_PAUSE_ABILITY BIT(1)
+#define ICE_AQ_PHY_ENA_LOW_POWER BIT(2)
+#define ICE_AQ_PHY_ENA_LINK BIT(3)
+#define ICE_AQ_PHY_ENA_ATOMIC_LINK BIT(5)
+ u8 low_power_ctrl;
+ __le16 eee_cap; /* Value from ice_aqc_get_phy_caps */
+ __le16 eeer_value;
+ u8 link_fec_opt; /* Use defines from ice_aqc_get_phy_caps */
+ u8 rsvd1;
+};
+
+/* Restart AN command data structure (direct 0x0605)
+ * Also used for response, with only the lport_num field present.
+ */
+struct ice_aqc_restart_an {
+ u8 lport_num;
+ u8 reserved;
+ u8 cmd_flags;
+#define ICE_AQC_RESTART_AN_LINK_RESTART BIT(1)
+#define ICE_AQC_RESTART_AN_LINK_ENABLE BIT(2)
+ u8 reserved2[13];
+};
+
+/* Get link status (indirect 0x0607), also used for Link Status Event */
+struct ice_aqc_get_link_status {
+ u8 lport_num;
+ u8 reserved;
+ __le16 cmd_flags;
+#define ICE_AQ_LSE_M 0x3
+#define ICE_AQ_LSE_NOP 0x0
+#define ICE_AQ_LSE_DIS 0x2
+#define ICE_AQ_LSE_ENA 0x3
+ /* only response uses this flag */
+#define ICE_AQ_LSE_IS_ENABLED 0x1
+ __le32 reserved2;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Get link status response data structure, also used for Link Status Event */
+struct ice_aqc_get_link_status_data {
+ u8 topo_media_conflict;
+#define ICE_AQ_LINK_TOPO_CONFLICT BIT(0)
+#define ICE_AQ_LINK_MEDIA_CONFLICT BIT(1)
+#define ICE_AQ_LINK_TOPO_CORRUPT BIT(2)
+ u8 reserved1;
+ u8 link_info;
+#define ICE_AQ_LINK_UP BIT(0) /* Link Status */
+#define ICE_AQ_LINK_FAULT BIT(1)
+#define ICE_AQ_LINK_FAULT_TX BIT(2)
+#define ICE_AQ_LINK_FAULT_RX BIT(3)
+#define ICE_AQ_LINK_FAULT_REMOTE BIT(4)
+#define ICE_AQ_LINK_UP_PORT BIT(5) /* External Port Link Status */
+#define ICE_AQ_MEDIA_AVAILABLE BIT(6)
+#define ICE_AQ_SIGNAL_DETECT BIT(7)
+ u8 an_info;
+#define ICE_AQ_AN_COMPLETED BIT(0)
+#define ICE_AQ_LP_AN_ABILITY BIT(1)
+#define ICE_AQ_PD_FAULT BIT(2) /* Parallel Detection Fault */
+#define ICE_AQ_FEC_EN BIT(3)
+#define ICE_AQ_PHY_LOW_POWER BIT(4) /* Low Power State */
+#define ICE_AQ_LINK_PAUSE_TX BIT(5)
+#define ICE_AQ_LINK_PAUSE_RX BIT(6)
+#define ICE_AQ_QUALIFIED_MODULE BIT(7)
+ u8 ext_info;
+#define ICE_AQ_LINK_PHY_TEMP_ALARM BIT(0)
+#define ICE_AQ_LINK_EXCESSIVE_ERRORS BIT(1) /* Excessive Link Errors */
+ /* Port TX Suspended */
+#define ICE_AQ_LINK_TX_S 2
+#define ICE_AQ_LINK_TX_M (0x03 << ICE_AQ_LINK_TX_S)
+#define ICE_AQ_LINK_TX_ACTIVE 0
+#define ICE_AQ_LINK_TX_DRAINED 1
+#define ICE_AQ_LINK_TX_FLUSHED 3
+ u8 reserved2;
+ __le16 max_frame_size;
+ u8 cfg;
+#define ICE_AQ_LINK_25G_KR_FEC_EN BIT(0)
+#define ICE_AQ_LINK_25G_RS_528_FEC_EN BIT(1)
+#define ICE_AQ_LINK_25G_RS_544_FEC_EN BIT(2)
+ /* Pacing Config */
+#define ICE_AQ_CFG_PACING_S 3
+#define ICE_AQ_CFG_PACING_M (0xF << ICE_AQ_CFG_PACING_S)
+#define ICE_AQ_CFG_PACING_TYPE_M BIT(7)
+#define ICE_AQ_CFG_PACING_TYPE_AVG 0
+#define ICE_AQ_CFG_PACING_TYPE_FIXED ICE_AQ_CFG_PACING_TYPE_M
+ /* External Device Power Ability */
+ u8 power_desc;
+#define ICE_AQ_PWR_CLASS_M 0x3
+#define ICE_AQ_LINK_PWR_BASET_LOW_HIGH 0
+#define ICE_AQ_LINK_PWR_BASET_HIGH 1
+#define ICE_AQ_LINK_PWR_QSFP_CLASS_1 0
+#define ICE_AQ_LINK_PWR_QSFP_CLASS_2 1
+#define ICE_AQ_LINK_PWR_QSFP_CLASS_3 2
+#define ICE_AQ_LINK_PWR_QSFP_CLASS_4 3
+ __le16 link_speed;
+#define ICE_AQ_LINK_SPEED_10MB BIT(0)
+#define ICE_AQ_LINK_SPEED_100MB BIT(1)
+#define ICE_AQ_LINK_SPEED_1000MB BIT(2)
+#define ICE_AQ_LINK_SPEED_2500MB BIT(3)
+#define ICE_AQ_LINK_SPEED_5GB BIT(4)
+#define ICE_AQ_LINK_SPEED_10GB BIT(5)
+#define ICE_AQ_LINK_SPEED_20GB BIT(6)
+#define ICE_AQ_LINK_SPEED_25GB BIT(7)
+#define ICE_AQ_LINK_SPEED_40GB BIT(8)
+#define ICE_AQ_LINK_SPEED_UNKNOWN BIT(15)
+ __le32 reserved3; /* Aligns next field to 8-byte boundary */
+ __le64 phy_type_low; /* Use values from ICE_PHY_TYPE_LOW_* */
+ __le64 reserved4;
+};
+
+/* Set event mask command (direct 0x0613) */
+struct ice_aqc_set_event_mask {
+ u8 lport_num;
+ u8 reserved[7];
+ __le16 event_mask;
+#define ICE_AQ_LINK_EVENT_UPDOWN BIT(1)
+#define ICE_AQ_LINK_EVENT_MEDIA_NA BIT(2)
+#define ICE_AQ_LINK_EVENT_LINK_FAULT BIT(3)
+#define ICE_AQ_LINK_EVENT_PHY_TEMP_ALARM BIT(4)
+#define ICE_AQ_LINK_EVENT_EXCESSIVE_ERRORS BIT(5)
+#define ICE_AQ_LINK_EVENT_SIGNAL_DETECT BIT(6)
+#define ICE_AQ_LINK_EVENT_AN_COMPLETED BIT(7)
+#define ICE_AQ_LINK_EVENT_MODULE_QUAL_FAIL BIT(8)
+#define ICE_AQ_LINK_EVENT_PORT_TX_SUSPENDED BIT(9)
+ u8 reserved1[6];
+};
+
+/* NVM Read command (indirect 0x0701)
+ * NVM Erase commands (direct 0x0702)
+ * NVM Update commands (indirect 0x0703)
+ */
+struct ice_aqc_nvm {
+ u8 cmd_flags;
+#define ICE_AQC_NVM_LAST_CMD BIT(0)
+#define ICE_AQC_NVM_PCIR_REQ BIT(0) /* Used by NVM Update reply */
+#define ICE_AQC_NVM_PRESERVATION_S 1
+#define ICE_AQC_NVM_PRESERVATION_M (3 << CSR_AQ_NVM_PRESERVATION_S)
+#define ICE_AQC_NVM_NO_PRESERVATION (0 << CSR_AQ_NVM_PRESERVATION_S)
+#define ICE_AQC_NVM_PRESERVE_ALL BIT(1)
+#define ICE_AQC_NVM_PRESERVE_SELECTED (3 << CSR_AQ_NVM_PRESERVATION_S)
+#define ICE_AQC_NVM_FLASH_ONLY BIT(7)
+ u8 module_typeid;
+ __le16 length;
+#define ICE_AQC_NVM_ERASE_LEN 0xFFFF
+ __le32 offset;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Get/Set RSS key (indirect 0x0B04/0x0B02) */
+struct ice_aqc_get_set_rss_key {
+#define ICE_AQC_GSET_RSS_KEY_VSI_VALID BIT(15)
+#define ICE_AQC_GSET_RSS_KEY_VSI_ID_S 0
+#define ICE_AQC_GSET_RSS_KEY_VSI_ID_M (0x3FF << ICE_AQC_GSET_RSS_KEY_VSI_ID_S)
+ __le16 vsi_id;
+ u8 reserved[6];
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+#define ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE 0x28
+#define ICE_AQC_GET_SET_RSS_KEY_DATA_HASH_KEY_SIZE 0xC
+
+struct ice_aqc_get_set_rss_keys {
+ u8 standard_rss_key[ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE];
+ u8 extended_hash_key[ICE_AQC_GET_SET_RSS_KEY_DATA_HASH_KEY_SIZE];
+};
+
+/* Get/Set RSS LUT (indirect 0x0B05/0x0B03) */
+struct ice_aqc_get_set_rss_lut {
+#define ICE_AQC_GSET_RSS_LUT_VSI_VALID BIT(15)
+#define ICE_AQC_GSET_RSS_LUT_VSI_ID_S 0
+#define ICE_AQC_GSET_RSS_LUT_VSI_ID_M (0x1FF << ICE_AQC_GSET_RSS_LUT_VSI_ID_S)
+ __le16 vsi_id;
+#define ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_S 0
+#define ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_M \
+ (0x3 << ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_S)
+
+#define ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_VSI 0
+#define ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_PF 1
+#define ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_GLOBAL 2
+
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_S 2
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_M \
+ (0x3 << ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_S)
+
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_128 128
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_128_FLAG 0
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_512 512
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_512_FLAG 1
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_2K 2048
+#define ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_2K_FLAG 2
+
+#define ICE_AQC_GSET_RSS_LUT_GLOBAL_IDX_S 4
+#define ICE_AQC_GSET_RSS_LUT_GLOBAL_IDX_M \
+ (0xF << ICE_AQC_GSET_RSS_LUT_GLOBAL_IDX_S)
+
+ __le16 flags;
+ __le32 reserved;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* Add TX LAN Queues (indirect 0x0C30) */
+struct ice_aqc_add_txqs {
+ u8 num_qgrps;
+ u8 reserved[3];
+ __le32 reserved1;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* This is the descriptor of each queue entry for the Add TX LAN Queues
+ * command (0x0C30). Only used within struct ice_aqc_add_tx_qgrp.
+ */
+struct ice_aqc_add_txqs_perq {
+ __le16 txq_id;
+ u8 rsvd[2];
+ __le32 q_teid;
+ u8 txq_ctx[22];
+ u8 rsvd2[2];
+ struct ice_aqc_txsched_elem info;
+};
+
+/* The format of the command buffer for Add TX LAN Queues (0x0C30)
+ * is an array of the following structs. Please note that the length of
+ * each struct ice_aqc_add_tx_qgrp is variable due
+ * to the variable number of queues in each group!
+ */
+struct ice_aqc_add_tx_qgrp {
+ __le32 parent_teid;
+ u8 num_txqs;
+ u8 rsvd[3];
+ struct ice_aqc_add_txqs_perq txqs[1];
+};
+
+/* Disable TX LAN Queues (indirect 0x0C31) */
+struct ice_aqc_dis_txqs {
+ u8 cmd_type;
+#define ICE_AQC_Q_DIS_CMD_S 0
+#define ICE_AQC_Q_DIS_CMD_M (0x3 << ICE_AQC_Q_DIS_CMD_S)
+#define ICE_AQC_Q_DIS_CMD_NO_FUNC_RESET (0 << ICE_AQC_Q_DIS_CMD_S)
+#define ICE_AQC_Q_DIS_CMD_VM_RESET BIT(ICE_AQC_Q_DIS_CMD_S)
+#define ICE_AQC_Q_DIS_CMD_VF_RESET (2 << ICE_AQC_Q_DIS_CMD_S)
+#define ICE_AQC_Q_DIS_CMD_PF_RESET (3 << ICE_AQC_Q_DIS_CMD_S)
+#define ICE_AQC_Q_DIS_CMD_SUBSEQ_CALL BIT(2)
+#define ICE_AQC_Q_DIS_CMD_FLUSH_PIPE BIT(3)
+ u8 num_entries;
+ __le16 vmvf_and_timeout;
+#define ICE_AQC_Q_DIS_VMVF_NUM_S 0
+#define ICE_AQC_Q_DIS_VMVF_NUM_M (0x3FF << ICE_AQC_Q_DIS_VMVF_NUM_S)
+#define ICE_AQC_Q_DIS_TIMEOUT_S 10
+#define ICE_AQC_Q_DIS_TIMEOUT_M (0x3F << ICE_AQC_Q_DIS_TIMEOUT_S)
+ __le32 blocked_cgds;
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* The buffer for Disable TX LAN Queues (indirect 0x0C31)
+ * contains the following structures, arrayed one after the
+ * other.
+ * Note: Since the q_id is 16 bits wide, if the
+ * number of queues is even, then 2 bytes of alignment MUST be
+ * added before the start of the next group, to allow correct
+ * alignment of the parent_teid field.
+ */
+struct ice_aqc_dis_txq_item {
+ __le32 parent_teid;
+ u8 num_qs;
+ u8 rsvd;
+ /* The length of the q_id array varies according to num_qs */
+ __le16 q_id[1];
+ /* This only applies from F8 onward */
+#define ICE_AQC_Q_DIS_BUF_ELEM_TYPE_S 15
+#define ICE_AQC_Q_DIS_BUF_ELEM_TYPE_LAN_Q \
+ (0 << ICE_AQC_Q_DIS_BUF_ELEM_TYPE_S)
+#define ICE_AQC_Q_DIS_BUF_ELEM_TYPE_RDMA_QSET \
+ (1 << ICE_AQC_Q_DIS_BUF_ELEM_TYPE_S)
+};
+
+struct ice_aqc_dis_txq {
+ struct ice_aqc_dis_txq_item qgrps[1];
+};
+
+/**
+ * struct ice_aq_desc - Admin Queue (AQ) descriptor
+ * @flags: ICE_AQ_FLAG_* flags
+ * @opcode: AQ command opcode
+ * @datalen: length in bytes of indirect/external data buffer
+ * @retval: return value from firmware
+ * @cookie_h: opaque data high-half
+ * @cookie_l: opaque data low-half
+ * @params: command-specific parameters
+ *
+ * Descriptor format for commands the driver posts on the Admin Transmit Queue
+ * (ATQ). The firmware writes back onto the command descriptor and returns
+ * the result of the command. Asynchronous events that are not an immediate
+ * result of the command are written to the Admin Receive Queue (ARQ) using
+ * the same descriptor format. Descriptors are in little-endian notation with
+ * 32-bit words.
+ */
+struct ice_aq_desc {
+ __le16 flags;
+ __le16 opcode;
+ __le16 datalen;
+ __le16 retval;
+ __le32 cookie_high;
+ __le32 cookie_low;
+ union {
+ u8 raw[16];
+ struct ice_aqc_generic generic;
+ struct ice_aqc_get_ver get_ver;
+ struct ice_aqc_q_shutdown q_shutdown;
+ struct ice_aqc_req_res res_owner;
+ struct ice_aqc_manage_mac_read mac_read;
+ struct ice_aqc_manage_mac_write mac_write;
+ struct ice_aqc_clear_pxe clear_pxe;
+ struct ice_aqc_list_caps get_cap;
+ struct ice_aqc_get_phy_caps get_phy;
+ struct ice_aqc_set_phy_cfg set_phy;
+ struct ice_aqc_restart_an restart_an;
+ struct ice_aqc_get_sw_cfg get_sw_conf;
+ struct ice_aqc_sw_rules sw_rules;
+ struct ice_aqc_get_topo get_topo;
+ struct ice_aqc_get_cfg_elem get_update_elem;
+ struct ice_aqc_query_txsched_res query_sched_res;
+ struct ice_aqc_add_move_delete_elem add_move_delete_elem;
+ struct ice_aqc_nvm nvm;
+ struct ice_aqc_get_set_rss_lut get_set_rss_lut;
+ struct ice_aqc_get_set_rss_key get_set_rss_key;
+ struct ice_aqc_add_txqs add_txqs;
+ struct ice_aqc_dis_txqs dis_txqs;
+ struct ice_aqc_add_get_update_free_vsi vsi_cmd;
+ struct ice_aqc_alloc_free_res_cmd sw_res_ctrl;
+ struct ice_aqc_set_event_mask set_event_mask;
+ struct ice_aqc_get_link_status get_link_status;
+ } params;
+};
+
+/* FW defined boundary for a large buffer, 4k >= Large buffer > 512 bytes */
+#define ICE_AQ_LG_BUF 512
+
+#define ICE_AQ_FLAG_ERR_S 2
+#define ICE_AQ_FLAG_LB_S 9
+#define ICE_AQ_FLAG_RD_S 10
+#define ICE_AQ_FLAG_BUF_S 12
+#define ICE_AQ_FLAG_SI_S 13
+
+#define ICE_AQ_FLAG_ERR BIT(ICE_AQ_FLAG_ERR_S) /* 0x4 */
+#define ICE_AQ_FLAG_LB BIT(ICE_AQ_FLAG_LB_S) /* 0x200 */
+#define ICE_AQ_FLAG_RD BIT(ICE_AQ_FLAG_RD_S) /* 0x400 */
+#define ICE_AQ_FLAG_BUF BIT(ICE_AQ_FLAG_BUF_S) /* 0x1000 */
+#define ICE_AQ_FLAG_SI BIT(ICE_AQ_FLAG_SI_S) /* 0x2000 */
+
+/* error codes */
+enum ice_aq_err {
+ ICE_AQ_RC_OK = 0, /* success */
+ ICE_AQ_RC_ENOMEM = 9, /* Out of memory */
+ ICE_AQ_RC_EBUSY = 12, /* Device or resource busy */
+ ICE_AQ_RC_EEXIST = 13, /* object already exists */
+ ICE_AQ_RC_ENOSPC = 16, /* No space left or allocation failure */
+};
+
+/* Admin Queue command opcodes */
+enum ice_adminq_opc {
+ /* AQ commands */
+ ice_aqc_opc_get_ver = 0x0001,
+ ice_aqc_opc_q_shutdown = 0x0003,
+
+ /* resource ownership */
+ ice_aqc_opc_req_res = 0x0008,
+ ice_aqc_opc_release_res = 0x0009,
+
+ /* device/function capabilities */
+ ice_aqc_opc_list_func_caps = 0x000A,
+ ice_aqc_opc_list_dev_caps = 0x000B,
+
+ /* manage MAC address */
+ ice_aqc_opc_manage_mac_read = 0x0107,
+ ice_aqc_opc_manage_mac_write = 0x0108,
+
+ /* PXE */
+ ice_aqc_opc_clear_pxe_mode = 0x0110,
+
+ /* internal switch commands */
+ ice_aqc_opc_get_sw_cfg = 0x0200,
+
+ /* Alloc/Free/Get Resources */
+ ice_aqc_opc_alloc_res = 0x0208,
+ ice_aqc_opc_free_res = 0x0209,
+
+ /* VSI commands */
+ ice_aqc_opc_add_vsi = 0x0210,
+ ice_aqc_opc_update_vsi = 0x0211,
+ ice_aqc_opc_free_vsi = 0x0213,
+
+ /* switch rules population commands */
+ ice_aqc_opc_add_sw_rules = 0x02A0,
+ ice_aqc_opc_update_sw_rules = 0x02A1,
+ ice_aqc_opc_remove_sw_rules = 0x02A2,
+
+ ice_aqc_opc_clear_pf_cfg = 0x02A4,
+
+ /* transmit scheduler commands */
+ ice_aqc_opc_get_dflt_topo = 0x0400,
+ ice_aqc_opc_add_sched_elems = 0x0401,
+ ice_aqc_opc_suspend_sched_elems = 0x0409,
+ ice_aqc_opc_resume_sched_elems = 0x040A,
+ ice_aqc_opc_delete_sched_elems = 0x040F,
+ ice_aqc_opc_query_sched_res = 0x0412,
+
+ /* PHY commands */
+ ice_aqc_opc_get_phy_caps = 0x0600,
+ ice_aqc_opc_set_phy_cfg = 0x0601,
+ ice_aqc_opc_restart_an = 0x0605,
+ ice_aqc_opc_get_link_status = 0x0607,
+ ice_aqc_opc_set_event_mask = 0x0613,
+
+ /* NVM commands */
+ ice_aqc_opc_nvm_read = 0x0701,
+
+ /* RSS commands */
+ ice_aqc_opc_set_rss_key = 0x0B02,
+ ice_aqc_opc_set_rss_lut = 0x0B03,
+ ice_aqc_opc_get_rss_key = 0x0B04,
+ ice_aqc_opc_get_rss_lut = 0x0B05,
+
+ /* TX queue handling commands/events */
+ ice_aqc_opc_add_txqs = 0x0C30,
+ ice_aqc_opc_dis_txqs = 0x0C31,
+};
+
+#endif /* _ICE_ADMINQ_CMD_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+#include "ice_common.h"
+#include "ice_sched.h"
+#include "ice_adminq_cmd.h"
+
+#define ICE_PF_RESET_WAIT_COUNT 200
+
+#define ICE_NIC_FLX_ENTRY(hw, mdid, idx) \
+ wr32((hw), GLFLXP_RXDID_FLX_WRD_##idx(ICE_RXDID_FLEX_NIC), \
+ ((ICE_RX_OPC_MDID << \
+ GLFLXP_RXDID_FLX_WRD_##idx##_RXDID_OPCODE_S) & \
+ GLFLXP_RXDID_FLX_WRD_##idx##_RXDID_OPCODE_M) | \
+ (((mdid) << GLFLXP_RXDID_FLX_WRD_##idx##_PROT_MDID_S) & \
+ GLFLXP_RXDID_FLX_WRD_##idx##_PROT_MDID_M))
+
+#define ICE_NIC_FLX_FLG_ENTRY(hw, flg_0, flg_1, flg_2, flg_3, idx) \
+ wr32((hw), GLFLXP_RXDID_FLAGS(ICE_RXDID_FLEX_NIC, idx), \
+ (((flg_0) << GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S) & \
+ GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M) | \
+ (((flg_1) << GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_1_S) & \
+ GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_1_M) | \
+ (((flg_2) << GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_2_S) & \
+ GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_2_M) | \
+ (((flg_3) << GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_3_S) & \
+ GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_3_M))
+
+/**
+ * ice_set_mac_type - Sets MAC type
+ * @hw: pointer to the HW structure
+ *
+ * This function sets the MAC type of the adapter based on the
+ * vendor ID and device ID stored in the hw structure.
+ */
+static enum ice_status ice_set_mac_type(struct ice_hw *hw)
+{
+ if (hw->vendor_id != PCI_VENDOR_ID_INTEL)
+ return ICE_ERR_DEVICE_NOT_SUPPORTED;
+
+ hw->mac_type = ICE_MAC_GENERIC;
+ return 0;
+}
+
+/**
+ * ice_clear_pf_cfg - Clear PF configuration
+ * @hw: pointer to the hardware structure
+ */
+enum ice_status ice_clear_pf_cfg(struct ice_hw *hw)
+{
+ struct ice_aq_desc desc;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_clear_pf_cfg);
+
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, NULL);
+}
+
+/**
+ * ice_aq_manage_mac_read - manage MAC address read command
+ * @hw: pointer to the hw struct
+ * @buf: a virtual buffer to hold the manage MAC read response
+ * @buf_size: Size of the virtual buffer
+ * @cd: pointer to command details structure or NULL
+ *
+ * This function is used to return per PF station MAC address (0x0107).
+ * NOTE: Upon successful completion of this command, MAC address information
+ * is returned in user specified buffer. Please interpret user specified
+ * buffer as "manage_mac_read" response.
+ * Response such as various MAC addresses are stored in HW struct (port.mac)
+ * ice_aq_discover_caps is expected to be called before this function is called.
+ */
+static enum ice_status
+ice_aq_manage_mac_read(struct ice_hw *hw, void *buf, u16 buf_size,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_manage_mac_read_resp *resp;
+ struct ice_aqc_manage_mac_read *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+ u16 flags;
+
+ cmd = &desc.params.mac_read;
+
+ if (buf_size < sizeof(*resp))
+ return ICE_ERR_BUF_TOO_SHORT;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_manage_mac_read);
+
+ status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+ if (status)
+ return status;
+
+ resp = (struct ice_aqc_manage_mac_read_resp *)buf;
+ flags = le16_to_cpu(cmd->flags) & ICE_AQC_MAN_MAC_READ_M;
+
+ if (!(flags & ICE_AQC_MAN_MAC_LAN_ADDR_VALID)) {
+ ice_debug(hw, ICE_DBG_LAN, "got invalid MAC address\n");
+ return ICE_ERR_CFG;
+ }
+
+ ether_addr_copy(hw->port_info->mac.lan_addr, resp->mac_addr);
+ ether_addr_copy(hw->port_info->mac.perm_addr, resp->mac_addr);
+ return 0;
+}
+
+/**
+ * ice_aq_get_phy_caps - returns PHY capabilities
+ * @pi: port information structure
+ * @qual_mods: report qualified modules
+ * @report_mode: report mode capabilities
+ * @pcaps: structure for PHY capabilities to be filled
+ * @cd: pointer to command details structure or NULL
+ *
+ * Returns the various PHY capabilities supported on the Port (0x0600)
+ */
+static enum ice_status
+ice_aq_get_phy_caps(struct ice_port_info *pi, bool qual_mods, u8 report_mode,
+ struct ice_aqc_get_phy_caps_data *pcaps,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_get_phy_caps *cmd;
+ u16 pcaps_size = sizeof(*pcaps);
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.get_phy;
+
+ if (!pcaps || (report_mode & ~ICE_AQC_REPORT_MODE_M) || !pi)
+ return ICE_ERR_PARAM;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_phy_caps);
+
+ if (qual_mods)
+ cmd->param0 |= cpu_to_le16(ICE_AQC_GET_PHY_RQM);
+
+ cmd->param0 |= cpu_to_le16(report_mode);
+ status = ice_aq_send_cmd(pi->hw, &desc, pcaps, pcaps_size, cd);
+
+ if (!status && report_mode == ICE_AQC_REPORT_TOPO_CAP)
+ pi->phy.phy_type_low = le64_to_cpu(pcaps->phy_type_low);
+
+ return status;
+}
+
+/**
+ * ice_get_media_type - Gets media type
+ * @pi: port information structure
+ */
+static enum ice_media_type ice_get_media_type(struct ice_port_info *pi)
+{
+ struct ice_link_status *hw_link_info;
+
+ if (!pi)
+ return ICE_MEDIA_UNKNOWN;
+
+ hw_link_info = &pi->phy.link_info;
+
+ if (hw_link_info->phy_type_low) {
+ switch (hw_link_info->phy_type_low) {
+ case ICE_PHY_TYPE_LOW_1000BASE_SX:
+ case ICE_PHY_TYPE_LOW_1000BASE_LX:
+ case ICE_PHY_TYPE_LOW_10GBASE_SR:
+ case ICE_PHY_TYPE_LOW_10GBASE_LR:
+ case ICE_PHY_TYPE_LOW_10G_SFI_C2C:
+ case ICE_PHY_TYPE_LOW_25GBASE_SR:
+ case ICE_PHY_TYPE_LOW_25GBASE_LR:
+ case ICE_PHY_TYPE_LOW_25G_AUI_C2C:
+ case ICE_PHY_TYPE_LOW_40GBASE_SR4:
+ case ICE_PHY_TYPE_LOW_40GBASE_LR4:
+ return ICE_MEDIA_FIBER;
+ case ICE_PHY_TYPE_LOW_100BASE_TX:
+ case ICE_PHY_TYPE_LOW_1000BASE_T:
+ case ICE_PHY_TYPE_LOW_2500BASE_T:
+ case ICE_PHY_TYPE_LOW_5GBASE_T:
+ case ICE_PHY_TYPE_LOW_10GBASE_T:
+ case ICE_PHY_TYPE_LOW_25GBASE_T:
+ return ICE_MEDIA_BASET;
+ case ICE_PHY_TYPE_LOW_10G_SFI_DA:
+ case ICE_PHY_TYPE_LOW_25GBASE_CR:
+ case ICE_PHY_TYPE_LOW_25GBASE_CR_S:
+ case ICE_PHY_TYPE_LOW_25GBASE_CR1:
+ case ICE_PHY_TYPE_LOW_40GBASE_CR4:
+ return ICE_MEDIA_DA;
+ case ICE_PHY_TYPE_LOW_1000BASE_KX:
+ case ICE_PHY_TYPE_LOW_2500BASE_KX:
+ case ICE_PHY_TYPE_LOW_2500BASE_X:
+ case ICE_PHY_TYPE_LOW_5GBASE_KR:
+ case ICE_PHY_TYPE_LOW_10GBASE_KR_CR1:
+ case ICE_PHY_TYPE_LOW_25GBASE_KR:
+ case ICE_PHY_TYPE_LOW_25GBASE_KR1:
+ case ICE_PHY_TYPE_LOW_25GBASE_KR_S:
+ case ICE_PHY_TYPE_LOW_40GBASE_KR4:
+ return ICE_MEDIA_BACKPLANE;
+ }
+ }
+
+ return ICE_MEDIA_UNKNOWN;
+}
+
+/**
+ * ice_aq_get_link_info
+ * @pi: port information structure
+ * @ena_lse: enable/disable LinkStatusEvent reporting
+ * @link: pointer to link status structure - optional
+ * @cd: pointer to command details structure or NULL
+ *
+ * Get Link Status (0x607). Returns the link status of the adapter.
+ */
+enum ice_status
+ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
+ struct ice_link_status *link, struct ice_sq_cd *cd)
+{
+ struct ice_link_status *hw_link_info_old, *hw_link_info;
+ struct ice_aqc_get_link_status_data link_data = { 0 };
+ struct ice_aqc_get_link_status *resp;
+ enum ice_media_type *hw_media_type;
+ struct ice_fc_info *hw_fc_info;
+ bool tx_pause, rx_pause;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+ u16 cmd_flags;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+ hw_link_info_old = &pi->phy.link_info_old;
+ hw_media_type = &pi->phy.media_type;
+ hw_link_info = &pi->phy.link_info;
+ hw_fc_info = &pi->fc;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_link_status);
+ cmd_flags = (ena_lse) ? ICE_AQ_LSE_ENA : ICE_AQ_LSE_DIS;
+ resp = &desc.params.get_link_status;
+ resp->cmd_flags = cpu_to_le16(cmd_flags);
+ resp->lport_num = pi->lport;
+
+ status = ice_aq_send_cmd(pi->hw, &desc, &link_data, sizeof(link_data),
+ cd);
+
+ if (status)
+ return status;
+
+ /* save off old link status information */
+ *hw_link_info_old = *hw_link_info;
+
+ /* update current link status information */
+ hw_link_info->link_speed = le16_to_cpu(link_data.link_speed);
+ hw_link_info->phy_type_low = le64_to_cpu(link_data.phy_type_low);
+ *hw_media_type = ice_get_media_type(pi);
+ hw_link_info->link_info = link_data.link_info;
+ hw_link_info->an_info = link_data.an_info;
+ hw_link_info->ext_info = link_data.ext_info;
+ hw_link_info->max_frame_size = le16_to_cpu(link_data.max_frame_size);
+ hw_link_info->pacing = link_data.cfg & ICE_AQ_CFG_PACING_M;
+
+ /* update fc info */
+ tx_pause = !!(link_data.an_info & ICE_AQ_LINK_PAUSE_TX);
+ rx_pause = !!(link_data.an_info & ICE_AQ_LINK_PAUSE_RX);
+ if (tx_pause && rx_pause)
+ hw_fc_info->current_mode = ICE_FC_FULL;
+ else if (tx_pause)
+ hw_fc_info->current_mode = ICE_FC_TX_PAUSE;
+ else if (rx_pause)
+ hw_fc_info->current_mode = ICE_FC_RX_PAUSE;
+ else
+ hw_fc_info->current_mode = ICE_FC_NONE;
+
+ hw_link_info->lse_ena =
+ !!(resp->cmd_flags & cpu_to_le16(ICE_AQ_LSE_IS_ENABLED));
+
+ /* save link status information */
+ if (link)
+ *link = *hw_link_info;
+
+ /* flag cleared so calling functions don't call AQ again */
+ pi->phy.get_link_info = false;
+
+ return status;
+}
+
+/**
+ * ice_init_flex_parser - initialize rx flex parser
+ * @hw: pointer to the hardware structure
+ *
+ * Function to initialize flex descriptors
+ */
+static void ice_init_flex_parser(struct ice_hw *hw)
+{
+ u8 idx = 0;
+
+ ICE_NIC_FLX_ENTRY(hw, ICE_RX_MDID_HASH_LOW, 0);
+ ICE_NIC_FLX_ENTRY(hw, ICE_RX_MDID_HASH_HIGH, 1);
+ ICE_NIC_FLX_ENTRY(hw, ICE_RX_MDID_FLOW_ID_LOWER, 2);
+ ICE_NIC_FLX_ENTRY(hw, ICE_RX_MDID_FLOW_ID_HIGH, 3);
+ ICE_NIC_FLX_FLG_ENTRY(hw, ICE_RXFLG_PKT_FRG, ICE_RXFLG_UDP_GRE,
+ ICE_RXFLG_PKT_DSI, ICE_RXFLG_FIN, idx++);
+ ICE_NIC_FLX_FLG_ENTRY(hw, ICE_RXFLG_SYN, ICE_RXFLG_RST,
+ ICE_RXFLG_PKT_DSI, ICE_RXFLG_PKT_DSI, idx++);
+ ICE_NIC_FLX_FLG_ENTRY(hw, ICE_RXFLG_PKT_DSI, ICE_RXFLG_PKT_DSI,
+ ICE_RXFLG_EVLAN_x8100, ICE_RXFLG_EVLAN_x9100,
+ idx++);
+ ICE_NIC_FLX_FLG_ENTRY(hw, ICE_RXFLG_VLAN_x8100, ICE_RXFLG_TNL_VLAN,
+ ICE_RXFLG_TNL_MAC, ICE_RXFLG_TNL0, idx++);
+ ICE_NIC_FLX_FLG_ENTRY(hw, ICE_RXFLG_TNL1, ICE_RXFLG_TNL2,
+ ICE_RXFLG_PKT_DSI, ICE_RXFLG_PKT_DSI, idx);
+}
+
+/**
+ * ice_init_fltr_mgmt_struct - initializes filter management list and locks
+ * @hw: pointer to the hw struct
+ */
+static enum ice_status ice_init_fltr_mgmt_struct(struct ice_hw *hw)
+{
+ struct ice_switch_info *sw;
+
+ hw->switch_info = devm_kzalloc(ice_hw_to_dev(hw),
+ sizeof(*hw->switch_info), GFP_KERNEL);
+ sw = hw->switch_info;
+
+ if (!sw)
+ return ICE_ERR_NO_MEMORY;
+
+ INIT_LIST_HEAD(&sw->vsi_list_map_head);
+
+ mutex_init(&sw->mac_list_lock);
+ INIT_LIST_HEAD(&sw->mac_list_head);
+
+ mutex_init(&sw->vlan_list_lock);
+ INIT_LIST_HEAD(&sw->vlan_list_head);
+
+ mutex_init(&sw->eth_m_list_lock);
+ INIT_LIST_HEAD(&sw->eth_m_list_head);
+
+ mutex_init(&sw->promisc_list_lock);
+ INIT_LIST_HEAD(&sw->promisc_list_head);
+
+ mutex_init(&sw->mac_vlan_list_lock);
+ INIT_LIST_HEAD(&sw->mac_vlan_list_head);
+
+ return 0;
+}
+
+/**
+ * ice_cleanup_fltr_mgmt_struct - cleanup filter management list and locks
+ * @hw: pointer to the hw struct
+ */
+static void ice_cleanup_fltr_mgmt_struct(struct ice_hw *hw)
+{
+ struct ice_switch_info *sw = hw->switch_info;
+ struct ice_vsi_list_map_info *v_pos_map;
+ struct ice_vsi_list_map_info *v_tmp_map;
+
+ list_for_each_entry_safe(v_pos_map, v_tmp_map, &sw->vsi_list_map_head,
+ list_entry) {
+ list_del(&v_pos_map->list_entry);
+ devm_kfree(ice_hw_to_dev(hw), v_pos_map);
+ }
+
+ mutex_destroy(&sw->mac_list_lock);
+ mutex_destroy(&sw->vlan_list_lock);
+ mutex_destroy(&sw->eth_m_list_lock);
+ mutex_destroy(&sw->promisc_list_lock);
+ mutex_destroy(&sw->mac_vlan_list_lock);
+
+ devm_kfree(ice_hw_to_dev(hw), sw);
+}
+
+/**
+ * ice_init_hw - main hardware initialization routine
+ * @hw: pointer to the hardware structure
+ */
+enum ice_status ice_init_hw(struct ice_hw *hw)
+{
+ struct ice_aqc_get_phy_caps_data *pcaps;
+ enum ice_status status;
+ u16 mac_buf_len;
+ void *mac_buf;
+
+ /* Set MAC type based on DeviceID */
+ status = ice_set_mac_type(hw);
+ if (status)
+ return status;
+
+ hw->pf_id = (u8)(rd32(hw, PF_FUNC_RID) &
+ PF_FUNC_RID_FUNC_NUM_M) >>
+ PF_FUNC_RID_FUNC_NUM_S;
+
+ status = ice_reset(hw, ICE_RESET_PFR);
+ if (status)
+ return status;
+
+ /* set these values to minimum allowed */
+ hw->itr_gran_200 = ICE_ITR_GRAN_MIN_200;
+ hw->itr_gran_100 = ICE_ITR_GRAN_MIN_100;
+ hw->itr_gran_50 = ICE_ITR_GRAN_MIN_50;
+ hw->itr_gran_25 = ICE_ITR_GRAN_MIN_25;
+
+ status = ice_init_all_ctrlq(hw);
+ if (status)
+ goto err_unroll_cqinit;
+
+ status = ice_clear_pf_cfg(hw);
+ if (status)
+ goto err_unroll_cqinit;
+
+ ice_clear_pxe_mode(hw);
+
+ status = ice_init_nvm(hw);
+ if (status)
+ goto err_unroll_cqinit;
+
+ status = ice_get_caps(hw);
+ if (status)
+ goto err_unroll_cqinit;
+
+ hw->port_info = devm_kzalloc(ice_hw_to_dev(hw),
+ sizeof(*hw->port_info), GFP_KERNEL);
+ if (!hw->port_info) {
+ status = ICE_ERR_NO_MEMORY;
+ goto err_unroll_cqinit;
+ }
+
+ /* set the back pointer to hw */
+ hw->port_info->hw = hw;
+
+ /* Initialize port_info struct with switch configuration data */
+ status = ice_get_initial_sw_cfg(hw);
+ if (status)
+ goto err_unroll_alloc;
+
+ hw->evb_veb = true;
+
+ /* Query the allocated resources for tx scheduler */
+ status = ice_sched_query_res_alloc(hw);
+ if (status) {
+ ice_debug(hw, ICE_DBG_SCHED,
+ "Failed to get scheduler allocated resources\n");
+ goto err_unroll_alloc;
+ }
+
+ /* Initialize port_info struct with scheduler data */
+ status = ice_sched_init_port(hw->port_info);
+ if (status)
+ goto err_unroll_sched;
+
+ pcaps = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*pcaps), GFP_KERNEL);
+ if (!pcaps) {
+ status = ICE_ERR_NO_MEMORY;
+ goto err_unroll_sched;
+ }
+
+ /* Initialize port_info struct with PHY capabilities */
+ status = ice_aq_get_phy_caps(hw->port_info, false,
+ ICE_AQC_REPORT_TOPO_CAP, pcaps, NULL);
+ devm_kfree(ice_hw_to_dev(hw), pcaps);
+ if (status)
+ goto err_unroll_sched;
+
+ /* Initialize port_info struct with link information */
+ status = ice_aq_get_link_info(hw->port_info, false, NULL, NULL);
+ if (status)
+ goto err_unroll_sched;
+
+ status = ice_init_fltr_mgmt_struct(hw);
+ if (status)
+ goto err_unroll_sched;
+
+ /* Get port MAC information */
+ mac_buf_len = sizeof(struct ice_aqc_manage_mac_read_resp);
+ mac_buf = devm_kzalloc(ice_hw_to_dev(hw), mac_buf_len, GFP_KERNEL);
+
+ if (!mac_buf)
+ goto err_unroll_fltr_mgmt_struct;
+
+ status = ice_aq_manage_mac_read(hw, mac_buf, mac_buf_len, NULL);
+ devm_kfree(ice_hw_to_dev(hw), mac_buf);
+
+ if (status)
+ goto err_unroll_fltr_mgmt_struct;
+
+ ice_init_flex_parser(hw);
+
+ return 0;
+
+err_unroll_fltr_mgmt_struct:
+ ice_cleanup_fltr_mgmt_struct(hw);
+err_unroll_sched:
+ ice_sched_cleanup_all(hw);
+err_unroll_alloc:
+ devm_kfree(ice_hw_to_dev(hw), hw->port_info);
+err_unroll_cqinit:
+ ice_shutdown_all_ctrlq(hw);
+ return status;
+}
+
+/**
+ * ice_deinit_hw - unroll initialization operations done by ice_init_hw
+ * @hw: pointer to the hardware structure
+ */
+void ice_deinit_hw(struct ice_hw *hw)
+{
+ ice_sched_cleanup_all(hw);
+ ice_shutdown_all_ctrlq(hw);
+
+ if (hw->port_info) {
+ devm_kfree(ice_hw_to_dev(hw), hw->port_info);
+ hw->port_info = NULL;
+ }
+
+ ice_cleanup_fltr_mgmt_struct(hw);
+}
+
+/**
+ * ice_check_reset - Check to see if a global reset is complete
+ * @hw: pointer to the hardware structure
+ */
+enum ice_status ice_check_reset(struct ice_hw *hw)
+{
+ u32 cnt, reg = 0, grst_delay;
+
+ /* Poll for Device Active state in case a recent CORER, GLOBR,
+ * or EMPR has occurred. The grst delay value is in 100ms units.
+ * Add 1sec for outstanding AQ commands that can take a long time.
+ */
+ grst_delay = ((rd32(hw, GLGEN_RSTCTL) & GLGEN_RSTCTL_GRSTDEL_M) >>
+ GLGEN_RSTCTL_GRSTDEL_S) + 10;
+
+ for (cnt = 0; cnt < grst_delay; cnt++) {
+ mdelay(100);
+ reg = rd32(hw, GLGEN_RSTAT);
+ if (!(reg & GLGEN_RSTAT_DEVSTATE_M))
+ break;
+ }
+
+ if (cnt == grst_delay) {
+ ice_debug(hw, ICE_DBG_INIT,
+ "Global reset polling failed to complete.\n");
+ return ICE_ERR_RESET_FAILED;
+ }
+
+#define ICE_RESET_DONE_MASK (GLNVM_ULD_CORER_DONE_M | \
+ GLNVM_ULD_GLOBR_DONE_M)
+
+ /* Device is Active; check Global Reset processes are done */
+ for (cnt = 0; cnt < ICE_PF_RESET_WAIT_COUNT; cnt++) {
+ reg = rd32(hw, GLNVM_ULD) & ICE_RESET_DONE_MASK;
+ if (reg == ICE_RESET_DONE_MASK) {
+ ice_debug(hw, ICE_DBG_INIT,
+ "Global reset processes done. %d\n", cnt);
+ break;
+ }
+ mdelay(10);
+ }
+
+ if (cnt == ICE_PF_RESET_WAIT_COUNT) {
+ ice_debug(hw, ICE_DBG_INIT,
+ "Wait for Reset Done timed out. GLNVM_ULD = 0x%x\n",
+ reg);
+ return ICE_ERR_RESET_FAILED;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_pf_reset - Reset the PF
+ * @hw: pointer to the hardware structure
+ *
+ * If a global reset has been triggered, this function checks
+ * for its completion and then issues the PF reset
+ */
+static enum ice_status ice_pf_reset(struct ice_hw *hw)
+{
+ u32 cnt, reg;
+
+ /* If at function entry a global reset was already in progress, i.e.
+ * state is not 'device active' or any of the reset done bits are not
+ * set in GLNVM_ULD, there is no need for a PF Reset; poll until the
+ * global reset is done.
+ */
+ if ((rd32(hw, GLGEN_RSTAT) & GLGEN_RSTAT_DEVSTATE_M) ||
+ (rd32(hw, GLNVM_ULD) & ICE_RESET_DONE_MASK) ^ ICE_RESET_DONE_MASK) {
+ /* poll on global reset currently in progress until done */
+ if (ice_check_reset(hw))
+ return ICE_ERR_RESET_FAILED;
+
+ return 0;
+ }
+
+ /* Reset the PF */
+ reg = rd32(hw, PFGEN_CTRL);
+
+ wr32(hw, PFGEN_CTRL, (reg | PFGEN_CTRL_PFSWR_M));
+
+ for (cnt = 0; cnt < ICE_PF_RESET_WAIT_COUNT; cnt++) {
+ reg = rd32(hw, PFGEN_CTRL);
+ if (!(reg & PFGEN_CTRL_PFSWR_M))
+ break;
+
+ mdelay(1);
+ }
+
+ if (cnt == ICE_PF_RESET_WAIT_COUNT) {
+ ice_debug(hw, ICE_DBG_INIT,
+ "PF reset polling failed to complete.\n");
+ return ICE_ERR_RESET_FAILED;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_reset - Perform different types of reset
+ * @hw: pointer to the hardware structure
+ * @req: reset request
+ *
+ * This function triggers a reset as specified by the req parameter.
+ *
+ * Note:
+ * If anything other than a PF reset is triggered, PXE mode is restored.
+ * This has to be cleared using ice_clear_pxe_mode again, once the AQ
+ * interface has been restored in the rebuild flow.
+ */
+enum ice_status ice_reset(struct ice_hw *hw, enum ice_reset_req req)
+{
+ u32 val = 0;
+
+ switch (req) {
+ case ICE_RESET_PFR:
+ return ice_pf_reset(hw);
+ case ICE_RESET_CORER:
+ ice_debug(hw, ICE_DBG_INIT, "CoreR requested\n");
+ val = GLGEN_RTRIG_CORER_M;
+ break;
+ case ICE_RESET_GLOBR:
+ ice_debug(hw, ICE_DBG_INIT, "GlobalR requested\n");
+ val = GLGEN_RTRIG_GLOBR_M;
+ break;
+ }
+
+ val |= rd32(hw, GLGEN_RTRIG);
+ wr32(hw, GLGEN_RTRIG, val);
+ ice_flush(hw);
+
+ /* wait for the FW to be ready */
+ return ice_check_reset(hw);
+}
+
+/**
+ * ice_copy_rxq_ctx_to_hw
+ * @hw: pointer to the hardware structure
+ * @ice_rxq_ctx: pointer to the rxq context
+ * @rxq_index: the index of the rx queue
+ *
+ * Copies rxq context from dense structure to hw register space
+ */
+static enum ice_status
+ice_copy_rxq_ctx_to_hw(struct ice_hw *hw, u8 *ice_rxq_ctx, u32 rxq_index)
+{
+ u8 i;
+
+ if (!ice_rxq_ctx)
+ return ICE_ERR_BAD_PTR;
+
+ if (rxq_index > QRX_CTRL_MAX_INDEX)
+ return ICE_ERR_PARAM;
+
+ /* Copy each dword separately to hw */
+ for (i = 0; i < ICE_RXQ_CTX_SIZE_DWORDS; i++) {
+ wr32(hw, QRX_CONTEXT(i, rxq_index),
+ *((u32 *)(ice_rxq_ctx + (i * sizeof(u32)))));
+
+ ice_debug(hw, ICE_DBG_QCTX, "qrxdata[%d]: %08X\n", i,
+ *((u32 *)(ice_rxq_ctx + (i * sizeof(u32)))));
+ }
+
+ return 0;
+}
+
+/* LAN Rx Queue Context */
+static const struct ice_ctx_ele ice_rlan_ctx_info[] = {
+ /* Field Width LSB */
+ ICE_CTX_STORE(ice_rlan_ctx, head, 13, 0),
+ ICE_CTX_STORE(ice_rlan_ctx, cpuid, 8, 13),
+ ICE_CTX_STORE(ice_rlan_ctx, base, 57, 32),
+ ICE_CTX_STORE(ice_rlan_ctx, qlen, 13, 89),
+ ICE_CTX_STORE(ice_rlan_ctx, dbuf, 7, 102),
+ ICE_CTX_STORE(ice_rlan_ctx, hbuf, 5, 109),
+ ICE_CTX_STORE(ice_rlan_ctx, dtype, 2, 114),
+ ICE_CTX_STORE(ice_rlan_ctx, dsize, 1, 116),
+ ICE_CTX_STORE(ice_rlan_ctx, crcstrip, 1, 117),
+ ICE_CTX_STORE(ice_rlan_ctx, l2tsel, 1, 119),
+ ICE_CTX_STORE(ice_rlan_ctx, hsplit_0, 4, 120),
+ ICE_CTX_STORE(ice_rlan_ctx, hsplit_1, 2, 124),
+ ICE_CTX_STORE(ice_rlan_ctx, showiv, 1, 127),
+ ICE_CTX_STORE(ice_rlan_ctx, rxmax, 14, 174),
+ ICE_CTX_STORE(ice_rlan_ctx, tphrdesc_ena, 1, 193),
+ ICE_CTX_STORE(ice_rlan_ctx, tphwdesc_ena, 1, 194),
+ ICE_CTX_STORE(ice_rlan_ctx, tphdata_ena, 1, 195),
+ ICE_CTX_STORE(ice_rlan_ctx, tphhead_ena, 1, 196),
+ ICE_CTX_STORE(ice_rlan_ctx, lrxqthresh, 3, 198),
+ { 0 }
+};
+
+/**
+ * ice_write_rxq_ctx
+ * @hw: pointer to the hardware structure
+ * @rlan_ctx: pointer to the rxq context
+ * @rxq_index: the index of the rx queue
+ *
+ * Converts rxq context from sparse to dense structure and then writes
+ * it to hw register space
+ */
+enum ice_status
+ice_write_rxq_ctx(struct ice_hw *hw, struct ice_rlan_ctx *rlan_ctx,
+ u32 rxq_index)
+{
+ u8 ctx_buf[ICE_RXQ_CTX_SZ] = { 0 };
+
+ ice_set_ctx((u8 *)rlan_ctx, ctx_buf, ice_rlan_ctx_info);
+ return ice_copy_rxq_ctx_to_hw(hw, ctx_buf, rxq_index);
+}
+
+/* LAN Tx Queue Context */
+const struct ice_ctx_ele ice_tlan_ctx_info[] = {
+ /* Field Width LSB */
+ ICE_CTX_STORE(ice_tlan_ctx, base, 57, 0),
+ ICE_CTX_STORE(ice_tlan_ctx, port_num, 3, 57),
+ ICE_CTX_STORE(ice_tlan_ctx, cgd_num, 5, 60),
+ ICE_CTX_STORE(ice_tlan_ctx, pf_num, 3, 65),
+ ICE_CTX_STORE(ice_tlan_ctx, vmvf_num, 10, 68),
+ ICE_CTX_STORE(ice_tlan_ctx, vmvf_type, 2, 78),
+ ICE_CTX_STORE(ice_tlan_ctx, src_vsi, 10, 80),
+ ICE_CTX_STORE(ice_tlan_ctx, tsyn_ena, 1, 90),
+ ICE_CTX_STORE(ice_tlan_ctx, alt_vlan, 1, 92),
+ ICE_CTX_STORE(ice_tlan_ctx, cpuid, 8, 93),
+ ICE_CTX_STORE(ice_tlan_ctx, wb_mode, 1, 101),
+ ICE_CTX_STORE(ice_tlan_ctx, tphrd_desc, 1, 102),
+ ICE_CTX_STORE(ice_tlan_ctx, tphrd, 1, 103),
+ ICE_CTX_STORE(ice_tlan_ctx, tphwr_desc, 1, 104),
+ ICE_CTX_STORE(ice_tlan_ctx, cmpq_id, 9, 105),
+ ICE_CTX_STORE(ice_tlan_ctx, qnum_in_func, 14, 114),
+ ICE_CTX_STORE(ice_tlan_ctx, itr_notification_mode, 1, 128),
+ ICE_CTX_STORE(ice_tlan_ctx, adjust_prof_id, 6, 129),
+ ICE_CTX_STORE(ice_tlan_ctx, qlen, 13, 135),
+ ICE_CTX_STORE(ice_tlan_ctx, quanta_prof_idx, 4, 148),
+ ICE_CTX_STORE(ice_tlan_ctx, tso_ena, 1, 152),
+ ICE_CTX_STORE(ice_tlan_ctx, tso_qnum, 11, 153),
+ ICE_CTX_STORE(ice_tlan_ctx, legacy_int, 1, 164),
+ ICE_CTX_STORE(ice_tlan_ctx, drop_ena, 1, 165),
+ ICE_CTX_STORE(ice_tlan_ctx, cache_prof_idx, 2, 166),
+ ICE_CTX_STORE(ice_tlan_ctx, pkt_shaper_prof_idx, 3, 168),
+ ICE_CTX_STORE(ice_tlan_ctx, int_q_state, 110, 171),
+ { 0 }
+};
+
+/**
+ * ice_debug_cq
+ * @hw: pointer to the hardware structure
+ * @mask: debug mask
+ * @desc: pointer to control queue descriptor
+ * @buf: pointer to command buffer
+ * @buf_len: max length of buf
+ *
+ * Dumps debug log about control command with descriptor contents.
+ */
+void ice_debug_cq(struct ice_hw *hw, u32 __maybe_unused mask, void *desc,
+ void *buf, u16 buf_len)
+{
+ struct ice_aq_desc *cq_desc = (struct ice_aq_desc *)desc;
+ u16 len;
+
+#ifndef CONFIG_DYNAMIC_DEBUG
+ if (!(mask & hw->debug_mask))
+ return;
+#endif
+
+ if (!desc)
+ return;
+
+ len = le16_to_cpu(cq_desc->datalen);
+
+ ice_debug(hw, mask,
+ "CQ CMD: opcode 0x%04X, flags 0x%04X, datalen 0x%04X, retval 0x%04X\n",
+ le16_to_cpu(cq_desc->opcode),
+ le16_to_cpu(cq_desc->flags),
+ le16_to_cpu(cq_desc->datalen), le16_to_cpu(cq_desc->retval));
+ ice_debug(hw, mask, "\tcookie (h,l) 0x%08X 0x%08X\n",
+ le32_to_cpu(cq_desc->cookie_high),
+ le32_to_cpu(cq_desc->cookie_low));
+ ice_debug(hw, mask, "\tparam (0,1) 0x%08X 0x%08X\n",
+ le32_to_cpu(cq_desc->params.generic.param0),
+ le32_to_cpu(cq_desc->params.generic.param1));
+ ice_debug(hw, mask, "\taddr (h,l) 0x%08X 0x%08X\n",
+ le32_to_cpu(cq_desc->params.generic.addr_high),
+ le32_to_cpu(cq_desc->params.generic.addr_low));
+ if (buf && cq_desc->datalen != 0) {
+ ice_debug(hw, mask, "Buffer:\n");
+ if (buf_len < len)
+ len = buf_len;
+
+ ice_debug_array(hw, mask, 16, 1, (u8 *)buf, len);
+ }
+}
+
+/* FW Admin Queue command wrappers */
+
+/**
+ * ice_aq_send_cmd - send FW Admin Queue command to FW Admin Queue
+ * @hw: pointer to the hw struct
+ * @desc: descriptor describing the command
+ * @buf: buffer to use for indirect commands (NULL for direct commands)
+ * @buf_size: size of buffer for indirect commands (0 for direct commands)
+ * @cd: pointer to command details structure
+ *
+ * Helper function to send FW Admin Queue commands to the FW Admin Queue.
+ */
+enum ice_status
+ice_aq_send_cmd(struct ice_hw *hw, struct ice_aq_desc *desc, void *buf,
+ u16 buf_size, struct ice_sq_cd *cd)
+{
+ return ice_sq_send_cmd(hw, &hw->adminq, desc, buf, buf_size, cd);
+}
+
+/**
+ * ice_aq_get_fw_ver
+ * @hw: pointer to the hw struct
+ * @cd: pointer to command details structure or NULL
+ *
+ * Get the firmware version (0x0001) from the admin queue commands
+ */
+enum ice_status ice_aq_get_fw_ver(struct ice_hw *hw, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_get_ver *resp;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ resp = &desc.params.get_ver;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_ver);
+
+ status = ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
+
+ if (!status) {
+ hw->fw_branch = resp->fw_branch;
+ hw->fw_maj_ver = resp->fw_major;
+ hw->fw_min_ver = resp->fw_minor;
+ hw->fw_patch = resp->fw_patch;
+ hw->fw_build = le32_to_cpu(resp->fw_build);
+ hw->api_branch = resp->api_branch;
+ hw->api_maj_ver = resp->api_major;
+ hw->api_min_ver = resp->api_minor;
+ hw->api_patch = resp->api_patch;
+ }
+
+ return status;
+}
+
+/**
+ * ice_aq_q_shutdown
+ * @hw: pointer to the hw struct
+ * @unloading: is the driver unloading itself
+ *
+ * Tell the Firmware that we're shutting down the AdminQ and whether
+ * or not the driver is unloading as well (0x0003).
+ */
+enum ice_status ice_aq_q_shutdown(struct ice_hw *hw, bool unloading)
+{
+ struct ice_aqc_q_shutdown *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.q_shutdown;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_q_shutdown);
+
+ if (unloading)
+ cmd->driver_unloading = cpu_to_le32(ICE_AQC_DRIVER_UNLOADING);
+
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, NULL);
+}
+
+/**
+ * ice_aq_req_res
+ * @hw: pointer to the hw struct
+ * @res: resource id
+ * @access: access type
+ * @sdp_number: resource number
+ * @timeout: the maximum time in ms that the driver may hold the resource
+ * @cd: pointer to command details structure or NULL
+ *
+ * requests common resource using the admin queue commands (0x0008)
+ */
+static enum ice_status
+ice_aq_req_res(struct ice_hw *hw, enum ice_aq_res_ids res,
+ enum ice_aq_res_access_type access, u8 sdp_number, u32 *timeout,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_req_res *cmd_resp;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd_resp = &desc.params.res_owner;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_req_res);
+
+ cmd_resp->res_id = cpu_to_le16(res);
+ cmd_resp->access_type = cpu_to_le16(access);
+ cmd_resp->res_number = cpu_to_le32(sdp_number);
+
+ status = ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
+ /* The completion specifies the maximum time in ms that the driver
+ * may hold the resource in the Timeout field.
+ * If the resource is held by someone else, the command completes with
+ * busy return value and the timeout field indicates the maximum time
+ * the current owner of the resource has to free it.
+ */
+ if (!status || hw->adminq.sq_last_status == ICE_AQ_RC_EBUSY)
+ *timeout = le32_to_cpu(cmd_resp->timeout);
+
+ return status;
+}
+
+/**
+ * ice_aq_release_res
+ * @hw: pointer to the hw struct
+ * @res: resource id
+ * @sdp_number: resource number
+ * @cd: pointer to command details structure or NULL
+ *
+ * release common resource using the admin queue commands (0x0009)
+ */
+static enum ice_status
+ice_aq_release_res(struct ice_hw *hw, enum ice_aq_res_ids res, u8 sdp_number,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_req_res *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.res_owner;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_release_res);
+
+ cmd->res_id = cpu_to_le16(res);
+ cmd->res_number = cpu_to_le32(sdp_number);
+
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
+}
+
+/**
+ * ice_acquire_res
+ * @hw: pointer to the HW structure
+ * @res: resource id
+ * @access: access type (read or write)
+ *
+ * This function will attempt to acquire the ownership of a resource.
+ */
+enum ice_status
+ice_acquire_res(struct ice_hw *hw, enum ice_aq_res_ids res,
+ enum ice_aq_res_access_type access)
+{
+#define ICE_RES_POLLING_DELAY_MS 10
+ u32 delay = ICE_RES_POLLING_DELAY_MS;
+ enum ice_status status;
+ u32 time_left = 0;
+ u32 timeout;
+
+ status = ice_aq_req_res(hw, res, access, 0, &time_left, NULL);
+
+ /* An admin queue return code of ICE_AQ_RC_EEXIST means that another
+ * driver has previously acquired the resource and performed any
+ * necessary updates; in this case the caller does not obtain the
+ * resource and has no further work to do.
+ */
+ if (hw->adminq.sq_last_status == ICE_AQ_RC_EEXIST) {
+ status = ICE_ERR_AQ_NO_WORK;
+ goto ice_acquire_res_exit;
+ }
+
+ if (status)
+ ice_debug(hw, ICE_DBG_RES,
+ "resource %d acquire type %d failed.\n", res, access);
+
+ /* If necessary, poll until the current lock owner timeouts */
+ timeout = time_left;
+ while (status && timeout && time_left) {
+ mdelay(delay);
+ timeout = (timeout > delay) ? timeout - delay : 0;
+ status = ice_aq_req_res(hw, res, access, 0, &time_left, NULL);
+
+ if (hw->adminq.sq_last_status == ICE_AQ_RC_EEXIST) {
+ /* lock free, but no work to do */
+ status = ICE_ERR_AQ_NO_WORK;
+ break;
+ }
+
+ if (!status)
+ /* lock acquired */
+ break;
+ }
+ if (status && status != ICE_ERR_AQ_NO_WORK)
+ ice_debug(hw, ICE_DBG_RES, "resource acquire timed out.\n");
+
+ice_acquire_res_exit:
+ if (status == ICE_ERR_AQ_NO_WORK) {
+ if (access == ICE_RES_WRITE)
+ ice_debug(hw, ICE_DBG_RES,
+ "resource indicates no work to do.\n");
+ else
+ ice_debug(hw, ICE_DBG_RES,
+ "Warning: ICE_ERR_AQ_NO_WORK not expected\n");
+ }
+ return status;
+}
+
+/**
+ * ice_release_res
+ * @hw: pointer to the HW structure
+ * @res: resource id
+ *
+ * This function will release a resource using the proper Admin Command.
+ */
+void ice_release_res(struct ice_hw *hw, enum ice_aq_res_ids res)
+{
+ enum ice_status status;
+ u32 total_delay = 0;
+
+ status = ice_aq_release_res(hw, res, 0, NULL);
+
+ /* there are some rare cases when trying to release the resource
+ * results in an admin Q timeout, so handle them correctly
+ */
+ while ((status == ICE_ERR_AQ_TIMEOUT) &&
+ (total_delay < hw->adminq.sq_cmd_timeout)) {
+ mdelay(1);
+ status = ice_aq_release_res(hw, res, 0, NULL);
+ total_delay++;
+ }
+}
+
+/**
+ * ice_parse_caps - parse function/device capabilities
+ * @hw: pointer to the hw struct
+ * @buf: pointer to a buffer containing function/device capability records
+ * @cap_count: number of capability records in the list
+ * @opc: type of capabilities list to parse
+ *
+ * Helper function to parse function(0x000a)/device(0x000b) capabilities list.
+ */
+static void
+ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,
+ enum ice_adminq_opc opc)
+{
+ struct ice_aqc_list_caps_elem *cap_resp;
+ struct ice_hw_func_caps *func_p = NULL;
+ struct ice_hw_dev_caps *dev_p = NULL;
+ struct ice_hw_common_caps *caps;
+ u32 i;
+
+ if (!buf)
+ return;
+
+ cap_resp = (struct ice_aqc_list_caps_elem *)buf;
+
+ if (opc == ice_aqc_opc_list_dev_caps) {
+ dev_p = &hw->dev_caps;
+ caps = &dev_p->common_cap;
+ } else if (opc == ice_aqc_opc_list_func_caps) {
+ func_p = &hw->func_caps;
+ caps = &func_p->common_cap;
+ } else {
+ ice_debug(hw, ICE_DBG_INIT, "wrong opcode\n");
+ return;
+ }
+
+ for (i = 0; caps && i < cap_count; i++, cap_resp++) {
+ u32 logical_id = le32_to_cpu(cap_resp->logical_id);
+ u32 phys_id = le32_to_cpu(cap_resp->phys_id);
+ u32 number = le32_to_cpu(cap_resp->number);
+ u16 cap = le16_to_cpu(cap_resp->cap);
+
+ switch (cap) {
+ case ICE_AQC_CAPS_VSI:
+ if (dev_p) {
+ dev_p->num_vsi_allocd_to_host = number;
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Dev.VSI cnt = %d\n",
+ dev_p->num_vsi_allocd_to_host);
+ } else if (func_p) {
+ func_p->guaranteed_num_vsi = number;
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Func.VSI cnt = %d\n",
+ func_p->guaranteed_num_vsi);
+ }
+ break;
+ case ICE_AQC_CAPS_RSS:
+ caps->rss_table_size = number;
+ caps->rss_table_entry_width = logical_id;
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: RSS table size = %d\n",
+ caps->rss_table_size);
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: RSS table width = %d\n",
+ caps->rss_table_entry_width);
+ break;
+ case ICE_AQC_CAPS_RXQS:
+ caps->num_rxq = number;
+ caps->rxq_first_id = phys_id;
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Num Rx Qs = %d\n", caps->num_rxq);
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Rx first queue ID = %d\n",
+ caps->rxq_first_id);
+ break;
+ case ICE_AQC_CAPS_TXQS:
+ caps->num_txq = number;
+ caps->txq_first_id = phys_id;
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Num Tx Qs = %d\n", caps->num_txq);
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Tx first queue ID = %d\n",
+ caps->txq_first_id);
+ break;
+ case ICE_AQC_CAPS_MSIX:
+ caps->num_msix_vectors = number;
+ caps->msix_vector_first_id = phys_id;
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: MSIX vector count = %d\n",
+ caps->num_msix_vectors);
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: MSIX first vector index = %d\n",
+ caps->msix_vector_first_id);
+ break;
+ case ICE_AQC_CAPS_MAX_MTU:
+ caps->max_mtu = number;
+ if (dev_p)
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Dev.MaxMTU = %d\n",
+ caps->max_mtu);
+ else if (func_p)
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: func.MaxMTU = %d\n",
+ caps->max_mtu);
+ break;
+ default:
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Unknown capability[%d]: 0x%x\n", i,
+ cap);
+ break;
+ }
+ }
+}
+
+/**
+ * ice_aq_discover_caps - query function/device capabilities
+ * @hw: pointer to the hw struct
+ * @buf: a virtual buffer to hold the capabilities
+ * @buf_size: Size of the virtual buffer
+ * @data_size: Size of the returned data, or buf size needed if AQ err==ENOMEM
+ * @opc: capabilities type to discover - pass in the command opcode
+ * @cd: pointer to command details structure or NULL
+ *
+ * Get the function(0x000a)/device(0x000b) capabilities description from
+ * the firmware.
+ */
+static enum ice_status
+ice_aq_discover_caps(struct ice_hw *hw, void *buf, u16 buf_size, u16 *data_size,
+ enum ice_adminq_opc opc, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_list_caps *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.get_cap;
+
+ if (opc != ice_aqc_opc_list_func_caps &&
+ opc != ice_aqc_opc_list_dev_caps)
+ return ICE_ERR_PARAM;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, opc);
+
+ status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+ if (!status)
+ ice_parse_caps(hw, buf, le32_to_cpu(cmd->count), opc);
+ *data_size = le16_to_cpu(desc.datalen);
+
+ return status;
+}
+
+/**
+ * ice_get_caps - get info about the HW
+ * @hw: pointer to the hardware structure
+ */
+enum ice_status ice_get_caps(struct ice_hw *hw)
+{
+ enum ice_status status;
+ u16 data_size = 0;
+ u16 cbuf_len;
+ u8 retries;
+
+ /* The driver doesn't know how many capabilities the device will return
+ * so the buffer size required isn't known ahead of time. The driver
+ * starts with cbuf_len and if this turns out to be insufficient, the
+ * device returns ICE_AQ_RC_ENOMEM and also the buffer size it needs.
+ * The driver then allocates the buffer of this size and retries the
+ * operation. So it follows that the retry count is 2.
+ */
+#define ICE_GET_CAP_BUF_COUNT 40
+#define ICE_GET_CAP_RETRY_COUNT 2
+
+ cbuf_len = ICE_GET_CAP_BUF_COUNT *
+ sizeof(struct ice_aqc_list_caps_elem);
+
+ retries = ICE_GET_CAP_RETRY_COUNT;
+
+ do {
+ void *cbuf;
+
+ cbuf = devm_kzalloc(ice_hw_to_dev(hw), cbuf_len, GFP_KERNEL);
+ if (!cbuf)
+ return ICE_ERR_NO_MEMORY;
+
+ status = ice_aq_discover_caps(hw, cbuf, cbuf_len, &data_size,
+ ice_aqc_opc_list_func_caps, NULL);
+ devm_kfree(ice_hw_to_dev(hw), cbuf);
+
+ if (!status || hw->adminq.sq_last_status != ICE_AQ_RC_ENOMEM)
+ break;
+
+ /* If ENOMEM is returned, try again with bigger buffer */
+ cbuf_len = data_size;
+ } while (--retries);
+
+ return status;
+}
+
+/**
+ * ice_aq_manage_mac_write - manage MAC address write command
+ * @hw: pointer to the hw struct
+ * @mac_addr: MAC address to be written as LAA/LAA+WoL/Port address
+ * @flags: flags to control write behavior
+ * @cd: pointer to command details structure or NULL
+ *
+ * This function is used to write MAC address to the NVM (0x0108).
+ */
+enum ice_status
+ice_aq_manage_mac_write(struct ice_hw *hw, u8 *mac_addr, u8 flags,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_manage_mac_write *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.mac_write;
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_manage_mac_write);
+
+ cmd->flags = flags;
+
+ /* Prep values for flags, sah, sal */
+ cmd->sah = htons(*((u16 *)mac_addr));
+ cmd->sal = htonl(*((u32 *)(mac_addr + 2)));
+
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
+}
+
+/**
+ * ice_aq_clear_pxe_mode
+ * @hw: pointer to the hw struct
+ *
+ * Tell the firmware that the driver is taking over from PXE (0x0110).
+ */
+static enum ice_status ice_aq_clear_pxe_mode(struct ice_hw *hw)
+{
+ struct ice_aq_desc desc;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_clear_pxe_mode);
+ desc.params.clear_pxe.rx_cnt = ICE_AQC_CLEAR_PXE_RX_CNT;
+
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, NULL);
+}
+
+/**
+ * ice_clear_pxe_mode - clear pxe operations mode
+ * @hw: pointer to the hw struct
+ *
+ * Make sure all PXE mode settings are cleared, including things
+ * like descriptor fetch/write-back mode.
+ */
+void ice_clear_pxe_mode(struct ice_hw *hw)
+{
+ if (ice_check_sq_alive(hw, &hw->adminq))
+ ice_aq_clear_pxe_mode(hw);
+}
+
+/**
+ * ice_aq_set_phy_cfg
+ * @hw: pointer to the hw struct
+ * @lport: logical port number
+ * @cfg: structure with PHY configuration data to be set
+ * @cd: pointer to command details structure or NULL
+ *
+ * Set the various PHY configuration parameters supported on the Port.
+ * One or more of the Set PHY config parameters may be ignored in an MFP
+ * mode as the PF may not have the privilege to set some of the PHY Config
+ * parameters. This status will be indicated by the command response (0x0601).
+ */
+static enum ice_status
+ice_aq_set_phy_cfg(struct ice_hw *hw, u8 lport,
+ struct ice_aqc_set_phy_cfg_data *cfg, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_set_phy_cfg *cmd;
+ struct ice_aq_desc desc;
+
+ if (!cfg)
+ return ICE_ERR_PARAM;
+
+ cmd = &desc.params.set_phy;
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_set_phy_cfg);
+ cmd->lport_num = lport;
+
+ return ice_aq_send_cmd(hw, &desc, cfg, sizeof(*cfg), cd);
+}
+
+/**
+ * ice_update_link_info - update status of the HW network link
+ * @pi: port info structure of the interested logical port
+ */
+static enum ice_status
+ice_update_link_info(struct ice_port_info *pi)
+{
+ struct ice_aqc_get_phy_caps_data *pcaps;
+ struct ice_phy_info *phy_info;
+ enum ice_status status;
+ struct ice_hw *hw;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+
+ hw = pi->hw;
+
+ pcaps = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*pcaps), GFP_KERNEL);
+ if (!pcaps)
+ return ICE_ERR_NO_MEMORY;
+
+ phy_info = &pi->phy;
+ status = ice_aq_get_link_info(pi, true, NULL, NULL);
+ if (status)
+ goto out;
+
+ if (phy_info->link_info.link_info & ICE_AQ_MEDIA_AVAILABLE) {
+ status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_SW_CFG,
+ pcaps, NULL);
+ if (status)
+ goto out;
+
+ memcpy(phy_info->link_info.module_type, &pcaps->module_type,
+ sizeof(phy_info->link_info.module_type));
+ }
+out:
+ devm_kfree(ice_hw_to_dev(hw), pcaps);
+ return status;
+}
+
+/**
+ * ice_set_fc
+ * @pi: port information structure
+ * @aq_failures: pointer to status code, specific to ice_set_fc routine
+ * @atomic_restart: enable automatic link update
+ *
+ * Set the requested flow control mode.
+ */
+enum ice_status
+ice_set_fc(struct ice_port_info *pi, u8 *aq_failures, bool atomic_restart)
+{
+ struct ice_aqc_set_phy_cfg_data cfg = { 0 };
+ struct ice_aqc_get_phy_caps_data *pcaps;
+ enum ice_status status;
+ u8 pause_mask = 0x0;
+ struct ice_hw *hw;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+ hw = pi->hw;
+ *aq_failures = ICE_SET_FC_AQ_FAIL_NONE;
+
+ switch (pi->fc.req_mode) {
+ case ICE_FC_FULL:
+ pause_mask |= ICE_AQC_PHY_EN_TX_LINK_PAUSE;
+ pause_mask |= ICE_AQC_PHY_EN_RX_LINK_PAUSE;
+ break;
+ case ICE_FC_RX_PAUSE:
+ pause_mask |= ICE_AQC_PHY_EN_RX_LINK_PAUSE;
+ break;
+ case ICE_FC_TX_PAUSE:
+ pause_mask |= ICE_AQC_PHY_EN_TX_LINK_PAUSE;
+ break;
+ default:
+ break;
+ }
+
+ pcaps = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*pcaps), GFP_KERNEL);
+ if (!pcaps)
+ return ICE_ERR_NO_MEMORY;
+
+ /* Get the current phy config */
+ status = ice_aq_get_phy_caps(pi, false, ICE_AQC_REPORT_SW_CFG, pcaps,
+ NULL);
+ if (status) {
+ *aq_failures = ICE_SET_FC_AQ_FAIL_GET;
+ goto out;
+ }
+
+ /* clear the old pause settings */
+ cfg.caps = pcaps->caps & ~(ICE_AQC_PHY_EN_TX_LINK_PAUSE |
+ ICE_AQC_PHY_EN_RX_LINK_PAUSE);
+ /* set the new capabilities */
+ cfg.caps |= pause_mask;
+ /* If the capabilities have changed, then set the new config */
+ if (cfg.caps != pcaps->caps) {
+ int retry_count, retry_max = 10;
+
+ /* Auto restart link so settings take effect */
+ if (atomic_restart)
+ cfg.caps |= ICE_AQ_PHY_ENA_ATOMIC_LINK;
+ /* Copy over all the old settings */
+ cfg.phy_type_low = pcaps->phy_type_low;
+ cfg.low_power_ctrl = pcaps->low_power_ctrl;
+ cfg.eee_cap = pcaps->eee_cap;
+ cfg.eeer_value = pcaps->eeer_value;
+ cfg.link_fec_opt = pcaps->link_fec_options;
+
+ status = ice_aq_set_phy_cfg(hw, pi->lport, &cfg, NULL);
+ if (status) {
+ *aq_failures = ICE_SET_FC_AQ_FAIL_SET;
+ goto out;
+ }
+
+ /* Update the link info
+ * It sometimes takes a really long time for link to
+ * come back from the atomic reset. Thus, we wait a
+ * little bit.
+ */
+ for (retry_count = 0; retry_count < retry_max; retry_count++) {
+ status = ice_update_link_info(pi);
+
+ if (!status)
+ break;
+
+ mdelay(100);
+ }
+
+ if (status)
+ *aq_failures = ICE_SET_FC_AQ_FAIL_UPDATE;
+ }
+
+out:
+ devm_kfree(ice_hw_to_dev(hw), pcaps);
+ return status;
+}
+
+/**
+ * ice_get_link_status - get status of the HW network link
+ * @pi: port information structure
+ * @link_up: pointer to bool (true/false = linkup/linkdown)
+ *
+ * Variable link_up is true if link is up, false if link is down.
+ * The variable link_up is invalid if status is non zero. As a
+ * result of this call, link status reporting becomes enabled
+ */
+enum ice_status ice_get_link_status(struct ice_port_info *pi, bool *link_up)
+{
+ struct ice_phy_info *phy_info;
+ enum ice_status status = 0;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+
+ phy_info = &pi->phy;
+
+ if (phy_info->get_link_info) {
+ status = ice_update_link_info(pi);
+
+ if (status)
+ ice_debug(pi->hw, ICE_DBG_LINK,
+ "get link status error, status = %d\n",
+ status);
+ }
+
+ *link_up = phy_info->link_info.link_info & ICE_AQ_LINK_UP;
+
+ return status;
+}
+
+/**
+ * ice_aq_set_link_restart_an
+ * @pi: pointer to the port information structure
+ * @ena_link: if true: enable link, if false: disable link
+ * @cd: pointer to command details structure or NULL
+ *
+ * Sets up the link and restarts the Auto-Negotiation over the link.
+ */
+enum ice_status
+ice_aq_set_link_restart_an(struct ice_port_info *pi, bool ena_link,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_restart_an *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.restart_an;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_restart_an);
+
+ cmd->cmd_flags = ICE_AQC_RESTART_AN_LINK_RESTART;
+ cmd->lport_num = pi->lport;
+ if (ena_link)
+ cmd->cmd_flags |= ICE_AQC_RESTART_AN_LINK_ENABLE;
+ else
+ cmd->cmd_flags &= ~ICE_AQC_RESTART_AN_LINK_ENABLE;
+
+ return ice_aq_send_cmd(pi->hw, &desc, NULL, 0, cd);
+}
+
+/**
+ * ice_aq_set_event_mask
+ * @hw: pointer to the hw struct
+ * @port_num: port number of the physical function
+ * @mask: event mask to be set
+ * @cd: pointer to command details structure or NULL
+ *
+ * Set event mask (0x0613)
+ */
+enum ice_status
+ice_aq_set_event_mask(struct ice_hw *hw, u8 port_num, u16 mask,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_set_event_mask *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.set_event_mask;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_set_event_mask);
+
+ cmd->lport_num = port_num;
+
+ cmd->event_mask = cpu_to_le16(mask);
+
+ return ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
+}
+
+/**
+ * __ice_aq_get_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: VSI FW index
+ * @lut_type: LUT table type
+ * @lut: pointer to the LUT buffer provided by the caller
+ * @lut_size: size of the LUT buffer
+ * @glob_lut_idx: global LUT index
+ * @set: set true to set the table, false to get the table
+ *
+ * Internal function to get (0x0B05) or set (0x0B03) RSS look up table
+ */
+static enum ice_status
+__ice_aq_get_set_rss_lut(struct ice_hw *hw, u16 vsi_id, u8 lut_type, u8 *lut,
+ u16 lut_size, u8 glob_lut_idx, bool set)
+{
+ struct ice_aqc_get_set_rss_lut *cmd_resp;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+ u16 flags = 0;
+
+ cmd_resp = &desc.params.get_set_rss_lut;
+
+ if (set) {
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_set_rss_lut);
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+ } else {
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_rss_lut);
+ }
+
+ cmd_resp->vsi_id = cpu_to_le16(((vsi_id <<
+ ICE_AQC_GSET_RSS_LUT_VSI_ID_S) &
+ ICE_AQC_GSET_RSS_LUT_VSI_ID_M) |
+ ICE_AQC_GSET_RSS_LUT_VSI_VALID);
+
+ switch (lut_type) {
+ case ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_VSI:
+ case ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_PF:
+ case ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_GLOBAL:
+ flags |= ((lut_type << ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_S) &
+ ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_M);
+ break;
+ default:
+ status = ICE_ERR_PARAM;
+ goto ice_aq_get_set_rss_lut_exit;
+ }
+
+ if (lut_type == ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_GLOBAL) {
+ flags |= ((glob_lut_idx << ICE_AQC_GSET_RSS_LUT_GLOBAL_IDX_S) &
+ ICE_AQC_GSET_RSS_LUT_GLOBAL_IDX_M);
+
+ if (!set)
+ goto ice_aq_get_set_rss_lut_send;
+ } else if (lut_type == ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_PF) {
+ if (!set)
+ goto ice_aq_get_set_rss_lut_send;
+ } else {
+ goto ice_aq_get_set_rss_lut_send;
+ }
+
+ /* LUT size is only valid for Global and PF table types */
+ if (lut_size == ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_128) {
+ flags |= (ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_128_FLAG <<
+ ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_S) &
+ ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_M;
+ } else if (lut_size == ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_512) {
+ flags |= (ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_512_FLAG <<
+ ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_S) &
+ ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_M;
+ } else if ((lut_size == ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_2K) &&
+ (lut_type == ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_PF)) {
+ flags |= (ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_2K_FLAG <<
+ ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_S) &
+ ICE_AQC_GSET_RSS_LUT_TABLE_SIZE_M;
+ } else {
+ status = ICE_ERR_PARAM;
+ goto ice_aq_get_set_rss_lut_exit;
+ }
+
+ice_aq_get_set_rss_lut_send:
+ cmd_resp->flags = cpu_to_le16(flags);
+ status = ice_aq_send_cmd(hw, &desc, lut, lut_size, NULL);
+
+ice_aq_get_set_rss_lut_exit:
+ return status;
+}
+
+/**
+ * ice_aq_get_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: VSI FW index
+ * @lut_type: LUT table type
+ * @lut: pointer to the LUT buffer provided by the caller
+ * @lut_size: size of the LUT buffer
+ *
+ * get the RSS lookup table, PF or VSI type
+ */
+enum ice_status
+ice_aq_get_rss_lut(struct ice_hw *hw, u16 vsi_id, u8 lut_type, u8 *lut,
+ u16 lut_size)
+{
+ return __ice_aq_get_set_rss_lut(hw, vsi_id, lut_type, lut, lut_size, 0,
+ false);
+}
+
+/**
+ * ice_aq_set_rss_lut
+ * @hw: pointer to the hardware structure
+ * @vsi_id: VSI FW index
+ * @lut_type: LUT table type
+ * @lut: pointer to the LUT buffer provided by the caller
+ * @lut_size: size of the LUT buffer
+ *
+ * set the RSS lookup table, PF or VSI type
+ */
+enum ice_status
+ice_aq_set_rss_lut(struct ice_hw *hw, u16 vsi_id, u8 lut_type, u8 *lut,
+ u16 lut_size)
+{
+ return __ice_aq_get_set_rss_lut(hw, vsi_id, lut_type, lut, lut_size, 0,
+ true);
+}
+
+/**
+ * __ice_aq_get_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: VSI FW index
+ * @key: pointer to key info struct
+ * @set: set true to set the key, false to get the key
+ *
+ * get (0x0B04) or set (0x0B02) the RSS key per VSI
+ */
+static enum
+ice_status __ice_aq_get_set_rss_key(struct ice_hw *hw, u16 vsi_id,
+ struct ice_aqc_get_set_rss_keys *key,
+ bool set)
+{
+ struct ice_aqc_get_set_rss_key *cmd_resp;
+ u16 key_size = sizeof(*key);
+ struct ice_aq_desc desc;
+
+ cmd_resp = &desc.params.get_set_rss_key;
+
+ if (set) {
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_set_rss_key);
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+ } else {
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_rss_key);
+ }
+
+ cmd_resp->vsi_id = cpu_to_le16(((vsi_id <<
+ ICE_AQC_GSET_RSS_KEY_VSI_ID_S) &
+ ICE_AQC_GSET_RSS_KEY_VSI_ID_M) |
+ ICE_AQC_GSET_RSS_KEY_VSI_VALID);
+
+ return ice_aq_send_cmd(hw, &desc, key, key_size, NULL);
+}
+
+/**
+ * ice_aq_get_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: VSI FW index
+ * @key: pointer to key info struct
+ *
+ * get the RSS key per VSI
+ */
+enum ice_status
+ice_aq_get_rss_key(struct ice_hw *hw, u16 vsi_id,
+ struct ice_aqc_get_set_rss_keys *key)
+{
+ return __ice_aq_get_set_rss_key(hw, vsi_id, key, false);
+}
+
+/**
+ * ice_aq_set_rss_key
+ * @hw: pointer to the hw struct
+ * @vsi_id: VSI FW index
+ * @keys: pointer to key info struct
+ *
+ * set the RSS key per VSI
+ */
+enum ice_status
+ice_aq_set_rss_key(struct ice_hw *hw, u16 vsi_id,
+ struct ice_aqc_get_set_rss_keys *keys)
+{
+ return __ice_aq_get_set_rss_key(hw, vsi_id, keys, true);
+}
+
+/**
+ * ice_aq_add_lan_txq
+ * @hw: pointer to the hardware structure
+ * @num_qgrps: Number of added queue groups
+ * @qg_list: list of queue groups to be added
+ * @buf_size: size of buffer for indirect command
+ * @cd: pointer to command details structure or NULL
+ *
+ * Add Tx LAN queue (0x0C30)
+ *
+ * NOTE:
+ * Prior to calling add Tx LAN queue:
+ * Initialize the following as part of the Tx queue context:
+ * Completion queue ID if the queue uses Completion queue, Quanta profile,
+ * Cache profile and Packet shaper profile.
+ *
+ * After add Tx LAN queue AQ command is completed:
+ * Interrupts should be associated with specific queues,
+ * Association of Tx queue to Doorbell queue is not part of Add LAN Tx queue
+ * flow.
+ */
+static enum ice_status
+ice_aq_add_lan_txq(struct ice_hw *hw, u8 num_qgrps,
+ struct ice_aqc_add_tx_qgrp *qg_list, u16 buf_size,
+ struct ice_sq_cd *cd)
+{
+ u16 i, sum_header_size, sum_q_size = 0;
+ struct ice_aqc_add_tx_qgrp *list;
+ struct ice_aqc_add_txqs *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.add_txqs;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_add_txqs);
+
+ if (!qg_list)
+ return ICE_ERR_PARAM;
+
+ if (num_qgrps > ICE_LAN_TXQ_MAX_QGRPS)
+ return ICE_ERR_PARAM;
+
+ sum_header_size = num_qgrps *
+ (sizeof(*qg_list) - sizeof(*qg_list->txqs));
+
+ list = qg_list;
+ for (i = 0; i < num_qgrps; i++) {
+ struct ice_aqc_add_txqs_perq *q = list->txqs;
+
+ sum_q_size += list->num_txqs * sizeof(*q);
+ list = (struct ice_aqc_add_tx_qgrp *)(q + list->num_txqs);
+ }
+
+ if (buf_size != (sum_header_size + sum_q_size))
+ return ICE_ERR_PARAM;
+
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+
+ cmd->num_qgrps = num_qgrps;
+
+ return ice_aq_send_cmd(hw, &desc, qg_list, buf_size, cd);
+}
+
+/**
+ * ice_aq_dis_lan_txq
+ * @hw: pointer to the hardware structure
+ * @num_qgrps: number of groups in the list
+ * @qg_list: the list of groups to disable
+ * @buf_size: the total size of the qg_list buffer in bytes
+ * @cd: pointer to command details structure or NULL
+ *
+ * Disable LAN Tx queue (0x0C31)
+ */
+static enum ice_status
+ice_aq_dis_lan_txq(struct ice_hw *hw, u8 num_qgrps,
+ struct ice_aqc_dis_txq_item *qg_list, u16 buf_size,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_dis_txqs *cmd;
+ struct ice_aq_desc desc;
+ u16 i, sz = 0;
+
+ cmd = &desc.params.dis_txqs;
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_dis_txqs);
+
+ if (!qg_list)
+ return ICE_ERR_PARAM;
+
+ if (num_qgrps > ICE_LAN_TXQ_MAX_QGRPS)
+ return ICE_ERR_PARAM;
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+ cmd->num_entries = num_qgrps;
+
+ for (i = 0; i < num_qgrps; ++i) {
+ /* Calculate the size taken up by the queue IDs in this group */
+ sz += qg_list[i].num_qs * sizeof(qg_list[i].q_id);
+
+ /* Add the size of the group header */
+ sz += sizeof(qg_list[i]) - sizeof(qg_list[i].q_id);
+
+ /* If the num of queues is even, add 2 bytes of padding */
+ if ((qg_list[i].num_qs % 2) == 0)
+ sz += 2;
+ }
+
+ if (buf_size != sz)
+ return ICE_ERR_PARAM;
+
+ return ice_aq_send_cmd(hw, &desc, qg_list, buf_size, cd);
+}
+
+/* End of FW Admin Queue command wrappers */
+
+/**
+ * ice_write_byte - write a byte to a packed context structure
+ * @src_ctx: the context structure to read from
+ * @dest_ctx: the context to be written to
+ * @ce_info: a description of the struct to be filled
+ */
+static void ice_write_byte(u8 *src_ctx, u8 *dest_ctx,
+ const struct ice_ctx_ele *ce_info)
+{
+ u8 src_byte, dest_byte, mask;
+ u8 *from, *dest;
+ u16 shift_width;
+
+ /* copy from the next struct field */
+ from = src_ctx + ce_info->offset;
+
+ /* prepare the bits and mask */
+ shift_width = ce_info->lsb % 8;
+ mask = (u8)(BIT(ce_info->width) - 1);
+
+ src_byte = *from;
+ src_byte &= mask;
+
+ /* shift to correct alignment */
+ mask <<= shift_width;
+ src_byte <<= shift_width;
+
+ /* get the current bits from the target bit string */
+ dest = dest_ctx + (ce_info->lsb / 8);
+
+ memcpy(&dest_byte, dest, sizeof(dest_byte));
+
+ dest_byte &= ~mask; /* get the bits not changing */
+ dest_byte |= src_byte; /* add in the new bits */
+
+ /* put it all back */
+ memcpy(dest, &dest_byte, sizeof(dest_byte));
+}
+
+/**
+ * ice_write_word - write a word to a packed context structure
+ * @src_ctx: the context structure to read from
+ * @dest_ctx: the context to be written to
+ * @ce_info: a description of the struct to be filled
+ */
+static void ice_write_word(u8 *src_ctx, u8 *dest_ctx,
+ const struct ice_ctx_ele *ce_info)
+{
+ u16 src_word, mask;
+ __le16 dest_word;
+ u8 *from, *dest;
+ u16 shift_width;
+
+ /* copy from the next struct field */
+ from = src_ctx + ce_info->offset;
+
+ /* prepare the bits and mask */
+ shift_width = ce_info->lsb % 8;
+ mask = BIT(ce_info->width) - 1;
+
+ /* don't swizzle the bits until after the mask because the mask bits
+ * will be in a different bit position on big endian machines
+ */
+ src_word = *(u16 *)from;
+ src_word &= mask;
+
+ /* shift to correct alignment */
+ mask <<= shift_width;
+ src_word <<= shift_width;
+
+ /* get the current bits from the target bit string */
+ dest = dest_ctx + (ce_info->lsb / 8);
+
+ memcpy(&dest_word, dest, sizeof(dest_word));
+
+ dest_word &= ~(cpu_to_le16(mask)); /* get the bits not changing */
+ dest_word |= cpu_to_le16(src_word); /* add in the new bits */
+
+ /* put it all back */
+ memcpy(dest, &dest_word, sizeof(dest_word));
+}
+
+/**
+ * ice_write_dword - write a dword to a packed context structure
+ * @src_ctx: the context structure to read from
+ * @dest_ctx: the context to be written to
+ * @ce_info: a description of the struct to be filled
+ */
+static void ice_write_dword(u8 *src_ctx, u8 *dest_ctx,
+ const struct ice_ctx_ele *ce_info)
+{
+ u32 src_dword, mask;
+ __le32 dest_dword;
+ u8 *from, *dest;
+ u16 shift_width;
+
+ /* copy from the next struct field */
+ from = src_ctx + ce_info->offset;
+
+ /* prepare the bits and mask */
+ shift_width = ce_info->lsb % 8;
+
+ /* if the field width is exactly 32 on an x86 machine, then the shift
+ * operation will not work because the SHL instructions count is masked
+ * to 5 bits so the shift will do nothing
+ */
+ if (ce_info->width < 32)
+ mask = BIT(ce_info->width) - 1;
+ else
+ mask = (u32)~0;
+
+ /* don't swizzle the bits until after the mask because the mask bits
+ * will be in a different bit position on big endian machines
+ */
+ src_dword = *(u32 *)from;
+ src_dword &= mask;
+
+ /* shift to correct alignment */
+ mask <<= shift_width;
+ src_dword <<= shift_width;
+
+ /* get the current bits from the target bit string */
+ dest = dest_ctx + (ce_info->lsb / 8);
+
+ memcpy(&dest_dword, dest, sizeof(dest_dword));
+
+ dest_dword &= ~(cpu_to_le32(mask)); /* get the bits not changing */
+ dest_dword |= cpu_to_le32(src_dword); /* add in the new bits */
+
+ /* put it all back */
+ memcpy(dest, &dest_dword, sizeof(dest_dword));
+}
+
+/**
+ * ice_write_qword - write a qword to a packed context structure
+ * @src_ctx: the context structure to read from
+ * @dest_ctx: the context to be written to
+ * @ce_info: a description of the struct to be filled
+ */
+static void ice_write_qword(u8 *src_ctx, u8 *dest_ctx,
+ const struct ice_ctx_ele *ce_info)
+{
+ u64 src_qword, mask;
+ __le64 dest_qword;
+ u8 *from, *dest;
+ u16 shift_width;
+
+ /* copy from the next struct field */
+ from = src_ctx + ce_info->offset;
+
+ /* prepare the bits and mask */
+ shift_width = ce_info->lsb % 8;
+
+ /* if the field width is exactly 64 on an x86 machine, then the shift
+ * operation will not work because the SHL instructions count is masked
+ * to 6 bits so the shift will do nothing
+ */
+ if (ce_info->width < 64)
+ mask = BIT_ULL(ce_info->width) - 1;
+ else
+ mask = (u64)~0;
+
+ /* don't swizzle the bits until after the mask because the mask bits
+ * will be in a different bit position on big endian machines
+ */
+ src_qword = *(u64 *)from;
+ src_qword &= mask;
+
+ /* shift to correct alignment */
+ mask <<= shift_width;
+ src_qword <<= shift_width;
+
+ /* get the current bits from the target bit string */
+ dest = dest_ctx + (ce_info->lsb / 8);
+
+ memcpy(&dest_qword, dest, sizeof(dest_qword));
+
+ dest_qword &= ~(cpu_to_le64(mask)); /* get the bits not changing */
+ dest_qword |= cpu_to_le64(src_qword); /* add in the new bits */
+
+ /* put it all back */
+ memcpy(dest, &dest_qword, sizeof(dest_qword));
+}
+
+/**
+ * ice_set_ctx - set context bits in packed structure
+ * @src_ctx: pointer to a generic non-packed context structure
+ * @dest_ctx: pointer to memory for the packed structure
+ * @ce_info: a description of the structure to be transformed
+ */
+enum ice_status
+ice_set_ctx(u8 *src_ctx, u8 *dest_ctx, const struct ice_ctx_ele *ce_info)
+{
+ int f;
+
+ for (f = 0; ce_info[f].width; f++) {
+ /* We have to deal with each element of the FW response
+ * using the correct size so that we are correct regardless
+ * of the endianness of the machine.
+ */
+ switch (ce_info[f].size_of) {
+ case sizeof(u8):
+ ice_write_byte(src_ctx, dest_ctx, &ce_info[f]);
+ break;
+ case sizeof(u16):
+ ice_write_word(src_ctx, dest_ctx, &ce_info[f]);
+ break;
+ case sizeof(u32):
+ ice_write_dword(src_ctx, dest_ctx, &ce_info[f]);
+ break;
+ case sizeof(u64):
+ ice_write_qword(src_ctx, dest_ctx, &ce_info[f]);
+ break;
+ default:
+ return ICE_ERR_INVAL_SIZE;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ice_ena_vsi_txq
+ * @pi: port information structure
+ * @vsi_id: VSI id
+ * @tc: tc number
+ * @num_qgrps: Number of added queue groups
+ * @buf: list of queue groups to be added
+ * @buf_size: size of buffer for indirect command
+ * @cd: pointer to command details structure or NULL
+ *
+ * This function adds one lan q
+ */
+enum ice_status
+ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_id, u8 tc, u8 num_qgrps,
+ struct ice_aqc_add_tx_qgrp *buf, u16 buf_size,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_txsched_elem_data node = { 0 };
+ struct ice_sched_node *parent;
+ enum ice_status status;
+ struct ice_hw *hw;
+
+ if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
+ return ICE_ERR_CFG;
+
+ if (num_qgrps > 1 || buf->num_txqs > 1)
+ return ICE_ERR_MAX_LIMIT;
+
+ hw = pi->hw;
+
+ mutex_lock(&pi->sched_lock);
+
+ /* find a parent node */
+ parent = ice_sched_get_free_qparent(pi, vsi_id, tc,
+ ICE_SCHED_NODE_OWNER_LAN);
+ if (!parent) {
+ status = ICE_ERR_PARAM;
+ goto ena_txq_exit;
+ }
+ buf->parent_teid = parent->info.node_teid;
+ node.parent_teid = parent->info.node_teid;
+ /* Mark that the values in the "generic" section as valid. The default
+ * value in the "generic" section is zero. This means that :
+ * - Scheduling mode is Bytes Per Second (BPS), indicated by Bit 0.
+ * - 0 priority among siblings, indicated by Bit 1-3.
+ * - WFQ, indicated by Bit 4.
+ * - 0 Adjustment value is used in PSM credit update flow, indicated by
+ * Bit 5-6.
+ * - Bit 7 is reserved.
+ * Without setting the generic section as valid in valid_sections, the
+ * Admin Q command will fail with error code ICE_AQ_RC_EINVAL.
+ */
+ buf->txqs[0].info.valid_sections = ICE_AQC_ELEM_VALID_GENERIC;
+
+ /* add the lan q */
+ status = ice_aq_add_lan_txq(hw, num_qgrps, buf, buf_size, cd);
+ if (status)
+ goto ena_txq_exit;
+
+ node.node_teid = buf->txqs[0].q_teid;
+ node.data.elem_type = ICE_AQC_ELEM_TYPE_LEAF;
+
+ /* add a leaf node into schduler tree q layer */
+ status = ice_sched_add_node(pi, hw->num_tx_sched_layers - 1, &node);
+
+ena_txq_exit:
+ mutex_unlock(&pi->sched_lock);
+ return status;
+}
+
+/**
+ * ice_dis_vsi_txq
+ * @pi: port information structure
+ * @num_queues: number of queues
+ * @q_ids: pointer to the q_id array
+ * @q_teids: pointer to queue node teids
+ * @cd: pointer to command details structure or NULL
+ *
+ * This function removes queues and their corresponding nodes in SW DB
+ */
+enum ice_status
+ice_dis_vsi_txq(struct ice_port_info *pi, u8 num_queues, u16 *q_ids,
+ u32 *q_teids, struct ice_sq_cd *cd)
+{
+ enum ice_status status = ICE_ERR_DOES_NOT_EXIST;
+ struct ice_aqc_dis_txq_item qg_list;
+ u16 i;
+
+ if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
+ return ICE_ERR_CFG;
+
+ mutex_lock(&pi->sched_lock);
+
+ for (i = 0; i < num_queues; i++) {
+ struct ice_sched_node *node;
+
+ node = ice_sched_find_node_by_teid(pi->root, q_teids[i]);
+ if (!node)
+ continue;
+ qg_list.parent_teid = node->info.parent_teid;
+ qg_list.num_qs = 1;
+ qg_list.q_id[0] = cpu_to_le16(q_ids[i]);
+ status = ice_aq_dis_lan_txq(pi->hw, 1, &qg_list,
+ sizeof(qg_list), cd);
+
+ if (status)
+ break;
+ ice_free_sched_node(pi, node);
+ }
+ mutex_unlock(&pi->sched_lock);
+ return status;
+}
+
+/**
+ * ice_cfg_vsi_qs - configure the new/exisiting VSI queues
+ * @pi: port information structure
+ * @vsi_id: VSI Id
+ * @tc_bitmap: TC bitmap
+ * @maxqs: max queues array per TC
+ * @owner: lan or rdma
+ *
+ * This function adds/updates the VSI queues per TC.
+ */
+static enum ice_status
+ice_cfg_vsi_qs(struct ice_port_info *pi, u16 vsi_id, u8 tc_bitmap,
+ u16 *maxqs, u8 owner)
+{
+ enum ice_status status = 0;
+ u8 i;
+
+ if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
+ return ICE_ERR_CFG;
+
+ mutex_lock(&pi->sched_lock);
+
+ for (i = 0; i < ICE_MAX_TRAFFIC_CLASS; i++) {
+ /* configuration is possible only if TC node is present */
+ if (!ice_sched_get_tc_node(pi, i))
+ continue;
+
+ status = ice_sched_cfg_vsi(pi, vsi_id, i, maxqs[i], owner,
+ ice_is_tc_ena(tc_bitmap, i));
+ if (status)
+ break;
+ }
+
+ mutex_unlock(&pi->sched_lock);
+ return status;
+}
+
+/**
+ * ice_cfg_vsi_lan - configure VSI lan queues
+ * @pi: port information structure
+ * @vsi_id: VSI Id
+ * @tc_bitmap: TC bitmap
+ * @max_lanqs: max lan queues array per TC
+ *
+ * This function adds/updates the VSI lan queues per TC.
+ */
+enum ice_status
+ice_cfg_vsi_lan(struct ice_port_info *pi, u16 vsi_id, u8 tc_bitmap,
+ u16 *max_lanqs)
+{
+ return ice_cfg_vsi_qs(pi, vsi_id, tc_bitmap, max_lanqs,
+ ICE_SCHED_NODE_OWNER_LAN);
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_COMMON_H_
+#define _ICE_COMMON_H_
+
+#include "ice.h"
+#include "ice_type.h"
+#include "ice_switch.h"
+
+void ice_debug_cq(struct ice_hw *hw, u32 mask, void *desc, void *buf,
+ u16 buf_len);
+enum ice_status ice_init_hw(struct ice_hw *hw);
+void ice_deinit_hw(struct ice_hw *hw);
+enum ice_status ice_check_reset(struct ice_hw *hw);
+enum ice_status ice_reset(struct ice_hw *hw, enum ice_reset_req req);
+enum ice_status ice_init_all_ctrlq(struct ice_hw *hw);
+void ice_shutdown_all_ctrlq(struct ice_hw *hw);
+enum ice_status
+ice_clean_rq_elem(struct ice_hw *hw, struct ice_ctl_q_info *cq,
+ struct ice_rq_event_info *e, u16 *pending);
+enum ice_status
+ice_get_link_status(struct ice_port_info *pi, bool *link_up);
+enum ice_status
+ice_acquire_res(struct ice_hw *hw, enum ice_aq_res_ids res,
+ enum ice_aq_res_access_type access);
+void ice_release_res(struct ice_hw *hw, enum ice_aq_res_ids res);
+enum ice_status ice_init_nvm(struct ice_hw *hw);
+enum ice_status
+ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
+ struct ice_aq_desc *desc, void *buf, u16 buf_size,
+ struct ice_sq_cd *cd);
+void ice_clear_pxe_mode(struct ice_hw *hw);
+enum ice_status ice_get_caps(struct ice_hw *hw);
+enum ice_status
+ice_write_rxq_ctx(struct ice_hw *hw, struct ice_rlan_ctx *rlan_ctx,
+ u32 rxq_index);
+
+enum ice_status
+ice_aq_get_rss_lut(struct ice_hw *hw, u16 vsi_id, u8 lut_type, u8 *lut,
+ u16 lut_size);
+enum ice_status
+ice_aq_set_rss_lut(struct ice_hw *hw, u16 vsi_id, u8 lut_type, u8 *lut,
+ u16 lut_size);
+enum ice_status
+ice_aq_get_rss_key(struct ice_hw *hw, u16 vsi_id,
+ struct ice_aqc_get_set_rss_keys *keys);
+enum ice_status
+ice_aq_set_rss_key(struct ice_hw *hw, u16 vsi_id,
+ struct ice_aqc_get_set_rss_keys *keys);
+bool ice_check_sq_alive(struct ice_hw *hw, struct ice_ctl_q_info *cq);
+enum ice_status ice_aq_q_shutdown(struct ice_hw *hw, bool unloading);
+void ice_fill_dflt_direct_cmd_desc(struct ice_aq_desc *desc, u16 opcode);
+extern const struct ice_ctx_ele ice_tlan_ctx_info[];
+enum ice_status
+ice_set_ctx(u8 *src_ctx, u8 *dest_ctx, const struct ice_ctx_ele *ce_info);
+enum ice_status
+ice_aq_send_cmd(struct ice_hw *hw, struct ice_aq_desc *desc,
+ void *buf, u16 buf_size, struct ice_sq_cd *cd);
+enum ice_status ice_aq_get_fw_ver(struct ice_hw *hw, struct ice_sq_cd *cd);
+enum ice_status
+ice_aq_manage_mac_write(struct ice_hw *hw, u8 *mac_addr, u8 flags,
+ struct ice_sq_cd *cd);
+enum ice_status ice_clear_pf_cfg(struct ice_hw *hw);
+enum ice_status
+ice_set_fc(struct ice_port_info *pi, u8 *aq_failures, bool atomic_restart);
+enum ice_status
+ice_aq_set_link_restart_an(struct ice_port_info *pi, bool ena_link,
+ struct ice_sq_cd *cd);
+enum ice_status
+ice_aq_get_link_info(struct ice_port_info *pi, bool ena_lse,
+ struct ice_link_status *link, struct ice_sq_cd *cd);
+enum ice_status
+ice_aq_set_event_mask(struct ice_hw *hw, u8 port_num, u16 mask,
+ struct ice_sq_cd *cd);
+enum ice_status
+ice_dis_vsi_txq(struct ice_port_info *pi, u8 num_queues, u16 *q_ids,
+ u32 *q_teids, struct ice_sq_cd *cmd_details);
+enum ice_status
+ice_cfg_vsi_lan(struct ice_port_info *pi, u16 vsi_id, u8 tc_bitmap,
+ u16 *max_lanqs);
+enum ice_status
+ice_ena_vsi_txq(struct ice_port_info *pi, u16 vsi_id, u8 tc, u8 num_qgrps,
+ struct ice_aqc_add_tx_qgrp *buf, u16 buf_size,
+ struct ice_sq_cd *cd);
+#endif /* _ICE_COMMON_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+#include "ice_common.h"
+
+/**
+ * ice_adminq_init_regs - Initialize AdminQ registers
+ * @hw: pointer to the hardware structure
+ *
+ * This assumes the alloc_sq and alloc_rq functions have already been called
+ */
+static void ice_adminq_init_regs(struct ice_hw *hw)
+{
+ struct ice_ctl_q_info *cq = &hw->adminq;
+
+ cq->sq.head = PF_FW_ATQH;
+ cq->sq.tail = PF_FW_ATQT;
+ cq->sq.len = PF_FW_ATQLEN;
+ cq->sq.bah = PF_FW_ATQBAH;
+ cq->sq.bal = PF_FW_ATQBAL;
+ cq->sq.len_mask = PF_FW_ATQLEN_ATQLEN_M;
+ cq->sq.len_ena_mask = PF_FW_ATQLEN_ATQENABLE_M;
+ cq->sq.head_mask = PF_FW_ATQH_ATQH_M;
+
+ cq->rq.head = PF_FW_ARQH;
+ cq->rq.tail = PF_FW_ARQT;
+ cq->rq.len = PF_FW_ARQLEN;
+ cq->rq.bah = PF_FW_ARQBAH;
+ cq->rq.bal = PF_FW_ARQBAL;
+ cq->rq.len_mask = PF_FW_ARQLEN_ARQLEN_M;
+ cq->rq.len_ena_mask = PF_FW_ARQLEN_ARQENABLE_M;
+ cq->rq.head_mask = PF_FW_ARQH_ARQH_M;
+}
+
+/**
+ * ice_check_sq_alive
+ * @hw: pointer to the hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Returns true if Queue is enabled else false.
+ */
+bool ice_check_sq_alive(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ /* check both queue-length and queue-enable fields */
+ if (cq->sq.len && cq->sq.len_mask && cq->sq.len_ena_mask)
+ return (rd32(hw, cq->sq.len) & (cq->sq.len_mask |
+ cq->sq.len_ena_mask)) ==
+ (cq->num_sq_entries | cq->sq.len_ena_mask);
+
+ return false;
+}
+
+/**
+ * ice_alloc_ctrlq_sq_ring - Allocate Control Transmit Queue (ATQ) rings
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ */
+static enum ice_status
+ice_alloc_ctrlq_sq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ size_t size = cq->num_sq_entries * sizeof(struct ice_aq_desc);
+
+ cq->sq.desc_buf.va = dmam_alloc_coherent(ice_hw_to_dev(hw), size,
+ &cq->sq.desc_buf.pa,
+ GFP_KERNEL | __GFP_ZERO);
+ if (!cq->sq.desc_buf.va)
+ return ICE_ERR_NO_MEMORY;
+ cq->sq.desc_buf.size = size;
+
+ cq->sq.cmd_buf = devm_kcalloc(ice_hw_to_dev(hw), cq->num_sq_entries,
+ sizeof(struct ice_sq_cd), GFP_KERNEL);
+ if (!cq->sq.cmd_buf) {
+ dmam_free_coherent(ice_hw_to_dev(hw), cq->sq.desc_buf.size,
+ cq->sq.desc_buf.va, cq->sq.desc_buf.pa);
+ cq->sq.desc_buf.va = NULL;
+ cq->sq.desc_buf.pa = 0;
+ cq->sq.desc_buf.size = 0;
+ return ICE_ERR_NO_MEMORY;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_alloc_ctrlq_rq_ring - Allocate Control Receive Queue (ARQ) rings
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ */
+static enum ice_status
+ice_alloc_ctrlq_rq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ size_t size = cq->num_rq_entries * sizeof(struct ice_aq_desc);
+
+ cq->rq.desc_buf.va = dmam_alloc_coherent(ice_hw_to_dev(hw), size,
+ &cq->rq.desc_buf.pa,
+ GFP_KERNEL | __GFP_ZERO);
+ if (!cq->rq.desc_buf.va)
+ return ICE_ERR_NO_MEMORY;
+ cq->rq.desc_buf.size = size;
+ return 0;
+}
+
+/**
+ * ice_free_ctrlq_sq_ring - Free Control Transmit Queue (ATQ) rings
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted send buffers have already been cleaned
+ * and de-allocated
+ */
+static void ice_free_ctrlq_sq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ dmam_free_coherent(ice_hw_to_dev(hw), cq->sq.desc_buf.size,
+ cq->sq.desc_buf.va, cq->sq.desc_buf.pa);
+ cq->sq.desc_buf.va = NULL;
+ cq->sq.desc_buf.pa = 0;
+ cq->sq.desc_buf.size = 0;
+}
+
+/**
+ * ice_free_ctrlq_rq_ring - Free Control Receive Queue (ARQ) rings
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * This assumes the posted receive buffers have already been cleaned
+ * and de-allocated
+ */
+static void ice_free_ctrlq_rq_ring(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ dmam_free_coherent(ice_hw_to_dev(hw), cq->rq.desc_buf.size,
+ cq->rq.desc_buf.va, cq->rq.desc_buf.pa);
+ cq->rq.desc_buf.va = NULL;
+ cq->rq.desc_buf.pa = 0;
+ cq->rq.desc_buf.size = 0;
+}
+
+/**
+ * ice_alloc_rq_bufs - Allocate pre-posted buffers for the ARQ
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ */
+static enum ice_status
+ice_alloc_rq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ int i;
+
+ /* We'll be allocating the buffer info memory first, then we can
+ * allocate the mapped buffers for the event processing
+ */
+ cq->rq.dma_head = devm_kcalloc(ice_hw_to_dev(hw), cq->num_rq_entries,
+ sizeof(cq->rq.desc_buf), GFP_KERNEL);
+ if (!cq->rq.dma_head)
+ return ICE_ERR_NO_MEMORY;
+ cq->rq.r.rq_bi = (struct ice_dma_mem *)cq->rq.dma_head;
+
+ /* allocate the mapped buffers */
+ for (i = 0; i < cq->num_rq_entries; i++) {
+ struct ice_aq_desc *desc;
+ struct ice_dma_mem *bi;
+
+ bi = &cq->rq.r.rq_bi[i];
+ bi->va = dmam_alloc_coherent(ice_hw_to_dev(hw),
+ cq->rq_buf_size, &bi->pa,
+ GFP_KERNEL | __GFP_ZERO);
+ if (!bi->va)
+ goto unwind_alloc_rq_bufs;
+ bi->size = cq->rq_buf_size;
+
+ /* now configure the descriptors for use */
+ desc = ICE_CTL_Q_DESC(cq->rq, i);
+
+ desc->flags = cpu_to_le16(ICE_AQ_FLAG_BUF);
+ if (cq->rq_buf_size > ICE_AQ_LG_BUF)
+ desc->flags |= cpu_to_le16(ICE_AQ_FLAG_LB);
+ desc->opcode = 0;
+ /* This is in accordance with Admin queue design, there is no
+ * register for buffer size configuration
+ */
+ desc->datalen = cpu_to_le16(bi->size);
+ desc->retval = 0;
+ desc->cookie_high = 0;
+ desc->cookie_low = 0;
+ desc->params.generic.addr_high =
+ cpu_to_le32(upper_32_bits(bi->pa));
+ desc->params.generic.addr_low =
+ cpu_to_le32(lower_32_bits(bi->pa));
+ desc->params.generic.param0 = 0;
+ desc->params.generic.param1 = 0;
+ }
+ return 0;
+
+unwind_alloc_rq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ dmam_free_coherent(ice_hw_to_dev(hw), cq->rq.r.rq_bi[i].size,
+ cq->rq.r.rq_bi[i].va, cq->rq.r.rq_bi[i].pa);
+ cq->rq.r.rq_bi[i].va = NULL;
+ cq->rq.r.rq_bi[i].pa = 0;
+ cq->rq.r.rq_bi[i].size = 0;
+ }
+ devm_kfree(ice_hw_to_dev(hw), cq->rq.dma_head);
+
+ return ICE_ERR_NO_MEMORY;
+}
+
+/**
+ * ice_alloc_sq_bufs - Allocate empty buffer structs for the ATQ
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ */
+static enum ice_status
+ice_alloc_sq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ int i;
+
+ /* No mapped memory needed yet, just the buffer info structures */
+ cq->sq.dma_head = devm_kcalloc(ice_hw_to_dev(hw), cq->num_sq_entries,
+ sizeof(cq->sq.desc_buf), GFP_KERNEL);
+ if (!cq->sq.dma_head)
+ return ICE_ERR_NO_MEMORY;
+ cq->sq.r.sq_bi = (struct ice_dma_mem *)cq->sq.dma_head;
+
+ /* allocate the mapped buffers */
+ for (i = 0; i < cq->num_sq_entries; i++) {
+ struct ice_dma_mem *bi;
+
+ bi = &cq->sq.r.sq_bi[i];
+ bi->va = dmam_alloc_coherent(ice_hw_to_dev(hw),
+ cq->sq_buf_size, &bi->pa,
+ GFP_KERNEL | __GFP_ZERO);
+ if (!bi->va)
+ goto unwind_alloc_sq_bufs;
+ bi->size = cq->sq_buf_size;
+ }
+ return 0;
+
+unwind_alloc_sq_bufs:
+ /* don't try to free the one that failed... */
+ i--;
+ for (; i >= 0; i--) {
+ dmam_free_coherent(ice_hw_to_dev(hw), cq->sq.r.sq_bi[i].size,
+ cq->sq.r.sq_bi[i].va, cq->sq.r.sq_bi[i].pa);
+ cq->sq.r.sq_bi[i].va = NULL;
+ cq->sq.r.sq_bi[i].pa = 0;
+ cq->sq.r.sq_bi[i].size = 0;
+ }
+ devm_kfree(ice_hw_to_dev(hw), cq->sq.dma_head);
+
+ return ICE_ERR_NO_MEMORY;
+}
+
+/**
+ * ice_free_rq_bufs - Free ARQ buffer info elements
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ */
+static void ice_free_rq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ int i;
+
+ /* free descriptors */
+ for (i = 0; i < cq->num_rq_entries; i++) {
+ dmam_free_coherent(ice_hw_to_dev(hw), cq->rq.r.rq_bi[i].size,
+ cq->rq.r.rq_bi[i].va, cq->rq.r.rq_bi[i].pa);
+ cq->rq.r.rq_bi[i].va = NULL;
+ cq->rq.r.rq_bi[i].pa = 0;
+ cq->rq.r.rq_bi[i].size = 0;
+ }
+
+ /* free the dma header */
+ devm_kfree(ice_hw_to_dev(hw), cq->rq.dma_head);
+}
+
+/**
+ * ice_free_sq_bufs - Free ATQ buffer info elements
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ */
+static void ice_free_sq_bufs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ int i;
+
+ /* only unmap if the address is non-NULL */
+ for (i = 0; i < cq->num_sq_entries; i++)
+ if (cq->sq.r.sq_bi[i].pa) {
+ dmam_free_coherent(ice_hw_to_dev(hw),
+ cq->sq.r.sq_bi[i].size,
+ cq->sq.r.sq_bi[i].va,
+ cq->sq.r.sq_bi[i].pa);
+ cq->sq.r.sq_bi[i].va = NULL;
+ cq->sq.r.sq_bi[i].pa = 0;
+ cq->sq.r.sq_bi[i].size = 0;
+ }
+
+ /* free the buffer info list */
+ devm_kfree(ice_hw_to_dev(hw), cq->sq.cmd_buf);
+
+ /* free the dma header */
+ devm_kfree(ice_hw_to_dev(hw), cq->sq.dma_head);
+}
+
+/**
+ * ice_cfg_sq_regs - configure Control ATQ registers
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * Configure base address and length registers for the transmit queue
+ */
+static enum ice_status
+ice_cfg_sq_regs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ u32 reg = 0;
+
+ /* Clear Head and Tail */
+ wr32(hw, cq->sq.head, 0);
+ wr32(hw, cq->sq.tail, 0);
+
+ /* set starting point */
+ wr32(hw, cq->sq.len, (cq->num_sq_entries | cq->sq.len_ena_mask));
+ wr32(hw, cq->sq.bal, lower_32_bits(cq->sq.desc_buf.pa));
+ wr32(hw, cq->sq.bah, upper_32_bits(cq->sq.desc_buf.pa));
+
+ /* Check one register to verify that config was applied */
+ reg = rd32(hw, cq->sq.bal);
+ if (reg != lower_32_bits(cq->sq.desc_buf.pa))
+ return ICE_ERR_AQ_ERROR;
+
+ return 0;
+}
+
+/**
+ * ice_cfg_rq_regs - configure Control ARQ register
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * Configure base address and length registers for the receive (event q)
+ */
+static enum ice_status
+ice_cfg_rq_regs(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ u32 reg = 0;
+
+ /* Clear Head and Tail */
+ wr32(hw, cq->rq.head, 0);
+ wr32(hw, cq->rq.tail, 0);
+
+ /* set starting point */
+ wr32(hw, cq->rq.len, (cq->num_rq_entries | cq->rq.len_ena_mask));
+ wr32(hw, cq->rq.bal, lower_32_bits(cq->rq.desc_buf.pa));
+ wr32(hw, cq->rq.bah, upper_32_bits(cq->rq.desc_buf.pa));
+
+ /* Update tail in the HW to post pre-allocated buffers */
+ wr32(hw, cq->rq.tail, (u32)(cq->num_rq_entries - 1));
+
+ /* Check one register to verify that config was applied */
+ reg = rd32(hw, cq->rq.bal);
+ if (reg != lower_32_bits(cq->rq.desc_buf.pa))
+ return ICE_ERR_AQ_ERROR;
+
+ return 0;
+}
+
+/**
+ * ice_init_sq - main initialization routine for Control ATQ
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * This is the main initialization routine for the Control Send Queue
+ * Prior to calling this function, drivers *MUST* set the following fields
+ * in the cq->structure:
+ * - cq->num_sq_entries
+ * - cq->sq_buf_size
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+static enum ice_status ice_init_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ enum ice_status ret_code;
+
+ if (cq->sq.count > 0) {
+ /* queue already initialized */
+ ret_code = ICE_ERR_NOT_READY;
+ goto init_ctrlq_exit;
+ }
+
+ /* verify input for valid configuration */
+ if (!cq->num_sq_entries || !cq->sq_buf_size) {
+ ret_code = ICE_ERR_CFG;
+ goto init_ctrlq_exit;
+ }
+
+ cq->sq.next_to_use = 0;
+ cq->sq.next_to_clean = 0;
+
+ /* allocate the ring memory */
+ ret_code = ice_alloc_ctrlq_sq_ring(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_exit;
+
+ /* allocate buffers in the rings */
+ ret_code = ice_alloc_sq_bufs(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_free_rings;
+
+ /* initialize base registers */
+ ret_code = ice_cfg_sq_regs(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_free_rings;
+
+ /* success! */
+ cq->sq.count = cq->num_sq_entries;
+ goto init_ctrlq_exit;
+
+init_ctrlq_free_rings:
+ ice_free_ctrlq_sq_ring(hw, cq);
+
+init_ctrlq_exit:
+ return ret_code;
+}
+
+/**
+ * ice_init_rq - initialize ARQ
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * The main initialization routine for the Admin Receive (Event) Queue.
+ * Prior to calling this function, drivers *MUST* set the following fields
+ * in the cq->structure:
+ * - cq->num_rq_entries
+ * - cq->rq_buf_size
+ *
+ * Do *NOT* hold the lock when calling this as the memory allocation routines
+ * called are not going to be atomic context safe
+ */
+static enum ice_status ice_init_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ enum ice_status ret_code;
+
+ if (cq->rq.count > 0) {
+ /* queue already initialized */
+ ret_code = ICE_ERR_NOT_READY;
+ goto init_ctrlq_exit;
+ }
+
+ /* verify input for valid configuration */
+ if (!cq->num_rq_entries || !cq->rq_buf_size) {
+ ret_code = ICE_ERR_CFG;
+ goto init_ctrlq_exit;
+ }
+
+ cq->rq.next_to_use = 0;
+ cq->rq.next_to_clean = 0;
+
+ /* allocate the ring memory */
+ ret_code = ice_alloc_ctrlq_rq_ring(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_exit;
+
+ /* allocate buffers in the rings */
+ ret_code = ice_alloc_rq_bufs(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_free_rings;
+
+ /* initialize base registers */
+ ret_code = ice_cfg_rq_regs(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_free_rings;
+
+ /* success! */
+ cq->rq.count = cq->num_rq_entries;
+ goto init_ctrlq_exit;
+
+init_ctrlq_free_rings:
+ ice_free_ctrlq_rq_ring(hw, cq);
+
+init_ctrlq_exit:
+ return ret_code;
+}
+
+/**
+ * ice_shutdown_sq - shutdown the Control ATQ
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for the Control Transmit Queue
+ */
+static enum ice_status
+ice_shutdown_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ enum ice_status ret_code = 0;
+
+ mutex_lock(&cq->sq_lock);
+
+ if (!cq->sq.count) {
+ ret_code = ICE_ERR_NOT_READY;
+ goto shutdown_sq_out;
+ }
+
+ /* Stop firmware AdminQ processing */
+ wr32(hw, cq->sq.head, 0);
+ wr32(hw, cq->sq.tail, 0);
+ wr32(hw, cq->sq.len, 0);
+ wr32(hw, cq->sq.bal, 0);
+ wr32(hw, cq->sq.bah, 0);
+
+ cq->sq.count = 0; /* to indicate uninitialized queue */
+
+ /* free ring buffers and the ring itself */
+ ice_free_sq_bufs(hw, cq);
+ ice_free_ctrlq_sq_ring(hw, cq);
+
+shutdown_sq_out:
+ mutex_unlock(&cq->sq_lock);
+ return ret_code;
+}
+
+/**
+ * ice_aq_ver_check - Check the reported AQ API version.
+ * @fw_branch: The "branch" of FW, typically describes the device type
+ * @fw_major: The major version of the FW API
+ * @fw_minor: The minor version increment of the FW API
+ *
+ * Checks if the driver should load on a given AQ API version.
+ *
+ * Return: 'true' iff the driver should attempt to load. 'false' otherwise.
+ */
+static bool ice_aq_ver_check(u8 fw_branch, u8 fw_major, u8 fw_minor)
+{
+ if (fw_branch != EXP_FW_API_VER_BRANCH)
+ return false;
+ if (fw_major != EXP_FW_API_VER_MAJOR)
+ return false;
+ if (fw_minor != EXP_FW_API_VER_MINOR)
+ return false;
+ return true;
+}
+
+/**
+ * ice_shutdown_rq - shutdown Control ARQ
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * The main shutdown routine for the Control Receive Queue
+ */
+static enum ice_status
+ice_shutdown_rq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ enum ice_status ret_code = 0;
+
+ mutex_lock(&cq->rq_lock);
+
+ if (!cq->rq.count) {
+ ret_code = ICE_ERR_NOT_READY;
+ goto shutdown_rq_out;
+ }
+
+ /* Stop Control Queue processing */
+ wr32(hw, cq->rq.head, 0);
+ wr32(hw, cq->rq.tail, 0);
+ wr32(hw, cq->rq.len, 0);
+ wr32(hw, cq->rq.bal, 0);
+ wr32(hw, cq->rq.bah, 0);
+
+ /* set rq.count to 0 to indicate uninitialized queue */
+ cq->rq.count = 0;
+
+ /* free ring buffers and the ring itself */
+ ice_free_rq_bufs(hw, cq);
+ ice_free_ctrlq_rq_ring(hw, cq);
+
+shutdown_rq_out:
+ mutex_unlock(&cq->rq_lock);
+ return ret_code;
+}
+
+/**
+ * ice_init_check_adminq - Check version for Admin Queue to know if its alive
+ * @hw: pointer to the hardware structure
+ */
+static enum ice_status ice_init_check_adminq(struct ice_hw *hw)
+{
+ struct ice_ctl_q_info *cq = &hw->adminq;
+ enum ice_status status;
+
+ status = ice_aq_get_fw_ver(hw, NULL);
+ if (status)
+ goto init_ctrlq_free_rq;
+
+ if (!ice_aq_ver_check(hw->api_branch, hw->api_maj_ver,
+ hw->api_min_ver)) {
+ status = ICE_ERR_FW_API_VER;
+ goto init_ctrlq_free_rq;
+ }
+
+ return 0;
+
+init_ctrlq_free_rq:
+ ice_shutdown_rq(hw, cq);
+ ice_shutdown_sq(hw, cq);
+ mutex_destroy(&cq->sq_lock);
+ mutex_destroy(&cq->rq_lock);
+ return status;
+}
+
+/**
+ * ice_init_ctrlq - main initialization routine for any control Queue
+ * @hw: pointer to the hardware structure
+ * @q_type: specific Control queue type
+ *
+ * Prior to calling this function, drivers *MUST* set the following fields
+ * in the cq->structure:
+ * - cq->num_sq_entries
+ * - cq->num_rq_entries
+ * - cq->rq_buf_size
+ * - cq->sq_buf_size
+ *
+ */
+static enum ice_status ice_init_ctrlq(struct ice_hw *hw, enum ice_ctl_q q_type)
+{
+ struct ice_ctl_q_info *cq;
+ enum ice_status ret_code;
+
+ switch (q_type) {
+ case ICE_CTL_Q_ADMIN:
+ ice_adminq_init_regs(hw);
+ cq = &hw->adminq;
+ break;
+ default:
+ return ICE_ERR_PARAM;
+ }
+ cq->qtype = q_type;
+
+ /* verify input for valid configuration */
+ if (!cq->num_rq_entries || !cq->num_sq_entries ||
+ !cq->rq_buf_size || !cq->sq_buf_size) {
+ return ICE_ERR_CFG;
+ }
+ mutex_init(&cq->sq_lock);
+ mutex_init(&cq->rq_lock);
+
+ /* setup SQ command write back timeout */
+ cq->sq_cmd_timeout = ICE_CTL_Q_SQ_CMD_TIMEOUT;
+
+ /* allocate the ATQ */
+ ret_code = ice_init_sq(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_destroy_locks;
+
+ /* allocate the ARQ */
+ ret_code = ice_init_rq(hw, cq);
+ if (ret_code)
+ goto init_ctrlq_free_sq;
+
+ /* success! */
+ return 0;
+
+init_ctrlq_free_sq:
+ ice_shutdown_sq(hw, cq);
+init_ctrlq_destroy_locks:
+ mutex_destroy(&cq->sq_lock);
+ mutex_destroy(&cq->rq_lock);
+ return ret_code;
+}
+
+/**
+ * ice_init_all_ctrlq - main initialization routine for all control queues
+ * @hw: pointer to the hardware structure
+ *
+ * Prior to calling this function, drivers *MUST* set the following fields
+ * in the cq->structure for all control queues:
+ * - cq->num_sq_entries
+ * - cq->num_rq_entries
+ * - cq->rq_buf_size
+ * - cq->sq_buf_size
+ */
+enum ice_status ice_init_all_ctrlq(struct ice_hw *hw)
+{
+ enum ice_status ret_code;
+
+ /* Init FW admin queue */
+ ret_code = ice_init_ctrlq(hw, ICE_CTL_Q_ADMIN);
+ if (ret_code)
+ return ret_code;
+
+ return ice_init_check_adminq(hw);
+}
+
+/**
+ * ice_shutdown_ctrlq - shutdown routine for any control queue
+ * @hw: pointer to the hardware structure
+ * @q_type: specific Control queue type
+ */
+static void ice_shutdown_ctrlq(struct ice_hw *hw, enum ice_ctl_q q_type)
+{
+ struct ice_ctl_q_info *cq;
+
+ switch (q_type) {
+ case ICE_CTL_Q_ADMIN:
+ cq = &hw->adminq;
+ if (ice_check_sq_alive(hw, cq))
+ ice_aq_q_shutdown(hw, true);
+ break;
+ default:
+ return;
+ }
+
+ ice_shutdown_sq(hw, cq);
+ ice_shutdown_rq(hw, cq);
+ mutex_destroy(&cq->sq_lock);
+ mutex_destroy(&cq->rq_lock);
+}
+
+/**
+ * ice_shutdown_all_ctrlq - shutdown routine for all control queues
+ * @hw: pointer to the hardware structure
+ */
+void ice_shutdown_all_ctrlq(struct ice_hw *hw)
+{
+ /* Shutdown FW admin queue */
+ ice_shutdown_ctrlq(hw, ICE_CTL_Q_ADMIN);
+}
+
+/**
+ * ice_clean_sq - cleans Admin send queue (ATQ)
+ * @hw: pointer to the hardware structure
+ * @cq: pointer to the specific Control queue
+ *
+ * returns the number of free desc
+ */
+static u16 ice_clean_sq(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ struct ice_ctl_q_ring *sq = &cq->sq;
+ u16 ntc = sq->next_to_clean;
+ struct ice_sq_cd *details;
+ struct ice_aq_desc *desc;
+
+ desc = ICE_CTL_Q_DESC(*sq, ntc);
+ details = ICE_CTL_Q_DETAILS(*sq, ntc);
+
+ while (rd32(hw, cq->sq.head) != ntc) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "ntc %d head %d.\n", ntc, rd32(hw, cq->sq.head));
+ memset(desc, 0, sizeof(*desc));
+ memset(details, 0, sizeof(*details));
+ ntc++;
+ if (ntc == sq->count)
+ ntc = 0;
+ desc = ICE_CTL_Q_DESC(*sq, ntc);
+ details = ICE_CTL_Q_DETAILS(*sq, ntc);
+ }
+
+ sq->next_to_clean = ntc;
+
+ return ICE_CTL_Q_DESC_UNUSED(sq);
+}
+
+/**
+ * ice_sq_done - check if FW has processed the Admin Send Queue (ATQ)
+ * @hw: pointer to the hw struct
+ * @cq: pointer to the specific Control queue
+ *
+ * Returns true if the firmware has processed all descriptors on the
+ * admin send queue. Returns false if there are still requests pending.
+ */
+static bool ice_sq_done(struct ice_hw *hw, struct ice_ctl_q_info *cq)
+{
+ /* AQ designers suggest use of head for better
+ * timing reliability than DD bit
+ */
+ return rd32(hw, cq->sq.head) == cq->sq.next_to_use;
+}
+
+/**
+ * ice_sq_send_cmd - send command to Control Queue (ATQ)
+ * @hw: pointer to the hw struct
+ * @cq: pointer to the specific Control queue
+ * @desc: prefilled descriptor describing the command (non DMA mem)
+ * @buf: buffer to use for indirect commands (or NULL for direct commands)
+ * @buf_size: size of buffer for indirect commands (or 0 for direct commands)
+ * @cd: pointer to command details structure
+ *
+ * This is the main send command routine for the ATQ. It runs the q,
+ * cleans the queue, etc.
+ */
+enum ice_status
+ice_sq_send_cmd(struct ice_hw *hw, struct ice_ctl_q_info *cq,
+ struct ice_aq_desc *desc, void *buf, u16 buf_size,
+ struct ice_sq_cd *cd)
+{
+ struct ice_dma_mem *dma_buf = NULL;
+ struct ice_aq_desc *desc_on_ring;
+ bool cmd_completed = false;
+ enum ice_status status = 0;
+ struct ice_sq_cd *details;
+ u32 total_delay = 0;
+ u16 retval = 0;
+ u32 val = 0;
+
+ mutex_lock(&cq->sq_lock);
+
+ cq->sq_last_status = ICE_AQ_RC_OK;
+
+ if (!cq->sq.count) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Control Send queue not initialized.\n");
+ status = ICE_ERR_AQ_EMPTY;
+ goto sq_send_command_error;
+ }
+
+ if ((buf && !buf_size) || (!buf && buf_size)) {
+ status = ICE_ERR_PARAM;
+ goto sq_send_command_error;
+ }
+
+ if (buf) {
+ if (buf_size > cq->sq_buf_size) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Invalid buffer size for Control Send queue: %d.\n",
+ buf_size);
+ status = ICE_ERR_INVAL_SIZE;
+ goto sq_send_command_error;
+ }
+
+ desc->flags |= cpu_to_le16(ICE_AQ_FLAG_BUF);
+ if (buf_size > ICE_AQ_LG_BUF)
+ desc->flags |= cpu_to_le16(ICE_AQ_FLAG_LB);
+ }
+
+ val = rd32(hw, cq->sq.head);
+ if (val >= cq->num_sq_entries) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "head overrun at %d in the Control Send Queue ring\n",
+ val);
+ status = ICE_ERR_AQ_EMPTY;
+ goto sq_send_command_error;
+ }
+
+ details = ICE_CTL_Q_DETAILS(cq->sq, cq->sq.next_to_use);
+ if (cd)
+ memcpy(details, cd, sizeof(*details));
+ else
+ memset(details, 0, sizeof(*details));
+
+ /* Call clean and check queue available function to reclaim the
+ * descriptors that were processed by FW/MBX; the function returns the
+ * number of desc available. The clean function called here could be
+ * called in a separate thread in case of asynchronous completions.
+ */
+ if (ice_clean_sq(hw, cq) == 0) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Error: Control Send Queue is full.\n");
+ status = ICE_ERR_AQ_FULL;
+ goto sq_send_command_error;
+ }
+
+ /* initialize the temp desc pointer with the right desc */
+ desc_on_ring = ICE_CTL_Q_DESC(cq->sq, cq->sq.next_to_use);
+
+ /* if the desc is available copy the temp desc to the right place */
+ memcpy(desc_on_ring, desc, sizeof(*desc_on_ring));
+
+ /* if buf is not NULL assume indirect command */
+ if (buf) {
+ dma_buf = &cq->sq.r.sq_bi[cq->sq.next_to_use];
+ /* copy the user buf into the respective DMA buf */
+ memcpy(dma_buf->va, buf, buf_size);
+ desc_on_ring->datalen = cpu_to_le16(buf_size);
+
+ /* Update the address values in the desc with the pa value
+ * for respective buffer
+ */
+ desc_on_ring->params.generic.addr_high =
+ cpu_to_le32(upper_32_bits(dma_buf->pa));
+ desc_on_ring->params.generic.addr_low =
+ cpu_to_le32(lower_32_bits(dma_buf->pa));
+ }
+
+ /* Debug desc and buffer */
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "ATQ: Control Send queue desc and buffer:\n");
+
+ ice_debug_cq(hw, ICE_DBG_AQ_CMD, (void *)desc_on_ring, buf, buf_size);
+
+ (cq->sq.next_to_use)++;
+ if (cq->sq.next_to_use == cq->sq.count)
+ cq->sq.next_to_use = 0;
+ wr32(hw, cq->sq.tail, cq->sq.next_to_use);
+
+ do {
+ if (ice_sq_done(hw, cq))
+ break;
+
+ mdelay(1);
+ total_delay++;
+ } while (total_delay < cq->sq_cmd_timeout);
+
+ /* if ready, copy the desc back to temp */
+ if (ice_sq_done(hw, cq)) {
+ memcpy(desc, desc_on_ring, sizeof(*desc));
+ if (buf) {
+ /* get returned length to copy */
+ u16 copy_size = le16_to_cpu(desc->datalen);
+
+ if (copy_size > buf_size) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Return len %d > than buf len %d\n",
+ copy_size, buf_size);
+ status = ICE_ERR_AQ_ERROR;
+ } else {
+ memcpy(buf, dma_buf->va, copy_size);
+ }
+ }
+ retval = le16_to_cpu(desc->retval);
+ if (retval) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Control Send Queue command completed with error 0x%x\n",
+ retval);
+
+ /* strip off FW internal code */
+ retval &= 0xff;
+ }
+ cmd_completed = true;
+ if (!status && retval != ICE_AQ_RC_OK)
+ status = ICE_ERR_AQ_ERROR;
+ cq->sq_last_status = (enum ice_aq_err)retval;
+ }
+
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "ATQ: desc and buffer writeback:\n");
+
+ ice_debug_cq(hw, ICE_DBG_AQ_CMD, (void *)desc, buf, buf_size);
+
+ /* save writeback AQ if requested */
+ if (details->wb_desc)
+ memcpy(details->wb_desc, desc_on_ring,
+ sizeof(*details->wb_desc));
+
+ /* update the error if time out occurred */
+ if (!cmd_completed) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Control Send Queue Writeback timeout.\n");
+ status = ICE_ERR_AQ_TIMEOUT;
+ }
+
+sq_send_command_error:
+ mutex_unlock(&cq->sq_lock);
+ return status;
+}
+
+/**
+ * ice_fill_dflt_direct_cmd_desc - AQ descriptor helper function
+ * @desc: pointer to the temp descriptor (non DMA mem)
+ * @opcode: the opcode can be used to decide which flags to turn off or on
+ *
+ * Fill the desc with default values
+ */
+void ice_fill_dflt_direct_cmd_desc(struct ice_aq_desc *desc, u16 opcode)
+{
+ /* zero out the desc */
+ memset(desc, 0, sizeof(*desc));
+ desc->opcode = cpu_to_le16(opcode);
+ desc->flags = cpu_to_le16(ICE_AQ_FLAG_SI);
+}
+
+/**
+ * ice_clean_rq_elem
+ * @hw: pointer to the hw struct
+ * @cq: pointer to the specific Control queue
+ * @e: event info from the receive descriptor, includes any buffers
+ * @pending: number of events that could be left to process
+ *
+ * This function cleans one Admin Receive Queue element and returns
+ * the contents through e. It can also return how many events are
+ * left to process through 'pending'.
+ */
+enum ice_status
+ice_clean_rq_elem(struct ice_hw *hw, struct ice_ctl_q_info *cq,
+ struct ice_rq_event_info *e, u16 *pending)
+{
+ u16 ntc = cq->rq.next_to_clean;
+ enum ice_status ret_code = 0;
+ struct ice_aq_desc *desc;
+ struct ice_dma_mem *bi;
+ u16 desc_idx;
+ u16 datalen;
+ u16 flags;
+ u16 ntu;
+
+ /* pre-clean the event info */
+ memset(&e->desc, 0, sizeof(e->desc));
+
+ /* take the lock before we start messing with the ring */
+ mutex_lock(&cq->rq_lock);
+
+ if (!cq->rq.count) {
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Control Receive queue not initialized.\n");
+ ret_code = ICE_ERR_AQ_EMPTY;
+ goto clean_rq_elem_err;
+ }
+
+ /* set next_to_use to head */
+ ntu = (u16)(rd32(hw, cq->rq.head) & cq->rq.head_mask);
+
+ if (ntu == ntc) {
+ /* nothing to do - shouldn't need to update ring's values */
+ ret_code = ICE_ERR_AQ_NO_WORK;
+ goto clean_rq_elem_out;
+ }
+
+ /* now clean the next descriptor */
+ desc = ICE_CTL_Q_DESC(cq->rq, ntc);
+ desc_idx = ntc;
+
+ flags = le16_to_cpu(desc->flags);
+ if (flags & ICE_AQ_FLAG_ERR) {
+ ret_code = ICE_ERR_AQ_ERROR;
+ cq->rq_last_status = (enum ice_aq_err)le16_to_cpu(desc->retval);
+ ice_debug(hw, ICE_DBG_AQ_MSG,
+ "Control Receive Queue Event received with error 0x%x\n",
+ cq->rq_last_status);
+ }
+ memcpy(&e->desc, desc, sizeof(e->desc));
+ datalen = le16_to_cpu(desc->datalen);
+ e->msg_len = min(datalen, e->buf_len);
+ if (e->msg_buf && e->msg_len)
+ memcpy(e->msg_buf, cq->rq.r.rq_bi[desc_idx].va, e->msg_len);
+
+ ice_debug(hw, ICE_DBG_AQ_MSG, "ARQ: desc and buffer:\n");
+
+ ice_debug_cq(hw, ICE_DBG_AQ_CMD, (void *)desc, e->msg_buf,
+ cq->rq_buf_size);
+
+ /* Restore the original datalen and buffer address in the desc,
+ * FW updates datalen to indicate the event message size
+ */
+ bi = &cq->rq.r.rq_bi[ntc];
+ memset(desc, 0, sizeof(*desc));
+
+ desc->flags = cpu_to_le16(ICE_AQ_FLAG_BUF);
+ if (cq->rq_buf_size > ICE_AQ_LG_BUF)
+ desc->flags |= cpu_to_le16(ICE_AQ_FLAG_LB);
+ desc->datalen = cpu_to_le16(bi->size);
+ desc->params.generic.addr_high = cpu_to_le32(upper_32_bits(bi->pa));
+ desc->params.generic.addr_low = cpu_to_le32(lower_32_bits(bi->pa));
+
+ /* set tail = the last cleaned desc index. */
+ wr32(hw, cq->rq.tail, ntc);
+ /* ntc is updated to tail + 1 */
+ ntc++;
+ if (ntc == cq->num_rq_entries)
+ ntc = 0;
+ cq->rq.next_to_clean = ntc;
+ cq->rq.next_to_use = ntu;
+
+clean_rq_elem_out:
+ /* Set pending if needed, unlock and return */
+ if (pending)
+ *pending = (u16)((ntc > ntu ? cq->rq.count : 0) + (ntu - ntc));
+clean_rq_elem_err:
+ mutex_unlock(&cq->rq_lock);
+
+ return ret_code;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_CONTROLQ_H_
+#define _ICE_CONTROLQ_H_
+
+#include "ice_adminq_cmd.h"
+
+/* Maximum buffer lengths for all control queue types */
+#define ICE_AQ_MAX_BUF_LEN 4096
+
+#define ICE_CTL_Q_DESC(R, i) \
+ (&(((struct ice_aq_desc *)((R).desc_buf.va))[i]))
+
+#define ICE_CTL_Q_DESC_UNUSED(R) \
+ (u16)((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->count) + \
+ (R)->next_to_clean - (R)->next_to_use - 1)
+
+/* Defines that help manage the driver vs FW API checks.
+ * Take a look at ice_aq_ver_check in ice_controlq.c for actual usage.
+ *
+ */
+#define EXP_FW_API_VER_BRANCH 0x00
+#define EXP_FW_API_VER_MAJOR 0x00
+#define EXP_FW_API_VER_MINOR 0x01
+
+/* Different control queue types: These are mainly for SW consumption. */
+enum ice_ctl_q {
+ ICE_CTL_Q_UNKNOWN = 0,
+ ICE_CTL_Q_ADMIN,
+};
+
+/* Control Queue default settings */
+#define ICE_CTL_Q_SQ_CMD_TIMEOUT 250 /* msecs */
+
+struct ice_ctl_q_ring {
+ void *dma_head; /* Virtual address to dma head */
+ struct ice_dma_mem desc_buf; /* descriptor ring memory */
+ void *cmd_buf; /* command buffer memory */
+
+ union {
+ struct ice_dma_mem *sq_bi;
+ struct ice_dma_mem *rq_bi;
+ } r;
+
+ u16 count; /* Number of descriptors */
+
+ /* used for interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+
+ /* used for queue tracking */
+ u32 head;
+ u32 tail;
+ u32 len;
+ u32 bah;
+ u32 bal;
+ u32 len_mask;
+ u32 len_ena_mask;
+ u32 head_mask;
+};
+
+/* sq transaction details */
+struct ice_sq_cd {
+ struct ice_aq_desc *wb_desc;
+};
+
+#define ICE_CTL_Q_DETAILS(R, i) (&(((struct ice_sq_cd *)((R).cmd_buf))[i]))
+
+/* rq event information */
+struct ice_rq_event_info {
+ struct ice_aq_desc desc;
+ u16 msg_len;
+ u16 buf_len;
+ u8 *msg_buf;
+};
+
+/* Control Queue information */
+struct ice_ctl_q_info {
+ enum ice_ctl_q qtype;
+ struct ice_ctl_q_ring rq; /* receive queue */
+ struct ice_ctl_q_ring sq; /* send queue */
+ u32 sq_cmd_timeout; /* send queue cmd write back timeout */
+ u16 num_rq_entries; /* receive queue depth */
+ u16 num_sq_entries; /* send queue depth */
+ u16 rq_buf_size; /* receive queue buffer size */
+ u16 sq_buf_size; /* send queue buffer size */
+ struct mutex sq_lock; /* Send queue lock */
+ struct mutex rq_lock; /* Receive queue lock */
+ enum ice_aq_err sq_last_status; /* last status on send queue */
+ enum ice_aq_err rq_last_status; /* last status on receive queue */
+};
+
+#endif /* _ICE_CONTROLQ_H_ */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_DEVIDS_H_
+#define _ICE_DEVIDS_H_
+
+/* Device IDs */
+/* Intel(R) Ethernet Controller C810 for backplane */
+#define ICE_DEV_ID_C810_BACKPLANE 0x1591
+/* Intel(R) Ethernet Controller C810 for QSFP */
+#define ICE_DEV_ID_C810_QSFP 0x1592
+/* Intel(R) Ethernet Controller C810 for SFP */
+#define ICE_DEV_ID_C810_SFP 0x1593
+/* Intel(R) Ethernet Controller C810/X557-AT 10GBASE-T */
+#define ICE_DEV_ID_C810_10G_BASE_T 0x1594
+/* Intel(R) Ethernet Controller C810 1GbE */
+#define ICE_DEV_ID_C810_SGMII 0x1595
+
+#endif /* _ICE_DEVIDS_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+/* ethtool support for ice */
+
+#include "ice.h"
+
+struct ice_stats {
+ char stat_string[ETH_GSTRING_LEN];
+ int sizeof_stat;
+ int stat_offset;
+};
+
+#define ICE_STAT(_type, _name, _stat) { \
+ .stat_string = _name, \
+ .sizeof_stat = FIELD_SIZEOF(_type, _stat), \
+ .stat_offset = offsetof(_type, _stat) \
+}
+
+#define ICE_VSI_STAT(_name, _stat) \
+ ICE_STAT(struct ice_vsi, _name, _stat)
+#define ICE_PF_STAT(_name, _stat) \
+ ICE_STAT(struct ice_pf, _name, _stat)
+
+static int ice_q_stats_len(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+
+ return ((np->vsi->num_txq + np->vsi->num_rxq) *
+ (sizeof(struct ice_q_stats) / sizeof(u64)));
+}
+
+#define ICE_PF_STATS_LEN ARRAY_SIZE(ice_gstrings_pf_stats)
+#define ICE_VSI_STATS_LEN ARRAY_SIZE(ice_gstrings_vsi_stats)
+
+#define ICE_ALL_STATS_LEN(n) (ICE_PF_STATS_LEN + ICE_VSI_STATS_LEN + \
+ ice_q_stats_len(n))
+
+static const struct ice_stats ice_gstrings_vsi_stats[] = {
+ ICE_VSI_STAT("tx_unicast", eth_stats.tx_unicast),
+ ICE_VSI_STAT("rx_unicast", eth_stats.rx_unicast),
+ ICE_VSI_STAT("tx_multicast", eth_stats.tx_multicast),
+ ICE_VSI_STAT("rx_multicast", eth_stats.rx_multicast),
+ ICE_VSI_STAT("tx_broadcast", eth_stats.tx_broadcast),
+ ICE_VSI_STAT("rx_broadcast", eth_stats.rx_broadcast),
+ ICE_VSI_STAT("tx_bytes", eth_stats.tx_bytes),
+ ICE_VSI_STAT("rx_bytes", eth_stats.rx_bytes),
+ ICE_VSI_STAT("rx_discards", eth_stats.rx_discards),
+ ICE_VSI_STAT("tx_errors", eth_stats.tx_errors),
+ ICE_VSI_STAT("tx_linearize", tx_linearize),
+ ICE_VSI_STAT("rx_unknown_protocol", eth_stats.rx_unknown_protocol),
+ ICE_VSI_STAT("rx_alloc_fail", rx_buf_failed),
+ ICE_VSI_STAT("rx_pg_alloc_fail", rx_page_failed),
+};
+
+/* These PF_STATs might look like duplicates of some NETDEV_STATs,
+ * but they aren't. This device is capable of supporting multiple
+ * VSIs/netdevs on a single PF. The NETDEV_STATs are for individual
+ * netdevs whereas the PF_STATs are for the physical function that's
+ * hosting these netdevs.
+ *
+ * The PF_STATs are appended to the netdev stats only when ethtool -S
+ * is queried on the base PF netdev.
+ */
+static struct ice_stats ice_gstrings_pf_stats[] = {
+ ICE_PF_STAT("tx_bytes", stats.eth.tx_bytes),
+ ICE_PF_STAT("rx_bytes", stats.eth.rx_bytes),
+ ICE_PF_STAT("tx_unicast", stats.eth.tx_unicast),
+ ICE_PF_STAT("rx_unicast", stats.eth.rx_unicast),
+ ICE_PF_STAT("tx_multicast", stats.eth.tx_multicast),
+ ICE_PF_STAT("rx_multicast", stats.eth.rx_multicast),
+ ICE_PF_STAT("tx_broadcast", stats.eth.tx_broadcast),
+ ICE_PF_STAT("rx_broadcast", stats.eth.rx_broadcast),
+ ICE_PF_STAT("tx_errors", stats.eth.tx_errors),
+ ICE_PF_STAT("tx_size_64", stats.tx_size_64),
+ ICE_PF_STAT("rx_size_64", stats.rx_size_64),
+ ICE_PF_STAT("tx_size_127", stats.tx_size_127),
+ ICE_PF_STAT("rx_size_127", stats.rx_size_127),
+ ICE_PF_STAT("tx_size_255", stats.tx_size_255),
+ ICE_PF_STAT("rx_size_255", stats.rx_size_255),
+ ICE_PF_STAT("tx_size_511", stats.tx_size_511),
+ ICE_PF_STAT("rx_size_511", stats.rx_size_511),
+ ICE_PF_STAT("tx_size_1023", stats.tx_size_1023),
+ ICE_PF_STAT("rx_size_1023", stats.rx_size_1023),
+ ICE_PF_STAT("tx_size_1522", stats.tx_size_1522),
+ ICE_PF_STAT("rx_size_1522", stats.rx_size_1522),
+ ICE_PF_STAT("tx_size_big", stats.tx_size_big),
+ ICE_PF_STAT("rx_size_big", stats.rx_size_big),
+ ICE_PF_STAT("link_xon_tx", stats.link_xon_tx),
+ ICE_PF_STAT("link_xon_rx", stats.link_xon_rx),
+ ICE_PF_STAT("link_xoff_tx", stats.link_xoff_tx),
+ ICE_PF_STAT("link_xoff_rx", stats.link_xoff_rx),
+ ICE_PF_STAT("tx_dropped_link_down", stats.tx_dropped_link_down),
+ ICE_PF_STAT("rx_undersize", stats.rx_undersize),
+ ICE_PF_STAT("rx_fragments", stats.rx_fragments),
+ ICE_PF_STAT("rx_oversize", stats.rx_oversize),
+ ICE_PF_STAT("rx_jabber", stats.rx_jabber),
+ ICE_PF_STAT("rx_csum_bad", hw_csum_rx_error),
+ ICE_PF_STAT("rx_length_errors", stats.rx_len_errors),
+ ICE_PF_STAT("rx_dropped", stats.eth.rx_discards),
+ ICE_PF_STAT("rx_crc_errors", stats.crc_errors),
+ ICE_PF_STAT("illegal_bytes", stats.illegal_bytes),
+ ICE_PF_STAT("mac_local_faults", stats.mac_local_faults),
+ ICE_PF_STAT("mac_remote_faults", stats.mac_remote_faults),
+};
+
+static u32 ice_regs_dump_list[] = {
+ PFGEN_STATE,
+ PRTGEN_STATUS,
+ QRX_CTRL(0),
+ QINT_TQCTL(0),
+ QINT_RQCTL(0),
+ PFINT_OICR_ENA,
+ QRX_ITR(0),
+};
+
+/**
+ * ice_nvm_version_str - format the NVM version strings
+ * @hw: ptr to the hardware info
+ */
+static char *ice_nvm_version_str(struct ice_hw *hw)
+{
+ static char buf[ICE_ETHTOOL_FWVER_LEN];
+ u8 ver, patch;
+ u32 full_ver;
+ u16 build;
+
+ full_ver = hw->nvm.oem_ver;
+ ver = (u8)((full_ver & ICE_OEM_VER_MASK) >> ICE_OEM_VER_SHIFT);
+ build = (u16)((full_ver & ICE_OEM_VER_BUILD_MASK) >>
+ ICE_OEM_VER_BUILD_SHIFT);
+ patch = (u8)(full_ver & ICE_OEM_VER_PATCH_MASK);
+
+ snprintf(buf, sizeof(buf), "%x.%02x 0x%x %d.%d.%d",
+ (hw->nvm.ver & ICE_NVM_VER_HI_MASK) >> ICE_NVM_VER_HI_SHIFT,
+ (hw->nvm.ver & ICE_NVM_VER_LO_MASK) >> ICE_NVM_VER_LO_SHIFT,
+ hw->nvm.eetrack, ver, build, patch);
+
+ return buf;
+}
+
+static void
+ice_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *drvinfo)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+
+ strlcpy(drvinfo->driver, KBUILD_MODNAME, sizeof(drvinfo->driver));
+ strlcpy(drvinfo->version, ice_drv_ver, sizeof(drvinfo->version));
+ strlcpy(drvinfo->fw_version, ice_nvm_version_str(&pf->hw),
+ sizeof(drvinfo->fw_version));
+ strlcpy(drvinfo->bus_info, pci_name(pf->pdev),
+ sizeof(drvinfo->bus_info));
+}
+
+static int ice_get_regs_len(struct net_device __always_unused *netdev)
+{
+ return ARRAY_SIZE(ice_regs_dump_list);
+}
+
+static void
+ice_get_regs(struct net_device *netdev, struct ethtool_regs *regs, void *p)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_pf *pf = np->vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ u32 *regs_buf = (u32 *)p;
+ int i;
+
+ regs->version = 1;
+
+ for (i = 0; i < ARRAY_SIZE(ice_regs_dump_list) / sizeof(u32); ++i)
+ regs_buf[i] = rd32(hw, ice_regs_dump_list[i]);
+}
+
+static u32 ice_get_msglevel(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_pf *pf = np->vsi->back;
+
+#ifndef CONFIG_DYNAMIC_DEBUG
+ if (pf->hw.debug_mask)
+ netdev_info(netdev, "hw debug_mask: 0x%llX\n",
+ pf->hw.debug_mask);
+#endif /* !CONFIG_DYNAMIC_DEBUG */
+
+ return pf->msg_enable;
+}
+
+static void ice_set_msglevel(struct net_device *netdev, u32 data)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_pf *pf = np->vsi->back;
+
+#ifndef CONFIG_DYNAMIC_DEBUG
+ if (ICE_DBG_USER & data)
+ pf->hw.debug_mask = data;
+ else
+ pf->msg_enable = data;
+#else
+ pf->msg_enable = data;
+#endif /* !CONFIG_DYNAMIC_DEBUG */
+}
+
+static void ice_get_strings(struct net_device *netdev, u32 stringset, u8 *data)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ char *p = (char *)data;
+ unsigned int i;
+
+ switch (stringset) {
+ case ETH_SS_STATS:
+ for (i = 0; i < ICE_VSI_STATS_LEN; i++) {
+ snprintf(p, ETH_GSTRING_LEN, "%s",
+ ice_gstrings_vsi_stats[i].stat_string);
+ p += ETH_GSTRING_LEN;
+ }
+
+ ice_for_each_txq(vsi, i) {
+ snprintf(p, ETH_GSTRING_LEN,
+ "tx-queue-%u.tx_packets", i);
+ p += ETH_GSTRING_LEN;
+ snprintf(p, ETH_GSTRING_LEN, "tx-queue-%u.tx_bytes", i);
+ p += ETH_GSTRING_LEN;
+ }
+
+ ice_for_each_rxq(vsi, i) {
+ snprintf(p, ETH_GSTRING_LEN,
+ "rx-queue-%u.rx_packets", i);
+ p += ETH_GSTRING_LEN;
+ snprintf(p, ETH_GSTRING_LEN, "rx-queue-%u.rx_bytes", i);
+ p += ETH_GSTRING_LEN;
+ }
+
+ if (vsi->type != ICE_VSI_PF)
+ return;
+
+ for (i = 0; i < ICE_PF_STATS_LEN; i++) {
+ snprintf(p, ETH_GSTRING_LEN, "port.%s",
+ ice_gstrings_pf_stats[i].stat_string);
+ p += ETH_GSTRING_LEN;
+ }
+
+ break;
+ default:
+ break;
+ }
+}
+
+static int ice_get_sset_count(struct net_device *netdev, int sset)
+{
+ switch (sset) {
+ case ETH_SS_STATS:
+ return ICE_ALL_STATS_LEN(netdev);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static void
+ice_get_ethtool_stats(struct net_device *netdev,
+ struct ethtool_stats __always_unused *stats, u64 *data)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+ struct ice_ring *ring;
+ unsigned int j = 0;
+ int i = 0;
+ char *p;
+
+ for (j = 0; j < ICE_VSI_STATS_LEN; j++) {
+ p = (char *)vsi + ice_gstrings_vsi_stats[j].stat_offset;
+ data[i++] = (ice_gstrings_vsi_stats[j].sizeof_stat ==
+ sizeof(u64)) ? *(u64 *)p : *(u32 *)p;
+ }
+
+ /* populate per queue stats */
+ rcu_read_lock();
+
+ ice_for_each_txq(vsi, j) {
+ ring = READ_ONCE(vsi->tx_rings[j]);
+ if (!ring)
+ continue;
+ data[i++] = ring->stats.pkts;
+ data[i++] = ring->stats.bytes;
+ }
+
+ ice_for_each_rxq(vsi, j) {
+ ring = READ_ONCE(vsi->rx_rings[j]);
+ data[i++] = ring->stats.pkts;
+ data[i++] = ring->stats.bytes;
+ }
+
+ rcu_read_unlock();
+
+ if (vsi->type != ICE_VSI_PF)
+ return;
+
+ for (j = 0; j < ICE_PF_STATS_LEN; j++) {
+ p = (char *)pf + ice_gstrings_pf_stats[j].stat_offset;
+ data[i++] = (ice_gstrings_pf_stats[j].sizeof_stat ==
+ sizeof(u64)) ? *(u64 *)p : *(u32 *)p;
+ }
+}
+
+static int
+ice_get_link_ksettings(struct net_device *netdev,
+ struct ethtool_link_ksettings *ks)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_link_status *hw_link_info;
+ struct ice_vsi *vsi = np->vsi;
+ bool link_up;
+
+ hw_link_info = &vsi->port_info->phy.link_info;
+ link_up = hw_link_info->link_info & ICE_AQ_LINK_UP;
+
+ ethtool_link_ksettings_add_link_mode(ks, supported,
+ 10000baseT_Full);
+ ethtool_link_ksettings_add_link_mode(ks, advertising,
+ 10000baseT_Full);
+
+ /* set speed and duplex */
+ if (link_up) {
+ switch (hw_link_info->link_speed) {
+ case ICE_AQ_LINK_SPEED_100MB:
+ ks->base.speed = SPEED_100;
+ break;
+ case ICE_AQ_LINK_SPEED_2500MB:
+ ks->base.speed = SPEED_2500;
+ break;
+ case ICE_AQ_LINK_SPEED_5GB:
+ ks->base.speed = SPEED_5000;
+ break;
+ case ICE_AQ_LINK_SPEED_10GB:
+ ks->base.speed = SPEED_10000;
+ break;
+ case ICE_AQ_LINK_SPEED_25GB:
+ ks->base.speed = SPEED_25000;
+ break;
+ case ICE_AQ_LINK_SPEED_40GB:
+ ks->base.speed = SPEED_40000;
+ break;
+ default:
+ ks->base.speed = SPEED_UNKNOWN;
+ break;
+ }
+
+ ks->base.duplex = DUPLEX_FULL;
+ } else {
+ ks->base.speed = SPEED_UNKNOWN;
+ ks->base.duplex = DUPLEX_UNKNOWN;
+ }
+
+ /* set autoneg settings */
+ ks->base.autoneg = ((hw_link_info->an_info & ICE_AQ_AN_COMPLETED) ?
+ AUTONEG_ENABLE : AUTONEG_DISABLE);
+
+ /* set media type settings */
+ switch (vsi->port_info->phy.media_type) {
+ case ICE_MEDIA_FIBER:
+ ethtool_link_ksettings_add_link_mode(ks, supported, FIBRE);
+ ks->base.port = PORT_FIBRE;
+ break;
+ case ICE_MEDIA_BASET:
+ ethtool_link_ksettings_add_link_mode(ks, supported, TP);
+ ethtool_link_ksettings_add_link_mode(ks, advertising, TP);
+ ks->base.port = PORT_TP;
+ break;
+ case ICE_MEDIA_BACKPLANE:
+ ethtool_link_ksettings_add_link_mode(ks, supported, Autoneg);
+ ethtool_link_ksettings_add_link_mode(ks, supported, Backplane);
+ ethtool_link_ksettings_add_link_mode(ks, advertising, Autoneg);
+ ethtool_link_ksettings_add_link_mode(ks, advertising,
+ Backplane);
+ ks->base.port = PORT_NONE;
+ break;
+ case ICE_MEDIA_DA:
+ ethtool_link_ksettings_add_link_mode(ks, supported, FIBRE);
+ ethtool_link_ksettings_add_link_mode(ks, advertising, FIBRE);
+ ks->base.port = PORT_DA;
+ break;
+ default:
+ ks->base.port = PORT_OTHER;
+ break;
+ }
+
+ /* flow control is symmetric and always supported */
+ ethtool_link_ksettings_add_link_mode(ks, supported, Pause);
+
+ switch (vsi->port_info->fc.req_mode) {
+ case ICE_FC_FULL:
+ ethtool_link_ksettings_add_link_mode(ks, advertising, Pause);
+ break;
+ case ICE_FC_TX_PAUSE:
+ ethtool_link_ksettings_add_link_mode(ks, advertising,
+ Asym_Pause);
+ break;
+ case ICE_FC_RX_PAUSE:
+ ethtool_link_ksettings_add_link_mode(ks, advertising, Pause);
+ ethtool_link_ksettings_add_link_mode(ks, advertising,
+ Asym_Pause);
+ break;
+ case ICE_FC_PFC:
+ default:
+ ethtool_link_ksettings_del_link_mode(ks, advertising, Pause);
+ ethtool_link_ksettings_del_link_mode(ks, advertising,
+ Asym_Pause);
+ break;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_get_rxnfc - command to get RX flow classification rules
+ * @netdev: network interface device structure
+ * @cmd: ethtool rxnfc command
+ * @rule_locs: buffer to rturn Rx flow classification rules
+ *
+ * Returns Success if the command is supported.
+ */
+static int ice_get_rxnfc(struct net_device *netdev, struct ethtool_rxnfc *cmd,
+ u32 __always_unused *rule_locs)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ int ret = -EOPNOTSUPP;
+
+ switch (cmd->cmd) {
+ case ETHTOOL_GRXRINGS:
+ cmd->data = vsi->rss_size;
+ ret = 0;
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+static void
+ice_get_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+
+ ring->rx_max_pending = ICE_MAX_NUM_DESC;
+ ring->tx_max_pending = ICE_MAX_NUM_DESC;
+ ring->rx_pending = vsi->rx_rings[0]->count;
+ ring->tx_pending = vsi->tx_rings[0]->count;
+ ring->rx_mini_pending = ICE_MIN_NUM_DESC;
+ ring->rx_mini_max_pending = 0;
+ ring->rx_jumbo_max_pending = 0;
+ ring->rx_jumbo_pending = 0;
+}
+
+static int
+ice_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring)
+{
+ struct ice_ring *tx_rings = NULL, *rx_rings = NULL;
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+ int i, timeout = 50, err = 0;
+ u32 new_rx_cnt, new_tx_cnt;
+
+ if (ring->tx_pending > ICE_MAX_NUM_DESC ||
+ ring->tx_pending < ICE_MIN_NUM_DESC ||
+ ring->rx_pending > ICE_MAX_NUM_DESC ||
+ ring->rx_pending < ICE_MIN_NUM_DESC) {
+ netdev_err(netdev, "Descriptors requested (Tx: %d / Rx: %d) out of range [%d-%d]\n",
+ ring->tx_pending, ring->rx_pending,
+ ICE_MIN_NUM_DESC, ICE_MAX_NUM_DESC);
+ return -EINVAL;
+ }
+
+ new_tx_cnt = ALIGN(ring->tx_pending, ICE_REQ_DESC_MULTIPLE);
+ new_rx_cnt = ALIGN(ring->rx_pending, ICE_REQ_DESC_MULTIPLE);
+
+ /* if nothing to do return success */
+ if (new_tx_cnt == vsi->tx_rings[0]->count &&
+ new_rx_cnt == vsi->rx_rings[0]->count) {
+ netdev_dbg(netdev, "Nothing to change, descriptor count is same as requested\n");
+ return 0;
+ }
+
+ while (test_and_set_bit(__ICE_CFG_BUSY, pf->state)) {
+ timeout--;
+ if (!timeout)
+ return -EBUSY;
+ usleep_range(1000, 2000);
+ }
+
+ /* set for the next time the netdev is started */
+ if (!netif_running(vsi->netdev)) {
+ for (i = 0; i < vsi->alloc_txq; i++)
+ vsi->tx_rings[i]->count = new_tx_cnt;
+ for (i = 0; i < vsi->alloc_rxq; i++)
+ vsi->rx_rings[i]->count = new_rx_cnt;
+ netdev_dbg(netdev, "Link is down, descriptor count change happens when link is brought up\n");
+ goto done;
+ }
+
+ if (new_tx_cnt == vsi->tx_rings[0]->count)
+ goto process_rx;
+
+ /* alloc updated Tx resources */
+ netdev_info(netdev, "Changing Tx descriptor count from %d to %d\n",
+ vsi->tx_rings[0]->count, new_tx_cnt);
+
+ tx_rings = devm_kcalloc(&pf->pdev->dev, vsi->alloc_txq,
+ sizeof(struct ice_ring), GFP_KERNEL);
+ if (!tx_rings) {
+ err = -ENOMEM;
+ goto done;
+ }
+
+ for (i = 0; i < vsi->num_txq; i++) {
+ /* clone ring and setup updated count */
+ tx_rings[i] = *vsi->tx_rings[i];
+ tx_rings[i].count = new_tx_cnt;
+ tx_rings[i].desc = NULL;
+ tx_rings[i].tx_buf = NULL;
+ err = ice_setup_tx_ring(&tx_rings[i]);
+ if (err) {
+ while (i) {
+ i--;
+ ice_clean_tx_ring(&tx_rings[i]);
+ }
+ devm_kfree(&pf->pdev->dev, tx_rings);
+ goto done;
+ }
+ }
+
+process_rx:
+ if (new_rx_cnt == vsi->rx_rings[0]->count)
+ goto process_link;
+
+ /* alloc updated Rx resources */
+ netdev_info(netdev, "Changing Rx descriptor count from %d to %d\n",
+ vsi->rx_rings[0]->count, new_rx_cnt);
+
+ rx_rings = devm_kcalloc(&pf->pdev->dev, vsi->alloc_rxq,
+ sizeof(struct ice_ring), GFP_KERNEL);
+ if (!rx_rings) {
+ err = -ENOMEM;
+ goto done;
+ }
+
+ for (i = 0; i < vsi->num_rxq; i++) {
+ /* clone ring and setup updated count */
+ rx_rings[i] = *vsi->rx_rings[i];
+ rx_rings[i].count = new_rx_cnt;
+ rx_rings[i].desc = NULL;
+ rx_rings[i].rx_buf = NULL;
+ /* this is to allow wr32 to have something to write to
+ * during early allocation of Rx buffers
+ */
+ rx_rings[i].tail = vsi->back->hw.hw_addr + PRTGEN_STATUS;
+
+ err = ice_setup_rx_ring(&rx_rings[i]);
+ if (err)
+ goto rx_unwind;
+
+ /* allocate Rx buffers */
+ err = ice_alloc_rx_bufs(&rx_rings[i],
+ ICE_DESC_UNUSED(&rx_rings[i]));
+rx_unwind:
+ if (err) {
+ while (i) {
+ i--;
+ ice_free_rx_ring(&rx_rings[i]);
+ }
+ devm_kfree(&pf->pdev->dev, rx_rings);
+ err = -ENOMEM;
+ goto free_tx;
+ }
+ }
+
+process_link:
+ /* Bring interface down, copy in the new ring info, then restore the
+ * interface. if VSI is up, bring it down and then back up
+ */
+ if (!test_and_set_bit(__ICE_DOWN, vsi->state)) {
+ ice_down(vsi);
+
+ if (tx_rings) {
+ for (i = 0; i < vsi->alloc_txq; i++) {
+ ice_free_tx_ring(vsi->tx_rings[i]);
+ *vsi->tx_rings[i] = tx_rings[i];
+ }
+ devm_kfree(&pf->pdev->dev, tx_rings);
+ }
+
+ if (rx_rings) {
+ for (i = 0; i < vsi->alloc_rxq; i++) {
+ ice_free_rx_ring(vsi->rx_rings[i]);
+ /* copy the real tail offset */
+ rx_rings[i].tail = vsi->rx_rings[i]->tail;
+ /* this is to fake out the allocation routine
+ * into thinking it has to realloc everything
+ * but the recycling logic will let us re-use
+ * the buffers allocated above
+ */
+ rx_rings[i].next_to_use = 0;
+ rx_rings[i].next_to_clean = 0;
+ rx_rings[i].next_to_alloc = 0;
+ *vsi->rx_rings[i] = rx_rings[i];
+ }
+ devm_kfree(&pf->pdev->dev, rx_rings);
+ }
+
+ ice_up(vsi);
+ }
+ goto done;
+
+free_tx:
+ /* error cleanup if the Rx allocations failed after getting Tx */
+ if (tx_rings) {
+ for (i = 0; i < vsi->alloc_txq; i++)
+ ice_free_tx_ring(&tx_rings[i]);
+ devm_kfree(&pf->pdev->dev, tx_rings);
+ }
+
+done:
+ clear_bit(__ICE_CFG_BUSY, pf->state);
+ return err;
+}
+
+static int ice_nway_reset(struct net_device *netdev)
+{
+ /* restart autonegotiation */
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_link_status *hw_link_info;
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_port_info *pi;
+ enum ice_status status;
+ bool link_up;
+
+ pi = vsi->port_info;
+ hw_link_info = &pi->phy.link_info;
+ link_up = hw_link_info->link_info & ICE_AQ_LINK_UP;
+
+ status = ice_aq_set_link_restart_an(pi, link_up, NULL);
+ if (status) {
+ netdev_info(netdev, "link restart failed, err %d aq_err %d\n",
+ status, pi->hw->adminq.sq_last_status);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_get_pauseparam - Get Flow Control status
+ * @netdev: network interface device structure
+ * @pause: ethernet pause (flow control) parameters
+ */
+static void
+ice_get_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_port_info *pi;
+
+ pi = np->vsi->port_info;
+ pause->autoneg =
+ ((pi->phy.link_info.an_info & ICE_AQ_AN_COMPLETED) ?
+ AUTONEG_ENABLE : AUTONEG_DISABLE);
+
+ if (pi->fc.current_mode == ICE_FC_RX_PAUSE) {
+ pause->rx_pause = 1;
+ } else if (pi->fc.current_mode == ICE_FC_TX_PAUSE) {
+ pause->tx_pause = 1;
+ } else if (pi->fc.current_mode == ICE_FC_FULL) {
+ pause->rx_pause = 1;
+ pause->tx_pause = 1;
+ }
+}
+
+/**
+ * ice_set_pauseparam - Set Flow Control parameter
+ * @netdev: network interface device structure
+ * @pause: return tx/rx flow control status
+ */
+static int
+ice_set_pauseparam(struct net_device *netdev, struct ethtool_pauseparam *pause)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_link_status *hw_link_info;
+ struct ice_pf *pf = np->vsi->back;
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_hw *hw = &pf->hw;
+ struct ice_port_info *pi;
+ enum ice_status status;
+ u8 aq_failures;
+ bool link_up;
+ int err = 0;
+
+ pi = vsi->port_info;
+ hw_link_info = &pi->phy.link_info;
+ link_up = hw_link_info->link_info & ICE_AQ_LINK_UP;
+
+ /* Changing the port's flow control is not supported if this isn't the
+ * PF VSI
+ */
+ if (vsi->type != ICE_VSI_PF) {
+ netdev_info(netdev, "Changing flow control parameters only supported for PF VSI\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (pause->autoneg != (hw_link_info->an_info & ICE_AQ_AN_COMPLETED)) {
+ netdev_info(netdev, "To change autoneg please use: ethtool -s <dev> autoneg <on|off>\n");
+ return -EOPNOTSUPP;
+ }
+
+ /* If we have link and don't have autoneg */
+ if (!test_bit(__ICE_DOWN, pf->state) &&
+ !(hw_link_info->an_info & ICE_AQ_AN_COMPLETED)) {
+ /* Send message that it might not necessarily work*/
+ netdev_info(netdev, "Autoneg did not complete so changing settings may not result in an actual change.\n");
+ }
+
+ if (pause->rx_pause && pause->tx_pause)
+ pi->fc.req_mode = ICE_FC_FULL;
+ else if (pause->rx_pause && !pause->tx_pause)
+ pi->fc.req_mode = ICE_FC_RX_PAUSE;
+ else if (!pause->rx_pause && pause->tx_pause)
+ pi->fc.req_mode = ICE_FC_TX_PAUSE;
+ else if (!pause->rx_pause && !pause->tx_pause)
+ pi->fc.req_mode = ICE_FC_NONE;
+ else
+ return -EINVAL;
+
+ /* Tell the OS link is going down, the link will go back up when fw
+ * says it is ready asynchronously
+ */
+ ice_print_link_msg(vsi, false);
+ netif_carrier_off(netdev);
+ netif_tx_stop_all_queues(netdev);
+
+ /* Set the FC mode and only restart AN if link is up */
+ status = ice_set_fc(pi, &aq_failures, link_up);
+
+ if (aq_failures & ICE_SET_FC_AQ_FAIL_GET) {
+ netdev_info(netdev, "Set fc failed on the get_phy_capabilities call with err %d aq_err %d\n",
+ status, hw->adminq.sq_last_status);
+ err = -EAGAIN;
+ } else if (aq_failures & ICE_SET_FC_AQ_FAIL_SET) {
+ netdev_info(netdev, "Set fc failed on the set_phy_config call with err %d aq_err %d\n",
+ status, hw->adminq.sq_last_status);
+ err = -EAGAIN;
+ } else if (aq_failures & ICE_SET_FC_AQ_FAIL_UPDATE) {
+ netdev_info(netdev, "Set fc failed on the get_link_info call with err %d aq_err %d\n",
+ status, hw->adminq.sq_last_status);
+ err = -EAGAIN;
+ }
+
+ if (!test_bit(__ICE_DOWN, pf->state)) {
+ /* Give it a little more time to try to come back */
+ msleep(75);
+ if (!test_bit(__ICE_DOWN, pf->state))
+ return ice_nway_reset(netdev);
+ }
+
+ return err;
+}
+
+/**
+ * ice_get_rxfh_key_size - get the RSS hash key size
+ * @netdev: network interface device structure
+ *
+ * Returns the table size.
+ */
+static u32 ice_get_rxfh_key_size(struct net_device __always_unused *netdev)
+{
+ return ICE_VSIQF_HKEY_ARRAY_SIZE;
+}
+
+/**
+ * ice_get_rxfh_indir_size - get the rx flow hash indirection table size
+ * @netdev: network interface device structure
+ *
+ * Returns the table size.
+ */
+static u32 ice_get_rxfh_indir_size(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+
+ return np->vsi->rss_table_size;
+}
+
+/**
+ * ice_get_rxfh - get the rx flow hash indirection table
+ * @netdev: network interface device structure
+ * @indir: indirection table
+ * @key: hash key
+ * @hfunc: hash function
+ *
+ * Reads the indirection table directly from the hardware.
+ */
+static int
+ice_get_rxfh(struct net_device *netdev, u32 *indir, u8 *key, u8 *hfunc)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+ int ret = 0, i;
+ u8 *lut;
+
+ if (hfunc)
+ *hfunc = ETH_RSS_HASH_TOP;
+
+ if (!indir)
+ return 0;
+
+ if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags)) {
+ /* RSS not supported return error here */
+ netdev_warn(netdev, "RSS is not configured on this VSI!\n");
+ return -EIO;
+ }
+
+ lut = devm_kzalloc(&pf->pdev->dev, vsi->rss_table_size, GFP_KERNEL);
+ if (!lut)
+ return -ENOMEM;
+
+ if (ice_get_rss(vsi, key, lut, vsi->rss_table_size)) {
+ ret = -EIO;
+ goto out;
+ }
+
+ for (i = 0; i < vsi->rss_table_size; i++)
+ indir[i] = (u32)(lut[i]);
+
+out:
+ devm_kfree(&pf->pdev->dev, lut);
+ return ret;
+}
+
+/**
+ * ice_set_rxfh - set the rx flow hash indirection table
+ * @netdev: network interface device structure
+ * @indir: indirection table
+ * @key: hash key
+ * @hfunc: hash function
+ *
+ * Returns -EINVAL if the table specifies an invalid queue id, otherwise
+ * returns 0 after programming the table.
+ */
+static int ice_set_rxfh(struct net_device *netdev, const u32 *indir,
+ const u8 *key, const u8 hfunc)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+ u8 *seed = NULL;
+
+ if (hfunc != ETH_RSS_HASH_NO_CHANGE && hfunc != ETH_RSS_HASH_TOP)
+ return -EOPNOTSUPP;
+
+ if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags)) {
+ /* RSS not supported return error here */
+ netdev_warn(netdev, "RSS is not configured on this VSI!\n");
+ return -EIO;
+ }
+
+ if (key) {
+ if (!vsi->rss_hkey_user) {
+ vsi->rss_hkey_user =
+ devm_kzalloc(&pf->pdev->dev,
+ ICE_VSIQF_HKEY_ARRAY_SIZE,
+ GFP_KERNEL);
+ if (!vsi->rss_hkey_user)
+ return -ENOMEM;
+ }
+ memcpy(vsi->rss_hkey_user, key, ICE_VSIQF_HKEY_ARRAY_SIZE);
+ seed = vsi->rss_hkey_user;
+ }
+
+ if (!vsi->rss_lut_user) {
+ vsi->rss_lut_user = devm_kzalloc(&pf->pdev->dev,
+ vsi->rss_table_size,
+ GFP_KERNEL);
+ if (!vsi->rss_lut_user)
+ return -ENOMEM;
+ }
+
+ /* Each 32 bits pointed by 'indir' is stored with a lut entry */
+ if (indir) {
+ int i;
+
+ for (i = 0; i < vsi->rss_table_size; i++)
+ vsi->rss_lut_user[i] = (u8)(indir[i]);
+ } else {
+ ice_fill_rss_lut(vsi->rss_lut_user, vsi->rss_table_size,
+ vsi->rss_size);
+ }
+
+ if (ice_set_rss(vsi, seed, vsi->rss_lut_user, vsi->rss_table_size))
+ return -EIO;
+
+ return 0;
+}
+
+static const struct ethtool_ops ice_ethtool_ops = {
+ .get_link_ksettings = ice_get_link_ksettings,
+ .get_drvinfo = ice_get_drvinfo,
+ .get_regs_len = ice_get_regs_len,
+ .get_regs = ice_get_regs,
+ .get_msglevel = ice_get_msglevel,
+ .set_msglevel = ice_set_msglevel,
+ .get_link = ethtool_op_get_link,
+ .get_strings = ice_get_strings,
+ .get_ethtool_stats = ice_get_ethtool_stats,
+ .get_sset_count = ice_get_sset_count,
+ .get_rxnfc = ice_get_rxnfc,
+ .get_ringparam = ice_get_ringparam,
+ .set_ringparam = ice_set_ringparam,
+ .nway_reset = ice_nway_reset,
+ .get_pauseparam = ice_get_pauseparam,
+ .set_pauseparam = ice_set_pauseparam,
+ .get_rxfh_key_size = ice_get_rxfh_key_size,
+ .get_rxfh_indir_size = ice_get_rxfh_indir_size,
+ .get_rxfh = ice_get_rxfh,
+ .set_rxfh = ice_set_rxfh,
+};
+
+/**
+ * ice_set_ethtool_ops - setup netdev ethtool ops
+ * @netdev: network interface device structure
+ *
+ * setup netdev ethtool ops with ice specific ops
+ */
+void ice_set_ethtool_ops(struct net_device *netdev)
+{
+ netdev->ethtool_ops = &ice_ethtool_ops;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+/* Machine-generated file */
+
+#ifndef _ICE_HW_AUTOGEN_H_
+#define _ICE_HW_AUTOGEN_H_
+
+#define QTX_COMM_DBELL(_DBQM) (0x002C0000 + ((_DBQM) * 4))
+#define PF_FW_ARQBAH 0x00080180
+#define PF_FW_ARQBAL 0x00080080
+#define PF_FW_ARQH 0x00080380
+#define PF_FW_ARQH_ARQH_S 0
+#define PF_FW_ARQH_ARQH_M ICE_M(0x3FF, PF_FW_ARQH_ARQH_S)
+#define PF_FW_ARQLEN 0x00080280
+#define PF_FW_ARQLEN_ARQLEN_S 0
+#define PF_FW_ARQLEN_ARQLEN_M ICE_M(0x3FF, PF_FW_ARQLEN_ARQLEN_S)
+#define PF_FW_ARQLEN_ARQVFE_S 28
+#define PF_FW_ARQLEN_ARQVFE_M BIT(PF_FW_ARQLEN_ARQVFE_S)
+#define PF_FW_ARQLEN_ARQOVFL_S 29
+#define PF_FW_ARQLEN_ARQOVFL_M BIT(PF_FW_ARQLEN_ARQOVFL_S)
+#define PF_FW_ARQLEN_ARQCRIT_S 30
+#define PF_FW_ARQLEN_ARQCRIT_M BIT(PF_FW_ARQLEN_ARQCRIT_S)
+#define PF_FW_ARQLEN_ARQENABLE_S 31
+#define PF_FW_ARQLEN_ARQENABLE_M BIT(PF_FW_ARQLEN_ARQENABLE_S)
+#define PF_FW_ARQT 0x00080480
+#define PF_FW_ATQBAH 0x00080100
+#define PF_FW_ATQBAL 0x00080000
+#define PF_FW_ATQH 0x00080300
+#define PF_FW_ATQH_ATQH_S 0
+#define PF_FW_ATQH_ATQH_M ICE_M(0x3FF, PF_FW_ATQH_ATQH_S)
+#define PF_FW_ATQLEN 0x00080200
+#define PF_FW_ATQLEN_ATQLEN_S 0
+#define PF_FW_ATQLEN_ATQLEN_M ICE_M(0x3FF, PF_FW_ATQLEN_ATQLEN_S)
+#define PF_FW_ATQLEN_ATQVFE_S 28
+#define PF_FW_ATQLEN_ATQVFE_M BIT(PF_FW_ATQLEN_ATQVFE_S)
+#define PF_FW_ATQLEN_ATQOVFL_S 29
+#define PF_FW_ATQLEN_ATQOVFL_M BIT(PF_FW_ATQLEN_ATQOVFL_S)
+#define PF_FW_ATQLEN_ATQCRIT_S 30
+#define PF_FW_ATQLEN_ATQCRIT_M BIT(PF_FW_ATQLEN_ATQCRIT_S)
+#define PF_FW_ATQLEN_ATQENABLE_S 31
+#define PF_FW_ATQLEN_ATQENABLE_M BIT(PF_FW_ATQLEN_ATQENABLE_S)
+#define PF_FW_ATQT 0x00080400
+
+#define GLFLXP_RXDID_FLAGS(_i, _j) (0x0045D000 + ((_i) * 4 + (_j) * 256))
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S 0
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M ICE_M(0x3F, GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S)
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_1_S 8
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_1_M ICE_M(0x3F, GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_1_S)
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_2_S 16
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_2_M ICE_M(0x3F, GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_2_S)
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_3_S 24
+#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_3_M ICE_M(0x3F, GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_3_S)
+#define GLFLXP_RXDID_FLX_WRD_0(_i) (0x0045c800 + ((_i) * 4))
+#define GLFLXP_RXDID_FLX_WRD_0_PROT_MDID_S 0
+#define GLFLXP_RXDID_FLX_WRD_0_PROT_MDID_M ICE_M(0xFF, GLFLXP_RXDID_FLX_WRD_0_PROT_MDID_S)
+#define GLFLXP_RXDID_FLX_WRD_0_RXDID_OPCODE_S 30
+#define GLFLXP_RXDID_FLX_WRD_0_RXDID_OPCODE_M ICE_M(0x3, GLFLXP_RXDID_FLX_WRD_0_RXDID_OPCODE_S)
+#define GLFLXP_RXDID_FLX_WRD_1(_i) (0x0045c900 + ((_i) * 4))
+#define GLFLXP_RXDID_FLX_WRD_1_PROT_MDID_S 0
+#define GLFLXP_RXDID_FLX_WRD_1_PROT_MDID_M ICE_M(0xFF, GLFLXP_RXDID_FLX_WRD_1_PROT_MDID_S)
+#define GLFLXP_RXDID_FLX_WRD_1_RXDID_OPCODE_S 30
+#define GLFLXP_RXDID_FLX_WRD_1_RXDID_OPCODE_M ICE_M(0x3, GLFLXP_RXDID_FLX_WRD_1_RXDID_OPCODE_S)
+#define GLFLXP_RXDID_FLX_WRD_2(_i) (0x0045ca00 + ((_i) * 4))
+#define GLFLXP_RXDID_FLX_WRD_2_PROT_MDID_S 0
+#define GLFLXP_RXDID_FLX_WRD_2_PROT_MDID_M ICE_M(0xFF, GLFLXP_RXDID_FLX_WRD_2_PROT_MDID_S)
+#define GLFLXP_RXDID_FLX_WRD_2_RXDID_OPCODE_S 30
+#define GLFLXP_RXDID_FLX_WRD_2_RXDID_OPCODE_M ICE_M(0x3, GLFLXP_RXDID_FLX_WRD_2_RXDID_OPCODE_S)
+#define GLFLXP_RXDID_FLX_WRD_3(_i) (0x0045cb00 + ((_i) * 4))
+#define GLFLXP_RXDID_FLX_WRD_3_PROT_MDID_S 0
+#define GLFLXP_RXDID_FLX_WRD_3_PROT_MDID_M ICE_M(0xFF, GLFLXP_RXDID_FLX_WRD_3_PROT_MDID_S)
+#define GLFLXP_RXDID_FLX_WRD_3_RXDID_OPCODE_S 30
+#define GLFLXP_RXDID_FLX_WRD_3_RXDID_OPCODE_M ICE_M(0x3, GLFLXP_RXDID_FLX_WRD_3_RXDID_OPCODE_S)
+
+#define QRXFLXP_CNTXT(_QRX) (0x00480000 + ((_QRX) * 4))
+#define QRXFLXP_CNTXT_RXDID_IDX_S 0
+#define QRXFLXP_CNTXT_RXDID_IDX_M ICE_M(0x3F, QRXFLXP_CNTXT_RXDID_IDX_S)
+#define QRXFLXP_CNTXT_RXDID_PRIO_S 8
+#define QRXFLXP_CNTXT_RXDID_PRIO_M ICE_M(0x7, QRXFLXP_CNTXT_RXDID_PRIO_S)
+#define QRXFLXP_CNTXT_TS_S 11
+#define QRXFLXP_CNTXT_TS_M BIT(QRXFLXP_CNTXT_TS_S)
+#define GLGEN_RSTAT 0x000B8188
+#define GLGEN_RSTAT_DEVSTATE_S 0
+#define GLGEN_RSTAT_DEVSTATE_M ICE_M(0x3, GLGEN_RSTAT_DEVSTATE_S)
+#define GLGEN_RSTCTL 0x000B8180
+#define GLGEN_RSTCTL_GRSTDEL_S 0
+#define GLGEN_RSTCTL_GRSTDEL_M ICE_M(0x3F, GLGEN_RSTCTL_GRSTDEL_S)
+#define GLGEN_RSTAT_RESET_TYPE_S 2
+#define GLGEN_RSTAT_RESET_TYPE_M ICE_M(0x3, GLGEN_RSTAT_RESET_TYPE_S)
+#define GLGEN_RTRIG 0x000B8190
+#define GLGEN_RTRIG_CORER_S 0
+#define GLGEN_RTRIG_CORER_M BIT(GLGEN_RTRIG_CORER_S)
+#define GLGEN_RTRIG_GLOBR_S 1
+#define GLGEN_RTRIG_GLOBR_M BIT(GLGEN_RTRIG_GLOBR_S)
+#define GLGEN_STAT 0x000B612C
+#define PFGEN_CTRL 0x00091000
+#define PFGEN_CTRL_PFSWR_S 0
+#define PFGEN_CTRL_PFSWR_M BIT(PFGEN_CTRL_PFSWR_S)
+#define PFGEN_STATE 0x00088000
+#define PRTGEN_STATUS 0x000B8100
+#define PFHMC_ERRORDATA 0x00520500
+#define PFHMC_ERRORINFO 0x00520400
+#define GLINT_DYN_CTL(_INT) (0x00160000 + ((_INT) * 4))
+#define GLINT_DYN_CTL_INTENA_S 0
+#define GLINT_DYN_CTL_INTENA_M BIT(GLINT_DYN_CTL_INTENA_S)
+#define GLINT_DYN_CTL_CLEARPBA_S 1
+#define GLINT_DYN_CTL_CLEARPBA_M BIT(GLINT_DYN_CTL_CLEARPBA_S)
+#define GLINT_DYN_CTL_SWINT_TRIG_S 2
+#define GLINT_DYN_CTL_SWINT_TRIG_M BIT(GLINT_DYN_CTL_SWINT_TRIG_S)
+#define GLINT_DYN_CTL_ITR_INDX_S 3
+#define GLINT_DYN_CTL_SW_ITR_INDX_S 25
+#define GLINT_DYN_CTL_SW_ITR_INDX_M ICE_M(0x3, GLINT_DYN_CTL_SW_ITR_INDX_S)
+#define GLINT_DYN_CTL_INTENA_MSK_S 31
+#define GLINT_DYN_CTL_INTENA_MSK_M BIT(GLINT_DYN_CTL_INTENA_MSK_S)
+#define GLINT_ITR(_i, _INT) (0x00154000 + ((_i) * 8192 + (_INT) * 4))
+#define PFINT_FW_CTL 0x0016C800
+#define PFINT_FW_CTL_MSIX_INDX_S 0
+#define PFINT_FW_CTL_MSIX_INDX_M ICE_M(0x7FF, PFINT_FW_CTL_MSIX_INDX_S)
+#define PFINT_FW_CTL_ITR_INDX_S 11
+#define PFINT_FW_CTL_ITR_INDX_M ICE_M(0x3, PFINT_FW_CTL_ITR_INDX_S)
+#define PFINT_FW_CTL_CAUSE_ENA_S 30
+#define PFINT_FW_CTL_CAUSE_ENA_M BIT(PFINT_FW_CTL_CAUSE_ENA_S)
+#define PFINT_OICR 0x0016CA00
+#define PFINT_OICR_INTEVENT_S 0
+#define PFINT_OICR_INTEVENT_M BIT(PFINT_OICR_INTEVENT_S)
+#define PFINT_OICR_HLP_RDY_S 14
+#define PFINT_OICR_HLP_RDY_M BIT(PFINT_OICR_HLP_RDY_S)
+#define PFINT_OICR_CPM_RDY_S 15
+#define PFINT_OICR_CPM_RDY_M BIT(PFINT_OICR_CPM_RDY_S)
+#define PFINT_OICR_ECC_ERR_S 16
+#define PFINT_OICR_ECC_ERR_M BIT(PFINT_OICR_ECC_ERR_S)
+#define PFINT_OICR_MAL_DETECT_S 19
+#define PFINT_OICR_MAL_DETECT_M BIT(PFINT_OICR_MAL_DETECT_S)
+#define PFINT_OICR_GRST_S 20
+#define PFINT_OICR_GRST_M BIT(PFINT_OICR_GRST_S)
+#define PFINT_OICR_PCI_EXCEPTION_S 21
+#define PFINT_OICR_PCI_EXCEPTION_M BIT(PFINT_OICR_PCI_EXCEPTION_S)
+#define PFINT_OICR_GPIO_S 22
+#define PFINT_OICR_GPIO_M BIT(PFINT_OICR_GPIO_S)
+#define PFINT_OICR_STORM_DETECT_S 24
+#define PFINT_OICR_STORM_DETECT_M BIT(PFINT_OICR_STORM_DETECT_S)
+#define PFINT_OICR_HMC_ERR_S 26
+#define PFINT_OICR_HMC_ERR_M BIT(PFINT_OICR_HMC_ERR_S)
+#define PFINT_OICR_PE_CRITERR_S 28
+#define PFINT_OICR_PE_CRITERR_M BIT(PFINT_OICR_PE_CRITERR_S)
+#define PFINT_OICR_CTL 0x0016CA80
+#define PFINT_OICR_CTL_MSIX_INDX_S 0
+#define PFINT_OICR_CTL_MSIX_INDX_M ICE_M(0x7FF, PFINT_OICR_CTL_MSIX_INDX_S)
+#define PFINT_OICR_CTL_ITR_INDX_S 11
+#define PFINT_OICR_CTL_ITR_INDX_M ICE_M(0x3, PFINT_OICR_CTL_ITR_INDX_S)
+#define PFINT_OICR_CTL_CAUSE_ENA_S 30
+#define PFINT_OICR_CTL_CAUSE_ENA_M BIT(PFINT_OICR_CTL_CAUSE_ENA_S)
+#define PFINT_OICR_ENA 0x0016C900
+#define QINT_RQCTL(_QRX) (0x00150000 + ((_QRX) * 4))
+#define QINT_RQCTL_MSIX_INDX_S 0
+#define QINT_RQCTL_ITR_INDX_S 11
+#define QINT_RQCTL_CAUSE_ENA_S 30
+#define QINT_RQCTL_CAUSE_ENA_M BIT(QINT_RQCTL_CAUSE_ENA_S)
+#define QINT_TQCTL(_DBQM) (0x00140000 + ((_DBQM) * 4))
+#define QINT_TQCTL_MSIX_INDX_S 0
+#define QINT_TQCTL_ITR_INDX_S 11
+#define QINT_TQCTL_CAUSE_ENA_S 30
+#define QINT_TQCTL_CAUSE_ENA_M BIT(QINT_TQCTL_CAUSE_ENA_S)
+#define GLLAN_RCTL_0 0x002941F8
+#define QRX_CONTEXT(_i, _QRX) (0x00280000 + ((_i) * 8192 + (_QRX) * 4))
+#define QRX_CTRL(_QRX) (0x00120000 + ((_QRX) * 4))
+#define QRX_CTRL_MAX_INDEX 2047
+#define QRX_CTRL_QENA_REQ_S 0
+#define QRX_CTRL_QENA_REQ_M BIT(QRX_CTRL_QENA_REQ_S)
+#define QRX_CTRL_QENA_STAT_S 2
+#define QRX_CTRL_QENA_STAT_M BIT(QRX_CTRL_QENA_STAT_S)
+#define QRX_ITR(_QRX) (0x00292000 + ((_QRX) * 4))
+#define QRX_TAIL(_QRX) (0x00290000 + ((_QRX) * 4))
+#define GLNVM_FLA 0x000B6108
+#define GLNVM_FLA_LOCKED_S 6
+#define GLNVM_FLA_LOCKED_M BIT(GLNVM_FLA_LOCKED_S)
+#define GLNVM_GENS 0x000B6100
+#define GLNVM_GENS_SR_SIZE_S 5
+#define GLNVM_GENS_SR_SIZE_M ICE_M(0x7, GLNVM_GENS_SR_SIZE_S)
+#define GLNVM_ULD 0x000B6008
+#define GLNVM_ULD_CORER_DONE_S 3
+#define GLNVM_ULD_CORER_DONE_M BIT(GLNVM_ULD_CORER_DONE_S)
+#define GLNVM_ULD_GLOBR_DONE_S 4
+#define GLNVM_ULD_GLOBR_DONE_M BIT(GLNVM_ULD_GLOBR_DONE_S)
+#define PF_FUNC_RID 0x0009E880
+#define PF_FUNC_RID_FUNC_NUM_S 0
+#define PF_FUNC_RID_FUNC_NUM_M ICE_M(0x7, PF_FUNC_RID_FUNC_NUM_S)
+#define GLPRT_BPRCH(_i) (0x00381384 + ((_i) * 8))
+#define GLPRT_BPRCL(_i) (0x00381380 + ((_i) * 8))
+#define GLPRT_BPTCH(_i) (0x00381244 + ((_i) * 8))
+#define GLPRT_BPTCL(_i) (0x00381240 + ((_i) * 8))
+#define GLPRT_CRCERRS(_i) (0x00380100 + ((_i) * 8))
+#define GLPRT_GORCH(_i) (0x00380004 + ((_i) * 8))
+#define GLPRT_GORCL(_i) (0x00380000 + ((_i) * 8))
+#define GLPRT_GOTCH(_i) (0x00380B44 + ((_i) * 8))
+#define GLPRT_GOTCL(_i) (0x00380B40 + ((_i) * 8))
+#define GLPRT_ILLERRC(_i) (0x003801C0 + ((_i) * 8))
+#define GLPRT_LXOFFRXC(_i) (0x003802C0 + ((_i) * 8))
+#define GLPRT_LXOFFTXC(_i) (0x00381180 + ((_i) * 8))
+#define GLPRT_LXONRXC(_i) (0x00380280 + ((_i) * 8))
+#define GLPRT_LXONTXC(_i) (0x00381140 + ((_i) * 8))
+#define GLPRT_MLFC(_i) (0x00380040 + ((_i) * 8))
+#define GLPRT_MPRCH(_i) (0x00381344 + ((_i) * 8))
+#define GLPRT_MPRCL(_i) (0x00381340 + ((_i) * 8))
+#define GLPRT_MPTCH(_i) (0x00381204 + ((_i) * 8))
+#define GLPRT_MPTCL(_i) (0x00381200 + ((_i) * 8))
+#define GLPRT_MRFC(_i) (0x00380080 + ((_i) * 8))
+#define GLPRT_PRC1023H(_i) (0x00380A04 + ((_i) * 8))
+#define GLPRT_PRC1023L(_i) (0x00380A00 + ((_i) * 8))
+#define GLPRT_PRC127H(_i) (0x00380944 + ((_i) * 8))
+#define GLPRT_PRC127L(_i) (0x00380940 + ((_i) * 8))
+#define GLPRT_PRC1522H(_i) (0x00380A44 + ((_i) * 8))
+#define GLPRT_PRC1522L(_i) (0x00380A40 + ((_i) * 8))
+#define GLPRT_PRC255H(_i) (0x00380984 + ((_i) * 8))
+#define GLPRT_PRC255L(_i) (0x00380980 + ((_i) * 8))
+#define GLPRT_PRC511H(_i) (0x003809C4 + ((_i) * 8))
+#define GLPRT_PRC511L(_i) (0x003809C0 + ((_i) * 8))
+#define GLPRT_PRC64H(_i) (0x00380904 + ((_i) * 8))
+#define GLPRT_PRC64L(_i) (0x00380900 + ((_i) * 8))
+#define GLPRT_PRC9522H(_i) (0x00380A84 + ((_i) * 8))
+#define GLPRT_PRC9522L(_i) (0x00380A80 + ((_i) * 8))
+#define GLPRT_PTC1023H(_i) (0x00380C84 + ((_i) * 8))
+#define GLPRT_PTC1023L(_i) (0x00380C80 + ((_i) * 8))
+#define GLPRT_PTC127H(_i) (0x00380BC4 + ((_i) * 8))
+#define GLPRT_PTC127L(_i) (0x00380BC0 + ((_i) * 8))
+#define GLPRT_PTC1522H(_i) (0x00380CC4 + ((_i) * 8))
+#define GLPRT_PTC1522L(_i) (0x00380CC0 + ((_i) * 8))
+#define GLPRT_PTC255H(_i) (0x00380C04 + ((_i) * 8))
+#define GLPRT_PTC255L(_i) (0x00380C00 + ((_i) * 8))
+#define GLPRT_PTC511H(_i) (0x00380C44 + ((_i) * 8))
+#define GLPRT_PTC511L(_i) (0x00380C40 + ((_i) * 8))
+#define GLPRT_PTC64H(_i) (0x00380B84 + ((_i) * 8))
+#define GLPRT_PTC64L(_i) (0x00380B80 + ((_i) * 8))
+#define GLPRT_PTC9522H(_i) (0x00380D04 + ((_i) * 8))
+#define GLPRT_PTC9522L(_i) (0x00380D00 + ((_i) * 8))
+#define GLPRT_RFC(_i) (0x00380AC0 + ((_i) * 8))
+#define GLPRT_RJC(_i) (0x00380B00 + ((_i) * 8))
+#define GLPRT_RLEC(_i) (0x00380140 + ((_i) * 8))
+#define GLPRT_ROC(_i) (0x00380240 + ((_i) * 8))
+#define GLPRT_RUC(_i) (0x00380200 + ((_i) * 8))
+#define GLPRT_TDOLD(_i) (0x00381280 + ((_i) * 8))
+#define GLPRT_UPRCH(_i) (0x00381304 + ((_i) * 8))
+#define GLPRT_UPRCL(_i) (0x00381300 + ((_i) * 8))
+#define GLPRT_UPTCH(_i) (0x003811C4 + ((_i) * 8))
+#define GLPRT_UPTCL(_i) (0x003811C0 + ((_i) * 8))
+#define GLV_BPRCH(_i) (0x003B6004 + ((_i) * 8))
+#define GLV_BPRCL(_i) (0x003B6000 + ((_i) * 8))
+#define GLV_BPTCH(_i) (0x0030E004 + ((_i) * 8))
+#define GLV_BPTCL(_i) (0x0030E000 + ((_i) * 8))
+#define GLV_GORCH(_i) (0x003B0004 + ((_i) * 8))
+#define GLV_GORCL(_i) (0x003B0000 + ((_i) * 8))
+#define GLV_GOTCH(_i) (0x00300004 + ((_i) * 8))
+#define GLV_GOTCL(_i) (0x00300000 + ((_i) * 8))
+#define GLV_MPRCH(_i) (0x003B4004 + ((_i) * 8))
+#define GLV_MPRCL(_i) (0x003B4000 + ((_i) * 8))
+#define GLV_MPTCH(_i) (0x0030C004 + ((_i) * 8))
+#define GLV_MPTCL(_i) (0x0030C000 + ((_i) * 8))
+#define GLV_RDPC(_i) (0x00294C04 + ((_i) * 4))
+#define GLV_TEPC(_VSI) (0x00312000 + ((_VSI) * 4))
+#define GLV_UPRCH(_i) (0x003B2004 + ((_i) * 8))
+#define GLV_UPRCL(_i) (0x003B2000 + ((_i) * 8))
+#define GLV_UPTCH(_i) (0x0030A004 + ((_i) * 8))
+#define GLV_UPTCL(_i) (0x0030A000 + ((_i) * 8))
+#define VSIQF_HKEY_MAX_INDEX 12
+
+#endif /* _ICE_HW_AUTOGEN_H_ */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_LAN_TX_RX_H_
+#define _ICE_LAN_TX_RX_H_
+
+union ice_32byte_rx_desc {
+ struct {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ /* bit 0 of hdr_addr is DD bit */
+ __le64 rsvd1;
+ __le64 rsvd2;
+ } read;
+ struct {
+ struct {
+ struct {
+ __le16 mirroring_status;
+ __le16 l2tag1;
+ } lo_dword;
+ union {
+ __le32 rss; /* RSS Hash */
+ __le32 fd_id; /* Flow Director filter id */
+ } hi_dword;
+ } qword0;
+ struct {
+ /* status/error/PTYPE/length */
+ __le64 status_error_len;
+ } qword1;
+ struct {
+ __le16 ext_status; /* extended status */
+ __le16 rsvd;
+ __le16 l2tag2_1;
+ __le16 l2tag2_2;
+ } qword2;
+ struct {
+ __le32 reserved;
+ __le32 fd_id;
+ } qword3;
+ } wb; /* writeback */
+};
+
+struct ice_rx_ptype_decoded {
+ u32 ptype:10;
+ u32 known:1;
+ u32 outer_ip:1;
+ u32 outer_ip_ver:2;
+ u32 outer_frag:1;
+ u32 tunnel_type:3;
+ u32 tunnel_end_prot:2;
+ u32 tunnel_end_frag:1;
+ u32 inner_prot:4;
+ u32 payload_layer:3;
+};
+
+enum ice_rx_ptype_outer_ip {
+ ICE_RX_PTYPE_OUTER_L2 = 0,
+ ICE_RX_PTYPE_OUTER_IP = 1,
+};
+
+enum ice_rx_ptype_outer_ip_ver {
+ ICE_RX_PTYPE_OUTER_NONE = 0,
+ ICE_RX_PTYPE_OUTER_IPV4 = 1,
+ ICE_RX_PTYPE_OUTER_IPV6 = 2,
+};
+
+enum ice_rx_ptype_outer_fragmented {
+ ICE_RX_PTYPE_NOT_FRAG = 0,
+ ICE_RX_PTYPE_FRAG = 1,
+};
+
+enum ice_rx_ptype_tunnel_type {
+ ICE_RX_PTYPE_TUNNEL_NONE = 0,
+ ICE_RX_PTYPE_TUNNEL_IP_IP = 1,
+ ICE_RX_PTYPE_TUNNEL_IP_GRENAT = 2,
+ ICE_RX_PTYPE_TUNNEL_IP_GRENAT_MAC = 3,
+ ICE_RX_PTYPE_TUNNEL_IP_GRENAT_MAC_VLAN = 4,
+};
+
+enum ice_rx_ptype_tunnel_end_prot {
+ ICE_RX_PTYPE_TUNNEL_END_NONE = 0,
+ ICE_RX_PTYPE_TUNNEL_END_IPV4 = 1,
+ ICE_RX_PTYPE_TUNNEL_END_IPV6 = 2,
+};
+
+enum ice_rx_ptype_inner_prot {
+ ICE_RX_PTYPE_INNER_PROT_NONE = 0,
+ ICE_RX_PTYPE_INNER_PROT_UDP = 1,
+ ICE_RX_PTYPE_INNER_PROT_TCP = 2,
+ ICE_RX_PTYPE_INNER_PROT_SCTP = 3,
+ ICE_RX_PTYPE_INNER_PROT_ICMP = 4,
+ ICE_RX_PTYPE_INNER_PROT_TIMESYNC = 5,
+};
+
+enum ice_rx_ptype_payload_layer {
+ ICE_RX_PTYPE_PAYLOAD_LAYER_NONE = 0,
+ ICE_RX_PTYPE_PAYLOAD_LAYER_PAY2 = 1,
+ ICE_RX_PTYPE_PAYLOAD_LAYER_PAY3 = 2,
+ ICE_RX_PTYPE_PAYLOAD_LAYER_PAY4 = 3,
+};
+
+/* RX Flex Descriptor
+ * This descriptor is used instead of the legacy version descriptor when
+ * ice_rlan_ctx.adv_desc is set
+ */
+union ice_32b_rx_flex_desc {
+ struct {
+ __le64 pkt_addr; /* Packet buffer address */
+ __le64 hdr_addr; /* Header buffer address */
+ /* bit 0 of hdr_addr is DD bit */
+ __le64 rsvd1;
+ __le64 rsvd2;
+ } read;
+ struct {
+ /* Qword 0 */
+ u8 rxdid; /* descriptor builder profile id */
+ u8 mir_id_umb_cast; /* mirror=[5:0], umb=[7:6] */
+ __le16 ptype_flex_flags0; /* ptype=[9:0], ff0=[15:10] */
+ __le16 pkt_len; /* [15:14] are reserved */
+ __le16 hdr_len_sph_flex_flags1; /* header=[10:0] */
+ /* sph=[11:11] */
+ /* ff1/ext=[15:12] */
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le16 flex_meta0;
+ __le16 flex_meta1;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flex_flags2;
+ u8 time_stamp_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le16 flex_meta2;
+ __le16 flex_meta3;
+ union {
+ struct {
+ __le16 flex_meta4;
+ __le16 flex_meta5;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+ } wb; /* writeback */
+};
+
+/* Rx Flex Descriptor NIC Profile
+ * This descriptor corresponds to RxDID 2 which contains
+ * metadata fields for RSS, flow id and timestamp info
+ */
+struct ice_32b_rx_flex_desc_nic {
+ /* Qword 0 */
+ u8 rxdid;
+ u8 mir_id_umb_cast;
+ __le16 ptype_flexi_flags0;
+ __le16 pkt_len;
+ __le16 hdr_len_sph_flex_flags1;
+
+ /* Qword 1 */
+ __le16 status_error0;
+ __le16 l2tag1;
+ __le32 rss_hash;
+
+ /* Qword 2 */
+ __le16 status_error1;
+ u8 flexi_flags2;
+ u8 ts_low;
+ __le16 l2tag2_1st;
+ __le16 l2tag2_2nd;
+
+ /* Qword 3 */
+ __le32 flow_id;
+ union {
+ struct {
+ __le16 vlan_id;
+ __le16 flow_id_ipv6;
+ } flex;
+ __le32 ts_high;
+ } flex_ts;
+};
+
+/* Receive Flex Descriptor profile IDs: There are a total
+ * of 64 profiles where profile IDs 0/1 are for legacy; and
+ * profiles 2-63 are flex profiles that can be programmed
+ * with a specific metadata (profile 7 reserved for HW)
+ */
+enum ice_rxdid {
+ ICE_RXDID_START = 0,
+ ICE_RXDID_LEGACY_0 = ICE_RXDID_START,
+ ICE_RXDID_LEGACY_1,
+ ICE_RXDID_FLX_START,
+ ICE_RXDID_FLEX_NIC = ICE_RXDID_FLX_START,
+ ICE_RXDID_FLX_LAST = 63,
+ ICE_RXDID_LAST = ICE_RXDID_FLX_LAST
+};
+
+/* Receive Flex Descriptor Rx opcode values */
+#define ICE_RX_OPC_MDID 0x01
+
+/* Receive Descriptor MDID values */
+#define ICE_RX_MDID_FLOW_ID_LOWER 5
+#define ICE_RX_MDID_FLOW_ID_HIGH 6
+#define ICE_RX_MDID_HASH_LOW 56
+#define ICE_RX_MDID_HASH_HIGH 57
+
+/* Rx Flag64 packet flag bits */
+enum ice_rx_flg64_bits {
+ ICE_RXFLG_PKT_DSI = 0,
+ ICE_RXFLG_EVLAN_x8100 = 15,
+ ICE_RXFLG_EVLAN_x9100,
+ ICE_RXFLG_VLAN_x8100,
+ ICE_RXFLG_TNL_MAC = 22,
+ ICE_RXFLG_TNL_VLAN,
+ ICE_RXFLG_PKT_FRG,
+ ICE_RXFLG_FIN = 32,
+ ICE_RXFLG_SYN,
+ ICE_RXFLG_RST,
+ ICE_RXFLG_TNL0 = 38,
+ ICE_RXFLG_TNL1,
+ ICE_RXFLG_TNL2,
+ ICE_RXFLG_UDP_GRE,
+ ICE_RXFLG_RSVD = 63
+};
+
+/* for ice_32byte_rx_flex_desc.ptype_flexi_flags0 member */
+#define ICE_RX_FLEX_DESC_PTYPE_M (0x3FF) /* 10-bits */
+
+/* for ice_32byte_rx_flex_desc.pkt_length member */
+#define ICE_RX_FLX_DESC_PKT_LEN_M (0x3FFF) /* 14-bits */
+
+enum ice_rx_flex_desc_status_error_0_bits {
+ /* Note: These are predefined bit offsets */
+ ICE_RX_FLEX_DESC_STATUS0_DD_S = 0,
+ ICE_RX_FLEX_DESC_STATUS0_EOF_S,
+ ICE_RX_FLEX_DESC_STATUS0_HBO_S,
+ ICE_RX_FLEX_DESC_STATUS0_L3L4P_S,
+ ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S,
+ ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S,
+ ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S,
+ ICE_RX_FLEX_DESC_STATUS0_XSUM_EUDPE_S,
+ ICE_RX_FLEX_DESC_STATUS0_LPBK_S,
+ ICE_RX_FLEX_DESC_STATUS0_IPV6EXADD_S,
+ ICE_RX_FLEX_DESC_STATUS0_RXE_S,
+ ICE_RX_FLEX_DESC_STATUS0_CRCP_S,
+ ICE_RX_FLEX_DESC_STATUS0_RSS_VALID_S,
+ ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S,
+ ICE_RX_FLEX_DESC_STATUS0_XTRMD0_VALID_S,
+ ICE_RX_FLEX_DESC_STATUS0_XTRMD1_VALID_S,
+ ICE_RX_FLEX_DESC_STATUS0_LAST /* this entry must be last!!! */
+};
+
+#define ICE_RXQ_CTX_SIZE_DWORDS 8
+#define ICE_RXQ_CTX_SZ (ICE_RXQ_CTX_SIZE_DWORDS * sizeof(u32))
+
+/* RLAN Rx queue context data
+ *
+ * The sizes of the variables may be larger than needed due to crossing byte
+ * boundaries. If we do not have the width of the variable set to the correct
+ * size then we could end up shifting bits off the top of the variable when the
+ * variable is at the top of a byte and crosses over into the next byte.
+ */
+struct ice_rlan_ctx {
+ u16 head;
+ u16 cpuid; /* bigger than needed, see above for reason */
+ u64 base;
+ u16 qlen;
+#define ICE_RLAN_CTX_DBUF_S 7
+ u16 dbuf; /* bigger than needed, see above for reason */
+#define ICE_RLAN_CTX_HBUF_S 6
+ u16 hbuf; /* bigger than needed, see above for reason */
+ u8 dtype;
+ u8 dsize;
+ u8 crcstrip;
+ u8 l2tsel;
+ u8 hsplit_0;
+ u8 hsplit_1;
+ u8 showiv;
+ u32 rxmax; /* bigger than needed, see above for reason */
+ u8 tphrdesc_ena;
+ u8 tphwdesc_ena;
+ u8 tphdata_ena;
+ u8 tphhead_ena;
+ u16 lrxqthresh; /* bigger than needed, see above for reason */
+};
+
+struct ice_ctx_ele {
+ u16 offset;
+ u16 size_of;
+ u16 width;
+ u16 lsb;
+};
+
+#define ICE_CTX_STORE(_struct, _ele, _width, _lsb) { \
+ .offset = offsetof(struct _struct, _ele), \
+ .size_of = FIELD_SIZEOF(struct _struct, _ele), \
+ .width = _width, \
+ .lsb = _lsb, \
+}
+
+/* for hsplit_0 field of Rx RLAN context */
+enum ice_rlan_ctx_rx_hsplit_0 {
+ ICE_RLAN_RX_HSPLIT_0_NO_SPLIT = 0,
+ ICE_RLAN_RX_HSPLIT_0_SPLIT_L2 = 1,
+ ICE_RLAN_RX_HSPLIT_0_SPLIT_IP = 2,
+ ICE_RLAN_RX_HSPLIT_0_SPLIT_TCP_UDP = 4,
+ ICE_RLAN_RX_HSPLIT_0_SPLIT_SCTP = 8,
+};
+
+/* for hsplit_1 field of Rx RLAN context */
+enum ice_rlan_ctx_rx_hsplit_1 {
+ ICE_RLAN_RX_HSPLIT_1_NO_SPLIT = 0,
+ ICE_RLAN_RX_HSPLIT_1_SPLIT_L2 = 1,
+ ICE_RLAN_RX_HSPLIT_1_SPLIT_ALWAYS = 2,
+};
+
+/* TX Descriptor */
+struct ice_tx_desc {
+ __le64 buf_addr; /* Address of descriptor's data buf */
+ __le64 cmd_type_offset_bsz;
+};
+
+enum ice_tx_desc_dtype_value {
+ ICE_TX_DESC_DTYPE_DATA = 0x0,
+ ICE_TX_DESC_DTYPE_CTX = 0x1,
+ /* DESC_DONE - HW has completed write-back of descriptor */
+ ICE_TX_DESC_DTYPE_DESC_DONE = 0xF,
+};
+
+#define ICE_TXD_QW1_CMD_S 4
+#define ICE_TXD_QW1_CMD_M (0xFFFUL << ICE_TXD_QW1_CMD_S)
+
+enum ice_tx_desc_cmd_bits {
+ ICE_TX_DESC_CMD_EOP = 0x0001,
+ ICE_TX_DESC_CMD_RS = 0x0002,
+ ICE_TX_DESC_CMD_IL2TAG1 = 0x0008,
+ ICE_TX_DESC_CMD_IIPT_IPV6 = 0x0020, /* 2 BITS */
+ ICE_TX_DESC_CMD_IIPT_IPV4 = 0x0040, /* 2 BITS */
+ ICE_TX_DESC_CMD_IIPT_IPV4_CSUM = 0x0060, /* 2 BITS */
+ ICE_TX_DESC_CMD_L4T_EOFT_TCP = 0x0100, /* 2 BITS */
+ ICE_TX_DESC_CMD_L4T_EOFT_UDP = 0x0300, /* 2 BITS */
+};
+
+#define ICE_TXD_QW1_OFFSET_S 16
+#define ICE_TXD_QW1_OFFSET_M (0x3FFFFULL << ICE_TXD_QW1_OFFSET_S)
+
+enum ice_tx_desc_len_fields {
+ /* Note: These are predefined bit offsets */
+ ICE_TX_DESC_LEN_MACLEN_S = 0, /* 7 BITS */
+ ICE_TX_DESC_LEN_IPLEN_S = 7, /* 7 BITS */
+ ICE_TX_DESC_LEN_L4_LEN_S = 14 /* 4 BITS */
+};
+
+#define ICE_TXD_QW1_MACLEN_M (0x7FUL << ICE_TX_DESC_LEN_MACLEN_S)
+#define ICE_TXD_QW1_IPLEN_M (0x7FUL << ICE_TX_DESC_LEN_IPLEN_S)
+#define ICE_TXD_QW1_L4LEN_M (0xFUL << ICE_TX_DESC_LEN_L4_LEN_S)
+
+/* Tx descriptor field limits in bytes */
+#define ICE_TXD_MACLEN_MAX ((ICE_TXD_QW1_MACLEN_M >> \
+ ICE_TX_DESC_LEN_MACLEN_S) * ICE_BYTES_PER_WORD)
+#define ICE_TXD_IPLEN_MAX ((ICE_TXD_QW1_IPLEN_M >> \
+ ICE_TX_DESC_LEN_IPLEN_S) * ICE_BYTES_PER_DWORD)
+#define ICE_TXD_L4LEN_MAX ((ICE_TXD_QW1_L4LEN_M >> \
+ ICE_TX_DESC_LEN_L4_LEN_S) * ICE_BYTES_PER_DWORD)
+
+#define ICE_TXD_QW1_TX_BUF_SZ_S 34
+#define ICE_TXD_QW1_L2TAG1_S 48
+
+/* Context descriptors */
+struct ice_tx_ctx_desc {
+ __le32 tunneling_params;
+ __le16 l2tag2;
+ __le16 rsvd;
+ __le64 qw1;
+};
+
+#define ICE_TXD_CTX_QW1_CMD_S 4
+#define ICE_TXD_CTX_QW1_CMD_M (0x7FUL << ICE_TXD_CTX_QW1_CMD_S)
+
+#define ICE_TXD_CTX_QW1_TSO_LEN_S 30
+#define ICE_TXD_CTX_QW1_TSO_LEN_M \
+ (0x3FFFFULL << ICE_TXD_CTX_QW1_TSO_LEN_S)
+
+#define ICE_TXD_CTX_QW1_MSS_S 50
+
+enum ice_tx_ctx_desc_cmd_bits {
+ ICE_TX_CTX_DESC_TSO = 0x01,
+ ICE_TX_CTX_DESC_TSYN = 0x02,
+ ICE_TX_CTX_DESC_IL2TAG2 = 0x04,
+ ICE_TX_CTX_DESC_IL2TAG2_IL2H = 0x08,
+ ICE_TX_CTX_DESC_SWTCH_NOTAG = 0x00,
+ ICE_TX_CTX_DESC_SWTCH_UPLINK = 0x10,
+ ICE_TX_CTX_DESC_SWTCH_LOCAL = 0x20,
+ ICE_TX_CTX_DESC_SWTCH_VSI = 0x30,
+ ICE_TX_CTX_DESC_RESERVED = 0x40
+};
+
+#define ICE_LAN_TXQ_MAX_QGRPS 127
+#define ICE_LAN_TXQ_MAX_QDIS 1023
+
+/* Tx queue context data
+ *
+ * The sizes of the variables may be larger than needed due to crossing byte
+ * boundaries. If we do not have the width of the variable set to the correct
+ * size then we could end up shifting bits off the top of the variable when the
+ * variable is at the top of a byte and crosses over into the next byte.
+ */
+struct ice_tlan_ctx {
+#define ICE_TLAN_CTX_BASE_S 7
+ u64 base; /* base is defined in 128-byte units */
+ u8 port_num;
+ u16 cgd_num; /* bigger than needed, see above for reason */
+ u8 pf_num;
+ u16 vmvf_num;
+ u8 vmvf_type;
+#define ICE_TLAN_CTX_VMVF_TYPE_VMQ 1
+#define ICE_TLAN_CTX_VMVF_TYPE_PF 2
+ u16 src_vsi;
+ u8 tsyn_ena;
+ u8 alt_vlan;
+ u16 cpuid; /* bigger than needed, see above for reason */
+ u8 wb_mode;
+ u8 tphrd_desc;
+ u8 tphrd;
+ u8 tphwr_desc;
+ u16 cmpq_id;
+ u16 qnum_in_func;
+ u8 itr_notification_mode;
+ u8 adjust_prof_id;
+ u32 qlen; /* bigger than needed, see above for reason */
+ u8 quanta_prof_idx;
+ u8 tso_ena;
+ u16 tso_qnum;
+ u8 legacy_int;
+ u8 drop_ena;
+ u8 cache_prof_idx;
+ u8 pkt_shaper_prof_idx;
+ u8 int_q_state; /* width not needed - internal do not write */
+};
+
+/* macro to make the table lines short */
+#define ICE_PTT(PTYPE, OUTER_IP, OUTER_IP_VER, OUTER_FRAG, T, TE, TEF, I, PL)\
+ { PTYPE, \
+ 1, \
+ ICE_RX_PTYPE_OUTER_##OUTER_IP, \
+ ICE_RX_PTYPE_OUTER_##OUTER_IP_VER, \
+ ICE_RX_PTYPE_##OUTER_FRAG, \
+ ICE_RX_PTYPE_TUNNEL_##T, \
+ ICE_RX_PTYPE_TUNNEL_END_##TE, \
+ ICE_RX_PTYPE_##TEF, \
+ ICE_RX_PTYPE_INNER_PROT_##I, \
+ ICE_RX_PTYPE_PAYLOAD_LAYER_##PL }
+
+#define ICE_PTT_UNUSED_ENTRY(PTYPE) { PTYPE, 0, 0, 0, 0, 0, 0, 0, 0, 0 }
+
+/* shorter macros makes the table fit but are terse */
+#define ICE_RX_PTYPE_NOF ICE_RX_PTYPE_NOT_FRAG
+
+/* Lookup table mapping the HW PTYPE to the bit field for decoding */
+static const struct ice_rx_ptype_decoded ice_ptype_lkup[] = {
+ /* L2 Packet types */
+ ICE_PTT_UNUSED_ENTRY(0),
+ ICE_PTT(1, L2, NONE, NOF, NONE, NONE, NOF, NONE, PAY2),
+ ICE_PTT(2, L2, NONE, NOF, NONE, NONE, NOF, NONE, NONE),
+};
+
+static inline struct ice_rx_ptype_decoded ice_decode_rx_desc_ptype(u16 ptype)
+{
+ return ice_ptype_lkup[ptype];
+}
+#endif /* _ICE_LAN_TX_RX_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+/* Intel(R) Ethernet Connection E800 Series Linux Driver */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include "ice.h"
+
+#define DRV_VERSION "ice-0.7.0-k"
+#define DRV_SUMMARY "Intel(R) Ethernet Connection E800 Series Linux Driver"
+const char ice_drv_ver[] = DRV_VERSION;
+static const char ice_driver_string[] = DRV_SUMMARY;
+static const char ice_copyright[] = "Copyright (c) 2018, Intel Corporation.";
+
+MODULE_AUTHOR("Intel Corporation, <linux.nics@intel.com>");
+MODULE_DESCRIPTION(DRV_SUMMARY);
+MODULE_LICENSE("GPL");
+MODULE_VERSION(DRV_VERSION);
+
+static int debug = -1;
+module_param(debug, int, 0644);
+#ifndef CONFIG_DYNAMIC_DEBUG
+MODULE_PARM_DESC(debug, "netif level (0=none,...,16=all), hw debug_mask (0x8XXXXXXX)");
+#else
+MODULE_PARM_DESC(debug, "netif level (0=none,...,16=all)");
+#endif /* !CONFIG_DYNAMIC_DEBUG */
+
+static struct workqueue_struct *ice_wq;
+static const struct net_device_ops ice_netdev_ops;
+
+static void ice_pf_dis_all_vsi(struct ice_pf *pf);
+static void ice_rebuild(struct ice_pf *pf);
+static int ice_vsi_release(struct ice_vsi *vsi);
+static void ice_update_vsi_stats(struct ice_vsi *vsi);
+static void ice_update_pf_stats(struct ice_pf *pf);
+
+/**
+ * ice_get_free_slot - get the next non-NULL location index in array
+ * @array: array to search
+ * @size: size of the array
+ * @curr: last known occupied index to be used as a search hint
+ *
+ * void * is being used to keep the functionality generic. This lets us use this
+ * function on any array of pointers.
+ */
+static int ice_get_free_slot(void *array, int size, int curr)
+{
+ int **tmp_array = (int **)array;
+ int next;
+
+ if (curr < (size - 1) && !tmp_array[curr + 1]) {
+ next = curr + 1;
+ } else {
+ int i = 0;
+
+ while ((i < size) && (tmp_array[i]))
+ i++;
+ if (i == size)
+ next = ICE_NO_VSI;
+ else
+ next = i;
+ }
+ return next;
+}
+
+/**
+ * ice_search_res - Search the tracker for a block of resources
+ * @res: pointer to the resource
+ * @needed: size of the block needed
+ * @id: identifier to track owner
+ * Returns the base item index of the block, or -ENOMEM for error
+ */
+static int ice_search_res(struct ice_res_tracker *res, u16 needed, u16 id)
+{
+ int start = res->search_hint;
+ int end = start;
+
+ id |= ICE_RES_VALID_BIT;
+
+ do {
+ /* skip already allocated entries */
+ if (res->list[end++] & ICE_RES_VALID_BIT) {
+ start = end;
+ if ((start + needed) > res->num_entries)
+ break;
+ }
+
+ if (end == (start + needed)) {
+ int i = start;
+
+ /* there was enough, so assign it to the requestor */
+ while (i != end)
+ res->list[i++] = id;
+
+ if (end == res->num_entries)
+ end = 0;
+
+ res->search_hint = end;
+ return start;
+ }
+ } while (1);
+
+ return -ENOMEM;
+}
+
+/**
+ * ice_get_res - get a block of resources
+ * @pf: board private structure
+ * @res: pointer to the resource
+ * @needed: size of the block needed
+ * @id: identifier to track owner
+ *
+ * Returns the base item index of the block, or -ENOMEM for error
+ * The search_hint trick and lack of advanced fit-finding only works
+ * because we're highly likely to have all the same sized requests.
+ * Linear search time and any fragmentation should be minimal.
+ */
+static int
+ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id)
+{
+ int ret;
+
+ if (!res || !pf)
+ return -EINVAL;
+
+ if (!needed || needed > res->num_entries || id >= ICE_RES_VALID_BIT) {
+ dev_err(&pf->pdev->dev,
+ "param err: needed=%d, num_entries = %d id=0x%04x\n",
+ needed, res->num_entries, id);
+ return -EINVAL;
+ }
+
+ /* search based on search_hint */
+ ret = ice_search_res(res, needed, id);
+
+ if (ret < 0) {
+ /* previous search failed. Reset search hint and try again */
+ res->search_hint = 0;
+ ret = ice_search_res(res, needed, id);
+ }
+
+ return ret;
+}
+
+/**
+ * ice_free_res - free a block of resources
+ * @res: pointer to the resource
+ * @index: starting index previously returned by ice_get_res
+ * @id: identifier to track owner
+ * Returns number of resources freed
+ */
+static int ice_free_res(struct ice_res_tracker *res, u16 index, u16 id)
+{
+ int count = 0;
+ int i;
+
+ if (!res || index >= res->num_entries)
+ return -EINVAL;
+
+ id |= ICE_RES_VALID_BIT;
+ for (i = index; i < res->num_entries && res->list[i] == id; i++) {
+ res->list[i] = 0;
+ count++;
+ }
+
+ return count;
+}
+
+/**
+ * ice_add_mac_to_list - Add a mac address filter entry to the list
+ * @vsi: the VSI to be forwarded to
+ * @add_list: pointer to the list which contains MAC filter entries
+ * @macaddr: the MAC address to be added.
+ *
+ * Adds mac address filter entry to the temp list
+ *
+ * Returns 0 on success or ENOMEM on failure.
+ */
+static int ice_add_mac_to_list(struct ice_vsi *vsi, struct list_head *add_list,
+ const u8 *macaddr)
+{
+ struct ice_fltr_list_entry *tmp;
+ struct ice_pf *pf = vsi->back;
+
+ tmp = devm_kzalloc(&pf->pdev->dev, sizeof(*tmp), GFP_ATOMIC);
+ if (!tmp)
+ return -ENOMEM;
+
+ tmp->fltr_info.flag = ICE_FLTR_TX;
+ tmp->fltr_info.src = vsi->vsi_num;
+ tmp->fltr_info.lkup_type = ICE_SW_LKUP_MAC;
+ tmp->fltr_info.fltr_act = ICE_FWD_TO_VSI;
+ tmp->fltr_info.fwd_id.vsi_id = vsi->vsi_num;
+ ether_addr_copy(tmp->fltr_info.l_data.mac.mac_addr, macaddr);
+
+ INIT_LIST_HEAD(&tmp->list_entry);
+ list_add(&tmp->list_entry, add_list);
+
+ return 0;
+}
+
+/**
+ * ice_add_mac_to_sync_list - creates list of mac addresses to be synced
+ * @netdev: the net device on which the sync is happening
+ * @addr: mac address to sync
+ *
+ * This is a callback function which is called by the in kernel device sync
+ * functions (like __dev_uc_sync, __dev_mc_sync, etc). This function only
+ * populates the tmp_sync_list, which is later used by ice_add_mac to add the
+ * mac filters from the hardware.
+ */
+static int ice_add_mac_to_sync_list(struct net_device *netdev, const u8 *addr)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+
+ if (ice_add_mac_to_list(vsi, &vsi->tmp_sync_list, addr))
+ return -EINVAL;
+
+ return 0;
+}
+
+/**
+ * ice_add_mac_to_unsync_list - creates list of mac addresses to be unsynced
+ * @netdev: the net device on which the unsync is happening
+ * @addr: mac address to unsync
+ *
+ * This is a callback function which is called by the in kernel device unsync
+ * functions (like __dev_uc_unsync, __dev_mc_unsync, etc). This function only
+ * populates the tmp_unsync_list, which is later used by ice_remove_mac to
+ * delete the mac filters from the hardware.
+ */
+static int ice_add_mac_to_unsync_list(struct net_device *netdev, const u8 *addr)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+
+ if (ice_add_mac_to_list(vsi, &vsi->tmp_unsync_list, addr))
+ return -EINVAL;
+
+ return 0;
+}
+
+/**
+ * ice_free_fltr_list - free filter lists helper
+ * @dev: pointer to the device struct
+ * @h: pointer to the list head to be freed
+ *
+ * Helper function to free filter lists previously created using
+ * ice_add_mac_to_list
+ */
+static void ice_free_fltr_list(struct device *dev, struct list_head *h)
+{
+ struct ice_fltr_list_entry *e, *tmp;
+
+ list_for_each_entry_safe(e, tmp, h, list_entry) {
+ list_del(&e->list_entry);
+ devm_kfree(dev, e);
+ }
+}
+
+/**
+ * ice_vsi_fltr_changed - check if filter state changed
+ * @vsi: VSI to be checked
+ *
+ * returns true if filter state has changed, false otherwise.
+ */
+static bool ice_vsi_fltr_changed(struct ice_vsi *vsi)
+{
+ return test_bit(ICE_VSI_FLAG_UMAC_FLTR_CHANGED, vsi->flags) ||
+ test_bit(ICE_VSI_FLAG_MMAC_FLTR_CHANGED, vsi->flags) ||
+ test_bit(ICE_VSI_FLAG_VLAN_FLTR_CHANGED, vsi->flags);
+}
+
+/**
+ * ice_vsi_sync_fltr - Update the VSI filter list to the HW
+ * @vsi: ptr to the VSI
+ *
+ * Push any outstanding VSI filter changes through the AdminQ.
+ */
+static int ice_vsi_sync_fltr(struct ice_vsi *vsi)
+{
+ struct device *dev = &vsi->back->pdev->dev;
+ struct net_device *netdev = vsi->netdev;
+ bool promisc_forced_on = false;
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ enum ice_status status = 0;
+ u32 changed_flags = 0;
+ int err = 0;
+
+ if (!vsi->netdev)
+ return -EINVAL;
+
+ while (test_and_set_bit(__ICE_CFG_BUSY, vsi->state))
+ usleep_range(1000, 2000);
+
+ changed_flags = vsi->current_netdev_flags ^ vsi->netdev->flags;
+ vsi->current_netdev_flags = vsi->netdev->flags;
+
+ INIT_LIST_HEAD(&vsi->tmp_sync_list);
+ INIT_LIST_HEAD(&vsi->tmp_unsync_list);
+
+ if (ice_vsi_fltr_changed(vsi)) {
+ clear_bit(ICE_VSI_FLAG_UMAC_FLTR_CHANGED, vsi->flags);
+ clear_bit(ICE_VSI_FLAG_MMAC_FLTR_CHANGED, vsi->flags);
+ clear_bit(ICE_VSI_FLAG_VLAN_FLTR_CHANGED, vsi->flags);
+
+ /* grab the netdev's addr_list_lock */
+ netif_addr_lock_bh(netdev);
+ __dev_uc_sync(netdev, ice_add_mac_to_sync_list,
+ ice_add_mac_to_unsync_list);
+ __dev_mc_sync(netdev, ice_add_mac_to_sync_list,
+ ice_add_mac_to_unsync_list);
+ /* our temp lists are populated. release lock */
+ netif_addr_unlock_bh(netdev);
+ }
+
+ /* Remove mac addresses in the unsync list */
+ status = ice_remove_mac(hw, &vsi->tmp_unsync_list);
+ ice_free_fltr_list(dev, &vsi->tmp_unsync_list);
+ if (status) {
+ netdev_err(netdev, "Failed to delete MAC filters\n");
+ /* if we failed because of alloc failures, just bail */
+ if (status == ICE_ERR_NO_MEMORY) {
+ err = -ENOMEM;
+ goto out;
+ }
+ }
+
+ /* Add mac addresses in the sync list */
+ status = ice_add_mac(hw, &vsi->tmp_sync_list);
+ ice_free_fltr_list(dev, &vsi->tmp_sync_list);
+ if (status) {
+ netdev_err(netdev, "Failed to add MAC filters\n");
+ /* If there is no more space for new umac filters, vsi
+ * should go into promiscuous mode. There should be some
+ * space reserved for promiscuous filters.
+ */
+ if (hw->adminq.sq_last_status == ICE_AQ_RC_ENOSPC &&
+ !test_and_set_bit(__ICE_FLTR_OVERFLOW_PROMISC,
+ vsi->state)) {
+ promisc_forced_on = true;
+ netdev_warn(netdev,
+ "Reached MAC filter limit, forcing promisc mode on VSI %d\n",
+ vsi->vsi_num);
+ } else {
+ err = -EIO;
+ goto out;
+ }
+ }
+ /* check for changes in promiscuous modes */
+ if (changed_flags & IFF_ALLMULTI)
+ netdev_warn(netdev, "Unsupported configuration\n");
+
+ if (((changed_flags & IFF_PROMISC) || promisc_forced_on) ||
+ test_bit(ICE_VSI_FLAG_PROMISC_CHANGED, vsi->flags)) {
+ clear_bit(ICE_VSI_FLAG_PROMISC_CHANGED, vsi->flags);
+ if (vsi->current_netdev_flags & IFF_PROMISC) {
+ /* Apply TX filter rule to get traffic from VMs */
+ status = ice_cfg_dflt_vsi(hw, vsi->vsi_num, true,
+ ICE_FLTR_TX);
+ if (status) {
+ netdev_err(netdev, "Error setting default VSI %i tx rule\n",
+ vsi->vsi_num);
+ vsi->current_netdev_flags &= ~IFF_PROMISC;
+ err = -EIO;
+ goto out_promisc;
+ }
+ /* Apply RX filter rule to get traffic from wire */
+ status = ice_cfg_dflt_vsi(hw, vsi->vsi_num, true,
+ ICE_FLTR_RX);
+ if (status) {
+ netdev_err(netdev, "Error setting default VSI %i rx rule\n",
+ vsi->vsi_num);
+ vsi->current_netdev_flags &= ~IFF_PROMISC;
+ err = -EIO;
+ goto out_promisc;
+ }
+ } else {
+ /* Clear TX filter rule to stop traffic from VMs */
+ status = ice_cfg_dflt_vsi(hw, vsi->vsi_num, false,
+ ICE_FLTR_TX);
+ if (status) {
+ netdev_err(netdev, "Error clearing default VSI %i tx rule\n",
+ vsi->vsi_num);
+ vsi->current_netdev_flags |= IFF_PROMISC;
+ err = -EIO;
+ goto out_promisc;
+ }
+ /* Clear filter RX to remove traffic from wire */
+ status = ice_cfg_dflt_vsi(hw, vsi->vsi_num, false,
+ ICE_FLTR_RX);
+ if (status) {
+ netdev_err(netdev, "Error clearing default VSI %i rx rule\n",
+ vsi->vsi_num);
+ vsi->current_netdev_flags |= IFF_PROMISC;
+ err = -EIO;
+ goto out_promisc;
+ }
+ }
+ }
+ goto exit;
+
+out_promisc:
+ set_bit(ICE_VSI_FLAG_PROMISC_CHANGED, vsi->flags);
+ goto exit;
+out:
+ /* if something went wrong then set the changed flag so we try again */
+ set_bit(ICE_VSI_FLAG_UMAC_FLTR_CHANGED, vsi->flags);
+ set_bit(ICE_VSI_FLAG_MMAC_FLTR_CHANGED, vsi->flags);
+exit:
+ clear_bit(__ICE_CFG_BUSY, vsi->state);
+ return err;
+}
+
+/**
+ * ice_sync_fltr_subtask - Sync the VSI filter list with HW
+ * @pf: board private structure
+ */
+static void ice_sync_fltr_subtask(struct ice_pf *pf)
+{
+ int v;
+
+ if (!pf || !(test_bit(ICE_FLAG_FLTR_SYNC, pf->flags)))
+ return;
+
+ clear_bit(ICE_FLAG_FLTR_SYNC, pf->flags);
+
+ for (v = 0; v < pf->num_alloc_vsi; v++)
+ if (pf->vsi[v] && ice_vsi_fltr_changed(pf->vsi[v]) &&
+ ice_vsi_sync_fltr(pf->vsi[v])) {
+ /* come back and try again later */
+ set_bit(ICE_FLAG_FLTR_SYNC, pf->flags);
+ break;
+ }
+}
+
+/**
+ * ice_is_reset_recovery_pending - schedule a reset
+ * @state: pf state field
+ */
+static bool ice_is_reset_recovery_pending(unsigned long int *state)
+{
+ return test_bit(__ICE_RESET_RECOVERY_PENDING, state);
+}
+
+/**
+ * ice_prepare_for_reset - prep for the core to reset
+ * @pf: board private structure
+ *
+ * Inform or close all dependent features in prep for reset.
+ */
+static void
+ice_prepare_for_reset(struct ice_pf *pf)
+{
+ struct ice_hw *hw = &pf->hw;
+ u32 v;
+
+ ice_for_each_vsi(pf, v)
+ if (pf->vsi[v])
+ ice_remove_vsi_fltr(hw, pf->vsi[v]->vsi_num);
+
+ dev_dbg(&pf->pdev->dev, "Tearing down internal switch for reset\n");
+
+ /* disable the VSIs and their queues that are not already DOWN */
+ /* pf_dis_all_vsi modifies netdev structures -rtnl_lock needed */
+ ice_pf_dis_all_vsi(pf);
+
+ ice_for_each_vsi(pf, v)
+ if (pf->vsi[v])
+ pf->vsi[v]->vsi_num = 0;
+
+ ice_shutdown_all_ctrlq(hw);
+}
+
+/**
+ * ice_do_reset - Initiate one of many types of resets
+ * @pf: board private structure
+ * @reset_type: reset type requested
+ * before this function was called.
+ */
+static void ice_do_reset(struct ice_pf *pf, enum ice_reset_req reset_type)
+{
+ struct device *dev = &pf->pdev->dev;
+ struct ice_hw *hw = &pf->hw;
+
+ dev_dbg(dev, "reset_type 0x%x requested\n", reset_type);
+ WARN_ON(in_interrupt());
+
+ /* PFR is a bit of a special case because it doesn't result in an OICR
+ * interrupt. So for PFR, we prepare for reset, issue the reset and
+ * rebuild sequentially.
+ */
+ if (reset_type == ICE_RESET_PFR) {
+ set_bit(__ICE_RESET_RECOVERY_PENDING, pf->state);
+ ice_prepare_for_reset(pf);
+ }
+
+ /* trigger the reset */
+ if (ice_reset(hw, reset_type)) {
+ dev_err(dev, "reset %d failed\n", reset_type);
+ set_bit(__ICE_RESET_FAILED, pf->state);
+ clear_bit(__ICE_RESET_RECOVERY_PENDING, pf->state);
+ return;
+ }
+
+ if (reset_type == ICE_RESET_PFR) {
+ pf->pfr_count++;
+ ice_rebuild(pf);
+ clear_bit(__ICE_RESET_RECOVERY_PENDING, pf->state);
+ }
+}
+
+/**
+ * ice_reset_subtask - Set up for resetting the device and driver
+ * @pf: board private structure
+ */
+static void ice_reset_subtask(struct ice_pf *pf)
+{
+ enum ice_reset_req reset_type;
+
+ rtnl_lock();
+
+ /* When a CORER/GLOBR/EMPR is about to happen, the hardware triggers an
+ * OICR interrupt. The OICR handler (ice_misc_intr) determines what
+ * type of reset happened and sets __ICE_RESET_RECOVERY_PENDING bit in
+ * pf->state. So if reset/recovery is pending (as indicated by this bit)
+ * we do a rebuild and return.
+ */
+ if (ice_is_reset_recovery_pending(pf->state)) {
+ clear_bit(__ICE_GLOBR_RECV, pf->state);
+ clear_bit(__ICE_CORER_RECV, pf->state);
+ ice_prepare_for_reset(pf);
+
+ /* make sure we are ready to rebuild */
+ if (ice_check_reset(&pf->hw))
+ set_bit(__ICE_RESET_FAILED, pf->state);
+ else
+ ice_rebuild(pf);
+ clear_bit(__ICE_RESET_RECOVERY_PENDING, pf->state);
+ goto unlock;
+ }
+
+ /* No pending resets to finish processing. Check for new resets */
+ if (test_and_clear_bit(__ICE_GLOBR_REQ, pf->state))
+ reset_type = ICE_RESET_GLOBR;
+ else if (test_and_clear_bit(__ICE_CORER_REQ, pf->state))
+ reset_type = ICE_RESET_CORER;
+ else if (test_and_clear_bit(__ICE_PFR_REQ, pf->state))
+ reset_type = ICE_RESET_PFR;
+ else
+ goto unlock;
+
+ /* reset if not already down or resetting */
+ if (!test_bit(__ICE_DOWN, pf->state) &&
+ !test_bit(__ICE_CFG_BUSY, pf->state)) {
+ ice_do_reset(pf, reset_type);
+ }
+
+unlock:
+ rtnl_unlock();
+}
+
+/**
+ * ice_watchdog_subtask - periodic tasks not using event driven scheduling
+ * @pf: board private structure
+ */
+static void ice_watchdog_subtask(struct ice_pf *pf)
+{
+ int i;
+
+ /* if interface is down do nothing */
+ if (test_bit(__ICE_DOWN, pf->state) ||
+ test_bit(__ICE_CFG_BUSY, pf->state))
+ return;
+
+ /* make sure we don't do these things too often */
+ if (time_before(jiffies,
+ pf->serv_tmr_prev + pf->serv_tmr_period))
+ return;
+
+ pf->serv_tmr_prev = jiffies;
+
+ /* Update the stats for active netdevs so the network stack
+ * can look at updated numbers whenever it cares to
+ */
+ ice_update_pf_stats(pf);
+ for (i = 0; i < pf->num_alloc_vsi; i++)
+ if (pf->vsi[i] && pf->vsi[i]->netdev)
+ ice_update_vsi_stats(pf->vsi[i]);
+}
+
+/**
+ * ice_print_link_msg - print link up or down message
+ * @vsi: the VSI whose link status is being queried
+ * @isup: boolean for if the link is now up or down
+ */
+void ice_print_link_msg(struct ice_vsi *vsi, bool isup)
+{
+ const char *speed;
+ const char *fc;
+
+ if (vsi->current_isup == isup)
+ return;
+
+ vsi->current_isup = isup;
+
+ if (!isup) {
+ netdev_info(vsi->netdev, "NIC Link is Down\n");
+ return;
+ }
+
+ switch (vsi->port_info->phy.link_info.link_speed) {
+ case ICE_AQ_LINK_SPEED_40GB:
+ speed = "40 G";
+ break;
+ case ICE_AQ_LINK_SPEED_25GB:
+ speed = "25 G";
+ break;
+ case ICE_AQ_LINK_SPEED_20GB:
+ speed = "20 G";
+ break;
+ case ICE_AQ_LINK_SPEED_10GB:
+ speed = "10 G";
+ break;
+ case ICE_AQ_LINK_SPEED_5GB:
+ speed = "5 G";
+ break;
+ case ICE_AQ_LINK_SPEED_2500MB:
+ speed = "2.5 G";
+ break;
+ case ICE_AQ_LINK_SPEED_1000MB:
+ speed = "1 G";
+ break;
+ case ICE_AQ_LINK_SPEED_100MB:
+ speed = "100 M";
+ break;
+ default:
+ speed = "Unknown";
+ break;
+ }
+
+ switch (vsi->port_info->fc.current_mode) {
+ case ICE_FC_FULL:
+ fc = "RX/TX";
+ break;
+ case ICE_FC_TX_PAUSE:
+ fc = "TX";
+ break;
+ case ICE_FC_RX_PAUSE:
+ fc = "RX";
+ break;
+ default:
+ fc = "Unknown";
+ break;
+ }
+
+ netdev_info(vsi->netdev, "NIC Link is up %sbps, Flow Control: %s\n",
+ speed, fc);
+}
+
+/**
+ * ice_init_link_events - enable/initialize link events
+ * @pi: pointer to the port_info instance
+ *
+ * Returns -EIO on failure, 0 on success
+ */
+static int ice_init_link_events(struct ice_port_info *pi)
+{
+ u16 mask;
+
+ mask = ~((u16)(ICE_AQ_LINK_EVENT_UPDOWN | ICE_AQ_LINK_EVENT_MEDIA_NA |
+ ICE_AQ_LINK_EVENT_MODULE_QUAL_FAIL));
+
+ if (ice_aq_set_event_mask(pi->hw, pi->lport, mask, NULL)) {
+ dev_dbg(ice_hw_to_dev(pi->hw),
+ "Failed to set link event mask for port %d\n",
+ pi->lport);
+ return -EIO;
+ }
+
+ if (ice_aq_get_link_info(pi, true, NULL, NULL)) {
+ dev_dbg(ice_hw_to_dev(pi->hw),
+ "Failed to enable link events for port %d\n",
+ pi->lport);
+ return -EIO;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_vsi_link_event - update the vsi's netdev
+ * @vsi: the vsi on which the link event occurred
+ * @link_up: whether or not the vsi needs to be set up or down
+ */
+static void ice_vsi_link_event(struct ice_vsi *vsi, bool link_up)
+{
+ if (!vsi || test_bit(__ICE_DOWN, vsi->state))
+ return;
+
+ if (vsi->type == ICE_VSI_PF) {
+ if (!vsi->netdev) {
+ dev_dbg(&vsi->back->pdev->dev,
+ "vsi->netdev is not initialized!\n");
+ return;
+ }
+ if (link_up) {
+ netif_carrier_on(vsi->netdev);
+ netif_tx_wake_all_queues(vsi->netdev);
+ } else {
+ netif_carrier_off(vsi->netdev);
+ netif_tx_stop_all_queues(vsi->netdev);
+ }
+ }
+}
+
+/**
+ * ice_link_event - process the link event
+ * @pf: pf that the link event is associated with
+ * @pi: port_info for the port that the link event is associated with
+ *
+ * Returns -EIO if ice_get_link_status() fails
+ * Returns 0 on success
+ */
+static int
+ice_link_event(struct ice_pf *pf, struct ice_port_info *pi)
+{
+ u8 new_link_speed, old_link_speed;
+ struct ice_phy_info *phy_info;
+ bool new_link_same_as_old;
+ bool new_link, old_link;
+ u8 lport;
+ u16 v;
+
+ phy_info = &pi->phy;
+ phy_info->link_info_old = phy_info->link_info;
+ /* Force ice_get_link_status() to update link info */
+ phy_info->get_link_info = true;
+
+ old_link = (phy_info->link_info_old.link_info & ICE_AQ_LINK_UP);
+ old_link_speed = phy_info->link_info_old.link_speed;
+
+ lport = pi->lport;
+ if (ice_get_link_status(pi, &new_link)) {
+ dev_dbg(&pf->pdev->dev,
+ "Could not get link status for port %d\n", lport);
+ return -EIO;
+ }
+
+ new_link_speed = phy_info->link_info.link_speed;
+
+ new_link_same_as_old = (new_link == old_link &&
+ new_link_speed == old_link_speed);
+
+ ice_for_each_vsi(pf, v) {
+ struct ice_vsi *vsi = pf->vsi[v];
+
+ if (!vsi || !vsi->port_info)
+ continue;
+
+ if (new_link_same_as_old &&
+ (test_bit(__ICE_DOWN, vsi->state) ||
+ new_link == netif_carrier_ok(vsi->netdev)))
+ continue;
+
+ if (vsi->port_info->lport == lport) {
+ ice_print_link_msg(vsi, new_link);
+ ice_vsi_link_event(vsi, new_link);
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ice_handle_link_event - handle link event via ARQ
+ * @pf: pf that the link event is associated with
+ *
+ * Return -EINVAL if port_info is null
+ * Return status on succes
+ */
+static int ice_handle_link_event(struct ice_pf *pf)
+{
+ struct ice_port_info *port_info;
+ int status;
+
+ port_info = pf->hw.port_info;
+ if (!port_info)
+ return -EINVAL;
+
+ status = ice_link_event(pf, port_info);
+ if (status)
+ dev_dbg(&pf->pdev->dev,
+ "Could not process link event, error %d\n", status);
+
+ return status;
+}
+
+/**
+ * __ice_clean_ctrlq - helper function to clean controlq rings
+ * @pf: ptr to struct ice_pf
+ * @q_type: specific Control queue type
+ */
+static int __ice_clean_ctrlq(struct ice_pf *pf, enum ice_ctl_q q_type)
+{
+ struct ice_rq_event_info event;
+ struct ice_hw *hw = &pf->hw;
+ struct ice_ctl_q_info *cq;
+ u16 pending, i = 0;
+ const char *qtype;
+ u32 oldval, val;
+
+ /* Do not clean control queue if/when PF reset fails */
+ if (test_bit(__ICE_RESET_FAILED, pf->state))
+ return 0;
+
+ switch (q_type) {
+ case ICE_CTL_Q_ADMIN:
+ cq = &hw->adminq;
+ qtype = "Admin";
+ break;
+ default:
+ dev_warn(&pf->pdev->dev, "Unknown control queue type 0x%x\n",
+ q_type);
+ return 0;
+ }
+
+ /* check for error indications - PF_xx_AxQLEN register layout for
+ * FW/MBX/SB are identical so just use defines for PF_FW_AxQLEN.
+ */
+ val = rd32(hw, cq->rq.len);
+ if (val & (PF_FW_ARQLEN_ARQVFE_M | PF_FW_ARQLEN_ARQOVFL_M |
+ PF_FW_ARQLEN_ARQCRIT_M)) {
+ oldval = val;
+ if (val & PF_FW_ARQLEN_ARQVFE_M)
+ dev_dbg(&pf->pdev->dev,
+ "%s Receive Queue VF Error detected\n", qtype);
+ if (val & PF_FW_ARQLEN_ARQOVFL_M) {
+ dev_dbg(&pf->pdev->dev,
+ "%s Receive Queue Overflow Error detected\n",
+ qtype);
+ }
+ if (val & PF_FW_ARQLEN_ARQCRIT_M)
+ dev_dbg(&pf->pdev->dev,
+ "%s Receive Queue Critical Error detected\n",
+ qtype);
+ val &= ~(PF_FW_ARQLEN_ARQVFE_M | PF_FW_ARQLEN_ARQOVFL_M |
+ PF_FW_ARQLEN_ARQCRIT_M);
+ if (oldval != val)
+ wr32(hw, cq->rq.len, val);
+ }
+
+ val = rd32(hw, cq->sq.len);
+ if (val & (PF_FW_ATQLEN_ATQVFE_M | PF_FW_ATQLEN_ATQOVFL_M |
+ PF_FW_ATQLEN_ATQCRIT_M)) {
+ oldval = val;
+ if (val & PF_FW_ATQLEN_ATQVFE_M)
+ dev_dbg(&pf->pdev->dev,
+ "%s Send Queue VF Error detected\n", qtype);
+ if (val & PF_FW_ATQLEN_ATQOVFL_M) {
+ dev_dbg(&pf->pdev->dev,
+ "%s Send Queue Overflow Error detected\n",
+ qtype);
+ }
+ if (val & PF_FW_ATQLEN_ATQCRIT_M)
+ dev_dbg(&pf->pdev->dev,
+ "%s Send Queue Critical Error detected\n",
+ qtype);
+ val &= ~(PF_FW_ATQLEN_ATQVFE_M | PF_FW_ATQLEN_ATQOVFL_M |
+ PF_FW_ATQLEN_ATQCRIT_M);
+ if (oldval != val)
+ wr32(hw, cq->sq.len, val);
+ }
+
+ event.buf_len = cq->rq_buf_size;
+ event.msg_buf = devm_kzalloc(&pf->pdev->dev, event.buf_len,
+ GFP_KERNEL);
+ if (!event.msg_buf)
+ return 0;
+
+ do {
+ enum ice_status ret;
+ u16 opcode;
+
+ ret = ice_clean_rq_elem(hw, cq, &event, &pending);
+ if (ret == ICE_ERR_AQ_NO_WORK)
+ break;
+ if (ret) {
+ dev_err(&pf->pdev->dev,
+ "%s Receive Queue event error %d\n", qtype,
+ ret);
+ break;
+ }
+
+ opcode = le16_to_cpu(event.desc.opcode);
+
+ switch (opcode) {
+ case ice_aqc_opc_get_link_status:
+ if (ice_handle_link_event(pf))
+ dev_err(&pf->pdev->dev,
+ "Could not handle link event");
+ break;
+ default:
+ dev_dbg(&pf->pdev->dev,
+ "%s Receive Queue unknown event 0x%04x ignored\n",
+ qtype, opcode);
+ break;
+ }
+ } while (pending && (i++ < ICE_DFLT_IRQ_WORK));
+
+ devm_kfree(&pf->pdev->dev, event.msg_buf);
+
+ return pending && (i == ICE_DFLT_IRQ_WORK);
+}
+
+/**
+ * ice_clean_adminq_subtask - clean the AdminQ rings
+ * @pf: board private structure
+ */
+static void ice_clean_adminq_subtask(struct ice_pf *pf)
+{
+ struct ice_hw *hw = &pf->hw;
+ u32 val;
+
+ if (!test_bit(__ICE_ADMINQ_EVENT_PENDING, pf->state))
+ return;
+
+ if (__ice_clean_ctrlq(pf, ICE_CTL_Q_ADMIN))
+ return;
+
+ clear_bit(__ICE_ADMINQ_EVENT_PENDING, pf->state);
+
+ /* re-enable Admin queue interrupt causes */
+ val = rd32(hw, PFINT_FW_CTL);
+ wr32(hw, PFINT_FW_CTL, (val | PFINT_FW_CTL_CAUSE_ENA_M));
+
+ ice_flush(hw);
+}
+
+/**
+ * ice_service_task_schedule - schedule the service task to wake up
+ * @pf: board private structure
+ *
+ * If not already scheduled, this puts the task into the work queue.
+ */
+static void ice_service_task_schedule(struct ice_pf *pf)
+{
+ if (!test_bit(__ICE_DOWN, pf->state) &&
+ !test_and_set_bit(__ICE_SERVICE_SCHED, pf->state))
+ queue_work(ice_wq, &pf->serv_task);
+}
+
+/**
+ * ice_service_task_complete - finish up the service task
+ * @pf: board private structure
+ */
+static void ice_service_task_complete(struct ice_pf *pf)
+{
+ WARN_ON(!test_bit(__ICE_SERVICE_SCHED, pf->state));
+
+ /* force memory (pf->state) to sync before next service task */
+ smp_mb__before_atomic();
+ clear_bit(__ICE_SERVICE_SCHED, pf->state);
+}
+
+/**
+ * ice_service_timer - timer callback to schedule service task
+ * @t: pointer to timer_list
+ */
+static void ice_service_timer(struct timer_list *t)
+{
+ struct ice_pf *pf = from_timer(pf, t, serv_tmr);
+
+ mod_timer(&pf->serv_tmr, round_jiffies(pf->serv_tmr_period + jiffies));
+ ice_service_task_schedule(pf);
+}
+
+/**
+ * ice_service_task - manage and run subtasks
+ * @work: pointer to work_struct contained by the PF struct
+ */
+static void ice_service_task(struct work_struct *work)
+{
+ struct ice_pf *pf = container_of(work, struct ice_pf, serv_task);
+ unsigned long start_time = jiffies;
+
+ /* subtasks */
+
+ /* process reset requests first */
+ ice_reset_subtask(pf);
+
+ /* bail if a reset/recovery cycle is pending */
+ if (ice_is_reset_recovery_pending(pf->state) ||
+ test_bit(__ICE_SUSPENDED, pf->state)) {
+ ice_service_task_complete(pf);
+ return;
+ }
+
+ ice_sync_fltr_subtask(pf);
+ ice_watchdog_subtask(pf);
+ ice_clean_adminq_subtask(pf);
+
+ /* Clear __ICE_SERVICE_SCHED flag to allow scheduling next event */
+ ice_service_task_complete(pf);
+
+ /* If the tasks have taken longer than one service timer period
+ * or there is more work to be done, reset the service timer to
+ * schedule the service task now.
+ */
+ if (time_after(jiffies, (start_time + pf->serv_tmr_period)) ||
+ test_bit(__ICE_ADMINQ_EVENT_PENDING, pf->state))
+ mod_timer(&pf->serv_tmr, jiffies);
+}
+
+/**
+ * ice_set_ctrlq_len - helper function to set controlq length
+ * @hw: pointer to the hw instance
+ */
+static void ice_set_ctrlq_len(struct ice_hw *hw)
+{
+ hw->adminq.num_rq_entries = ICE_AQ_LEN;
+ hw->adminq.num_sq_entries = ICE_AQ_LEN;
+ hw->adminq.rq_buf_size = ICE_AQ_MAX_BUF_LEN;
+ hw->adminq.sq_buf_size = ICE_AQ_MAX_BUF_LEN;
+}
+
+/**
+ * ice_irq_affinity_notify - Callback for affinity changes
+ * @notify: context as to what irq was changed
+ * @mask: the new affinity mask
+ *
+ * This is a callback function used by the irq_set_affinity_notifier function
+ * so that we may register to receive changes to the irq affinity masks.
+ */
+static void ice_irq_affinity_notify(struct irq_affinity_notify *notify,
+ const cpumask_t *mask)
+{
+ struct ice_q_vector *q_vector =
+ container_of(notify, struct ice_q_vector, affinity_notify);
+
+ cpumask_copy(&q_vector->affinity_mask, mask);
+}
+
+/**
+ * ice_irq_affinity_release - Callback for affinity notifier release
+ * @ref: internal core kernel usage
+ *
+ * This is a callback function used by the irq_set_affinity_notifier function
+ * to inform the current notification subscriber that they will no longer
+ * receive notifications.
+ */
+static void ice_irq_affinity_release(struct kref __always_unused *ref) {}
+
+/**
+ * ice_vsi_dis_irq - Mask off queue interrupt generation on the VSI
+ * @vsi: the VSI being un-configured
+ */
+static void ice_vsi_dis_irq(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ int base = vsi->base_vector;
+ u32 val;
+ int i;
+
+ /* disable interrupt causation from each queue */
+ if (vsi->tx_rings) {
+ ice_for_each_txq(vsi, i) {
+ if (vsi->tx_rings[i]) {
+ u16 reg;
+
+ reg = vsi->tx_rings[i]->reg_idx;
+ val = rd32(hw, QINT_TQCTL(reg));
+ val &= ~QINT_TQCTL_CAUSE_ENA_M;
+ wr32(hw, QINT_TQCTL(reg), val);
+ }
+ }
+ }
+
+ if (vsi->rx_rings) {
+ ice_for_each_rxq(vsi, i) {
+ if (vsi->rx_rings[i]) {
+ u16 reg;
+
+ reg = vsi->rx_rings[i]->reg_idx;
+ val = rd32(hw, QINT_RQCTL(reg));
+ val &= ~QINT_RQCTL_CAUSE_ENA_M;
+ wr32(hw, QINT_RQCTL(reg), val);
+ }
+ }
+ }
+
+ /* disable each interrupt */
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags)) {
+ for (i = vsi->base_vector;
+ i < (vsi->num_q_vectors + vsi->base_vector); i++)
+ wr32(hw, GLINT_DYN_CTL(i), 0);
+
+ ice_flush(hw);
+ for (i = 0; i < vsi->num_q_vectors; i++)
+ synchronize_irq(pf->msix_entries[i + base].vector);
+ }
+}
+
+/**
+ * ice_vsi_ena_irq - Enable IRQ for the given VSI
+ * @vsi: the VSI being configured
+ */
+static int ice_vsi_ena_irq(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags)) {
+ int i;
+
+ for (i = 0; i < vsi->num_q_vectors; i++)
+ ice_irq_dynamic_ena(hw, vsi, vsi->q_vectors[i]);
+ }
+
+ ice_flush(hw);
+ return 0;
+}
+
+/**
+ * ice_vsi_delete - delete a VSI from the switch
+ * @vsi: pointer to VSI being removed
+ */
+static void ice_vsi_delete(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_vsi_ctx ctxt;
+ enum ice_status status;
+
+ ctxt.vsi_num = vsi->vsi_num;
+
+ memcpy(&ctxt.info, &vsi->info, sizeof(struct ice_aqc_vsi_props));
+
+ status = ice_aq_free_vsi(&pf->hw, &ctxt, false, NULL);
+ if (status)
+ dev_err(&pf->pdev->dev, "Failed to delete VSI %i in FW\n",
+ vsi->vsi_num);
+}
+
+/**
+ * ice_vsi_req_irq_msix - get MSI-X vectors from the OS for the VSI
+ * @vsi: the VSI being configured
+ * @basename: name for the vector
+ */
+static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
+{
+ int q_vectors = vsi->num_q_vectors;
+ struct ice_pf *pf = vsi->back;
+ int base = vsi->base_vector;
+ int rx_int_idx = 0;
+ int tx_int_idx = 0;
+ int vector, err;
+ int irq_num;
+
+ for (vector = 0; vector < q_vectors; vector++) {
+ struct ice_q_vector *q_vector = vsi->q_vectors[vector];
+
+ irq_num = pf->msix_entries[base + vector].vector;
+
+ if (q_vector->tx.ring && q_vector->rx.ring) {
+ snprintf(q_vector->name, sizeof(q_vector->name) - 1,
+ "%s-%s-%d", basename, "TxRx", rx_int_idx++);
+ tx_int_idx++;
+ } else if (q_vector->rx.ring) {
+ snprintf(q_vector->name, sizeof(q_vector->name) - 1,
+ "%s-%s-%d", basename, "rx", rx_int_idx++);
+ } else if (q_vector->tx.ring) {
+ snprintf(q_vector->name, sizeof(q_vector->name) - 1,
+ "%s-%s-%d", basename, "tx", tx_int_idx++);
+ } else {
+ /* skip this unused q_vector */
+ continue;
+ }
+ err = devm_request_irq(&pf->pdev->dev,
+ pf->msix_entries[base + vector].vector,
+ vsi->irq_handler, 0, q_vector->name,
+ q_vector);
+ if (err) {
+ netdev_err(vsi->netdev,
+ "MSIX request_irq failed, error: %d\n", err);
+ goto free_q_irqs;
+ }
+
+ /* register for affinity change notifications */
+ q_vector->affinity_notify.notify = ice_irq_affinity_notify;
+ q_vector->affinity_notify.release = ice_irq_affinity_release;
+ irq_set_affinity_notifier(irq_num, &q_vector->affinity_notify);
+
+ /* assign the mask for this irq */
+ irq_set_affinity_hint(irq_num, &q_vector->affinity_mask);
+ }
+
+ vsi->irqs_ready = true;
+ return 0;
+
+free_q_irqs:
+ while (vector) {
+ vector--;
+ irq_num = pf->msix_entries[base + vector].vector,
+ irq_set_affinity_notifier(irq_num, NULL);
+ irq_set_affinity_hint(irq_num, NULL);
+ devm_free_irq(&pf->pdev->dev, irq_num, &vsi->q_vectors[vector]);
+ }
+ return err;
+}
+
+/**
+ * ice_vsi_set_rss_params - Setup RSS capabilities per VSI type
+ * @vsi: the VSI being configured
+ */
+static void ice_vsi_set_rss_params(struct ice_vsi *vsi)
+{
+ struct ice_hw_common_caps *cap;
+ struct ice_pf *pf = vsi->back;
+
+ if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags)) {
+ vsi->rss_size = 1;
+ return;
+ }
+
+ cap = &pf->hw.func_caps.common_cap;
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ /* PF VSI will inherit RSS instance of PF */
+ vsi->rss_table_size = cap->rss_table_size;
+ vsi->rss_size = min_t(int, num_online_cpus(),
+ BIT(cap->rss_table_entry_width));
+ vsi->rss_lut_type = ICE_AQC_GSET_RSS_LUT_TABLE_TYPE_PF;
+ break;
+ default:
+ dev_warn(&pf->pdev->dev, "Unknown VSI type %d\n", vsi->type);
+ break;
+ }
+}
+
+/**
+ * ice_vsi_setup_q_map - Setup a VSI queue map
+ * @vsi: the VSI being configured
+ * @ctxt: VSI context structure
+ */
+static void ice_vsi_setup_q_map(struct ice_vsi *vsi, struct ice_vsi_ctx *ctxt)
+{
+ u16 offset = 0, qmap = 0, numq_tc;
+ u16 pow = 0, max_rss = 0, qcount;
+ u16 qcount_tx = vsi->alloc_txq;
+ u16 qcount_rx = vsi->alloc_rxq;
+ bool ena_tc0 = false;
+ int i;
+
+ /* at least TC0 should be enabled by default */
+ if (vsi->tc_cfg.numtc) {
+ if (!(vsi->tc_cfg.ena_tc & BIT(0)))
+ ena_tc0 = true;
+ } else {
+ ena_tc0 = true;
+ }
+
+ if (ena_tc0) {
+ vsi->tc_cfg.numtc++;
+ vsi->tc_cfg.ena_tc |= 1;
+ }
+
+ numq_tc = qcount_rx / vsi->tc_cfg.numtc;
+
+ /* TC mapping is a function of the number of Rx queues assigned to the
+ * VSI for each traffic class and the offset of these queues.
+ * The first 10 bits are for queue offset for TC0, next 4 bits for no:of
+ * queues allocated to TC0. No:of queues is a power-of-2.
+ *
+ * If TC is not enabled, the queue offset is set to 0, and allocate one
+ * queue, this way, traffic for the given TC will be sent to the default
+ * queue.
+ *
+ * Setup number and offset of Rx queues for all TCs for the VSI
+ */
+
+ /* qcount will change if RSS is enabled */
+ if (test_bit(ICE_FLAG_RSS_ENA, vsi->back->flags)) {
+ if (vsi->type == ICE_VSI_PF)
+ max_rss = ICE_MAX_LG_RSS_QS;
+ else
+ max_rss = ICE_MAX_SMALL_RSS_QS;
+
+ qcount = min_t(int, numq_tc, max_rss);
+ qcount = min_t(int, qcount, vsi->rss_size);
+ } else {
+ qcount = numq_tc;
+ }
+
+ /* find higher power-of-2 of qcount */
+ pow = ilog2(qcount);
+
+ if (!is_power_of_2(qcount))
+ pow++;
+
+ for (i = 0; i < ICE_MAX_TRAFFIC_CLASS; i++) {
+ if (!(vsi->tc_cfg.ena_tc & BIT(i))) {
+ /* TC is not enabled */
+ vsi->tc_cfg.tc_info[i].qoffset = 0;
+ vsi->tc_cfg.tc_info[i].qcount = 1;
+ ctxt->info.tc_mapping[i] = 0;
+ continue;
+ }
+
+ /* TC is enabled */
+ vsi->tc_cfg.tc_info[i].qoffset = offset;
+ vsi->tc_cfg.tc_info[i].qcount = qcount;
+
+ qmap = ((offset << ICE_AQ_VSI_TC_Q_OFFSET_S) &
+ ICE_AQ_VSI_TC_Q_OFFSET_M) |
+ ((pow << ICE_AQ_VSI_TC_Q_NUM_S) &
+ ICE_AQ_VSI_TC_Q_NUM_M);
+ offset += qcount;
+ ctxt->info.tc_mapping[i] = cpu_to_le16(qmap);
+ }
+
+ vsi->num_txq = qcount_tx;
+ vsi->num_rxq = offset;
+
+ /* Rx queue mapping */
+ ctxt->info.mapping_flags |= cpu_to_le16(ICE_AQ_VSI_Q_MAP_CONTIG);
+ /* q_mapping buffer holds the info for the first queue allocated for
+ * this VSI in the PF space and also the number of queues associated
+ * with this VSI.
+ */
+ ctxt->info.q_mapping[0] = cpu_to_le16(vsi->rxq_map[0]);
+ ctxt->info.q_mapping[1] = cpu_to_le16(vsi->num_rxq);
+}
+
+/**
+ * ice_set_dflt_vsi_ctx - Set default VSI context before adding a VSI
+ * @ctxt: the VSI context being set
+ *
+ * This initializes a default VSI context for all sections except the Queues.
+ */
+static void ice_set_dflt_vsi_ctx(struct ice_vsi_ctx *ctxt)
+{
+ u32 table = 0;
+
+ memset(&ctxt->info, 0, sizeof(ctxt->info));
+ /* VSI's should be allocated from shared pool */
+ ctxt->alloc_from_pool = true;
+ /* Src pruning enabled by default */
+ ctxt->info.sw_flags = ICE_AQ_VSI_SW_FLAG_SRC_PRUNE;
+ /* Traffic from VSI can be sent to LAN */
+ ctxt->info.sw_flags2 = ICE_AQ_VSI_SW_FLAG_LAN_ENA;
+ /* Allow all packets untagged/tagged */
+ ctxt->info.port_vlan_flags = ((ICE_AQ_VSI_PVLAN_MODE_ALL &
+ ICE_AQ_VSI_PVLAN_MODE_M) >>
+ ICE_AQ_VSI_PVLAN_MODE_S);
+ /* Show VLAN/UP from packets in Rx descriptors */
+ ctxt->info.port_vlan_flags |= ((ICE_AQ_VSI_PVLAN_EMOD_STR_BOTH &
+ ICE_AQ_VSI_PVLAN_EMOD_M) >>
+ ICE_AQ_VSI_PVLAN_EMOD_S);
+ /* Have 1:1 UP mapping for both ingress/egress tables */
+ table |= ICE_UP_TABLE_TRANSLATE(0, 0);
+ table |= ICE_UP_TABLE_TRANSLATE(1, 1);
+ table |= ICE_UP_TABLE_TRANSLATE(2, 2);
+ table |= ICE_UP_TABLE_TRANSLATE(3, 3);
+ table |= ICE_UP_TABLE_TRANSLATE(4, 4);
+ table |= ICE_UP_TABLE_TRANSLATE(5, 5);
+ table |= ICE_UP_TABLE_TRANSLATE(6, 6);
+ table |= ICE_UP_TABLE_TRANSLATE(7, 7);
+ ctxt->info.ingress_table = cpu_to_le32(table);
+ ctxt->info.egress_table = cpu_to_le32(table);
+ /* Have 1:1 UP mapping for outer to inner UP table */
+ ctxt->info.outer_up_table = cpu_to_le32(table);
+ /* No Outer tag support outer_tag_flags remains to zero */
+}
+
+/**
+ * ice_set_rss_vsi_ctx - Set RSS VSI context before adding a VSI
+ * @ctxt: the VSI context being set
+ * @vsi: the VSI being configured
+ */
+static void ice_set_rss_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi)
+{
+ u8 lut_type, hash_type;
+
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ /* PF VSI will inherit RSS instance of PF */
+ lut_type = ICE_AQ_VSI_Q_OPT_RSS_LUT_PF;
+ hash_type = ICE_AQ_VSI_Q_OPT_RSS_TPLZ;
+ break;
+ default:
+ dev_warn(&vsi->back->pdev->dev, "Unknown VSI type %d\n",
+ vsi->type);
+ return;
+ }
+
+ ctxt->info.q_opt_rss = ((lut_type << ICE_AQ_VSI_Q_OPT_RSS_LUT_S) &
+ ICE_AQ_VSI_Q_OPT_RSS_LUT_M) |
+ ((hash_type << ICE_AQ_VSI_Q_OPT_RSS_HASH_S) &
+ ICE_AQ_VSI_Q_OPT_RSS_HASH_M);
+}
+
+/**
+ * ice_vsi_add - Create a new VSI or fetch preallocated VSI
+ * @vsi: the VSI being configured
+ *
+ * This initializes a VSI context depending on the VSI type to be added and
+ * passes it down to the add_vsi aq command to create a new VSI.
+ */
+static int ice_vsi_add(struct ice_vsi *vsi)
+{
+ struct ice_vsi_ctx ctxt = { 0 };
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ int ret = 0;
+
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ ctxt.flags = ICE_AQ_VSI_TYPE_PF;
+ break;
+ default:
+ return -ENODEV;
+ }
+
+ ice_set_dflt_vsi_ctx(&ctxt);
+ /* if the switch is in VEB mode, allow VSI loopback */
+ if (vsi->vsw->bridge_mode == BRIDGE_MODE_VEB)
+ ctxt.info.sw_flags |= ICE_AQ_VSI_SW_FLAG_ALLOW_LB;
+
+ /* Set LUT type and HASH type if RSS is enabled */
+ if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
+ ice_set_rss_vsi_ctx(&ctxt, vsi);
+
+ ctxt.info.sw_id = vsi->port_info->sw_id;
+ ice_vsi_setup_q_map(vsi, &ctxt);
+
+ ret = ice_aq_add_vsi(hw, &ctxt, NULL);
+ if (ret) {
+ dev_err(&vsi->back->pdev->dev,
+ "Add VSI AQ call failed, err %d\n", ret);
+ return -EIO;
+ }
+ vsi->info = ctxt.info;
+ vsi->vsi_num = ctxt.vsi_num;
+
+ return ret;
+}
+
+/**
+ * ice_vsi_release_msix - Clear the queue to Interrupt mapping in HW
+ * @vsi: the VSI being cleaned up
+ */
+static void ice_vsi_release_msix(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ u16 vector = vsi->base_vector;
+ struct ice_hw *hw = &pf->hw;
+ u32 txq = 0;
+ u32 rxq = 0;
+ int i, q;
+
+ for (i = 0; i < vsi->num_q_vectors; i++, vector++) {
+ struct ice_q_vector *q_vector = vsi->q_vectors[i];
+
+ wr32(hw, GLINT_ITR(ICE_RX_ITR, vector), 0);
+ wr32(hw, GLINT_ITR(ICE_TX_ITR, vector), 0);
+ for (q = 0; q < q_vector->num_ring_tx; q++) {
+ wr32(hw, QINT_TQCTL(vsi->txq_map[txq]), 0);
+ txq++;
+ }
+
+ for (q = 0; q < q_vector->num_ring_rx; q++) {
+ wr32(hw, QINT_RQCTL(vsi->rxq_map[rxq]), 0);
+ rxq++;
+ }
+ }
+
+ ice_flush(hw);
+}
+
+/**
+ * ice_vsi_clear_rings - Deallocates the Tx and Rx rings for VSI
+ * @vsi: the VSI having rings deallocated
+ */
+static void ice_vsi_clear_rings(struct ice_vsi *vsi)
+{
+ int i;
+
+ if (vsi->tx_rings) {
+ for (i = 0; i < vsi->alloc_txq; i++) {
+ if (vsi->tx_rings[i]) {
+ kfree_rcu(vsi->tx_rings[i], rcu);
+ vsi->tx_rings[i] = NULL;
+ }
+ }
+ }
+ if (vsi->rx_rings) {
+ for (i = 0; i < vsi->alloc_rxq; i++) {
+ if (vsi->rx_rings[i]) {
+ kfree_rcu(vsi->rx_rings[i], rcu);
+ vsi->rx_rings[i] = NULL;
+ }
+ }
+ }
+}
+
+/**
+ * ice_vsi_alloc_rings - Allocates Tx and Rx rings for the VSI
+ * @vsi: VSI which is having rings allocated
+ */
+static int ice_vsi_alloc_rings(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int i;
+
+ /* Allocate tx_rings */
+ for (i = 0; i < vsi->alloc_txq; i++) {
+ struct ice_ring *ring;
+
+ /* allocate with kzalloc(), free with kfree_rcu() */
+ ring = kzalloc(sizeof(*ring), GFP_KERNEL);
+
+ if (!ring)
+ goto err_out;
+
+ ring->q_index = i;
+ ring->reg_idx = vsi->txq_map[i];
+ ring->ring_active = false;
+ ring->vsi = vsi;
+ ring->netdev = vsi->netdev;
+ ring->dev = &pf->pdev->dev;
+ ring->count = vsi->num_desc;
+
+ vsi->tx_rings[i] = ring;
+ }
+
+ /* Allocate rx_rings */
+ for (i = 0; i < vsi->alloc_rxq; i++) {
+ struct ice_ring *ring;
+
+ /* allocate with kzalloc(), free with kfree_rcu() */
+ ring = kzalloc(sizeof(*ring), GFP_KERNEL);
+ if (!ring)
+ goto err_out;
+
+ ring->q_index = i;
+ ring->reg_idx = vsi->rxq_map[i];
+ ring->ring_active = false;
+ ring->vsi = vsi;
+ ring->netdev = vsi->netdev;
+ ring->dev = &pf->pdev->dev;
+ ring->count = vsi->num_desc;
+ vsi->rx_rings[i] = ring;
+ }
+
+ return 0;
+
+err_out:
+ ice_vsi_clear_rings(vsi);
+ return -ENOMEM;
+}
+
+/**
+ * ice_vsi_free_irq - Free the irq association with the OS
+ * @vsi: the VSI being configured
+ */
+static void ice_vsi_free_irq(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int base = vsi->base_vector;
+
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags)) {
+ int i;
+
+ if (!vsi->q_vectors || !vsi->irqs_ready)
+ return;
+
+ vsi->irqs_ready = false;
+ for (i = 0; i < vsi->num_q_vectors; i++) {
+ u16 vector = i + base;
+ int irq_num;
+
+ irq_num = pf->msix_entries[vector].vector;
+
+ /* free only the irqs that were actually requested */
+ if (!vsi->q_vectors[i] ||
+ !(vsi->q_vectors[i]->num_ring_tx ||
+ vsi->q_vectors[i]->num_ring_rx))
+ continue;
+
+ /* clear the affinity notifier in the IRQ descriptor */
+ irq_set_affinity_notifier(irq_num, NULL);
+
+ /* clear the affinity_mask in the IRQ descriptor */
+ irq_set_affinity_hint(irq_num, NULL);
+ synchronize_irq(irq_num);
+ devm_free_irq(&pf->pdev->dev, irq_num,
+ vsi->q_vectors[i]);
+ }
+ ice_vsi_release_msix(vsi);
+ }
+}
+
+/**
+ * ice_vsi_cfg_msix - MSIX mode Interrupt Config in the HW
+ * @vsi: the VSI being configured
+ */
+static void ice_vsi_cfg_msix(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ u16 vector = vsi->base_vector;
+ struct ice_hw *hw = &pf->hw;
+ u32 txq = 0, rxq = 0;
+ int i, q, itr;
+ u8 itr_gran;
+
+ for (i = 0; i < vsi->num_q_vectors; i++, vector++) {
+ struct ice_q_vector *q_vector = vsi->q_vectors[i];
+
+ itr_gran = hw->itr_gran_200;
+
+ if (q_vector->num_ring_rx) {
+ q_vector->rx.itr =
+ ITR_TO_REG(vsi->rx_rings[rxq]->rx_itr_setting,
+ itr_gran);
+ q_vector->rx.latency_range = ICE_LOW_LATENCY;
+ }
+
+ if (q_vector->num_ring_tx) {
+ q_vector->tx.itr =
+ ITR_TO_REG(vsi->tx_rings[txq]->tx_itr_setting,
+ itr_gran);
+ q_vector->tx.latency_range = ICE_LOW_LATENCY;
+ }
+ wr32(hw, GLINT_ITR(ICE_RX_ITR, vector), q_vector->rx.itr);
+ wr32(hw, GLINT_ITR(ICE_TX_ITR, vector), q_vector->tx.itr);
+
+ /* Both Transmit Queue Interrupt Cause Control register
+ * and Receive Queue Interrupt Cause control register
+ * expects MSIX_INDX field to be the vector index
+ * within the function space and not the absolute
+ * vector index across PF or across device.
+ * For SR-IOV VF VSIs queue vector index always starts
+ * with 1 since first vector index(0) is used for OICR
+ * in VF space. Since VMDq and other PF VSIs are withtin
+ * the PF function space, use the vector index thats
+ * tracked for this PF.
+ */
+ for (q = 0; q < q_vector->num_ring_tx; q++) {
+ u32 val;
+
+ itr = ICE_TX_ITR;
+ val = QINT_TQCTL_CAUSE_ENA_M |
+ (itr << QINT_TQCTL_ITR_INDX_S) |
+ (vector << QINT_TQCTL_MSIX_INDX_S);
+ wr32(hw, QINT_TQCTL(vsi->txq_map[txq]), val);
+ txq++;
+ }
+
+ for (q = 0; q < q_vector->num_ring_rx; q++) {
+ u32 val;
+
+ itr = ICE_RX_ITR;
+ val = QINT_RQCTL_CAUSE_ENA_M |
+ (itr << QINT_RQCTL_ITR_INDX_S) |
+ (vector << QINT_RQCTL_MSIX_INDX_S);
+ wr32(hw, QINT_RQCTL(vsi->rxq_map[rxq]), val);
+ rxq++;
+ }
+ }
+
+ ice_flush(hw);
+}
+
+/**
+ * ice_ena_misc_vector - enable the non-queue interrupts
+ * @pf: board private structure
+ */
+static void ice_ena_misc_vector(struct ice_pf *pf)
+{
+ struct ice_hw *hw = &pf->hw;
+ u32 val;
+
+ /* clear things first */
+ wr32(hw, PFINT_OICR_ENA, 0); /* disable all */
+ rd32(hw, PFINT_OICR); /* read to clear */
+
+ val = (PFINT_OICR_HLP_RDY_M |
+ PFINT_OICR_CPM_RDY_M |
+ PFINT_OICR_ECC_ERR_M |
+ PFINT_OICR_MAL_DETECT_M |
+ PFINT_OICR_GRST_M |
+ PFINT_OICR_PCI_EXCEPTION_M |
+ PFINT_OICR_GPIO_M |
+ PFINT_OICR_STORM_DETECT_M |
+ PFINT_OICR_HMC_ERR_M);
+
+ wr32(hw, PFINT_OICR_ENA, val);
+
+ /* SW_ITR_IDX = 0, but don't change INTENA */
+ wr32(hw, GLINT_DYN_CTL(pf->oicr_idx),
+ GLINT_DYN_CTL_SW_ITR_INDX_M | GLINT_DYN_CTL_INTENA_MSK_M);
+}
+
+/**
+ * ice_misc_intr - misc interrupt handler
+ * @irq: interrupt number
+ * @data: pointer to a q_vector
+ */
+static irqreturn_t ice_misc_intr(int __always_unused irq, void *data)
+{
+ struct ice_pf *pf = (struct ice_pf *)data;
+ struct ice_hw *hw = &pf->hw;
+ irqreturn_t ret = IRQ_NONE;
+ u32 oicr, ena_mask;
+
+ set_bit(__ICE_ADMINQ_EVENT_PENDING, pf->state);
+
+ oicr = rd32(hw, PFINT_OICR);
+ ena_mask = rd32(hw, PFINT_OICR_ENA);
+
+ if (!(oicr & PFINT_OICR_INTEVENT_M))
+ goto ena_intr;
+
+ if (oicr & PFINT_OICR_GRST_M) {
+ u32 reset;
+ /* we have a reset warning */
+ ena_mask &= ~PFINT_OICR_GRST_M;
+ reset = (rd32(hw, GLGEN_RSTAT) & GLGEN_RSTAT_RESET_TYPE_M) >>
+ GLGEN_RSTAT_RESET_TYPE_S;
+
+ if (reset == ICE_RESET_CORER)
+ pf->corer_count++;
+ else if (reset == ICE_RESET_GLOBR)
+ pf->globr_count++;
+ else
+ pf->empr_count++;
+
+ /* If a reset cycle isn't already in progress, we set a bit in
+ * pf->state so that the service task can start a reset/rebuild.
+ * We also make note of which reset happened so that peer
+ * devices/drivers can be informed.
+ */
+ if (!test_bit(__ICE_RESET_RECOVERY_PENDING, pf->state)) {
+ if (reset == ICE_RESET_CORER)
+ set_bit(__ICE_CORER_RECV, pf->state);
+ else if (reset == ICE_RESET_GLOBR)
+ set_bit(__ICE_GLOBR_RECV, pf->state);
+ else
+ set_bit(__ICE_EMPR_RECV, pf->state);
+
+ set_bit(__ICE_RESET_RECOVERY_PENDING, pf->state);
+ }
+ }
+
+ if (oicr & PFINT_OICR_HMC_ERR_M) {
+ ena_mask &= ~PFINT_OICR_HMC_ERR_M;
+ dev_dbg(&pf->pdev->dev,
+ "HMC Error interrupt - info 0x%x, data 0x%x\n",
+ rd32(hw, PFHMC_ERRORINFO),
+ rd32(hw, PFHMC_ERRORDATA));
+ }
+
+ /* Report and mask off any remaining unexpected interrupts */
+ oicr &= ena_mask;
+ if (oicr) {
+ dev_dbg(&pf->pdev->dev, "unhandled interrupt oicr=0x%08x\n",
+ oicr);
+ /* If a critical error is pending there is no choice but to
+ * reset the device.
+ */
+ if (oicr & (PFINT_OICR_PE_CRITERR_M |
+ PFINT_OICR_PCI_EXCEPTION_M |
+ PFINT_OICR_ECC_ERR_M)) {
+ set_bit(__ICE_PFR_REQ, pf->state);
+ ice_service_task_schedule(pf);
+ }
+ ena_mask &= ~oicr;
+ }
+ ret = IRQ_HANDLED;
+
+ena_intr:
+ /* re-enable interrupt causes that are not handled during this pass */
+ wr32(hw, PFINT_OICR_ENA, ena_mask);
+ if (!test_bit(__ICE_DOWN, pf->state)) {
+ ice_service_task_schedule(pf);
+ ice_irq_dynamic_ena(hw, NULL, NULL);
+ }
+
+ return ret;
+}
+
+/**
+ * ice_vsi_map_rings_to_vectors - Map VSI rings to interrupt vectors
+ * @vsi: the VSI being configured
+ *
+ * This function maps descriptor rings to the queue-specific vectors allotted
+ * through the MSI-X enabling code. On a constrained vector budget, we map Tx
+ * and Rx rings to the vector as "efficiently" as possible.
+ */
+static void ice_vsi_map_rings_to_vectors(struct ice_vsi *vsi)
+{
+ int q_vectors = vsi->num_q_vectors;
+ int tx_rings_rem, rx_rings_rem;
+ int v_id;
+
+ /* initially assigning remaining rings count to VSIs num queue value */
+ tx_rings_rem = vsi->num_txq;
+ rx_rings_rem = vsi->num_rxq;
+
+ for (v_id = 0; v_id < q_vectors; v_id++) {
+ struct ice_q_vector *q_vector = vsi->q_vectors[v_id];
+ int tx_rings_per_v, rx_rings_per_v, q_id, q_base;
+
+ /* Tx rings mapping to vector */
+ tx_rings_per_v = DIV_ROUND_UP(tx_rings_rem, q_vectors - v_id);
+ q_vector->num_ring_tx = tx_rings_per_v;
+ q_vector->tx.ring = NULL;
+ q_base = vsi->num_txq - tx_rings_rem;
+
+ for (q_id = q_base; q_id < (q_base + tx_rings_per_v); q_id++) {
+ struct ice_ring *tx_ring = vsi->tx_rings[q_id];
+
+ tx_ring->q_vector = q_vector;
+ tx_ring->next = q_vector->tx.ring;
+ q_vector->tx.ring = tx_ring;
+ }
+ tx_rings_rem -= tx_rings_per_v;
+
+ /* Rx rings mapping to vector */
+ rx_rings_per_v = DIV_ROUND_UP(rx_rings_rem, q_vectors - v_id);
+ q_vector->num_ring_rx = rx_rings_per_v;
+ q_vector->rx.ring = NULL;
+ q_base = vsi->num_rxq - rx_rings_rem;
+
+ for (q_id = q_base; q_id < (q_base + rx_rings_per_v); q_id++) {
+ struct ice_ring *rx_ring = vsi->rx_rings[q_id];
+
+ rx_ring->q_vector = q_vector;
+ rx_ring->next = q_vector->rx.ring;
+ q_vector->rx.ring = rx_ring;
+ }
+ rx_rings_rem -= rx_rings_per_v;
+ }
+}
+
+/**
+ * ice_vsi_set_num_qs - Set num queues, descriptors and vectors for a VSI
+ * @vsi: the VSI being configured
+ *
+ * Return 0 on success and a negative value on error
+ */
+static void ice_vsi_set_num_qs(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ vsi->alloc_txq = pf->num_lan_tx;
+ vsi->alloc_rxq = pf->num_lan_rx;
+ vsi->num_desc = ALIGN(ICE_DFLT_NUM_DESC, ICE_REQ_DESC_MULTIPLE);
+ vsi->num_q_vectors = max_t(int, pf->num_lan_rx, pf->num_lan_tx);
+ break;
+ default:
+ dev_warn(&vsi->back->pdev->dev, "Unknown VSI type %d\n",
+ vsi->type);
+ break;
+ }
+}
+
+/**
+ * ice_vsi_alloc_arrays - Allocate queue and vector pointer arrays for the vsi
+ * @vsi: VSI pointer
+ * @alloc_qvectors: a bool to specify if q_vectors need to be allocated.
+ *
+ * On error: returns error code (negative)
+ * On success: returns 0
+ */
+static int ice_vsi_alloc_arrays(struct ice_vsi *vsi, bool alloc_qvectors)
+{
+ struct ice_pf *pf = vsi->back;
+
+ /* allocate memory for both Tx and Rx ring pointers */
+ vsi->tx_rings = devm_kcalloc(&pf->pdev->dev, vsi->alloc_txq,
+ sizeof(struct ice_ring *), GFP_KERNEL);
+ if (!vsi->tx_rings)
+ goto err_txrings;
+
+ vsi->rx_rings = devm_kcalloc(&pf->pdev->dev, vsi->alloc_rxq,
+ sizeof(struct ice_ring *), GFP_KERNEL);
+ if (!vsi->rx_rings)
+ goto err_rxrings;
+
+ if (alloc_qvectors) {
+ /* allocate memory for q_vector pointers */
+ vsi->q_vectors = devm_kcalloc(&pf->pdev->dev,
+ vsi->num_q_vectors,
+ sizeof(struct ice_q_vector *),
+ GFP_KERNEL);
+ if (!vsi->q_vectors)
+ goto err_vectors;
+ }
+
+ return 0;
+
+err_vectors:
+ devm_kfree(&pf->pdev->dev, vsi->rx_rings);
+err_rxrings:
+ devm_kfree(&pf->pdev->dev, vsi->tx_rings);
+err_txrings:
+ return -ENOMEM;
+}
+
+/**
+ * ice_msix_clean_rings - MSIX mode Interrupt Handler
+ * @irq: interrupt number
+ * @data: pointer to a q_vector
+ */
+static irqreturn_t ice_msix_clean_rings(int __always_unused irq, void *data)
+{
+ struct ice_q_vector *q_vector = (struct ice_q_vector *)data;
+
+ if (!q_vector->tx.ring && !q_vector->rx.ring)
+ return IRQ_HANDLED;
+
+ napi_schedule(&q_vector->napi);
+
+ return IRQ_HANDLED;
+}
+
+/**
+ * ice_vsi_alloc - Allocates the next available struct vsi in the PF
+ * @pf: board private structure
+ * @type: type of VSI
+ *
+ * returns a pointer to a VSI on success, NULL on failure.
+ */
+static struct ice_vsi *ice_vsi_alloc(struct ice_pf *pf, enum ice_vsi_type type)
+{
+ struct ice_vsi *vsi = NULL;
+
+ /* Need to protect the allocation of the VSIs at the PF level */
+ mutex_lock(&pf->sw_mutex);
+
+ /* If we have already allocated our maximum number of VSIs,
+ * pf->next_vsi will be ICE_NO_VSI. If not, pf->next_vsi index
+ * is available to be populated
+ */
+ if (pf->next_vsi == ICE_NO_VSI) {
+ dev_dbg(&pf->pdev->dev, "out of VSI slots!\n");
+ goto unlock_pf;
+ }
+
+ vsi = devm_kzalloc(&pf->pdev->dev, sizeof(*vsi), GFP_KERNEL);
+ if (!vsi)
+ goto unlock_pf;
+
+ vsi->type = type;
+ vsi->back = pf;
+ set_bit(__ICE_DOWN, vsi->state);
+ vsi->idx = pf->next_vsi;
+ vsi->work_lmt = ICE_DFLT_IRQ_WORK;
+
+ ice_vsi_set_num_qs(vsi);
+
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ if (ice_vsi_alloc_arrays(vsi, true))
+ goto err_rings;
+
+ /* Setup default MSIX irq handler for VSI */
+ vsi->irq_handler = ice_msix_clean_rings;
+ break;
+ default:
+ dev_warn(&pf->pdev->dev, "Unknown VSI type %d\n", vsi->type);
+ goto unlock_pf;
+ }
+
+ /* fill VSI slot in the PF struct */
+ pf->vsi[pf->next_vsi] = vsi;
+
+ /* prepare pf->next_vsi for next use */
+ pf->next_vsi = ice_get_free_slot(pf->vsi, pf->num_alloc_vsi,
+ pf->next_vsi);
+ goto unlock_pf;
+
+err_rings:
+ devm_kfree(&pf->pdev->dev, vsi);
+ vsi = NULL;
+unlock_pf:
+ mutex_unlock(&pf->sw_mutex);
+ return vsi;
+}
+
+/**
+ * ice_free_irq_msix_misc - Unroll misc vector setup
+ * @pf: board private structure
+ */
+static void ice_free_irq_msix_misc(struct ice_pf *pf)
+{
+ /* disable OICR interrupt */
+ wr32(&pf->hw, PFINT_OICR_ENA, 0);
+ ice_flush(&pf->hw);
+
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags) && pf->msix_entries) {
+ synchronize_irq(pf->msix_entries[pf->oicr_idx].vector);
+ devm_free_irq(&pf->pdev->dev,
+ pf->msix_entries[pf->oicr_idx].vector, pf);
+ }
+
+ ice_free_res(pf->irq_tracker, pf->oicr_idx, ICE_RES_MISC_VEC_ID);
+}
+
+/**
+ * ice_req_irq_msix_misc - Setup the misc vector to handle non queue events
+ * @pf: board private structure
+ *
+ * This sets up the handler for MSIX 0, which is used to manage the
+ * non-queue interrupts, e.g. AdminQ and errors. This is not used
+ * when in MSI or Legacy interrupt mode.
+ */
+static int ice_req_irq_msix_misc(struct ice_pf *pf)
+{
+ struct ice_hw *hw = &pf->hw;
+ int oicr_idx, err = 0;
+ u8 itr_gran;
+ u32 val;
+
+ if (!pf->int_name[0])
+ snprintf(pf->int_name, sizeof(pf->int_name) - 1, "%s-%s:misc",
+ dev_driver_string(&pf->pdev->dev),
+ dev_name(&pf->pdev->dev));
+
+ /* Do not request IRQ but do enable OICR interrupt since settings are
+ * lost during reset. Note that this function is called only during
+ * rebuild path and not while reset is in progress.
+ */
+ if (ice_is_reset_recovery_pending(pf->state))
+ goto skip_req_irq;
+
+ /* reserve one vector in irq_tracker for misc interrupts */
+ oicr_idx = ice_get_res(pf, pf->irq_tracker, 1, ICE_RES_MISC_VEC_ID);
+ if (oicr_idx < 0)
+ return oicr_idx;
+
+ pf->oicr_idx = oicr_idx;
+
+ err = devm_request_irq(&pf->pdev->dev,
+ pf->msix_entries[pf->oicr_idx].vector,
+ ice_misc_intr, 0, pf->int_name, pf);
+ if (err) {
+ dev_err(&pf->pdev->dev,
+ "devm_request_irq for %s failed: %d\n",
+ pf->int_name, err);
+ ice_free_res(pf->irq_tracker, 1, ICE_RES_MISC_VEC_ID);
+ return err;
+ }
+
+skip_req_irq:
+ ice_ena_misc_vector(pf);
+
+ val = (pf->oicr_idx & PFINT_OICR_CTL_MSIX_INDX_M) |
+ (ICE_RX_ITR & PFINT_OICR_CTL_ITR_INDX_M) |
+ PFINT_OICR_CTL_CAUSE_ENA_M;
+ wr32(hw, PFINT_OICR_CTL, val);
+
+ /* This enables Admin queue Interrupt causes */
+ val = (pf->oicr_idx & PFINT_FW_CTL_MSIX_INDX_M) |
+ (ICE_RX_ITR & PFINT_FW_CTL_ITR_INDX_M) |
+ PFINT_FW_CTL_CAUSE_ENA_M;
+ wr32(hw, PFINT_FW_CTL, val);
+
+ itr_gran = hw->itr_gran_200;
+
+ wr32(hw, GLINT_ITR(ICE_RX_ITR, pf->oicr_idx),
+ ITR_TO_REG(ICE_ITR_8K, itr_gran));
+
+ ice_flush(hw);
+ ice_irq_dynamic_ena(hw, NULL, NULL);
+
+ return 0;
+}
+
+/**
+ * ice_vsi_get_qs_contig - Assign a contiguous chunk of queues to VSI
+ * @vsi: the VSI getting queues
+ *
+ * Return 0 on success and a negative value on error
+ */
+static int ice_vsi_get_qs_contig(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int offset, ret = 0;
+
+ mutex_lock(&pf->avail_q_mutex);
+ /* look for contiguous block of queues for tx */
+ offset = bitmap_find_next_zero_area(pf->avail_txqs, ICE_MAX_TXQS,
+ 0, vsi->alloc_txq, 0);
+ if (offset < ICE_MAX_TXQS) {
+ int i;
+
+ bitmap_set(pf->avail_txqs, offset, vsi->alloc_txq);
+ for (i = 0; i < vsi->alloc_txq; i++)
+ vsi->txq_map[i] = i + offset;
+ } else {
+ ret = -ENOMEM;
+ vsi->tx_mapping_mode = ICE_VSI_MAP_SCATTER;
+ }
+
+ /* look for contiguous block of queues for rx */
+ offset = bitmap_find_next_zero_area(pf->avail_rxqs, ICE_MAX_RXQS,
+ 0, vsi->alloc_rxq, 0);
+ if (offset < ICE_MAX_RXQS) {
+ int i;
+
+ bitmap_set(pf->avail_rxqs, offset, vsi->alloc_rxq);
+ for (i = 0; i < vsi->alloc_rxq; i++)
+ vsi->rxq_map[i] = i + offset;
+ } else {
+ ret = -ENOMEM;
+ vsi->rx_mapping_mode = ICE_VSI_MAP_SCATTER;
+ }
+ mutex_unlock(&pf->avail_q_mutex);
+
+ return ret;
+}
+
+/**
+ * ice_vsi_get_qs_scatter - Assign a scattered queues to VSI
+ * @vsi: the VSI getting queues
+ *
+ * Return 0 on success and a negative value on error
+ */
+static int ice_vsi_get_qs_scatter(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int i, index = 0;
+
+ mutex_lock(&pf->avail_q_mutex);
+
+ if (vsi->tx_mapping_mode == ICE_VSI_MAP_SCATTER) {
+ for (i = 0; i < vsi->alloc_txq; i++) {
+ index = find_next_zero_bit(pf->avail_txqs,
+ ICE_MAX_TXQS, index);
+ if (index < ICE_MAX_TXQS) {
+ set_bit(index, pf->avail_txqs);
+ vsi->txq_map[i] = index;
+ } else {
+ goto err_scatter_tx;
+ }
+ }
+ }
+
+ if (vsi->rx_mapping_mode == ICE_VSI_MAP_SCATTER) {
+ for (i = 0; i < vsi->alloc_rxq; i++) {
+ index = find_next_zero_bit(pf->avail_rxqs,
+ ICE_MAX_RXQS, index);
+ if (index < ICE_MAX_RXQS) {
+ set_bit(index, pf->avail_rxqs);
+ vsi->rxq_map[i] = index;
+ } else {
+ goto err_scatter_rx;
+ }
+ }
+ }
+
+ mutex_unlock(&pf->avail_q_mutex);
+ return 0;
+
+err_scatter_rx:
+ /* unflag any queues we have grabbed (i is failed position) */
+ for (index = 0; index < i; index++) {
+ clear_bit(vsi->rxq_map[index], pf->avail_rxqs);
+ vsi->rxq_map[index] = 0;
+ }
+ i = vsi->alloc_txq;
+err_scatter_tx:
+ /* i is either position of failed attempt or vsi->alloc_txq */
+ for (index = 0; index < i; index++) {
+ clear_bit(vsi->txq_map[index], pf->avail_txqs);
+ vsi->txq_map[index] = 0;
+ }
+
+ mutex_unlock(&pf->avail_q_mutex);
+ return -ENOMEM;
+}
+
+/**
+ * ice_vsi_get_qs - Assign queues from PF to VSI
+ * @vsi: the VSI to assign queues to
+ *
+ * Returns 0 on success and a negative value on error
+ */
+static int ice_vsi_get_qs(struct ice_vsi *vsi)
+{
+ int ret = 0;
+
+ vsi->tx_mapping_mode = ICE_VSI_MAP_CONTIG;
+ vsi->rx_mapping_mode = ICE_VSI_MAP_CONTIG;
+
+ /* NOTE: ice_vsi_get_qs_contig() will set the rx/tx mapping
+ * modes individually to scatter if assigning contiguous queues
+ * to rx or tx fails
+ */
+ ret = ice_vsi_get_qs_contig(vsi);
+ if (ret < 0) {
+ if (vsi->tx_mapping_mode == ICE_VSI_MAP_SCATTER)
+ vsi->alloc_txq = max_t(u16, vsi->alloc_txq,
+ ICE_MAX_SCATTER_TXQS);
+ if (vsi->rx_mapping_mode == ICE_VSI_MAP_SCATTER)
+ vsi->alloc_rxq = max_t(u16, vsi->alloc_rxq,
+ ICE_MAX_SCATTER_RXQS);
+ ret = ice_vsi_get_qs_scatter(vsi);
+ }
+
+ return ret;
+}
+
+/**
+ * ice_vsi_put_qs - Release queues from VSI to PF
+ * @vsi: the VSI thats going to release queues
+ */
+static void ice_vsi_put_qs(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int i;
+
+ mutex_lock(&pf->avail_q_mutex);
+
+ for (i = 0; i < vsi->alloc_txq; i++) {
+ clear_bit(vsi->txq_map[i], pf->avail_txqs);
+ vsi->txq_map[i] = ICE_INVAL_Q_INDEX;
+ }
+
+ for (i = 0; i < vsi->alloc_rxq; i++) {
+ clear_bit(vsi->rxq_map[i], pf->avail_rxqs);
+ vsi->rxq_map[i] = ICE_INVAL_Q_INDEX;
+ }
+
+ mutex_unlock(&pf->avail_q_mutex);
+}
+
+/**
+ * ice_free_q_vector - Free memory allocated for a specific interrupt vector
+ * @vsi: VSI having the memory freed
+ * @v_idx: index of the vector to be freed
+ */
+static void ice_free_q_vector(struct ice_vsi *vsi, int v_idx)
+{
+ struct ice_q_vector *q_vector;
+ struct ice_ring *ring;
+
+ if (!vsi->q_vectors[v_idx]) {
+ dev_dbg(&vsi->back->pdev->dev, "Queue vector at index %d not found\n",
+ v_idx);
+ return;
+ }
+ q_vector = vsi->q_vectors[v_idx];
+
+ ice_for_each_ring(ring, q_vector->tx)
+ ring->q_vector = NULL;
+ ice_for_each_ring(ring, q_vector->rx)
+ ring->q_vector = NULL;
+
+ /* only VSI with an associated netdev is set up with NAPI */
+ if (vsi->netdev)
+ netif_napi_del(&q_vector->napi);
+
+ devm_kfree(&vsi->back->pdev->dev, q_vector);
+ vsi->q_vectors[v_idx] = NULL;
+}
+
+/**
+ * ice_vsi_free_q_vectors - Free memory allocated for interrupt vectors
+ * @vsi: the VSI having memory freed
+ */
+static void ice_vsi_free_q_vectors(struct ice_vsi *vsi)
+{
+ int v_idx;
+
+ for (v_idx = 0; v_idx < vsi->num_q_vectors; v_idx++)
+ ice_free_q_vector(vsi, v_idx);
+}
+
+/**
+ * ice_cfg_netdev - Setup the netdev flags
+ * @vsi: the VSI being configured
+ *
+ * Returns 0 on success, negative value on failure
+ */
+static int ice_cfg_netdev(struct ice_vsi *vsi)
+{
+ netdev_features_t csumo_features;
+ netdev_features_t vlano_features;
+ netdev_features_t dflt_features;
+ netdev_features_t tso_features;
+ struct ice_netdev_priv *np;
+ struct net_device *netdev;
+ u8 mac_addr[ETH_ALEN];
+
+ netdev = alloc_etherdev_mqs(sizeof(struct ice_netdev_priv),
+ vsi->alloc_txq, vsi->alloc_rxq);
+ if (!netdev)
+ return -ENOMEM;
+
+ vsi->netdev = netdev;
+ np = netdev_priv(netdev);
+ np->vsi = vsi;
+
+ dflt_features = NETIF_F_SG |
+ NETIF_F_HIGHDMA |
+ NETIF_F_RXHASH;
+
+ csumo_features = NETIF_F_RXCSUM |
+ NETIF_F_IP_CSUM |
+ NETIF_F_IPV6_CSUM;
+
+ vlano_features = NETIF_F_HW_VLAN_CTAG_FILTER |
+ NETIF_F_HW_VLAN_CTAG_TX |
+ NETIF_F_HW_VLAN_CTAG_RX;
+
+ tso_features = NETIF_F_TSO;
+
+ /* set features that user can change */
+ netdev->hw_features = dflt_features | csumo_features |
+ vlano_features | tso_features;
+
+ /* enable features */
+ netdev->features |= netdev->hw_features;
+ /* encap and VLAN devices inherit default, csumo and tso features */
+ netdev->hw_enc_features |= dflt_features | csumo_features |
+ tso_features;
+ netdev->vlan_features |= dflt_features | csumo_features |
+ tso_features;
+
+ if (vsi->type == ICE_VSI_PF) {
+ SET_NETDEV_DEV(netdev, &vsi->back->pdev->dev);
+ ether_addr_copy(mac_addr, vsi->port_info->mac.perm_addr);
+
+ ether_addr_copy(netdev->dev_addr, mac_addr);
+ ether_addr_copy(netdev->perm_addr, mac_addr);
+ }
+
+ netdev->priv_flags |= IFF_UNICAST_FLT;
+
+ /* assign netdev_ops */
+ netdev->netdev_ops = &ice_netdev_ops;
+
+ /* setup watchdog timeout value to be 5 second */
+ netdev->watchdog_timeo = 5 * HZ;
+
+ ice_set_ethtool_ops(netdev);
+
+ netdev->min_mtu = ETH_MIN_MTU;
+ netdev->max_mtu = ICE_MAX_MTU;
+
+ return 0;
+}
+
+/**
+ * ice_vsi_free_arrays - clean up vsi resources
+ * @vsi: pointer to VSI being cleared
+ * @free_qvectors: bool to specify if q_vectors should be deallocated
+ */
+static void ice_vsi_free_arrays(struct ice_vsi *vsi, bool free_qvectors)
+{
+ struct ice_pf *pf = vsi->back;
+
+ /* free the ring and vector containers */
+ if (free_qvectors && vsi->q_vectors) {
+ devm_kfree(&pf->pdev->dev, vsi->q_vectors);
+ vsi->q_vectors = NULL;
+ }
+ if (vsi->tx_rings) {
+ devm_kfree(&pf->pdev->dev, vsi->tx_rings);
+ vsi->tx_rings = NULL;
+ }
+ if (vsi->rx_rings) {
+ devm_kfree(&pf->pdev->dev, vsi->rx_rings);
+ vsi->rx_rings = NULL;
+ }
+}
+
+/**
+ * ice_vsi_clear - clean up and deallocate the provided vsi
+ * @vsi: pointer to VSI being cleared
+ *
+ * This deallocates the vsi's queue resources, removes it from the PF's
+ * VSI array if necessary, and deallocates the VSI
+ *
+ * Returns 0 on success, negative on failure
+ */
+static int ice_vsi_clear(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = NULL;
+
+ if (!vsi)
+ return 0;
+
+ if (!vsi->back)
+ return -EINVAL;
+
+ pf = vsi->back;
+
+ if (!pf->vsi[vsi->idx] || pf->vsi[vsi->idx] != vsi) {
+ dev_dbg(&pf->pdev->dev, "vsi does not exist at pf->vsi[%d]\n",
+ vsi->idx);
+ return -EINVAL;
+ }
+
+ mutex_lock(&pf->sw_mutex);
+ /* updates the PF for this cleared vsi */
+
+ pf->vsi[vsi->idx] = NULL;
+ if (vsi->idx < pf->next_vsi)
+ pf->next_vsi = vsi->idx;
+
+ ice_vsi_free_arrays(vsi, true);
+ mutex_unlock(&pf->sw_mutex);
+ devm_kfree(&pf->pdev->dev, vsi);
+
+ return 0;
+}
+
+/**
+ * ice_vsi_alloc_q_vector - Allocate memory for a single interrupt vector
+ * @vsi: the VSI being configured
+ * @v_idx: index of the vector in the vsi struct
+ *
+ * We allocate one q_vector. If allocation fails we return -ENOMEM.
+ */
+static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, int v_idx)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_q_vector *q_vector;
+
+ /* allocate q_vector */
+ q_vector = devm_kzalloc(&pf->pdev->dev, sizeof(*q_vector), GFP_KERNEL);
+ if (!q_vector)
+ return -ENOMEM;
+
+ q_vector->vsi = vsi;
+ q_vector->v_idx = v_idx;
+ /* only set affinity_mask if the CPU is online */
+ if (cpu_online(v_idx))
+ cpumask_set_cpu(v_idx, &q_vector->affinity_mask);
+
+ if (vsi->netdev)
+ netif_napi_add(vsi->netdev, &q_vector->napi, ice_napi_poll,
+ NAPI_POLL_WEIGHT);
+ /* tie q_vector and vsi together */
+ vsi->q_vectors[v_idx] = q_vector;
+
+ return 0;
+}
+
+/**
+ * ice_vsi_alloc_q_vectors - Allocate memory for interrupt vectors
+ * @vsi: the VSI being configured
+ *
+ * We allocate one q_vector per queue interrupt. If allocation fails we
+ * return -ENOMEM.
+ */
+static int ice_vsi_alloc_q_vectors(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int v_idx = 0, num_q_vectors;
+ int err;
+
+ if (vsi->q_vectors[0]) {
+ dev_dbg(&pf->pdev->dev, "VSI %d has existing q_vectors\n",
+ vsi->vsi_num);
+ return -EEXIST;
+ }
+
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags)) {
+ num_q_vectors = vsi->num_q_vectors;
+ } else {
+ err = -EINVAL;
+ goto err_out;
+ }
+
+ for (v_idx = 0; v_idx < num_q_vectors; v_idx++) {
+ err = ice_vsi_alloc_q_vector(vsi, v_idx);
+ if (err)
+ goto err_out;
+ }
+
+ return 0;
+
+err_out:
+ while (v_idx--)
+ ice_free_q_vector(vsi, v_idx);
+
+ dev_err(&pf->pdev->dev,
+ "Failed to allocate %d q_vector for VSI %d, ret=%d\n",
+ vsi->num_q_vectors, vsi->vsi_num, err);
+ vsi->num_q_vectors = 0;
+ return err;
+}
+
+/**
+ * ice_vsi_setup_vector_base - Set up the base vector for the given VSI
+ * @vsi: ptr to the VSI
+ *
+ * This should only be called after ice_vsi_alloc() which allocates the
+ * corresponding SW VSI structure and initializes num_queue_pairs for the
+ * newly allocated VSI.
+ *
+ * Returns 0 on success or negative on failure
+ */
+static int ice_vsi_setup_vector_base(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int num_q_vectors = 0;
+
+ if (vsi->base_vector) {
+ dev_dbg(&pf->pdev->dev, "VSI %d has non-zero base vector %d\n",
+ vsi->vsi_num, vsi->base_vector);
+ return -EEXIST;
+ }
+
+ if (!test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
+ return -ENOENT;
+
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ num_q_vectors = vsi->num_q_vectors;
+ break;
+ default:
+ dev_warn(&vsi->back->pdev->dev, "Unknown VSI type %d\n",
+ vsi->type);
+ break;
+ }
+
+ if (num_q_vectors)
+ vsi->base_vector = ice_get_res(pf, pf->irq_tracker,
+ num_q_vectors, vsi->idx);
+
+ if (vsi->base_vector < 0) {
+ dev_err(&pf->pdev->dev,
+ "Failed to get tracking for %d vectors for VSI %d, err=%d\n",
+ num_q_vectors, vsi->vsi_num, vsi->base_vector);
+ return -ENOENT;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_fill_rss_lut - Fill the RSS lookup table with default values
+ * @lut: Lookup table
+ * @rss_table_size: Lookup table size
+ * @rss_size: Range of queue number for hashing
+ */
+void ice_fill_rss_lut(u8 *lut, u16 rss_table_size, u16 rss_size)
+{
+ u16 i;
+
+ for (i = 0; i < rss_table_size; i++)
+ lut[i] = i % rss_size;
+}
+
+/**
+ * ice_vsi_cfg_rss - Configure RSS params for a VSI
+ * @vsi: VSI to be configured
+ */
+static int ice_vsi_cfg_rss(struct ice_vsi *vsi)
+{
+ u8 seed[ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE];
+ struct ice_aqc_get_set_rss_keys *key;
+ struct ice_pf *pf = vsi->back;
+ enum ice_status status;
+ int err = 0;
+ u8 *lut;
+
+ vsi->rss_size = min_t(int, vsi->rss_size, vsi->num_rxq);
+
+ lut = devm_kzalloc(&pf->pdev->dev, vsi->rss_table_size, GFP_KERNEL);
+ if (!lut)
+ return -ENOMEM;
+
+ if (vsi->rss_lut_user)
+ memcpy(lut, vsi->rss_lut_user, vsi->rss_table_size);
+ else
+ ice_fill_rss_lut(lut, vsi->rss_table_size, vsi->rss_size);
+
+ status = ice_aq_set_rss_lut(&pf->hw, vsi->vsi_num, vsi->rss_lut_type,
+ lut, vsi->rss_table_size);
+
+ if (status) {
+ dev_err(&vsi->back->pdev->dev,
+ "set_rss_lut failed, error %d\n", status);
+ err = -EIO;
+ goto ice_vsi_cfg_rss_exit;
+ }
+
+ key = devm_kzalloc(&vsi->back->pdev->dev, sizeof(*key), GFP_KERNEL);
+ if (!key) {
+ err = -ENOMEM;
+ goto ice_vsi_cfg_rss_exit;
+ }
+
+ if (vsi->rss_hkey_user)
+ memcpy(seed, vsi->rss_hkey_user,
+ ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE);
+ else
+ netdev_rss_key_fill((void *)seed,
+ ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE);
+ memcpy(&key->standard_rss_key, seed,
+ ICE_AQC_GET_SET_RSS_KEY_DATA_RSS_KEY_SIZE);
+
+ status = ice_aq_set_rss_key(&pf->hw, vsi->vsi_num, key);
+
+ if (status) {
+ dev_err(&vsi->back->pdev->dev, "set_rss_key failed, error %d\n",
+ status);
+ err = -EIO;
+ }
+
+ devm_kfree(&pf->pdev->dev, key);
+ice_vsi_cfg_rss_exit:
+ devm_kfree(&pf->pdev->dev, lut);
+ return err;
+}
+
+/**
+ * ice_vsi_reinit_setup - return resource and reallocate resource for a VSI
+ * @vsi: pointer to the ice_vsi
+ *
+ * This reallocates the VSIs queue resources
+ *
+ * Returns 0 on success and negative value on failure
+ */
+static int ice_vsi_reinit_setup(struct ice_vsi *vsi)
+{
+ u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };
+ int ret, i;
+
+ if (!vsi)
+ return -EINVAL;
+
+ ice_vsi_free_q_vectors(vsi);
+ ice_free_res(vsi->back->irq_tracker, vsi->base_vector, vsi->idx);
+ vsi->base_vector = 0;
+ ice_vsi_clear_rings(vsi);
+ ice_vsi_free_arrays(vsi, false);
+ ice_vsi_set_num_qs(vsi);
+
+ /* Initialize VSI struct elements and create VSI in FW */
+ ret = ice_vsi_add(vsi);
+ if (ret < 0)
+ goto err_vsi;
+
+ ret = ice_vsi_alloc_arrays(vsi, false);
+ if (ret < 0)
+ goto err_vsi;
+
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ if (!vsi->netdev) {
+ ret = ice_cfg_netdev(vsi);
+ if (ret)
+ goto err_rings;
+
+ ret = register_netdev(vsi->netdev);
+ if (ret)
+ goto err_rings;
+
+ netif_carrier_off(vsi->netdev);
+ netif_tx_stop_all_queues(vsi->netdev);
+ }
+
+ ret = ice_vsi_alloc_q_vectors(vsi);
+ if (ret)
+ goto err_rings;
+
+ ret = ice_vsi_setup_vector_base(vsi);
+ if (ret)
+ goto err_vectors;
+
+ ret = ice_vsi_alloc_rings(vsi);
+ if (ret)
+ goto err_vectors;
+
+ ice_vsi_map_rings_to_vectors(vsi);
+ break;
+ default:
+ break;
+ }
+
+ ice_vsi_set_tc_cfg(vsi);
+
+ /* configure VSI nodes based on number of queues and TC's */
+ for (i = 0; i < vsi->tc_cfg.numtc; i++)
+ max_txqs[i] = vsi->num_txq;
+
+ ret = ice_cfg_vsi_lan(vsi->port_info, vsi->vsi_num,
+ vsi->tc_cfg.ena_tc, max_txqs);
+ if (ret) {
+ dev_info(&vsi->back->pdev->dev,
+ "Failed VSI lan queue config\n");
+ goto err_vectors;
+ }
+ return 0;
+
+err_vectors:
+ ice_vsi_free_q_vectors(vsi);
+err_rings:
+ if (vsi->netdev) {
+ vsi->current_netdev_flags = 0;
+ unregister_netdev(vsi->netdev);
+ free_netdev(vsi->netdev);
+ vsi->netdev = NULL;
+ }
+err_vsi:
+ ice_vsi_clear(vsi);
+ set_bit(__ICE_RESET_FAILED, vsi->back->state);
+ return ret;
+}
+
+/**
+ * ice_vsi_setup - Set up a VSI by a given type
+ * @pf: board private structure
+ * @type: VSI type
+ * @pi: pointer to the port_info instance
+ *
+ * This allocates the sw VSI structure and its queue resources.
+ *
+ * Returns pointer to the successfully allocated and configure VSI sw struct on
+ * success, otherwise returns NULL on failure.
+ */
+static struct ice_vsi *
+ice_vsi_setup(struct ice_pf *pf, enum ice_vsi_type type,
+ struct ice_port_info *pi)
+{
+ u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };
+ struct device *dev = &pf->pdev->dev;
+ struct ice_vsi_ctx ctxt = { 0 };
+ struct ice_vsi *vsi;
+ int ret, i;
+
+ vsi = ice_vsi_alloc(pf, type);
+ if (!vsi) {
+ dev_err(dev, "could not allocate VSI\n");
+ return NULL;
+ }
+
+ vsi->port_info = pi;
+ vsi->vsw = pf->first_sw;
+
+ if (ice_vsi_get_qs(vsi)) {
+ dev_err(dev, "Failed to allocate queues. vsi->idx = %d\n",
+ vsi->idx);
+ goto err_get_qs;
+ }
+
+ /* set RSS capabilities */
+ ice_vsi_set_rss_params(vsi);
+
+ /* create the VSI */
+ ret = ice_vsi_add(vsi);
+ if (ret)
+ goto err_vsi;
+
+ ctxt.vsi_num = vsi->vsi_num;
+
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ ret = ice_cfg_netdev(vsi);
+ if (ret)
+ goto err_cfg_netdev;
+
+ ret = register_netdev(vsi->netdev);
+ if (ret)
+ goto err_register_netdev;
+
+ netif_carrier_off(vsi->netdev);
+
+ /* make sure transmit queues start off as stopped */
+ netif_tx_stop_all_queues(vsi->netdev);
+ ret = ice_vsi_alloc_q_vectors(vsi);
+ if (ret)
+ goto err_msix;
+
+ ret = ice_vsi_setup_vector_base(vsi);
+ if (ret)
+ goto err_rings;
+
+ ret = ice_vsi_alloc_rings(vsi);
+ if (ret)
+ goto err_rings;
+
+ ice_vsi_map_rings_to_vectors(vsi);
+
+ /* Do not exit if configuring RSS had an issue, at least
+ * receive traffic on first queue. Hence no need to capture
+ * return value
+ */
+ if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
+ ice_vsi_cfg_rss(vsi);
+ break;
+ default:
+ /* if vsi type is not recognized, clean up the resources and
+ * exit
+ */
+ goto err_rings;
+ }
+
+ ice_vsi_set_tc_cfg(vsi);
+
+ /* configure VSI nodes based on number of queues and TC's */
+ for (i = 0; i < vsi->tc_cfg.numtc; i++)
+ max_txqs[i] = vsi->num_txq;
+
+ ret = ice_cfg_vsi_lan(vsi->port_info, vsi->vsi_num,
+ vsi->tc_cfg.ena_tc, max_txqs);
+ if (ret) {
+ dev_info(&pf->pdev->dev, "Failed VSI lan queue config\n");
+ goto err_rings;
+ }
+
+ return vsi;
+
+err_rings:
+ ice_vsi_free_q_vectors(vsi);
+err_msix:
+ if (vsi->netdev && vsi->netdev->reg_state == NETREG_REGISTERED)
+ unregister_netdev(vsi->netdev);
+err_register_netdev:
+ if (vsi->netdev) {
+ free_netdev(vsi->netdev);
+ vsi->netdev = NULL;
+ }
+err_cfg_netdev:
+ ret = ice_aq_free_vsi(&pf->hw, &ctxt, false, NULL);
+ if (ret)
+ dev_err(&vsi->back->pdev->dev,
+ "Free VSI AQ call failed, err %d\n", ret);
+err_vsi:
+ ice_vsi_put_qs(vsi);
+err_get_qs:
+ pf->q_left_tx += vsi->alloc_txq;
+ pf->q_left_rx += vsi->alloc_rxq;
+ ice_vsi_clear(vsi);
+
+ return NULL;
+}
+
+/**
+ * ice_vsi_add_vlan - Add vsi membership for given vlan
+ * @vsi: the vsi being configured
+ * @vid: vlan id to be added
+ */
+static int ice_vsi_add_vlan(struct ice_vsi *vsi, u16 vid)
+{
+ struct ice_fltr_list_entry *tmp;
+ struct ice_pf *pf = vsi->back;
+ LIST_HEAD(tmp_add_list);
+ enum ice_status status;
+ int err = 0;
+
+ tmp = devm_kzalloc(&pf->pdev->dev, sizeof(*tmp), GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ tmp->fltr_info.lkup_type = ICE_SW_LKUP_VLAN;
+ tmp->fltr_info.fltr_act = ICE_FWD_TO_VSI;
+ tmp->fltr_info.flag = ICE_FLTR_TX;
+ tmp->fltr_info.src = vsi->vsi_num;
+ tmp->fltr_info.fwd_id.vsi_id = vsi->vsi_num;
+ tmp->fltr_info.l_data.vlan.vlan_id = vid;
+
+ INIT_LIST_HEAD(&tmp->list_entry);
+ list_add(&tmp->list_entry, &tmp_add_list);
+
+ status = ice_add_vlan(&pf->hw, &tmp_add_list);
+ if (status) {
+ err = -ENODEV;
+ dev_err(&pf->pdev->dev, "Failure Adding VLAN %d on VSI %i\n",
+ vid, vsi->vsi_num);
+ }
+
+ ice_free_fltr_list(&pf->pdev->dev, &tmp_add_list);
+ return err;
+}
+
+/**
+ * ice_vlan_rx_add_vid - Add a vlan id filter to HW offload
+ * @netdev: network interface to be adjusted
+ * @proto: unused protocol
+ * @vid: vlan id to be added
+ *
+ * net_device_ops implementation for adding vlan ids
+ */
+static int ice_vlan_rx_add_vid(struct net_device *netdev,
+ __always_unused __be16 proto, u16 vid)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ int ret = 0;
+
+ if (vid >= VLAN_N_VID) {
+ netdev_err(netdev, "VLAN id requested %d is out of range %d\n",
+ vid, VLAN_N_VID);
+ return -EINVAL;
+ }
+
+ if (vsi->info.pvid)
+ return -EINVAL;
+
+ /* Add all VLAN ids including 0 to the switch filter. VLAN id 0 is
+ * needed to continue allowing all untagged packets since VLAN prune
+ * list is applied to all packets by the switch
+ */
+ ret = ice_vsi_add_vlan(vsi, vid);
+
+ if (!ret)
+ set_bit(vid, vsi->active_vlans);
+
+ return ret;
+}
+
+/**
+ * ice_vsi_kill_vlan - Remove VSI membership for a given VLAN
+ * @vsi: the VSI being configured
+ * @vid: VLAN id to be removed
+ */
+static void ice_vsi_kill_vlan(struct ice_vsi *vsi, u16 vid)
+{
+ struct ice_fltr_list_entry *list;
+ struct ice_pf *pf = vsi->back;
+ LIST_HEAD(tmp_add_list);
+
+ list = devm_kzalloc(&pf->pdev->dev, sizeof(*list), GFP_KERNEL);
+ if (!list)
+ return;
+
+ list->fltr_info.lkup_type = ICE_SW_LKUP_VLAN;
+ list->fltr_info.fwd_id.vsi_id = vsi->vsi_num;
+ list->fltr_info.fltr_act = ICE_FWD_TO_VSI;
+ list->fltr_info.l_data.vlan.vlan_id = vid;
+ list->fltr_info.flag = ICE_FLTR_TX;
+ list->fltr_info.src = vsi->vsi_num;
+
+ INIT_LIST_HEAD(&list->list_entry);
+ list_add(&list->list_entry, &tmp_add_list);
+
+ if (ice_remove_vlan(&pf->hw, &tmp_add_list))
+ dev_err(&pf->pdev->dev, "Error removing VLAN %d on vsi %i\n",
+ vid, vsi->vsi_num);
+
+ ice_free_fltr_list(&pf->pdev->dev, &tmp_add_list);
+}
+
+/**
+ * ice_vlan_rx_kill_vid - Remove a vlan id filter from HW offload
+ * @netdev: network interface to be adjusted
+ * @proto: unused protocol
+ * @vid: vlan id to be removed
+ *
+ * net_device_ops implementation for removing vlan ids
+ */
+static int ice_vlan_rx_kill_vid(struct net_device *netdev,
+ __always_unused __be16 proto, u16 vid)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+
+ if (vsi->info.pvid)
+ return -EINVAL;
+
+ /* return code is ignored as there is nothing a user
+ * can do about failure to remove and a log message was
+ * already printed from the other function
+ */
+ ice_vsi_kill_vlan(vsi, vid);
+
+ clear_bit(vid, vsi->active_vlans);
+
+ return 0;
+}
+
+/**
+ * ice_setup_pf_sw - Setup the HW switch on startup or after reset
+ * @pf: board private structure
+ *
+ * Returns 0 on success, negative value on failure
+ */
+static int ice_setup_pf_sw(struct ice_pf *pf)
+{
+ LIST_HEAD(tmp_add_list);
+ u8 broadcast[ETH_ALEN];
+ struct ice_vsi *vsi;
+ int status = 0;
+
+ if (!ice_is_reset_recovery_pending(pf->state)) {
+ vsi = ice_vsi_setup(pf, ICE_VSI_PF, pf->hw.port_info);
+ if (!vsi) {
+ status = -ENOMEM;
+ goto error_exit;
+ }
+ } else {
+ vsi = pf->vsi[0];
+ status = ice_vsi_reinit_setup(vsi);
+ if (status < 0)
+ return -EIO;
+ }
+
+ /* tmp_add_list contains a list of MAC addresses for which MAC
+ * filters need to be programmed. Add the VSI's unicast MAC to
+ * this list
+ */
+ status = ice_add_mac_to_list(vsi, &tmp_add_list,
+ vsi->port_info->mac.perm_addr);
+ if (status)
+ goto error_exit;
+
+ /* VSI needs to receive broadcast traffic, so add the broadcast
+ * MAC address to the list.
+ */
+ eth_broadcast_addr(broadcast);
+ status = ice_add_mac_to_list(vsi, &tmp_add_list, broadcast);
+ if (status)
+ goto error_exit;
+
+ /* program MAC filters for entries in tmp_add_list */
+ status = ice_add_mac(&pf->hw, &tmp_add_list);
+ if (status) {
+ dev_err(&pf->pdev->dev, "Could not add MAC filters\n");
+ status = -ENOMEM;
+ goto error_exit;
+ }
+
+ ice_free_fltr_list(&pf->pdev->dev, &tmp_add_list);
+ return status;
+
+error_exit:
+ ice_free_fltr_list(&pf->pdev->dev, &tmp_add_list);
+
+ if (vsi) {
+ ice_vsi_free_q_vectors(vsi);
+ if (vsi->netdev && vsi->netdev->reg_state == NETREG_REGISTERED)
+ unregister_netdev(vsi->netdev);
+ if (vsi->netdev) {
+ free_netdev(vsi->netdev);
+ vsi->netdev = NULL;
+ }
+
+ ice_vsi_delete(vsi);
+ ice_vsi_put_qs(vsi);
+ pf->q_left_tx += vsi->alloc_txq;
+ pf->q_left_rx += vsi->alloc_rxq;
+ ice_vsi_clear(vsi);
+ }
+ return status;
+}
+
+/**
+ * ice_determine_q_usage - Calculate queue distribution
+ * @pf: board private structure
+ *
+ * Return -ENOMEM if we don't get enough queues for all ports
+ */
+static void ice_determine_q_usage(struct ice_pf *pf)
+{
+ u16 q_left_tx, q_left_rx;
+
+ q_left_tx = pf->hw.func_caps.common_cap.num_txq;
+ q_left_rx = pf->hw.func_caps.common_cap.num_rxq;
+
+ pf->num_lan_tx = min_t(int, q_left_tx, num_online_cpus());
+
+ /* only 1 rx queue unless RSS is enabled */
+ if (!test_bit(ICE_FLAG_RSS_ENA, pf->flags))
+ pf->num_lan_rx = 1;
+ else
+ pf->num_lan_rx = min_t(int, q_left_rx, num_online_cpus());
+
+ pf->q_left_tx = q_left_tx - pf->num_lan_tx;
+ pf->q_left_rx = q_left_rx - pf->num_lan_rx;
+}
+
+/**
+ * ice_deinit_pf - Unrolls initialziations done by ice_init_pf
+ * @pf: board private structure to initialize
+ */
+static void ice_deinit_pf(struct ice_pf *pf)
+{
+ if (pf->serv_tmr.function)
+ del_timer_sync(&pf->serv_tmr);
+ if (pf->serv_task.func)
+ cancel_work_sync(&pf->serv_task);
+ mutex_destroy(&pf->sw_mutex);
+ mutex_destroy(&pf->avail_q_mutex);
+}
+
+/**
+ * ice_init_pf - Initialize general software structures (struct ice_pf)
+ * @pf: board private structure to initialize
+ */
+static void ice_init_pf(struct ice_pf *pf)
+{
+ bitmap_zero(pf->flags, ICE_PF_FLAGS_NBITS);
+ set_bit(ICE_FLAG_MSIX_ENA, pf->flags);
+
+ mutex_init(&pf->sw_mutex);
+ mutex_init(&pf->avail_q_mutex);
+
+ /* Clear avail_[t|r]x_qs bitmaps (set all to avail) */
+ mutex_lock(&pf->avail_q_mutex);
+ bitmap_zero(pf->avail_txqs, ICE_MAX_TXQS);
+ bitmap_zero(pf->avail_rxqs, ICE_MAX_RXQS);
+ mutex_unlock(&pf->avail_q_mutex);
+
+ if (pf->hw.func_caps.common_cap.rss_table_size)
+ set_bit(ICE_FLAG_RSS_ENA, pf->flags);
+
+ /* setup service timer and periodic service task */
+ timer_setup(&pf->serv_tmr, ice_service_timer, 0);
+ pf->serv_tmr_period = HZ;
+ INIT_WORK(&pf->serv_task, ice_service_task);
+ clear_bit(__ICE_SERVICE_SCHED, pf->state);
+}
+
+/**
+ * ice_ena_msix_range - Request a range of MSIX vectors from the OS
+ * @pf: board private structure
+ *
+ * compute the number of MSIX vectors required (v_budget) and request from
+ * the OS. Return the number of vectors reserved or negative on failure
+ */
+static int ice_ena_msix_range(struct ice_pf *pf)
+{
+ int v_left, v_actual, v_budget = 0;
+ int needed, err, i;
+
+ v_left = pf->hw.func_caps.common_cap.num_msix_vectors;
+
+ /* reserve one vector for miscellaneous handler */
+ needed = 1;
+ v_budget += needed;
+ v_left -= needed;
+
+ /* reserve vectors for LAN traffic */
+ pf->num_lan_msix = min_t(int, num_online_cpus(), v_left);
+ v_budget += pf->num_lan_msix;
+
+ pf->msix_entries = devm_kcalloc(&pf->pdev->dev, v_budget,
+ sizeof(struct msix_entry), GFP_KERNEL);
+
+ if (!pf->msix_entries) {
+ err = -ENOMEM;
+ goto exit_err;
+ }
+
+ for (i = 0; i < v_budget; i++)
+ pf->msix_entries[i].entry = i;
+
+ /* actually reserve the vectors */
+ v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries,
+ ICE_MIN_MSIX, v_budget);
+
+ if (v_actual < 0) {
+ dev_err(&pf->pdev->dev, "unable to reserve MSI-X vectors\n");
+ err = v_actual;
+ goto msix_err;
+ }
+
+ if (v_actual < v_budget) {
+ dev_warn(&pf->pdev->dev,
+ "not enough vectors. requested = %d, obtained = %d\n",
+ v_budget, v_actual);
+ if (v_actual >= (pf->num_lan_msix + 1)) {
+ pf->num_avail_msix = v_actual - (pf->num_lan_msix + 1);
+ } else if (v_actual >= 2) {
+ pf->num_lan_msix = 1;
+ pf->num_avail_msix = v_actual - 2;
+ } else {
+ pci_disable_msix(pf->pdev);
+ err = -ERANGE;
+ goto msix_err;
+ }
+ }
+
+ return v_actual;
+
+msix_err:
+ devm_kfree(&pf->pdev->dev, pf->msix_entries);
+ goto exit_err;
+
+exit_err:
+ pf->num_lan_msix = 0;
+ clear_bit(ICE_FLAG_MSIX_ENA, pf->flags);
+ return err;
+}
+
+/**
+ * ice_dis_msix - Disable MSI-X interrupt setup in OS
+ * @pf: board private structure
+ */
+static void ice_dis_msix(struct ice_pf *pf)
+{
+ pci_disable_msix(pf->pdev);
+ devm_kfree(&pf->pdev->dev, pf->msix_entries);
+ pf->msix_entries = NULL;
+ clear_bit(ICE_FLAG_MSIX_ENA, pf->flags);
+}
+
+/**
+ * ice_init_interrupt_scheme - Determine proper interrupt scheme
+ * @pf: board private structure to initialize
+ */
+static int ice_init_interrupt_scheme(struct ice_pf *pf)
+{
+ int vectors = 0;
+ ssize_t size;
+
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
+ vectors = ice_ena_msix_range(pf);
+ else
+ return -ENODEV;
+
+ if (vectors < 0)
+ return vectors;
+
+ /* set up vector assignment tracking */
+ size = sizeof(struct ice_res_tracker) + (sizeof(u16) * vectors);
+
+ pf->irq_tracker = devm_kzalloc(&pf->pdev->dev, size, GFP_KERNEL);
+ if (!pf->irq_tracker) {
+ ice_dis_msix(pf);
+ return -ENOMEM;
+ }
+
+ pf->irq_tracker->num_entries = vectors;
+
+ return 0;
+}
+
+/**
+ * ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme
+ * @pf: board private structure
+ */
+static void ice_clear_interrupt_scheme(struct ice_pf *pf)
+{
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
+ ice_dis_msix(pf);
+
+ devm_kfree(&pf->pdev->dev, pf->irq_tracker);
+ pf->irq_tracker = NULL;
+}
+
+/**
+ * ice_probe - Device initialization routine
+ * @pdev: PCI device information struct
+ * @ent: entry in ice_pci_tbl
+ *
+ * Returns 0 on success, negative on failure
+ */
+static int ice_probe(struct pci_dev *pdev,
+ const struct pci_device_id __always_unused *ent)
+{
+ struct ice_pf *pf;
+ struct ice_hw *hw;
+ int err;
+
+ /* this driver uses devres, see Documentation/driver-model/devres.txt */
+ err = pcim_enable_device(pdev);
+ if (err)
+ return err;
+
+ err = pcim_iomap_regions(pdev, BIT(ICE_BAR0), pci_name(pdev));
+ if (err) {
+ dev_err(&pdev->dev, "I/O map error %d\n", err);
+ return err;
+ }
+
+ pf = devm_kzalloc(&pdev->dev, sizeof(*pf), GFP_KERNEL);
+ if (!pf)
+ return -ENOMEM;
+
+ /* set up for high or low dma */
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (err)
+ err = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(&pdev->dev, "DMA configuration failed: 0x%x\n", err);
+ return err;
+ }
+
+ pci_enable_pcie_error_reporting(pdev);
+ pci_set_master(pdev);
+
+ pf->pdev = pdev;
+ pci_set_drvdata(pdev, pf);
+ set_bit(__ICE_DOWN, pf->state);
+
+ hw = &pf->hw;
+ hw->hw_addr = pcim_iomap_table(pdev)[ICE_BAR0];
+ hw->back = pf;
+ hw->vendor_id = pdev->vendor;
+ hw->device_id = pdev->device;
+ pci_read_config_byte(pdev, PCI_REVISION_ID, &hw->revision_id);
+ hw->subsystem_vendor_id = pdev->subsystem_vendor;
+ hw->subsystem_device_id = pdev->subsystem_device;
+ hw->bus.device = PCI_SLOT(pdev->devfn);
+ hw->bus.func = PCI_FUNC(pdev->devfn);
+ ice_set_ctrlq_len(hw);
+
+ pf->msg_enable = netif_msg_init(debug, ICE_DFLT_NETIF_M);
+
+#ifndef CONFIG_DYNAMIC_DEBUG
+ if (debug < -1)
+ hw->debug_mask = debug;
+#endif
+
+ err = ice_init_hw(hw);
+ if (err) {
+ dev_err(&pdev->dev, "ice_init_hw failed: %d\n", err);
+ err = -EIO;
+ goto err_exit_unroll;
+ }
+
+ dev_info(&pdev->dev, "firmware %d.%d.%05d api %d.%d\n",
+ hw->fw_maj_ver, hw->fw_min_ver, hw->fw_build,
+ hw->api_maj_ver, hw->api_min_ver);
+
+ ice_init_pf(pf);
+
+ ice_determine_q_usage(pf);
+
+ pf->num_alloc_vsi = min_t(u16, ICE_MAX_VSI_ALLOC,
+ hw->func_caps.guaranteed_num_vsi);
+ if (!pf->num_alloc_vsi) {
+ err = -EIO;
+ goto err_init_pf_unroll;
+ }
+
+ pf->vsi = devm_kcalloc(&pdev->dev, pf->num_alloc_vsi,
+ sizeof(struct ice_vsi *), GFP_KERNEL);
+ if (!pf->vsi) {
+ err = -ENOMEM;
+ goto err_init_pf_unroll;
+ }
+
+ err = ice_init_interrupt_scheme(pf);
+ if (err) {
+ dev_err(&pdev->dev,
+ "ice_init_interrupt_scheme failed: %d\n", err);
+ err = -EIO;
+ goto err_init_interrupt_unroll;
+ }
+
+ /* In case of MSIX we are going to setup the misc vector right here
+ * to handle admin queue events etc. In case of legacy and MSI
+ * the misc functionality and queue processing is combined in
+ * the same vector and that gets setup at open.
+ */
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags)) {
+ err = ice_req_irq_msix_misc(pf);
+ if (err) {
+ dev_err(&pdev->dev,
+ "setup of misc vector failed: %d\n", err);
+ goto err_init_interrupt_unroll;
+ }
+ }
+
+ /* create switch struct for the switch element created by FW on boot */
+ pf->first_sw = devm_kzalloc(&pdev->dev, sizeof(struct ice_sw),
+ GFP_KERNEL);
+ if (!pf->first_sw) {
+ err = -ENOMEM;
+ goto err_msix_misc_unroll;
+ }
+
+ pf->first_sw->bridge_mode = BRIDGE_MODE_VEB;
+ pf->first_sw->pf = pf;
+
+ /* record the sw_id available for later use */
+ pf->first_sw->sw_id = hw->port_info->sw_id;
+
+ err = ice_setup_pf_sw(pf);
+ if (err) {
+ dev_err(&pdev->dev,
+ "probe failed due to setup pf switch:%d\n", err);
+ goto err_alloc_sw_unroll;
+ }
+
+ /* Driver is mostly up */
+ clear_bit(__ICE_DOWN, pf->state);
+
+ /* since everything is good, start the service timer */
+ mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period));
+
+ err = ice_init_link_events(pf->hw.port_info);
+ if (err) {
+ dev_err(&pdev->dev, "ice_init_link_events failed: %d\n", err);
+ goto err_alloc_sw_unroll;
+ }
+
+ return 0;
+
+err_alloc_sw_unroll:
+ set_bit(__ICE_DOWN, pf->state);
+ devm_kfree(&pf->pdev->dev, pf->first_sw);
+err_msix_misc_unroll:
+ ice_free_irq_msix_misc(pf);
+err_init_interrupt_unroll:
+ ice_clear_interrupt_scheme(pf);
+ devm_kfree(&pdev->dev, pf->vsi);
+err_init_pf_unroll:
+ ice_deinit_pf(pf);
+ ice_deinit_hw(hw);
+err_exit_unroll:
+ pci_disable_pcie_error_reporting(pdev);
+ return err;
+}
+
+/**
+ * ice_remove - Device removal routine
+ * @pdev: PCI device information struct
+ */
+static void ice_remove(struct pci_dev *pdev)
+{
+ struct ice_pf *pf = pci_get_drvdata(pdev);
+ int i = 0;
+ int err;
+
+ if (!pf)
+ return;
+
+ set_bit(__ICE_DOWN, pf->state);
+
+ for (i = 0; i < pf->num_alloc_vsi; i++) {
+ if (!pf->vsi[i])
+ continue;
+
+ err = ice_vsi_release(pf->vsi[i]);
+ if (err)
+ dev_dbg(&pf->pdev->dev, "Failed to release VSI index %d (err %d)\n",
+ i, err);
+ }
+
+ ice_free_irq_msix_misc(pf);
+ ice_clear_interrupt_scheme(pf);
+ ice_deinit_pf(pf);
+ ice_deinit_hw(&pf->hw);
+ pci_disable_pcie_error_reporting(pdev);
+}
+
+/* ice_pci_tbl - PCI Device ID Table
+ *
+ * Wildcard entries (PCI_ANY_ID) should come last
+ * Last entry must be all 0s
+ *
+ * { Vendor ID, Device ID, SubVendor ID, SubDevice ID,
+ * Class, Class Mask, private data (not used) }
+ */
+static const struct pci_device_id ice_pci_tbl[] = {
+ { PCI_VDEVICE(INTEL, ICE_DEV_ID_C810_BACKPLANE), 0 },
+ { PCI_VDEVICE(INTEL, ICE_DEV_ID_C810_QSFP), 0 },
+ { PCI_VDEVICE(INTEL, ICE_DEV_ID_C810_SFP), 0 },
+ { PCI_VDEVICE(INTEL, ICE_DEV_ID_C810_10G_BASE_T), 0 },
+ { PCI_VDEVICE(INTEL, ICE_DEV_ID_C810_SGMII), 0 },
+ /* required last entry */
+ { 0, }
+};
+MODULE_DEVICE_TABLE(pci, ice_pci_tbl);
+
+static struct pci_driver ice_driver = {
+ .name = KBUILD_MODNAME,
+ .id_table = ice_pci_tbl,
+ .probe = ice_probe,
+ .remove = ice_remove,
+};
+
+/**
+ * ice_module_init - Driver registration routine
+ *
+ * ice_module_init is the first routine called when the driver is
+ * loaded. All it does is register with the PCI subsystem.
+ */
+static int __init ice_module_init(void)
+{
+ int status;
+
+ pr_info("%s - version %s\n", ice_driver_string, ice_drv_ver);
+ pr_info("%s\n", ice_copyright);
+
+ ice_wq = alloc_ordered_workqueue("%s", WQ_MEM_RECLAIM, KBUILD_MODNAME);
+ if (!ice_wq) {
+ pr_err("Failed to create workqueue\n");
+ return -ENOMEM;
+ }
+
+ status = pci_register_driver(&ice_driver);
+ if (status) {
+ pr_err("failed to register pci driver, err %d\n", status);
+ destroy_workqueue(ice_wq);
+ }
+
+ return status;
+}
+module_init(ice_module_init);
+
+/**
+ * ice_module_exit - Driver exit cleanup routine
+ *
+ * ice_module_exit is called just before the driver is removed
+ * from memory.
+ */
+static void __exit ice_module_exit(void)
+{
+ pci_unregister_driver(&ice_driver);
+ destroy_workqueue(ice_wq);
+ pr_info("module unloaded\n");
+}
+module_exit(ice_module_exit);
+
+/**
+ * ice_set_mac_address - NDO callback to set mac address
+ * @netdev: network interface device structure
+ * @pi: pointer to an address structure
+ *
+ * Returns 0 on success, negative on failure
+ */
+static int ice_set_mac_address(struct net_device *netdev, void *pi)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ struct sockaddr *addr = pi;
+ enum ice_status status;
+ LIST_HEAD(a_mac_list);
+ LIST_HEAD(r_mac_list);
+ u8 flags = 0;
+ int err;
+ u8 *mac;
+
+ mac = (u8 *)addr->sa_data;
+
+ if (!is_valid_ether_addr(mac))
+ return -EADDRNOTAVAIL;
+
+ if (ether_addr_equal(netdev->dev_addr, mac)) {
+ netdev_warn(netdev, "already using mac %pM\n", mac);
+ return 0;
+ }
+
+ if (test_bit(__ICE_DOWN, pf->state) ||
+ ice_is_reset_recovery_pending(pf->state)) {
+ netdev_err(netdev, "can't set mac %pM. device not ready\n",
+ mac);
+ return -EBUSY;
+ }
+
+ /* When we change the mac address we also have to change the mac address
+ * based filter rules that were created previously for the old mac
+ * address. So first, we remove the old filter rule using ice_remove_mac
+ * and then create a new filter rule using ice_add_mac. Note that for
+ * both these operations, we first need to form a "list" of mac
+ * addresses (even though in this case, we have only 1 mac address to be
+ * added/removed) and this done using ice_add_mac_to_list. Depending on
+ * the ensuing operation this "list" of mac addresses is either to be
+ * added or removed from the filter.
+ */
+ err = ice_add_mac_to_list(vsi, &r_mac_list, netdev->dev_addr);
+ if (err) {
+ err = -EADDRNOTAVAIL;
+ goto free_lists;
+ }
+
+ status = ice_remove_mac(hw, &r_mac_list);
+ if (status) {
+ err = -EADDRNOTAVAIL;
+ goto free_lists;
+ }
+
+ err = ice_add_mac_to_list(vsi, &a_mac_list, mac);
+ if (err) {
+ err = -EADDRNOTAVAIL;
+ goto free_lists;
+ }
+
+ status = ice_add_mac(hw, &a_mac_list);
+ if (status) {
+ err = -EADDRNOTAVAIL;
+ goto free_lists;
+ }
+
+free_lists:
+ /* free list entries */
+ ice_free_fltr_list(&pf->pdev->dev, &r_mac_list);
+ ice_free_fltr_list(&pf->pdev->dev, &a_mac_list);
+
+ if (err) {
+ netdev_err(netdev, "can't set mac %pM. filter update failed\n",
+ mac);
+ return err;
+ }
+
+ /* change the netdev's mac address */
+ memcpy(netdev->dev_addr, mac, netdev->addr_len);
+ netdev_dbg(vsi->netdev, "updated mac address to %pM\n",
+ netdev->dev_addr);
+
+ /* write new mac address to the firmware */
+ flags = ICE_AQC_MAN_MAC_UPDATE_LAA_WOL;
+ status = ice_aq_manage_mac_write(hw, mac, flags, NULL);
+ if (status) {
+ netdev_err(netdev, "can't set mac %pM. write to firmware failed.\n",
+ mac);
+ }
+ return 0;
+}
+
+/**
+ * ice_set_rx_mode - NDO callback to set the netdev filters
+ * @netdev: network interface device structure
+ */
+static void ice_set_rx_mode(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+
+ if (!vsi)
+ return;
+
+ /* Set the flags to synchronize filters
+ * ndo_set_rx_mode may be triggered even without a change in netdev
+ * flags
+ */
+ set_bit(ICE_VSI_FLAG_UMAC_FLTR_CHANGED, vsi->flags);
+ set_bit(ICE_VSI_FLAG_MMAC_FLTR_CHANGED, vsi->flags);
+ set_bit(ICE_FLAG_FLTR_SYNC, vsi->back->flags);
+
+ /* schedule our worker thread which will take care of
+ * applying the new filter changes
+ */
+ ice_service_task_schedule(vsi->back);
+}
+
+/**
+ * ice_fdb_add - add an entry to the hardware database
+ * @ndm: the input from the stack
+ * @tb: pointer to array of nladdr (unused)
+ * @dev: the net device pointer
+ * @addr: the MAC address entry being added
+ * @vid: VLAN id
+ * @flags: instructions from stack about fdb operation
+ */
+static int ice_fdb_add(struct ndmsg *ndm, struct nlattr __always_unused *tb[],
+ struct net_device *dev, const unsigned char *addr,
+ u16 vid, u16 flags)
+{
+ int err;
+
+ if (vid) {
+ netdev_err(dev, "VLANs aren't supported yet for dev_uc|mc_add()\n");
+ return -EINVAL;
+ }
+ if (ndm->ndm_state && !(ndm->ndm_state & NUD_PERMANENT)) {
+ netdev_err(dev, "FDB only supports static addresses\n");
+ return -EINVAL;
+ }
+
+ if (is_unicast_ether_addr(addr) || is_link_local_ether_addr(addr))
+ err = dev_uc_add_excl(dev, addr);
+ else if (is_multicast_ether_addr(addr))
+ err = dev_mc_add_excl(dev, addr);
+ else
+ err = -EINVAL;
+
+ /* Only return duplicate errors if NLM_F_EXCL is set */
+ if (err == -EEXIST && !(flags & NLM_F_EXCL))
+ err = 0;
+
+ return err;
+}
+
+/**
+ * ice_fdb_del - delete an entry from the hardware database
+ * @ndm: the input from the stack
+ * @tb: pointer to array of nladdr (unused)
+ * @dev: the net device pointer
+ * @addr: the MAC address entry being added
+ * @vid: VLAN id
+ */
+static int ice_fdb_del(struct ndmsg *ndm, __always_unused struct nlattr *tb[],
+ struct net_device *dev, const unsigned char *addr,
+ __always_unused u16 vid)
+{
+ int err;
+
+ if (ndm->ndm_state & NUD_PERMANENT) {
+ netdev_err(dev, "FDB only supports static addresses\n");
+ return -EINVAL;
+ }
+
+ if (is_unicast_ether_addr(addr))
+ err = dev_uc_del(dev, addr);
+ else if (is_multicast_ether_addr(addr))
+ err = dev_mc_del(dev, addr);
+ else
+ err = -EINVAL;
+
+ return err;
+}
+
+/**
+ * ice_vsi_manage_vlan_insertion - Manage VLAN insertion for the VSI for Tx
+ * @vsi: the vsi being changed
+ */
+static int ice_vsi_manage_vlan_insertion(struct ice_vsi *vsi)
+{
+ struct device *dev = &vsi->back->pdev->dev;
+ struct ice_hw *hw = &vsi->back->hw;
+ struct ice_vsi_ctx ctxt = { 0 };
+ enum ice_status status;
+
+ /* Here we are configuring the VSI to let the driver add VLAN tags by
+ * setting port_vlan_flags to ICE_AQ_VSI_PVLAN_MODE_ALL. The actual VLAN
+ * tag insertion happens in the Tx hot path, in ice_tx_map.
+ */
+ ctxt.info.port_vlan_flags = ICE_AQ_VSI_PVLAN_MODE_ALL;
+
+ ctxt.info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_VLAN_VALID);
+ ctxt.vsi_num = vsi->vsi_num;
+
+ status = ice_aq_update_vsi(hw, &ctxt, NULL);
+ if (status) {
+ dev_err(dev, "update VSI for VLAN insert failed, err %d aq_err %d\n",
+ status, hw->adminq.sq_last_status);
+ return -EIO;
+ }
+
+ vsi->info.port_vlan_flags = ctxt.info.port_vlan_flags;
+ return 0;
+}
+
+/**
+ * ice_vsi_manage_vlan_stripping - Manage VLAN stripping for the VSI for Rx
+ * @vsi: the vsi being changed
+ * @ena: boolean value indicating if this is a enable or disable request
+ */
+static int ice_vsi_manage_vlan_stripping(struct ice_vsi *vsi, bool ena)
+{
+ struct device *dev = &vsi->back->pdev->dev;
+ struct ice_hw *hw = &vsi->back->hw;
+ struct ice_vsi_ctx ctxt = { 0 };
+ enum ice_status status;
+
+ /* Here we are configuring what the VSI should do with the VLAN tag in
+ * the Rx packet. We can either leave the tag in the packet or put it in
+ * the Rx descriptor.
+ */
+ if (ena) {
+ /* Strip VLAN tag from Rx packet and put it in the desc */
+ ctxt.info.port_vlan_flags = ICE_AQ_VSI_PVLAN_EMOD_STR_BOTH;
+ } else {
+ /* Disable stripping. Leave tag in packet */
+ ctxt.info.port_vlan_flags = ICE_AQ_VSI_PVLAN_EMOD_NOTHING;
+ }
+
+ ctxt.info.valid_sections = cpu_to_le16(ICE_AQ_VSI_PROP_VLAN_VALID);
+ ctxt.vsi_num = vsi->vsi_num;
+
+ status = ice_aq_update_vsi(hw, &ctxt, NULL);
+ if (status) {
+ dev_err(dev, "update VSI for VALN strip failed, ena = %d err %d aq_err %d\n",
+ ena, status, hw->adminq.sq_last_status);
+ return -EIO;
+ }
+
+ vsi->info.port_vlan_flags = ctxt.info.port_vlan_flags;
+ return 0;
+}
+
+/**
+ * ice_set_features - set the netdev feature flags
+ * @netdev: ptr to the netdev being adjusted
+ * @features: the feature set that the stack is suggesting
+ */
+static int ice_set_features(struct net_device *netdev,
+ netdev_features_t features)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ int ret = 0;
+
+ if ((features & NETIF_F_HW_VLAN_CTAG_RX) &&
+ !(netdev->features & NETIF_F_HW_VLAN_CTAG_RX))
+ ret = ice_vsi_manage_vlan_stripping(vsi, true);
+ else if (!(features & NETIF_F_HW_VLAN_CTAG_RX) &&
+ (netdev->features & NETIF_F_HW_VLAN_CTAG_RX))
+ ret = ice_vsi_manage_vlan_stripping(vsi, false);
+ else if ((features & NETIF_F_HW_VLAN_CTAG_TX) &&
+ !(netdev->features & NETIF_F_HW_VLAN_CTAG_TX))
+ ret = ice_vsi_manage_vlan_insertion(vsi);
+ else if (!(features & NETIF_F_HW_VLAN_CTAG_TX) &&
+ (netdev->features & NETIF_F_HW_VLAN_CTAG_TX))
+ ret = ice_vsi_manage_vlan_insertion(vsi);
+
+ return ret;
+}
+
+/**
+ * ice_vsi_vlan_setup - Setup vlan offload properties on a VSI
+ * @vsi: VSI to setup vlan properties for
+ */
+static int ice_vsi_vlan_setup(struct ice_vsi *vsi)
+{
+ int ret = 0;
+
+ if (vsi->netdev->features & NETIF_F_HW_VLAN_CTAG_RX)
+ ret = ice_vsi_manage_vlan_stripping(vsi, true);
+ if (vsi->netdev->features & NETIF_F_HW_VLAN_CTAG_TX)
+ ret = ice_vsi_manage_vlan_insertion(vsi);
+
+ return ret;
+}
+
+/**
+ * ice_restore_vlan - Reinstate VLANs when vsi/netdev comes back up
+ * @vsi: the VSI being brought back up
+ */
+static int ice_restore_vlan(struct ice_vsi *vsi)
+{
+ int err;
+ u16 vid;
+
+ if (!vsi->netdev)
+ return -EINVAL;
+
+ err = ice_vsi_vlan_setup(vsi);
+ if (err)
+ return err;
+
+ for_each_set_bit(vid, vsi->active_vlans, VLAN_N_VID) {
+ err = ice_vlan_rx_add_vid(vsi->netdev, htons(ETH_P_8021Q), vid);
+ if (err)
+ break;
+ }
+
+ return err;
+}
+
+/**
+ * ice_setup_tx_ctx - setup a struct ice_tlan_ctx instance
+ * @ring: The Tx ring to configure
+ * @tlan_ctx: Pointer to the Tx LAN queue context structure to be initialized
+ * @pf_q: queue index in the PF space
+ *
+ * Configure the Tx descriptor ring in TLAN context.
+ */
+static void
+ice_setup_tx_ctx(struct ice_ring *ring, struct ice_tlan_ctx *tlan_ctx, u16 pf_q)
+{
+ struct ice_vsi *vsi = ring->vsi;
+ struct ice_hw *hw = &vsi->back->hw;
+
+ tlan_ctx->base = ring->dma >> ICE_TLAN_CTX_BASE_S;
+
+ tlan_ctx->port_num = vsi->port_info->lport;
+
+ /* Transmit Queue Length */
+ tlan_ctx->qlen = ring->count;
+
+ /* PF number */
+ tlan_ctx->pf_num = hw->pf_id;
+
+ /* queue belongs to a specific VSI type
+ * VF / VM index should be programmed per vmvf_type setting:
+ * for vmvf_type = VF, it is VF number between 0-256
+ * for vmvf_type = VM, it is VM number between 0-767
+ * for PF or EMP this field should be set to zero
+ */
+ switch (vsi->type) {
+ case ICE_VSI_PF:
+ tlan_ctx->vmvf_type = ICE_TLAN_CTX_VMVF_TYPE_PF;
+ break;
+ default:
+ return;
+ }
+
+ /* make sure the context is associated with the right VSI */
+ tlan_ctx->src_vsi = vsi->vsi_num;
+
+ tlan_ctx->tso_ena = ICE_TX_LEGACY;
+ tlan_ctx->tso_qnum = pf_q;
+
+ /* Legacy or Advanced Host Interface:
+ * 0: Advanced Host Interface
+ * 1: Legacy Host Interface
+ */
+ tlan_ctx->legacy_int = ICE_TX_LEGACY;
+}
+
+/**
+ * ice_vsi_cfg_txqs - Configure the VSI for Tx
+ * @vsi: the VSI being configured
+ *
+ * Return 0 on success and a negative value on error
+ * Configure the Tx VSI for operation.
+ */
+static int ice_vsi_cfg_txqs(struct ice_vsi *vsi)
+{
+ struct ice_aqc_add_tx_qgrp *qg_buf;
+ struct ice_aqc_add_txqs_perq *txq;
+ struct ice_pf *pf = vsi->back;
+ enum ice_status status;
+ u16 buf_len, i, pf_q;
+ int err = 0, tc = 0;
+ u8 num_q_grps;
+
+ buf_len = sizeof(struct ice_aqc_add_tx_qgrp);
+ qg_buf = devm_kzalloc(&pf->pdev->dev, buf_len, GFP_KERNEL);
+ if (!qg_buf)
+ return -ENOMEM;
+
+ if (vsi->num_txq > ICE_MAX_TXQ_PER_TXQG) {
+ err = -EINVAL;
+ goto err_cfg_txqs;
+ }
+ qg_buf->num_txqs = 1;
+ num_q_grps = 1;
+
+ /* set up and configure the tx queues */
+ ice_for_each_txq(vsi, i) {
+ struct ice_tlan_ctx tlan_ctx = { 0 };
+
+ pf_q = vsi->txq_map[i];
+ ice_setup_tx_ctx(vsi->tx_rings[i], &tlan_ctx, pf_q);
+ /* copy context contents into the qg_buf */
+ qg_buf->txqs[0].txq_id = cpu_to_le16(pf_q);
+ ice_set_ctx((u8 *)&tlan_ctx, qg_buf->txqs[0].txq_ctx,
+ ice_tlan_ctx_info);
+
+ /* init queue specific tail reg. It is referred as transmit
+ * comm scheduler queue doorbell.
+ */
+ vsi->tx_rings[i]->tail = pf->hw.hw_addr + QTX_COMM_DBELL(pf_q);
+ status = ice_ena_vsi_txq(vsi->port_info, vsi->vsi_num, tc,
+ num_q_grps, qg_buf, buf_len, NULL);
+ if (status) {
+ dev_err(&vsi->back->pdev->dev,
+ "Failed to set LAN Tx queue context, error: %d\n",
+ status);
+ err = -ENODEV;
+ goto err_cfg_txqs;
+ }
+
+ /* Add Tx Queue TEID into the VSI tx ring from the response
+ * This will complete configuring and enabling the queue.
+ */
+ txq = &qg_buf->txqs[0];
+ if (pf_q == le16_to_cpu(txq->txq_id))
+ vsi->tx_rings[i]->txq_teid =
+ le32_to_cpu(txq->q_teid);
+ }
+err_cfg_txqs:
+ devm_kfree(&pf->pdev->dev, qg_buf);
+ return err;
+}
+
+/**
+ * ice_setup_rx_ctx - Configure a receive ring context
+ * @ring: The Rx ring to configure
+ *
+ * Configure the Rx descriptor ring in RLAN context.
+ */
+static int ice_setup_rx_ctx(struct ice_ring *ring)
+{
+ struct ice_vsi *vsi = ring->vsi;
+ struct ice_hw *hw = &vsi->back->hw;
+ u32 rxdid = ICE_RXDID_FLEX_NIC;
+ struct ice_rlan_ctx rlan_ctx;
+ u32 regval;
+ u16 pf_q;
+ int err;
+
+ /* what is RX queue number in global space of 2K rx queues */
+ pf_q = vsi->rxq_map[ring->q_index];
+
+ /* clear the context structure first */
+ memset(&rlan_ctx, 0, sizeof(rlan_ctx));
+
+ rlan_ctx.base = ring->dma >> 7;
+
+ rlan_ctx.qlen = ring->count;
+
+ /* Receive Packet Data Buffer Size.
+ * The Packet Data Buffer Size is defined in 128 byte units.
+ */
+ rlan_ctx.dbuf = vsi->rx_buf_len >> ICE_RLAN_CTX_DBUF_S;
+
+ /* use 32 byte descriptors */
+ rlan_ctx.dsize = 1;
+
+ /* Strip the Ethernet CRC bytes before the packet is posted to host
+ * memory.
+ */
+ rlan_ctx.crcstrip = 1;
+
+ /* L2TSEL flag defines the reported L2 Tags in the receive descriptor */
+ rlan_ctx.l2tsel = 1;
+
+ rlan_ctx.dtype = ICE_RX_DTYPE_NO_SPLIT;
+ rlan_ctx.hsplit_0 = ICE_RLAN_RX_HSPLIT_0_NO_SPLIT;
+ rlan_ctx.hsplit_1 = ICE_RLAN_RX_HSPLIT_1_NO_SPLIT;
+
+ /* This controls whether VLAN is stripped from inner headers
+ * The VLAN in the inner L2 header is stripped to the receive
+ * descriptor if enabled by this flag.
+ */
+ rlan_ctx.showiv = 0;
+
+ /* Max packet size for this queue - must not be set to a larger value
+ * than 5 x DBUF
+ */
+ rlan_ctx.rxmax = min_t(u16, vsi->max_frame,
+ ICE_MAX_CHAINED_RX_BUFS * vsi->rx_buf_len);
+
+ /* Rx queue threshold in units of 64 */
+ rlan_ctx.lrxqthresh = 1;
+
+ /* Enable Flexible Descriptors in the queue context which
+ * allows this driver to select a specific receive descriptor format
+ */
+ regval = rd32(hw, QRXFLXP_CNTXT(pf_q));
+ regval |= (rxdid << QRXFLXP_CNTXT_RXDID_IDX_S) &
+ QRXFLXP_CNTXT_RXDID_IDX_M;
+
+ /* increasing context priority to pick up profile id;
+ * default is 0x01; setting to 0x03 to ensure profile
+ * is programming if prev context is of same priority
+ */
+ regval |= (0x03 << QRXFLXP_CNTXT_RXDID_PRIO_S) &
+ QRXFLXP_CNTXT_RXDID_PRIO_M;
+
+ wr32(hw, QRXFLXP_CNTXT(pf_q), regval);
+
+ /* Absolute queue number out of 2K needs to be passed */
+ err = ice_write_rxq_ctx(hw, &rlan_ctx, pf_q);
+ if (err) {
+ dev_err(&vsi->back->pdev->dev,
+ "Failed to set LAN Rx queue context for absolute Rx queue %d error: %d\n",
+ pf_q, err);
+ return -EIO;
+ }
+
+ /* init queue specific tail register */
+ ring->tail = hw->hw_addr + QRX_TAIL(pf_q);
+ writel(0, ring->tail);
+ ice_alloc_rx_bufs(ring, ICE_DESC_UNUSED(ring));
+
+ return 0;
+}
+
+/**
+ * ice_vsi_cfg_rxqs - Configure the VSI for Rx
+ * @vsi: the VSI being configured
+ *
+ * Return 0 on success and a negative value on error
+ * Configure the Rx VSI for operation.
+ */
+static int ice_vsi_cfg_rxqs(struct ice_vsi *vsi)
+{
+ int err = 0;
+ u16 i;
+
+ if (vsi->netdev && vsi->netdev->mtu > ETH_DATA_LEN)
+ vsi->max_frame = vsi->netdev->mtu +
+ ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN;
+ else
+ vsi->max_frame = ICE_RXBUF_2048;
+
+ vsi->rx_buf_len = ICE_RXBUF_2048;
+ /* set up individual rings */
+ for (i = 0; i < vsi->num_rxq && !err; i++)
+ err = ice_setup_rx_ctx(vsi->rx_rings[i]);
+
+ if (err) {
+ dev_err(&vsi->back->pdev->dev, "ice_setup_rx_ctx failed\n");
+ return -EIO;
+ }
+ return err;
+}
+
+/**
+ * ice_vsi_cfg - Setup the VSI
+ * @vsi: the VSI being configured
+ *
+ * Return 0 on success and negative value on error
+ */
+static int ice_vsi_cfg(struct ice_vsi *vsi)
+{
+ int err;
+
+ ice_set_rx_mode(vsi->netdev);
+
+ err = ice_restore_vlan(vsi);
+ if (err)
+ return err;
+
+ err = ice_vsi_cfg_txqs(vsi);
+ if (!err)
+ err = ice_vsi_cfg_rxqs(vsi);
+
+ return err;
+}
+
+/**
+ * ice_vsi_stop_tx_rings - Disable Tx rings
+ * @vsi: the VSI being configured
+ */
+static int ice_vsi_stop_tx_rings(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ enum ice_status status;
+ u32 *q_teids, val;
+ u16 *q_ids, i;
+ int err = 0;
+
+ if (vsi->num_txq > ICE_LAN_TXQ_MAX_QDIS)
+ return -EINVAL;
+
+ q_teids = devm_kcalloc(&pf->pdev->dev, vsi->num_txq, sizeof(*q_teids),
+ GFP_KERNEL);
+ if (!q_teids)
+ return -ENOMEM;
+
+ q_ids = devm_kcalloc(&pf->pdev->dev, vsi->num_txq, sizeof(*q_ids),
+ GFP_KERNEL);
+ if (!q_ids) {
+ err = -ENOMEM;
+ goto err_alloc_q_ids;
+ }
+
+ /* set up the tx queue list to be disabled */
+ ice_for_each_txq(vsi, i) {
+ u16 v_idx;
+
+ if (!vsi->tx_rings || !vsi->tx_rings[i]) {
+ err = -EINVAL;
+ goto err_out;
+ }
+
+ q_ids[i] = vsi->txq_map[i];
+ q_teids[i] = vsi->tx_rings[i]->txq_teid;
+
+ /* clear cause_ena bit for disabled queues */
+ val = rd32(hw, QINT_TQCTL(vsi->tx_rings[i]->reg_idx));
+ val &= ~QINT_TQCTL_CAUSE_ENA_M;
+ wr32(hw, QINT_TQCTL(vsi->tx_rings[i]->reg_idx), val);
+
+ /* software is expected to wait for 100 ns */
+ ndelay(100);
+
+ /* trigger a software interrupt for the vector associated to
+ * the queue to schedule napi handler
+ */
+ v_idx = vsi->tx_rings[i]->q_vector->v_idx;
+ wr32(hw, GLINT_DYN_CTL(vsi->base_vector + v_idx),
+ GLINT_DYN_CTL_SWINT_TRIG_M | GLINT_DYN_CTL_INTENA_MSK_M);
+ }
+ status = ice_dis_vsi_txq(vsi->port_info, vsi->num_txq, q_ids, q_teids,
+ NULL);
+ if (status) {
+ dev_err(&pf->pdev->dev,
+ "Failed to disable LAN Tx queues, error: %d\n",
+ status);
+ err = -ENODEV;
+ }
+
+err_out:
+ devm_kfree(&pf->pdev->dev, q_ids);
+
+err_alloc_q_ids:
+ devm_kfree(&pf->pdev->dev, q_teids);
+
+ return err;
+}
+
+/**
+ * ice_pf_rxq_wait - Wait for a PF's Rx queue to be enabled or disabled
+ * @pf: the PF being configured
+ * @pf_q: the PF queue
+ * @ena: enable or disable state of the queue
+ *
+ * This routine will wait for the given Rx queue of the PF to reach the
+ * enabled or disabled state.
+ * Returns -ETIMEDOUT in case of failing to reach the requested state after
+ * multiple retries; else will return 0 in case of success.
+ */
+static int ice_pf_rxq_wait(struct ice_pf *pf, int pf_q, bool ena)
+{
+ int i;
+
+ for (i = 0; i < ICE_Q_WAIT_RETRY_LIMIT; i++) {
+ u32 rx_reg = rd32(&pf->hw, QRX_CTRL(pf_q));
+
+ if (ena == !!(rx_reg & QRX_CTRL_QENA_STAT_M))
+ break;
+
+ usleep_range(10, 20);
+ }
+ if (i >= ICE_Q_WAIT_RETRY_LIMIT)
+ return -ETIMEDOUT;
+
+ return 0;
+}
+
+/**
+ * ice_vsi_ctrl_rx_rings - Start or stop a VSI's rx rings
+ * @vsi: the VSI being configured
+ * @ena: start or stop the rx rings
+ */
+static int ice_vsi_ctrl_rx_rings(struct ice_vsi *vsi, bool ena)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ int i, j, ret = 0;
+
+ for (i = 0; i < vsi->num_rxq; i++) {
+ int pf_q = vsi->rxq_map[i];
+ u32 rx_reg;
+
+ for (j = 0; j < ICE_Q_WAIT_MAX_RETRY; j++) {
+ rx_reg = rd32(hw, QRX_CTRL(pf_q));
+ if (((rx_reg >> QRX_CTRL_QENA_REQ_S) & 1) ==
+ ((rx_reg >> QRX_CTRL_QENA_STAT_S) & 1))
+ break;
+ usleep_range(1000, 2000);
+ }
+
+ /* Skip if the queue is already in the requested state */
+ if (ena == !!(rx_reg & QRX_CTRL_QENA_STAT_M))
+ continue;
+
+ /* turn on/off the queue */
+ if (ena)
+ rx_reg |= QRX_CTRL_QENA_REQ_M;
+ else
+ rx_reg &= ~QRX_CTRL_QENA_REQ_M;
+ wr32(hw, QRX_CTRL(pf_q), rx_reg);
+
+ /* wait for the change to finish */
+ ret = ice_pf_rxq_wait(pf, pf_q, ena);
+ if (ret) {
+ dev_err(&pf->pdev->dev,
+ "VSI idx %d Rx ring %d %sable timeout\n",
+ vsi->idx, pf_q, (ena ? "en" : "dis"));
+ break;
+ }
+ }
+
+ return ret;
+}
+
+/**
+ * ice_vsi_start_rx_rings - start VSI's rx rings
+ * @vsi: the VSI whose rings are to be started
+ *
+ * Returns 0 on success and a negative value on error
+ */
+static int ice_vsi_start_rx_rings(struct ice_vsi *vsi)
+{
+ return ice_vsi_ctrl_rx_rings(vsi, true);
+}
+
+/**
+ * ice_vsi_stop_rx_rings - stop VSI's rx rings
+ * @vsi: the VSI
+ *
+ * Returns 0 on success and a negative value on error
+ */
+static int ice_vsi_stop_rx_rings(struct ice_vsi *vsi)
+{
+ return ice_vsi_ctrl_rx_rings(vsi, false);
+}
+
+/**
+ * ice_vsi_stop_tx_rx_rings - stop VSI's tx and rx rings
+ * @vsi: the VSI
+ * Returns 0 on success and a negative value on error
+ */
+static int ice_vsi_stop_tx_rx_rings(struct ice_vsi *vsi)
+{
+ int err_tx, err_rx;
+
+ err_tx = ice_vsi_stop_tx_rings(vsi);
+ if (err_tx)
+ dev_dbg(&vsi->back->pdev->dev, "Failed to disable Tx rings\n");
+
+ err_rx = ice_vsi_stop_rx_rings(vsi);
+ if (err_rx)
+ dev_dbg(&vsi->back->pdev->dev, "Failed to disable Rx rings\n");
+
+ if (err_tx || err_rx)
+ return -EIO;
+
+ return 0;
+}
+
+/**
+ * ice_napi_enable_all - Enable NAPI for all q_vectors in the VSI
+ * @vsi: the VSI being configured
+ */
+static void ice_napi_enable_all(struct ice_vsi *vsi)
+{
+ int q_idx;
+
+ if (!vsi->netdev)
+ return;
+
+ for (q_idx = 0; q_idx < vsi->num_q_vectors; q_idx++)
+ napi_enable(&vsi->q_vectors[q_idx]->napi);
+}
+
+/**
+ * ice_up_complete - Finish the last steps of bringing up a connection
+ * @vsi: The VSI being configured
+ *
+ * Return 0 on success and negative value on error
+ */
+static int ice_up_complete(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf = vsi->back;
+ int err;
+
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
+ ice_vsi_cfg_msix(vsi);
+ else
+ return -ENOTSUPP;
+
+ /* Enable only Rx rings, Tx rings were enabled by the FW when the
+ * Tx queue group list was configured and the context bits were
+ * programmed using ice_vsi_cfg_txqs
+ */
+ err = ice_vsi_start_rx_rings(vsi);
+ if (err)
+ return err;
+
+ clear_bit(__ICE_DOWN, vsi->state);
+ ice_napi_enable_all(vsi);
+ ice_vsi_ena_irq(vsi);
+
+ if (vsi->port_info &&
+ (vsi->port_info->phy.link_info.link_info & ICE_AQ_LINK_UP) &&
+ vsi->netdev) {
+ ice_print_link_msg(vsi, true);
+ netif_tx_start_all_queues(vsi->netdev);
+ netif_carrier_on(vsi->netdev);
+ }
+
+ ice_service_task_schedule(pf);
+
+ return err;
+}
+
+/**
+ * ice_up - Bring the connection back up after being down
+ * @vsi: VSI being configured
+ */
+int ice_up(struct ice_vsi *vsi)
+{
+ int err;
+
+ err = ice_vsi_cfg(vsi);
+ if (!err)
+ err = ice_up_complete(vsi);
+
+ return err;
+}
+
+/**
+ * ice_fetch_u64_stats_per_ring - get packets and bytes stats per ring
+ * @ring: Tx or Rx ring to read stats from
+ * @pkts: packets stats counter
+ * @bytes: bytes stats counter
+ *
+ * This function fetches stats from the ring considering the atomic operations
+ * that needs to be performed to read u64 values in 32 bit machine.
+ */
+static void ice_fetch_u64_stats_per_ring(struct ice_ring *ring, u64 *pkts,
+ u64 *bytes)
+{
+ unsigned int start;
+ *pkts = 0;
+ *bytes = 0;
+
+ if (!ring)
+ return;
+ do {
+ start = u64_stats_fetch_begin_irq(&ring->syncp);
+ *pkts = ring->stats.pkts;
+ *bytes = ring->stats.bytes;
+ } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+}
+
+/**
+ * ice_stat_update40 - read 40 bit stat from the chip and update stat values
+ * @hw: ptr to the hardware info
+ * @hireg: high 32 bit HW register to read from
+ * @loreg: low 32 bit HW register to read from
+ * @prev_stat_loaded: bool to specify if previous stats are loaded
+ * @prev_stat: ptr to previous loaded stat value
+ * @cur_stat: ptr to current stat value
+ */
+static void ice_stat_update40(struct ice_hw *hw, u32 hireg, u32 loreg,
+ bool prev_stat_loaded, u64 *prev_stat,
+ u64 *cur_stat)
+{
+ u64 new_data;
+
+ new_data = rd32(hw, loreg);
+ new_data |= ((u64)(rd32(hw, hireg) & 0xFFFF)) << 32;
+
+ /* device stats are not reset at PFR, they likely will not be zeroed
+ * when the driver starts. So save the first values read and use them as
+ * offsets to be subtracted from the raw values in order to report stats
+ * that count from zero.
+ */
+ if (!prev_stat_loaded)
+ *prev_stat = new_data;
+ if (likely(new_data >= *prev_stat))
+ *cur_stat = new_data - *prev_stat;
+ else
+ /* to manage the potential roll-over */
+ *cur_stat = (new_data + BIT_ULL(40)) - *prev_stat;
+ *cur_stat &= 0xFFFFFFFFFFULL;
+}
+
+/**
+ * ice_stat_update32 - read 32 bit stat from the chip and update stat values
+ * @hw: ptr to the hardware info
+ * @reg: HW register to read from
+ * @prev_stat_loaded: bool to specify if previous stats are loaded
+ * @prev_stat: ptr to previous loaded stat value
+ * @cur_stat: ptr to current stat value
+ */
+static void ice_stat_update32(struct ice_hw *hw, u32 reg, bool prev_stat_loaded,
+ u64 *prev_stat, u64 *cur_stat)
+{
+ u32 new_data;
+
+ new_data = rd32(hw, reg);
+
+ /* device stats are not reset at PFR, they likely will not be zeroed
+ * when the driver starts. So save the first values read and use them as
+ * offsets to be subtracted from the raw values in order to report stats
+ * that count from zero.
+ */
+ if (!prev_stat_loaded)
+ *prev_stat = new_data;
+ if (likely(new_data >= *prev_stat))
+ *cur_stat = new_data - *prev_stat;
+ else
+ /* to manage the potential roll-over */
+ *cur_stat = (new_data + BIT_ULL(32)) - *prev_stat;
+}
+
+/**
+ * ice_update_eth_stats - Update VSI-specific ethernet statistics counters
+ * @vsi: the VSI to be updated
+ */
+static void ice_update_eth_stats(struct ice_vsi *vsi)
+{
+ struct ice_eth_stats *prev_es, *cur_es;
+ struct ice_hw *hw = &vsi->back->hw;
+ u16 vsi_num = vsi->vsi_num; /* HW absolute index of a VSI */
+
+ prev_es = &vsi->eth_stats_prev;
+ cur_es = &vsi->eth_stats;
+
+ ice_stat_update40(hw, GLV_GORCH(vsi_num), GLV_GORCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->rx_bytes,
+ &cur_es->rx_bytes);
+
+ ice_stat_update40(hw, GLV_UPRCH(vsi_num), GLV_UPRCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->rx_unicast,
+ &cur_es->rx_unicast);
+
+ ice_stat_update40(hw, GLV_MPRCH(vsi_num), GLV_MPRCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->rx_multicast,
+ &cur_es->rx_multicast);
+
+ ice_stat_update40(hw, GLV_BPRCH(vsi_num), GLV_BPRCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->rx_broadcast,
+ &cur_es->rx_broadcast);
+
+ ice_stat_update32(hw, GLV_RDPC(vsi_num), vsi->stat_offsets_loaded,
+ &prev_es->rx_discards, &cur_es->rx_discards);
+
+ ice_stat_update40(hw, GLV_GOTCH(vsi_num), GLV_GOTCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->tx_bytes,
+ &cur_es->tx_bytes);
+
+ ice_stat_update40(hw, GLV_UPTCH(vsi_num), GLV_UPTCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->tx_unicast,
+ &cur_es->tx_unicast);
+
+ ice_stat_update40(hw, GLV_MPTCH(vsi_num), GLV_MPTCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->tx_multicast,
+ &cur_es->tx_multicast);
+
+ ice_stat_update40(hw, GLV_BPTCH(vsi_num), GLV_BPTCL(vsi_num),
+ vsi->stat_offsets_loaded, &prev_es->tx_broadcast,
+ &cur_es->tx_broadcast);
+
+ ice_stat_update32(hw, GLV_TEPC(vsi_num), vsi->stat_offsets_loaded,
+ &prev_es->tx_errors, &cur_es->tx_errors);
+
+ vsi->stat_offsets_loaded = true;
+}
+
+/**
+ * ice_update_vsi_ring_stats - Update VSI stats counters
+ * @vsi: the VSI to be updated
+ */
+static void ice_update_vsi_ring_stats(struct ice_vsi *vsi)
+{
+ struct rtnl_link_stats64 *vsi_stats = &vsi->net_stats;
+ struct ice_ring *ring;
+ u64 pkts, bytes;
+ int i;
+
+ /* reset netdev stats */
+ vsi_stats->tx_packets = 0;
+ vsi_stats->tx_bytes = 0;
+ vsi_stats->rx_packets = 0;
+ vsi_stats->rx_bytes = 0;
+
+ /* reset non-netdev (extended) stats */
+ vsi->tx_restart = 0;
+ vsi->tx_busy = 0;
+ vsi->tx_linearize = 0;
+ vsi->rx_buf_failed = 0;
+ vsi->rx_page_failed = 0;
+
+ rcu_read_lock();
+
+ /* update Tx rings counters */
+ ice_for_each_txq(vsi, i) {
+ ring = READ_ONCE(vsi->tx_rings[i]);
+ ice_fetch_u64_stats_per_ring(ring, &pkts, &bytes);
+ vsi_stats->tx_packets += pkts;
+ vsi_stats->tx_bytes += bytes;
+ vsi->tx_restart += ring->tx_stats.restart_q;
+ vsi->tx_busy += ring->tx_stats.tx_busy;
+ vsi->tx_linearize += ring->tx_stats.tx_linearize;
+ }
+
+ /* update Rx rings counters */
+ ice_for_each_rxq(vsi, i) {
+ ring = READ_ONCE(vsi->rx_rings[i]);
+ ice_fetch_u64_stats_per_ring(ring, &pkts, &bytes);
+ vsi_stats->rx_packets += pkts;
+ vsi_stats->rx_bytes += bytes;
+ vsi->rx_buf_failed += ring->rx_stats.alloc_buf_failed;
+ vsi->rx_page_failed += ring->rx_stats.alloc_page_failed;
+ }
+
+ rcu_read_unlock();
+}
+
+/**
+ * ice_update_vsi_stats - Update VSI stats counters
+ * @vsi: the VSI to be updated
+ */
+static void ice_update_vsi_stats(struct ice_vsi *vsi)
+{
+ struct rtnl_link_stats64 *cur_ns = &vsi->net_stats;
+ struct ice_eth_stats *cur_es = &vsi->eth_stats;
+ struct ice_pf *pf = vsi->back;
+
+ if (test_bit(__ICE_DOWN, vsi->state) ||
+ test_bit(__ICE_CFG_BUSY, pf->state))
+ return;
+
+ /* get stats as recorded by Tx/Rx rings */
+ ice_update_vsi_ring_stats(vsi);
+
+ /* get VSI stats as recorded by the hardware */
+ ice_update_eth_stats(vsi);
+
+ cur_ns->tx_errors = cur_es->tx_errors;
+ cur_ns->rx_dropped = cur_es->rx_discards;
+ cur_ns->tx_dropped = cur_es->tx_discards;
+ cur_ns->multicast = cur_es->rx_multicast;
+
+ /* update some more netdev stats if this is main VSI */
+ if (vsi->type == ICE_VSI_PF) {
+ cur_ns->rx_crc_errors = pf->stats.crc_errors;
+ cur_ns->rx_errors = pf->stats.crc_errors +
+ pf->stats.illegal_bytes;
+ cur_ns->rx_length_errors = pf->stats.rx_len_errors;
+ }
+}
+
+/**
+ * ice_update_pf_stats - Update PF port stats counters
+ * @pf: PF whose stats needs to be updated
+ */
+static void ice_update_pf_stats(struct ice_pf *pf)
+{
+ struct ice_hw_port_stats *prev_ps, *cur_ps;
+ struct ice_hw *hw = &pf->hw;
+ u8 pf_id;
+
+ prev_ps = &pf->stats_prev;
+ cur_ps = &pf->stats;
+ pf_id = hw->pf_id;
+
+ ice_stat_update40(hw, GLPRT_GORCH(pf_id), GLPRT_GORCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.rx_bytes,
+ &cur_ps->eth.rx_bytes);
+
+ ice_stat_update40(hw, GLPRT_UPRCH(pf_id), GLPRT_UPRCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.rx_unicast,
+ &cur_ps->eth.rx_unicast);
+
+ ice_stat_update40(hw, GLPRT_MPRCH(pf_id), GLPRT_MPRCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.rx_multicast,
+ &cur_ps->eth.rx_multicast);
+
+ ice_stat_update40(hw, GLPRT_BPRCH(pf_id), GLPRT_BPRCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.rx_broadcast,
+ &cur_ps->eth.rx_broadcast);
+
+ ice_stat_update40(hw, GLPRT_GOTCH(pf_id), GLPRT_GOTCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.tx_bytes,
+ &cur_ps->eth.tx_bytes);
+
+ ice_stat_update40(hw, GLPRT_UPTCH(pf_id), GLPRT_UPTCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.tx_unicast,
+ &cur_ps->eth.tx_unicast);
+
+ ice_stat_update40(hw, GLPRT_MPTCH(pf_id), GLPRT_MPTCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.tx_multicast,
+ &cur_ps->eth.tx_multicast);
+
+ ice_stat_update40(hw, GLPRT_BPTCH(pf_id), GLPRT_BPTCL(pf_id),
+ pf->stat_prev_loaded, &prev_ps->eth.tx_broadcast,
+ &cur_ps->eth.tx_broadcast);
+
+ ice_stat_update32(hw, GLPRT_TDOLD(pf_id), pf->stat_prev_loaded,
+ &prev_ps->tx_dropped_link_down,
+ &cur_ps->tx_dropped_link_down);
+
+ ice_stat_update40(hw, GLPRT_PRC64H(pf_id), GLPRT_PRC64L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->rx_size_64,
+ &cur_ps->rx_size_64);
+
+ ice_stat_update40(hw, GLPRT_PRC127H(pf_id), GLPRT_PRC127L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->rx_size_127,
+ &cur_ps->rx_size_127);
+
+ ice_stat_update40(hw, GLPRT_PRC255H(pf_id), GLPRT_PRC255L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->rx_size_255,
+ &cur_ps->rx_size_255);
+
+ ice_stat_update40(hw, GLPRT_PRC511H(pf_id), GLPRT_PRC511L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->rx_size_511,
+ &cur_ps->rx_size_511);
+
+ ice_stat_update40(hw, GLPRT_PRC1023H(pf_id),
+ GLPRT_PRC1023L(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_size_1023, &cur_ps->rx_size_1023);
+
+ ice_stat_update40(hw, GLPRT_PRC1522H(pf_id),
+ GLPRT_PRC1522L(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_size_1522, &cur_ps->rx_size_1522);
+
+ ice_stat_update40(hw, GLPRT_PRC9522H(pf_id),
+ GLPRT_PRC9522L(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_size_big, &cur_ps->rx_size_big);
+
+ ice_stat_update40(hw, GLPRT_PTC64H(pf_id), GLPRT_PTC64L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->tx_size_64,
+ &cur_ps->tx_size_64);
+
+ ice_stat_update40(hw, GLPRT_PTC127H(pf_id), GLPRT_PTC127L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->tx_size_127,
+ &cur_ps->tx_size_127);
+
+ ice_stat_update40(hw, GLPRT_PTC255H(pf_id), GLPRT_PTC255L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->tx_size_255,
+ &cur_ps->tx_size_255);
+
+ ice_stat_update40(hw, GLPRT_PTC511H(pf_id), GLPRT_PTC511L(pf_id),
+ pf->stat_prev_loaded, &prev_ps->tx_size_511,
+ &cur_ps->tx_size_511);
+
+ ice_stat_update40(hw, GLPRT_PTC1023H(pf_id),
+ GLPRT_PTC1023L(pf_id), pf->stat_prev_loaded,
+ &prev_ps->tx_size_1023, &cur_ps->tx_size_1023);
+
+ ice_stat_update40(hw, GLPRT_PTC1522H(pf_id),
+ GLPRT_PTC1522L(pf_id), pf->stat_prev_loaded,
+ &prev_ps->tx_size_1522, &cur_ps->tx_size_1522);
+
+ ice_stat_update40(hw, GLPRT_PTC9522H(pf_id),
+ GLPRT_PTC9522L(pf_id), pf->stat_prev_loaded,
+ &prev_ps->tx_size_big, &cur_ps->tx_size_big);
+
+ ice_stat_update32(hw, GLPRT_LXONRXC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->link_xon_rx, &cur_ps->link_xon_rx);
+
+ ice_stat_update32(hw, GLPRT_LXOFFRXC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->link_xoff_rx, &cur_ps->link_xoff_rx);
+
+ ice_stat_update32(hw, GLPRT_LXONTXC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->link_xon_tx, &cur_ps->link_xon_tx);
+
+ ice_stat_update32(hw, GLPRT_LXOFFTXC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->link_xoff_tx, &cur_ps->link_xoff_tx);
+
+ ice_stat_update32(hw, GLPRT_CRCERRS(pf_id), pf->stat_prev_loaded,
+ &prev_ps->crc_errors, &cur_ps->crc_errors);
+
+ ice_stat_update32(hw, GLPRT_ILLERRC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->illegal_bytes, &cur_ps->illegal_bytes);
+
+ ice_stat_update32(hw, GLPRT_MLFC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->mac_local_faults,
+ &cur_ps->mac_local_faults);
+
+ ice_stat_update32(hw, GLPRT_MRFC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->mac_remote_faults,
+ &cur_ps->mac_remote_faults);
+
+ ice_stat_update32(hw, GLPRT_RLEC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_len_errors, &cur_ps->rx_len_errors);
+
+ ice_stat_update32(hw, GLPRT_RUC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_undersize, &cur_ps->rx_undersize);
+
+ ice_stat_update32(hw, GLPRT_RFC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_fragments, &cur_ps->rx_fragments);
+
+ ice_stat_update32(hw, GLPRT_ROC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_oversize, &cur_ps->rx_oversize);
+
+ ice_stat_update32(hw, GLPRT_RJC(pf_id), pf->stat_prev_loaded,
+ &prev_ps->rx_jabber, &cur_ps->rx_jabber);
+
+ pf->stat_prev_loaded = true;
+}
+
+/**
+ * ice_get_stats64 - get statistics for network device structure
+ * @netdev: network interface device structure
+ * @stats: main device statistics structure
+ */
+static
+void ice_get_stats64(struct net_device *netdev, struct rtnl_link_stats64 *stats)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct rtnl_link_stats64 *vsi_stats;
+ struct ice_vsi *vsi = np->vsi;
+
+ vsi_stats = &vsi->net_stats;
+
+ if (test_bit(__ICE_DOWN, vsi->state) || !vsi->num_txq || !vsi->num_rxq)
+ return;
+ /* netdev packet/byte stats come from ring counter. These are obtained
+ * by summing up ring counters (done by ice_update_vsi_ring_stats).
+ */
+ ice_update_vsi_ring_stats(vsi);
+ stats->tx_packets = vsi_stats->tx_packets;
+ stats->tx_bytes = vsi_stats->tx_bytes;
+ stats->rx_packets = vsi_stats->rx_packets;
+ stats->rx_bytes = vsi_stats->rx_bytes;
+
+ /* The rest of the stats can be read from the hardware but instead we
+ * just return values that the watchdog task has already obtained from
+ * the hardware.
+ */
+ stats->multicast = vsi_stats->multicast;
+ stats->tx_errors = vsi_stats->tx_errors;
+ stats->tx_dropped = vsi_stats->tx_dropped;
+ stats->rx_errors = vsi_stats->rx_errors;
+ stats->rx_dropped = vsi_stats->rx_dropped;
+ stats->rx_crc_errors = vsi_stats->rx_crc_errors;
+ stats->rx_length_errors = vsi_stats->rx_length_errors;
+}
+
+#ifdef CONFIG_NET_POLL_CONTROLLER
+/**
+ * ice_netpoll - polling "interrupt" handler
+ * @netdev: network interface device structure
+ *
+ * Used by netconsole to send skbs without having to re-enable interrupts.
+ * This is not called in the normal interrupt path.
+ */
+static void ice_netpoll(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+ int i;
+
+ if (test_bit(__ICE_DOWN, vsi->state) ||
+ !test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
+ return;
+
+ for (i = 0; i < vsi->num_q_vectors; i++)
+ ice_msix_clean_rings(0, vsi->q_vectors[i]);
+}
+#endif /* CONFIG_NET_POLL_CONTROLLER */
+
+/**
+ * ice_napi_disable_all - Disable NAPI for all q_vectors in the VSI
+ * @vsi: VSI having NAPI disabled
+ */
+static void ice_napi_disable_all(struct ice_vsi *vsi)
+{
+ int q_idx;
+
+ if (!vsi->netdev)
+ return;
+
+ for (q_idx = 0; q_idx < vsi->num_q_vectors; q_idx++)
+ napi_disable(&vsi->q_vectors[q_idx]->napi);
+}
+
+/**
+ * ice_down - Shutdown the connection
+ * @vsi: The VSI being stopped
+ */
+int ice_down(struct ice_vsi *vsi)
+{
+ int i, err;
+
+ /* Caller of this function is expected to set the
+ * vsi->state __ICE_DOWN bit
+ */
+ if (vsi->netdev) {
+ netif_carrier_off(vsi->netdev);
+ netif_tx_disable(vsi->netdev);
+ }
+
+ ice_vsi_dis_irq(vsi);
+ err = ice_vsi_stop_tx_rx_rings(vsi);
+ ice_napi_disable_all(vsi);
+
+ ice_for_each_txq(vsi, i)
+ ice_clean_tx_ring(vsi->tx_rings[i]);
+
+ ice_for_each_rxq(vsi, i)
+ ice_clean_rx_ring(vsi->rx_rings[i]);
+
+ if (err)
+ netdev_err(vsi->netdev, "Failed to close VSI 0x%04X on switch 0x%04X\n",
+ vsi->vsi_num, vsi->vsw->sw_id);
+ return err;
+}
+
+/**
+ * ice_vsi_setup_tx_rings - Allocate VSI Tx queue resources
+ * @vsi: VSI having resources allocated
+ *
+ * Return 0 on success, negative on failure
+ */
+static int ice_vsi_setup_tx_rings(struct ice_vsi *vsi)
+{
+ int i, err;
+
+ if (!vsi->num_txq) {
+ dev_err(&vsi->back->pdev->dev, "VSI %d has 0 Tx queues\n",
+ vsi->vsi_num);
+ return -EINVAL;
+ }
+
+ ice_for_each_txq(vsi, i) {
+ err = ice_setup_tx_ring(vsi->tx_rings[i]);
+ if (err)
+ break;
+ }
+
+ return err;
+}
+
+/**
+ * ice_vsi_setup_rx_rings - Allocate VSI Rx queue resources
+ * @vsi: VSI having resources allocated
+ *
+ * Return 0 on success, negative on failure
+ */
+static int ice_vsi_setup_rx_rings(struct ice_vsi *vsi)
+{
+ int i, err;
+
+ if (!vsi->num_rxq) {
+ dev_err(&vsi->back->pdev->dev, "VSI %d has 0 Rx queues\n",
+ vsi->vsi_num);
+ return -EINVAL;
+ }
+
+ ice_for_each_rxq(vsi, i) {
+ err = ice_setup_rx_ring(vsi->rx_rings[i]);
+ if (err)
+ break;
+ }
+
+ return err;
+}
+
+/**
+ * ice_vsi_req_irq - Request IRQ from the OS
+ * @vsi: The VSI IRQ is being requested for
+ * @basename: name for the vector
+ *
+ * Return 0 on success and a negative value on error
+ */
+static int ice_vsi_req_irq(struct ice_vsi *vsi, char *basename)
+{
+ struct ice_pf *pf = vsi->back;
+ int err = -EINVAL;
+
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
+ err = ice_vsi_req_irq_msix(vsi, basename);
+
+ return err;
+}
+
+/**
+ * ice_vsi_free_tx_rings - Free Tx resources for VSI queues
+ * @vsi: the VSI having resources freed
+ */
+static void ice_vsi_free_tx_rings(struct ice_vsi *vsi)
+{
+ int i;
+
+ if (!vsi->tx_rings)
+ return;
+
+ ice_for_each_txq(vsi, i)
+ if (vsi->tx_rings[i] && vsi->tx_rings[i]->desc)
+ ice_free_tx_ring(vsi->tx_rings[i]);
+}
+
+/**
+ * ice_vsi_free_rx_rings - Free Rx resources for VSI queues
+ * @vsi: the VSI having resources freed
+ */
+static void ice_vsi_free_rx_rings(struct ice_vsi *vsi)
+{
+ int i;
+
+ if (!vsi->rx_rings)
+ return;
+
+ ice_for_each_rxq(vsi, i)
+ if (vsi->rx_rings[i] && vsi->rx_rings[i]->desc)
+ ice_free_rx_ring(vsi->rx_rings[i]);
+}
+
+/**
+ * ice_vsi_open - Called when a network interface is made active
+ * @vsi: the VSI to open
+ *
+ * Initialization of the VSI
+ *
+ * Returns 0 on success, negative value on error
+ */
+static int ice_vsi_open(struct ice_vsi *vsi)
+{
+ char int_name[ICE_INT_NAME_STR_LEN];
+ struct ice_pf *pf = vsi->back;
+ int err;
+
+ /* allocate descriptors */
+ err = ice_vsi_setup_tx_rings(vsi);
+ if (err)
+ goto err_setup_tx;
+
+ err = ice_vsi_setup_rx_rings(vsi);
+ if (err)
+ goto err_setup_rx;
+
+ err = ice_vsi_cfg(vsi);
+ if (err)
+ goto err_setup_rx;
+
+ snprintf(int_name, sizeof(int_name) - 1, "%s-%s",
+ dev_driver_string(&pf->pdev->dev), vsi->netdev->name);
+ err = ice_vsi_req_irq(vsi, int_name);
+ if (err)
+ goto err_setup_rx;
+
+ /* Notify the stack of the actual queue counts. */
+ err = netif_set_real_num_tx_queues(vsi->netdev, vsi->num_txq);
+ if (err)
+ goto err_set_qs;
+
+ err = netif_set_real_num_rx_queues(vsi->netdev, vsi->num_rxq);
+ if (err)
+ goto err_set_qs;
+
+ err = ice_up_complete(vsi);
+ if (err)
+ goto err_up_complete;
+
+ return 0;
+
+err_up_complete:
+ ice_down(vsi);
+err_set_qs:
+ ice_vsi_free_irq(vsi);
+err_setup_rx:
+ ice_vsi_free_rx_rings(vsi);
+err_setup_tx:
+ ice_vsi_free_tx_rings(vsi);
+
+ return err;
+}
+
+/**
+ * ice_vsi_close - Shut down a VSI
+ * @vsi: the VSI being shut down
+ */
+static void ice_vsi_close(struct ice_vsi *vsi)
+{
+ if (!test_and_set_bit(__ICE_DOWN, vsi->state))
+ ice_down(vsi);
+
+ ice_vsi_free_irq(vsi);
+ ice_vsi_free_tx_rings(vsi);
+ ice_vsi_free_rx_rings(vsi);
+}
+
+/**
+ * ice_rss_clean - Delete RSS related VSI structures that hold user inputs
+ * @vsi: the VSI being removed
+ */
+static void ice_rss_clean(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf;
+
+ pf = vsi->back;
+
+ if (vsi->rss_hkey_user)
+ devm_kfree(&pf->pdev->dev, vsi->rss_hkey_user);
+ if (vsi->rss_lut_user)
+ devm_kfree(&pf->pdev->dev, vsi->rss_lut_user);
+}
+
+/**
+ * ice_vsi_release - Delete a VSI and free its resources
+ * @vsi: the VSI being removed
+ *
+ * Returns 0 on success or < 0 on error
+ */
+static int ice_vsi_release(struct ice_vsi *vsi)
+{
+ struct ice_pf *pf;
+
+ if (!vsi->back)
+ return -ENODEV;
+ pf = vsi->back;
+
+ if (vsi->netdev) {
+ unregister_netdev(vsi->netdev);
+ free_netdev(vsi->netdev);
+ vsi->netdev = NULL;
+ }
+
+ if (test_bit(ICE_FLAG_RSS_ENA, pf->flags))
+ ice_rss_clean(vsi);
+
+ /* Disable VSI and free resources */
+ ice_vsi_dis_irq(vsi);
+ ice_vsi_close(vsi);
+
+ /* reclaim interrupt vectors back to PF */
+ ice_free_res(vsi->back->irq_tracker, vsi->base_vector, vsi->idx);
+ pf->num_avail_msix += vsi->num_q_vectors;
+
+ ice_remove_vsi_fltr(&pf->hw, vsi->vsi_num);
+ ice_vsi_delete(vsi);
+ ice_vsi_free_q_vectors(vsi);
+ ice_vsi_clear_rings(vsi);
+
+ ice_vsi_put_qs(vsi);
+ pf->q_left_tx += vsi->alloc_txq;
+ pf->q_left_rx += vsi->alloc_rxq;
+
+ ice_vsi_clear(vsi);
+
+ return 0;
+}
+
+/**
+ * ice_dis_vsi - pause a VSI
+ * @vsi: the VSI being paused
+ */
+static void ice_dis_vsi(struct ice_vsi *vsi)
+{
+ if (test_bit(__ICE_DOWN, vsi->state))
+ return;
+
+ set_bit(__ICE_NEEDS_RESTART, vsi->state);
+
+ if (vsi->netdev && netif_running(vsi->netdev) &&
+ vsi->type == ICE_VSI_PF)
+ vsi->netdev->netdev_ops->ndo_stop(vsi->netdev);
+
+ ice_vsi_close(vsi);
+}
+
+/**
+ * ice_ena_vsi - resume a VSI
+ * @vsi: the VSI being resume
+ */
+static void ice_ena_vsi(struct ice_vsi *vsi)
+{
+ if (!test_and_clear_bit(__ICE_NEEDS_RESTART, vsi->state))
+ return;
+
+ if (vsi->netdev && netif_running(vsi->netdev))
+ vsi->netdev->netdev_ops->ndo_open(vsi->netdev);
+ else if (ice_vsi_open(vsi))
+ /* this clears the DOWN bit */
+ dev_dbg(&vsi->back->pdev->dev, "Failed open VSI 0x%04X on switch 0x%04X\n",
+ vsi->vsi_num, vsi->vsw->sw_id);
+}
+
+/**
+ * ice_pf_dis_all_vsi - Pause all VSIs on a PF
+ * @pf: the PF
+ */
+static void ice_pf_dis_all_vsi(struct ice_pf *pf)
+{
+ int v;
+
+ ice_for_each_vsi(pf, v)
+ if (pf->vsi[v])
+ ice_dis_vsi(pf->vsi[v]);
+}
+
+/**
+ * ice_pf_ena_all_vsi - Resume all VSIs on a PF
+ * @pf: the PF
+ */
+static void ice_pf_ena_all_vsi(struct ice_pf *pf)
+{
+ int v;
+
+ ice_for_each_vsi(pf, v)
+ if (pf->vsi[v])
+ ice_ena_vsi(pf->vsi[v]);
+}
+
+/**
+ * ice_rebuild - rebuild after reset
+ * @pf: pf to rebuild
+ */
+static void ice_rebuild(struct ice_pf *pf)
+{
+ struct device *dev = &pf->pdev->dev;
+ struct ice_hw *hw = &pf->hw;
+ enum ice_status ret;
+ int err;
+
+ if (test_bit(__ICE_DOWN, pf->state))
+ goto clear_recovery;
+
+ dev_dbg(dev, "rebuilding pf\n");
+
+ ret = ice_init_all_ctrlq(hw);
+ if (ret) {
+ dev_err(dev, "control queues init failed %d\n", ret);
+ goto fail_reset;
+ }
+
+ ret = ice_clear_pf_cfg(hw);
+ if (ret) {
+ dev_err(dev, "clear PF configuration failed %d\n", ret);
+ goto fail_reset;
+ }
+
+ ice_clear_pxe_mode(hw);
+
+ ret = ice_get_caps(hw);
+ if (ret) {
+ dev_err(dev, "ice_get_caps failed %d\n", ret);
+ goto fail_reset;
+ }
+
+ /* basic nic switch setup */
+ err = ice_setup_pf_sw(pf);
+ if (err) {
+ dev_err(dev, "ice_setup_pf_sw failed\n");
+ goto fail_reset;
+ }
+
+ /* start misc vector */
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags)) {
+ err = ice_req_irq_msix_misc(pf);
+ if (err) {
+ dev_err(dev, "misc vector setup failed: %d\n", err);
+ goto fail_reset;
+ }
+ }
+
+ /* restart the VSIs that were rebuilt and running before the reset */
+ ice_pf_ena_all_vsi(pf);
+
+ return;
+
+fail_reset:
+ ice_shutdown_all_ctrlq(hw);
+ set_bit(__ICE_RESET_FAILED, pf->state);
+clear_recovery:
+ set_bit(__ICE_RESET_RECOVERY_PENDING, pf->state);
+}
+
+/**
+ * ice_change_mtu - NDO callback to change the MTU
+ * @netdev: network interface device structure
+ * @new_mtu: new value for maximum frame size
+ *
+ * Returns 0 on success, negative on failure
+ */
+static int ice_change_mtu(struct net_device *netdev, int new_mtu)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_pf *pf = vsi->back;
+ u8 count = 0;
+
+ if (new_mtu == netdev->mtu) {
+ netdev_warn(netdev, "mtu is already %d\n", netdev->mtu);
+ return 0;
+ }
+
+ if (new_mtu < netdev->min_mtu) {
+ netdev_err(netdev, "new mtu invalid. min_mtu is %d\n",
+ netdev->min_mtu);
+ return -EINVAL;
+ } else if (new_mtu > netdev->max_mtu) {
+ netdev_err(netdev, "new mtu invalid. max_mtu is %d\n",
+ netdev->min_mtu);
+ return -EINVAL;
+ }
+ /* if a reset is in progress, wait for some time for it to complete */
+ do {
+ if (ice_is_reset_recovery_pending(pf->state)) {
+ count++;
+ usleep_range(1000, 2000);
+ } else {
+ break;
+ }
+
+ } while (count < 100);
+
+ if (count == 100) {
+ netdev_err(netdev, "can't change mtu. Device is busy\n");
+ return -EBUSY;
+ }
+
+ netdev->mtu = new_mtu;
+
+ /* if VSI is up, bring it down and then back up */
+ if (!test_and_set_bit(__ICE_DOWN, vsi->state)) {
+ int err;
+
+ err = ice_down(vsi);
+ if (err) {
+ netdev_err(netdev, "change mtu if_up err %d\n", err);
+ return err;
+ }
+
+ err = ice_up(vsi);
+ if (err) {
+ netdev_err(netdev, "change mtu if_up err %d\n", err);
+ return err;
+ }
+ }
+
+ netdev_dbg(netdev, "changed mtu to %d\n", new_mtu);
+ return 0;
+}
+
+/**
+ * ice_set_rss - Set RSS keys and lut
+ * @vsi: Pointer to VSI structure
+ * @seed: RSS hash seed
+ * @lut: Lookup table
+ * @lut_size: Lookup table size
+ *
+ * Returns 0 on success, negative on failure
+ */
+int ice_set_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ enum ice_status status;
+
+ if (seed) {
+ struct ice_aqc_get_set_rss_keys *buf =
+ (struct ice_aqc_get_set_rss_keys *)seed;
+
+ status = ice_aq_set_rss_key(hw, vsi->vsi_num, buf);
+
+ if (status) {
+ dev_err(&pf->pdev->dev,
+ "Cannot set RSS key, err %d aq_err %d\n",
+ status, hw->adminq.rq_last_status);
+ return -EIO;
+ }
+ }
+
+ if (lut) {
+ status = ice_aq_set_rss_lut(hw, vsi->vsi_num,
+ vsi->rss_lut_type, lut, lut_size);
+ if (status) {
+ dev_err(&pf->pdev->dev,
+ "Cannot set RSS lut, err %d aq_err %d\n",
+ status, hw->adminq.rq_last_status);
+ return -EIO;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ice_get_rss - Get RSS keys and lut
+ * @vsi: Pointer to VSI structure
+ * @seed: Buffer to store the keys
+ * @lut: Buffer to store the lookup table entries
+ * @lut_size: Size of buffer to store the lookup table entries
+ *
+ * Returns 0 on success, negative on failure
+ */
+int ice_get_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size)
+{
+ struct ice_pf *pf = vsi->back;
+ struct ice_hw *hw = &pf->hw;
+ enum ice_status status;
+
+ if (seed) {
+ struct ice_aqc_get_set_rss_keys *buf =
+ (struct ice_aqc_get_set_rss_keys *)seed;
+
+ status = ice_aq_get_rss_key(hw, vsi->vsi_num, buf);
+ if (status) {
+ dev_err(&pf->pdev->dev,
+ "Cannot get RSS key, err %d aq_err %d\n",
+ status, hw->adminq.rq_last_status);
+ return -EIO;
+ }
+ }
+
+ if (lut) {
+ status = ice_aq_get_rss_lut(hw, vsi->vsi_num,
+ vsi->rss_lut_type, lut, lut_size);
+ if (status) {
+ dev_err(&pf->pdev->dev,
+ "Cannot get RSS lut, err %d aq_err %d\n",
+ status, hw->adminq.rq_last_status);
+ return -EIO;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ice_open - Called when a network interface becomes active
+ * @netdev: network interface device structure
+ *
+ * The open entry point is called when a network interface is made
+ * active by the system (IFF_UP). At this point all resources needed
+ * for transmit and receive operations are allocated, the interrupt
+ * handler is registered with the OS, the netdev watchdog is enabled,
+ * and the stack is notified that the interface is ready.
+ *
+ * Returns 0 on success, negative value on failure
+ */
+static int ice_open(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ int err;
+
+ netif_carrier_off(netdev);
+
+ err = ice_vsi_open(vsi);
+
+ if (err)
+ netdev_err(netdev, "Failed to open VSI 0x%04X on switch 0x%04X\n",
+ vsi->vsi_num, vsi->vsw->sw_id);
+ return err;
+}
+
+/**
+ * ice_stop - Disables a network interface
+ * @netdev: network interface device structure
+ *
+ * The stop entry point is called when an interface is de-activated by the OS,
+ * and the netdevice enters the DOWN state. The hardware is still under the
+ * driver's control, but the netdev interface is disabled.
+ *
+ * Returns success only - not allowed to fail
+ */
+static int ice_stop(struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+
+ ice_vsi_close(vsi);
+
+ return 0;
+}
+
+/**
+ * ice_features_check - Validate encapsulated packet conforms to limits
+ * @skb: skb buffer
+ * @netdev: This port's netdev
+ * @features: Offload features that the stack believes apply
+ */
+static netdev_features_t
+ice_features_check(struct sk_buff *skb,
+ struct net_device __always_unused *netdev,
+ netdev_features_t features)
+{
+ size_t len;
+
+ /* No point in doing any of this if neither checksum nor GSO are
+ * being requested for this frame. We can rule out both by just
+ * checking for CHECKSUM_PARTIAL
+ */
+ if (skb->ip_summed != CHECKSUM_PARTIAL)
+ return features;
+
+ /* We cannot support GSO if the MSS is going to be less than
+ * 64 bytes. If it is then we need to drop support for GSO.
+ */
+ if (skb_is_gso(skb) && (skb_shinfo(skb)->gso_size < 64))
+ features &= ~NETIF_F_GSO_MASK;
+
+ len = skb_network_header(skb) - skb->data;
+ if (len & ~(ICE_TXD_MACLEN_MAX))
+ goto out_rm_features;
+
+ len = skb_transport_header(skb) - skb_network_header(skb);
+ if (len & ~(ICE_TXD_IPLEN_MAX))
+ goto out_rm_features;
+
+ if (skb->encapsulation) {
+ len = skb_inner_network_header(skb) - skb_transport_header(skb);
+ if (len & ~(ICE_TXD_L4LEN_MAX))
+ goto out_rm_features;
+
+ len = skb_inner_transport_header(skb) -
+ skb_inner_network_header(skb);
+ if (len & ~(ICE_TXD_IPLEN_MAX))
+ goto out_rm_features;
+ }
+
+ return features;
+out_rm_features:
+ return features & ~(NETIF_F_CSUM_MASK | NETIF_F_GSO_MASK);
+}
+
+static const struct net_device_ops ice_netdev_ops = {
+ .ndo_open = ice_open,
+ .ndo_stop = ice_stop,
+ .ndo_start_xmit = ice_start_xmit,
+ .ndo_features_check = ice_features_check,
+ .ndo_set_rx_mode = ice_set_rx_mode,
+ .ndo_set_mac_address = ice_set_mac_address,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_change_mtu = ice_change_mtu,
+ .ndo_get_stats64 = ice_get_stats64,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = ice_netpoll,
+#endif /* CONFIG_NET_POLL_CONTROLLER */
+ .ndo_vlan_rx_add_vid = ice_vlan_rx_add_vid,
+ .ndo_vlan_rx_kill_vid = ice_vlan_rx_kill_vid,
+ .ndo_set_features = ice_set_features,
+ .ndo_fdb_add = ice_fdb_add,
+ .ndo_fdb_del = ice_fdb_del,
+};
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+#include "ice_common.h"
+
+/**
+ * ice_aq_read_nvm
+ * @hw: pointer to the hw struct
+ * @module_typeid: module pointer location in words from the NVM beginning
+ * @offset: byte offset from the module beginning
+ * @length: length of the section to be read (in bytes from the offset)
+ * @data: command buffer (size [bytes] = length)
+ * @last_command: tells if this is the last command in a series
+ * @cd: pointer to command details structure or NULL
+ *
+ * Read the NVM using the admin queue commands (0x0701)
+ */
+static enum ice_status
+ice_aq_read_nvm(struct ice_hw *hw, u8 module_typeid, u32 offset, u16 length,
+ void *data, bool last_command, struct ice_sq_cd *cd)
+{
+ struct ice_aq_desc desc;
+ struct ice_aqc_nvm *cmd;
+
+ cmd = &desc.params.nvm;
+
+ /* In offset the highest byte must be zeroed. */
+ if (offset & 0xFF000000)
+ return ICE_ERR_PARAM;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_nvm_read);
+
+ /* If this is the last command in a series, set the proper flag. */
+ if (last_command)
+ cmd->cmd_flags |= ICE_AQC_NVM_LAST_CMD;
+ cmd->module_typeid = module_typeid;
+ cmd->offset = cpu_to_le32(offset);
+ cmd->length = cpu_to_le16(length);
+
+ return ice_aq_send_cmd(hw, &desc, data, length, cd);
+}
+
+/**
+ * ice_check_sr_access_params - verify params for Shadow RAM R/W operations.
+ * @hw: pointer to the HW structure
+ * @offset: offset in words from module start
+ * @words: number of words to access
+ */
+static enum ice_status
+ice_check_sr_access_params(struct ice_hw *hw, u32 offset, u16 words)
+{
+ if ((offset + words) > hw->nvm.sr_words) {
+ ice_debug(hw, ICE_DBG_NVM,
+ "NVM error: offset beyond SR lmt.\n");
+ return ICE_ERR_PARAM;
+ }
+
+ if (words > ICE_SR_SECTOR_SIZE_IN_WORDS) {
+ /* We can access only up to 4KB (one sector), in one AQ write */
+ ice_debug(hw, ICE_DBG_NVM,
+ "NVM error: tried to access %d words, limit is %d.\n",
+ words, ICE_SR_SECTOR_SIZE_IN_WORDS);
+ return ICE_ERR_PARAM;
+ }
+
+ if (((offset + (words - 1)) / ICE_SR_SECTOR_SIZE_IN_WORDS) !=
+ (offset / ICE_SR_SECTOR_SIZE_IN_WORDS)) {
+ /* A single access cannot spread over two sectors */
+ ice_debug(hw, ICE_DBG_NVM,
+ "NVM error: cannot spread over two sectors.\n");
+ return ICE_ERR_PARAM;
+ }
+
+ return 0;
+}
+
+/**
+ * ice_read_sr_aq - Read Shadow RAM.
+ * @hw: pointer to the HW structure
+ * @offset: offset in words from module start
+ * @words: number of words to read
+ * @data: buffer for words reads from Shadow RAM
+ * @last_command: tells the AdminQ that this is the last command
+ *
+ * Reads 16-bit word buffers from the Shadow RAM using the admin command.
+ */
+static enum ice_status
+ice_read_sr_aq(struct ice_hw *hw, u32 offset, u16 words, u16 *data,
+ bool last_command)
+{
+ enum ice_status status;
+
+ status = ice_check_sr_access_params(hw, offset, words);
+
+ /* values in "offset" and "words" parameters are sized as words
+ * (16 bits) but ice_aq_read_nvm expects these values in bytes.
+ * So do this conversion while calling ice_aq_read_nvm.
+ */
+ if (!status)
+ status = ice_aq_read_nvm(hw, 0, 2 * offset, 2 * words, data,
+ last_command, NULL);
+
+ return status;
+}
+
+/**
+ * ice_read_sr_word_aq - Reads Shadow RAM via AQ
+ * @hw: pointer to the HW structure
+ * @offset: offset of the Shadow RAM word to read (0x000000 - 0x001FFF)
+ * @data: word read from the Shadow RAM
+ *
+ * Reads one 16 bit word from the Shadow RAM using the ice_read_sr_aq method.
+ */
+static enum ice_status
+ice_read_sr_word_aq(struct ice_hw *hw, u16 offset, u16 *data)
+{
+ enum ice_status status;
+
+ status = ice_read_sr_aq(hw, offset, 1, data, true);
+ if (!status)
+ *data = le16_to_cpu(*(__le16 *)data);
+
+ return status;
+}
+
+/**
+ * ice_acquire_nvm - Generic request for acquiring the NVM ownership
+ * @hw: pointer to the HW structure
+ * @access: NVM access type (read or write)
+ *
+ * This function will request NVM ownership.
+ */
+static enum
+ice_status ice_acquire_nvm(struct ice_hw *hw,
+ enum ice_aq_res_access_type access)
+{
+ if (hw->nvm.blank_nvm_mode)
+ return 0;
+
+ return ice_acquire_res(hw, ICE_NVM_RES_ID, access);
+}
+
+/**
+ * ice_release_nvm - Generic request for releasing the NVM ownership
+ * @hw: pointer to the HW structure
+ *
+ * This function will release NVM ownership.
+ */
+static void ice_release_nvm(struct ice_hw *hw)
+{
+ if (hw->nvm.blank_nvm_mode)
+ return;
+
+ ice_release_res(hw, ICE_NVM_RES_ID);
+}
+
+/**
+ * ice_read_sr_word - Reads Shadow RAM word and acquire NVM if necessary
+ * @hw: pointer to the HW structure
+ * @offset: offset of the Shadow RAM word to read (0x000000 - 0x001FFF)
+ * @data: word read from the Shadow RAM
+ *
+ * Reads one 16 bit word from the Shadow RAM using the ice_read_sr_word_aq.
+ */
+static enum ice_status
+ice_read_sr_word(struct ice_hw *hw, u16 offset, u16 *data)
+{
+ enum ice_status status;
+
+ status = ice_acquire_nvm(hw, ICE_RES_READ);
+ if (!status) {
+ status = ice_read_sr_word_aq(hw, offset, data);
+ ice_release_nvm(hw);
+ }
+
+ return status;
+}
+
+/**
+ * ice_init_nvm - initializes NVM setting
+ * @hw: pointer to the hw struct
+ *
+ * This function reads and populates NVM settings such as Shadow RAM size,
+ * max_timeout, and blank_nvm_mode
+ */
+enum ice_status ice_init_nvm(struct ice_hw *hw)
+{
+ struct ice_nvm_info *nvm = &hw->nvm;
+ u16 eetrack_lo, eetrack_hi;
+ enum ice_status status = 0;
+ u32 fla, gens_stat;
+ u8 sr_size;
+
+ /* The SR size is stored regardless of the nvm programming mode
+ * as the blank mode may be used in the factory line.
+ */
+ gens_stat = rd32(hw, GLNVM_GENS);
+ sr_size = (gens_stat & GLNVM_GENS_SR_SIZE_M) >> GLNVM_GENS_SR_SIZE_S;
+
+ /* Switching to words (sr_size contains power of 2) */
+ nvm->sr_words = BIT(sr_size) * ICE_SR_WORDS_IN_1KB;
+
+ /* Check if we are in the normal or blank NVM programming mode */
+ fla = rd32(hw, GLNVM_FLA);
+ if (fla & GLNVM_FLA_LOCKED_M) { /* Normal programming mode */
+ nvm->blank_nvm_mode = false;
+ } else { /* Blank programming mode */
+ nvm->blank_nvm_mode = true;
+ status = ICE_ERR_NVM_BLANK_MODE;
+ ice_debug(hw, ICE_DBG_NVM,
+ "NVM init error: unsupported blank mode.\n");
+ return status;
+ }
+
+ status = ice_read_sr_word(hw, ICE_SR_NVM_DEV_STARTER_VER, &hw->nvm.ver);
+ if (status) {
+ ice_debug(hw, ICE_DBG_INIT,
+ "Failed to read DEV starter version.\n");
+ return status;
+ }
+
+ status = ice_read_sr_word(hw, ICE_SR_NVM_EETRACK_LO, &eetrack_lo);
+ if (status) {
+ ice_debug(hw, ICE_DBG_INIT, "Failed to read EETRACK lo.\n");
+ return status;
+ }
+ status = ice_read_sr_word(hw, ICE_SR_NVM_EETRACK_HI, &eetrack_hi);
+ if (status) {
+ ice_debug(hw, ICE_DBG_INIT, "Failed to read EETRACK hi.\n");
+ return status;
+ }
+
+ hw->nvm.eetrack = (eetrack_hi << 16) | eetrack_lo;
+
+ return status;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_OSDEP_H_
+#define _ICE_OSDEP_H_
+
+#include <linux/types.h>
+#include <linux/io.h>
+#ifndef CONFIG_64BIT
+#include <linux/io-64-nonatomic-lo-hi.h>
+#endif
+
+#define wr32(a, reg, value) writel((value), ((a)->hw_addr + (reg)))
+#define rd32(a, reg) readl((a)->hw_addr + (reg))
+#define wr64(a, reg, value) writeq((value), ((a)->hw_addr + (reg)))
+#define rd64(a, reg) readq((a)->hw_addr + (reg))
+
+#define ice_flush(a) rd32((a), GLGEN_STAT)
+#define ICE_M(m, s) ((m) << (s))
+
+struct ice_dma_mem {
+ void *va;
+ dma_addr_t pa;
+ size_t size;
+};
+
+#define ice_hw_to_dev(ptr) \
+ (&(container_of((ptr), struct ice_pf, hw))->pdev->dev)
+
+#ifdef CONFIG_DYNAMIC_DEBUG
+#define ice_debug(hw, type, fmt, args...) \
+ dev_dbg(ice_hw_to_dev(hw), fmt, ##args)
+
+#define ice_debug_array(hw, type, rowsize, groupsize, buf, len) \
+ print_hex_dump_debug(KBUILD_MODNAME " ", \
+ DUMP_PREFIX_OFFSET, rowsize, \
+ groupsize, buf, len, false)
+#else
+#define ice_debug(hw, type, fmt, args...) \
+do { \
+ if ((type) & (hw)->debug_mask) \
+ dev_info(ice_hw_to_dev(hw), fmt, ##args); \
+} while (0)
+
+#ifdef DEBUG
+#define ice_debug_array(hw, type, rowsize, groupsize, buf, len) \
+do { \
+ if ((type) & (hw)->debug_mask) \
+ print_hex_dump_debug(KBUILD_MODNAME, \
+ DUMP_PREFIX_OFFSET, \
+ rowsize, groupsize, buf, \
+ len, false); \
+} while (0)
+#else
+#define ice_debug_array(hw, type, rowsize, groupsize, buf, len) \
+do { \
+ struct ice_hw *hw_l = hw; \
+ if ((type) & (hw_l)->debug_mask) { \
+ u16 len_l = len; \
+ u8 *buf_l = buf; \
+ int i; \
+ for (i = 0; i < (len_l - 16); i += 16) \
+ ice_debug(hw_l, type, "0x%04X %16ph\n",\
+ i, ((buf_l) + i)); \
+ if (i < len_l) \
+ ice_debug(hw_l, type, "0x%04X %*ph\n", \
+ i, ((len_l) - i), ((buf_l) + i));\
+ } \
+} while (0)
+#endif /* DEBUG */
+#endif /* CONFIG_DYNAMIC_DEBUG */
+
+#endif /* _ICE_OSDEP_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+#include "ice_sched.h"
+
+/**
+ * ice_sched_add_root_node - Insert the Tx scheduler root node in SW DB
+ * @pi: port information structure
+ * @info: Scheduler element information from firmware
+ *
+ * This function inserts the root node of the scheduling tree topology
+ * to the SW DB.
+ */
+static enum ice_status
+ice_sched_add_root_node(struct ice_port_info *pi,
+ struct ice_aqc_txsched_elem_data *info)
+{
+ struct ice_sched_node *root;
+ struct ice_hw *hw;
+ u16 max_children;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+
+ hw = pi->hw;
+
+ root = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*root), GFP_KERNEL);
+ if (!root)
+ return ICE_ERR_NO_MEMORY;
+
+ max_children = le16_to_cpu(hw->layer_info[0].max_children);
+ root->children = devm_kcalloc(ice_hw_to_dev(hw), max_children,
+ sizeof(*root), GFP_KERNEL);
+ if (!root->children) {
+ devm_kfree(ice_hw_to_dev(hw), root);
+ return ICE_ERR_NO_MEMORY;
+ }
+
+ memcpy(&root->info, info, sizeof(*info));
+ pi->root = root;
+ return 0;
+}
+
+/**
+ * ice_sched_find_node_by_teid - Find the Tx scheduler node in SW DB
+ * @start_node: pointer to the starting ice_sched_node struct in a sub-tree
+ * @teid: node teid to search
+ *
+ * This function searches for a node matching the teid in the scheduling tree
+ * from the SW DB. The search is recursive and is restricted by the number of
+ * layers it has searched through; stopping at the max supported layer.
+ *
+ * This function needs to be called when holding the port_info->sched_lock
+ */
+struct ice_sched_node *
+ice_sched_find_node_by_teid(struct ice_sched_node *start_node, u32 teid)
+{
+ u16 i;
+
+ /* The TEID is same as that of the start_node */
+ if (ICE_TXSCHED_GET_NODE_TEID(start_node) == teid)
+ return start_node;
+
+ /* The node has no children or is at the max layer */
+ if (!start_node->num_children ||
+ start_node->tx_sched_layer >= ICE_AQC_TOPO_MAX_LEVEL_NUM ||
+ start_node->info.data.elem_type == ICE_AQC_ELEM_TYPE_LEAF)
+ return NULL;
+
+ /* Check if teid matches to any of the children nodes */
+ for (i = 0; i < start_node->num_children; i++)
+ if (ICE_TXSCHED_GET_NODE_TEID(start_node->children[i]) == teid)
+ return start_node->children[i];
+
+ /* Search within each child's sub-tree */
+ for (i = 0; i < start_node->num_children; i++) {
+ struct ice_sched_node *tmp;
+
+ tmp = ice_sched_find_node_by_teid(start_node->children[i],
+ teid);
+ if (tmp)
+ return tmp;
+ }
+
+ return NULL;
+}
+
+/**
+ * ice_sched_add_node - Insert the Tx scheduler node in SW DB
+ * @pi: port information structure
+ * @layer: Scheduler layer of the node
+ * @info: Scheduler element information from firmware
+ *
+ * This function inserts a scheduler node to the SW DB.
+ */
+enum ice_status
+ice_sched_add_node(struct ice_port_info *pi, u8 layer,
+ struct ice_aqc_txsched_elem_data *info)
+{
+ struct ice_sched_node *parent;
+ struct ice_sched_node *node;
+ struct ice_hw *hw;
+ u16 max_children;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+
+ hw = pi->hw;
+
+ /* A valid parent node should be there */
+ parent = ice_sched_find_node_by_teid(pi->root,
+ le32_to_cpu(info->parent_teid));
+ if (!parent) {
+ ice_debug(hw, ICE_DBG_SCHED,
+ "Parent Node not found for parent_teid=0x%x\n",
+ le32_to_cpu(info->parent_teid));
+ return ICE_ERR_PARAM;
+ }
+
+ node = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*node), GFP_KERNEL);
+ if (!node)
+ return ICE_ERR_NO_MEMORY;
+ max_children = le16_to_cpu(hw->layer_info[layer].max_children);
+ if (max_children) {
+ node->children = devm_kcalloc(ice_hw_to_dev(hw), max_children,
+ sizeof(*node), GFP_KERNEL);
+ if (!node->children) {
+ devm_kfree(ice_hw_to_dev(hw), node);
+ return ICE_ERR_NO_MEMORY;
+ }
+ }
+
+ node->in_use = true;
+ node->parent = parent;
+ node->tx_sched_layer = layer;
+ parent->children[parent->num_children++] = node;
+ memcpy(&node->info, info, sizeof(*info));
+ return 0;
+}
+
+/**
+ * ice_aq_delete_sched_elems - delete scheduler elements
+ * @hw: pointer to the hw struct
+ * @grps_req: number of groups to delete
+ * @buf: pointer to buffer
+ * @buf_size: buffer size in bytes
+ * @grps_del: returns total number of elements deleted
+ * @cd: pointer to command details structure or NULL
+ *
+ * Delete scheduling elements (0x040F)
+ */
+static enum ice_status
+ice_aq_delete_sched_elems(struct ice_hw *hw, u16 grps_req,
+ struct ice_aqc_delete_elem *buf, u16 buf_size,
+ u16 *grps_del, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_add_move_delete_elem *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.add_move_delete_elem;
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_delete_sched_elems);
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+ cmd->num_grps_req = cpu_to_le16(grps_req);
+
+ status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+ if (!status && grps_del)
+ *grps_del = le16_to_cpu(cmd->num_grps_updated);
+
+ return status;
+}
+
+/**
+ * ice_sched_remove_elems - remove nodes from hw
+ * @hw: pointer to the hw struct
+ * @parent: pointer to the parent node
+ * @num_nodes: number of nodes
+ * @node_teids: array of node teids to be deleted
+ *
+ * This function remove nodes from hw
+ */
+static enum ice_status
+ice_sched_remove_elems(struct ice_hw *hw, struct ice_sched_node *parent,
+ u16 num_nodes, u32 *node_teids)
+{
+ struct ice_aqc_delete_elem *buf;
+ u16 i, num_groups_removed = 0;
+ enum ice_status status;
+ u16 buf_size;
+
+ buf_size = sizeof(*buf) + sizeof(u32) * (num_nodes - 1);
+ buf = devm_kzalloc(ice_hw_to_dev(hw), buf_size, GFP_KERNEL);
+ if (!buf)
+ return ICE_ERR_NO_MEMORY;
+ buf->hdr.parent_teid = parent->info.node_teid;
+ buf->hdr.num_elems = cpu_to_le16(num_nodes);
+ for (i = 0; i < num_nodes; i++)
+ buf->teid[i] = cpu_to_le32(node_teids[i]);
+ status = ice_aq_delete_sched_elems(hw, 1, buf, buf_size,
+ &num_groups_removed, NULL);
+ if (status || num_groups_removed != 1)
+ ice_debug(hw, ICE_DBG_SCHED, "remove elements failed\n");
+ devm_kfree(ice_hw_to_dev(hw), buf);
+ return status;
+}
+
+/**
+ * ice_sched_get_first_node - get the first node of the given layer
+ * @hw: pointer to the hw struct
+ * @parent: pointer the base node of the subtree
+ * @layer: layer number
+ *
+ * This function retrieves the first node of the given layer from the subtree
+ */
+static struct ice_sched_node *
+ice_sched_get_first_node(struct ice_hw *hw, struct ice_sched_node *parent,
+ u8 layer)
+{
+ u8 i;
+
+ if (layer < hw->sw_entry_point_layer)
+ return NULL;
+ for (i = 0; i < parent->num_children; i++) {
+ struct ice_sched_node *node = parent->children[i];
+
+ if (node) {
+ if (node->tx_sched_layer == layer)
+ return node;
+ /* this recursion is intentional, and wouldn't
+ * go more than 9 calls
+ */
+ return ice_sched_get_first_node(hw, node, layer);
+ }
+ }
+ return NULL;
+}
+
+/**
+ * ice_sched_get_tc_node - get pointer to TC node
+ * @pi: port information structure
+ * @tc: TC number
+ *
+ * This function returns the TC node pointer
+ */
+struct ice_sched_node *ice_sched_get_tc_node(struct ice_port_info *pi, u8 tc)
+{
+ u8 i;
+
+ if (!pi)
+ return NULL;
+ for (i = 0; i < pi->root->num_children; i++)
+ if (pi->root->children[i]->tc_num == tc)
+ return pi->root->children[i];
+ return NULL;
+}
+
+/**
+ * ice_free_sched_node - Free a Tx scheduler node from SW DB
+ * @pi: port information structure
+ * @node: pointer to the ice_sched_node struct
+ *
+ * This function frees up a node from SW DB as well as from HW
+ *
+ * This function needs to be called with the port_info->sched_lock held
+ */
+void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node)
+{
+ struct ice_sched_node *parent;
+ struct ice_hw *hw = pi->hw;
+ u8 i, j;
+
+ /* Free the children before freeing up the parent node
+ * The parent array is updated below and that shifts the nodes
+ * in the array. So always pick the first child if num children > 0
+ */
+ while (node->num_children)
+ ice_free_sched_node(pi, node->children[0]);
+
+ /* Leaf, TC and root nodes can't be deleted by SW */
+ if (node->tx_sched_layer >= hw->sw_entry_point_layer &&
+ node->info.data.elem_type != ICE_AQC_ELEM_TYPE_TC &&
+ node->info.data.elem_type != ICE_AQC_ELEM_TYPE_ROOT_PORT &&
+ node->info.data.elem_type != ICE_AQC_ELEM_TYPE_LEAF) {
+ u32 teid = le32_to_cpu(node->info.node_teid);
+ enum ice_status status;
+
+ status = ice_sched_remove_elems(hw, node->parent, 1, &teid);
+ if (status)
+ ice_debug(hw, ICE_DBG_SCHED,
+ "remove element failed %d\n", status);
+ }
+ parent = node->parent;
+ /* root has no parent */
+ if (parent) {
+ struct ice_sched_node *p, *tc_node;
+
+ /* update the parent */
+ for (i = 0; i < parent->num_children; i++)
+ if (parent->children[i] == node) {
+ for (j = i + 1; j < parent->num_children; j++)
+ parent->children[j - 1] =
+ parent->children[j];
+ parent->num_children--;
+ break;
+ }
+
+ /* search for previous sibling that points to this node and
+ * remove the reference
+ */
+ tc_node = ice_sched_get_tc_node(pi, node->tc_num);
+ if (!tc_node) {
+ ice_debug(hw, ICE_DBG_SCHED,
+ "Invalid TC number %d\n", node->tc_num);
+ goto err_exit;
+ }
+ p = ice_sched_get_first_node(hw, tc_node, node->tx_sched_layer);
+ while (p) {
+ if (p->sibling == node) {
+ p->sibling = node->sibling;
+ break;
+ }
+ p = p->sibling;
+ }
+ }
+err_exit:
+ /* leaf nodes have no children */
+ if (node->children)
+ devm_kfree(ice_hw_to_dev(hw), node->children);
+ devm_kfree(ice_hw_to_dev(hw), node);
+}
+
+/**
+ * ice_aq_get_dflt_topo - gets default scheduler topology
+ * @hw: pointer to the hw struct
+ * @lport: logical port number
+ * @buf: pointer to buffer
+ * @buf_size: buffer size in bytes
+ * @num_branches: returns total number of queue to port branches
+ * @cd: pointer to command details structure or NULL
+ *
+ * Get default scheduler topology (0x400)
+ */
+static enum ice_status
+ice_aq_get_dflt_topo(struct ice_hw *hw, u8 lport,
+ struct ice_aqc_get_topo_elem *buf, u16 buf_size,
+ u8 *num_branches, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_get_topo *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.get_topo;
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_dflt_topo);
+ cmd->port_num = lport;
+ status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+ if (!status && num_branches)
+ *num_branches = cmd->num_branches;
+
+ return status;
+}
+
+/**
+ * ice_aq_add_sched_elems - adds scheduling element
+ * @hw: pointer to the hw struct
+ * @grps_req: the number of groups that are requested to be added
+ * @buf: pointer to buffer
+ * @buf_size: buffer size in bytes
+ * @grps_added: returns total number of groups added
+ * @cd: pointer to command details structure or NULL
+ *
+ * Add scheduling elements (0x0401)
+ */
+static enum ice_status
+ice_aq_add_sched_elems(struct ice_hw *hw, u16 grps_req,
+ struct ice_aqc_add_elem *buf, u16 buf_size,
+ u16 *grps_added, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_add_move_delete_elem *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.add_move_delete_elem;
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_add_sched_elems);
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+
+ cmd->num_grps_req = cpu_to_le16(grps_req);
+ status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+ if (!status && grps_added)
+ *grps_added = le16_to_cpu(cmd->num_grps_updated);
+
+ return status;
+}
+
+/**
+ * ice_suspend_resume_elems - suspend/resume scheduler elements
+ * @hw: pointer to the hw struct
+ * @elems_req: number of elements to suspend
+ * @buf: pointer to buffer
+ * @buf_size: buffer size in bytes
+ * @elems_ret: returns total number of elements suspended
+ * @cd: pointer to command details structure or NULL
+ * @cmd_code: command code for suspend or resume
+ *
+ * suspend/resume scheduler elements
+ */
+static enum ice_status
+ice_suspend_resume_elems(struct ice_hw *hw, u16 elems_req,
+ struct ice_aqc_suspend_resume_elem *buf, u16 buf_size,
+ u16 *elems_ret, struct ice_sq_cd *cd,
+ enum ice_adminq_opc cmd_code)
+{
+ struct ice_aqc_get_cfg_elem *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.get_update_elem;
+ ice_fill_dflt_direct_cmd_desc(&desc, cmd_code);
+ cmd->num_elem_req = cpu_to_le16(elems_req);
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+ status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+ if (!status && elems_ret)
+ *elems_ret = le16_to_cpu(cmd->num_elem_resp);
+ return status;
+}
+
+/**
+ * ice_aq_suspend_sched_elems - suspend scheduler elements
+ * @hw: pointer to the hw struct
+ * @elems_req: number of elements to suspend
+ * @buf: pointer to buffer
+ * @buf_size: buffer size in bytes
+ * @elems_ret: returns total number of elements suspended
+ * @cd: pointer to command details structure or NULL
+ *
+ * Suspend scheduling elements (0x0409)
+ */
+static enum ice_status
+ice_aq_suspend_sched_elems(struct ice_hw *hw, u16 elems_req,
+ struct ice_aqc_suspend_resume_elem *buf,
+ u16 buf_size, u16 *elems_ret, struct ice_sq_cd *cd)
+{
+ return ice_suspend_resume_elems(hw, elems_req, buf, buf_size, elems_ret,
+ cd, ice_aqc_opc_suspend_sched_elems);
+}
+
+/**
+ * ice_aq_resume_sched_elems - resume scheduler elements
+ * @hw: pointer to the hw struct
+ * @elems_req: number of elements to resume
+ * @buf: pointer to buffer
+ * @buf_size: buffer size in bytes
+ * @elems_ret: returns total number of elements resumed
+ * @cd: pointer to command details structure or NULL
+ *
+ * resume scheduling elements (0x040A)
+ */
+static enum ice_status
+ice_aq_resume_sched_elems(struct ice_hw *hw, u16 elems_req,
+ struct ice_aqc_suspend_resume_elem *buf,
+ u16 buf_size, u16 *elems_ret, struct ice_sq_cd *cd)
+{
+ return ice_suspend_resume_elems(hw, elems_req, buf, buf_size, elems_ret,
+ cd, ice_aqc_opc_resume_sched_elems);
+}
+
+/**
+ * ice_aq_query_sched_res - query scheduler resource
+ * @hw: pointer to the hw struct
+ * @buf_size: buffer size in bytes
+ * @buf: pointer to buffer
+ * @cd: pointer to command details structure or NULL
+ *
+ * Query scheduler resource allocation (0x0412)
+ */
+static enum ice_status
+ice_aq_query_sched_res(struct ice_hw *hw, u16 buf_size,
+ struct ice_aqc_query_txsched_res_resp *buf,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aq_desc desc;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_query_sched_res);
+ return ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+}
+
+/**
+ * ice_sched_suspend_resume_elems - suspend or resume hw nodes
+ * @hw: pointer to the hw struct
+ * @num_nodes: number of nodes
+ * @node_teids: array of node teids to be suspended or resumed
+ * @suspend: true means suspend / false means resume
+ *
+ * This function suspends or resumes hw nodes
+ */
+static enum ice_status
+ice_sched_suspend_resume_elems(struct ice_hw *hw, u8 num_nodes, u32 *node_teids,
+ bool suspend)
+{
+ struct ice_aqc_suspend_resume_elem *buf;
+ u16 i, buf_size, num_elem_ret = 0;
+ enum ice_status status;
+
+ buf_size = sizeof(*buf) * num_nodes;
+ buf = devm_kzalloc(ice_hw_to_dev(hw), buf_size, GFP_KERNEL);
+ if (!buf)
+ return ICE_ERR_NO_MEMORY;
+
+ for (i = 0; i < num_nodes; i++)
+ buf->teid[i] = cpu_to_le32(node_teids[i]);
+
+ if (suspend)
+ status = ice_aq_suspend_sched_elems(hw, num_nodes, buf,
+ buf_size, &num_elem_ret,
+ NULL);
+ else
+ status = ice_aq_resume_sched_elems(hw, num_nodes, buf,
+ buf_size, &num_elem_ret,
+ NULL);
+ if (status || num_elem_ret != num_nodes)
+ ice_debug(hw, ICE_DBG_SCHED, "suspend/resume failed\n");
+
+ devm_kfree(ice_hw_to_dev(hw), buf);
+ return status;
+}
+
+/**
+ * ice_sched_clear_tx_topo - clears the schduler tree nodes
+ * @pi: port information structure
+ *
+ * This function removes all the nodes from HW as well as from SW DB.
+ */
+static void ice_sched_clear_tx_topo(struct ice_port_info *pi)
+{
+ struct ice_sched_agg_info *agg_info;
+ struct ice_sched_vsi_info *vsi_elem;
+ struct ice_sched_agg_info *atmp;
+ struct ice_sched_vsi_info *tmp;
+ struct ice_hw *hw;
+
+ if (!pi)
+ return;
+
+ hw = pi->hw;
+
+ list_for_each_entry_safe(agg_info, atmp, &pi->agg_list, list_entry) {
+ struct ice_sched_agg_vsi_info *agg_vsi_info;
+ struct ice_sched_agg_vsi_info *vtmp;
+
+ list_for_each_entry_safe(agg_vsi_info, vtmp,
+ &agg_info->agg_vsi_list, list_entry) {
+ list_del(&agg_vsi_info->list_entry);
+ devm_kfree(ice_hw_to_dev(hw), agg_vsi_info);
+ }
+ }
+
+ /* remove the vsi list */
+ list_for_each_entry_safe(vsi_elem, tmp, &pi->vsi_info_list,
+ list_entry) {
+ list_del(&vsi_elem->list_entry);
+ devm_kfree(ice_hw_to_dev(hw), vsi_elem);
+ }
+
+ if (pi->root) {
+ ice_free_sched_node(pi, pi->root);
+ pi->root = NULL;
+ }
+}
+
+/**
+ * ice_sched_clear_port - clear the scheduler elements from SW DB for a port
+ * @pi: port information structure
+ *
+ * Cleanup scheduling elements from SW DB
+ */
+static void ice_sched_clear_port(struct ice_port_info *pi)
+{
+ if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
+ return;
+
+ pi->port_state = ICE_SCHED_PORT_STATE_INIT;
+ mutex_lock(&pi->sched_lock);
+ ice_sched_clear_tx_topo(pi);
+ mutex_unlock(&pi->sched_lock);
+ mutex_destroy(&pi->sched_lock);
+}
+
+/**
+ * ice_sched_cleanup_all - cleanup scheduler elements from SW DB for all ports
+ * @hw: pointer to the hw struct
+ *
+ * Cleanup scheduling elements from SW DB for all the ports
+ */
+void ice_sched_cleanup_all(struct ice_hw *hw)
+{
+ if (!hw || !hw->port_info)
+ return;
+
+ if (hw->layer_info)
+ devm_kfree(ice_hw_to_dev(hw), hw->layer_info);
+
+ ice_sched_clear_port(hw->port_info);
+
+ hw->num_tx_sched_layers = 0;
+ hw->num_tx_sched_phys_layers = 0;
+ hw->flattened_layers = 0;
+ hw->max_cgds = 0;
+}
+
+/**
+ * ice_sched_create_vsi_info_entry - create an empty new VSI entry
+ * @pi: port information structure
+ * @vsi_id: VSI Id
+ *
+ * This function creates a new VSI entry and adds it to list
+ */
+static struct ice_sched_vsi_info *
+ice_sched_create_vsi_info_entry(struct ice_port_info *pi, u16 vsi_id)
+{
+ struct ice_sched_vsi_info *vsi_elem;
+
+ if (!pi)
+ return NULL;
+
+ vsi_elem = devm_kzalloc(ice_hw_to_dev(pi->hw), sizeof(*vsi_elem),
+ GFP_KERNEL);
+ if (!vsi_elem)
+ return NULL;
+
+ list_add(&vsi_elem->list_entry, &pi->vsi_info_list);
+ vsi_elem->vsi_id = vsi_id;
+ return vsi_elem;
+}
+
+/**
+ * ice_sched_add_elems - add nodes to hw and SW DB
+ * @pi: port information structure
+ * @tc_node: pointer to the branch node
+ * @parent: pointer to the parent node
+ * @layer: layer number to add nodes
+ * @num_nodes: number of nodes
+ * @num_nodes_added: pointer to num nodes added
+ * @first_node_teid: if new nodes are added then return the teid of first node
+ *
+ * This function add nodes to hw as well as to SW DB for a given layer
+ */
+static enum ice_status
+ice_sched_add_elems(struct ice_port_info *pi, struct ice_sched_node *tc_node,
+ struct ice_sched_node *parent, u8 layer, u16 num_nodes,
+ u16 *num_nodes_added, u32 *first_node_teid)
+{
+ struct ice_sched_node *prev, *new_node;
+ struct ice_aqc_add_elem *buf;
+ u16 i, num_groups_added = 0;
+ enum ice_status status = 0;
+ struct ice_hw *hw = pi->hw;
+ u16 buf_size;
+ u32 teid;
+
+ buf_size = sizeof(*buf) + sizeof(*buf->generic) * (num_nodes - 1);
+ buf = devm_kzalloc(ice_hw_to_dev(hw), buf_size, GFP_KERNEL);
+ if (!buf)
+ return ICE_ERR_NO_MEMORY;
+
+ buf->hdr.parent_teid = parent->info.node_teid;
+ buf->hdr.num_elems = cpu_to_le16(num_nodes);
+ for (i = 0; i < num_nodes; i++) {
+ buf->generic[i].parent_teid = parent->info.node_teid;
+ buf->generic[i].data.elem_type = ICE_AQC_ELEM_TYPE_SE_GENERIC;
+ buf->generic[i].data.valid_sections =
+ ICE_AQC_ELEM_VALID_GENERIC | ICE_AQC_ELEM_VALID_CIR |
+ ICE_AQC_ELEM_VALID_EIR;
+ buf->generic[i].data.generic = 0;
+ buf->generic[i].data.cir_bw.bw_profile_idx =
+ ICE_SCHED_DFLT_RL_PROF_ID;
+ buf->generic[i].data.eir_bw.bw_profile_idx =
+ ICE_SCHED_DFLT_RL_PROF_ID;
+ }
+
+ status = ice_aq_add_sched_elems(hw, 1, buf, buf_size,
+ &num_groups_added, NULL);
+ if (status || num_groups_added != 1) {
+ ice_debug(hw, ICE_DBG_SCHED, "add elements failed\n");
+ devm_kfree(ice_hw_to_dev(hw), buf);
+ return ICE_ERR_CFG;
+ }
+
+ *num_nodes_added = num_nodes;
+ /* add nodes to the SW DB */
+ for (i = 0; i < num_nodes; i++) {
+ status = ice_sched_add_node(pi, layer, &buf->generic[i]);
+ if (status) {
+ ice_debug(hw, ICE_DBG_SCHED,
+ "add nodes in SW DB failed status =%d\n",
+ status);
+ break;
+ }
+
+ teid = le32_to_cpu(buf->generic[i].node_teid);
+ new_node = ice_sched_find_node_by_teid(parent, teid);
+
+ if (!new_node) {
+ ice_debug(hw, ICE_DBG_SCHED,
+ "Node is missing for teid =%d\n", teid);
+ break;
+ }
+
+ new_node->sibling = NULL;
+ new_node->tc_num = tc_node->tc_num;
+
+ /* add it to previous node sibling pointer */
+ /* Note: siblings are not linked across branches */
+ prev = ice_sched_get_first_node(hw, tc_node, layer);
+
+ if (prev && prev != new_node) {
+ while (prev->sibling)
+ prev = prev->sibling;
+ prev->sibling = new_node;
+ }
+
+ if (i == 0)
+ *first_node_teid = teid;
+ }
+
+ devm_kfree(ice_hw_to_dev(hw), buf);
+ return status;
+}
+
+/**
+ * ice_sched_add_nodes_to_layer - Add nodes to a given layer
+ * @pi: port information structure
+ * @tc_node: pointer to TC node
+ * @parent: pointer to parent node
+ * @layer: layer number to add nodes
+ * @num_nodes: number of nodes to be added
+ * @first_node_teid: pointer to the first node teid
+ * @num_nodes_added: pointer to number of nodes added
+ *
+ * This function add nodes to a given layer.
+ */
+static enum ice_status
+ice_sched_add_nodes_to_layer(struct ice_port_info *pi,
+ struct ice_sched_node *tc_node,
+ struct ice_sched_node *parent, u8 layer,
+ u16 num_nodes, u32 *first_node_teid,
+ u16 *num_nodes_added)
+{
+ u32 *first_teid_ptr = first_node_teid;
+ u16 new_num_nodes, max_child_nodes;
+ enum ice_status status = 0;
+ struct ice_hw *hw = pi->hw;
+ u16 num_added = 0;
+ u32 temp;
+
+ if (!num_nodes)
+ return status;
+
+ if (!parent || layer < hw->sw_entry_point_layer)
+ return ICE_ERR_PARAM;
+
+ *num_nodes_added = 0;
+
+ /* max children per node per layer */
+ max_child_nodes =
+ le16_to_cpu(hw->layer_info[parent->tx_sched_layer].max_children);
+
+ /* current number of children + required nodes exceed max children ? */
+ if ((parent->num_children + num_nodes) > max_child_nodes) {
+ /* Fail if the parent is a TC node */
+ if (parent == tc_node)
+ return ICE_ERR_CFG;
+
+ /* utilize all the spaces if the parent is not full */
+ if (parent->num_children < max_child_nodes) {
+ new_num_nodes = max_child_nodes - parent->num_children;
+ /* this recursion is intentional, and wouldn't
+ * go more than 2 calls
+ */
+ status = ice_sched_add_nodes_to_layer(pi, tc_node,
+ parent, layer,
+ new_num_nodes,
+ first_node_teid,
+ &num_added);
+ if (status)
+ return status;
+
+ *num_nodes_added += num_added;
+ }
+ /* Don't modify the first node teid memory if the first node was
+ * added already in the above call. Instead send some temp
+ * memory for all other recursive calls.
+ */
+ if (num_added)
+ first_teid_ptr = &temp;
+
+ new_num_nodes = num_nodes - num_added;
+
+ /* This parent is full, try the next sibling */
+ parent = parent->sibling;
+
+ /* this recursion is intentional, for 1024 queues
+ * per VSI, it goes max of 16 iterations.
+ * 1024 / 8 = 128 layer 8 nodes
+ * 128 /8 = 16 (add 8 nodes per iteration)
+ */
+ status = ice_sched_add_nodes_to_layer(pi, tc_node, parent,
+ layer, new_num_nodes,
+ first_teid_ptr,
+ &num_added);
+ *num_nodes_added += num_added;
+ return status;
+ }
+
+ status = ice_sched_add_elems(pi, tc_node, parent, layer, num_nodes,
+ num_nodes_added, first_node_teid);
+ return status;
+}
+
+/**
+ * ice_sched_get_qgrp_layer - get the current queue group layer number
+ * @hw: pointer to the hw struct
+ *
+ * This function returns the current queue group layer number
+ */
+static u8 ice_sched_get_qgrp_layer(struct ice_hw *hw)
+{
+ /* It's always total layers - 1, the array is 0 relative so -2 */
+ return hw->num_tx_sched_layers - ICE_QGRP_LAYER_OFFSET;
+}
+
+/**
+ * ice_sched_get_vsi_layer - get the current VSI layer number
+ * @hw: pointer to the hw struct
+ *
+ * This function returns the current VSI layer number
+ */
+static u8 ice_sched_get_vsi_layer(struct ice_hw *hw)
+{
+ /* Num Layers VSI layer
+ * 9 6
+ * 7 4
+ * 5 or less sw_entry_point_layer
+ */
+ /* calculate the vsi layer based on number of layers. */
+ if (hw->num_tx_sched_layers > ICE_VSI_LAYER_OFFSET + 1) {
+ u8 layer = hw->num_tx_sched_layers - ICE_VSI_LAYER_OFFSET;
+
+ if (layer > hw->sw_entry_point_layer)
+ return layer;
+ }
+ return hw->sw_entry_point_layer;
+}
+
+/**
+ * ice_sched_get_num_nodes_per_layer - Get the total number of nodes per layer
+ * @pi: pointer to the port info struct
+ * @layer: layer number
+ *
+ * This function calculates the number of nodes present in the scheduler tree
+ * including all the branches for a given layer
+ */
+static u16
+ice_sched_get_num_nodes_per_layer(struct ice_port_info *pi, u8 layer)
+{
+ struct ice_hw *hw;
+ u16 num_nodes = 0;
+ u8 i;
+
+ if (!pi)
+ return num_nodes;
+
+ hw = pi->hw;
+
+ /* Calculate the number of nodes for all TCs */
+ for (i = 0; i < pi->root->num_children; i++) {
+ struct ice_sched_node *tc_node, *node;
+
+ tc_node = pi->root->children[i];
+
+ /* Get the first node */
+ node = ice_sched_get_first_node(hw, tc_node, layer);
+ if (!node)
+ continue;
+
+ /* count the siblings */
+ while (node) {
+ num_nodes++;
+ node = node->sibling;
+ }
+ }
+
+ return num_nodes;
+}
+
+/**
+ * ice_sched_val_max_nodes - check max number of nodes reached or not
+ * @pi: port information structure
+ * @new_num_nodes_per_layer: pointer to the new number of nodes array
+ *
+ * This function checks whether the scheduler tree layers have enough space to
+ * add new nodes
+ */
+static enum ice_status
+ice_sched_validate_for_max_nodes(struct ice_port_info *pi,
+ u16 *new_num_nodes_per_layer)
+{
+ struct ice_hw *hw = pi->hw;
+ u8 i, qg_layer;
+ u16 num_nodes;
+
+ qg_layer = ice_sched_get_qgrp_layer(hw);
+
+ /* walk through all the layers from SW entry point to qgroup layer */
+ for (i = hw->sw_entry_point_layer; i <= qg_layer; i++) {
+ num_nodes = ice_sched_get_num_nodes_per_layer(pi, i);
+ if (num_nodes + new_num_nodes_per_layer[i] >
+ le16_to_cpu(hw->layer_info[i].max_pf_nodes)) {
+ ice_debug(hw, ICE_DBG_SCHED,
+ "max nodes reached for layer = %d\n", i);
+ return ICE_ERR_CFG;
+ }
+ }
+ return 0;
+}
+
+/**
+ * ice_rm_dflt_leaf_node - remove the default leaf node in the tree
+ * @pi: port information structure
+ *
+ * This function removes the leaf node that was created by the FW
+ * during initialization
+ */
+static void
+ice_rm_dflt_leaf_node(struct ice_port_info *pi)
+{
+ struct ice_sched_node *node;
+
+ node = pi->root;
+ while (node) {
+ if (!node->num_children)
+ break;
+ node = node->children[0];
+ }
+ if (node && node->info.data.elem_type == ICE_AQC_ELEM_TYPE_LEAF) {
+ u32 teid = le32_to_cpu(node->info.node_teid);
+ enum ice_status status;
+
+ /* remove the default leaf node */
+ status = ice_sched_remove_elems(pi->hw, node->parent, 1, &teid);
+ if (!status)
+ ice_free_sched_node(pi, node);
+ }
+}
+
+/**
+ * ice_sched_rm_dflt_nodes - free the default nodes in the tree
+ * @pi: port information structure
+ *
+ * This function frees all the nodes except root and TC that were created by
+ * the FW during initialization
+ */
+static void
+ice_sched_rm_dflt_nodes(struct ice_port_info *pi)
+{
+ struct ice_sched_node *node;
+
+ ice_rm_dflt_leaf_node(pi);
+
+ /* remove the default nodes except TC and root nodes */
+ node = pi->root;
+ while (node) {
+ if (node->tx_sched_layer >= pi->hw->sw_entry_point_layer &&
+ node->info.data.elem_type != ICE_AQC_ELEM_TYPE_TC &&
+ node->info.data.elem_type != ICE_AQC_ELEM_TYPE_ROOT_PORT) {
+ ice_free_sched_node(pi, node);
+ break;
+ }
+
+ if (!node->num_children)
+ break;
+ node = node->children[0];
+ }
+}
+
+/**
+ * ice_sched_init_port - Initialize scheduler by querying information from FW
+ * @pi: port info structure for the tree to cleanup
+ *
+ * This function is the initial call to find the total number of Tx scheduler
+ * resources, default topology created by firmware and storing the information
+ * in SW DB.
+ */
+enum ice_status ice_sched_init_port(struct ice_port_info *pi)
+{
+ struct ice_aqc_get_topo_elem *buf;
+ enum ice_status status;
+ struct ice_hw *hw;
+ u8 num_branches;
+ u16 num_elems;
+ u8 i, j;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+ hw = pi->hw;
+
+ /* Query the Default Topology from FW */
+ buf = devm_kcalloc(ice_hw_to_dev(hw), ICE_TXSCHED_MAX_BRANCHES,
+ sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ return ICE_ERR_NO_MEMORY;
+
+ /* Query default scheduling tree topology */
+ status = ice_aq_get_dflt_topo(hw, pi->lport, buf,
+ sizeof(*buf) * ICE_TXSCHED_MAX_BRANCHES,
+ &num_branches, NULL);
+ if (status)
+ goto err_init_port;
+
+ /* num_branches should be between 1-8 */
+ if (num_branches < 1 || num_branches > ICE_TXSCHED_MAX_BRANCHES) {
+ ice_debug(hw, ICE_DBG_SCHED, "num_branches unexpected %d\n",
+ num_branches);
+ status = ICE_ERR_PARAM;
+ goto err_init_port;
+ }
+
+ /* get the number of elements on the default/first branch */
+ num_elems = le16_to_cpu(buf[0].hdr.num_elems);
+
+ /* num_elems should always be between 1-9 */
+ if (num_elems < 1 || num_elems > ICE_AQC_TOPO_MAX_LEVEL_NUM) {
+ ice_debug(hw, ICE_DBG_SCHED, "num_elems unexpected %d\n",
+ num_elems);
+ status = ICE_ERR_PARAM;
+ goto err_init_port;
+ }
+
+ /* If the last node is a leaf node then the index of the Q group
+ * layer is two less than the number of elements.
+ */
+ if (num_elems > 2 && buf[0].generic[num_elems - 1].data.elem_type ==
+ ICE_AQC_ELEM_TYPE_LEAF)
+ pi->last_node_teid =
+ le32_to_cpu(buf[0].generic[num_elems - 2].node_teid);
+ else
+ pi->last_node_teid =
+ le32_to_cpu(buf[0].generic[num_elems - 1].node_teid);
+
+ /* Insert the Tx Sched root node */
+ status = ice_sched_add_root_node(pi, &buf[0].generic[0]);
+ if (status)
+ goto err_init_port;
+
+ /* Parse the default tree and cache the information */
+ for (i = 0; i < num_branches; i++) {
+ num_elems = le16_to_cpu(buf[i].hdr.num_elems);
+
+ /* Skip root element as already inserted */
+ for (j = 1; j < num_elems; j++) {
+ /* update the sw entry point */
+ if (buf[0].generic[j].data.elem_type ==
+ ICE_AQC_ELEM_TYPE_ENTRY_POINT)
+ hw->sw_entry_point_layer = j;
+
+ status = ice_sched_add_node(pi, j, &buf[i].generic[j]);
+ if (status)
+ goto err_init_port;
+ }
+ }
+
+ /* Remove the default nodes. */
+ if (pi->root)
+ ice_sched_rm_dflt_nodes(pi);
+
+ /* initialize the port for handling the scheduler tree */
+ pi->port_state = ICE_SCHED_PORT_STATE_READY;
+ mutex_init(&pi->sched_lock);
+ INIT_LIST_HEAD(&pi->agg_list);
+ INIT_LIST_HEAD(&pi->vsi_info_list);
+
+err_init_port:
+ if (status && pi->root) {
+ ice_free_sched_node(pi, pi->root);
+ pi->root = NULL;
+ }
+
+ devm_kfree(ice_hw_to_dev(hw), buf);
+ return status;
+}
+
+/**
+ * ice_sched_query_res_alloc - query the FW for num of logical sched layers
+ * @hw: pointer to the HW struct
+ *
+ * query FW for allocated scheduler resources and store in HW struct
+ */
+enum ice_status ice_sched_query_res_alloc(struct ice_hw *hw)
+{
+ struct ice_aqc_query_txsched_res_resp *buf;
+ enum ice_status status = 0;
+
+ if (hw->layer_info)
+ return status;
+
+ buf = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ return ICE_ERR_NO_MEMORY;
+
+ status = ice_aq_query_sched_res(hw, sizeof(*buf), buf, NULL);
+ if (status)
+ goto sched_query_out;
+
+ hw->num_tx_sched_layers = le16_to_cpu(buf->sched_props.logical_levels);
+ hw->num_tx_sched_phys_layers =
+ le16_to_cpu(buf->sched_props.phys_levels);
+ hw->flattened_layers = buf->sched_props.flattening_bitmap;
+ hw->max_cgds = buf->sched_props.max_pf_cgds;
+
+ hw->layer_info = devm_kmemdup(ice_hw_to_dev(hw), buf->layer_props,
+ (hw->num_tx_sched_layers *
+ sizeof(*hw->layer_info)),
+ GFP_KERNEL);
+ if (!hw->layer_info) {
+ status = ICE_ERR_NO_MEMORY;
+ goto sched_query_out;
+ }
+
+sched_query_out:
+ devm_kfree(ice_hw_to_dev(hw), buf);
+ return status;
+}
+
+/**
+ * ice_sched_get_vsi_info_entry - Get the vsi entry list for given vsi_id
+ * @pi: port information structure
+ * @vsi_id: vsi id
+ *
+ * This function retrieves the vsi list for the given vsi id
+ */
+static struct ice_sched_vsi_info *
+ice_sched_get_vsi_info_entry(struct ice_port_info *pi, u16 vsi_id)
+{
+ struct ice_sched_vsi_info *list_elem;
+
+ if (!pi)
+ return NULL;
+
+ list_for_each_entry(list_elem, &pi->vsi_info_list, list_entry)
+ if (list_elem->vsi_id == vsi_id)
+ return list_elem;
+ return NULL;
+}
+
+/**
+ * ice_sched_find_node_in_subtree - Find node in part of base node subtree
+ * @hw: pointer to the hw struct
+ * @base: pointer to the base node
+ * @node: pointer to the node to search
+ *
+ * This function checks whether a given node is part of the base node
+ * subtree or not
+ */
+static bool
+ice_sched_find_node_in_subtree(struct ice_hw *hw, struct ice_sched_node *base,
+ struct ice_sched_node *node)
+{
+ u8 i;
+
+ for (i = 0; i < base->num_children; i++) {
+ struct ice_sched_node *child = base->children[i];
+
+ if (node == child)
+ return true;
+
+ if (child->tx_sched_layer > node->tx_sched_layer)
+ return false;
+
+ /* this recursion is intentional, and wouldn't
+ * go more than 8 calls
+ */
+ if (ice_sched_find_node_in_subtree(hw, child, node))
+ return true;
+ }
+ return false;
+}
+
+/**
+ * ice_sched_get_free_qparent - Get a free lan or rdma q group node
+ * @pi: port information structure
+ * @vsi_id: vsi id
+ * @tc: branch number
+ * @owner: lan or rdma
+ *
+ * This function retrieves a free lan or rdma q group node
+ */
+struct ice_sched_node *
+ice_sched_get_free_qparent(struct ice_port_info *pi, u16 vsi_id, u8 tc,
+ u8 owner)
+{
+ struct ice_sched_node *vsi_node, *qgrp_node = NULL;
+ struct ice_sched_vsi_info *list_elem;
+ u16 max_children;
+ u8 qgrp_layer;
+
+ qgrp_layer = ice_sched_get_qgrp_layer(pi->hw);
+ max_children = le16_to_cpu(pi->hw->layer_info[qgrp_layer].max_children);
+
+ list_elem = ice_sched_get_vsi_info_entry(pi, vsi_id);
+ if (!list_elem)
+ goto lan_q_exit;
+
+ vsi_node = list_elem->vsi_node[tc];
+
+ /* validate invalid VSI id */
+ if (!vsi_node)
+ goto lan_q_exit;
+
+ /* get the first q group node from VSI sub-tree */
+ qgrp_node = ice_sched_get_first_node(pi->hw, vsi_node, qgrp_layer);
+ while (qgrp_node) {
+ /* make sure the qgroup node is part of the VSI subtree */
+ if (ice_sched_find_node_in_subtree(pi->hw, vsi_node, qgrp_node))
+ if (qgrp_node->num_children < max_children &&
+ qgrp_node->owner == owner)
+ break;
+ qgrp_node = qgrp_node->sibling;
+ }
+
+lan_q_exit:
+ return qgrp_node;
+}
+
+/**
+ * ice_sched_get_vsi_node - Get a VSI node based on VSI id
+ * @hw: pointer to the hw struct
+ * @tc_node: pointer to the TC node
+ * @vsi_id: VSI id
+ *
+ * This function retrieves a VSI node for a given VSI id from a given
+ * TC branch
+ */
+static struct ice_sched_node *
+ice_sched_get_vsi_node(struct ice_hw *hw, struct ice_sched_node *tc_node,
+ u16 vsi_id)
+{
+ struct ice_sched_node *node;
+ u8 vsi_layer;
+
+ vsi_layer = ice_sched_get_vsi_layer(hw);
+ node = ice_sched_get_first_node(hw, tc_node, vsi_layer);
+
+ /* Check whether it already exists */
+ while (node) {
+ if (node->vsi_id == vsi_id)
+ return node;
+ node = node->sibling;
+ }
+
+ return node;
+}
+
+/**
+ * ice_sched_calc_vsi_child_nodes - calculate number of VSI child nodes
+ * @hw: pointer to the hw struct
+ * @num_qs: number of queues
+ * @num_nodes: num nodes array
+ *
+ * This function calculates the number of VSI child nodes based on the
+ * number of queues.
+ */
+static void
+ice_sched_calc_vsi_child_nodes(struct ice_hw *hw, u16 num_qs, u16 *num_nodes)
+{
+ u16 num = num_qs;
+ u8 i, qgl, vsil;
+
+ qgl = ice_sched_get_qgrp_layer(hw);
+ vsil = ice_sched_get_vsi_layer(hw);
+
+ /* calculate num nodes from q group to VSI layer */
+ for (i = qgl; i > vsil; i--) {
+ u16 max_children = le16_to_cpu(hw->layer_info[i].max_children);
+
+ /* round to the next integer if there is a remainder */
+ num = DIV_ROUND_UP(num, max_children);
+
+ /* need at least one node */
+ num_nodes[i] = num ? num : 1;
+ }
+}
+
+/**
+ * ice_sched_add_vsi_child_nodes - add VSI child nodes to tree
+ * @pi: port information structure
+ * @vsi_id: VSI id
+ * @tc_node: pointer to the TC node
+ * @num_nodes: pointer to the num nodes that needs to be added per layer
+ * @owner: node owner (lan or rdma)
+ *
+ * This function adds the VSI child nodes to tree. It gets called for
+ * lan and rdma separately.
+ */
+static enum ice_status
+ice_sched_add_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_id,
+ struct ice_sched_node *tc_node, u16 *num_nodes,
+ u8 owner)
+{
+ struct ice_sched_node *parent, *node;
+ struct ice_hw *hw = pi->hw;
+ enum ice_status status;
+ u32 first_node_teid;
+ u16 num_added = 0;
+ u8 i, qgl, vsil;
+
+ status = ice_sched_validate_for_max_nodes(pi, num_nodes);
+ if (status)
+ return status;
+
+ qgl = ice_sched_get_qgrp_layer(hw);
+ vsil = ice_sched_get_vsi_layer(hw);
+ parent = ice_sched_get_vsi_node(hw, tc_node, vsi_id);
+ for (i = vsil + 1; i <= qgl; i++) {
+ if (!parent)
+ return ICE_ERR_CFG;
+ status = ice_sched_add_nodes_to_layer(pi, tc_node, parent, i,
+ num_nodes[i],
+ &first_node_teid,
+ &num_added);
+ if (status || num_nodes[i] != num_added)
+ return ICE_ERR_CFG;
+
+ /* The newly added node can be a new parent for the next
+ * layer nodes
+ */
+ if (num_added) {
+ parent = ice_sched_find_node_by_teid(tc_node,
+ first_node_teid);
+ node = parent;
+ while (node) {
+ node->owner = owner;
+ node = node->sibling;
+ }
+ } else {
+ parent = parent->children[0];
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ice_sched_rm_vsi_child_nodes - remove VSI child nodes from the tree
+ * @pi: port information structure
+ * @vsi_node: pointer to the VSI node
+ * @num_nodes: pointer to the num nodes that needs to be removed per layer
+ * @owner: node owner (lan or rdma)
+ *
+ * This function removes the VSI child nodes from the tree. It gets called for
+ * lan and rdma separately.
+ */
+static void
+ice_sched_rm_vsi_child_nodes(struct ice_port_info *pi,
+ struct ice_sched_node *vsi_node, u16 *num_nodes,
+ u8 owner)
+{
+ struct ice_sched_node *node, *next;
+ u8 i, qgl, vsil;
+ u16 num;
+
+ qgl = ice_sched_get_qgrp_layer(pi->hw);
+ vsil = ice_sched_get_vsi_layer(pi->hw);
+
+ for (i = qgl; i > vsil; i--) {
+ num = num_nodes[i];
+ node = ice_sched_get_first_node(pi->hw, vsi_node, i);
+ while (node && num) {
+ next = node->sibling;
+ if (node->owner == owner && !node->num_children) {
+ ice_free_sched_node(pi, node);
+ num--;
+ }
+ node = next;
+ }
+ }
+}
+
+/**
+ * ice_sched_calc_vsi_support_nodes - calculate number of VSI support nodes
+ * @hw: pointer to the hw struct
+ * @tc_node: pointer to TC node
+ * @num_nodes: pointer to num nodes array
+ *
+ * This function calculates the number of supported nodes needed to add this
+ * VSI into tx tree including the VSI, parent and intermediate nodes in below
+ * layers
+ */
+static void
+ice_sched_calc_vsi_support_nodes(struct ice_hw *hw,
+ struct ice_sched_node *tc_node, u16 *num_nodes)
+{
+ struct ice_sched_node *node;
+ u16 max_child;
+ u8 i, vsil;
+
+ vsil = ice_sched_get_vsi_layer(hw);
+ for (i = vsil; i >= hw->sw_entry_point_layer; i--)
+ /* Add intermediate nodes if TC has no children and
+ * need at least one node for VSI
+ */
+ if (!tc_node->num_children || i == vsil) {
+ num_nodes[i]++;
+ } else {
+ /* If intermediate nodes are reached max children
+ * then add a new one.
+ */
+ node = ice_sched_get_first_node(hw, tc_node, i);
+ max_child = le16_to_cpu(hw->layer_info[i].max_children);
+
+ /* scan all the siblings */
+ while (node) {
+ if (node->num_children < max_child)
+ break;
+ node = node->sibling;
+ }
+
+ /* all the nodes are full, allocate a new one */
+ if (!node)
+ num_nodes[i]++;
+ }
+}
+
+/**
+ * ice_sched_add_vsi_support_nodes - add VSI supported nodes into tx tree
+ * @pi: port information structure
+ * @vsi_id: VSI Id
+ * @tc_node: pointer to TC node
+ * @num_nodes: pointer to num nodes array
+ *
+ * This function adds the VSI supported nodes into tx tree including the
+ * VSI, its parent and intermediate nodes in below layers
+ */
+static enum ice_status
+ice_sched_add_vsi_support_nodes(struct ice_port_info *pi, u16 vsi_id,
+ struct ice_sched_node *tc_node, u16 *num_nodes)
+{
+ struct ice_sched_node *parent = tc_node;
+ enum ice_status status;
+ u32 first_node_teid;
+ u16 num_added = 0;
+ u8 i, vsil;
+
+ if (!pi)
+ return ICE_ERR_PARAM;
+
+ status = ice_sched_validate_for_max_nodes(pi, num_nodes);
+ if (status)
+ return status;
+
+ vsil = ice_sched_get_vsi_layer(pi->hw);
+ for (i = pi->hw->sw_entry_point_layer; i <= vsil; i++) {
+ status = ice_sched_add_nodes_to_layer(pi, tc_node, parent,
+ i, num_nodes[i],
+ &first_node_teid,
+ &num_added);
+ if (status || num_nodes[i] != num_added)
+ return ICE_ERR_CFG;
+
+ /* The newly added node can be a new parent for the next
+ * layer nodes
+ */
+ if (num_added)
+ parent = ice_sched_find_node_by_teid(tc_node,
+ first_node_teid);
+ else
+ parent = parent->children[0];
+
+ if (!parent)
+ return ICE_ERR_CFG;
+
+ if (i == vsil)
+ parent->vsi_id = vsi_id;
+ }
+ return 0;
+}
+
+/**
+ * ice_sched_add_vsi_to_topo - add a new VSI into tree
+ * @pi: port information structure
+ * @vsi_id: VSI Id
+ * @tc: TC number
+ *
+ * This function adds a new VSI into scheduler tree
+ */
+static enum ice_status
+ice_sched_add_vsi_to_topo(struct ice_port_info *pi, u16 vsi_id, u8 tc)
+{
+ u16 num_nodes[ICE_AQC_TOPO_MAX_LEVEL_NUM] = { 0 };
+ struct ice_sched_node *tc_node;
+ struct ice_hw *hw = pi->hw;
+
+ tc_node = ice_sched_get_tc_node(pi, tc);
+ if (!tc_node)
+ return ICE_ERR_PARAM;
+
+ /* calculate number of supported nodes needed for this VSI */
+ ice_sched_calc_vsi_support_nodes(hw, tc_node, num_nodes);
+
+ /* add vsi supported nodes to tc subtree */
+ return ice_sched_add_vsi_support_nodes(pi, vsi_id, tc_node, num_nodes);
+}
+
+/**
+ * ice_sched_update_vsi_child_nodes - update VSI child nodes
+ * @pi: port information structure
+ * @vsi_id: VSI Id
+ * @tc: TC number
+ * @new_numqs: new number of max queues
+ * @owner: owner of this subtree
+ *
+ * This function updates the VSI child nodes based on the number of queues
+ */
+static enum ice_status
+ice_sched_update_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_id, u8 tc,
+ u16 new_numqs, u8 owner)
+{
+ u16 prev_num_nodes[ICE_AQC_TOPO_MAX_LEVEL_NUM] = { 0 };
+ u16 new_num_nodes[ICE_AQC_TOPO_MAX_LEVEL_NUM] = { 0 };
+ struct ice_sched_node *vsi_node;
+ struct ice_sched_node *tc_node;
+ struct ice_sched_vsi_info *vsi;
+ enum ice_status status = 0;
+ struct ice_hw *hw = pi->hw;
+ u16 prev_numqs;
+ u8 i;
+
+ tc_node = ice_sched_get_tc_node(pi, tc);
+ if (!tc_node)
+ return ICE_ERR_CFG;
+
+ vsi_node = ice_sched_get_vsi_node(hw, tc_node, vsi_id);
+ if (!vsi_node)
+ return ICE_ERR_CFG;
+
+ vsi = ice_sched_get_vsi_info_entry(pi, vsi_id);
+ if (!vsi)
+ return ICE_ERR_CFG;
+
+ if (owner == ICE_SCHED_NODE_OWNER_LAN)
+ prev_numqs = vsi->max_lanq[tc];
+ else
+ return ICE_ERR_PARAM;
+
+ /* num queues are not changed */
+ if (prev_numqs == new_numqs)
+ return status;
+
+ /* calculate number of nodes based on prev/new number of qs */
+ if (prev_numqs)
+ ice_sched_calc_vsi_child_nodes(hw, prev_numqs, prev_num_nodes);
+
+ if (new_numqs)
+ ice_sched_calc_vsi_child_nodes(hw, new_numqs, new_num_nodes);
+
+ if (prev_numqs > new_numqs) {
+ for (i = 0; i < ICE_AQC_TOPO_MAX_LEVEL_NUM; i++)
+ new_num_nodes[i] = prev_num_nodes[i] - new_num_nodes[i];
+
+ ice_sched_rm_vsi_child_nodes(pi, vsi_node, new_num_nodes,
+ owner);
+ } else {
+ for (i = 0; i < ICE_AQC_TOPO_MAX_LEVEL_NUM; i++)
+ new_num_nodes[i] -= prev_num_nodes[i];
+
+ status = ice_sched_add_vsi_child_nodes(pi, vsi_id, tc_node,
+ new_num_nodes, owner);
+ if (status)
+ return status;
+ }
+
+ if (owner == ICE_SCHED_NODE_OWNER_LAN)
+ vsi->max_lanq[tc] = new_numqs;
+
+ return status;
+}
+
+/**
+ * ice_sched_cfg_vsi - configure the new/exisiting VSI
+ * @pi: port information structure
+ * @vsi_id: VSI Id
+ * @tc: TC number
+ * @maxqs: max number of queues
+ * @owner: lan or rdma
+ * @enable: TC enabled or disabled
+ *
+ * This function adds/updates VSI nodes based on the number of queues. If TC is
+ * enabled and VSI is in suspended state then resume the VSI back. If TC is
+ * disabled then suspend the VSI if it is not already.
+ */
+enum ice_status
+ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_id, u8 tc, u16 maxqs,
+ u8 owner, bool enable)
+{
+ struct ice_sched_node *vsi_node, *tc_node;
+ struct ice_sched_vsi_info *vsi;
+ enum ice_status status = 0;
+ struct ice_hw *hw = pi->hw;
+
+ tc_node = ice_sched_get_tc_node(pi, tc);
+ if (!tc_node)
+ return ICE_ERR_PARAM;
+
+ vsi = ice_sched_get_vsi_info_entry(pi, vsi_id);
+ if (!vsi)
+ vsi = ice_sched_create_vsi_info_entry(pi, vsi_id);
+ if (!vsi)
+ return ICE_ERR_NO_MEMORY;
+
+ vsi_node = ice_sched_get_vsi_node(hw, tc_node, vsi_id);
+
+ /* suspend the VSI if tc is not enabled */
+ if (!enable) {
+ if (vsi_node && vsi_node->in_use) {
+ u32 teid = le32_to_cpu(vsi_node->info.node_teid);
+
+ status = ice_sched_suspend_resume_elems(hw, 1, &teid,
+ true);
+ if (!status)
+ vsi_node->in_use = false;
+ }
+ return status;
+ }
+
+ /* TC is enabled, if it is a new VSI then add it to the tree */
+ if (!vsi_node) {
+ status = ice_sched_add_vsi_to_topo(pi, vsi_id, tc);
+ if (status)
+ return status;
+ vsi_node = ice_sched_get_vsi_node(hw, tc_node, vsi_id);
+ if (!vsi_node)
+ return ICE_ERR_CFG;
+ vsi->vsi_node[tc] = vsi_node;
+ vsi_node->in_use = true;
+ }
+
+ /* update the VSI child nodes */
+ status = ice_sched_update_vsi_child_nodes(pi, vsi_id, tc, maxqs, owner);
+ if (status)
+ return status;
+
+ /* TC is enabled, resume the VSI if it is in the suspend state */
+ if (!vsi_node->in_use) {
+ u32 teid = le32_to_cpu(vsi_node->info.node_teid);
+
+ status = ice_sched_suspend_resume_elems(hw, 1, &teid, false);
+ if (!status)
+ vsi_node->in_use = true;
+ }
+
+ return status;
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_SCHED_H_
+#define _ICE_SCHED_H_
+
+#include "ice_common.h"
+
+#define ICE_QGRP_LAYER_OFFSET 2
+#define ICE_VSI_LAYER_OFFSET 4
+
+struct ice_sched_agg_vsi_info {
+ struct list_head list_entry;
+ DECLARE_BITMAP(tc_bitmap, ICE_MAX_TRAFFIC_CLASS);
+ u16 vsi_id;
+};
+
+struct ice_sched_agg_info {
+ struct list_head agg_vsi_list;
+ struct list_head list_entry;
+ DECLARE_BITMAP(tc_bitmap, ICE_MAX_TRAFFIC_CLASS);
+ u32 agg_id;
+ enum ice_agg_type agg_type;
+};
+
+/* FW AQ command calls */
+enum ice_status ice_sched_init_port(struct ice_port_info *pi);
+enum ice_status ice_sched_query_res_alloc(struct ice_hw *hw);
+void ice_sched_cleanup_all(struct ice_hw *hw);
+struct ice_sched_node *
+ice_sched_find_node_by_teid(struct ice_sched_node *start_node, u32 teid);
+enum ice_status
+ice_sched_add_node(struct ice_port_info *pi, u8 layer,
+ struct ice_aqc_txsched_elem_data *info);
+void ice_free_sched_node(struct ice_port_info *pi, struct ice_sched_node *node);
+struct ice_sched_node *ice_sched_get_tc_node(struct ice_port_info *pi, u8 tc);
+struct ice_sched_node *
+ice_sched_get_free_qparent(struct ice_port_info *pi, u16 vsi_id, u8 tc,
+ u8 owner);
+enum ice_status
+ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_id, u8 tc, u16 maxqs,
+ u8 owner, bool enable);
+#endif /* _ICE_SCHED_H_ */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_STATUS_H_
+#define _ICE_STATUS_H_
+
+/* Error Codes */
+enum ice_status {
+ ICE_ERR_PARAM = -1,
+ ICE_ERR_NOT_IMPL = -2,
+ ICE_ERR_NOT_READY = -3,
+ ICE_ERR_BAD_PTR = -5,
+ ICE_ERR_INVAL_SIZE = -6,
+ ICE_ERR_DEVICE_NOT_SUPPORTED = -8,
+ ICE_ERR_RESET_FAILED = -9,
+ ICE_ERR_FW_API_VER = -10,
+ ICE_ERR_NO_MEMORY = -11,
+ ICE_ERR_CFG = -12,
+ ICE_ERR_OUT_OF_RANGE = -13,
+ ICE_ERR_ALREADY_EXISTS = -14,
+ ICE_ERR_DOES_NOT_EXIST = -15,
+ ICE_ERR_MAX_LIMIT = -17,
+ ICE_ERR_BUF_TOO_SHORT = -52,
+ ICE_ERR_NVM_BLANK_MODE = -53,
+ ICE_ERR_AQ_ERROR = -100,
+ ICE_ERR_AQ_TIMEOUT = -101,
+ ICE_ERR_AQ_FULL = -102,
+ ICE_ERR_AQ_NO_WORK = -103,
+ ICE_ERR_AQ_EMPTY = -104,
+};
+
+#endif /* _ICE_STATUS_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+#include "ice_switch.h"
+
+#define ICE_ETH_DA_OFFSET 0
+#define ICE_ETH_ETHTYPE_OFFSET 12
+#define ICE_ETH_VLAN_TCI_OFFSET 14
+#define ICE_MAX_VLAN_ID 0xFFF
+
+/* Dummy ethernet header needed in the ice_aqc_sw_rules_elem
+ * struct to configure any switch filter rules.
+ * {DA (6 bytes), SA(6 bytes),
+ * Ether type (2 bytes for header without VLAN tag) OR
+ * VLAN tag (4 bytes for header with VLAN tag) }
+ *
+ * Word on Hardcoded values
+ * byte 0 = 0x2: to identify it as locally administered DA MAC
+ * byte 6 = 0x2: to identify it as locally administered SA MAC
+ * byte 12 = 0x81 & byte 13 = 0x00:
+ * In case of VLAN filter first two bytes defines ether type (0x8100)
+ * and remaining two bytes are placeholder for programming a given VLAN id
+ * In case of Ether type filter it is treated as header without VLAN tag
+ * and byte 12 and 13 is used to program a given Ether type instead
+ */
+#define DUMMY_ETH_HDR_LEN 16
+static const u8 dummy_eth_header[DUMMY_ETH_HDR_LEN] = { 0x2, 0, 0, 0, 0, 0,
+ 0x2, 0, 0, 0, 0, 0,
+ 0x81, 0, 0, 0};
+
+#define ICE_SW_RULE_RX_TX_ETH_HDR_SIZE \
+ (sizeof(struct ice_aqc_sw_rules_elem) - \
+ sizeof(((struct ice_aqc_sw_rules_elem *)0)->pdata) + \
+ sizeof(struct ice_sw_rule_lkup_rx_tx) + DUMMY_ETH_HDR_LEN - 1)
+#define ICE_SW_RULE_RX_TX_NO_HDR_SIZE \
+ (sizeof(struct ice_aqc_sw_rules_elem) - \
+ sizeof(((struct ice_aqc_sw_rules_elem *)0)->pdata) + \
+ sizeof(struct ice_sw_rule_lkup_rx_tx) - 1)
+#define ICE_SW_RULE_LG_ACT_SIZE(n) \
+ (sizeof(struct ice_aqc_sw_rules_elem) - \
+ sizeof(((struct ice_aqc_sw_rules_elem *)0)->pdata) + \
+ sizeof(struct ice_sw_rule_lg_act) - \
+ sizeof(((struct ice_sw_rule_lg_act *)0)->act) + \
+ ((n) * sizeof(((struct ice_sw_rule_lg_act *)0)->act)))
+#define ICE_SW_RULE_VSI_LIST_SIZE(n) \
+ (sizeof(struct ice_aqc_sw_rules_elem) - \
+ sizeof(((struct ice_aqc_sw_rules_elem *)0)->pdata) + \
+ sizeof(struct ice_sw_rule_vsi_list) - \
+ sizeof(((struct ice_sw_rule_vsi_list *)0)->vsi) + \
+ ((n) * sizeof(((struct ice_sw_rule_vsi_list *)0)->vsi)))
+
+/**
+ * ice_aq_alloc_free_res - command to allocate/free resources
+ * @hw: pointer to the hw struct
+ * @num_entries: number of resource entries in buffer
+ * @buf: Indirect buffer to hold data parameters and response
+ * @buf_size: size of buffer for indirect commands
+ * @opc: pass in the command opcode
+ * @cd: pointer to command details structure or NULL
+ *
+ * Helper function to allocate/free resources using the admin queue commands
+ */
+static enum ice_status
+ice_aq_alloc_free_res(struct ice_hw *hw, u16 num_entries,
+ struct ice_aqc_alloc_free_res_elem *buf, u16 buf_size,
+ enum ice_adminq_opc opc, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_alloc_free_res_cmd *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.sw_res_ctrl;
+
+ if (!buf)
+ return ICE_ERR_PARAM;
+
+ if (buf_size < (num_entries * sizeof(buf->elem[0])))
+ return ICE_ERR_PARAM;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, opc);
+
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+
+ cmd->num_entries = cpu_to_le16(num_entries);
+
+ return ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+}
+
+/**
+ * ice_aq_get_sw_cfg - get switch configuration
+ * @hw: pointer to the hardware structure
+ * @buf: pointer to the result buffer
+ * @buf_size: length of the buffer available for response
+ * @req_desc: pointer to requested descriptor
+ * @num_elems: pointer to number of elements
+ * @cd: pointer to command details structure or NULL
+ *
+ * Get switch configuration (0x0200) to be placed in 'buff'.
+ * This admin command returns information such as initial VSI/port number
+ * and switch ID it belongs to.
+ *
+ * NOTE: *req_desc is both an input/output parameter.
+ * The caller of this function first calls this function with *request_desc set
+ * to 0. If the response from f/w has *req_desc set to 0, all the switch
+ * configuration information has been returned; if non-zero (meaning not all
+ * the information was returned), the caller should call this function again
+ * with *req_desc set to the previous value returned by f/w to get the
+ * next block of switch configuration information.
+ *
+ * *num_elems is output only parameter. This reflects the number of elements
+ * in response buffer. The caller of this function to use *num_elems while
+ * parsing the response buffer.
+ */
+static enum ice_status
+ice_aq_get_sw_cfg(struct ice_hw *hw, struct ice_aqc_get_sw_cfg_resp *buf,
+ u16 buf_size, u16 *req_desc, u16 *num_elems,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_get_sw_cfg *cmd;
+ enum ice_status status;
+ struct ice_aq_desc desc;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_get_sw_cfg);
+ cmd = &desc.params.get_sw_conf;
+ cmd->element = cpu_to_le16(*req_desc);
+
+ status = ice_aq_send_cmd(hw, &desc, buf, buf_size, cd);
+ if (!status) {
+ *req_desc = le16_to_cpu(cmd->element);
+ *num_elems = le16_to_cpu(cmd->num_elems);
+ }
+
+ return status;
+}
+
+/**
+ * ice_aq_add_vsi
+ * @hw: pointer to the hw struct
+ * @vsi_ctx: pointer to a VSI context struct
+ * @cd: pointer to command details structure or NULL
+ *
+ * Add a VSI context to the hardware (0x0210)
+ */
+enum ice_status
+ice_aq_add_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_add_update_free_vsi_resp *res;
+ struct ice_aqc_add_get_update_free_vsi *cmd;
+ enum ice_status status;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.vsi_cmd;
+ res = (struct ice_aqc_add_update_free_vsi_resp *)&desc.params.raw;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_add_vsi);
+
+ if (!vsi_ctx->alloc_from_pool)
+ cmd->vsi_num = cpu_to_le16(vsi_ctx->vsi_num |
+ ICE_AQ_VSI_IS_VALID);
+
+ cmd->vsi_flags = cpu_to_le16(vsi_ctx->flags);
+
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+
+ status = ice_aq_send_cmd(hw, &desc, &vsi_ctx->info,
+ sizeof(vsi_ctx->info), cd);
+
+ if (!status) {
+ vsi_ctx->vsi_num = le16_to_cpu(res->vsi_num) & ICE_AQ_VSI_NUM_M;
+ vsi_ctx->vsis_allocd = le16_to_cpu(res->vsi_used);
+ vsi_ctx->vsis_unallocated = le16_to_cpu(res->vsi_free);
+ }
+
+ return status;
+}
+
+/**
+ * ice_aq_update_vsi
+ * @hw: pointer to the hw struct
+ * @vsi_ctx: pointer to a VSI context struct
+ * @cd: pointer to command details structure or NULL
+ *
+ * Update VSI context in the hardware (0x0211)
+ */
+enum ice_status
+ice_aq_update_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
+ struct ice_sq_cd *cd)
+{
+ struct ice_aqc_add_update_free_vsi_resp *resp;
+ struct ice_aqc_add_get_update_free_vsi *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.vsi_cmd;
+ resp = (struct ice_aqc_add_update_free_vsi_resp *)&desc.params.raw;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_update_vsi);
+
+ cmd->vsi_num = cpu_to_le16(vsi_ctx->vsi_num | ICE_AQ_VSI_IS_VALID);
+
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+
+ status = ice_aq_send_cmd(hw, &desc, &vsi_ctx->info,
+ sizeof(vsi_ctx->info), cd);
+
+ if (!status) {
+ vsi_ctx->vsis_allocd = le16_to_cpu(resp->vsi_used);
+ vsi_ctx->vsis_unallocated = le16_to_cpu(resp->vsi_free);
+ }
+
+ return status;
+}
+
+/**
+ * ice_aq_free_vsi
+ * @hw: pointer to the hw struct
+ * @vsi_ctx: pointer to a VSI context struct
+ * @keep_vsi_alloc: keep VSI allocation as part of this PF's resources
+ * @cd: pointer to command details structure or NULL
+ *
+ * Get VSI context info from hardware (0x0213)
+ */
+enum ice_status
+ice_aq_free_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
+ bool keep_vsi_alloc, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_add_update_free_vsi_resp *resp;
+ struct ice_aqc_add_get_update_free_vsi *cmd;
+ struct ice_aq_desc desc;
+ enum ice_status status;
+
+ cmd = &desc.params.vsi_cmd;
+ resp = (struct ice_aqc_add_update_free_vsi_resp *)&desc.params.raw;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_free_vsi);
+
+ cmd->vsi_num = cpu_to_le16(vsi_ctx->vsi_num | ICE_AQ_VSI_IS_VALID);
+ if (keep_vsi_alloc)
+ cmd->cmd_flags = cpu_to_le16(ICE_AQ_VSI_KEEP_ALLOC);
+
+ status = ice_aq_send_cmd(hw, &desc, NULL, 0, cd);
+ if (!status) {
+ vsi_ctx->vsis_allocd = le16_to_cpu(resp->vsi_used);
+ vsi_ctx->vsis_unallocated = le16_to_cpu(resp->vsi_free);
+ }
+
+ return status;
+}
+
+/**
+ * ice_aq_alloc_free_vsi_list
+ * @hw: pointer to the hw struct
+ * @vsi_list_id: VSI list id returned or used for lookup
+ * @lkup_type: switch rule filter lookup type
+ * @opc: switch rules population command type - pass in the command opcode
+ *
+ * allocates or free a VSI list resource
+ */
+static enum ice_status
+ice_aq_alloc_free_vsi_list(struct ice_hw *hw, u16 *vsi_list_id,
+ enum ice_sw_lkup_type lkup_type,
+ enum ice_adminq_opc opc)
+{
+ struct ice_aqc_alloc_free_res_elem *sw_buf;
+ struct ice_aqc_res_elem *vsi_ele;
+ enum ice_status status;
+ u16 buf_len;
+
+ buf_len = sizeof(*sw_buf);
+ sw_buf = devm_kzalloc(ice_hw_to_dev(hw), buf_len, GFP_KERNEL);
+ if (!sw_buf)
+ return ICE_ERR_NO_MEMORY;
+ sw_buf->num_elems = cpu_to_le16(1);
+
+ if (lkup_type == ICE_SW_LKUP_MAC ||
+ lkup_type == ICE_SW_LKUP_MAC_VLAN ||
+ lkup_type == ICE_SW_LKUP_ETHERTYPE ||
+ lkup_type == ICE_SW_LKUP_ETHERTYPE_MAC ||
+ lkup_type == ICE_SW_LKUP_PROMISC ||
+ lkup_type == ICE_SW_LKUP_PROMISC_VLAN) {
+ sw_buf->res_type = cpu_to_le16(ICE_AQC_RES_TYPE_VSI_LIST_REP);
+ } else if (lkup_type == ICE_SW_LKUP_VLAN) {
+ sw_buf->res_type =
+ cpu_to_le16(ICE_AQC_RES_TYPE_VSI_LIST_PRUNE);
+ } else {
+ status = ICE_ERR_PARAM;
+ goto ice_aq_alloc_free_vsi_list_exit;
+ }
+
+ if (opc == ice_aqc_opc_free_res)
+ sw_buf->elem[0].e.sw_resp = cpu_to_le16(*vsi_list_id);
+
+ status = ice_aq_alloc_free_res(hw, 1, sw_buf, buf_len, opc, NULL);
+ if (status)
+ goto ice_aq_alloc_free_vsi_list_exit;
+
+ if (opc == ice_aqc_opc_alloc_res) {
+ vsi_ele = &sw_buf->elem[0];
+ *vsi_list_id = le16_to_cpu(vsi_ele->e.sw_resp);
+ }
+
+ice_aq_alloc_free_vsi_list_exit:
+ devm_kfree(ice_hw_to_dev(hw), sw_buf);
+ return status;
+}
+
+/**
+ * ice_aq_sw_rules - add/update/remove switch rules
+ * @hw: pointer to the hw struct
+ * @rule_list: pointer to switch rule population list
+ * @rule_list_sz: total size of the rule list in bytes
+ * @num_rules: number of switch rules in the rule_list
+ * @opc: switch rules population command type - pass in the command opcode
+ * @cd: pointer to command details structure or NULL
+ *
+ * Add(0x02a0)/Update(0x02a1)/Remove(0x02a2) switch rules commands to firmware
+ */
+static enum ice_status
+ice_aq_sw_rules(struct ice_hw *hw, void *rule_list, u16 rule_list_sz,
+ u8 num_rules, enum ice_adminq_opc opc, struct ice_sq_cd *cd)
+{
+ struct ice_aq_desc desc;
+
+ if (opc != ice_aqc_opc_add_sw_rules &&
+ opc != ice_aqc_opc_update_sw_rules &&
+ opc != ice_aqc_opc_remove_sw_rules)
+ return ICE_ERR_PARAM;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, opc);
+
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+ desc.params.sw_rules.num_rules_fltr_entry_index =
+ cpu_to_le16(num_rules);
+ return ice_aq_send_cmd(hw, &desc, rule_list, rule_list_sz, cd);
+}
+
+/* ice_init_port_info - Initialize port_info with switch configuration data
+ * @pi: pointer to port_info
+ * @vsi_port_num: VSI number or port number
+ * @type: Type of switch element (port or VSI)
+ * @swid: switch ID of the switch the element is attached to
+ * @pf_vf_num: PF or VF number
+ * @is_vf: true if the element is a VF, false otherwise
+ */
+static void
+ice_init_port_info(struct ice_port_info *pi, u16 vsi_port_num, u8 type,
+ u16 swid, u16 pf_vf_num, bool is_vf)
+{
+ switch (type) {
+ case ICE_AQC_GET_SW_CONF_RESP_PHYS_PORT:
+ pi->lport = (u8)(vsi_port_num & ICE_LPORT_MASK);
+ pi->sw_id = swid;
+ pi->pf_vf_num = pf_vf_num;
+ pi->is_vf = is_vf;
+ pi->dflt_tx_vsi_num = ICE_DFLT_VSI_INVAL;
+ pi->dflt_rx_vsi_num = ICE_DFLT_VSI_INVAL;
+ break;
+ default:
+ ice_debug(pi->hw, ICE_DBG_SW,
+ "incorrect VSI/port type received\n");
+ break;
+ }
+}
+
+/* ice_get_initial_sw_cfg - Get initial port and default VSI data
+ * @hw: pointer to the hardware structure
+ */
+enum ice_status ice_get_initial_sw_cfg(struct ice_hw *hw)
+{
+ struct ice_aqc_get_sw_cfg_resp *rbuf;
+ enum ice_status status;
+ u16 req_desc = 0;
+ u16 num_elems;
+ u16 i;
+
+ rbuf = devm_kzalloc(ice_hw_to_dev(hw), ICE_SW_CFG_MAX_BUF_LEN,
+ GFP_KERNEL);
+
+ if (!rbuf)
+ return ICE_ERR_NO_MEMORY;
+
+ /* Multiple calls to ice_aq_get_sw_cfg may be required
+ * to get all the switch configuration information. The need
+ * for additional calls is indicated by ice_aq_get_sw_cfg
+ * writing a non-zero value in req_desc
+ */
+ do {
+ status = ice_aq_get_sw_cfg(hw, rbuf, ICE_SW_CFG_MAX_BUF_LEN,
+ &req_desc, &num_elems, NULL);
+
+ if (status)
+ break;
+
+ for (i = 0; i < num_elems; i++) {
+ struct ice_aqc_get_sw_cfg_resp_elem *ele;
+ u16 pf_vf_num, swid, vsi_port_num;
+ bool is_vf = false;
+ u8 type;
+
+ ele = rbuf[i].elements;
+ vsi_port_num = le16_to_cpu(ele->vsi_port_num) &
+ ICE_AQC_GET_SW_CONF_RESP_VSI_PORT_NUM_M;
+
+ pf_vf_num = le16_to_cpu(ele->pf_vf_num) &
+ ICE_AQC_GET_SW_CONF_RESP_FUNC_NUM_M;
+
+ swid = le16_to_cpu(ele->swid);
+
+ if (le16_to_cpu(ele->pf_vf_num) &
+ ICE_AQC_GET_SW_CONF_RESP_IS_VF)
+ is_vf = true;
+
+ type = le16_to_cpu(ele->vsi_port_num) >>
+ ICE_AQC_GET_SW_CONF_RESP_TYPE_S;
+
+ if (type == ICE_AQC_GET_SW_CONF_RESP_VSI) {
+ /* FW VSI is not needed. Just continue. */
+ continue;
+ }
+
+ ice_init_port_info(hw->port_info, vsi_port_num,
+ type, swid, pf_vf_num, is_vf);
+ }
+ } while (req_desc && !status);
+
+ devm_kfree(ice_hw_to_dev(hw), (void *)rbuf);
+ return status;
+}
+
+/**
+ * ice_fill_sw_info - Helper function to populate lb_en and lan_en
+ * @hw: pointer to the hardware structure
+ * @f_info: filter info structure to fill/update
+ *
+ * This helper function populates the lb_en and lan_en elements of the provided
+ * ice_fltr_info struct using the switch's type and characteristics of the
+ * switch rule being configured.
+ */
+static void ice_fill_sw_info(struct ice_hw *hw, struct ice_fltr_info *f_info)
+{
+ f_info->lb_en = false;
+ f_info->lan_en = false;
+ if ((f_info->flag & ICE_FLTR_TX) &&
+ (f_info->fltr_act == ICE_FWD_TO_VSI ||
+ f_info->fltr_act == ICE_FWD_TO_VSI_LIST ||
+ f_info->fltr_act == ICE_FWD_TO_Q ||
+ f_info->fltr_act == ICE_FWD_TO_QGRP)) {
+ f_info->lb_en = true;
+ if (!(hw->evb_veb && f_info->lkup_type == ICE_SW_LKUP_MAC &&
+ is_unicast_ether_addr(f_info->l_data.mac.mac_addr)))
+ f_info->lan_en = true;
+ }
+}
+
+/**
+ * ice_fill_sw_rule - Helper function to fill switch rule structure
+ * @hw: pointer to the hardware structure
+ * @f_info: entry containing packet forwarding information
+ * @s_rule: switch rule structure to be filled in based on mac_entry
+ * @opc: switch rules population command type - pass in the command opcode
+ */
+static void
+ice_fill_sw_rule(struct ice_hw *hw, struct ice_fltr_info *f_info,
+ struct ice_aqc_sw_rules_elem *s_rule, enum ice_adminq_opc opc)
+{
+ u16 vlan_id = ICE_MAX_VLAN_ID + 1;
+ u8 eth_hdr[DUMMY_ETH_HDR_LEN];
+ void *daddr = NULL;
+ u32 act = 0;
+ __be16 *off;
+
+ if (opc == ice_aqc_opc_remove_sw_rules) {
+ s_rule->pdata.lkup_tx_rx.act = 0;
+ s_rule->pdata.lkup_tx_rx.index =
+ cpu_to_le16(f_info->fltr_rule_id);
+ s_rule->pdata.lkup_tx_rx.hdr_len = 0;
+ return;
+ }
+
+ /* initialize the ether header with a dummy header */
+ memcpy(eth_hdr, dummy_eth_header, sizeof(dummy_eth_header));
+ ice_fill_sw_info(hw, f_info);
+
+ switch (f_info->fltr_act) {
+ case ICE_FWD_TO_VSI:
+ act |= (f_info->fwd_id.vsi_id << ICE_SINGLE_ACT_VSI_ID_S) &
+ ICE_SINGLE_ACT_VSI_ID_M;
+ if (f_info->lkup_type != ICE_SW_LKUP_VLAN)
+ act |= ICE_SINGLE_ACT_VSI_FORWARDING |
+ ICE_SINGLE_ACT_VALID_BIT;
+ break;
+ case ICE_FWD_TO_VSI_LIST:
+ act |= ICE_SINGLE_ACT_VSI_LIST;
+ act |= (f_info->fwd_id.vsi_list_id <<
+ ICE_SINGLE_ACT_VSI_LIST_ID_S) &
+ ICE_SINGLE_ACT_VSI_LIST_ID_M;
+ if (f_info->lkup_type != ICE_SW_LKUP_VLAN)
+ act |= ICE_SINGLE_ACT_VSI_FORWARDING |
+ ICE_SINGLE_ACT_VALID_BIT;
+ break;
+ case ICE_FWD_TO_Q:
+ act |= ICE_SINGLE_ACT_TO_Q;
+ act |= (f_info->fwd_id.q_id << ICE_SINGLE_ACT_Q_INDEX_S) &
+ ICE_SINGLE_ACT_Q_INDEX_M;
+ break;
+ case ICE_FWD_TO_QGRP:
+ act |= ICE_SINGLE_ACT_TO_Q;
+ act |= (f_info->qgrp_size << ICE_SINGLE_ACT_Q_REGION_S) &
+ ICE_SINGLE_ACT_Q_REGION_M;
+ break;
+ case ICE_DROP_PACKET:
+ act |= ICE_SINGLE_ACT_VSI_FORWARDING | ICE_SINGLE_ACT_DROP;
+ break;
+ default:
+ return;
+ }
+
+ if (f_info->lb_en)
+ act |= ICE_SINGLE_ACT_LB_ENABLE;
+ if (f_info->lan_en)
+ act |= ICE_SINGLE_ACT_LAN_ENABLE;
+
+ switch (f_info->lkup_type) {
+ case ICE_SW_LKUP_MAC:
+ daddr = f_info->l_data.mac.mac_addr;
+ break;
+ case ICE_SW_LKUP_VLAN:
+ vlan_id = f_info->l_data.vlan.vlan_id;
+ if (f_info->fltr_act == ICE_FWD_TO_VSI ||
+ f_info->fltr_act == ICE_FWD_TO_VSI_LIST) {
+ act |= ICE_SINGLE_ACT_PRUNE;
+ act |= ICE_SINGLE_ACT_EGRESS | ICE_SINGLE_ACT_INGRESS;
+ }
+ break;
+ case ICE_SW_LKUP_ETHERTYPE_MAC:
+ daddr = f_info->l_data.ethertype_mac.mac_addr;
+ /* fall-through */
+ case ICE_SW_LKUP_ETHERTYPE:
+ off = (__be16 *)ð_hdr[ICE_ETH_ETHTYPE_OFFSET];
+ *off = cpu_to_be16(f_info->l_data.ethertype_mac.ethertype);
+ break;
+ case ICE_SW_LKUP_MAC_VLAN:
+ daddr = f_info->l_data.mac_vlan.mac_addr;
+ vlan_id = f_info->l_data.mac_vlan.vlan_id;
+ break;
+ case ICE_SW_LKUP_PROMISC_VLAN:
+ vlan_id = f_info->l_data.mac_vlan.vlan_id;
+ /* fall-through */
+ case ICE_SW_LKUP_PROMISC:
+ daddr = f_info->l_data.mac_vlan.mac_addr;
+ break;
+ default:
+ break;
+ }
+
+ s_rule->type = (f_info->flag & ICE_FLTR_RX) ?
+ cpu_to_le16(ICE_AQC_SW_RULES_T_LKUP_RX) :
+ cpu_to_le16(ICE_AQC_SW_RULES_T_LKUP_TX);
+
+ /* Recipe set depending on lookup type */
+ s_rule->pdata.lkup_tx_rx.recipe_id = cpu_to_le16(f_info->lkup_type);
+ s_rule->pdata.lkup_tx_rx.src = cpu_to_le16(f_info->src);
+ s_rule->pdata.lkup_tx_rx.act = cpu_to_le32(act);
+
+ if (daddr)
+ ether_addr_copy(ð_hdr[ICE_ETH_DA_OFFSET], daddr);
+
+ if (!(vlan_id > ICE_MAX_VLAN_ID)) {
+ off = (__be16 *)ð_hdr[ICE_ETH_VLAN_TCI_OFFSET];
+ *off = cpu_to_be16(vlan_id);
+ }
+
+ /* Create the switch rule with the final dummy Ethernet header */
+ if (opc != ice_aqc_opc_update_sw_rules)
+ s_rule->pdata.lkup_tx_rx.hdr_len = cpu_to_le16(sizeof(eth_hdr));
+
+ memcpy(s_rule->pdata.lkup_tx_rx.hdr, eth_hdr, sizeof(eth_hdr));
+}
+
+/**
+ * ice_add_marker_act
+ * @hw: pointer to the hardware structure
+ * @m_ent: the management entry for which sw marker needs to be added
+ * @sw_marker: sw marker to tag the Rx descriptor with
+ * @l_id: large action resource id
+ *
+ * Create a large action to hold software marker and update the switch rule
+ * entry pointed by m_ent with newly created large action
+ */
+static enum ice_status
+ice_add_marker_act(struct ice_hw *hw, struct ice_fltr_mgmt_list_entry *m_ent,
+ u16 sw_marker, u16 l_id)
+{
+ struct ice_aqc_sw_rules_elem *lg_act, *rx_tx;
+ /* For software marker we need 3 large actions
+ * 1. FWD action: FWD TO VSI or VSI LIST
+ * 2. GENERIC VALUE action to hold the profile id
+ * 3. GENERIC VALUE action to hold the software marker id
+ */
+ const u16 num_lg_acts = 3;
+ enum ice_status status;
+ u16 lg_act_size;
+ u16 rules_size;
+ u16 vsi_info;
+ u32 act;
+
+ if (m_ent->fltr_info.lkup_type != ICE_SW_LKUP_MAC)
+ return ICE_ERR_PARAM;
+
+ /* Create two back-to-back switch rules and submit them to the HW using
+ * one memory buffer:
+ * 1. Large Action
+ * 2. Look up tx rx
+ */
+ lg_act_size = (u16)ICE_SW_RULE_LG_ACT_SIZE(num_lg_acts);
+ rules_size = lg_act_size + ICE_SW_RULE_RX_TX_ETH_HDR_SIZE;
+ lg_act = devm_kzalloc(ice_hw_to_dev(hw), rules_size, GFP_KERNEL);
+ if (!lg_act)
+ return ICE_ERR_NO_MEMORY;
+
+ rx_tx = (struct ice_aqc_sw_rules_elem *)((u8 *)lg_act + lg_act_size);
+
+ /* Fill in the first switch rule i.e. large action */
+ lg_act->type = cpu_to_le16(ICE_AQC_SW_RULES_T_LG_ACT);
+ lg_act->pdata.lg_act.index = cpu_to_le16(l_id);
+ lg_act->pdata.lg_act.size = cpu_to_le16(num_lg_acts);
+
+ /* First action VSI forwarding or VSI list forwarding depending on how
+ * many VSIs
+ */
+ vsi_info = (m_ent->vsi_count > 1) ?
+ m_ent->fltr_info.fwd_id.vsi_list_id :
+ m_ent->fltr_info.fwd_id.vsi_id;
+
+ act = ICE_LG_ACT_VSI_FORWARDING | ICE_LG_ACT_VALID_BIT;
+ act |= (vsi_info << ICE_LG_ACT_VSI_LIST_ID_S) &
+ ICE_LG_ACT_VSI_LIST_ID_M;
+ if (m_ent->vsi_count > 1)
+ act |= ICE_LG_ACT_VSI_LIST;
+ lg_act->pdata.lg_act.act[0] = cpu_to_le32(act);
+
+ /* Second action descriptor type */
+ act = ICE_LG_ACT_GENERIC;
+
+ act |= (1 << ICE_LG_ACT_GENERIC_VALUE_S) & ICE_LG_ACT_GENERIC_VALUE_M;
+ lg_act->pdata.lg_act.act[1] = cpu_to_le32(act);
+
+ act = (7 << ICE_LG_ACT_GENERIC_OFFSET_S) & ICE_LG_ACT_GENERIC_VALUE_M;
+
+ /* Third action Marker value */
+ act |= ICE_LG_ACT_GENERIC;
+ act |= (sw_marker << ICE_LG_ACT_GENERIC_VALUE_S) &
+ ICE_LG_ACT_GENERIC_VALUE_M;
+
+ act |= (0 << ICE_LG_ACT_GENERIC_OFFSET_S) & ICE_LG_ACT_GENERIC_VALUE_M;
+ lg_act->pdata.lg_act.act[2] = cpu_to_le32(act);
+
+ /* call the fill switch rule to fill the lookup tx rx structure */
+ ice_fill_sw_rule(hw, &m_ent->fltr_info, rx_tx,
+ ice_aqc_opc_update_sw_rules);
+
+ /* Update the action to point to the large action id */
+ rx_tx->pdata.lkup_tx_rx.act =
+ cpu_to_le32(ICE_SINGLE_ACT_PTR |
+ ((l_id << ICE_SINGLE_ACT_PTR_VAL_S) &
+ ICE_SINGLE_ACT_PTR_VAL_M));
+
+ /* Use the filter rule id of the previously created rule with single
+ * act. Once the update happens, hardware will treat this as large
+ * action
+ */
+ rx_tx->pdata.lkup_tx_rx.index =
+ cpu_to_le16(m_ent->fltr_info.fltr_rule_id);
+
+ status = ice_aq_sw_rules(hw, lg_act, rules_size, 2,
+ ice_aqc_opc_update_sw_rules, NULL);
+ if (!status) {
+ m_ent->lg_act_idx = l_id;
+ m_ent->sw_marker_id = sw_marker;
+ }
+
+ devm_kfree(ice_hw_to_dev(hw), lg_act);
+ return status;
+}
+
+/**
+ * ice_create_vsi_list_map
+ * @hw: pointer to the hardware structure
+ * @vsi_array: array of VSIs to form a VSI list
+ * @num_vsi: num VSI in the array
+ * @vsi_list_id: VSI list id generated as part of allocate resource
+ *
+ * Helper function to create a new entry of VSI list id to VSI mapping
+ * using the given VSI list id
+ */
+static struct ice_vsi_list_map_info *
+ice_create_vsi_list_map(struct ice_hw *hw, u16 *vsi_array, u16 num_vsi,
+ u16 vsi_list_id)
+{
+ struct ice_switch_info *sw = hw->switch_info;
+ struct ice_vsi_list_map_info *v_map;
+ int i;
+
+ v_map = devm_kcalloc(ice_hw_to_dev(hw), 1, sizeof(*v_map), GFP_KERNEL);
+ if (!v_map)
+ return NULL;
+
+ v_map->vsi_list_id = vsi_list_id;
+
+ for (i = 0; i < num_vsi; i++)
+ set_bit(vsi_array[i], v_map->vsi_map);
+
+ list_add(&v_map->list_entry, &sw->vsi_list_map_head);
+ return v_map;
+}
+
+/**
+ * ice_update_vsi_list_rule
+ * @hw: pointer to the hardware structure
+ * @vsi_array: array of VSIs to form a VSI list
+ * @num_vsi: num VSI in the array
+ * @vsi_list_id: VSI list id generated as part of allocate resource
+ * @remove: Boolean value to indicate if this is a remove action
+ * @opc: switch rules population command type - pass in the command opcode
+ * @lkup_type: lookup type of the filter
+ *
+ * Call AQ command to add a new switch rule or update existing switch rule
+ * using the given VSI list id
+ */
+static enum ice_status
+ice_update_vsi_list_rule(struct ice_hw *hw, u16 *vsi_array, u16 num_vsi,
+ u16 vsi_list_id, bool remove, enum ice_adminq_opc opc,
+ enum ice_sw_lkup_type lkup_type)
+{
+ struct ice_aqc_sw_rules_elem *s_rule;
+ enum ice_status status;
+ u16 s_rule_size;
+ u16 type;
+ int i;
+
+ if (!num_vsi)
+ return ICE_ERR_PARAM;
+
+ if (lkup_type == ICE_SW_LKUP_MAC ||
+ lkup_type == ICE_SW_LKUP_MAC_VLAN ||
+ lkup_type == ICE_SW_LKUP_ETHERTYPE ||
+ lkup_type == ICE_SW_LKUP_ETHERTYPE_MAC ||
+ lkup_type == ICE_SW_LKUP_PROMISC ||
+ lkup_type == ICE_SW_LKUP_PROMISC_VLAN)
+ type = remove ? ICE_AQC_SW_RULES_T_VSI_LIST_CLEAR :
+ ICE_AQC_SW_RULES_T_VSI_LIST_SET;
+ else if (lkup_type == ICE_SW_LKUP_VLAN)
+ type = remove ? ICE_AQC_SW_RULES_T_PRUNE_LIST_CLEAR :
+ ICE_AQC_SW_RULES_T_PRUNE_LIST_SET;
+ else
+ return ICE_ERR_PARAM;
+
+ s_rule_size = (u16)ICE_SW_RULE_VSI_LIST_SIZE(num_vsi);
+ s_rule = devm_kzalloc(ice_hw_to_dev(hw), s_rule_size, GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ for (i = 0; i < num_vsi; i++)
+ s_rule->pdata.vsi_list.vsi[i] = cpu_to_le16(vsi_array[i]);
+
+ s_rule->type = cpu_to_le16(type);
+ s_rule->pdata.vsi_list.number_vsi = cpu_to_le16(num_vsi);
+ s_rule->pdata.vsi_list.index = cpu_to_le16(vsi_list_id);
+
+ status = ice_aq_sw_rules(hw, s_rule, s_rule_size, 1, opc, NULL);
+
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ return status;
+}
+
+/**
+ * ice_create_vsi_list_rule - Creates and populates a VSI list rule
+ * @hw: pointer to the hw struct
+ * @vsi_array: array of VSIs to form a VSI list
+ * @num_vsi: number of VSIs in the array
+ * @vsi_list_id: stores the ID of the VSI list to be created
+ * @lkup_type: switch rule filter's lookup type
+ */
+static enum ice_status
+ice_create_vsi_list_rule(struct ice_hw *hw, u16 *vsi_array, u16 num_vsi,
+ u16 *vsi_list_id, enum ice_sw_lkup_type lkup_type)
+{
+ enum ice_status status;
+ int i;
+
+ for (i = 0; i < num_vsi; i++)
+ if (vsi_array[i] >= ICE_MAX_VSI)
+ return ICE_ERR_OUT_OF_RANGE;
+
+ status = ice_aq_alloc_free_vsi_list(hw, vsi_list_id, lkup_type,
+ ice_aqc_opc_alloc_res);
+ if (status)
+ return status;
+
+ /* Update the newly created VSI list to include the specified VSIs */
+ return ice_update_vsi_list_rule(hw, vsi_array, num_vsi, *vsi_list_id,
+ false, ice_aqc_opc_add_sw_rules,
+ lkup_type);
+}
+
+/**
+ * ice_create_pkt_fwd_rule
+ * @hw: pointer to the hardware structure
+ * @f_entry: entry containing packet forwarding information
+ *
+ * Create switch rule with given filter information and add an entry
+ * to the corresponding filter management list to track this switch rule
+ * and VSI mapping
+ */
+static enum ice_status
+ice_create_pkt_fwd_rule(struct ice_hw *hw,
+ struct ice_fltr_list_entry *f_entry)
+{
+ struct ice_switch_info *sw = hw->switch_info;
+ struct ice_fltr_mgmt_list_entry *fm_entry;
+ struct ice_aqc_sw_rules_elem *s_rule;
+ enum ice_sw_lkup_type l_type;
+ enum ice_status status;
+
+ s_rule = devm_kzalloc(ice_hw_to_dev(hw),
+ ICE_SW_RULE_RX_TX_ETH_HDR_SIZE, GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+ fm_entry = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*fm_entry),
+ GFP_KERNEL);
+ if (!fm_entry) {
+ status = ICE_ERR_NO_MEMORY;
+ goto ice_create_pkt_fwd_rule_exit;
+ }
+
+ fm_entry->fltr_info = f_entry->fltr_info;
+
+ /* Initialize all the fields for the management entry */
+ fm_entry->vsi_count = 1;
+ fm_entry->lg_act_idx = ICE_INVAL_LG_ACT_INDEX;
+ fm_entry->sw_marker_id = ICE_INVAL_SW_MARKER_ID;
+ fm_entry->counter_index = ICE_INVAL_COUNTER_ID;
+
+ ice_fill_sw_rule(hw, &fm_entry->fltr_info, s_rule,
+ ice_aqc_opc_add_sw_rules);
+
+ status = ice_aq_sw_rules(hw, s_rule, ICE_SW_RULE_RX_TX_ETH_HDR_SIZE, 1,
+ ice_aqc_opc_add_sw_rules, NULL);
+ if (status) {
+ devm_kfree(ice_hw_to_dev(hw), fm_entry);
+ goto ice_create_pkt_fwd_rule_exit;
+ }
+
+ f_entry->fltr_info.fltr_rule_id =
+ le16_to_cpu(s_rule->pdata.lkup_tx_rx.index);
+ fm_entry->fltr_info.fltr_rule_id =
+ le16_to_cpu(s_rule->pdata.lkup_tx_rx.index);
+
+ /* The book keeping entries will get removed when base driver
+ * calls remove filter AQ command
+ */
+ l_type = fm_entry->fltr_info.lkup_type;
+ if (l_type == ICE_SW_LKUP_MAC) {
+ mutex_lock(&sw->mac_list_lock);
+ list_add(&fm_entry->list_entry, &sw->mac_list_head);
+ mutex_unlock(&sw->mac_list_lock);
+ } else if (l_type == ICE_SW_LKUP_VLAN) {
+ mutex_lock(&sw->vlan_list_lock);
+ list_add(&fm_entry->list_entry, &sw->vlan_list_head);
+ mutex_unlock(&sw->vlan_list_lock);
+ } else if (l_type == ICE_SW_LKUP_ETHERTYPE ||
+ l_type == ICE_SW_LKUP_ETHERTYPE_MAC) {
+ mutex_lock(&sw->eth_m_list_lock);
+ list_add(&fm_entry->list_entry, &sw->eth_m_list_head);
+ mutex_unlock(&sw->eth_m_list_lock);
+ } else if (l_type == ICE_SW_LKUP_PROMISC ||
+ l_type == ICE_SW_LKUP_PROMISC_VLAN) {
+ mutex_lock(&sw->promisc_list_lock);
+ list_add(&fm_entry->list_entry, &sw->promisc_list_head);
+ mutex_unlock(&sw->promisc_list_lock);
+ } else if (fm_entry->fltr_info.lkup_type == ICE_SW_LKUP_MAC_VLAN) {
+ mutex_lock(&sw->mac_vlan_list_lock);
+ list_add(&fm_entry->list_entry, &sw->mac_vlan_list_head);
+ mutex_unlock(&sw->mac_vlan_list_lock);
+ } else {
+ status = ICE_ERR_NOT_IMPL;
+ }
+ice_create_pkt_fwd_rule_exit:
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ return status;
+}
+
+/**
+ * ice_update_pkt_fwd_rule
+ * @hw: pointer to the hardware structure
+ * @rule_id: rule of previously created switch rule to update
+ * @vsi_list_id: VSI list id to be updated with
+ * @f_info: ice_fltr_info to pull other information for switch rule
+ *
+ * Call AQ command to update a previously created switch rule with a
+ * VSI list id
+ */
+static enum ice_status
+ice_update_pkt_fwd_rule(struct ice_hw *hw, u16 rule_id, u16 vsi_list_id,
+ struct ice_fltr_info f_info)
+{
+ struct ice_aqc_sw_rules_elem *s_rule;
+ struct ice_fltr_info tmp_fltr;
+ enum ice_status status;
+
+ s_rule = devm_kzalloc(ice_hw_to_dev(hw),
+ ICE_SW_RULE_RX_TX_ETH_HDR_SIZE, GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ tmp_fltr = f_info;
+ tmp_fltr.fltr_act = ICE_FWD_TO_VSI_LIST;
+ tmp_fltr.fwd_id.vsi_list_id = vsi_list_id;
+
+ ice_fill_sw_rule(hw, &tmp_fltr, s_rule,
+ ice_aqc_opc_update_sw_rules);
+
+ s_rule->pdata.lkup_tx_rx.index = cpu_to_le16(rule_id);
+
+ /* Update switch rule with new rule set to forward VSI list */
+ status = ice_aq_sw_rules(hw, s_rule, ICE_SW_RULE_RX_TX_ETH_HDR_SIZE, 1,
+ ice_aqc_opc_update_sw_rules, NULL);
+
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ return status;
+}
+
+/**
+ * ice_handle_vsi_list_mgmt
+ * @hw: pointer to the hardware structure
+ * @m_entry: pointer to current filter management list entry
+ * @cur_fltr: filter information from the book keeping entry
+ * @new_fltr: filter information with the new VSI to be added
+ *
+ * Call AQ command to add or update previously created VSI list with new VSI.
+ *
+ * Helper function to do book keeping associated with adding filter information
+ * The algorithm to do the booking keeping is described below :
+ * When a VSI needs to subscribe to a given filter( MAC/VLAN/Ethtype etc.)
+ * if only one VSI has been added till now
+ * Allocate a new VSI list and add two VSIs
+ * to this list using switch rule command
+ * Update the previously created switch rule with the
+ * newly created VSI list id
+ * if a VSI list was previously created
+ * Add the new VSI to the previously created VSI list set
+ * using the update switch rule command
+ */
+static enum ice_status
+ice_handle_vsi_list_mgmt(struct ice_hw *hw,
+ struct ice_fltr_mgmt_list_entry *m_entry,
+ struct ice_fltr_info *cur_fltr,
+ struct ice_fltr_info *new_fltr)
+{
+ enum ice_status status = 0;
+ u16 vsi_list_id = 0;
+
+ if ((cur_fltr->fltr_act == ICE_FWD_TO_Q ||
+ cur_fltr->fltr_act == ICE_FWD_TO_QGRP))
+ return ICE_ERR_NOT_IMPL;
+
+ if ((new_fltr->fltr_act == ICE_FWD_TO_Q ||
+ new_fltr->fltr_act == ICE_FWD_TO_QGRP) &&
+ (cur_fltr->fltr_act == ICE_FWD_TO_VSI ||
+ cur_fltr->fltr_act == ICE_FWD_TO_VSI_LIST))
+ return ICE_ERR_NOT_IMPL;
+
+ if (m_entry->vsi_count < 2 && !m_entry->vsi_list_info) {
+ /* Only one entry existed in the mapping and it was not already
+ * a part of a VSI list. So, create a VSI list with the old and
+ * new VSIs.
+ */
+ u16 vsi_id_arr[2];
+ u16 fltr_rule;
+
+ /* A rule already exists with the new VSI being added */
+ if (cur_fltr->fwd_id.vsi_id == new_fltr->fwd_id.vsi_id)
+ return ICE_ERR_ALREADY_EXISTS;
+
+ vsi_id_arr[0] = cur_fltr->fwd_id.vsi_id;
+ vsi_id_arr[1] = new_fltr->fwd_id.vsi_id;
+ status = ice_create_vsi_list_rule(hw, &vsi_id_arr[0], 2,
+ &vsi_list_id,
+ new_fltr->lkup_type);
+ if (status)
+ return status;
+
+ fltr_rule = cur_fltr->fltr_rule_id;
+ /* Update the previous switch rule of "MAC forward to VSI" to
+ * "MAC fwd to VSI list"
+ */
+ status = ice_update_pkt_fwd_rule(hw, fltr_rule, vsi_list_id,
+ *new_fltr);
+ if (status)
+ return status;
+
+ cur_fltr->fwd_id.vsi_list_id = vsi_list_id;
+ cur_fltr->fltr_act = ICE_FWD_TO_VSI_LIST;
+ m_entry->vsi_list_info =
+ ice_create_vsi_list_map(hw, &vsi_id_arr[0], 2,
+ vsi_list_id);
+
+ /* If this entry was large action then the large action needs
+ * to be updated to point to FWD to VSI list
+ */
+ if (m_entry->sw_marker_id != ICE_INVAL_SW_MARKER_ID)
+ status =
+ ice_add_marker_act(hw, m_entry,
+ m_entry->sw_marker_id,
+ m_entry->lg_act_idx);
+ } else {
+ u16 vsi_id = new_fltr->fwd_id.vsi_id;
+ enum ice_adminq_opc opcode;
+
+ /* A rule already exists with the new VSI being added */
+ if (test_bit(vsi_id, m_entry->vsi_list_info->vsi_map))
+ return 0;
+
+ /* Update the previously created VSI list set with
+ * the new VSI id passed in
+ */
+ vsi_list_id = cur_fltr->fwd_id.vsi_list_id;
+ opcode = ice_aqc_opc_update_sw_rules;
+
+ status = ice_update_vsi_list_rule(hw, &vsi_id, 1, vsi_list_id,
+ false, opcode,
+ new_fltr->lkup_type);
+ /* update VSI list mapping info with new VSI id */
+ if (!status)
+ set_bit(vsi_id, m_entry->vsi_list_info->vsi_map);
+ }
+ if (!status)
+ m_entry->vsi_count++;
+ return status;
+}
+
+/**
+ * ice_find_mac_entry
+ * @hw: pointer to the hardware structure
+ * @mac_addr: MAC address to search for
+ *
+ * Helper function to search for a MAC entry using a given MAC address
+ * Returns pointer to the entry if found.
+ */
+static struct ice_fltr_mgmt_list_entry *
+ice_find_mac_entry(struct ice_hw *hw, u8 *mac_addr)
+{
+ struct ice_fltr_mgmt_list_entry *m_list_itr, *mac_ret = NULL;
+ struct ice_switch_info *sw = hw->switch_info;
+
+ mutex_lock(&sw->mac_list_lock);
+ list_for_each_entry(m_list_itr, &sw->mac_list_head, list_entry) {
+ u8 *buf = &m_list_itr->fltr_info.l_data.mac.mac_addr[0];
+
+ if (ether_addr_equal(buf, mac_addr)) {
+ mac_ret = m_list_itr;
+ break;
+ }
+ }
+ mutex_unlock(&sw->mac_list_lock);
+ return mac_ret;
+}
+
+/**
+ * ice_add_shared_mac - Add one MAC shared filter rule
+ * @hw: pointer to the hardware structure
+ * @f_entry: structure containing MAC forwarding information
+ *
+ * Adds or updates the book keeping list for the MAC addresses
+ */
+static enum ice_status
+ice_add_shared_mac(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
+{
+ struct ice_fltr_info *new_fltr, *cur_fltr;
+ struct ice_fltr_mgmt_list_entry *m_entry;
+
+ new_fltr = &f_entry->fltr_info;
+
+ m_entry = ice_find_mac_entry(hw, &new_fltr->l_data.mac.mac_addr[0]);
+ if (!m_entry)
+ return ice_create_pkt_fwd_rule(hw, f_entry);
+
+ cur_fltr = &m_entry->fltr_info;
+
+ return ice_handle_vsi_list_mgmt(hw, m_entry, cur_fltr, new_fltr);
+}
+
+/**
+ * ice_add_mac - Add a MAC address based filter rule
+ * @hw: pointer to the hardware structure
+ * @m_list: list of MAC addresses and forwarding information
+ *
+ * IMPORTANT: When the ucast_shared flag is set to false and m_list has
+ * multiple unicast addresses, the function assumes that all the
+ * addresses are unique in a given add_mac call. It doesn't
+ * check for duplicates in this case, removing duplicates from a given
+ * list should be taken care of in the caller of this function.
+ */
+enum ice_status
+ice_add_mac(struct ice_hw *hw, struct list_head *m_list)
+{
+ struct ice_aqc_sw_rules_elem *s_rule, *r_iter;
+ struct ice_fltr_list_entry *m_list_itr;
+ u16 elem_sent, total_elem_left;
+ enum ice_status status = 0;
+ u16 num_unicast = 0;
+ u16 s_rule_size;
+
+ if (!m_list || !hw)
+ return ICE_ERR_PARAM;
+
+ list_for_each_entry(m_list_itr, m_list, list_entry) {
+ u8 *add = &m_list_itr->fltr_info.l_data.mac.mac_addr[0];
+
+ if (m_list_itr->fltr_info.lkup_type != ICE_SW_LKUP_MAC)
+ return ICE_ERR_PARAM;
+ if (is_zero_ether_addr(add))
+ return ICE_ERR_PARAM;
+ if (is_unicast_ether_addr(add) && !hw->ucast_shared) {
+ /* Don't overwrite the unicast address */
+ if (ice_find_mac_entry(hw, add))
+ return ICE_ERR_ALREADY_EXISTS;
+ num_unicast++;
+ } else if (is_multicast_ether_addr(add) ||
+ (is_unicast_ether_addr(add) && hw->ucast_shared)) {
+ status = ice_add_shared_mac(hw, m_list_itr);
+ if (status) {
+ m_list_itr->status = ICE_FLTR_STATUS_FW_FAIL;
+ return status;
+ }
+ m_list_itr->status = ICE_FLTR_STATUS_FW_SUCCESS;
+ }
+ }
+
+ /* Exit if no suitable entries were found for adding bulk switch rule */
+ if (!num_unicast)
+ return 0;
+
+ /* Allocate switch rule buffer for the bulk update for unicast */
+ s_rule_size = ICE_SW_RULE_RX_TX_ETH_HDR_SIZE;
+ s_rule = devm_kcalloc(ice_hw_to_dev(hw), num_unicast, s_rule_size,
+ GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ r_iter = s_rule;
+ list_for_each_entry(m_list_itr, m_list, list_entry) {
+ struct ice_fltr_info *f_info = &m_list_itr->fltr_info;
+ u8 *addr = &f_info->l_data.mac.mac_addr[0];
+
+ if (is_unicast_ether_addr(addr)) {
+ ice_fill_sw_rule(hw, &m_list_itr->fltr_info,
+ r_iter, ice_aqc_opc_add_sw_rules);
+ r_iter = (struct ice_aqc_sw_rules_elem *)
+ ((u8 *)r_iter + s_rule_size);
+ }
+ }
+
+ /* Call AQ bulk switch rule update for all unicast addresses */
+ r_iter = s_rule;
+ /* Call AQ switch rule in AQ_MAX chunk */
+ for (total_elem_left = num_unicast; total_elem_left > 0;
+ total_elem_left -= elem_sent) {
+ struct ice_aqc_sw_rules_elem *entry = r_iter;
+
+ elem_sent = min(total_elem_left,
+ (u16)(ICE_AQ_MAX_BUF_LEN / s_rule_size));
+ status = ice_aq_sw_rules(hw, entry, elem_sent * s_rule_size,
+ elem_sent, ice_aqc_opc_add_sw_rules,
+ NULL);
+ if (status)
+ goto ice_add_mac_exit;
+ r_iter = (struct ice_aqc_sw_rules_elem *)
+ ((u8 *)r_iter + (elem_sent * s_rule_size));
+ }
+
+ /* Fill up rule id based on the value returned from FW */
+ r_iter = s_rule;
+ list_for_each_entry(m_list_itr, m_list, list_entry) {
+ struct ice_fltr_info *f_info = &m_list_itr->fltr_info;
+ u8 *addr = &f_info->l_data.mac.mac_addr[0];
+ struct ice_switch_info *sw = hw->switch_info;
+ struct ice_fltr_mgmt_list_entry *fm_entry;
+
+ if (is_unicast_ether_addr(addr)) {
+ f_info->fltr_rule_id =
+ le16_to_cpu(r_iter->pdata.lkup_tx_rx.index);
+ f_info->fltr_act = ICE_FWD_TO_VSI;
+ /* Create an entry to track this MAC address */
+ fm_entry = devm_kzalloc(ice_hw_to_dev(hw),
+ sizeof(*fm_entry), GFP_KERNEL);
+ if (!fm_entry) {
+ status = ICE_ERR_NO_MEMORY;
+ goto ice_add_mac_exit;
+ }
+ fm_entry->fltr_info = *f_info;
+ fm_entry->vsi_count = 1;
+ /* The book keeping entries will get removed when
+ * base driver calls remove filter AQ command
+ */
+ mutex_lock(&sw->mac_list_lock);
+ list_add(&fm_entry->list_entry, &sw->mac_list_head);
+ mutex_unlock(&sw->mac_list_lock);
+
+ r_iter = (struct ice_aqc_sw_rules_elem *)
+ ((u8 *)r_iter + s_rule_size);
+ }
+ }
+
+ice_add_mac_exit:
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ return status;
+}
+
+/**
+ * ice_find_vlan_entry
+ * @hw: pointer to the hardware structure
+ * @vlan_id: VLAN id to search for
+ *
+ * Helper function to search for a VLAN entry using a given VLAN id
+ * Returns pointer to the entry if found.
+ */
+static struct ice_fltr_mgmt_list_entry *
+ice_find_vlan_entry(struct ice_hw *hw, u16 vlan_id)
+{
+ struct ice_fltr_mgmt_list_entry *vlan_list_itr, *vlan_ret = NULL;
+ struct ice_switch_info *sw = hw->switch_info;
+
+ mutex_lock(&sw->vlan_list_lock);
+ list_for_each_entry(vlan_list_itr, &sw->vlan_list_head, list_entry)
+ if (vlan_list_itr->fltr_info.l_data.vlan.vlan_id == vlan_id) {
+ vlan_ret = vlan_list_itr;
+ break;
+ }
+
+ mutex_unlock(&sw->vlan_list_lock);
+ return vlan_ret;
+}
+
+/**
+ * ice_add_vlan_internal - Add one VLAN based filter rule
+ * @hw: pointer to the hardware structure
+ * @f_entry: filter entry containing one VLAN information
+ */
+static enum ice_status
+ice_add_vlan_internal(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
+{
+ struct ice_fltr_info *new_fltr, *cur_fltr;
+ struct ice_fltr_mgmt_list_entry *v_list_itr;
+ u16 vlan_id;
+
+ new_fltr = &f_entry->fltr_info;
+ /* VLAN id should only be 12 bits */
+ if (new_fltr->l_data.vlan.vlan_id > ICE_MAX_VLAN_ID)
+ return ICE_ERR_PARAM;
+
+ vlan_id = new_fltr->l_data.vlan.vlan_id;
+ v_list_itr = ice_find_vlan_entry(hw, vlan_id);
+ if (!v_list_itr) {
+ u16 vsi_id = ICE_VSI_INVAL_ID;
+ enum ice_status status;
+ u16 vsi_list_id = 0;
+
+ if (new_fltr->fltr_act == ICE_FWD_TO_VSI) {
+ enum ice_sw_lkup_type lkup_type = new_fltr->lkup_type;
+
+ /* All VLAN pruning rules use a VSI list.
+ * Convert the action to forwarding to a VSI list.
+ */
+ vsi_id = new_fltr->fwd_id.vsi_id;
+ status = ice_create_vsi_list_rule(hw, &vsi_id, 1,
+ &vsi_list_id,
+ lkup_type);
+ if (status)
+ return status;
+ new_fltr->fltr_act = ICE_FWD_TO_VSI_LIST;
+ new_fltr->fwd_id.vsi_list_id = vsi_list_id;
+ }
+
+ status = ice_create_pkt_fwd_rule(hw, f_entry);
+ if (!status && vsi_id != ICE_VSI_INVAL_ID) {
+ v_list_itr = ice_find_vlan_entry(hw, vlan_id);
+ if (!v_list_itr)
+ return ICE_ERR_DOES_NOT_EXIST;
+ v_list_itr->vsi_list_info =
+ ice_create_vsi_list_map(hw, &vsi_id, 1,
+ vsi_list_id);
+ }
+
+ return status;
+ }
+
+ cur_fltr = &v_list_itr->fltr_info;
+ return ice_handle_vsi_list_mgmt(hw, v_list_itr, cur_fltr, new_fltr);
+}
+
+/**
+ * ice_add_vlan - Add VLAN based filter rule
+ * @hw: pointer to the hardware structure
+ * @v_list: list of VLAN entries and forwarding information
+ */
+enum ice_status
+ice_add_vlan(struct ice_hw *hw, struct list_head *v_list)
+{
+ struct ice_fltr_list_entry *v_list_itr;
+
+ if (!v_list || !hw)
+ return ICE_ERR_PARAM;
+
+ list_for_each_entry(v_list_itr, v_list, list_entry) {
+ enum ice_status status;
+
+ if (v_list_itr->fltr_info.lkup_type != ICE_SW_LKUP_VLAN)
+ return ICE_ERR_PARAM;
+
+ status = ice_add_vlan_internal(hw, v_list_itr);
+ if (status) {
+ v_list_itr->status = ICE_FLTR_STATUS_FW_FAIL;
+ return status;
+ }
+ v_list_itr->status = ICE_FLTR_STATUS_FW_SUCCESS;
+ }
+ return 0;
+}
+
+/**
+ * ice_remove_vsi_list_rule
+ * @hw: pointer to the hardware structure
+ * @vsi_list_id: VSI list id generated as part of allocate resource
+ * @lkup_type: switch rule filter lookup type
+ */
+static enum ice_status
+ice_remove_vsi_list_rule(struct ice_hw *hw, u16 vsi_list_id,
+ enum ice_sw_lkup_type lkup_type)
+{
+ struct ice_aqc_sw_rules_elem *s_rule;
+ enum ice_status status;
+ u16 s_rule_size;
+
+ s_rule_size = (u16)ICE_SW_RULE_VSI_LIST_SIZE(0);
+ s_rule = devm_kzalloc(ice_hw_to_dev(hw), s_rule_size, GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ s_rule->type = cpu_to_le16(ICE_AQC_SW_RULES_T_VSI_LIST_CLEAR);
+ s_rule->pdata.vsi_list.index = cpu_to_le16(vsi_list_id);
+ /* FW expects number of VSIs in vsi_list resource to be 0 for clear
+ * command. Since memory is zero'ed out during initialization, it's not
+ * necessary to explicitly initialize the variable to 0.
+ */
+
+ status = ice_aq_sw_rules(hw, s_rule, s_rule_size, 1,
+ ice_aqc_opc_remove_sw_rules, NULL);
+ if (!status)
+ /* Free the vsi_list resource that we allocated */
+ status = ice_aq_alloc_free_vsi_list(hw, &vsi_list_id, lkup_type,
+ ice_aqc_opc_free_res);
+
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ return status;
+}
+
+/**
+ * ice_handle_rem_vsi_list_mgmt
+ * @hw: pointer to the hardware structure
+ * @vsi_id: ID of the VSI to remove
+ * @fm_list_itr: filter management entry for which the VSI list management
+ * needs to be done
+ */
+static enum ice_status
+ice_handle_rem_vsi_list_mgmt(struct ice_hw *hw, u16 vsi_id,
+ struct ice_fltr_mgmt_list_entry *fm_list_itr)
+{
+ struct ice_switch_info *sw = hw->switch_info;
+ enum ice_status status = 0;
+ enum ice_sw_lkup_type lkup_type;
+ bool is_last_elem = true;
+ bool conv_list = false;
+ bool del_list = false;
+ u16 vsi_list_id;
+
+ lkup_type = fm_list_itr->fltr_info.lkup_type;
+ vsi_list_id = fm_list_itr->fltr_info.fwd_id.vsi_list_id;
+
+ if (fm_list_itr->vsi_count > 1) {
+ status = ice_update_vsi_list_rule(hw, &vsi_id, 1, vsi_list_id,
+ true,
+ ice_aqc_opc_update_sw_rules,
+ lkup_type);
+ if (status)
+ return status;
+ fm_list_itr->vsi_count--;
+ is_last_elem = false;
+ clear_bit(vsi_id, fm_list_itr->vsi_list_info->vsi_map);
+ }
+
+ /* For non-VLAN rules that forward packets to a VSI list, convert them
+ * to forwarding packets to a VSI if there is only one VSI left in the
+ * list. Unused lists are then removed.
+ * VLAN rules need to use VSI lists even with only one VSI.
+ */
+ if (fm_list_itr->fltr_info.fltr_act == ICE_FWD_TO_VSI_LIST) {
+ if (lkup_type == ICE_SW_LKUP_VLAN) {
+ del_list = is_last_elem;
+ } else if (fm_list_itr->vsi_count == 1) {
+ conv_list = true;
+ del_list = true;
+ }
+ }
+
+ if (del_list) {
+ /* Remove the VSI list since it is no longer used */
+ struct ice_vsi_list_map_info *vsi_list_info =
+ fm_list_itr->vsi_list_info;
+
+ status = ice_remove_vsi_list_rule(hw, vsi_list_id, lkup_type);
+ if (status)
+ return status;
+
+ if (conv_list) {
+ u16 rem_vsi_id;
+
+ rem_vsi_id = find_first_bit(vsi_list_info->vsi_map,
+ ICE_MAX_VSI);
+
+ /* Error out when the expected last element is not in
+ * the VSI list map
+ */
+ if (rem_vsi_id == ICE_MAX_VSI)
+ return ICE_ERR_OUT_OF_RANGE;
+
+ /* Change the list entry action from VSI_LIST to VSI */
+ fm_list_itr->fltr_info.fltr_act = ICE_FWD_TO_VSI;
+ fm_list_itr->fltr_info.fwd_id.vsi_id = rem_vsi_id;
+ }
+
+ list_del(&vsi_list_info->list_entry);
+ devm_kfree(ice_hw_to_dev(hw), vsi_list_info);
+ fm_list_itr->vsi_list_info = NULL;
+ }
+
+ if (conv_list) {
+ /* Convert the rule's forward action to forwarding packets to
+ * a VSI
+ */
+ struct ice_aqc_sw_rules_elem *s_rule;
+
+ s_rule = devm_kzalloc(ice_hw_to_dev(hw),
+ ICE_SW_RULE_RX_TX_ETH_HDR_SIZE,
+ GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ ice_fill_sw_rule(hw, &fm_list_itr->fltr_info, s_rule,
+ ice_aqc_opc_update_sw_rules);
+
+ s_rule->pdata.lkup_tx_rx.index =
+ cpu_to_le16(fm_list_itr->fltr_info.fltr_rule_id);
+
+ status = ice_aq_sw_rules(hw, s_rule,
+ ICE_SW_RULE_RX_TX_ETH_HDR_SIZE, 1,
+ ice_aqc_opc_update_sw_rules, NULL);
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ if (status)
+ return status;
+ }
+
+ if (is_last_elem) {
+ /* Remove the lookup rule */
+ struct ice_aqc_sw_rules_elem *s_rule;
+
+ s_rule = devm_kzalloc(ice_hw_to_dev(hw),
+ ICE_SW_RULE_RX_TX_NO_HDR_SIZE,
+ GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ ice_fill_sw_rule(hw, &fm_list_itr->fltr_info, s_rule,
+ ice_aqc_opc_remove_sw_rules);
+
+ status = ice_aq_sw_rules(hw, s_rule,
+ ICE_SW_RULE_RX_TX_NO_HDR_SIZE, 1,
+ ice_aqc_opc_remove_sw_rules, NULL);
+ if (status)
+ return status;
+
+ /* Remove a book keeping entry from the MAC address list */
+ mutex_lock(&sw->mac_list_lock);
+ list_del(&fm_list_itr->list_entry);
+ mutex_unlock(&sw->mac_list_lock);
+ devm_kfree(ice_hw_to_dev(hw), fm_list_itr);
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ }
+ return status;
+}
+
+/**
+ * ice_remove_mac_entry
+ * @hw: pointer to the hardware structure
+ * @f_entry: structure containing MAC forwarding information
+ */
+static enum ice_status
+ice_remove_mac_entry(struct ice_hw *hw, struct ice_fltr_list_entry *f_entry)
+{
+ struct ice_fltr_mgmt_list_entry *m_entry;
+ u16 vsi_id;
+ u8 *add;
+
+ add = &f_entry->fltr_info.l_data.mac.mac_addr[0];
+
+ m_entry = ice_find_mac_entry(hw, add);
+ if (!m_entry)
+ return ICE_ERR_PARAM;
+
+ vsi_id = f_entry->fltr_info.fwd_id.vsi_id;
+ return ice_handle_rem_vsi_list_mgmt(hw, vsi_id, m_entry);
+}
+
+/**
+ * ice_remove_mac - remove a MAC address based filter rule
+ * @hw: pointer to the hardware structure
+ * @m_list: list of MAC addresses and forwarding information
+ *
+ * This function removes either a MAC filter rule or a specific VSI from a
+ * VSI list for a multicast MAC address.
+ *
+ * Returns ICE_ERR_DOES_NOT_EXIST if a given entry was not added by
+ * ice_add_mac. Caller should be aware that this call will only work if all
+ * the entries passed into m_list were added previously. It will not attempt to
+ * do a partial remove of entries that were found.
+ */
+enum ice_status
+ice_remove_mac(struct ice_hw *hw, struct list_head *m_list)
+{
+ struct ice_aqc_sw_rules_elem *s_rule, *r_iter;
+ u8 s_rule_size = ICE_SW_RULE_RX_TX_NO_HDR_SIZE;
+ struct ice_switch_info *sw = hw->switch_info;
+ struct ice_fltr_mgmt_list_entry *m_entry;
+ struct ice_fltr_list_entry *m_list_itr;
+ u16 elem_sent, total_elem_left;
+ enum ice_status status = 0;
+ u16 num_unicast = 0;
+
+ if (!m_list)
+ return ICE_ERR_PARAM;
+
+ list_for_each_entry(m_list_itr, m_list, list_entry) {
+ u8 *addr = m_list_itr->fltr_info.l_data.mac.mac_addr;
+
+ if (is_unicast_ether_addr(addr) && !hw->ucast_shared)
+ num_unicast++;
+ else if (is_multicast_ether_addr(addr) ||
+ (is_unicast_ether_addr(addr) && hw->ucast_shared))
+ ice_remove_mac_entry(hw, m_list_itr);
+ }
+
+ /* Exit if no unicast addresses found. Multicast switch rules
+ * were added individually
+ */
+ if (!num_unicast)
+ return 0;
+
+ /* Allocate switch rule buffer for the bulk update for unicast */
+ s_rule = devm_kcalloc(ice_hw_to_dev(hw), num_unicast, s_rule_size,
+ GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ r_iter = s_rule;
+ list_for_each_entry(m_list_itr, m_list, list_entry) {
+ u8 *addr = m_list_itr->fltr_info.l_data.mac.mac_addr;
+
+ if (is_unicast_ether_addr(addr)) {
+ m_entry = ice_find_mac_entry(hw, addr);
+ if (!m_entry) {
+ status = ICE_ERR_DOES_NOT_EXIST;
+ goto ice_remove_mac_exit;
+ }
+
+ ice_fill_sw_rule(hw, &m_entry->fltr_info,
+ r_iter, ice_aqc_opc_remove_sw_rules);
+ r_iter = (struct ice_aqc_sw_rules_elem *)
+ ((u8 *)r_iter + s_rule_size);
+ }
+ }
+
+ /* Call AQ bulk switch rule update for all unicast addresses */
+ r_iter = s_rule;
+ /* Call AQ switch rule in AQ_MAX chunk */
+ for (total_elem_left = num_unicast; total_elem_left > 0;
+ total_elem_left -= elem_sent) {
+ struct ice_aqc_sw_rules_elem *entry = r_iter;
+
+ elem_sent = min(total_elem_left,
+ (u16)(ICE_AQ_MAX_BUF_LEN / s_rule_size));
+ status = ice_aq_sw_rules(hw, entry, elem_sent * s_rule_size,
+ elem_sent, ice_aqc_opc_remove_sw_rules,
+ NULL);
+ if (status)
+ break;
+ r_iter = (struct ice_aqc_sw_rules_elem *)
+ ((u8 *)r_iter + s_rule_size);
+ }
+
+ list_for_each_entry(m_list_itr, m_list, list_entry) {
+ u8 *addr = m_list_itr->fltr_info.l_data.mac.mac_addr;
+
+ if (is_unicast_ether_addr(addr)) {
+ m_entry = ice_find_mac_entry(hw, addr);
+ if (!m_entry)
+ return ICE_ERR_OUT_OF_RANGE;
+ mutex_lock(&sw->mac_list_lock);
+ list_del(&m_entry->list_entry);
+ mutex_unlock(&sw->mac_list_lock);
+ devm_kfree(ice_hw_to_dev(hw), m_entry);
+ }
+ }
+
+ice_remove_mac_exit:
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ return status;
+}
+
+/**
+ * ice_cfg_dflt_vsi - add filter rule to set/unset given VSI as default
+ * VSI for the switch (represented by swid)
+ * @hw: pointer to the hardware structure
+ * @vsi_id: number of VSI to set as default
+ * @set: true to add the above mentioned switch rule, false to remove it
+ * @direction: ICE_FLTR_RX or ICE_FLTR_TX
+ */
+enum ice_status
+ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_id, bool set, u8 direction)
+{
+ struct ice_aqc_sw_rules_elem *s_rule;
+ struct ice_fltr_info f_info;
+ enum ice_adminq_opc opcode;
+ enum ice_status status;
+ u16 s_rule_size;
+
+ s_rule_size = set ? ICE_SW_RULE_RX_TX_ETH_HDR_SIZE :
+ ICE_SW_RULE_RX_TX_NO_HDR_SIZE;
+ s_rule = devm_kzalloc(ice_hw_to_dev(hw), s_rule_size, GFP_KERNEL);
+ if (!s_rule)
+ return ICE_ERR_NO_MEMORY;
+
+ memset(&f_info, 0, sizeof(f_info));
+
+ f_info.lkup_type = ICE_SW_LKUP_DFLT;
+ f_info.flag = direction;
+ f_info.fltr_act = ICE_FWD_TO_VSI;
+ f_info.fwd_id.vsi_id = vsi_id;
+
+ if (f_info.flag & ICE_FLTR_RX) {
+ f_info.src = hw->port_info->lport;
+ if (!set)
+ f_info.fltr_rule_id =
+ hw->port_info->dflt_rx_vsi_rule_id;
+ } else if (f_info.flag & ICE_FLTR_TX) {
+ f_info.src = vsi_id;
+ if (!set)
+ f_info.fltr_rule_id =
+ hw->port_info->dflt_tx_vsi_rule_id;
+ }
+
+ if (set)
+ opcode = ice_aqc_opc_add_sw_rules;
+ else
+ opcode = ice_aqc_opc_remove_sw_rules;
+
+ ice_fill_sw_rule(hw, &f_info, s_rule, opcode);
+
+ status = ice_aq_sw_rules(hw, s_rule, s_rule_size, 1, opcode, NULL);
+ if (status || !(f_info.flag & ICE_FLTR_TX_RX))
+ goto out;
+ if (set) {
+ u16 index = le16_to_cpu(s_rule->pdata.lkup_tx_rx.index);
+
+ if (f_info.flag & ICE_FLTR_TX) {
+ hw->port_info->dflt_tx_vsi_num = vsi_id;
+ hw->port_info->dflt_tx_vsi_rule_id = index;
+ } else if (f_info.flag & ICE_FLTR_RX) {
+ hw->port_info->dflt_rx_vsi_num = vsi_id;
+ hw->port_info->dflt_rx_vsi_rule_id = index;
+ }
+ } else {
+ if (f_info.flag & ICE_FLTR_TX) {
+ hw->port_info->dflt_tx_vsi_num = ICE_DFLT_VSI_INVAL;
+ hw->port_info->dflt_tx_vsi_rule_id = ICE_INVAL_ACT;
+ } else if (f_info.flag & ICE_FLTR_RX) {
+ hw->port_info->dflt_rx_vsi_num = ICE_DFLT_VSI_INVAL;
+ hw->port_info->dflt_rx_vsi_rule_id = ICE_INVAL_ACT;
+ }
+ }
+
+out:
+ devm_kfree(ice_hw_to_dev(hw), s_rule);
+ return status;
+}
+
+/**
+ * ice_remove_vlan_internal - Remove one VLAN based filter rule
+ * @hw: pointer to the hardware structure
+ * @f_entry: filter entry containing one VLAN information
+ */
+static enum ice_status
+ice_remove_vlan_internal(struct ice_hw *hw,
+ struct ice_fltr_list_entry *f_entry)
+{
+ struct ice_fltr_info *new_fltr;
+ struct ice_fltr_mgmt_list_entry *v_list_elem;
+ u16 vsi_id;
+
+ new_fltr = &f_entry->fltr_info;
+
+ v_list_elem = ice_find_vlan_entry(hw, new_fltr->l_data.vlan.vlan_id);
+ if (!v_list_elem)
+ return ICE_ERR_PARAM;
+
+ vsi_id = f_entry->fltr_info.fwd_id.vsi_id;
+ return ice_handle_rem_vsi_list_mgmt(hw, vsi_id, v_list_elem);
+}
+
+/**
+ * ice_remove_vlan - Remove VLAN based filter rule
+ * @hw: pointer to the hardware structure
+ * @v_list: list of VLAN entries and forwarding information
+ */
+enum ice_status
+ice_remove_vlan(struct ice_hw *hw, struct list_head *v_list)
+{
+ struct ice_fltr_list_entry *v_list_itr;
+ enum ice_status status = 0;
+
+ if (!v_list || !hw)
+ return ICE_ERR_PARAM;
+
+ list_for_each_entry(v_list_itr, v_list, list_entry) {
+ status = ice_remove_vlan_internal(hw, v_list_itr);
+ if (status) {
+ v_list_itr->status = ICE_FLTR_STATUS_FW_FAIL;
+ return status;
+ }
+ v_list_itr->status = ICE_FLTR_STATUS_FW_SUCCESS;
+ }
+ return status;
+}
+
+/**
+ * ice_add_to_vsi_fltr_list - Add VSI filters to the list
+ * @hw: pointer to the hardware structure
+ * @vsi_id: ID of VSI to remove filters from
+ * @lkup_list_head: pointer to the list that has certain lookup type filters
+ * @vsi_list_head: pointer to the list pertaining to VSI with vsi_id
+ */
+static enum ice_status
+ice_add_to_vsi_fltr_list(struct ice_hw *hw, u16 vsi_id,
+ struct list_head *lkup_list_head,
+ struct list_head *vsi_list_head)
+{
+ struct ice_fltr_mgmt_list_entry *fm_entry;
+
+ /* check to make sure VSI id is valid and within boundary */
+ if (vsi_id >=
+ (sizeof(fm_entry->vsi_list_info->vsi_map) * BITS_PER_BYTE - 1))
+ return ICE_ERR_PARAM;
+
+ list_for_each_entry(fm_entry, lkup_list_head, list_entry) {
+ struct ice_fltr_info *fi;
+
+ fi = &fm_entry->fltr_info;
+ if ((fi->fltr_act == ICE_FWD_TO_VSI &&
+ fi->fwd_id.vsi_id == vsi_id) ||
+ (fi->fltr_act == ICE_FWD_TO_VSI_LIST &&
+ (test_bit(vsi_id, fm_entry->vsi_list_info->vsi_map)))) {
+ struct ice_fltr_list_entry *tmp;
+
+ /* this memory is freed up in the caller function
+ * ice_remove_vsi_lkup_fltr() once filters for
+ * this VSI are removed
+ */
+ tmp = devm_kzalloc(ice_hw_to_dev(hw), sizeof(*tmp),
+ GFP_KERNEL);
+ if (!tmp)
+ return ICE_ERR_NO_MEMORY;
+
+ memcpy(&tmp->fltr_info, fi, sizeof(*fi));
+
+ /* Expected below fields to be set to ICE_FWD_TO_VSI and
+ * the particular VSI id since we are only removing this
+ * one VSI
+ */
+ if (fi->fltr_act == ICE_FWD_TO_VSI_LIST) {
+ tmp->fltr_info.fltr_act = ICE_FWD_TO_VSI;
+ tmp->fltr_info.fwd_id.vsi_id = vsi_id;
+ }
+
+ list_add(&tmp->list_entry, vsi_list_head);
+ }
+ }
+ return 0;
+}
+
+/**
+ * ice_remove_vsi_lkup_fltr - Remove lookup type filters for a VSI
+ * @hw: pointer to the hardware structure
+ * @vsi_id: ID of VSI to remove filters from
+ * @lkup: switch rule filter lookup type
+ */
+static void
+ice_remove_vsi_lkup_fltr(struct ice_hw *hw, u16 vsi_id,
+ enum ice_sw_lkup_type lkup)
+{
+ struct ice_switch_info *sw = hw->switch_info;
+ struct ice_fltr_list_entry *fm_entry;
+ struct list_head remove_list_head;
+ struct ice_fltr_list_entry *tmp;
+ enum ice_status status;
+
+ INIT_LIST_HEAD(&remove_list_head);
+ switch (lkup) {
+ case ICE_SW_LKUP_MAC:
+ mutex_lock(&sw->mac_list_lock);
+ status = ice_add_to_vsi_fltr_list(hw, vsi_id,
+ &sw->mac_list_head,
+ &remove_list_head);
+ mutex_unlock(&sw->mac_list_lock);
+ if (!status) {
+ ice_remove_mac(hw, &remove_list_head);
+ goto free_fltr_list;
+ }
+ break;
+ case ICE_SW_LKUP_VLAN:
+ mutex_lock(&sw->vlan_list_lock);
+ status = ice_add_to_vsi_fltr_list(hw, vsi_id,
+ &sw->vlan_list_head,
+ &remove_list_head);
+ mutex_unlock(&sw->vlan_list_lock);
+ if (!status) {
+ ice_remove_vlan(hw, &remove_list_head);
+ goto free_fltr_list;
+ }
+ break;
+ case ICE_SW_LKUP_MAC_VLAN:
+ case ICE_SW_LKUP_ETHERTYPE:
+ case ICE_SW_LKUP_ETHERTYPE_MAC:
+ case ICE_SW_LKUP_PROMISC:
+ case ICE_SW_LKUP_PROMISC_VLAN:
+ case ICE_SW_LKUP_DFLT:
+ ice_debug(hw, ICE_DBG_SW,
+ "Remove filters for this lookup type hasn't been implemented yet\n");
+ break;
+ }
+
+ return;
+free_fltr_list:
+ list_for_each_entry_safe(fm_entry, tmp, &remove_list_head, list_entry) {
+ list_del(&fm_entry->list_entry);
+ devm_kfree(ice_hw_to_dev(hw), fm_entry);
+ }
+}
+
+/**
+ * ice_remove_vsi_fltr - Remove all filters for a VSI
+ * @hw: pointer to the hardware structure
+ * @vsi_id: ID of VSI to remove filters from
+ */
+void ice_remove_vsi_fltr(struct ice_hw *hw, u16 vsi_id)
+{
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_MAC);
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_MAC_VLAN);
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_PROMISC);
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_VLAN);
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_DFLT);
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_ETHERTYPE);
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_ETHERTYPE_MAC);
+ ice_remove_vsi_lkup_fltr(hw, vsi_id, ICE_SW_LKUP_PROMISC_VLAN);
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_SWITCH_H_
+#define _ICE_SWITCH_H_
+
+#include "ice_common.h"
+
+#define ICE_SW_CFG_MAX_BUF_LEN 2048
+#define ICE_DFLT_VSI_INVAL 0xff
+#define ICE_VSI_INVAL_ID 0xffff
+
+/* VSI context structure for add/get/update/free operations */
+struct ice_vsi_ctx {
+ u16 vsi_num;
+ u16 vsis_allocd;
+ u16 vsis_unallocated;
+ u16 flags;
+ struct ice_aqc_vsi_props info;
+ bool alloc_from_pool;
+};
+
+enum ice_sw_fwd_act_type {
+ ICE_FWD_TO_VSI = 0,
+ ICE_FWD_TO_VSI_LIST, /* Do not use this when adding filter */
+ ICE_FWD_TO_Q,
+ ICE_FWD_TO_QGRP,
+ ICE_DROP_PACKET,
+ ICE_INVAL_ACT
+};
+
+/* Switch recipe ID enum values are specific to hardware */
+enum ice_sw_lkup_type {
+ ICE_SW_LKUP_ETHERTYPE = 0,
+ ICE_SW_LKUP_MAC = 1,
+ ICE_SW_LKUP_MAC_VLAN = 2,
+ ICE_SW_LKUP_PROMISC = 3,
+ ICE_SW_LKUP_VLAN = 4,
+ ICE_SW_LKUP_DFLT = 5,
+ ICE_SW_LKUP_ETHERTYPE_MAC = 8,
+ ICE_SW_LKUP_PROMISC_VLAN = 9,
+};
+
+struct ice_fltr_info {
+ /* Look up information: how to look up packet */
+ enum ice_sw_lkup_type lkup_type;
+ /* Forward action: filter action to do after lookup */
+ enum ice_sw_fwd_act_type fltr_act;
+ /* rule ID returned by firmware once filter rule is created */
+ u16 fltr_rule_id;
+ u16 flag;
+#define ICE_FLTR_RX BIT(0)
+#define ICE_FLTR_TX BIT(1)
+#define ICE_FLTR_TX_RX (ICE_FLTR_RX | ICE_FLTR_TX)
+
+ /* Source VSI for LOOKUP_TX or source port for LOOKUP_RX */
+ u16 src;
+
+ union {
+ struct {
+ u8 mac_addr[ETH_ALEN];
+ } mac;
+ struct {
+ u8 mac_addr[ETH_ALEN];
+ u16 vlan_id;
+ } mac_vlan;
+ struct {
+ u16 vlan_id;
+ } vlan;
+ /* Set lkup_type as ICE_SW_LKUP_ETHERTYPE
+ * if just using ethertype as filter. Set lkup_type as
+ * ICE_SW_LKUP_ETHERTYPE_MAC if MAC also needs to be
+ * passed in as filter.
+ */
+ struct {
+ u16 ethertype;
+ u8 mac_addr[ETH_ALEN]; /* optional */
+ } ethertype_mac;
+ } l_data;
+
+ /* Depending on filter action */
+ union {
+ /* queue id in case of ICE_FWD_TO_Q and starting
+ * queue id in case of ICE_FWD_TO_QGRP.
+ */
+ u16 q_id:11;
+ u16 vsi_id:10;
+ u16 vsi_list_id:10;
+ } fwd_id;
+
+ /* Set to num_queues if action is ICE_FWD_TO_QGRP. This field
+ * determines the range of queues the packet needs to be forwarded to
+ */
+ u8 qgrp_size;
+
+ /* Rule creations populate these indicators basing on the switch type */
+ bool lb_en; /* Indicate if packet can be looped back */
+ bool lan_en; /* Indicate if packet can be forwarded to the uplink */
+};
+
+/* Bookkeeping structure to hold bitmap of VSIs corresponding to VSI list id */
+struct ice_vsi_list_map_info {
+ struct list_head list_entry;
+ DECLARE_BITMAP(vsi_map, ICE_MAX_VSI);
+ u16 vsi_list_id;
+};
+
+enum ice_sw_fltr_status {
+ ICE_FLTR_STATUS_NEW = 0,
+ ICE_FLTR_STATUS_FW_SUCCESS,
+ ICE_FLTR_STATUS_FW_FAIL,
+};
+
+struct ice_fltr_list_entry {
+ struct list_head list_entry;
+ enum ice_sw_fltr_status status;
+ struct ice_fltr_info fltr_info;
+};
+
+/* This defines an entry in the list that maintains MAC or VLAN membership
+ * to HW list mapping, since multiple VSIs can subscribe to the same MAC or
+ * VLAN. As an optimization the VSI list should be created only when a
+ * second VSI becomes a subscriber to the VLAN address.
+ */
+struct ice_fltr_mgmt_list_entry {
+ /* back pointer to VSI list id to VSI list mapping */
+ struct ice_vsi_list_map_info *vsi_list_info;
+ u16 vsi_count;
+#define ICE_INVAL_LG_ACT_INDEX 0xffff
+ u16 lg_act_idx;
+#define ICE_INVAL_SW_MARKER_ID 0xffff
+ u16 sw_marker_id;
+ struct list_head list_entry;
+ struct ice_fltr_info fltr_info;
+#define ICE_INVAL_COUNTER_ID 0xff
+ u8 counter_index;
+};
+
+/* VSI related commands */
+enum ice_status
+ice_aq_add_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
+ struct ice_sq_cd *cd);
+enum ice_status
+ice_aq_update_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
+ struct ice_sq_cd *cd);
+enum ice_status
+ice_aq_free_vsi(struct ice_hw *hw, struct ice_vsi_ctx *vsi_ctx,
+ bool keep_vsi_alloc, struct ice_sq_cd *cd);
+
+enum ice_status ice_get_initial_sw_cfg(struct ice_hw *hw);
+
+/* Switch/bridge related commands */
+enum ice_status ice_add_mac(struct ice_hw *hw, struct list_head *m_lst);
+enum ice_status ice_remove_mac(struct ice_hw *hw, struct list_head *m_lst);
+void ice_remove_vsi_fltr(struct ice_hw *hw, u16 vsi_id);
+enum ice_status ice_add_vlan(struct ice_hw *hw, struct list_head *m_list);
+enum ice_status ice_remove_vlan(struct ice_hw *hw, struct list_head *v_list);
+enum ice_status
+ice_cfg_dflt_vsi(struct ice_hw *hw, u16 vsi_id, bool set, u8 direction);
+
+#endif /* _ICE_SWITCH_H_ */
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018, Intel Corporation. */
+
+/* The driver transmit and receive code */
+
+#include <linux/prefetch.h>
+#include <linux/mm.h>
+#include "ice.h"
+
+#define ICE_RX_HDR_SIZE 256
+
+/**
+ * ice_unmap_and_free_tx_buf - Release a Tx buffer
+ * @ring: the ring that owns the buffer
+ * @tx_buf: the buffer to free
+ */
+static void
+ice_unmap_and_free_tx_buf(struct ice_ring *ring, struct ice_tx_buf *tx_buf)
+{
+ if (tx_buf->skb) {
+ dev_kfree_skb_any(tx_buf->skb);
+ if (dma_unmap_len(tx_buf, len))
+ dma_unmap_single(ring->dev,
+ dma_unmap_addr(tx_buf, dma),
+ dma_unmap_len(tx_buf, len),
+ DMA_TO_DEVICE);
+ } else if (dma_unmap_len(tx_buf, len)) {
+ dma_unmap_page(ring->dev,
+ dma_unmap_addr(tx_buf, dma),
+ dma_unmap_len(tx_buf, len),
+ DMA_TO_DEVICE);
+ }
+
+ tx_buf->next_to_watch = NULL;
+ tx_buf->skb = NULL;
+ dma_unmap_len_set(tx_buf, len, 0);
+ /* tx_buf must be completely set up in the transmit path */
+}
+
+static struct netdev_queue *txring_txq(const struct ice_ring *ring)
+{
+ return netdev_get_tx_queue(ring->netdev, ring->q_index);
+}
+
+/**
+ * ice_clean_tx_ring - Free any empty Tx buffers
+ * @tx_ring: ring to be cleaned
+ */
+void ice_clean_tx_ring(struct ice_ring *tx_ring)
+{
+ unsigned long size;
+ u16 i;
+
+ /* ring already cleared, nothing to do */
+ if (!tx_ring->tx_buf)
+ return;
+
+ /* Free all the Tx ring sk_bufss */
+ for (i = 0; i < tx_ring->count; i++)
+ ice_unmap_and_free_tx_buf(tx_ring, &tx_ring->tx_buf[i]);
+
+ size = sizeof(struct ice_tx_buf) * tx_ring->count;
+ memset(tx_ring->tx_buf, 0, size);
+
+ /* Zero out the descriptor ring */
+ memset(tx_ring->desc, 0, tx_ring->size);
+
+ tx_ring->next_to_use = 0;
+ tx_ring->next_to_clean = 0;
+
+ if (!tx_ring->netdev)
+ return;
+
+ /* cleanup Tx queue statistics */
+ netdev_tx_reset_queue(txring_txq(tx_ring));
+}
+
+/**
+ * ice_free_tx_ring - Free Tx resources per queue
+ * @tx_ring: Tx descriptor ring for a specific queue
+ *
+ * Free all transmit software resources
+ */
+void ice_free_tx_ring(struct ice_ring *tx_ring)
+{
+ ice_clean_tx_ring(tx_ring);
+ devm_kfree(tx_ring->dev, tx_ring->tx_buf);
+ tx_ring->tx_buf = NULL;
+
+ if (tx_ring->desc) {
+ dmam_free_coherent(tx_ring->dev, tx_ring->size,
+ tx_ring->desc, tx_ring->dma);
+ tx_ring->desc = NULL;
+ }
+}
+
+/**
+ * ice_clean_tx_irq - Reclaim resources after transmit completes
+ * @vsi: the VSI we care about
+ * @tx_ring: Tx ring to clean
+ * @napi_budget: Used to determine if we are in netpoll
+ *
+ * Returns true if there's any budget left (e.g. the clean is finished)
+ */
+static bool ice_clean_tx_irq(struct ice_vsi *vsi, struct ice_ring *tx_ring,
+ int napi_budget)
+{
+ unsigned int total_bytes = 0, total_pkts = 0;
+ unsigned int budget = vsi->work_lmt;
+ s16 i = tx_ring->next_to_clean;
+ struct ice_tx_desc *tx_desc;
+ struct ice_tx_buf *tx_buf;
+
+ tx_buf = &tx_ring->tx_buf[i];
+ tx_desc = ICE_TX_DESC(tx_ring, i);
+ i -= tx_ring->count;
+
+ do {
+ struct ice_tx_desc *eop_desc = tx_buf->next_to_watch;
+
+ /* if next_to_watch is not set then there is no work pending */
+ if (!eop_desc)
+ break;
+
+ smp_rmb(); /* prevent any other reads prior to eop_desc */
+
+ /* if the descriptor isn't done, no work yet to do */
+ if (!(eop_desc->cmd_type_offset_bsz &
+ cpu_to_le64(ICE_TX_DESC_DTYPE_DESC_DONE)))
+ break;
+
+ /* clear next_to_watch to prevent false hangs */
+ tx_buf->next_to_watch = NULL;
+
+ /* update the statistics for this packet */
+ total_bytes += tx_buf->bytecount;
+ total_pkts += tx_buf->gso_segs;
+
+ /* free the skb */
+ napi_consume_skb(tx_buf->skb, napi_budget);
+
+ /* unmap skb header data */
+ dma_unmap_single(tx_ring->dev,
+ dma_unmap_addr(tx_buf, dma),
+ dma_unmap_len(tx_buf, len),
+ DMA_TO_DEVICE);
+
+ /* clear tx_buf data */
+ tx_buf->skb = NULL;
+ dma_unmap_len_set(tx_buf, len, 0);
+
+ /* unmap remaining buffers */
+ while (tx_desc != eop_desc) {
+ tx_buf++;
+ tx_desc++;
+ i++;
+ if (unlikely(!i)) {
+ i -= tx_ring->count;
+ tx_buf = tx_ring->tx_buf;
+ tx_desc = ICE_TX_DESC(tx_ring, 0);
+ }
+
+ /* unmap any remaining paged data */
+ if (dma_unmap_len(tx_buf, len)) {
+ dma_unmap_page(tx_ring->dev,
+ dma_unmap_addr(tx_buf, dma),
+ dma_unmap_len(tx_buf, len),
+ DMA_TO_DEVICE);
+ dma_unmap_len_set(tx_buf, len, 0);
+ }
+ }
+
+ /* move us one more past the eop_desc for start of next pkt */
+ tx_buf++;
+ tx_desc++;
+ i++;
+ if (unlikely(!i)) {
+ i -= tx_ring->count;
+ tx_buf = tx_ring->tx_buf;
+ tx_desc = ICE_TX_DESC(tx_ring, 0);
+ }
+
+ prefetch(tx_desc);
+
+ /* update budget accounting */
+ budget--;
+ } while (likely(budget));
+
+ i += tx_ring->count;
+ tx_ring->next_to_clean = i;
+ u64_stats_update_begin(&tx_ring->syncp);
+ tx_ring->stats.bytes += total_bytes;
+ tx_ring->stats.pkts += total_pkts;
+ u64_stats_update_end(&tx_ring->syncp);
+ tx_ring->q_vector->tx.total_bytes += total_bytes;
+ tx_ring->q_vector->tx.total_pkts += total_pkts;
+
+ netdev_tx_completed_queue(txring_txq(tx_ring), total_pkts,
+ total_bytes);
+
+#define TX_WAKE_THRESHOLD ((s16)(DESC_NEEDED * 2))
+ if (unlikely(total_pkts && netif_carrier_ok(tx_ring->netdev) &&
+ (ICE_DESC_UNUSED(tx_ring) >= TX_WAKE_THRESHOLD))) {
+ /* Make sure that anybody stopping the queue after this
+ * sees the new next_to_clean.
+ */
+ smp_mb();
+ if (__netif_subqueue_stopped(tx_ring->netdev,
+ tx_ring->q_index) &&
+ !test_bit(__ICE_DOWN, vsi->state)) {
+ netif_wake_subqueue(tx_ring->netdev,
+ tx_ring->q_index);
+ ++tx_ring->tx_stats.restart_q;
+ }
+ }
+
+ return !!budget;
+}
+
+/**
+ * ice_setup_tx_ring - Allocate the Tx descriptors
+ * @tx_ring: the tx ring to set up
+ *
+ * Return 0 on success, negative on error
+ */
+int ice_setup_tx_ring(struct ice_ring *tx_ring)
+{
+ struct device *dev = tx_ring->dev;
+ int bi_size;
+
+ if (!dev)
+ return -ENOMEM;
+
+ /* warn if we are about to overwrite the pointer */
+ WARN_ON(tx_ring->tx_buf);
+ bi_size = sizeof(struct ice_tx_buf) * tx_ring->count;
+ tx_ring->tx_buf = devm_kzalloc(dev, bi_size, GFP_KERNEL);
+ if (!tx_ring->tx_buf)
+ return -ENOMEM;
+
+ /* round up to nearest 4K */
+ tx_ring->size = tx_ring->count * sizeof(struct ice_tx_desc);
+ tx_ring->size = ALIGN(tx_ring->size, 4096);
+ tx_ring->desc = dmam_alloc_coherent(dev, tx_ring->size, &tx_ring->dma,
+ GFP_KERNEL);
+ if (!tx_ring->desc) {
+ dev_err(dev, "Unable to allocate memory for the Tx descriptor ring, size=%d\n",
+ tx_ring->size);
+ goto err;
+ }
+
+ tx_ring->next_to_use = 0;
+ tx_ring->next_to_clean = 0;
+ return 0;
+
+err:
+ devm_kfree(dev, tx_ring->tx_buf);
+ tx_ring->tx_buf = NULL;
+ return -ENOMEM;
+}
+
+/**
+ * ice_clean_rx_ring - Free Rx buffers
+ * @rx_ring: ring to be cleaned
+ */
+void ice_clean_rx_ring(struct ice_ring *rx_ring)
+{
+ struct device *dev = rx_ring->dev;
+ unsigned long size;
+ u16 i;
+
+ /* ring already cleared, nothing to do */
+ if (!rx_ring->rx_buf)
+ return;
+
+ /* Free all the Rx ring sk_buffs */
+ for (i = 0; i < rx_ring->count; i++) {
+ struct ice_rx_buf *rx_buf = &rx_ring->rx_buf[i];
+
+ if (rx_buf->skb) {
+ dev_kfree_skb(rx_buf->skb);
+ rx_buf->skb = NULL;
+ }
+ if (!rx_buf->page)
+ continue;
+
+ dma_unmap_page(dev, rx_buf->dma, PAGE_SIZE, DMA_FROM_DEVICE);
+ __free_pages(rx_buf->page, 0);
+
+ rx_buf->page = NULL;
+ rx_buf->page_offset = 0;
+ }
+
+ size = sizeof(struct ice_rx_buf) * rx_ring->count;
+ memset(rx_ring->rx_buf, 0, size);
+
+ /* Zero out the descriptor ring */
+ memset(rx_ring->desc, 0, rx_ring->size);
+
+ rx_ring->next_to_alloc = 0;
+ rx_ring->next_to_clean = 0;
+ rx_ring->next_to_use = 0;
+}
+
+/**
+ * ice_free_rx_ring - Free Rx resources
+ * @rx_ring: ring to clean the resources from
+ *
+ * Free all receive software resources
+ */
+void ice_free_rx_ring(struct ice_ring *rx_ring)
+{
+ ice_clean_rx_ring(rx_ring);
+ devm_kfree(rx_ring->dev, rx_ring->rx_buf);
+ rx_ring->rx_buf = NULL;
+
+ if (rx_ring->desc) {
+ dmam_free_coherent(rx_ring->dev, rx_ring->size,
+ rx_ring->desc, rx_ring->dma);
+ rx_ring->desc = NULL;
+ }
+}
+
+/**
+ * ice_setup_rx_ring - Allocate the Rx descriptors
+ * @rx_ring: the rx ring to set up
+ *
+ * Return 0 on success, negative on error
+ */
+int ice_setup_rx_ring(struct ice_ring *rx_ring)
+{
+ struct device *dev = rx_ring->dev;
+ int bi_size;
+
+ if (!dev)
+ return -ENOMEM;
+
+ /* warn if we are about to overwrite the pointer */
+ WARN_ON(rx_ring->rx_buf);
+ bi_size = sizeof(struct ice_rx_buf) * rx_ring->count;
+ rx_ring->rx_buf = devm_kzalloc(dev, bi_size, GFP_KERNEL);
+ if (!rx_ring->rx_buf)
+ return -ENOMEM;
+
+ /* round up to nearest 4K */
+ rx_ring->size = rx_ring->count * sizeof(union ice_32byte_rx_desc);
+ rx_ring->size = ALIGN(rx_ring->size, 4096);
+ rx_ring->desc = dmam_alloc_coherent(dev, rx_ring->size, &rx_ring->dma,
+ GFP_KERNEL);
+ if (!rx_ring->desc) {
+ dev_err(dev, "Unable to allocate memory for the Rx descriptor ring, size=%d\n",
+ rx_ring->size);
+ goto err;
+ }
+
+ rx_ring->next_to_use = 0;
+ rx_ring->next_to_clean = 0;
+ return 0;
+
+err:
+ devm_kfree(dev, rx_ring->rx_buf);
+ rx_ring->rx_buf = NULL;
+ return -ENOMEM;
+}
+
+/**
+ * ice_release_rx_desc - Store the new tail and head values
+ * @rx_ring: ring to bump
+ * @val: new head index
+ */
+static void ice_release_rx_desc(struct ice_ring *rx_ring, u32 val)
+{
+ rx_ring->next_to_use = val;
+
+ /* update next to alloc since we have filled the ring */
+ rx_ring->next_to_alloc = val;
+
+ /* Force memory writes to complete before letting h/w
+ * know there are new descriptors to fetch. (Only
+ * applicable for weak-ordered memory model archs,
+ * such as IA-64).
+ */
+ wmb();
+ writel(val, rx_ring->tail);
+}
+
+/**
+ * ice_alloc_mapped_page - recycle or make a new page
+ * @rx_ring: ring to use
+ * @bi: rx_buf struct to modify
+ *
+ * Returns true if the page was successfully allocated or
+ * reused.
+ */
+static bool ice_alloc_mapped_page(struct ice_ring *rx_ring,
+ struct ice_rx_buf *bi)
+{
+ struct page *page = bi->page;
+ dma_addr_t dma;
+
+ /* since we are recycling buffers we should seldom need to alloc */
+ if (likely(page)) {
+ rx_ring->rx_stats.page_reuse_count++;
+ return true;
+ }
+
+ /* alloc new page for storage */
+ page = alloc_page(GFP_ATOMIC | __GFP_NOWARN);
+ if (unlikely(!page)) {
+ rx_ring->rx_stats.alloc_page_failed++;
+ return false;
+ }
+
+ /* map page for use */
+ dma = dma_map_page(rx_ring->dev, page, 0, PAGE_SIZE, DMA_FROM_DEVICE);
+
+ /* if mapping failed free memory back to system since
+ * there isn't much point in holding memory we can't use
+ */
+ if (dma_mapping_error(rx_ring->dev, dma)) {
+ __free_pages(page, 0);
+ rx_ring->rx_stats.alloc_page_failed++;
+ return false;
+ }
+
+ bi->dma = dma;
+ bi->page = page;
+ bi->page_offset = 0;
+
+ return true;
+}
+
+/**
+ * ice_alloc_rx_bufs - Replace used receive buffers
+ * @rx_ring: ring to place buffers on
+ * @cleaned_count: number of buffers to replace
+ *
+ * Returns false if all allocations were successful, true if any fail
+ */
+bool ice_alloc_rx_bufs(struct ice_ring *rx_ring, u16 cleaned_count)
+{
+ union ice_32b_rx_flex_desc *rx_desc;
+ u16 ntu = rx_ring->next_to_use;
+ struct ice_rx_buf *bi;
+
+ /* do nothing if no valid netdev defined */
+ if (!rx_ring->netdev || !cleaned_count)
+ return false;
+
+ /* get the RX descriptor and buffer based on next_to_use */
+ rx_desc = ICE_RX_DESC(rx_ring, ntu);
+ bi = &rx_ring->rx_buf[ntu];
+
+ do {
+ if (!ice_alloc_mapped_page(rx_ring, bi))
+ goto no_bufs;
+
+ /* Refresh the desc even if buffer_addrs didn't change
+ * because each write-back erases this info.
+ */
+ rx_desc->read.pkt_addr = cpu_to_le64(bi->dma + bi->page_offset);
+
+ rx_desc++;
+ bi++;
+ ntu++;
+ if (unlikely(ntu == rx_ring->count)) {
+ rx_desc = ICE_RX_DESC(rx_ring, 0);
+ bi = rx_ring->rx_buf;
+ ntu = 0;
+ }
+
+ /* clear the status bits for the next_to_use descriptor */
+ rx_desc->wb.status_error0 = 0;
+
+ cleaned_count--;
+ } while (cleaned_count);
+
+ if (rx_ring->next_to_use != ntu)
+ ice_release_rx_desc(rx_ring, ntu);
+
+ return false;
+
+no_bufs:
+ if (rx_ring->next_to_use != ntu)
+ ice_release_rx_desc(rx_ring, ntu);
+
+ /* make sure to come back via polling to try again after
+ * allocation failure
+ */
+ return true;
+}
+
+/**
+ * ice_page_is_reserved - check if reuse is possible
+ * @page: page struct to check
+ */
+static bool ice_page_is_reserved(struct page *page)
+{
+ return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page);
+}
+
+/**
+ * ice_add_rx_frag - Add contents of Rx buffer to sk_buff
+ * @rx_buf: buffer containing page to add
+ * @rx_desc: descriptor containing length of buffer written by hardware
+ * @skb: sk_buf to place the data into
+ *
+ * This function will add the data contained in rx_buf->page to the skb.
+ * This is done either through a direct copy if the data in the buffer is
+ * less than the skb header size, otherwise it will just attach the page as
+ * a frag to the skb.
+ *
+ * The function will then update the page offset if necessary and return
+ * true if the buffer can be reused by the adapter.
+ */
+static bool ice_add_rx_frag(struct ice_rx_buf *rx_buf,
+ union ice_32b_rx_flex_desc *rx_desc,
+ struct sk_buff *skb)
+{
+#if (PAGE_SIZE < 8192)
+ unsigned int truesize = ICE_RXBUF_2048;
+#else
+ unsigned int last_offset = PAGE_SIZE - ICE_RXBUF_2048;
+ unsigned int truesize;
+#endif /* PAGE_SIZE < 8192) */
+
+ struct page *page;
+ unsigned int size;
+
+ size = le16_to_cpu(rx_desc->wb.pkt_len) &
+ ICE_RX_FLX_DESC_PKT_LEN_M;
+
+ page = rx_buf->page;
+
+#if (PAGE_SIZE >= 8192)
+ truesize = ALIGN(size, L1_CACHE_BYTES);
+#endif /* PAGE_SIZE >= 8192) */
+
+ /* will the data fit in the skb we allocated? if so, just
+ * copy it as it is pretty small anyway
+ */
+ if (size <= ICE_RX_HDR_SIZE && !skb_is_nonlinear(skb)) {
+ unsigned char *va = page_address(page) + rx_buf->page_offset;
+
+ memcpy(__skb_put(skb, size), va, ALIGN(size, sizeof(long)));
+
+ /* page is not reserved, we can reuse buffer as-is */
+ if (likely(!ice_page_is_reserved(page)))
+ return true;
+
+ /* this page cannot be reused so discard it */
+ __free_pages(page, 0);
+ return false;
+ }
+
+ skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
+ rx_buf->page_offset, size, truesize);
+
+ /* avoid re-using remote pages */
+ if (unlikely(ice_page_is_reserved(page)))
+ return false;
+
+#if (PAGE_SIZE < 8192)
+ /* if we are only owner of page we can reuse it */
+ if (unlikely(page_count(page) != 1))
+ return false;
+
+ /* flip page offset to other buffer */
+ rx_buf->page_offset ^= truesize;
+#else
+ /* move offset up to the next cache line */
+ rx_buf->page_offset += truesize;
+
+ if (rx_buf->page_offset > last_offset)
+ return false;
+#endif /* PAGE_SIZE < 8192) */
+
+ /* Even if we own the page, we are not allowed to use atomic_set()
+ * This would break get_page_unless_zero() users.
+ */
+ get_page(rx_buf->page);
+
+ return true;
+}
+
+/**
+ * ice_reuse_rx_page - page flip buffer and store it back on the ring
+ * @rx_ring: rx descriptor ring to store buffers on
+ * @old_buf: donor buffer to have page reused
+ *
+ * Synchronizes page for reuse by the adapter
+ */
+static void ice_reuse_rx_page(struct ice_ring *rx_ring,
+ struct ice_rx_buf *old_buf)
+{
+ u16 nta = rx_ring->next_to_alloc;
+ struct ice_rx_buf *new_buf;
+
+ new_buf = &rx_ring->rx_buf[nta];
+
+ /* update, and store next to alloc */
+ nta++;
+ rx_ring->next_to_alloc = (nta < rx_ring->count) ? nta : 0;
+
+ /* transfer page from old buffer to new buffer */
+ *new_buf = *old_buf;
+}
+
+/**
+ * ice_fetch_rx_buf - Allocate skb and populate it
+ * @rx_ring: rx descriptor ring to transact packets on
+ * @rx_desc: descriptor containing info written by hardware
+ *
+ * This function allocates an skb on the fly, and populates it with the page
+ * data from the current receive descriptor, taking care to set up the skb
+ * correctly, as well as handling calling the page recycle function if
+ * necessary.
+ */
+static struct sk_buff *ice_fetch_rx_buf(struct ice_ring *rx_ring,
+ union ice_32b_rx_flex_desc *rx_desc)
+{
+ struct ice_rx_buf *rx_buf;
+ struct sk_buff *skb;
+ struct page *page;
+
+ rx_buf = &rx_ring->rx_buf[rx_ring->next_to_clean];
+ page = rx_buf->page;
+ prefetchw(page);
+
+ skb = rx_buf->skb;
+
+ if (likely(!skb)) {
+ u8 *page_addr = page_address(page) + rx_buf->page_offset;
+
+ /* prefetch first cache line of first page */
+ prefetch(page_addr);
+#if L1_CACHE_BYTES < 128
+ prefetch((void *)(page_addr + L1_CACHE_BYTES));
+#endif /* L1_CACHE_BYTES */
+
+ /* allocate a skb to store the frags */
+ skb = __napi_alloc_skb(&rx_ring->q_vector->napi,
+ ICE_RX_HDR_SIZE,
+ GFP_ATOMIC | __GFP_NOWARN);
+ if (unlikely(!skb)) {
+ rx_ring->rx_stats.alloc_buf_failed++;
+ return NULL;
+ }
+
+ /* we will be copying header into skb->data in
+ * pskb_may_pull so it is in our interest to prefetch
+ * it now to avoid a possible cache miss
+ */
+ prefetchw(skb->data);
+
+ skb_record_rx_queue(skb, rx_ring->q_index);
+ } else {
+ /* we are reusing so sync this buffer for CPU use */
+ dma_sync_single_range_for_cpu(rx_ring->dev, rx_buf->dma,
+ rx_buf->page_offset,
+ ICE_RXBUF_2048,
+ DMA_FROM_DEVICE);
+
+ rx_buf->skb = NULL;
+ }
+
+ /* pull page into skb */
+ if (ice_add_rx_frag(rx_buf, rx_desc, skb)) {
+ /* hand second half of page back to the ring */
+ ice_reuse_rx_page(rx_ring, rx_buf);
+ rx_ring->rx_stats.page_reuse_count++;
+ } else {
+ /* we are not reusing the buffer so unmap it */
+ dma_unmap_page(rx_ring->dev, rx_buf->dma, PAGE_SIZE,
+ DMA_FROM_DEVICE);
+ }
+
+ /* clear contents of buffer_info */
+ rx_buf->page = NULL;
+
+ return skb;
+}
+
+/**
+ * ice_pull_tail - ice specific version of skb_pull_tail
+ * @skb: pointer to current skb being adjusted
+ *
+ * This function is an ice specific version of __pskb_pull_tail. The
+ * main difference between this version and the original function is that
+ * this function can make several assumptions about the state of things
+ * that allow for significant optimizations versus the standard function.
+ * As a result we can do things like drop a frag and maintain an accurate
+ * truesize for the skb.
+ */
+static void ice_pull_tail(struct sk_buff *skb)
+{
+ struct skb_frag_struct *frag = &skb_shinfo(skb)->frags[0];
+ unsigned int pull_len;
+ unsigned char *va;
+
+ /* it is valid to use page_address instead of kmap since we are
+ * working with pages allocated out of the lomem pool per
+ * alloc_page(GFP_ATOMIC)
+ */
+ va = skb_frag_address(frag);
+
+ /* we need the header to contain the greater of either ETH_HLEN or
+ * 60 bytes if the skb->len is less than 60 for skb_pad.
+ */
+ pull_len = eth_get_headlen(va, ICE_RX_HDR_SIZE);
+
+ /* align pull length to size of long to optimize memcpy performance */
+ skb_copy_to_linear_data(skb, va, ALIGN(pull_len, sizeof(long)));
+
+ /* update all of the pointers */
+ skb_frag_size_sub(frag, pull_len);
+ frag->page_offset += pull_len;
+ skb->data_len -= pull_len;
+ skb->tail += pull_len;
+}
+
+/**
+ * ice_cleanup_headers - Correct empty headers
+ * @skb: pointer to current skb being fixed
+ *
+ * Also address the case where we are pulling data in on pages only
+ * and as such no data is present in the skb header.
+ *
+ * In addition if skb is not at least 60 bytes we need to pad it so that
+ * it is large enough to qualify as a valid Ethernet frame.
+ *
+ * Returns true if an error was encountered and skb was freed.
+ */
+static bool ice_cleanup_headers(struct sk_buff *skb)
+{
+ /* place header in linear portion of buffer */
+ if (skb_is_nonlinear(skb))
+ ice_pull_tail(skb);
+
+ /* if eth_skb_pad returns an error the skb was freed */
+ if (eth_skb_pad(skb))
+ return true;
+
+ return false;
+}
+
+/**
+ * ice_test_staterr - tests bits in Rx descriptor status and error fields
+ * @rx_desc: pointer to receive descriptor (in le64 format)
+ * @stat_err_bits: value to mask
+ *
+ * This function does some fast chicanery in order to return the
+ * value of the mask which is really only used for boolean tests.
+ * The status_error_len doesn't need to be shifted because it begins
+ * at offset zero.
+ */
+static bool ice_test_staterr(union ice_32b_rx_flex_desc *rx_desc,
+ const u16 stat_err_bits)
+{
+ return !!(rx_desc->wb.status_error0 &
+ cpu_to_le16(stat_err_bits));
+}
+
+/**
+ * ice_is_non_eop - process handling of non-EOP buffers
+ * @rx_ring: Rx ring being processed
+ * @rx_desc: Rx descriptor for current buffer
+ * @skb: Current socket buffer containing buffer in progress
+ *
+ * This function updates next to clean. If the buffer is an EOP buffer
+ * this function exits returning false, otherwise it will place the
+ * sk_buff in the next buffer to be chained and return true indicating
+ * that this is in fact a non-EOP buffer.
+ */
+static bool ice_is_non_eop(struct ice_ring *rx_ring,
+ union ice_32b_rx_flex_desc *rx_desc,
+ struct sk_buff *skb)
+{
+ u32 ntc = rx_ring->next_to_clean + 1;
+
+ /* fetch, update, and store next to clean */
+ ntc = (ntc < rx_ring->count) ? ntc : 0;
+ rx_ring->next_to_clean = ntc;
+
+ prefetch(ICE_RX_DESC(rx_ring, ntc));
+
+ /* if we are the last buffer then there is nothing else to do */
+#define ICE_RXD_EOF BIT(ICE_RX_FLEX_DESC_STATUS0_EOF_S)
+ if (likely(ice_test_staterr(rx_desc, ICE_RXD_EOF)))
+ return false;
+
+ /* place skb in next buffer to be received */
+ rx_ring->rx_buf[ntc].skb = skb;
+ rx_ring->rx_stats.non_eop_descs++;
+
+ return true;
+}
+
+/**
+ * ice_ptype_to_htype - get a hash type
+ * @ptype: the ptype value from the descriptor
+ *
+ * Returns a hash type to be used by skb_set_hash
+ */
+static enum pkt_hash_types ice_ptype_to_htype(u8 __always_unused ptype)
+{
+ return PKT_HASH_TYPE_NONE;
+}
+
+/**
+ * ice_rx_hash - set the hash value in the skb
+ * @rx_ring: descriptor ring
+ * @rx_desc: specific descriptor
+ * @skb: pointer to current skb
+ * @rx_ptype: the ptype value from the descriptor
+ */
+static void
+ice_rx_hash(struct ice_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc,
+ struct sk_buff *skb, u8 rx_ptype)
+{
+ struct ice_32b_rx_flex_desc_nic *nic_mdid;
+ u32 hash;
+
+ if (!(rx_ring->netdev->features & NETIF_F_RXHASH))
+ return;
+
+ if (rx_desc->wb.rxdid != ICE_RXDID_FLEX_NIC)
+ return;
+
+ nic_mdid = (struct ice_32b_rx_flex_desc_nic *)rx_desc;
+ hash = le32_to_cpu(nic_mdid->rss_hash);
+ skb_set_hash(skb, hash, ice_ptype_to_htype(rx_ptype));
+}
+
+/**
+ * ice_rx_csum - Indicate in skb if checksum is good
+ * @vsi: the VSI we care about
+ * @skb: skb currently being received and modified
+ * @rx_desc: the receive descriptor
+ * @ptype: the packet type decoded by hardware
+ *
+ * skb->protocol must be set before this function is called
+ */
+static void ice_rx_csum(struct ice_vsi *vsi, struct sk_buff *skb,
+ union ice_32b_rx_flex_desc *rx_desc, u8 ptype)
+{
+ struct ice_rx_ptype_decoded decoded;
+ u32 rx_error, rx_status;
+ bool ipv4, ipv6;
+
+ rx_status = le16_to_cpu(rx_desc->wb.status_error0);
+ rx_error = rx_status;
+
+ decoded = ice_decode_rx_desc_ptype(ptype);
+
+ /* Start with CHECKSUM_NONE and by default csum_level = 0 */
+ skb->ip_summed = CHECKSUM_NONE;
+ skb_checksum_none_assert(skb);
+
+ /* check if Rx checksum is enabled */
+ if (!(vsi->netdev->features & NETIF_F_RXCSUM))
+ return;
+
+ /* check if HW has decoded the packet and checksum */
+ if (!(rx_status & BIT(ICE_RX_FLEX_DESC_STATUS0_L3L4P_S)))
+ return;
+
+ if (!(decoded.known && decoded.outer_ip))
+ return;
+
+ ipv4 = (decoded.outer_ip == ICE_RX_PTYPE_OUTER_IP) &&
+ (decoded.outer_ip_ver == ICE_RX_PTYPE_OUTER_IPV4);
+ ipv6 = (decoded.outer_ip == ICE_RX_PTYPE_OUTER_IP) &&
+ (decoded.outer_ip_ver == ICE_RX_PTYPE_OUTER_IPV6);
+
+ if (ipv4 && (rx_error & (BIT(ICE_RX_FLEX_DESC_STATUS0_XSUM_IPE_S) |
+ BIT(ICE_RX_FLEX_DESC_STATUS0_XSUM_EIPE_S))))
+ goto checksum_fail;
+ else if (ipv6 && (rx_status &
+ (BIT(ICE_RX_FLEX_DESC_STATUS0_IPV6EXADD_S))))
+ goto checksum_fail;
+
+ /* check for L4 errors and handle packets that were not able to be
+ * checksummed due to arrival speed
+ */
+ if (rx_error & BIT(ICE_RX_FLEX_DESC_STATUS0_XSUM_L4E_S))
+ goto checksum_fail;
+
+ /* Only report checksum unnecessary for TCP, UDP, or SCTP */
+ switch (decoded.inner_prot) {
+ case ICE_RX_PTYPE_INNER_PROT_TCP:
+ case ICE_RX_PTYPE_INNER_PROT_UDP:
+ case ICE_RX_PTYPE_INNER_PROT_SCTP:
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ default:
+ break;
+ }
+ return;
+
+checksum_fail:
+ vsi->back->hw_csum_rx_error++;
+}
+
+/**
+ * ice_process_skb_fields - Populate skb header fields from Rx descriptor
+ * @rx_ring: rx descriptor ring packet is being transacted on
+ * @rx_desc: pointer to the EOP Rx descriptor
+ * @skb: pointer to current skb being populated
+ * @ptype: the packet type decoded by hardware
+ *
+ * This function checks the ring, descriptor, and packet information in
+ * order to populate the hash, checksum, VLAN, protocol, and
+ * other fields within the skb.
+ */
+static void ice_process_skb_fields(struct ice_ring *rx_ring,
+ union ice_32b_rx_flex_desc *rx_desc,
+ struct sk_buff *skb, u8 ptype)
+{
+ ice_rx_hash(rx_ring, rx_desc, skb, ptype);
+
+ /* modifies the skb - consumes the enet header */
+ skb->protocol = eth_type_trans(skb, rx_ring->netdev);
+
+ ice_rx_csum(rx_ring->vsi, skb, rx_desc, ptype);
+}
+
+/**
+ * ice_receive_skb - Send a completed packet up the stack
+ * @rx_ring: rx ring in play
+ * @skb: packet to send up
+ * @vlan_tag: vlan tag for packet
+ *
+ * This function sends the completed packet (via. skb) up the stack using
+ * gro receive functions (with/without vlan tag)
+ */
+static void ice_receive_skb(struct ice_ring *rx_ring, struct sk_buff *skb,
+ u16 vlan_tag)
+{
+ if ((rx_ring->netdev->features & NETIF_F_HW_VLAN_CTAG_RX) &&
+ (vlan_tag & VLAN_VID_MASK)) {
+ __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q), vlan_tag);
+ }
+ napi_gro_receive(&rx_ring->q_vector->napi, skb);
+}
+
+/**
+ * ice_clean_rx_irq - Clean completed descriptors from Rx ring - bounce buf
+ * @rx_ring: rx descriptor ring to transact packets on
+ * @budget: Total limit on number of packets to process
+ *
+ * This function provides a "bounce buffer" approach to Rx interrupt
+ * processing. The advantage to this is that on systems that have
+ * expensive overhead for IOMMU access this provides a means of avoiding
+ * it by maintaining the mapping of the page to the system.
+ *
+ * Returns amount of work completed
+ */
+static int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
+{
+ unsigned int total_rx_bytes = 0, total_rx_pkts = 0;
+ u16 cleaned_count = ICE_DESC_UNUSED(rx_ring);
+ bool failure = false;
+
+ /* start the loop to process RX packets bounded by 'budget' */
+ while (likely(total_rx_pkts < (unsigned int)budget)) {
+ union ice_32b_rx_flex_desc *rx_desc;
+ struct sk_buff *skb;
+ u16 stat_err_bits;
+ u16 vlan_tag = 0;
+ u8 rx_ptype;
+
+ /* return some buffers to hardware, one at a time is too slow */
+ if (cleaned_count >= ICE_RX_BUF_WRITE) {
+ failure = failure ||
+ ice_alloc_rx_bufs(rx_ring, cleaned_count);
+ cleaned_count = 0;
+ }
+
+ /* get the RX desc from RX ring based on 'next_to_clean' */
+ rx_desc = ICE_RX_DESC(rx_ring, rx_ring->next_to_clean);
+
+ /* status_error_len will always be zero for unused descriptors
+ * because it's cleared in cleanup, and overlaps with hdr_addr
+ * which is always zero because packet split isn't used, if the
+ * hardware wrote DD then it will be non-zero
+ */
+ stat_err_bits = BIT(ICE_RX_FLEX_DESC_STATUS0_DD_S);
+ if (!ice_test_staterr(rx_desc, stat_err_bits))
+ break;
+
+ /* This memory barrier is needed to keep us from reading
+ * any other fields out of the rx_desc until we know the
+ * DD bit is set.
+ */
+ dma_rmb();
+
+ /* allocate (if needed) and populate skb */
+ skb = ice_fetch_rx_buf(rx_ring, rx_desc);
+ if (!skb)
+ break;
+
+ cleaned_count++;
+
+ /* skip if it is NOP desc */
+ if (ice_is_non_eop(rx_ring, rx_desc, skb))
+ continue;
+
+ stat_err_bits = BIT(ICE_RX_FLEX_DESC_STATUS0_RXE_S);
+ if (unlikely(ice_test_staterr(rx_desc, stat_err_bits))) {
+ dev_kfree_skb_any(skb);
+ continue;
+ }
+
+ rx_ptype = le16_to_cpu(rx_desc->wb.ptype_flex_flags0) &
+ ICE_RX_FLEX_DESC_PTYPE_M;
+
+ stat_err_bits = BIT(ICE_RX_FLEX_DESC_STATUS0_L2TAG1P_S);
+ if (ice_test_staterr(rx_desc, stat_err_bits))
+ vlan_tag = le16_to_cpu(rx_desc->wb.l2tag1);
+
+ /* correct empty headers and pad skb if needed (to make valid
+ * ethernet frame
+ */
+ if (ice_cleanup_headers(skb)) {
+ skb = NULL;
+ continue;
+ }
+
+ /* probably a little skewed due to removing CRC */
+ total_rx_bytes += skb->len;
+
+ /* populate checksum, VLAN, and protocol */
+ ice_process_skb_fields(rx_ring, rx_desc, skb, rx_ptype);
+
+ /* send completed skb up the stack */
+ ice_receive_skb(rx_ring, skb, vlan_tag);
+
+ /* update budget accounting */
+ total_rx_pkts++;
+ }
+
+ /* update queue and vector specific stats */
+ u64_stats_update_begin(&rx_ring->syncp);
+ rx_ring->stats.pkts += total_rx_pkts;
+ rx_ring->stats.bytes += total_rx_bytes;
+ u64_stats_update_end(&rx_ring->syncp);
+ rx_ring->q_vector->rx.total_pkts += total_rx_pkts;
+ rx_ring->q_vector->rx.total_bytes += total_rx_bytes;
+
+ /* guarantee a trip back through this routine if there was a failure */
+ return failure ? budget : (int)total_rx_pkts;
+}
+
+/**
+ * ice_napi_poll - NAPI polling Rx/Tx cleanup routine
+ * @napi: napi struct with our devices info in it
+ * @budget: amount of work driver is allowed to do this pass, in packets
+ *
+ * This function will clean all queues associated with a q_vector.
+ *
+ * Returns the amount of work done
+ */
+int ice_napi_poll(struct napi_struct *napi, int budget)
+{
+ struct ice_q_vector *q_vector =
+ container_of(napi, struct ice_q_vector, napi);
+ struct ice_vsi *vsi = q_vector->vsi;
+ struct ice_pf *pf = vsi->back;
+ bool clean_complete = true;
+ int budget_per_ring = 0;
+ struct ice_ring *ring;
+ int work_done = 0;
+
+ /* Since the actual Tx work is minimal, we can give the Tx a larger
+ * budget and be more aggressive about cleaning up the Tx descriptors.
+ */
+ ice_for_each_ring(ring, q_vector->tx)
+ if (!ice_clean_tx_irq(vsi, ring, budget))
+ clean_complete = false;
+
+ /* Handle case where we are called by netpoll with a budget of 0 */
+ if (budget <= 0)
+ return budget;
+
+ /* We attempt to distribute budget to each Rx queue fairly, but don't
+ * allow the budget to go below 1 because that would exit polling early.
+ */
+ if (q_vector->num_ring_rx)
+ budget_per_ring = max(budget / q_vector->num_ring_rx, 1);
+
+ ice_for_each_ring(ring, q_vector->rx) {
+ int cleaned;
+
+ cleaned = ice_clean_rx_irq(ring, budget_per_ring);
+ work_done += cleaned;
+ /* if we clean as many as budgeted, we must not be done */
+ if (cleaned >= budget_per_ring)
+ clean_complete = false;
+ }
+
+ /* If work not completed, return budget and polling will return */
+ if (!clean_complete)
+ return budget;
+
+ /* Work is done so exit the polling mode and re-enable the interrupt */
+ napi_complete_done(napi, work_done);
+ if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
+ ice_irq_dynamic_ena(&vsi->back->hw, vsi, q_vector);
+ return 0;
+}
+
+/* helper function for building cmd/type/offset */
+static __le64
+build_ctob(u64 td_cmd, u64 td_offset, unsigned int size, u64 td_tag)
+{
+ return cpu_to_le64(ICE_TX_DESC_DTYPE_DATA |
+ (td_cmd << ICE_TXD_QW1_CMD_S) |
+ (td_offset << ICE_TXD_QW1_OFFSET_S) |
+ ((u64)size << ICE_TXD_QW1_TX_BUF_SZ_S) |
+ (td_tag << ICE_TXD_QW1_L2TAG1_S));
+}
+
+/**
+ * __ice_maybe_stop_tx - 2nd level check for tx stop conditions
+ * @tx_ring: the ring to be checked
+ * @size: the size buffer we want to assure is available
+ *
+ * Returns -EBUSY if a stop is needed, else 0
+ */
+static int __ice_maybe_stop_tx(struct ice_ring *tx_ring, unsigned int size)
+{
+ netif_stop_subqueue(tx_ring->netdev, tx_ring->q_index);
+ /* Memory barrier before checking head and tail */
+ smp_mb();
+
+ /* Check again in a case another CPU has just made room available. */
+ if (likely(ICE_DESC_UNUSED(tx_ring) < size))
+ return -EBUSY;
+
+ /* A reprieve! - use start_subqueue because it doesn't call schedule */
+ netif_start_subqueue(tx_ring->netdev, tx_ring->q_index);
+ ++tx_ring->tx_stats.restart_q;
+ return 0;
+}
+
+/**
+ * ice_maybe_stop_tx - 1st level check for tx stop conditions
+ * @tx_ring: the ring to be checked
+ * @size: the size buffer we want to assure is available
+ *
+ * Returns 0 if stop is not needed
+ */
+static int ice_maybe_stop_tx(struct ice_ring *tx_ring, unsigned int size)
+{
+ if (likely(ICE_DESC_UNUSED(tx_ring) >= size))
+ return 0;
+ return __ice_maybe_stop_tx(tx_ring, size);
+}
+
+/**
+ * ice_tx_map - Build the Tx descriptor
+ * @tx_ring: ring to send buffer on
+ * @first: first buffer info buffer to use
+ * @off: pointer to struct that holds offload parameters
+ *
+ * This function loops over the skb data pointed to by *first
+ * and gets a physical address for each memory location and programs
+ * it and the length into the transmit descriptor.
+ */
+static void
+ice_tx_map(struct ice_ring *tx_ring, struct ice_tx_buf *first,
+ struct ice_tx_offload_params *off)
+{
+ u64 td_offset, td_tag, td_cmd;
+ u16 i = tx_ring->next_to_use;
+ struct skb_frag_struct *frag;
+ unsigned int data_len, size;
+ struct ice_tx_desc *tx_desc;
+ struct ice_tx_buf *tx_buf;
+ struct sk_buff *skb;
+ dma_addr_t dma;
+
+ td_tag = off->td_l2tag1;
+ td_cmd = off->td_cmd;
+ td_offset = off->td_offset;
+ skb = first->skb;
+
+ data_len = skb->data_len;
+ size = skb_headlen(skb);
+
+ tx_desc = ICE_TX_DESC(tx_ring, i);
+
+ if (first->tx_flags & ICE_TX_FLAGS_HW_VLAN) {
+ td_cmd |= (u64)ICE_TX_DESC_CMD_IL2TAG1;
+ td_tag = (first->tx_flags & ICE_TX_FLAGS_VLAN_M) >>
+ ICE_TX_FLAGS_VLAN_S;
+ }
+
+ dma = dma_map_single(tx_ring->dev, skb->data, size, DMA_TO_DEVICE);
+
+ tx_buf = first;
+
+ for (frag = &skb_shinfo(skb)->frags[0];; frag++) {
+ unsigned int max_data = ICE_MAX_DATA_PER_TXD_ALIGNED;
+
+ if (dma_mapping_error(tx_ring->dev, dma))
+ goto dma_error;
+
+ /* record length, and DMA address */
+ dma_unmap_len_set(tx_buf, len, size);
+ dma_unmap_addr_set(tx_buf, dma, dma);
+
+ /* align size to end of page */
+ max_data += -dma & (ICE_MAX_READ_REQ_SIZE - 1);
+ tx_desc->buf_addr = cpu_to_le64(dma);
+
+ /* account for data chunks larger than the hardware
+ * can handle
+ */
+ while (unlikely(size > ICE_MAX_DATA_PER_TXD)) {
+ tx_desc->cmd_type_offset_bsz =
+ build_ctob(td_cmd, td_offset, max_data, td_tag);
+
+ tx_desc++;
+ i++;
+
+ if (i == tx_ring->count) {
+ tx_desc = ICE_TX_DESC(tx_ring, 0);
+ i = 0;
+ }
+
+ dma += max_data;
+ size -= max_data;
+
+ max_data = ICE_MAX_DATA_PER_TXD_ALIGNED;
+ tx_desc->buf_addr = cpu_to_le64(dma);
+ }
+
+ if (likely(!data_len))
+ break;
+
+ tx_desc->cmd_type_offset_bsz = build_ctob(td_cmd, td_offset,
+ size, td_tag);
+
+ tx_desc++;
+ i++;
+
+ if (i == tx_ring->count) {
+ tx_desc = ICE_TX_DESC(tx_ring, 0);
+ i = 0;
+ }
+
+ size = skb_frag_size(frag);
+ data_len -= size;
+
+ dma = skb_frag_dma_map(tx_ring->dev, frag, 0, size,
+ DMA_TO_DEVICE);
+
+ tx_buf = &tx_ring->tx_buf[i];
+ }
+
+ /* record bytecount for BQL */
+ netdev_tx_sent_queue(txring_txq(tx_ring), first->bytecount);
+
+ /* record SW timestamp if HW timestamp is not available */
+ skb_tx_timestamp(first->skb);
+
+ i++;
+ if (i == tx_ring->count)
+ i = 0;
+
+ /* write last descriptor with RS and EOP bits */
+ td_cmd |= (u64)(ICE_TX_DESC_CMD_EOP | ICE_TX_DESC_CMD_RS);
+ tx_desc->cmd_type_offset_bsz =
+ build_ctob(td_cmd, td_offset, size, td_tag);
+
+ /* Force memory writes to complete before letting h/w know there
+ * are new descriptors to fetch.
+ *
+ * We also use this memory barrier to make certain all of the
+ * status bits have been updated before next_to_watch is written.
+ */
+ wmb();
+
+ /* set next_to_watch value indicating a packet is present */
+ first->next_to_watch = tx_desc;
+
+ tx_ring->next_to_use = i;
+
+ ice_maybe_stop_tx(tx_ring, DESC_NEEDED);
+
+ /* notify HW of packet */
+ if (netif_xmit_stopped(txring_txq(tx_ring)) || !skb->xmit_more) {
+ writel(i, tx_ring->tail);
+
+ /* we need this if more than one processor can write to our tail
+ * at a time, it synchronizes IO on IA64/Altix systems
+ */
+ mmiowb();
+ }
+
+ return;
+
+dma_error:
+ /* clear dma mappings for failed tx_buf map */
+ for (;;) {
+ tx_buf = &tx_ring->tx_buf[i];
+ ice_unmap_and_free_tx_buf(tx_ring, tx_buf);
+ if (tx_buf == first)
+ break;
+ if (i == 0)
+ i = tx_ring->count;
+ i--;
+ }
+
+ tx_ring->next_to_use = i;
+}
+
+/**
+ * ice_tx_csum - Enable Tx checksum offloads
+ * @first: pointer to the first descriptor
+ * @off: pointer to struct that holds offload parameters
+ *
+ * Returns 0 or error (negative) if checksum offload can't happen, 1 otherwise.
+ */
+static
+int ice_tx_csum(struct ice_tx_buf *first, struct ice_tx_offload_params *off)
+{
+ u32 l4_len = 0, l3_len = 0, l2_len = 0;
+ struct sk_buff *skb = first->skb;
+ union {
+ struct iphdr *v4;
+ struct ipv6hdr *v6;
+ unsigned char *hdr;
+ } ip;
+ union {
+ struct tcphdr *tcp;
+ unsigned char *hdr;
+ } l4;
+ __be16 frag_off, protocol;
+ unsigned char *exthdr;
+ u32 offset, cmd = 0;
+ u8 l4_proto = 0;
+
+ if (skb->ip_summed != CHECKSUM_PARTIAL)
+ return 0;
+
+ ip.hdr = skb_network_header(skb);
+ l4.hdr = skb_transport_header(skb);
+
+ /* compute outer L2 header size */
+ l2_len = ip.hdr - skb->data;
+ offset = (l2_len / 2) << ICE_TX_DESC_LEN_MACLEN_S;
+
+ if (skb->encapsulation)
+ return -1;
+
+ /* Enable IP checksum offloads */
+ protocol = vlan_get_protocol(skb);
+ if (protocol == htons(ETH_P_IP)) {
+ l4_proto = ip.v4->protocol;
+ /* the stack computes the IP header already, the only time we
+ * need the hardware to recompute it is in the case of TSO.
+ */
+ if (first->tx_flags & ICE_TX_FLAGS_TSO)
+ cmd |= ICE_TX_DESC_CMD_IIPT_IPV4_CSUM;
+ else
+ cmd |= ICE_TX_DESC_CMD_IIPT_IPV4;
+
+ } else if (protocol == htons(ETH_P_IPV6)) {
+ cmd |= ICE_TX_DESC_CMD_IIPT_IPV6;
+ exthdr = ip.hdr + sizeof(*ip.v6);
+ l4_proto = ip.v6->nexthdr;
+ if (l4.hdr != exthdr)
+ ipv6_skip_exthdr(skb, exthdr - skb->data, &l4_proto,
+ &frag_off);
+ } else {
+ return -1;
+ }
+
+ /* compute inner L3 header size */
+ l3_len = l4.hdr - ip.hdr;
+ offset |= (l3_len / 4) << ICE_TX_DESC_LEN_IPLEN_S;
+
+ /* Enable L4 checksum offloads */
+ switch (l4_proto) {
+ case IPPROTO_TCP:
+ /* enable checksum offloads */
+ cmd |= ICE_TX_DESC_CMD_L4T_EOFT_TCP;
+ l4_len = l4.tcp->doff;
+ offset |= l4_len << ICE_TX_DESC_LEN_L4_LEN_S;
+ break;
+ case IPPROTO_UDP:
+ /* enable UDP checksum offload */
+ cmd |= ICE_TX_DESC_CMD_L4T_EOFT_UDP;
+ l4_len = (sizeof(struct udphdr) >> 2);
+ offset |= l4_len << ICE_TX_DESC_LEN_L4_LEN_S;
+ break;
+ case IPPROTO_SCTP:
+ default:
+ if (first->tx_flags & ICE_TX_FLAGS_TSO)
+ return -1;
+ skb_checksum_help(skb);
+ return 0;
+ }
+
+ off->td_cmd |= cmd;
+ off->td_offset |= offset;
+ return 1;
+}
+
+/**
+ * ice_tx_prepare_vlan_flags - prepare generic TX VLAN tagging flags for HW
+ * @tx_ring: ring to send buffer on
+ * @first: pointer to struct ice_tx_buf
+ *
+ * Checks the skb and set up correspondingly several generic transmit flags
+ * related to VLAN tagging for the HW, such as VLAN, DCB, etc.
+ *
+ * Returns error code indicate the frame should be dropped upon error and the
+ * otherwise returns 0 to indicate the flags has been set properly.
+ */
+static int
+ice_tx_prepare_vlan_flags(struct ice_ring *tx_ring, struct ice_tx_buf *first)
+{
+ struct sk_buff *skb = first->skb;
+ __be16 protocol = skb->protocol;
+
+ if (protocol == htons(ETH_P_8021Q) &&
+ !(tx_ring->netdev->features & NETIF_F_HW_VLAN_CTAG_TX)) {
+ /* when HW VLAN acceleration is turned off by the user the
+ * stack sets the protocol to 8021q so that the driver
+ * can take any steps required to support the SW only
+ * VLAN handling. In our case the driver doesn't need
+ * to take any further steps so just set the protocol
+ * to the encapsulated ethertype.
+ */
+ skb->protocol = vlan_get_protocol(skb);
+ goto out;
+ }
+
+ /* if we have a HW VLAN tag being added, default to the HW one */
+ if (skb_vlan_tag_present(skb)) {
+ first->tx_flags |= skb_vlan_tag_get(skb) << ICE_TX_FLAGS_VLAN_S;
+ first->tx_flags |= ICE_TX_FLAGS_HW_VLAN;
+ } else if (protocol == htons(ETH_P_8021Q)) {
+ struct vlan_hdr *vhdr, _vhdr;
+
+ /* for SW VLAN, check the next protocol and store the tag */
+ vhdr = (struct vlan_hdr *)skb_header_pointer(skb, ETH_HLEN,
+ sizeof(_vhdr),
+ &_vhdr);
+ if (!vhdr)
+ return -EINVAL;
+
+ first->tx_flags |= ntohs(vhdr->h_vlan_TCI) <<
+ ICE_TX_FLAGS_VLAN_S;
+ first->tx_flags |= ICE_TX_FLAGS_SW_VLAN;
+ }
+
+out:
+ return 0;
+}
+
+/**
+ * ice_tso - computes mss and TSO length to prepare for TSO
+ * @first: pointer to struct ice_tx_buf
+ * @off: pointer to struct that holds offload parameters
+ *
+ * Returns 0 or error (negative) if TSO can't happen, 1 otherwise.
+ */
+static
+int ice_tso(struct ice_tx_buf *first, struct ice_tx_offload_params *off)
+{
+ struct sk_buff *skb = first->skb;
+ union {
+ struct iphdr *v4;
+ struct ipv6hdr *v6;
+ unsigned char *hdr;
+ } ip;
+ union {
+ struct tcphdr *tcp;
+ unsigned char *hdr;
+ } l4;
+ u64 cd_mss, cd_tso_len;
+ u32 paylen, l4_start;
+ int err;
+
+ if (skb->ip_summed != CHECKSUM_PARTIAL)
+ return 0;
+
+ if (!skb_is_gso(skb))
+ return 0;
+
+ err = skb_cow_head(skb, 0);
+ if (err < 0)
+ return err;
+
+ ip.hdr = skb_network_header(skb);
+ l4.hdr = skb_transport_header(skb);
+
+ /* initialize outer IP header fields */
+ if (ip.v4->version == 4) {
+ ip.v4->tot_len = 0;
+ ip.v4->check = 0;
+ } else {
+ ip.v6->payload_len = 0;
+ }
+
+ /* determine offset of transport header */
+ l4_start = l4.hdr - skb->data;
+
+ /* remove payload length from checksum */
+ paylen = skb->len - l4_start;
+ csum_replace_by_diff(&l4.tcp->check, (__force __wsum)htonl(paylen));
+
+ /* compute length of segmentation header */
+ off->header_len = (l4.tcp->doff * 4) + l4_start;
+
+ /* update gso_segs and bytecount */
+ first->gso_segs = skb_shinfo(skb)->gso_segs;
+ first->bytecount = (first->gso_segs - 1) * off->header_len;
+
+ cd_tso_len = skb->len - off->header_len;
+ cd_mss = skb_shinfo(skb)->gso_size;
+
+ /* record cdesc_qw1 with TSO parameters */
+ off->cd_qw1 |= ICE_TX_DESC_DTYPE_CTX |
+ (ICE_TX_CTX_DESC_TSO << ICE_TXD_CTX_QW1_CMD_S) |
+ (cd_tso_len << ICE_TXD_CTX_QW1_TSO_LEN_S) |
+ (cd_mss << ICE_TXD_CTX_QW1_MSS_S);
+ first->tx_flags |= ICE_TX_FLAGS_TSO;
+ return 1;
+}
+
+/**
+ * ice_txd_use_count - estimate the number of descriptors needed for Tx
+ * @size: transmit request size in bytes
+ *
+ * Due to hardware alignment restrictions (4K alignment), we need to
+ * assume that we can have no more than 12K of data per descriptor, even
+ * though each descriptor can take up to 16K - 1 bytes of aligned memory.
+ * Thus, we need to divide by 12K. But division is slow! Instead,
+ * we decompose the operation into shifts and one relatively cheap
+ * multiply operation.
+ *
+ * To divide by 12K, we first divide by 4K, then divide by 3:
+ * To divide by 4K, shift right by 12 bits
+ * To divide by 3, multiply by 85, then divide by 256
+ * (Divide by 256 is done by shifting right by 8 bits)
+ * Finally, we add one to round up. Because 256 isn't an exact multiple of
+ * 3, we'll underestimate near each multiple of 12K. This is actually more
+ * accurate as we have 4K - 1 of wiggle room that we can fit into the last
+ * segment. For our purposes this is accurate out to 1M which is orders of
+ * magnitude greater than our largest possible GSO size.
+ *
+ * This would then be implemented as:
+ * return (((size >> 12) * 85) >> 8) + 1;
+ *
+ * Since multiplication and division are commutative, we can reorder
+ * operations into:
+ * return ((size * 85) >> 20) + 1;
+ */
+static unsigned int ice_txd_use_count(unsigned int size)
+{
+ return ((size * 85) >> 20) + 1;
+}
+
+/**
+ * ice_xmit_desc_count - calculate number of tx descriptors needed
+ * @skb: send buffer
+ *
+ * Returns number of data descriptors needed for this skb.
+ */
+static unsigned int ice_xmit_desc_count(struct sk_buff *skb)
+{
+ const struct skb_frag_struct *frag = &skb_shinfo(skb)->frags[0];
+ unsigned int nr_frags = skb_shinfo(skb)->nr_frags;
+ unsigned int count = 0, size = skb_headlen(skb);
+
+ for (;;) {
+ count += ice_txd_use_count(size);
+
+ if (!nr_frags--)
+ break;
+
+ size = skb_frag_size(frag++);
+ }
+
+ return count;
+}
+
+/**
+ * __ice_chk_linearize - Check if there are more than 8 buffers per packet
+ * @skb: send buffer
+ *
+ * Note: This HW can't DMA more than 8 buffers to build a packet on the wire
+ * and so we need to figure out the cases where we need to linearize the skb.
+ *
+ * For TSO we need to count the TSO header and segment payload separately.
+ * As such we need to check cases where we have 7 fragments or more as we
+ * can potentially require 9 DMA transactions, 1 for the TSO header, 1 for
+ * the segment payload in the first descriptor, and another 7 for the
+ * fragments.
+ */
+static bool __ice_chk_linearize(struct sk_buff *skb)
+{
+ const struct skb_frag_struct *frag, *stale;
+ int nr_frags, sum;
+
+ /* no need to check if number of frags is less than 7 */
+ nr_frags = skb_shinfo(skb)->nr_frags;
+ if (nr_frags < (ICE_MAX_BUF_TXD - 1))
+ return false;
+
+ /* We need to walk through the list and validate that each group
+ * of 6 fragments totals at least gso_size.
+ */
+ nr_frags -= ICE_MAX_BUF_TXD - 2;
+ frag = &skb_shinfo(skb)->frags[0];
+
+ /* Initialize size to the negative value of gso_size minus 1. We
+ * use this as the worst case scenerio in which the frag ahead
+ * of us only provides one byte which is why we are limited to 6
+ * descriptors for a single transmit as the header and previous
+ * fragment are already consuming 2 descriptors.
+ */
+ sum = 1 - skb_shinfo(skb)->gso_size;
+
+ /* Add size of frags 0 through 4 to create our initial sum */
+ sum += skb_frag_size(frag++);
+ sum += skb_frag_size(frag++);
+ sum += skb_frag_size(frag++);
+ sum += skb_frag_size(frag++);
+ sum += skb_frag_size(frag++);
+
+ /* Walk through fragments adding latest fragment, testing it, and
+ * then removing stale fragments from the sum.
+ */
+ stale = &skb_shinfo(skb)->frags[0];
+ for (;;) {
+ sum += skb_frag_size(frag++);
+
+ /* if sum is negative we failed to make sufficient progress */
+ if (sum < 0)
+ return true;
+
+ if (!nr_frags--)
+ break;
+
+ sum -= skb_frag_size(stale++);
+ }
+
+ return false;
+}
+
+/**
+ * ice_chk_linearize - Check if there are more than 8 fragments per packet
+ * @skb: send buffer
+ * @count: number of buffers used
+ *
+ * Note: Our HW can't scatter-gather more than 8 fragments to build
+ * a packet on the wire and so we need to figure out the cases where we
+ * need to linearize the skb.
+ */
+static bool ice_chk_linearize(struct sk_buff *skb, unsigned int count)
+{
+ /* Both TSO and single send will work if count is less than 8 */
+ if (likely(count < ICE_MAX_BUF_TXD))
+ return false;
+
+ if (skb_is_gso(skb))
+ return __ice_chk_linearize(skb);
+
+ /* we can support up to 8 data buffers for a single send */
+ return count != ICE_MAX_BUF_TXD;
+}
+
+/**
+ * ice_xmit_frame_ring - Sends buffer on Tx ring
+ * @skb: send buffer
+ * @tx_ring: ring to send buffer on
+ *
+ * Returns NETDEV_TX_OK if sent, else an error code
+ */
+static netdev_tx_t
+ice_xmit_frame_ring(struct sk_buff *skb, struct ice_ring *tx_ring)
+{
+ struct ice_tx_offload_params offload = { 0 };
+ struct ice_tx_buf *first;
+ unsigned int count;
+ int tso, csum;
+
+ count = ice_xmit_desc_count(skb);
+ if (ice_chk_linearize(skb, count)) {
+ if (__skb_linearize(skb))
+ goto out_drop;
+ count = ice_txd_use_count(skb->len);
+ tx_ring->tx_stats.tx_linearize++;
+ }
+
+ /* need: 1 descriptor per page * PAGE_SIZE/ICE_MAX_DATA_PER_TXD,
+ * + 1 desc for skb_head_len/ICE_MAX_DATA_PER_TXD,
+ * + 4 desc gap to avoid the cache line where head is,
+ * + 1 desc for context descriptor,
+ * otherwise try next time
+ */
+ if (ice_maybe_stop_tx(tx_ring, count + 4 + 1)) {
+ tx_ring->tx_stats.tx_busy++;
+ return NETDEV_TX_BUSY;
+ }
+
+ offload.tx_ring = tx_ring;
+
+ /* record the location of the first descriptor for this packet */
+ first = &tx_ring->tx_buf[tx_ring->next_to_use];
+ first->skb = skb;
+ first->bytecount = max_t(unsigned int, skb->len, ETH_ZLEN);
+ first->gso_segs = 1;
+ first->tx_flags = 0;
+
+ /* prepare the VLAN tagging flags for Tx */
+ if (ice_tx_prepare_vlan_flags(tx_ring, first))
+ goto out_drop;
+
+ /* set up TSO offload */
+ tso = ice_tso(first, &offload);
+ if (tso < 0)
+ goto out_drop;
+
+ /* always set up Tx checksum offload */
+ csum = ice_tx_csum(first, &offload);
+ if (csum < 0)
+ goto out_drop;
+
+ if (tso || offload.cd_tunnel_params) {
+ struct ice_tx_ctx_desc *cdesc;
+ int i = tx_ring->next_to_use;
+
+ /* grab the next descriptor */
+ cdesc = ICE_TX_CTX_DESC(tx_ring, i);
+ i++;
+ tx_ring->next_to_use = (i < tx_ring->count) ? i : 0;
+
+ /* setup context descriptor */
+ cdesc->tunneling_params = cpu_to_le32(offload.cd_tunnel_params);
+ cdesc->l2tag2 = cpu_to_le16(offload.cd_l2tag2);
+ cdesc->rsvd = cpu_to_le16(0);
+ cdesc->qw1 = cpu_to_le64(offload.cd_qw1);
+ }
+
+ ice_tx_map(tx_ring, first, &offload);
+ return NETDEV_TX_OK;
+
+out_drop:
+ dev_kfree_skb_any(skb);
+ return NETDEV_TX_OK;
+}
+
+/**
+ * ice_start_xmit - Selects the correct VSI and Tx queue to send buffer
+ * @skb: send buffer
+ * @netdev: network interface device structure
+ *
+ * Returns NETDEV_TX_OK if sent, else an error code
+ */
+netdev_tx_t ice_start_xmit(struct sk_buff *skb, struct net_device *netdev)
+{
+ struct ice_netdev_priv *np = netdev_priv(netdev);
+ struct ice_vsi *vsi = np->vsi;
+ struct ice_ring *tx_ring;
+
+ tx_ring = vsi->tx_rings[skb->queue_mapping];
+
+ /* hardware can't handle really short frames, hardware padding works
+ * beyond this point
+ */
+ if (skb_put_padto(skb, ICE_MIN_TX_LEN))
+ return NETDEV_TX_OK;
+
+ return ice_xmit_frame_ring(skb, tx_ring);
+}
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_TXRX_H_
+#define _ICE_TXRX_H_
+
+#define ICE_DFLT_IRQ_WORK 256
+#define ICE_RXBUF_2048 2048
+#define ICE_MAX_CHAINED_RX_BUFS 5
+#define ICE_MAX_BUF_TXD 8
+#define ICE_MIN_TX_LEN 17
+
+/* The size limit for a transmit buffer in a descriptor is (16K - 1).
+ * In order to align with the read requests we will align the value to
+ * the nearest 4K which represents our maximum read request size.
+ */
+#define ICE_MAX_READ_REQ_SIZE 4096
+#define ICE_MAX_DATA_PER_TXD (16 * 1024 - 1)
+#define ICE_MAX_DATA_PER_TXD_ALIGNED \
+ (~(ICE_MAX_READ_REQ_SIZE - 1) & ICE_MAX_DATA_PER_TXD)
+
+#define ICE_RX_BUF_WRITE 16 /* Must be power of 2 */
+#define ICE_MAX_TXQ_PER_TXQG 128
+
+/* Tx Descriptors needed, worst case */
+#define DESC_NEEDED (MAX_SKB_FRAGS + 4)
+#define ICE_DESC_UNUSED(R) \
+ ((((R)->next_to_clean > (R)->next_to_use) ? 0 : (R)->count) + \
+ (R)->next_to_clean - (R)->next_to_use - 1)
+
+#define ICE_TX_FLAGS_TSO BIT(0)
+#define ICE_TX_FLAGS_HW_VLAN BIT(1)
+#define ICE_TX_FLAGS_SW_VLAN BIT(2)
+#define ICE_TX_FLAGS_VLAN_M 0xffff0000
+#define ICE_TX_FLAGS_VLAN_S 16
+
+struct ice_tx_buf {
+ struct ice_tx_desc *next_to_watch;
+ struct sk_buff *skb;
+ unsigned int bytecount;
+ unsigned short gso_segs;
+ u32 tx_flags;
+ DEFINE_DMA_UNMAP_ADDR(dma);
+ DEFINE_DMA_UNMAP_LEN(len);
+};
+
+struct ice_tx_offload_params {
+ u8 header_len;
+ u32 td_cmd;
+ u32 td_offset;
+ u32 td_l2tag1;
+ u16 cd_l2tag2;
+ u32 cd_tunnel_params;
+ u64 cd_qw1;
+ struct ice_ring *tx_ring;
+};
+
+struct ice_rx_buf {
+ struct sk_buff *skb;
+ dma_addr_t dma;
+ struct page *page;
+ unsigned int page_offset;
+};
+
+struct ice_q_stats {
+ u64 pkts;
+ u64 bytes;
+};
+
+struct ice_txq_stats {
+ u64 restart_q;
+ u64 tx_busy;
+ u64 tx_linearize;
+};
+
+struct ice_rxq_stats {
+ u64 non_eop_descs;
+ u64 alloc_page_failed;
+ u64 alloc_buf_failed;
+ u64 page_reuse_count;
+};
+
+/* this enum matches hardware bits and is meant to be used by DYN_CTLN
+ * registers and QINT registers or more generally anywhere in the manual
+ * mentioning ITR_INDX, ITR_NONE cannot be used as an index 'n' into any
+ * register but instead is a special value meaning "don't update" ITR0/1/2.
+ */
+enum ice_dyn_idx_t {
+ ICE_IDX_ITR0 = 0,
+ ICE_IDX_ITR1 = 1,
+ ICE_IDX_ITR2 = 2,
+ ICE_ITR_NONE = 3 /* ITR_NONE must not be used as an index */
+};
+
+/* Header split modes defined by DTYPE field of Rx RLAN context */
+enum ice_rx_dtype {
+ ICE_RX_DTYPE_NO_SPLIT = 0,
+ ICE_RX_DTYPE_HEADER_SPLIT = 1,
+ ICE_RX_DTYPE_SPLIT_ALWAYS = 2,
+};
+
+/* indices into GLINT_ITR registers */
+#define ICE_RX_ITR ICE_IDX_ITR0
+#define ICE_TX_ITR ICE_IDX_ITR1
+#define ICE_ITR_DYNAMIC 0x8000 /* use top bit as a flag */
+#define ICE_ITR_8K 0x003E
+
+/* apply ITR HW granularity translation to program the HW registers */
+#define ITR_TO_REG(val, itr_gran) (((val) & ~ICE_ITR_DYNAMIC) >> (itr_gran))
+
+/* Legacy or Advanced Mode Queue */
+#define ICE_TX_ADVANCED 0
+#define ICE_TX_LEGACY 1
+
+/* descriptor ring, associated with a VSI */
+struct ice_ring {
+ struct ice_ring *next; /* pointer to next ring in q_vector */
+ void *desc; /* Descriptor ring memory */
+ struct device *dev; /* Used for DMA mapping */
+ struct net_device *netdev; /* netdev ring maps to */
+ struct ice_vsi *vsi; /* Backreference to associated VSI */
+ struct ice_q_vector *q_vector; /* Backreference to associated vector */
+ u8 __iomem *tail;
+ union {
+ struct ice_tx_buf *tx_buf;
+ struct ice_rx_buf *rx_buf;
+ };
+ u16 q_index; /* Queue number of ring */
+ u32 txq_teid; /* Added Tx queue TEID */
+
+ /* high bit set means dynamic, use accessor routines to read/write.
+ * hardware supports 2us/1us resolution for the ITR registers.
+ * these values always store the USER setting, and must be converted
+ * before programming to a register.
+ */
+ u16 rx_itr_setting;
+ u16 tx_itr_setting;
+
+ u16 count; /* Number of descriptors */
+ u16 reg_idx; /* HW register index of the ring */
+
+ /* used in interrupt processing */
+ u16 next_to_use;
+ u16 next_to_clean;
+
+ bool ring_active; /* is ring online or not */
+
+ /* stats structs */
+ struct ice_q_stats stats;
+ struct u64_stats_sync syncp;
+ union {
+ struct ice_txq_stats tx_stats;
+ struct ice_rxq_stats rx_stats;
+ };
+
+ unsigned int size; /* length of descriptor ring in bytes */
+ dma_addr_t dma; /* physical address of ring */
+ struct rcu_head rcu; /* to avoid race on free */
+ u16 next_to_alloc;
+} ____cacheline_internodealigned_in_smp;
+
+enum ice_latency_range {
+ ICE_LOWEST_LATENCY = 0,
+ ICE_LOW_LATENCY = 1,
+ ICE_BULK_LATENCY = 2,
+ ICE_ULTRA_LATENCY = 3,
+};
+
+struct ice_ring_container {
+ /* array of pointers to rings */
+ struct ice_ring *ring;
+ unsigned int total_bytes; /* total bytes processed this int */
+ unsigned int total_pkts; /* total packets processed this int */
+ enum ice_latency_range latency_range;
+ u16 itr;
+};
+
+/* iterator for handling rings in ring container */
+#define ice_for_each_ring(pos, head) \
+ for (pos = (head).ring; pos; pos = pos->next)
+
+bool ice_alloc_rx_bufs(struct ice_ring *rxr, u16 cleaned_count);
+netdev_tx_t ice_start_xmit(struct sk_buff *skb, struct net_device *netdev);
+void ice_clean_tx_ring(struct ice_ring *tx_ring);
+void ice_clean_rx_ring(struct ice_ring *rx_ring);
+int ice_setup_tx_ring(struct ice_ring *tx_ring);
+int ice_setup_rx_ring(struct ice_ring *rx_ring);
+void ice_free_tx_ring(struct ice_ring *tx_ring);
+void ice_free_rx_ring(struct ice_ring *rx_ring);
+int ice_napi_poll(struct napi_struct *napi, int budget);
+
+#endif /* _ICE_TXRX_H_ */
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018, Intel Corporation. */
+
+#ifndef _ICE_TYPE_H_
+#define _ICE_TYPE_H_
+
+#include "ice_status.h"
+#include "ice_hw_autogen.h"
+#include "ice_osdep.h"
+#include "ice_controlq.h"
+#include "ice_lan_tx_rx.h"
+
+#define ICE_BYTES_PER_WORD 2
+#define ICE_BYTES_PER_DWORD 4
+
+static inline bool ice_is_tc_ena(u8 bitmap, u8 tc)
+{
+ return test_bit(tc, (unsigned long *)&bitmap);
+}
+
+/* debug masks - set these bits in hw->debug_mask to control output */
+#define ICE_DBG_INIT BIT_ULL(1)
+#define ICE_DBG_LINK BIT_ULL(4)
+#define ICE_DBG_QCTX BIT_ULL(6)
+#define ICE_DBG_NVM BIT_ULL(7)
+#define ICE_DBG_LAN BIT_ULL(8)
+#define ICE_DBG_SW BIT_ULL(13)
+#define ICE_DBG_SCHED BIT_ULL(14)
+#define ICE_DBG_RES BIT_ULL(17)
+#define ICE_DBG_AQ_MSG BIT_ULL(24)
+#define ICE_DBG_AQ_CMD BIT_ULL(27)
+#define ICE_DBG_USER BIT_ULL(31)
+
+enum ice_aq_res_ids {
+ ICE_NVM_RES_ID = 1,
+ ICE_SPD_RES_ID,
+ ICE_GLOBAL_CFG_LOCK_RES_ID,
+ ICE_CHANGE_LOCK_RES_ID
+};
+
+enum ice_aq_res_access_type {
+ ICE_RES_READ = 1,
+ ICE_RES_WRITE
+};
+
+enum ice_fc_mode {
+ ICE_FC_NONE = 0,
+ ICE_FC_RX_PAUSE,
+ ICE_FC_TX_PAUSE,
+ ICE_FC_FULL,
+ ICE_FC_PFC,
+ ICE_FC_DFLT
+};
+
+enum ice_set_fc_aq_failures {
+ ICE_SET_FC_AQ_FAIL_NONE = 0,
+ ICE_SET_FC_AQ_FAIL_GET,
+ ICE_SET_FC_AQ_FAIL_SET,
+ ICE_SET_FC_AQ_FAIL_UPDATE
+};
+
+/* Various MAC types */
+enum ice_mac_type {
+ ICE_MAC_UNKNOWN = 0,
+ ICE_MAC_GENERIC,
+};
+
+/* Media Types */
+enum ice_media_type {
+ ICE_MEDIA_UNKNOWN = 0,
+ ICE_MEDIA_FIBER,
+ ICE_MEDIA_BASET,
+ ICE_MEDIA_BACKPLANE,
+ ICE_MEDIA_DA,
+};
+
+enum ice_vsi_type {
+ ICE_VSI_PF = 0,
+};
+
+struct ice_link_status {
+ /* Refer to ice_aq_phy_type for bits definition */
+ u64 phy_type_low;
+ u16 max_frame_size;
+ u16 link_speed;
+ bool lse_ena; /* Link Status Event notification */
+ u8 link_info;
+ u8 an_info;
+ u8 ext_info;
+ u8 pacing;
+ u8 req_speeds;
+ /* Refer to #define from module_type[ICE_MODULE_TYPE_TOTAL_BYTE] of
+ * ice_aqc_get_phy_caps structure
+ */
+ u8 module_type[ICE_MODULE_TYPE_TOTAL_BYTE];
+};
+
+/* PHY info such as phy_type, etc... */
+struct ice_phy_info {
+ struct ice_link_status link_info;
+ struct ice_link_status link_info_old;
+ u64 phy_type_low;
+ enum ice_media_type media_type;
+ bool get_link_info;
+};
+
+/* Common HW capabilities for SW use */
+struct ice_hw_common_caps {
+ /* TX/RX queues */
+ u16 num_rxq; /* Number/Total RX queues */
+ u16 rxq_first_id; /* First queue ID for RX queues */
+ u16 num_txq; /* Number/Total TX queues */
+ u16 txq_first_id; /* First queue ID for TX queues */
+
+ /* MSI-X vectors */
+ u16 num_msix_vectors;
+ u16 msix_vector_first_id;
+
+ /* Max MTU for function or device */
+ u16 max_mtu;
+
+ /* RSS related capabilities */
+ u16 rss_table_size; /* 512 for PFs and 64 for VFs */
+ u8 rss_table_entry_width; /* RSS Entry width in bits */
+};
+
+/* Function specific capabilities */
+struct ice_hw_func_caps {
+ struct ice_hw_common_caps common_cap;
+ u32 guaranteed_num_vsi;
+};
+
+/* Device wide capabilities */
+struct ice_hw_dev_caps {
+ struct ice_hw_common_caps common_cap;
+ u32 num_vsi_allocd_to_host; /* Excluding EMP VSI */
+};
+
+/* MAC info */
+struct ice_mac_info {
+ u8 lan_addr[ETH_ALEN];
+ u8 perm_addr[ETH_ALEN];
+};
+
+/* Various RESET request, These are not tied with HW reset types */
+enum ice_reset_req {
+ ICE_RESET_PFR = 0,
+ ICE_RESET_CORER = 1,
+ ICE_RESET_GLOBR = 2,
+};
+
+/* Bus parameters */
+struct ice_bus_info {
+ u16 device;
+ u8 func;
+};
+
+/* Flow control (FC) parameters */
+struct ice_fc_info {
+ enum ice_fc_mode current_mode; /* FC mode in effect */
+ enum ice_fc_mode req_mode; /* FC mode requested by caller */
+};
+
+/* NVM Information */
+struct ice_nvm_info {
+ u32 eetrack; /* NVM data version */
+ u32 oem_ver; /* OEM version info */
+ u16 sr_words; /* Shadow RAM size in words */
+ u16 ver; /* NVM package version */
+ bool blank_nvm_mode; /* is NVM empty (no FW present) */
+};
+
+/* Max number of port to queue branches w.r.t topology */
+#define ICE_MAX_TRAFFIC_CLASS 8
+#define ICE_TXSCHED_MAX_BRANCHES ICE_MAX_TRAFFIC_CLASS
+
+struct ice_sched_node {
+ struct ice_sched_node *parent;
+ struct ice_sched_node *sibling; /* next sibling in the same layer */
+ struct ice_sched_node **children;
+ struct ice_aqc_txsched_elem_data info;
+ u32 agg_id; /* aggregator group id */
+ u16 vsi_id;
+ bool in_use; /* suspended or in use */
+ u8 tx_sched_layer; /* Logical Layer (1-9) */
+ u8 num_children;
+ u8 tc_num;
+ u8 owner;
+#define ICE_SCHED_NODE_OWNER_LAN 0
+};
+
+/* Access Macros for Tx Sched Elements data */
+#define ICE_TXSCHED_GET_NODE_TEID(x) le32_to_cpu((x)->info.node_teid)
+
+/* The aggregator type determines if identifier is for a VSI group,
+ * aggregator group, aggregator of queues, or queue group.
+ */
+enum ice_agg_type {
+ ICE_AGG_TYPE_UNKNOWN = 0,
+ ICE_AGG_TYPE_VSI,
+ ICE_AGG_TYPE_AGG, /* aggregator */
+ ICE_AGG_TYPE_Q,
+ ICE_AGG_TYPE_QG
+};
+
+#define ICE_SCHED_DFLT_RL_PROF_ID 0
+
+/* vsi type list entry to locate corresponding vsi/ag nodes */
+struct ice_sched_vsi_info {
+ struct ice_sched_node *vsi_node[ICE_MAX_TRAFFIC_CLASS];
+ struct ice_sched_node *ag_node[ICE_MAX_TRAFFIC_CLASS];
+ struct list_head list_entry;
+ u16 max_lanq[ICE_MAX_TRAFFIC_CLASS];
+ u16 vsi_id;
+};
+
+/* driver defines the policy */
+struct ice_sched_tx_policy {
+ u16 max_num_vsis;
+ u8 max_num_lan_qs_per_tc[ICE_MAX_TRAFFIC_CLASS];
+ bool rdma_ena;
+};
+
+struct ice_port_info {
+ struct ice_sched_node *root; /* Root Node per Port */
+ struct ice_hw *hw; /* back pointer to hw instance */
+ u32 last_node_teid; /* scheduler last node info */
+ u16 sw_id; /* Initial switch ID belongs to port */
+ u16 pf_vf_num;
+ u8 port_state;
+#define ICE_SCHED_PORT_STATE_INIT 0x0
+#define ICE_SCHED_PORT_STATE_READY 0x1
+ u16 dflt_tx_vsi_rule_id;
+ u16 dflt_tx_vsi_num;
+ u16 dflt_rx_vsi_rule_id;
+ u16 dflt_rx_vsi_num;
+ struct ice_fc_info fc;
+ struct ice_mac_info mac;
+ struct ice_phy_info phy;
+ struct mutex sched_lock; /* protect access to TXSched tree */
+ struct ice_sched_tx_policy sched_policy;
+ struct list_head vsi_info_list;
+ struct list_head agg_list; /* lists all aggregator */
+ u8 lport;
+#define ICE_LPORT_MASK 0xff
+ bool is_vf;
+};
+
+struct ice_switch_info {
+ /* Switch VSI lists to MAC/VLAN translation */
+ struct mutex mac_list_lock; /* protect MAC list */
+ struct list_head mac_list_head;
+ struct mutex vlan_list_lock; /* protect VLAN list */
+ struct list_head vlan_list_head;
+ struct mutex eth_m_list_lock; /* protect ethtype list */
+ struct list_head eth_m_list_head;
+ struct mutex promisc_list_lock; /* protect promisc mode list */
+ struct list_head promisc_list_head;
+ struct mutex mac_vlan_list_lock; /* protect MAC-VLAN list */
+ struct list_head mac_vlan_list_head;
+
+ struct list_head vsi_list_map_head;
+};
+
+/* Port hardware description */
+struct ice_hw {
+ u8 __iomem *hw_addr;
+ void *back;
+ struct ice_aqc_layer_props *layer_info;
+ struct ice_port_info *port_info;
+ u64 debug_mask; /* bitmap for debug mask */
+ enum ice_mac_type mac_type;
+
+ /* pci info */
+ u16 device_id;
+ u16 vendor_id;
+ u16 subsystem_device_id;
+ u16 subsystem_vendor_id;
+ u8 revision_id;
+
+ u8 pf_id; /* device profile info */
+
+ /* TX Scheduler values */
+ u16 num_tx_sched_layers;
+ u16 num_tx_sched_phys_layers;
+ u8 flattened_layers;
+ u8 max_cgds;
+ u8 sw_entry_point_layer;
+
+ bool evb_veb; /* true for VEB, false for VEPA */
+ struct ice_bus_info bus;
+ struct ice_nvm_info nvm;
+ struct ice_hw_dev_caps dev_caps; /* device capabilities */
+ struct ice_hw_func_caps func_caps; /* function capabilities */
+
+ struct ice_switch_info *switch_info; /* switch filter lists */
+
+ /* Control Queue info */
+ struct ice_ctl_q_info adminq;
+
+ u8 api_branch; /* API branch version */
+ u8 api_maj_ver; /* API major version */
+ u8 api_min_ver; /* API minor version */
+ u8 api_patch; /* API patch version */
+ u8 fw_branch; /* firmware branch version */
+ u8 fw_maj_ver; /* firmware major version */
+ u8 fw_min_ver; /* firmware minor version */
+ u8 fw_patch; /* firmware patch version */
+ u32 fw_build; /* firmware build number */
+
+ /* minimum allowed value for different speeds */
+#define ICE_ITR_GRAN_MIN_200 1
+#define ICE_ITR_GRAN_MIN_100 1
+#define ICE_ITR_GRAN_MIN_50 2
+#define ICE_ITR_GRAN_MIN_25 4
+ /* ITR granularity in 1 us */
+ u8 itr_gran_200;
+ u8 itr_gran_100;
+ u8 itr_gran_50;
+ u8 itr_gran_25;
+ bool ucast_shared; /* true if VSIs can share unicast addr */
+
+};
+
+/* Statistics collected by each port, VSI, VEB, and S-channel */
+struct ice_eth_stats {
+ u64 rx_bytes; /* gorc */
+ u64 rx_unicast; /* uprc */
+ u64 rx_multicast; /* mprc */
+ u64 rx_broadcast; /* bprc */
+ u64 rx_discards; /* rdpc */
+ u64 rx_unknown_protocol; /* rupp */
+ u64 tx_bytes; /* gotc */
+ u64 tx_unicast; /* uptc */
+ u64 tx_multicast; /* mptc */
+ u64 tx_broadcast; /* bptc */
+ u64 tx_discards; /* tdpc */
+ u64 tx_errors; /* tepc */
+};
+
+/* Statistics collected by the MAC */
+struct ice_hw_port_stats {
+ /* eth stats collected by the port */
+ struct ice_eth_stats eth;
+ /* additional port specific stats */
+ u64 tx_dropped_link_down; /* tdold */
+ u64 crc_errors; /* crcerrs */
+ u64 illegal_bytes; /* illerrc */
+ u64 error_bytes; /* errbc */
+ u64 mac_local_faults; /* mlfc */
+ u64 mac_remote_faults; /* mrfc */
+ u64 rx_len_errors; /* rlec */
+ u64 link_xon_rx; /* lxonrxc */
+ u64 link_xoff_rx; /* lxoffrxc */
+ u64 link_xon_tx; /* lxontxc */
+ u64 link_xoff_tx; /* lxofftxc */
+ u64 rx_size_64; /* prc64 */
+ u64 rx_size_127; /* prc127 */
+ u64 rx_size_255; /* prc255 */
+ u64 rx_size_511; /* prc511 */
+ u64 rx_size_1023; /* prc1023 */
+ u64 rx_size_1522; /* prc1522 */
+ u64 rx_size_big; /* prc9522 */
+ u64 rx_undersize; /* ruc */
+ u64 rx_fragments; /* rfc */
+ u64 rx_oversize; /* roc */
+ u64 rx_jabber; /* rjc */
+ u64 tx_size_64; /* ptc64 */
+ u64 tx_size_127; /* ptc127 */
+ u64 tx_size_255; /* ptc255 */
+ u64 tx_size_511; /* ptc511 */
+ u64 tx_size_1023; /* ptc1023 */
+ u64 tx_size_1522; /* ptc1522 */
+ u64 tx_size_big; /* ptc9522 */
+};
+
+/* Checksum and Shadow RAM pointers */
+#define ICE_SR_NVM_DEV_STARTER_VER 0x18
+#define ICE_SR_NVM_EETRACK_LO 0x2D
+#define ICE_SR_NVM_EETRACK_HI 0x2E
+#define ICE_NVM_VER_LO_SHIFT 0
+#define ICE_NVM_VER_LO_MASK (0xff << ICE_NVM_VER_LO_SHIFT)
+#define ICE_NVM_VER_HI_SHIFT 12
+#define ICE_NVM_VER_HI_MASK (0xf << ICE_NVM_VER_HI_SHIFT)
+#define ICE_OEM_VER_PATCH_SHIFT 0
+#define ICE_OEM_VER_PATCH_MASK (0xff << ICE_OEM_VER_PATCH_SHIFT)
+#define ICE_OEM_VER_BUILD_SHIFT 8
+#define ICE_OEM_VER_BUILD_MASK (0xffff << ICE_OEM_VER_BUILD_SHIFT)
+#define ICE_OEM_VER_SHIFT 24
+#define ICE_OEM_VER_MASK (0xff << ICE_OEM_VER_SHIFT)
+#define ICE_SR_SECTOR_SIZE_IN_WORDS 0x800
+#define ICE_SR_WORDS_IN_1KB 512
+
+#endif /* _ICE_TYPE_H_ */
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel 82575 PCI-Express Ethernet Linux driver
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
* This program is free software; you can redistribute it and/or modify it
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2015 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+/* SPDX-License-Identifier: GPL-2.0 */
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
&adapter->hw.mac.thermal_sensor_data.sensor[offset];
igb_attr->hw = &adapter->hw;
igb_attr->dev_attr.store = NULL;
- igb_attr->dev_attr.attr.mode = S_IRUGO;
+ igb_attr->dev_attr.attr.mode = 0444;
igb_attr->dev_attr.attr.name = igb_attr->name;
sysfs_attr_init(&igb_attr->dev_attr.attr);
+// SPDX-License-Identifier: GPL-2.0
/* Intel(R) Gigabit Ethernet Linux driver
* Copyright(c) 2007-2014 Intel Corporation.
*
+// SPDX-License-Identifier: GPL-2.0+
/* PTP Hardware Clock (PHC) driver for the Intel 82576 and 82580
*
* Copyright (C) 2011 Richard Cochran <richardcochran@gmail.com>
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel(R) 82576 Virtual Function Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel(R) 82576 Virtual Function Linux driver
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel PRO/10GbE Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/10GbE Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/10GbE Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/10GbE Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/10GbE Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel PRO/10GbE Linux driver
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
void ixgbe_set_soft_rate_select_speed(struct ixgbe_hw *hw,
ixgbe_link_speed speed);
+#define IXGBE_FAILED_READ_RETRIES 5
#define IXGBE_FAILED_READ_REG 0xffffffffU
#define IXGBE_FAILED_READ_CFG_DWORD 0xffffffffU
#define IXGBE_FAILED_READ_CFG_WORD 0xffffU
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+// SPDX-License-Identifier: GPL-2.0
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
first->tx_flags |= IXGBE_TX_FLAGS_IPSEC | IXGBE_TX_FLAGS_CC;
- itd->flags = 0;
if (xs->id.proto == IPPROTO_ESP) {
- struct sk_buff *skb = first->skb;
- int ret, authlen, trailerlen;
- u8 padlen;
itd->flags |= IXGBE_ADVTXD_TUCMD_IPSEC_TYPE_ESP |
IXGBE_ADVTXD_TUCMD_L4T_TCP;
* padlen bytes of padding. This ends up not the same
* as the static value found in xs->props.trailer_len (21).
*
- * The "correct" way to get the auth length would be to use
- * authlen = crypto_aead_authsize(xs->data);
- * but since we know we only have one size to worry about
- * we can let the compiler use the constant and save us a
- * few CPU cycles.
+ * ... but if we're doing GSO, don't bother as the stack
+ * doesn't add a trailer for those.
*/
- authlen = IXGBE_IPSEC_AUTH_BITS / 8;
-
- ret = skb_copy_bits(skb, skb->len - (authlen + 2), &padlen, 1);
- if (unlikely(ret))
- return 0;
- trailerlen = authlen + 2 + padlen;
- itd->trailer_len = trailerlen;
+ if (!skb_is_gso(first->skb)) {
+ /* The "correct" way to get the auth length would be
+ * to use
+ * authlen = crypto_aead_authsize(xs->data);
+ * but since we know we only have one size to worry
+ * about * we can let the compiler use the constant
+ * and save us a few CPU cycles.
+ */
+ const int authlen = IXGBE_IPSEC_AUTH_BITS / 8;
+ struct sk_buff *skb = first->skb;
+ u8 padlen;
+ int ret;
+
+ ret = skb_copy_bits(skb, skb->len - (authlen + 2),
+ &padlen, 1);
+ if (unlikely(ret))
+ return 0;
+ itd->trailer_len = authlen + 2 + padlen;
+ }
}
if (tsa->encrypt)
itd->flags |= IXGBE_ADVTXD_TUCMD_IPSEC_ENCRYPT_EN;
ixgbe_ipsec_clear_hw_tables(adapter);
adapter->netdev->xfrmdev_ops = &ixgbe_xfrmdev_ops;
- adapter->netdev->features |= NETIF_F_HW_ESP;
- adapter->netdev->hw_enc_features |= NETIF_F_HW_ESP;
+
+#define IXGBE_ESP_FEATURES (NETIF_F_HW_ESP | \
+ NETIF_F_HW_ESP_TX_CSUM | \
+ NETIF_F_GSO_ESP)
+
+ adapter->netdev->features |= IXGBE_ESP_FEATURES;
+ adapter->netdev->hw_enc_features |= IXGBE_ESP_FEATURES;
return;
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
ixgbe_service_event_schedule(adapter);
}
-static void ixgbe_check_remove(struct ixgbe_hw *hw, u32 reg)
+static u32 ixgbe_check_remove(struct ixgbe_hw *hw, u32 reg)
{
+ u8 __iomem *reg_addr;
u32 value;
+ int i;
+
+ reg_addr = READ_ONCE(hw->hw_addr);
+ if (ixgbe_removed(reg_addr))
+ return IXGBE_FAILED_READ_REG;
- /* The following check not only optimizes a bit by not
- * performing a read on the status register when the
- * register just read was a status register read that
- * returned IXGBE_FAILED_READ_REG. It also blocks any
- * potential recursion.
+ /* Register read of 0xFFFFFFF can indicate the adapter has been removed,
+ * so perform several status register reads to determine if the adapter
+ * has been removed.
*/
- if (reg == IXGBE_STATUS) {
- ixgbe_remove_adapter(hw);
- return;
+ for (i = 0; i < IXGBE_FAILED_READ_RETRIES; i++) {
+ value = readl(reg_addr + IXGBE_STATUS);
+ if (value != IXGBE_FAILED_READ_REG)
+ break;
+ mdelay(3);
}
- value = ixgbe_read_reg(hw, IXGBE_STATUS);
+
if (value == IXGBE_FAILED_READ_REG)
ixgbe_remove_adapter(hw);
+ else
+ value = readl(reg_addr + reg);
+ return value;
}
/**
writes_completed:
value = readl(reg_addr + reg);
if (unlikely(value == IXGBE_FAILED_READ_REG))
- ixgbe_check_remove(hw, reg);
+ value = ixgbe_check_remove(hw, reg);
return value;
}
bi->dma = dma;
bi->page = page;
bi->page_offset = ixgbe_rx_offset(rx_ring);
- bi->pagecnt_bias = 1;
+ page_ref_add(page, USHRT_MAX - 1);
+ bi->pagecnt_bias = USHRT_MAX;
rx_ring->rx_stats.alloc_rx_page++;
return true;
* the pagecnt_bias and page count so that we fully restock the
* number of references the driver holds.
*/
- if (unlikely(!pagecnt_bias)) {
- page_ref_add(page, USHRT_MAX);
+ if (unlikely(pagecnt_bias == 1)) {
+ page_ref_add(page, USHRT_MAX - 1);
rx_buffer->pagecnt_bias = USHRT_MAX;
}
static int ixgbe_tso(struct ixgbe_ring *tx_ring,
struct ixgbe_tx_buffer *first,
- u8 *hdr_len)
+ u8 *hdr_len,
+ struct ixgbe_ipsec_tx_data *itd)
{
u32 vlan_macip_lens, type_tucmd, mss_l4len_idx;
struct sk_buff *skb = first->skb;
unsigned char *hdr;
} l4;
u32 paylen, l4_offset;
+ u32 fceof_saidx = 0;
int err;
if (skb->ip_summed != CHECKSUM_PARTIAL)
if (ip.v4->version == 4) {
unsigned char *csum_start = skb_checksum_start(skb);
unsigned char *trans_start = ip.hdr + (ip.v4->ihl * 4);
+ int len = csum_start - trans_start;
/* IP header will have to cancel out any data that
- * is not a part of the outer IP header
+ * is not a part of the outer IP header, so set to
+ * a reverse csum if needed, else init check to 0.
*/
- ip.v4->check = csum_fold(csum_partial(trans_start,
- csum_start - trans_start,
- 0));
+ ip.v4->check = (skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL) ?
+ csum_fold(csum_partial(trans_start,
+ len, 0)) : 0;
type_tucmd |= IXGBE_ADVTXD_TUCMD_IPV4;
ip.v4->tot_len = 0;
mss_l4len_idx = (*hdr_len - l4_offset) << IXGBE_ADVTXD_L4LEN_SHIFT;
mss_l4len_idx |= skb_shinfo(skb)->gso_size << IXGBE_ADVTXD_MSS_SHIFT;
+ fceof_saidx |= itd->sa_idx;
+ type_tucmd |= itd->flags | itd->trailer_len;
+
/* vlan_macip_lens: HEADLEN, MACLEN, VLAN tag */
vlan_macip_lens = l4.hdr - ip.hdr;
vlan_macip_lens |= (ip.hdr - skb->data) << IXGBE_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IXGBE_TX_FLAGS_VLAN_MASK;
- ixgbe_tx_ctxtdesc(tx_ring, vlan_macip_lens, 0, type_tucmd,
+ ixgbe_tx_ctxtdesc(tx_ring, vlan_macip_lens, fceof_saidx, type_tucmd,
mss_l4len_idx);
return 1;
vlan_macip_lens |= skb_network_offset(skb) << IXGBE_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IXGBE_TX_FLAGS_VLAN_MASK;
- if (first->tx_flags & IXGBE_TX_FLAGS_IPSEC) {
- fceof_saidx |= itd->sa_idx;
- type_tucmd |= itd->flags | itd->trailer_len;
- }
+ fceof_saidx |= itd->sa_idx;
+ type_tucmd |= itd->flags | itd->trailer_len;
ixgbe_tx_ctxtdesc(tx_ring, vlan_macip_lens, fceof_saidx, type_tucmd, 0);
}
if (skb->sp && !ixgbe_ipsec_tx(tx_ring, first, &ipsec_tx))
goto out_drop;
#endif
- tso = ixgbe_tso(tx_ring, first, &hdr_len);
+ tso = ixgbe_tso(tx_ring, first, &hdr_len, &ipsec_tx);
if (tso < 0)
goto out_drop;
else if (!tso)
/* We can only support IPV4 TSO in tunnels if we can mangle the
* inner IP ID field, so strip TSO if MANGLEID is not supported.
+ * IPsec offoad sets skb->encapsulation but still can handle
+ * the TSO, so it's the exception.
*/
- if (skb->encapsulation && !(features & NETIF_F_TSO_MANGLEID))
- features &= ~NETIF_F_TSO;
-
-#ifdef CONFIG_XFRM_OFFLOAD
- /* IPsec offload doesn't get along well with others *yet* */
- if (skb->sp)
- features &= ~(NETIF_F_TSO | NETIF_F_HW_CSUM);
+ if (skb->encapsulation && !(features & NETIF_F_TSO_MANGLEID)) {
+#ifdef CONFIG_XFRM
+ if (!skb->sp)
#endif
+ features &= ~NETIF_F_TSO;
+ }
return features;
}
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel 10 Gigabit PCI Express Linux drive
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
&adapter->hw.mac.thermal_sensor_data.sensor[offset];
ixgbe_attr->hw = &adapter->hw;
ixgbe_attr->dev_attr.store = NULL;
- ixgbe_attr->dev_attr.attr.mode = S_IRUGO;
+ ixgbe_attr->dev_attr.attr.mode = 0444;
ixgbe_attr->dev_attr.attr.name = ixgbe_attr->name;
sysfs_attr_init(&ixgbe_attr->dev_attr.attr);
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 10 Gigabit PCI Express Linux driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
*
* Intel 10 Gigabit PCI Express Linux driver
(IXGBE_CS4227_EDC_MODE_SR << 1));
if (setup_linear)
- reg_phy_ext = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 1;
+ reg_phy_ext |= (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 1;
else
- reg_phy_ext = (IXGBE_CS4227_EDC_MODE_SR << 1) | 1;
+ reg_phy_ext |= (IXGBE_CS4227_EDC_MODE_SR << 1) | 1;
ret_val = hw->phy.ops.write_reg(hw, reg_slice,
IXGBE_MDIO_ZERO_DEV_TYPE, reg_phy_ext);
+# SPDX-License-Identifier: GPL-2.0
################################################################################
#
# Intel 82599 Virtual Function driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 82599 Virtual Function driver
/*******************************************************************************
Intel 82599 Virtual Function driver
- Copyright(c) 1999 - 2015 Intel Corporation.
+ Copyright(c) 1999 - 2018 Intel Corporation.
This program is free software; you can redistribute it and/or modify it
under the terms and conditions of the GNU General Public License,
#define IXGBEVF_QUEUE_STATS_LEN ( \
(((struct ixgbevf_adapter *)netdev_priv(netdev))->num_tx_queues + \
+ ((struct ixgbevf_adapter *)netdev_priv(netdev))->num_xdp_queues + \
((struct ixgbevf_adapter *)netdev_priv(netdev))->num_rx_queues) * \
(sizeof(struct ixgbevf_stats) / sizeof(u64)))
#define IXGBEVF_GLOBAL_STATS_LEN ARRAY_SIZE(ixgbevf_gstrings_stats)
struct ixgbevf_adapter *adapter = netdev_priv(netdev);
struct ixgbevf_ring *tx_ring = NULL, *rx_ring = NULL;
u32 new_rx_count, new_tx_count;
- int i, err = 0;
+ int i, j, err = 0;
if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending))
return -EINVAL;
if (!netif_running(adapter->netdev)) {
for (i = 0; i < adapter->num_tx_queues; i++)
adapter->tx_ring[i]->count = new_tx_count;
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ adapter->xdp_ring[i]->count = new_tx_count;
for (i = 0; i < adapter->num_rx_queues; i++)
adapter->rx_ring[i]->count = new_rx_count;
adapter->tx_ring_count = new_tx_count;
+ adapter->xdp_ring_count = new_tx_count;
adapter->rx_ring_count = new_rx_count;
goto clear_reset;
}
if (new_tx_count != adapter->tx_ring_count) {
- tx_ring = vmalloc(adapter->num_tx_queues * sizeof(*tx_ring));
+ tx_ring = vmalloc((adapter->num_tx_queues +
+ adapter->num_xdp_queues) * sizeof(*tx_ring));
if (!tx_ring) {
err = -ENOMEM;
goto clear_reset;
goto clear_reset;
}
}
+
+ for (j = 0; j < adapter->num_xdp_queues; i++, j++) {
+ /* clone ring and setup updated count */
+ tx_ring[i] = *adapter->xdp_ring[j];
+ tx_ring[i].count = new_tx_count;
+ err = ixgbevf_setup_tx_resources(&tx_ring[i]);
+ if (err) {
+ while (i) {
+ i--;
+ ixgbevf_free_tx_resources(&tx_ring[i]);
+ }
+
+ vfree(tx_ring);
+ tx_ring = NULL;
+
+ goto clear_reset;
+ }
+ }
}
if (new_rx_count != adapter->rx_ring_count) {
for (i = 0; i < adapter->num_rx_queues; i++) {
/* clone ring and setup updated count */
rx_ring[i] = *adapter->rx_ring[i];
+
+ /* Clear copied XDP RX-queue info */
+ memset(&rx_ring[i].xdp_rxq, 0,
+ sizeof(rx_ring[i].xdp_rxq));
+
rx_ring[i].count = new_rx_count;
- err = ixgbevf_setup_rx_resources(&rx_ring[i]);
+ err = ixgbevf_setup_rx_resources(adapter, &rx_ring[i]);
if (err) {
while (i) {
i--;
}
adapter->tx_ring_count = new_tx_count;
+ for (j = 0; j < adapter->num_xdp_queues; i++, j++) {
+ ixgbevf_free_tx_resources(adapter->xdp_ring[j]);
+ *adapter->xdp_ring[j] = tx_ring[i];
+ }
+ adapter->xdp_ring_count = new_tx_count;
+
vfree(tx_ring);
tx_ring = NULL;
}
clear_reset:
/* free Tx resources if Rx error is encountered */
if (tx_ring) {
- for (i = 0; i < adapter->num_tx_queues; i++)
+ for (i = 0;
+ i < adapter->num_tx_queues + adapter->num_xdp_queues; i++)
ixgbevf_free_tx_resources(&tx_ring[i]);
vfree(tx_ring);
}
i += 2;
}
+ /* populate XDP queue data */
+ for (j = 0; j < adapter->num_xdp_queues; j++) {
+ ring = adapter->xdp_ring[j];
+ if (!ring) {
+ data[i++] = 0;
+ data[i++] = 0;
+ continue;
+ }
+
+ do {
+ start = u64_stats_fetch_begin_irq(&ring->syncp);
+ data[i] = ring->stats.packets;
+ data[i + 1] = ring->stats.bytes;
+ } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+ i += 2;
+ }
+
/* populate Rx queue data */
for (j = 0; j < adapter->num_rx_queues; j++) {
ring = adapter->rx_ring[j];
sprintf(p, "tx_queue_%u_bytes", i);
p += ETH_GSTRING_LEN;
}
+ for (i = 0; i < adapter->num_xdp_queues; i++) {
+ sprintf(p, "xdp_queue_%u_packets", i);
+ p += ETH_GSTRING_LEN;
+ sprintf(p, "xdp_queue_%u_bytes", i);
+ p += ETH_GSTRING_LEN;
+ }
for (i = 0; i < adapter->num_rx_queues; i++) {
sprintf(p, "rx_queue_%u_packets", i);
p += ETH_GSTRING_LEN;
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 82599 Virtual Function driver
- Copyright(c) 1999 - 2015 Intel Corporation.
+ Copyright(c) 1999 - 2018 Intel Corporation.
This program is free software; you can redistribute it and/or modify it
under the terms and conditions of the GNU General Public License,
#include <linux/netdevice.h>
#include <linux/if_vlan.h>
#include <linux/u64_stats_sync.h>
+#include <net/xdp.h>
#include "vf.h"
struct ixgbevf_tx_buffer {
union ixgbe_adv_tx_desc *next_to_watch;
unsigned long time_stamp;
- struct sk_buff *skb;
+ union {
+ struct sk_buff *skb;
+ /* XDP uses address ptr on irq_clean */
+ void *data;
+ };
unsigned int bytecount;
unsigned short gso_segs;
__be16 protocol;
__IXGBEVF_RX_BUILD_SKB_ENABLED,
__IXGBEVF_TX_DETECT_HANG,
__IXGBEVF_HANG_CHECK_ARMED,
+ __IXGBEVF_TX_XDP_RING,
};
+#define ring_is_xdp(ring) \
+ test_bit(__IXGBEVF_TX_XDP_RING, &(ring)->state)
+#define set_ring_xdp(ring) \
+ set_bit(__IXGBEVF_TX_XDP_RING, &(ring)->state)
+#define clear_ring_xdp(ring) \
+ clear_bit(__IXGBEVF_TX_XDP_RING, &(ring)->state)
+
struct ixgbevf_ring {
struct ixgbevf_ring *next;
struct ixgbevf_q_vector *q_vector; /* backpointer to q_vector */
struct net_device *netdev;
+ struct bpf_prog *xdp_prog;
struct device *dev;
void *desc; /* descriptor ring memory */
dma_addr_t dma; /* phys. address of descriptor ring */
struct ixgbevf_tx_queue_stats tx_stats;
struct ixgbevf_rx_queue_stats rx_stats;
};
-
+ struct xdp_rxq_info xdp_rxq;
u64 hw_csum_rx_error;
u8 __iomem *tail;
struct sk_buff *skb;
#define MAX_RX_QUEUES IXGBE_VF_MAX_RX_QUEUES
#define MAX_TX_QUEUES IXGBE_VF_MAX_TX_QUEUES
+#define MAX_XDP_QUEUES IXGBE_VF_MAX_TX_QUEUES
#define IXGBEVF_MAX_RSS_QUEUES 2
#define IXGBEVF_82599_RETA_SIZE 128 /* 128 entries */
#define IXGBEVF_X550_VFRETA_SIZE 64 /* 64 entries */
u32 eims_enable_mask;
u32 eims_other;
+ /* XDP */
+ int num_xdp_queues;
+ struct ixgbevf_ring *xdp_ring[MAX_XDP_QUEUES];
+
/* TX */
int num_tx_queues;
struct ixgbevf_ring *tx_ring[MAX_TX_QUEUES]; /* One per active queue */
/* OS defined structs */
struct net_device *netdev;
+ struct bpf_prog *xdp_prog;
struct pci_dev *pdev;
/* structs defined in ixgbe_vf.h */
unsigned long state;
u64 tx_busy;
unsigned int tx_ring_count;
+ unsigned int xdp_ring_count;
unsigned int rx_ring_count;
u8 __iomem *io_addr; /* Mainly for iounmap use */
void ixgbevf_reinit_locked(struct ixgbevf_adapter *adapter);
void ixgbevf_reset(struct ixgbevf_adapter *adapter);
void ixgbevf_set_ethtool_ops(struct net_device *netdev);
-int ixgbevf_setup_rx_resources(struct ixgbevf_ring *);
+int ixgbevf_setup_rx_resources(struct ixgbevf_adapter *adapter,
+ struct ixgbevf_ring *rx_ring);
int ixgbevf_setup_tx_resources(struct ixgbevf_ring *);
void ixgbevf_free_rx_resources(struct ixgbevf_ring *);
void ixgbevf_free_tx_resources(struct ixgbevf_ring *);
/*******************************************************************************
Intel 82599 Virtual Function driver
- Copyright(c) 1999 - 2015 Intel Corporation.
+ Copyright(c) 1999 - 2018 Intel Corporation.
This program is free software; you can redistribute it and/or modify it
under the terms and conditions of the GNU General Public License,
#include <linux/if_vlan.h>
#include <linux/prefetch.h>
#include <net/mpls.h>
+#include <linux/bpf.h>
+#include <linux/bpf_trace.h>
+#include <linux/atomic.h>
#include "ixgbevf.h"
total_packets += tx_buffer->gso_segs;
/* free the skb */
- napi_consume_skb(tx_buffer->skb, napi_budget);
+ if (ring_is_xdp(tx_ring))
+ page_frag_free(tx_buffer->data);
+ else
+ napi_consume_skb(tx_buffer->skb, napi_budget);
/* unmap skb header data */
dma_unmap_single(tx_ring->dev,
eop_desc = tx_ring->tx_buffer_info[i].next_to_watch;
- pr_err("Detected Tx Unit Hang\n"
+ pr_err("Detected Tx Unit Hang%s\n"
" Tx Queue <%d>\n"
" TDH, TDT <%x>, <%x>\n"
" next_to_use <%x>\n"
" eop_desc->wb.status <%x>\n"
" time_stamp <%lx>\n"
" jiffies <%lx>\n",
+ ring_is_xdp(tx_ring) ? " XDP" : "",
tx_ring->queue_index,
IXGBE_READ_REG(hw, IXGBE_VFTDH(tx_ring->reg_idx)),
IXGBE_READ_REG(hw, IXGBE_VFTDT(tx_ring->reg_idx)),
eop_desc, (eop_desc ? eop_desc->wb.status : 0),
tx_ring->tx_buffer_info[i].time_stamp, jiffies);
- netif_stop_subqueue(tx_ring->netdev, tx_ring->queue_index);
+ if (!ring_is_xdp(tx_ring))
+ netif_stop_subqueue(tx_ring->netdev,
+ tx_ring->queue_index);
/* schedule immediate reset if we believe we hung */
ixgbevf_tx_timeout_reset(adapter);
return true;
}
+ if (ring_is_xdp(tx_ring))
+ return !!budget;
+
#define TX_WAKE_THRESHOLD (DESC_NEEDED * 2)
if (unlikely(total_packets && netif_carrier_ok(tx_ring->netdev) &&
(ixgbevf_desc_unused(tx_ring) >= TX_WAKE_THRESHOLD))) {
}
static void ixgbevf_put_rx_buffer(struct ixgbevf_ring *rx_ring,
- struct ixgbevf_rx_buffer *rx_buffer)
+ struct ixgbevf_rx_buffer *rx_buffer,
+ struct sk_buff *skb)
{
if (ixgbevf_can_reuse_rx_page(rx_buffer)) {
/* hand second half of page back to the ring */
ixgbevf_reuse_rx_page(rx_ring, rx_buffer);
} else {
- /* We are not reusing the buffer so unmap it and free
- * any references we are holding to it
- */
- dma_unmap_page_attrs(rx_ring->dev, rx_buffer->dma,
- ixgbevf_rx_pg_size(rx_ring),
- DMA_FROM_DEVICE,
- IXGBEVF_RX_DMA_ATTR);
+ if (IS_ERR(skb))
+ /* We are not reusing the buffer so unmap it and free
+ * any references we are holding to it
+ */
+ dma_unmap_page_attrs(rx_ring->dev, rx_buffer->dma,
+ ixgbevf_rx_pg_size(rx_ring),
+ DMA_FROM_DEVICE,
+ IXGBEVF_RX_DMA_ATTR);
__page_frag_cache_drain(rx_buffer->page,
rx_buffer->pagecnt_bias);
}
union ixgbe_adv_rx_desc *rx_desc,
struct sk_buff *skb)
{
+ /* XDP packets use error pointer so abort at this point */
+ if (IS_ERR(skb))
+ return true;
+
/* verify that the packet does not have any known errors */
if (unlikely(ixgbevf_test_staterr(rx_desc,
IXGBE_RXDADV_ERR_FRAME_ERR_MASK))) {
static
struct sk_buff *ixgbevf_construct_skb(struct ixgbevf_ring *rx_ring,
struct ixgbevf_rx_buffer *rx_buffer,
- union ixgbe_adv_rx_desc *rx_desc,
- unsigned int size)
+ struct xdp_buff *xdp,
+ union ixgbe_adv_rx_desc *rx_desc)
{
- void *va = page_address(rx_buffer->page) + rx_buffer->page_offset;
+ unsigned int size = xdp->data_end - xdp->data;
#if (PAGE_SIZE < 8192)
unsigned int truesize = ixgbevf_rx_pg_size(rx_ring) / 2;
#else
- unsigned int truesize = SKB_DATA_ALIGN(size);
+ unsigned int truesize = SKB_DATA_ALIGN(xdp->data_end -
+ xdp->data_hard_start);
#endif
unsigned int headlen;
struct sk_buff *skb;
/* prefetch first cache line of first page */
- prefetch(va);
+ prefetch(xdp->data);
#if L1_CACHE_BYTES < 128
- prefetch(va + L1_CACHE_BYTES);
+ prefetch(xdp->data + L1_CACHE_BYTES);
#endif
+ /* Note, we get here by enabling legacy-rx via:
+ *
+ * ethtool --set-priv-flags <dev> legacy-rx on
+ *
+ * In this mode, we currently get 0 extra XDP headroom as
+ * opposed to having legacy-rx off, where we process XDP
+ * packets going to stack via ixgbevf_build_skb().
+ *
+ * For ixgbevf_construct_skb() mode it means that the
+ * xdp->data_meta will always point to xdp->data, since
+ * the helper cannot expand the head. Should this ever
+ * changed in future for legacy-rx mode on, then lets also
+ * add xdp->data_meta handling here.
+ */
/* allocate a skb to store the frags */
skb = napi_alloc_skb(&rx_ring->q_vector->napi, IXGBEVF_RX_HDR_SIZE);
/* Determine available headroom for copy */
headlen = size;
if (headlen > IXGBEVF_RX_HDR_SIZE)
- headlen = eth_get_headlen(va, IXGBEVF_RX_HDR_SIZE);
+ headlen = eth_get_headlen(xdp->data, IXGBEVF_RX_HDR_SIZE);
/* align pull length to size of long to optimize memcpy performance */
- memcpy(__skb_put(skb, headlen), va, ALIGN(headlen, sizeof(long)));
+ memcpy(__skb_put(skb, headlen), xdp->data,
+ ALIGN(headlen, sizeof(long)));
/* update all of the pointers */
size -= headlen;
if (size) {
skb_add_rx_frag(skb, 0, rx_buffer->page,
- (va + headlen) - page_address(rx_buffer->page),
+ (xdp->data + headlen) -
+ page_address(rx_buffer->page),
size, truesize);
#if (PAGE_SIZE < 8192)
rx_buffer->page_offset ^= truesize;
static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
struct ixgbevf_rx_buffer *rx_buffer,
- union ixgbe_adv_rx_desc *rx_desc,
- unsigned int size)
+ struct xdp_buff *xdp,
+ union ixgbe_adv_rx_desc *rx_desc)
{
- void *va = page_address(rx_buffer->page) + rx_buffer->page_offset;
+ unsigned int metasize = xdp->data - xdp->data_meta;
#if (PAGE_SIZE < 8192)
unsigned int truesize = ixgbevf_rx_pg_size(rx_ring) / 2;
#else
unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) +
- SKB_DATA_ALIGN(IXGBEVF_SKB_PAD + size);
+ SKB_DATA_ALIGN(xdp->data_end -
+ xdp->data_hard_start);
#endif
struct sk_buff *skb;
- /* prefetch first cache line of first page */
- prefetch(va);
+ /* Prefetch first cache line of first page. If xdp->data_meta
+ * is unused, this points to xdp->data, otherwise, we likely
+ * have a consumer accessing first few bytes of meta data,
+ * and then actual data.
+ */
+ prefetch(xdp->data_meta);
#if L1_CACHE_BYTES < 128
- prefetch(va + L1_CACHE_BYTES);
+ prefetch(xdp->data_meta + L1_CACHE_BYTES);
#endif
- /* build an skb to around the page buffer */
- skb = build_skb(va - IXGBEVF_SKB_PAD, truesize);
+ /* build an skb around the page buffer */
+ skb = build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
/* update pointers within the skb to store the data */
- skb_reserve(skb, IXGBEVF_SKB_PAD);
- __skb_put(skb, size);
+ skb_reserve(skb, xdp->data - xdp->data_hard_start);
+ __skb_put(skb, xdp->data_end - xdp->data);
+ if (metasize)
+ skb_metadata_set(skb, metasize);
/* update buffer offset */
#if (PAGE_SIZE < 8192)
return skb;
}
+
+#define IXGBEVF_XDP_PASS 0
+#define IXGBEVF_XDP_CONSUMED 1
+#define IXGBEVF_XDP_TX 2
+
+static int ixgbevf_xmit_xdp_ring(struct ixgbevf_ring *ring,
+ struct xdp_buff *xdp)
+{
+ struct ixgbevf_tx_buffer *tx_buffer;
+ union ixgbe_adv_tx_desc *tx_desc;
+ u32 len, cmd_type;
+ dma_addr_t dma;
+ u16 i;
+
+ len = xdp->data_end - xdp->data;
+
+ if (unlikely(!ixgbevf_desc_unused(ring)))
+ return IXGBEVF_XDP_CONSUMED;
+
+ dma = dma_map_single(ring->dev, xdp->data, len, DMA_TO_DEVICE);
+ if (dma_mapping_error(ring->dev, dma))
+ return IXGBEVF_XDP_CONSUMED;
+
+ /* record the location of the first descriptor for this packet */
+ tx_buffer = &ring->tx_buffer_info[ring->next_to_use];
+ tx_buffer->bytecount = len;
+ tx_buffer->gso_segs = 1;
+ tx_buffer->protocol = 0;
+
+ i = ring->next_to_use;
+ tx_desc = IXGBEVF_TX_DESC(ring, i);
+
+ dma_unmap_len_set(tx_buffer, len, len);
+ dma_unmap_addr_set(tx_buffer, dma, dma);
+ tx_buffer->data = xdp->data;
+ tx_desc->read.buffer_addr = cpu_to_le64(dma);
+
+ /* put descriptor type bits */
+ cmd_type = IXGBE_ADVTXD_DTYP_DATA |
+ IXGBE_ADVTXD_DCMD_DEXT |
+ IXGBE_ADVTXD_DCMD_IFCS;
+ cmd_type |= len | IXGBE_TXD_CMD;
+ tx_desc->read.cmd_type_len = cpu_to_le32(cmd_type);
+ tx_desc->read.olinfo_status =
+ cpu_to_le32((len << IXGBE_ADVTXD_PAYLEN_SHIFT) |
+ IXGBE_ADVTXD_CC);
+
+ /* Avoid any potential race with cleanup */
+ smp_wmb();
+
+ /* set next_to_watch value indicating a packet is present */
+ i++;
+ if (i == ring->count)
+ i = 0;
+
+ tx_buffer->next_to_watch = tx_desc;
+ ring->next_to_use = i;
+
+ return IXGBEVF_XDP_TX;
+}
+
+static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter,
+ struct ixgbevf_ring *rx_ring,
+ struct xdp_buff *xdp)
+{
+ int result = IXGBEVF_XDP_PASS;
+ struct ixgbevf_ring *xdp_ring;
+ struct bpf_prog *xdp_prog;
+ u32 act;
+
+ rcu_read_lock();
+ xdp_prog = READ_ONCE(rx_ring->xdp_prog);
+
+ if (!xdp_prog)
+ goto xdp_out;
+
+ act = bpf_prog_run_xdp(xdp_prog, xdp);
+ switch (act) {
+ case XDP_PASS:
+ break;
+ case XDP_TX:
+ xdp_ring = adapter->xdp_ring[rx_ring->queue_index];
+ result = ixgbevf_xmit_xdp_ring(xdp_ring, xdp);
+ break;
+ default:
+ bpf_warn_invalid_xdp_action(act);
+ /* fallthrough */
+ case XDP_ABORTED:
+ trace_xdp_exception(rx_ring->netdev, xdp_prog, act);
+ /* fallthrough -- handle aborts by dropping packet */
+ case XDP_DROP:
+ result = IXGBEVF_XDP_CONSUMED;
+ break;
+ }
+xdp_out:
+ rcu_read_unlock();
+ return ERR_PTR(-result);
+}
+
+static void ixgbevf_rx_buffer_flip(struct ixgbevf_ring *rx_ring,
+ struct ixgbevf_rx_buffer *rx_buffer,
+ unsigned int size)
+{
+#if (PAGE_SIZE < 8192)
+ unsigned int truesize = ixgbevf_rx_pg_size(rx_ring) / 2;
+
+ rx_buffer->page_offset ^= truesize;
+#else
+ unsigned int truesize = ring_uses_build_skb(rx_ring) ?
+ SKB_DATA_ALIGN(IXGBEVF_SKB_PAD + size) :
+ SKB_DATA_ALIGN(size);
+
+ rx_buffer->page_offset += truesize;
+#endif
+}
+
static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
struct ixgbevf_ring *rx_ring,
int budget)
{
unsigned int total_rx_bytes = 0, total_rx_packets = 0;
+ struct ixgbevf_adapter *adapter = q_vector->adapter;
u16 cleaned_count = ixgbevf_desc_unused(rx_ring);
struct sk_buff *skb = rx_ring->skb;
+ bool xdp_xmit = false;
+ struct xdp_buff xdp;
+
+ xdp.rxq = &rx_ring->xdp_rxq;
while (likely(total_rx_packets < budget)) {
- union ixgbe_adv_rx_desc *rx_desc;
struct ixgbevf_rx_buffer *rx_buffer;
+ union ixgbe_adv_rx_desc *rx_desc;
unsigned int size;
/* return some buffers to hardware, one at a time is too slow */
rx_buffer = ixgbevf_get_rx_buffer(rx_ring, size);
/* retrieve a buffer from the ring */
- if (skb)
+ if (!skb) {
+ xdp.data = page_address(rx_buffer->page) +
+ rx_buffer->page_offset;
+ xdp.data_meta = xdp.data;
+ xdp.data_hard_start = xdp.data -
+ ixgbevf_rx_offset(rx_ring);
+ xdp.data_end = xdp.data + size;
+
+ skb = ixgbevf_run_xdp(adapter, rx_ring, &xdp);
+ }
+
+ if (IS_ERR(skb)) {
+ if (PTR_ERR(skb) == -IXGBEVF_XDP_TX) {
+ xdp_xmit = true;
+ ixgbevf_rx_buffer_flip(rx_ring, rx_buffer,
+ size);
+ } else {
+ rx_buffer->pagecnt_bias++;
+ }
+ total_rx_packets++;
+ total_rx_bytes += size;
+ } else if (skb) {
ixgbevf_add_rx_frag(rx_ring, rx_buffer, skb, size);
- else if (ring_uses_build_skb(rx_ring))
+ } else if (ring_uses_build_skb(rx_ring)) {
skb = ixgbevf_build_skb(rx_ring, rx_buffer,
- rx_desc, size);
- else
+ &xdp, rx_desc);
+ } else {
skb = ixgbevf_construct_skb(rx_ring, rx_buffer,
- rx_desc, size);
+ &xdp, rx_desc);
+ }
/* exit if we failed to retrieve a buffer */
if (!skb) {
break;
}
- ixgbevf_put_rx_buffer(rx_ring, rx_buffer);
+ ixgbevf_put_rx_buffer(rx_ring, rx_buffer, skb);
cleaned_count++;
/* fetch next buffer in frame if non-eop */
/* place incomplete frames back on ring for completion */
rx_ring->skb = skb;
+ if (xdp_xmit) {
+ struct ixgbevf_ring *xdp_ring =
+ adapter->xdp_ring[rx_ring->queue_index];
+
+ /* Force memory writes to complete before letting h/w
+ * know there are new descriptors to fetch.
+ */
+ wmb();
+ ixgbevf_write_tail(xdp_ring, xdp_ring->next_to_use);
+ }
+
u64_stats_update_begin(&rx_ring->syncp);
rx_ring->stats.packets += total_rx_packets;
rx_ring->stats.bytes += total_rx_bytes;
/* Setup the HW Tx Head and Tail descriptor pointers */
for (i = 0; i < adapter->num_tx_queues; i++)
ixgbevf_configure_tx_ring(adapter, adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ ixgbevf_configure_tx_ring(adapter, adapter->xdp_ring[i]);
}
#define IXGBE_SRRCTL_BSIZEHDRSIZE_SHIFT 2
union ixgbe_adv_tx_desc *eop_desc, *tx_desc;
/* Free all the Tx ring sk_buffs */
- dev_kfree_skb_any(tx_buffer->skb);
+ if (ring_is_xdp(tx_ring))
+ page_frag_free(tx_buffer->data);
+ else
+ dev_kfree_skb_any(tx_buffer->skb);
/* unmap skb header data */
dma_unmap_single(tx_ring->dev,
for (i = 0; i < adapter->num_tx_queues; i++)
ixgbevf_clean_tx_ring(adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ ixgbevf_clean_tx_ring(adapter->xdp_ring[i]);
}
void ixgbevf_down(struct ixgbevf_adapter *adapter)
IXGBE_TXDCTL_SWFLSH);
}
+ for (i = 0; i < adapter->num_xdp_queues; i++) {
+ u8 reg_idx = adapter->xdp_ring[i]->reg_idx;
+
+ IXGBE_WRITE_REG(hw, IXGBE_VFTXDCTL(reg_idx),
+ IXGBE_TXDCTL_SWFLSH);
+ }
+
if (!pci_channel_offline(adapter->pdev))
ixgbevf_reset(adapter);
/* Start with base case */
adapter->num_rx_queues = 1;
adapter->num_tx_queues = 1;
+ adapter->num_xdp_queues = 0;
spin_lock_bh(&adapter->mbx_lock);
case ixgbe_mbox_api_11:
case ixgbe_mbox_api_12:
case ixgbe_mbox_api_13:
+ if (adapter->xdp_prog &&
+ hw->mac.max_tx_queues == rss)
+ rss = rss > 3 ? 2 : 1;
+
adapter->num_rx_queues = rss;
adapter->num_tx_queues = rss;
+ adapter->num_xdp_queues = adapter->xdp_prog ? rss : 0;
default:
break;
}
* @v_idx: index of vector in adapter struct
* @txr_count: number of Tx rings for q vector
* @txr_idx: index of first Tx ring to assign
+ * @xdp_count: total number of XDP rings to allocate
+ * @xdp_idx: index of first XDP ring to allocate
* @rxr_count: number of Rx rings for q vector
* @rxr_idx: index of first Rx ring to assign
*
**/
static int ixgbevf_alloc_q_vector(struct ixgbevf_adapter *adapter, int v_idx,
int txr_count, int txr_idx,
+ int xdp_count, int xdp_idx,
int rxr_count, int rxr_idx)
{
struct ixgbevf_q_vector *q_vector;
+ int reg_idx = txr_idx + xdp_idx;
struct ixgbevf_ring *ring;
int ring_count, size;
- ring_count = txr_count + rxr_count;
+ ring_count = txr_count + xdp_count + rxr_count;
size = sizeof(*q_vector) + (sizeof(*ring) * ring_count);
/* allocate q_vector and rings */
/* apply Tx specific ring traits */
ring->count = adapter->tx_ring_count;
ring->queue_index = txr_idx;
- ring->reg_idx = txr_idx;
+ ring->reg_idx = reg_idx;
/* assign ring to adapter */
adapter->tx_ring[txr_idx] = ring;
/* update count and index */
txr_count--;
txr_idx++;
+ reg_idx++;
+
+ /* push pointer to next ring */
+ ring++;
+ }
+
+ while (xdp_count) {
+ /* assign generic ring traits */
+ ring->dev = &adapter->pdev->dev;
+ ring->netdev = adapter->netdev;
+
+ /* configure backlink on ring */
+ ring->q_vector = q_vector;
+
+ /* update q_vector Tx values */
+ ixgbevf_add_ring(ring, &q_vector->tx);
+
+ /* apply Tx specific ring traits */
+ ring->count = adapter->tx_ring_count;
+ ring->queue_index = xdp_idx;
+ ring->reg_idx = reg_idx;
+ set_ring_xdp(ring);
+
+ /* assign ring to adapter */
+ adapter->xdp_ring[xdp_idx] = ring;
+
+ /* update count and index */
+ xdp_count--;
+ xdp_idx++;
+ reg_idx++;
/* push pointer to next ring */
ring++;
struct ixgbevf_q_vector *q_vector = adapter->q_vector[v_idx];
struct ixgbevf_ring *ring;
- ixgbevf_for_each_ring(ring, q_vector->tx)
- adapter->tx_ring[ring->queue_index] = NULL;
+ ixgbevf_for_each_ring(ring, q_vector->tx) {
+ if (ring_is_xdp(ring))
+ adapter->xdp_ring[ring->queue_index] = NULL;
+ else
+ adapter->tx_ring[ring->queue_index] = NULL;
+ }
ixgbevf_for_each_ring(ring, q_vector->rx)
adapter->rx_ring[ring->queue_index] = NULL;
int q_vectors = adapter->num_msix_vectors - NON_Q_VECTORS;
int rxr_remaining = adapter->num_rx_queues;
int txr_remaining = adapter->num_tx_queues;
- int rxr_idx = 0, txr_idx = 0, v_idx = 0;
+ int xdp_remaining = adapter->num_xdp_queues;
+ int rxr_idx = 0, txr_idx = 0, xdp_idx = 0, v_idx = 0;
int err;
- if (q_vectors >= (rxr_remaining + txr_remaining)) {
+ if (q_vectors >= (rxr_remaining + txr_remaining + xdp_remaining)) {
for (; rxr_remaining; v_idx++, q_vectors--) {
int rqpv = DIV_ROUND_UP(rxr_remaining, q_vectors);
err = ixgbevf_alloc_q_vector(adapter, v_idx,
- 0, 0, rqpv, rxr_idx);
+ 0, 0, 0, 0, rqpv, rxr_idx);
if (err)
goto err_out;
for (; q_vectors; v_idx++, q_vectors--) {
int rqpv = DIV_ROUND_UP(rxr_remaining, q_vectors);
int tqpv = DIV_ROUND_UP(txr_remaining, q_vectors);
+ int xqpv = DIV_ROUND_UP(xdp_remaining, q_vectors);
err = ixgbevf_alloc_q_vector(adapter, v_idx,
tqpv, txr_idx,
+ xqpv, xdp_idx,
rqpv, rxr_idx);
if (err)
rxr_idx += rqpv;
txr_remaining -= tqpv;
txr_idx += tqpv;
+ xdp_remaining -= xqpv;
+ xdp_idx += xqpv;
}
return 0;
goto err_alloc_q_vectors;
}
- hw_dbg(&adapter->hw, "Multiqueue %s: Rx Queue count = %u, Tx Queue count = %u\n",
- (adapter->num_rx_queues > 1) ? "Enabled" :
- "Disabled", adapter->num_rx_queues, adapter->num_tx_queues);
+ hw_dbg(&adapter->hw, "Multiqueue %s: Rx Queue count = %u, Tx Queue count = %u XDP Queue count %u\n",
+ (adapter->num_rx_queues > 1) ? "Enabled" : "Disabled",
+ adapter->num_rx_queues, adapter->num_tx_queues,
+ adapter->num_xdp_queues);
set_bit(__IXGBEVF_DOWN, &adapter->state);
static void ixgbevf_clear_interrupt_scheme(struct ixgbevf_adapter *adapter)
{
adapter->num_tx_queues = 0;
+ adapter->num_xdp_queues = 0;
adapter->num_rx_queues = 0;
ixgbevf_free_q_vectors(adapter);
if (netif_carrier_ok(adapter->netdev)) {
for (i = 0; i < adapter->num_tx_queues; i++)
set_check_for_tx_hang(adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ set_check_for_tx_hang(adapter->xdp_ring[i]);
}
/* get one bit for every active Tx/Rx interrupt vector */
for (i = 0; i < adapter->num_tx_queues; i++)
if (adapter->tx_ring[i]->desc)
ixgbevf_free_tx_resources(adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ if (adapter->xdp_ring[i]->desc)
+ ixgbevf_free_tx_resources(adapter->xdp_ring[i]);
}
/**
**/
static int ixgbevf_setup_all_tx_resources(struct ixgbevf_adapter *adapter)
{
- int i, err = 0;
+ int i, j = 0, err = 0;
for (i = 0; i < adapter->num_tx_queues; i++) {
err = ixgbevf_setup_tx_resources(adapter->tx_ring[i]);
goto err_setup_tx;
}
+ for (j = 0; j < adapter->num_xdp_queues; j++) {
+ err = ixgbevf_setup_tx_resources(adapter->xdp_ring[j]);
+ if (!err)
+ continue;
+ hw_dbg(&adapter->hw, "Allocation for XDP Queue %u failed\n", j);
+ break;
+ }
+
return 0;
err_setup_tx:
/* rewind the index freeing the rings as we go */
+ while (j--)
+ ixgbevf_free_tx_resources(adapter->xdp_ring[j]);
while (i--)
ixgbevf_free_tx_resources(adapter->tx_ring[i]);
+
return err;
}
/**
* ixgbevf_setup_rx_resources - allocate Rx resources (Descriptors)
+ * @adapter: board private structure
* @rx_ring: Rx descriptor ring (for a specific queue) to setup
*
* Returns 0 on success, negative on failure
**/
-int ixgbevf_setup_rx_resources(struct ixgbevf_ring *rx_ring)
+int ixgbevf_setup_rx_resources(struct ixgbevf_adapter *adapter,
+ struct ixgbevf_ring *rx_ring)
{
int size;
if (!rx_ring->desc)
goto err;
+ /* XDP RX-queue info */
+ if (xdp_rxq_info_reg(&rx_ring->xdp_rxq, adapter->netdev,
+ rx_ring->queue_index) < 0)
+ goto err;
+
+ rx_ring->xdp_prog = adapter->xdp_prog;
+
return 0;
err:
vfree(rx_ring->rx_buffer_info);
int i, err = 0;
for (i = 0; i < adapter->num_rx_queues; i++) {
- err = ixgbevf_setup_rx_resources(adapter->rx_ring[i]);
+ err = ixgbevf_setup_rx_resources(adapter, adapter->rx_ring[i]);
if (!err)
continue;
hw_dbg(&adapter->hw, "Allocation for Rx Queue %u failed\n", i);
{
ixgbevf_clean_rx_ring(rx_ring);
+ rx_ring->xdp_prog = NULL;
+ xdp_rxq_info_unreg(&rx_ring->xdp_rxq);
vfree(rx_ring->rx_buffer_info);
rx_ring->rx_buffer_info = NULL;
int max_frame = new_mtu + ETH_HLEN + ETH_FCS_LEN;
int ret;
+ /* prevent MTU being changed to a size unsupported by XDP */
+ if (adapter->xdp_prog) {
+ dev_warn(&adapter->pdev->dev, "MTU cannot be changed while XDP program is loaded\n");
+ return -EPERM;
+ }
+
spin_lock_bh(&adapter->mbx_lock);
/* notify the PF of our intent to use this size of frame */
ret = hw->mac.ops.set_rlpml(hw, max_frame);
ixgbevf_suspend(pdev, PMSG_SUSPEND);
}
+static void ixgbevf_get_tx_ring_stats(struct rtnl_link_stats64 *stats,
+ const struct ixgbevf_ring *ring)
+{
+ u64 bytes, packets;
+ unsigned int start;
+
+ if (ring) {
+ do {
+ start = u64_stats_fetch_begin_irq(&ring->syncp);
+ bytes = ring->stats.bytes;
+ packets = ring->stats.packets;
+ } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+ stats->tx_bytes += bytes;
+ stats->tx_packets += packets;
+ }
+}
+
static void ixgbevf_get_stats(struct net_device *netdev,
struct rtnl_link_stats64 *stats)
{
for (i = 0; i < adapter->num_tx_queues; i++) {
ring = adapter->tx_ring[i];
- do {
- start = u64_stats_fetch_begin_irq(&ring->syncp);
- bytes = ring->stats.bytes;
- packets = ring->stats.packets;
- } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
- stats->tx_bytes += bytes;
- stats->tx_packets += packets;
+ ixgbevf_get_tx_ring_stats(stats, ring);
+ }
+
+ for (i = 0; i < adapter->num_xdp_queues; i++) {
+ ring = adapter->xdp_ring[i];
+ ixgbevf_get_tx_ring_stats(stats, ring);
}
rcu_read_unlock();
}
return features;
}
+static int ixgbevf_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
+{
+ int i, frame_size = dev->mtu + ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN;
+ struct ixgbevf_adapter *adapter = netdev_priv(dev);
+ struct bpf_prog *old_prog;
+
+ /* verify ixgbevf ring attributes are sufficient for XDP */
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ struct ixgbevf_ring *ring = adapter->rx_ring[i];
+
+ if (frame_size > ixgbevf_rx_bufsz(ring))
+ return -EINVAL;
+ }
+
+ old_prog = xchg(&adapter->xdp_prog, prog);
+
+ /* If transitioning XDP modes reconfigure rings */
+ if (!!prog != !!old_prog) {
+ /* Hardware has to reinitialize queues and interrupts to
+ * match packet buffer alignment. Unfortunately, the
+ * hardware is not flexible enough to do this dynamically.
+ */
+ if (netif_running(dev))
+ ixgbevf_close(dev);
+
+ ixgbevf_clear_interrupt_scheme(adapter);
+ ixgbevf_init_interrupt_scheme(adapter);
+
+ if (netif_running(dev))
+ ixgbevf_open(dev);
+ } else {
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ xchg(&adapter->rx_ring[i]->xdp_prog, adapter->xdp_prog);
+ }
+
+ if (old_prog)
+ bpf_prog_put(old_prog);
+
+ return 0;
+}
+
+static int ixgbevf_xdp(struct net_device *dev, struct netdev_bpf *xdp)
+{
+ struct ixgbevf_adapter *adapter = netdev_priv(dev);
+
+ switch (xdp->command) {
+ case XDP_SETUP_PROG:
+ return ixgbevf_xdp_setup(dev, xdp->prog);
+ case XDP_QUERY_PROG:
+ xdp->prog_attached = !!(adapter->xdp_prog);
+ xdp->prog_id = adapter->xdp_prog ?
+ adapter->xdp_prog->aux->id : 0;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
static const struct net_device_ops ixgbevf_netdev_ops = {
.ndo_open = ixgbevf_open,
.ndo_stop = ixgbevf_close,
.ndo_poll_controller = ixgbevf_netpoll,
#endif
.ndo_features_check = ixgbevf_features_check,
+ .ndo_bpf = ixgbevf_xdp,
};
static void ixgbevf_assign_netdev_ops(struct net_device *dev)
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 82599 Virtual Function driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 82599 Virtual Function driver
+/* SPDX-License-Identifier: GPL-2.0 */
/*******************************************************************************
Intel 82599 Virtual Function driver
MODULE_AUTHOR("Rami Rosen <rosenr@marvell.com>, Thomas Petazzoni <thomas.petazzoni@free-electrons.com>");
MODULE_LICENSE("GPL");
-module_param(rxq_number, int, S_IRUGO);
-module_param(txq_number, int, S_IRUGO);
+module_param(rxq_number, int, 0444);
+module_param(txq_number, int, 0444);
-module_param(rxq_def, int, S_IRUGO);
-module_param(rx_copybreak, int, S_IRUGO | S_IWUSR);
+module_param(rxq_def, int, 0444);
+module_param(rx_copybreak, int, 0644);
return readl(priv->swth_base[0] + offset);
}
+static u32 mvpp2_read_relaxed(struct mvpp2 *priv, u32 offset)
+{
+ return readl_relaxed(priv->swth_base[0] + offset);
+}
/* These accessors should be used to access:
*
* - per-CPU registers, where each CPU has its own copy of the
return readl(priv->swth_base[cpu] + offset);
}
+static void mvpp2_percpu_write_relaxed(struct mvpp2 *priv, int cpu,
+ u32 offset, u32 data)
+{
+ writel_relaxed(data, priv->swth_base[cpu] + offset);
+}
+
+static u32 mvpp2_percpu_read_relaxed(struct mvpp2 *priv, int cpu,
+ u32 offset)
+{
+ return readl_relaxed(priv->swth_base[cpu] + offset);
+}
+
static dma_addr_t mvpp2_txdesc_dma_addr_get(struct mvpp2_port *port,
struct mvpp2_tx_desc *tx_desc)
{
return 0;
}
-/* Read tcam entry from hw */
-static int mvpp2_prs_hw_read(struct mvpp2 *priv, struct mvpp2_prs_entry *pe)
+/* Initialize tcam entry from hw */
+static int mvpp2_prs_init_from_hw(struct mvpp2 *priv,
+ struct mvpp2_prs_entry *pe, int tid)
{
int i;
if (pe->index > MVPP2_PRS_TCAM_SRAM_SIZE - 1)
return -EINVAL;
+ memset(pe, 0, sizeof(*pe));
+ pe->index = tid;
+
/* Write tcam index - indirect access */
mvpp2_write(priv, MVPP2_PRS_TCAM_IDX_REG, pe->index);
}
/* Find parser flow entry */
-static struct mvpp2_prs_entry *mvpp2_prs_flow_find(struct mvpp2 *priv, int flow)
+static int mvpp2_prs_flow_find(struct mvpp2 *priv, int flow)
{
- struct mvpp2_prs_entry *pe;
+ struct mvpp2_prs_entry pe;
int tid;
- pe = kzalloc(sizeof(*pe), GFP_KERNEL);
- if (!pe)
- return NULL;
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_FLOWS);
-
/* Go through the all entires with MVPP2_PRS_LU_FLOWS */
for (tid = MVPP2_PRS_TCAM_SRAM_SIZE - 1; tid >= 0; tid--) {
u8 bits;
priv->prs_shadow[tid].lu != MVPP2_PRS_LU_FLOWS)
continue;
- pe->index = tid;
- mvpp2_prs_hw_read(priv, pe);
- bits = mvpp2_prs_sram_ai_get(pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
+ bits = mvpp2_prs_sram_ai_get(&pe);
/* Sram store classification lookup ID in AI bits [5:0] */
if ((bits & MVPP2_PRS_FLOW_ID_MASK) == flow)
- return pe;
+ return tid;
}
- kfree(pe);
- return NULL;
+ return -ENOENT;
}
/* Return first free tcam index, seeking from start to end */
if (priv->prs_shadow[MVPP2_PE_DROP_ALL].valid) {
/* Entry exist - update port only */
- pe.index = MVPP2_PE_DROP_ALL;
- mvpp2_prs_hw_read(priv, &pe);
+ mvpp2_prs_init_from_hw(priv, &pe, MVPP2_PE_DROP_ALL);
} else {
/* Entry doesn't exist - create new */
memset(&pe, 0, sizeof(pe));
/* promiscuous mode - Accept unknown unicast or multicast packets */
if (priv->prs_shadow[tid].valid) {
- pe.index = tid;
- mvpp2_prs_hw_read(priv, &pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
} else {
memset(&pe, 0, sizeof(pe));
mvpp2_prs_tcam_lu_set(&pe, MVPP2_PRS_LU_MAC);
if (priv->prs_shadow[tid].valid) {
/* Entry exist - update port only */
- pe.index = tid;
- mvpp2_prs_hw_read(priv, &pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
} else {
/* Entry doesn't exist - create new */
memset(&pe, 0, sizeof(pe));
if (priv->prs_shadow[tid].valid) {
/* Entry exist - update port only */
- pe.index = tid;
- mvpp2_prs_hw_read(priv, &pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
} else {
/* Entry doesn't exist - create new */
memset(&pe, 0, sizeof(pe));
}
/* Search for existing single/triple vlan entry */
-static struct mvpp2_prs_entry *mvpp2_prs_vlan_find(struct mvpp2 *priv,
- unsigned short tpid, int ai)
+static int mvpp2_prs_vlan_find(struct mvpp2 *priv, unsigned short tpid, int ai)
{
- struct mvpp2_prs_entry *pe;
+ struct mvpp2_prs_entry pe;
int tid;
- pe = kzalloc(sizeof(*pe), GFP_KERNEL);
- if (!pe)
- return NULL;
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_VLAN);
-
/* Go through the all entries with MVPP2_PRS_LU_VLAN */
for (tid = MVPP2_PE_FIRST_FREE_TID;
tid <= MVPP2_PE_LAST_FREE_TID; tid++) {
priv->prs_shadow[tid].lu != MVPP2_PRS_LU_VLAN)
continue;
- pe->index = tid;
-
- mvpp2_prs_hw_read(priv, pe);
- match = mvpp2_prs_tcam_data_cmp(pe, 0, swab16(tpid));
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
+ match = mvpp2_prs_tcam_data_cmp(&pe, 0, swab16(tpid));
if (!match)
continue;
/* Get vlan type */
- ri_bits = mvpp2_prs_sram_ri_get(pe);
+ ri_bits = mvpp2_prs_sram_ri_get(&pe);
ri_bits &= MVPP2_PRS_RI_VLAN_MASK;
/* Get current ai value from tcam */
- ai_bits = mvpp2_prs_tcam_ai_get(pe);
+ ai_bits = mvpp2_prs_tcam_ai_get(&pe);
/* Clear double vlan bit */
ai_bits &= ~MVPP2_PRS_DBL_VLAN_AI_BIT;
if (ri_bits == MVPP2_PRS_RI_VLAN_SINGLE ||
ri_bits == MVPP2_PRS_RI_VLAN_TRIPLE)
- return pe;
+ return tid;
}
- kfree(pe);
- return NULL;
+ return -ENOENT;
}
/* Add/update single/triple vlan entry */
static int mvpp2_prs_vlan_add(struct mvpp2 *priv, unsigned short tpid, int ai,
unsigned int port_map)
{
- struct mvpp2_prs_entry *pe;
+ struct mvpp2_prs_entry pe;
int tid_aux, tid;
int ret = 0;
- pe = mvpp2_prs_vlan_find(priv, tpid, ai);
+ memset(&pe, 0, sizeof(pe));
+
+ tid = mvpp2_prs_vlan_find(priv, tpid, ai);
- if (!pe) {
+ if (tid < 0) {
/* Create new tcam entry */
tid = mvpp2_prs_tcam_first_free(priv, MVPP2_PE_LAST_FREE_TID,
MVPP2_PE_FIRST_FREE_TID);
if (tid < 0)
return tid;
- pe = kzalloc(sizeof(*pe), GFP_KERNEL);
- if (!pe)
- return -ENOMEM;
-
/* Get last double vlan tid */
for (tid_aux = MVPP2_PE_LAST_FREE_TID;
tid_aux >= MVPP2_PE_FIRST_FREE_TID; tid_aux--) {
priv->prs_shadow[tid_aux].lu != MVPP2_PRS_LU_VLAN)
continue;
- pe->index = tid_aux;
- mvpp2_prs_hw_read(priv, pe);
- ri_bits = mvpp2_prs_sram_ri_get(pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid_aux);
+ ri_bits = mvpp2_prs_sram_ri_get(&pe);
if ((ri_bits & MVPP2_PRS_RI_VLAN_MASK) ==
MVPP2_PRS_RI_VLAN_DOUBLE)
break;
}
- if (tid <= tid_aux) {
- ret = -EINVAL;
- goto free_pe;
- }
+ if (tid <= tid_aux)
+ return -EINVAL;
- memset(pe, 0, sizeof(*pe));
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_VLAN);
- pe->index = tid;
+ memset(&pe, 0, sizeof(pe));
+ pe.index = tid;
+ mvpp2_prs_tcam_lu_set(&pe, MVPP2_PRS_LU_VLAN);
- mvpp2_prs_match_etype(pe, 0, tpid);
+ mvpp2_prs_match_etype(&pe, 0, tpid);
/* VLAN tag detected, proceed with VID filtering */
- mvpp2_prs_sram_next_lu_set(pe, MVPP2_PRS_LU_VID);
+ mvpp2_prs_sram_next_lu_set(&pe, MVPP2_PRS_LU_VID);
/* Clear all ai bits for next iteration */
- mvpp2_prs_sram_ai_update(pe, 0, MVPP2_PRS_SRAM_AI_MASK);
+ mvpp2_prs_sram_ai_update(&pe, 0, MVPP2_PRS_SRAM_AI_MASK);
if (ai == MVPP2_PRS_SINGLE_VLAN_AI) {
- mvpp2_prs_sram_ri_update(pe, MVPP2_PRS_RI_VLAN_SINGLE,
+ mvpp2_prs_sram_ri_update(&pe, MVPP2_PRS_RI_VLAN_SINGLE,
MVPP2_PRS_RI_VLAN_MASK);
} else {
ai |= MVPP2_PRS_DBL_VLAN_AI_BIT;
- mvpp2_prs_sram_ri_update(pe, MVPP2_PRS_RI_VLAN_TRIPLE,
+ mvpp2_prs_sram_ri_update(&pe, MVPP2_PRS_RI_VLAN_TRIPLE,
MVPP2_PRS_RI_VLAN_MASK);
}
- mvpp2_prs_tcam_ai_update(pe, ai, MVPP2_PRS_SRAM_AI_MASK);
+ mvpp2_prs_tcam_ai_update(&pe, ai, MVPP2_PRS_SRAM_AI_MASK);
- mvpp2_prs_shadow_set(priv, pe->index, MVPP2_PRS_LU_VLAN);
+ mvpp2_prs_shadow_set(priv, pe.index, MVPP2_PRS_LU_VLAN);
+ } else {
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
}
/* Update ports' mask */
- mvpp2_prs_tcam_port_map_set(pe, port_map);
+ mvpp2_prs_tcam_port_map_set(&pe, port_map);
- mvpp2_prs_hw_write(priv, pe);
-free_pe:
- kfree(pe);
+ mvpp2_prs_hw_write(priv, &pe);
return ret;
}
}
/* Search for existing double vlan entry */
-static struct mvpp2_prs_entry *mvpp2_prs_double_vlan_find(struct mvpp2 *priv,
- unsigned short tpid1,
- unsigned short tpid2)
+static int mvpp2_prs_double_vlan_find(struct mvpp2 *priv, unsigned short tpid1,
+ unsigned short tpid2)
{
- struct mvpp2_prs_entry *pe;
+ struct mvpp2_prs_entry pe;
int tid;
- pe = kzalloc(sizeof(*pe), GFP_KERNEL);
- if (!pe)
- return NULL;
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_VLAN);
-
/* Go through the all entries with MVPP2_PRS_LU_VLAN */
for (tid = MVPP2_PE_FIRST_FREE_TID;
tid <= MVPP2_PE_LAST_FREE_TID; tid++) {
priv->prs_shadow[tid].lu != MVPP2_PRS_LU_VLAN)
continue;
- pe->index = tid;
- mvpp2_prs_hw_read(priv, pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
- match = mvpp2_prs_tcam_data_cmp(pe, 0, swab16(tpid1))
- && mvpp2_prs_tcam_data_cmp(pe, 4, swab16(tpid2));
+ match = mvpp2_prs_tcam_data_cmp(&pe, 0, swab16(tpid1)) &&
+ mvpp2_prs_tcam_data_cmp(&pe, 4, swab16(tpid2));
if (!match)
continue;
- ri_mask = mvpp2_prs_sram_ri_get(pe) & MVPP2_PRS_RI_VLAN_MASK;
+ ri_mask = mvpp2_prs_sram_ri_get(&pe) & MVPP2_PRS_RI_VLAN_MASK;
if (ri_mask == MVPP2_PRS_RI_VLAN_DOUBLE)
- return pe;
+ return tid;
}
- kfree(pe);
- return NULL;
+ return -ENOENT;
}
/* Add or update double vlan entry */
unsigned short tpid2,
unsigned int port_map)
{
- struct mvpp2_prs_entry *pe;
int tid_aux, tid, ai, ret = 0;
+ struct mvpp2_prs_entry pe;
+
+ memset(&pe, 0, sizeof(pe));
- pe = mvpp2_prs_double_vlan_find(priv, tpid1, tpid2);
+ tid = mvpp2_prs_double_vlan_find(priv, tpid1, tpid2);
- if (!pe) {
+ if (tid < 0) {
/* Create new tcam entry */
tid = mvpp2_prs_tcam_first_free(priv, MVPP2_PE_FIRST_FREE_TID,
MVPP2_PE_LAST_FREE_TID);
if (tid < 0)
return tid;
- pe = kzalloc(sizeof(*pe), GFP_KERNEL);
- if (!pe)
- return -ENOMEM;
-
/* Set ai value for new double vlan entry */
ai = mvpp2_prs_double_vlan_ai_free_get(priv);
- if (ai < 0) {
- ret = ai;
- goto free_pe;
- }
+ if (ai < 0)
+ return ai;
/* Get first single/triple vlan tid */
for (tid_aux = MVPP2_PE_FIRST_FREE_TID;
priv->prs_shadow[tid_aux].lu != MVPP2_PRS_LU_VLAN)
continue;
- pe->index = tid_aux;
- mvpp2_prs_hw_read(priv, pe);
- ri_bits = mvpp2_prs_sram_ri_get(pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid_aux);
+ ri_bits = mvpp2_prs_sram_ri_get(&pe);
ri_bits &= MVPP2_PRS_RI_VLAN_MASK;
if (ri_bits == MVPP2_PRS_RI_VLAN_SINGLE ||
ri_bits == MVPP2_PRS_RI_VLAN_TRIPLE)
break;
}
- if (tid >= tid_aux) {
- ret = -ERANGE;
- goto free_pe;
- }
+ if (tid >= tid_aux)
+ return -ERANGE;
- memset(pe, 0, sizeof(*pe));
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_VLAN);
- pe->index = tid;
+ memset(&pe, 0, sizeof(pe));
+ mvpp2_prs_tcam_lu_set(&pe, MVPP2_PRS_LU_VLAN);
+ pe.index = tid;
priv->prs_double_vlans[ai] = true;
- mvpp2_prs_match_etype(pe, 0, tpid1);
- mvpp2_prs_match_etype(pe, 4, tpid2);
+ mvpp2_prs_match_etype(&pe, 0, tpid1);
+ mvpp2_prs_match_etype(&pe, 4, tpid2);
- mvpp2_prs_sram_next_lu_set(pe, MVPP2_PRS_LU_VLAN);
+ mvpp2_prs_sram_next_lu_set(&pe, MVPP2_PRS_LU_VLAN);
/* Shift 4 bytes - skip outer vlan tag */
- mvpp2_prs_sram_shift_set(pe, MVPP2_VLAN_TAG_LEN,
+ mvpp2_prs_sram_shift_set(&pe, MVPP2_VLAN_TAG_LEN,
MVPP2_PRS_SRAM_OP_SEL_SHIFT_ADD);
- mvpp2_prs_sram_ri_update(pe, MVPP2_PRS_RI_VLAN_DOUBLE,
+ mvpp2_prs_sram_ri_update(&pe, MVPP2_PRS_RI_VLAN_DOUBLE,
MVPP2_PRS_RI_VLAN_MASK);
- mvpp2_prs_sram_ai_update(pe, ai | MVPP2_PRS_DBL_VLAN_AI_BIT,
+ mvpp2_prs_sram_ai_update(&pe, ai | MVPP2_PRS_DBL_VLAN_AI_BIT,
MVPP2_PRS_SRAM_AI_MASK);
- mvpp2_prs_shadow_set(priv, pe->index, MVPP2_PRS_LU_VLAN);
+ mvpp2_prs_shadow_set(priv, pe.index, MVPP2_PRS_LU_VLAN);
+ } else {
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
}
/* Update ports' mask */
- mvpp2_prs_tcam_port_map_set(pe, port_map);
- mvpp2_prs_hw_write(priv, pe);
-free_pe:
- kfree(pe);
+ mvpp2_prs_tcam_port_map_set(&pe, port_map);
+ mvpp2_prs_hw_write(priv, &pe);
+
return ret;
}
priv->prs_shadow[tid].lu != MVPP2_PRS_LU_VID)
continue;
- pe.index = tid;
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
- mvpp2_prs_hw_read(priv, &pe);
mvpp2_prs_tcam_data_byte_get(&pe, 2, &byte[0], &enable[0]);
mvpp2_prs_tcam_data_byte_get(&pe, 3, &byte[1], &enable[1]);
return tid;
}
- return 0;
+ return -ENOENT;
}
/* Write parser entry for VID filtering */
struct mvpp2_prs_entry pe;
int tid;
+ memset(&pe, 0, sizeof(pe));
+
/* Scan TCAM and see if entry with this <vid,port> already exist */
tid = mvpp2_prs_vid_range_find(priv, (1 << port->id), vid, mask);
shift = MVPP2_VLAN_TAG_LEN;
/* No such entry */
- if (!tid) {
- memset(&pe, 0, sizeof(pe));
+ if (tid < 0) {
/* Go through all entries from first to last in vlan range */
tid = mvpp2_prs_tcam_first_free(priv, vid_start,
/* Mask all ports */
mvpp2_prs_tcam_port_map_set(&pe, 0);
} else {
- mvpp2_prs_hw_read(priv, &pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
}
/* Enable the current port */
tid = mvpp2_prs_vid_range_find(priv, (1 << port->id), vid, 0xfff);
/* No such entry */
- if (tid)
+ if (tid < 0)
return;
mvpp2_prs_hw_inv(priv, tid);
}
/* Find tcam entry with matched pair <MAC DA, port> */
-static struct mvpp2_prs_entry *
+static int
mvpp2_prs_mac_da_range_find(struct mvpp2 *priv, int pmap, const u8 *da,
unsigned char *mask, int udf_type)
{
- struct mvpp2_prs_entry *pe;
+ struct mvpp2_prs_entry pe;
int tid;
- pe = kzalloc(sizeof(*pe), GFP_ATOMIC);
- if (!pe)
- return NULL;
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_MAC);
-
/* Go through the all entires with MVPP2_PRS_LU_MAC */
for (tid = MVPP2_PE_MAC_RANGE_START;
tid <= MVPP2_PE_MAC_RANGE_END; tid++) {
(priv->prs_shadow[tid].udf != udf_type))
continue;
- pe->index = tid;
- mvpp2_prs_hw_read(priv, pe);
- entry_pmap = mvpp2_prs_tcam_port_map_get(pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
+ entry_pmap = mvpp2_prs_tcam_port_map_get(&pe);
- if (mvpp2_prs_mac_range_equals(pe, da, mask) &&
+ if (mvpp2_prs_mac_range_equals(&pe, da, mask) &&
entry_pmap == pmap)
- return pe;
+ return tid;
}
- kfree(pe);
- return NULL;
+ return -ENOENT;
}
/* Update parser's mac da entry */
unsigned char mask[ETH_ALEN] = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
struct mvpp2 *priv = port->priv;
unsigned int pmap, len, ri;
- struct mvpp2_prs_entry *pe;
+ struct mvpp2_prs_entry pe;
int tid;
+ memset(&pe, 0, sizeof(pe));
+
/* Scan TCAM and see if entry with this <MAC DA, port> already exist */
- pe = mvpp2_prs_mac_da_range_find(priv, BIT(port->id), da, mask,
- MVPP2_PRS_UDF_MAC_DEF);
+ tid = mvpp2_prs_mac_da_range_find(priv, BIT(port->id), da, mask,
+ MVPP2_PRS_UDF_MAC_DEF);
/* No such entry */
- if (!pe) {
+ if (tid < 0) {
if (!add)
return 0;
if (tid < 0)
return tid;
- pe = kzalloc(sizeof(*pe), GFP_ATOMIC);
- if (!pe)
- return -ENOMEM;
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_MAC);
- pe->index = tid;
+ pe.index = tid;
/* Mask all ports */
- mvpp2_prs_tcam_port_map_set(pe, 0);
+ mvpp2_prs_tcam_port_map_set(&pe, 0);
+ } else {
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
}
+ mvpp2_prs_tcam_lu_set(&pe, MVPP2_PRS_LU_MAC);
+
/* Update port mask */
- mvpp2_prs_tcam_port_set(pe, port->id, add);
+ mvpp2_prs_tcam_port_set(&pe, port->id, add);
/* Invalidate the entry if no ports are left enabled */
- pmap = mvpp2_prs_tcam_port_map_get(pe);
+ pmap = mvpp2_prs_tcam_port_map_get(&pe);
if (pmap == 0) {
- if (add) {
- kfree(pe);
+ if (add)
return -EINVAL;
- }
- mvpp2_prs_hw_inv(priv, pe->index);
- priv->prs_shadow[pe->index].valid = false;
- kfree(pe);
+
+ mvpp2_prs_hw_inv(priv, pe.index);
+ priv->prs_shadow[pe.index].valid = false;
return 0;
}
/* Continue - set next lookup */
- mvpp2_prs_sram_next_lu_set(pe, MVPP2_PRS_LU_DSA);
+ mvpp2_prs_sram_next_lu_set(&pe, MVPP2_PRS_LU_DSA);
/* Set match on DA */
len = ETH_ALEN;
while (len--)
- mvpp2_prs_tcam_data_byte_set(pe, len, da[len], 0xff);
+ mvpp2_prs_tcam_data_byte_set(&pe, len, da[len], 0xff);
/* Set result info bits */
if (is_broadcast_ether_addr(da)) {
ri |= MVPP2_PRS_RI_MAC_ME_MASK;
}
- mvpp2_prs_sram_ri_update(pe, ri, MVPP2_PRS_RI_L2_CAST_MASK |
+ mvpp2_prs_sram_ri_update(&pe, ri, MVPP2_PRS_RI_L2_CAST_MASK |
MVPP2_PRS_RI_MAC_ME_MASK);
- mvpp2_prs_shadow_ri_set(priv, pe->index, ri, MVPP2_PRS_RI_L2_CAST_MASK |
+ mvpp2_prs_shadow_ri_set(priv, pe.index, ri, MVPP2_PRS_RI_L2_CAST_MASK |
MVPP2_PRS_RI_MAC_ME_MASK);
/* Shift to ethertype */
- mvpp2_prs_sram_shift_set(pe, 2 * ETH_ALEN,
+ mvpp2_prs_sram_shift_set(&pe, 2 * ETH_ALEN,
MVPP2_PRS_SRAM_OP_SEL_SHIFT_ADD);
/* Update shadow table and hw entry */
- priv->prs_shadow[pe->index].udf = MVPP2_PRS_UDF_MAC_DEF;
- mvpp2_prs_shadow_set(priv, pe->index, MVPP2_PRS_LU_MAC);
- mvpp2_prs_hw_write(priv, pe);
-
- kfree(pe);
+ priv->prs_shadow[pe.index].udf = MVPP2_PRS_UDF_MAC_DEF;
+ mvpp2_prs_shadow_set(priv, pe.index, MVPP2_PRS_LU_MAC);
+ mvpp2_prs_hw_write(priv, &pe);
return 0;
}
(priv->prs_shadow[tid].udf != MVPP2_PRS_UDF_MAC_DEF))
continue;
- pe.index = tid;
- mvpp2_prs_hw_read(priv, &pe);
+ mvpp2_prs_init_from_hw(priv, &pe, tid);
pmap = mvpp2_prs_tcam_port_map_get(&pe);
/* Set prs flow for the port */
static int mvpp2_prs_def_flow(struct mvpp2_port *port)
{
- struct mvpp2_prs_entry *pe;
+ struct mvpp2_prs_entry pe;
int tid;
- pe = mvpp2_prs_flow_find(port->priv, port->id);
+ memset(&pe, 0, sizeof(pe));
+
+ tid = mvpp2_prs_flow_find(port->priv, port->id);
/* Such entry not exist */
- if (!pe) {
+ if (tid < 0) {
/* Go through the all entires from last to first */
tid = mvpp2_prs_tcam_first_free(port->priv,
MVPP2_PE_LAST_FREE_TID,
if (tid < 0)
return tid;
- pe = kzalloc(sizeof(*pe), GFP_KERNEL);
- if (!pe)
- return -ENOMEM;
-
- mvpp2_prs_tcam_lu_set(pe, MVPP2_PRS_LU_FLOWS);
- pe->index = tid;
+ pe.index = tid;
/* Set flow ID*/
- mvpp2_prs_sram_ai_update(pe, port->id, MVPP2_PRS_FLOW_ID_MASK);
- mvpp2_prs_sram_bits_set(pe, MVPP2_PRS_SRAM_LU_DONE_BIT, 1);
+ mvpp2_prs_sram_ai_update(&pe, port->id, MVPP2_PRS_FLOW_ID_MASK);
+ mvpp2_prs_sram_bits_set(&pe, MVPP2_PRS_SRAM_LU_DONE_BIT, 1);
/* Update shadow table */
- mvpp2_prs_shadow_set(port->priv, pe->index, MVPP2_PRS_LU_FLOWS);
+ mvpp2_prs_shadow_set(port->priv, pe.index, MVPP2_PRS_LU_FLOWS);
+ } else {
+ mvpp2_prs_init_from_hw(port->priv, &pe, tid);
}
- mvpp2_prs_tcam_port_map_set(pe, (1 << port->id));
- mvpp2_prs_hw_write(port->priv, pe);
- kfree(pe);
+ mvpp2_prs_tcam_lu_set(&pe, MVPP2_PRS_LU_FLOWS);
+ mvpp2_prs_tcam_port_map_set(&pe, (1 << port->id));
+ mvpp2_prs_hw_write(port->priv, &pe);
return 0;
}
<< MVPP22_BM_ADDR_HIGH_VIRT_RLS_SHIFT) &
MVPP22_BM_ADDR_HIGH_VIRT_RLS_MASK;
- mvpp2_percpu_write(port->priv, cpu,
- MVPP22_BM_ADDR_HIGH_RLS_REG, val);
+ mvpp2_percpu_write_relaxed(port->priv, cpu,
+ MVPP22_BM_ADDR_HIGH_RLS_REG, val);
}
/* MVPP2_BM_VIRT_RLS_REG is not interpreted by HW, and simply
* descriptor. Instead of storing the virtual address, we
* store the physical address
*/
- mvpp2_percpu_write(port->priv, cpu,
- MVPP2_BM_VIRT_RLS_REG, buf_phys_addr);
- mvpp2_percpu_write(port->priv, cpu,
- MVPP2_BM_PHY_RLS_REG(pool), buf_dma_addr);
+ mvpp2_percpu_write_relaxed(port->priv, cpu,
+ MVPP2_BM_VIRT_RLS_REG, buf_phys_addr);
+ mvpp2_percpu_write_relaxed(port->priv, cpu,
+ MVPP2_BM_PHY_RLS_REG(pool), buf_dma_addr);
put_cpu();
}
if (!port->pool_short) {
port->pool_short =
mvpp2_bm_pool_use(port, short_log_pool,
- mvpp2_pools[long_log_pool].pkt_size);
+ mvpp2_pools[short_log_pool].pkt_size);
if (!port->pool_short)
return -ENOMEM;
if ((aggr_txq->count + num) > MVPP2_AGGR_TXQ_SIZE) {
/* Update number of occupied aggregated Tx descriptors */
int cpu = smp_processor_id();
- u32 val = mvpp2_read(priv, MVPP2_AGGR_TXQ_STATUS_REG(cpu));
+ u32 val = mvpp2_read_relaxed(priv,
+ MVPP2_AGGR_TXQ_STATUS_REG(cpu));
aggr_txq->count = val & MVPP2_AGGR_TXQ_PENDING_MASK;
}
int cpu = smp_processor_id();
val = (txq->id << MVPP2_TXQ_RSVD_REQ_Q_OFFSET) | num;
- mvpp2_percpu_write(priv, cpu, MVPP2_TXQ_RSVD_REQ_REG, val);
+ mvpp2_percpu_write_relaxed(priv, cpu, MVPP2_TXQ_RSVD_REQ_REG, val);
- val = mvpp2_percpu_read(priv, cpu, MVPP2_TXQ_RSVD_RSLT_REG);
+ val = mvpp2_percpu_read_relaxed(priv, cpu, MVPP2_TXQ_RSVD_RSLT_REG);
return val & MVPP2_TXQ_RSVD_RSLT_MASK;
}
u32 val;
/* Reading status reg resets transmitted descriptor counter */
- val = mvpp2_percpu_read(port->priv, smp_processor_id(),
- MVPP2_TXQ_SENT_REG(txq->id));
+ val = mvpp2_percpu_read_relaxed(port->priv, smp_processor_id(),
+ MVPP2_TXQ_SENT_REG(txq->id));
return (val & MVPP2_TRANSMITTED_COUNT_MASK) >>
MVPP2_TRANSMITTED_COUNT_OFFSET;
*
* Each CPU has its own Rx/Tx cause register
*/
- cause_rx_tx = mvpp2_percpu_read(port->priv, qv->sw_thread_id,
- MVPP2_ISR_RX_TX_CAUSE_REG(port->id));
+ cause_rx_tx = mvpp2_percpu_read_relaxed(port->priv, qv->sw_thread_id,
+ MVPP2_ISR_RX_TX_CAUSE_REG(port->id));
cause_misc = cause_rx_tx & MVPP2_CAUSE_MISC_SUM_MASK;
if (cause_misc) {
break;
case NETDEV_UP:
- d = debugfs_create_file(dev->name, S_IRUGO,
+ d = debugfs_create_file(dev->name, 0444,
skge_debug, dev,
&skge_debug_fops);
if (!d || IS_ERR(d))
break;
case NETDEV_UP:
- sky2->debugfs = debugfs_create_file(dev->name, S_IRUGO,
+ sky2->debugfs = debugfs_create_file(dev->name, 0444,
sky2_debug, dev,
&sky2_debug_fops);
if (IS_ERR(sky2->debugfs))
sprintf(info->dev_name, "mlx4_port%d", port);
info->port_attr.attr.name = info->dev_name;
- if (mlx4_is_mfunc(dev))
- info->port_attr.attr.mode = S_IRUGO;
- else {
- info->port_attr.attr.mode = S_IRUGO | S_IWUSR;
+ if (mlx4_is_mfunc(dev)) {
+ info->port_attr.attr.mode = 0444;
+ } else {
+ info->port_attr.attr.mode = 0644;
info->port_attr.store = set_port_type;
}
info->port_attr.show = show_port_type;
sprintf(info->dev_mtu_name, "mlx4_port%d_mtu", port);
info->port_mtu_attr.attr.name = info->dev_mtu_name;
- if (mlx4_is_mfunc(dev))
- info->port_mtu_attr.attr.mode = S_IRUGO;
- else {
- info->port_mtu_attr.attr.mode = S_IRUGO | S_IWUSR;
+ if (mlx4_is_mfunc(dev)) {
+ info->port_mtu_attr.attr.mode = 0444;
+ } else {
+ info->port_mtu_attr.attr.mode = 0644;
info->port_mtu_attr.store = set_port_ib_mtu;
}
info->port_mtu_attr.show = show_port_ib_mtu;
case MLX5_CMD_OP_MODIFY_HCA_VPORT_CONTEXT:
case MLX5_CMD_OP_QUERY_HCA_VPORT_GID:
case MLX5_CMD_OP_QUERY_HCA_VPORT_PKEY:
+ case MLX5_CMD_OP_QUERY_VNIC_ENV:
case MLX5_CMD_OP_QUERY_VPORT_COUNTER:
case MLX5_CMD_OP_ALLOC_Q_COUNTER:
case MLX5_CMD_OP_QUERY_Q_COUNTER:
MLX5_COMMAND_STR_CASE(MODIFY_HCA_VPORT_CONTEXT);
MLX5_COMMAND_STR_CASE(QUERY_HCA_VPORT_GID);
MLX5_COMMAND_STR_CASE(QUERY_HCA_VPORT_PKEY);
+ MLX5_COMMAND_STR_CASE(QUERY_VNIC_ENV);
MLX5_COMMAND_STR_CASE(QUERY_VPORT_COUNTER);
MLX5_COMMAND_STR_CASE(ALLOC_Q_COUNTER);
MLX5_COMMAND_STR_CASE(DEALLOC_Q_COUNTER);
cmd->checksum_disabled = 1;
cmd->max_reg_cmds = (1 << cmd->log_sz) - 1;
- cmd->bitmask = (1 << cmd->max_reg_cmds) - 1;
+ cmd->bitmask = (1UL << cmd->max_reg_cmds) - 1;
cmd->cmdif_rev = ioread32be(&dev->iseg->cmdif_rev_fw_sub) >> 16;
if (cmd->cmdif_rev > CMD_IF_REV) {
{MLX5_FLOW_CONTEXT_ACTION_ENCAP, "ENCAP"},\
{MLX5_FLOW_CONTEXT_ACTION_DECAP, "DECAP"},\
{MLX5_FLOW_CONTEXT_ACTION_MOD_HDR, "MOD_HDR"},\
+ {MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH, "VLAN_PUSH"},\
+ {MLX5_FLOW_CONTEXT_ACTION_VLAN_POP, "VLAN_POP"},\
{MLX5_FLOW_CONTEXT_ACTION_FWD_NEXT_PRIO, "NEXT_PRIO"}
TRACE_EVENT(mlx5_fs_set_fte,
#define MLX5_MPWRQ_WQE_PAGE_ORDER (MLX5_MPWRQ_LOG_WQE_SZ - PAGE_SHIFT > 0 ? \
MLX5_MPWRQ_LOG_WQE_SZ - PAGE_SHIFT : 0)
#define MLX5_MPWRQ_PAGES_PER_WQE BIT(MLX5_MPWRQ_WQE_PAGE_ORDER)
-#define MLX5_MPWRQ_STRIDES_PER_PAGE (MLX5_MPWRQ_NUM_STRIDES >> \
- MLX5_MPWRQ_WQE_PAGE_ORDER)
#define MLX5_MTT_OCTW(npages) (ALIGN(npages, 8) / 2)
#define MLX5E_REQUIRED_MTTS(wqes) \
#define MLX5E_MAX_NUM_SQS (MLX5E_MAX_NUM_CHANNELS * MLX5E_MAX_NUM_TC)
#define MLX5E_TX_CQ_POLL_BUDGET 128
#define MLX5E_UPDATE_STATS_INTERVAL 200 /* msecs */
+#define MLX5E_SQ_RECOVER_MIN_INTERVAL 500 /* msecs */
#define MLX5E_ICOSQ_MAX_WQEBBS \
(DIV_ROUND_UP(sizeof(struct mlx5e_umr_wqe), MLX5_SEND_WQE_BB))
"rx_cqe_moder",
"tx_cqe_moder",
"rx_cqe_compress",
+ "rx_striding_rq",
};
enum mlx5e_priv_flag {
MLX5E_PFLAG_RX_CQE_BASED_MODER = (1 << 0),
MLX5E_PFLAG_TX_CQE_BASED_MODER = (1 << 1),
MLX5E_PFLAG_RX_CQE_COMPRESS = (1 << 2),
+ MLX5E_PFLAG_RX_STRIDING_RQ = (1 << 3),
};
#define MLX5E_SET_PFLAG(params, pflag, enable) \
struct mlx5e_params {
u8 log_sq_size;
u8 rq_wq_type;
- u16 rq_headroom;
- u8 mpwqe_log_stride_sz;
- u8 mpwqe_log_num_strides;
u8 log_rq_size;
u16 num_channels;
u8 num_tc;
struct net_dim_cq_moder tx_cq_moderation;
bool lro_en;
u32 lro_wqe_sz;
- u16 tx_max_inline;
u8 tx_min_inline_mode;
u8 rss_hfunc;
u8 toeplitz_hash_key[40];
enum {
MLX5E_SQ_STATE_ENABLED,
+ MLX5E_SQ_STATE_RECOVERING,
MLX5E_SQ_STATE_IPSEC,
};
void __iomem *uar_map;
struct netdev_queue *txq;
u32 sqn;
- u16 max_inline;
u8 min_inline_mode;
u16 edge;
struct device *pdev;
struct mlx5e_channel *channel;
int txq_ix;
u32 rate_limit;
+ struct mlx5e_txqsq_recover {
+ struct work_struct recover_work;
+ u64 last_recover;
+ } recover;
} ____cacheline_aligned_in_smp;
struct mlx5e_xdpsq {
struct net_device *netdev;
struct mlx5e_stats stats;
struct hwtstamp_config tstamp;
- u16 q_counter;
+ u16 q_counter;
+ u16 drop_rq_q_counter;
#ifdef CONFIG_MLX5_CORE_EN_DCB
struct mlx5e_dcbx dcbx;
#endif
void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq);
void mlx5e_free_xdpsq_descs(struct mlx5e_xdpsq *sq);
+bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev);
+bool mlx5e_striding_rq_possible(struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params);
+
void mlx5e_page_release(struct mlx5e_rq *rq, struct mlx5e_dma_info *dma_info,
bool recycle);
void mlx5e_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe);
void mlx5e_dealloc_rx_mpwqe(struct mlx5e_rq *rq, u16 ix);
void mlx5e_free_rx_mpwqe(struct mlx5e_rq *rq, struct mlx5e_mpw_info *wi);
+u8 mlx5e_mpwqe_get_log_stride_size(struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params);
+u8 mlx5e_mpwqe_get_log_num_strides(struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params);
+
void mlx5e_update_stats(struct mlx5e_priv *priv);
int mlx5e_create_flow_steering(struct mlx5e_priv *priv);
u8 cq_period_mode);
void mlx5e_set_rx_cq_mode_params(struct mlx5e_params *params,
u8 cq_period_mode);
+void mlx5e_set_rq_type(struct mlx5_core_dev *mdev, struct mlx5e_params *params);
void mlx5e_init_rq_type_params(struct mlx5_core_dev *mdev,
- struct mlx5e_params *params,
- u8 rq_type);
+ struct mlx5e_params *params);
static inline bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev)
{
u16 rxq_index, u32 flow_id);
#endif
-u16 mlx5e_get_max_inline_cap(struct mlx5_core_dev *mdev);
int mlx5e_create_tir(struct mlx5_core_dev *mdev,
struct mlx5e_tir *tir, u32 *in, int inlen);
void mlx5e_destroy_tir(struct mlx5_core_dev *mdev,
int mlx5e_close(struct net_device *netdev);
int mlx5e_open(struct net_device *netdev);
void mlx5e_update_stats_work(struct work_struct *work);
-u32 mlx5e_choose_lro_timeout(struct mlx5_core_dev *mdev, u32 wanted_timeout);
int mlx5e_bits_invert(unsigned long a, int size);
{
int i, idx = 0;
- if (!data)
- return;
-
mutex_lock(&priv->state_lock);
mlx5e_update_stats(priv);
mutex_unlock(&priv->state_lock);
if (rq_wq_type != MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ)
return num_wqe;
- stride_size = 1 << priv->channels.params.mpwqe_log_stride_sz;
- num_strides = 1 << priv->channels.params.mpwqe_log_num_strides;
+ stride_size = 1 << mlx5e_mpwqe_get_log_stride_size(priv->mdev, &priv->channels.params);
+ num_strides = 1 << mlx5e_mpwqe_get_log_num_strides(priv->mdev, &priv->channels.params);
wqe_size = stride_size * num_strides;
packets_per_wqe = wqe_size /
if (rq_wq_type != MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ)
return num_packets;
- stride_size = 1 << priv->channels.params.mpwqe_log_stride_sz;
- num_strides = 1 << priv->channels.params.mpwqe_log_num_strides;
+ stride_size = 1 << mlx5e_mpwqe_get_log_stride_size(priv->mdev, &priv->channels.params);
+ num_strides = 1 << mlx5e_mpwqe_get_log_num_strides(priv->mdev, &priv->channels.params);
wqe_size = stride_size * num_strides;
num_packets = (1 << order_base_2(num_packets));
return err;
}
+#define MLX5E_PFC_PREVEN_AUTO_TOUT_MSEC 100
+#define MLX5E_PFC_PREVEN_TOUT_MAX_MSEC 8000
+#define MLX5E_PFC_PREVEN_MINOR_PRECENT 85
+#define MLX5E_PFC_PREVEN_TOUT_MIN_MSEC 80
+#define MLX5E_DEVICE_STALL_MINOR_WATERMARK(critical_tout) \
+ max_t(u16, MLX5E_PFC_PREVEN_TOUT_MIN_MSEC, \
+ (critical_tout * MLX5E_PFC_PREVEN_MINOR_PRECENT) / 100)
+
+static int mlx5e_get_pfc_prevention_tout(struct net_device *netdev,
+ u16 *pfc_prevention_tout)
+{
+ struct mlx5e_priv *priv = netdev_priv(netdev);
+ struct mlx5_core_dev *mdev = priv->mdev;
+
+ if (!MLX5_CAP_PCAM_FEATURE((priv)->mdev, pfcc_mask) ||
+ !MLX5_CAP_DEBUG((priv)->mdev, stall_detect))
+ return -EOPNOTSUPP;
+
+ return mlx5_query_port_stall_watermark(mdev, pfc_prevention_tout, NULL);
+}
+
+static int mlx5e_set_pfc_prevention_tout(struct net_device *netdev,
+ u16 pfc_preven)
+{
+ struct mlx5e_priv *priv = netdev_priv(netdev);
+ struct mlx5_core_dev *mdev = priv->mdev;
+ u16 critical_tout;
+ u16 minor;
+
+ if (!MLX5_CAP_PCAM_FEATURE((priv)->mdev, pfcc_mask) ||
+ !MLX5_CAP_DEBUG((priv)->mdev, stall_detect))
+ return -EOPNOTSUPP;
+
+ critical_tout = (pfc_preven == PFC_STORM_PREVENTION_AUTO) ?
+ MLX5E_PFC_PREVEN_AUTO_TOUT_MSEC :
+ pfc_preven;
+
+ if (critical_tout != PFC_STORM_PREVENTION_DISABLE &&
+ (critical_tout > MLX5E_PFC_PREVEN_TOUT_MAX_MSEC ||
+ critical_tout < MLX5E_PFC_PREVEN_TOUT_MIN_MSEC)) {
+ netdev_info(netdev, "%s: pfc prevention tout not in range (%d-%d)\n",
+ __func__, MLX5E_PFC_PREVEN_TOUT_MIN_MSEC,
+ MLX5E_PFC_PREVEN_TOUT_MAX_MSEC);
+ return -EINVAL;
+ }
+
+ minor = MLX5E_DEVICE_STALL_MINOR_WATERMARK(critical_tout);
+ return mlx5_set_port_stall_watermark(mdev, critical_tout,
+ minor);
+}
+
static int mlx5e_get_tunable(struct net_device *dev,
const struct ethtool_tunable *tuna,
void *data)
{
- const struct mlx5e_priv *priv = netdev_priv(dev);
- int err = 0;
+ int err;
switch (tuna->id) {
- case ETHTOOL_TX_COPYBREAK:
- *(u32 *)data = priv->channels.params.tx_max_inline;
+ case ETHTOOL_PFC_PREVENTION_TOUT:
+ err = mlx5e_get_pfc_prevention_tout(dev, data);
break;
default:
err = -EINVAL;
const void *data)
{
struct mlx5e_priv *priv = netdev_priv(dev);
- struct mlx5_core_dev *mdev = priv->mdev;
- struct mlx5e_channels new_channels = {};
- int err = 0;
- u32 val;
+ int err;
mutex_lock(&priv->state_lock);
switch (tuna->id) {
- case ETHTOOL_TX_COPYBREAK:
- val = *(u32 *)data;
- if (val > mlx5e_get_max_inline_cap(mdev)) {
- err = -EINVAL;
- break;
- }
-
- new_channels.params = priv->channels.params;
- new_channels.params.tx_max_inline = val;
-
- if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) {
- priv->channels.params = new_channels.params;
- break;
- }
-
- err = mlx5e_open_channels(priv, &new_channels);
- if (err)
- break;
- mlx5e_switch_priv_channels(priv, &new_channels, NULL);
-
+ case ETHTOOL_PFC_PREVENTION_TOUT:
+ err = mlx5e_set_pfc_prevention_tout(dev, *(u16 *)data);
break;
default:
err = -EINVAL;
new_channels.params = priv->channels.params;
MLX5E_SET_PFLAG(&new_channels.params, MLX5E_PFLAG_RX_CQE_COMPRESS, new_val);
- new_channels.params.mpwqe_log_stride_sz =
- MLX5E_MPWQE_STRIDE_SZ(priv->mdev, new_val);
- new_channels.params.mpwqe_log_num_strides =
- MLX5_MPWRQ_LOG_WQE_SZ - new_channels.params.mpwqe_log_stride_sz;
-
if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) {
priv->channels.params = new_channels.params;
return 0;
return 0;
}
+static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
+{
+ struct mlx5e_priv *priv = netdev_priv(netdev);
+ struct mlx5_core_dev *mdev = priv->mdev;
+ struct mlx5e_channels new_channels = {};
+ int err;
+
+ if (enable) {
+ if (!mlx5e_check_fragmented_striding_rq_cap(mdev))
+ return -EOPNOTSUPP;
+ if (!mlx5e_striding_rq_possible(mdev, &priv->channels.params))
+ return -EINVAL;
+ }
+
+ new_channels.params = priv->channels.params;
+
+ MLX5E_SET_PFLAG(&new_channels.params, MLX5E_PFLAG_RX_STRIDING_RQ, enable);
+ mlx5e_set_rq_type(mdev, &new_channels.params);
+
+ if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) {
+ priv->channels.params = new_channels.params;
+ return 0;
+ }
+
+ err = mlx5e_open_channels(priv, &new_channels);
+ if (err)
+ return err;
+
+ mlx5e_switch_priv_channels(priv, &new_channels, NULL);
+ return 0;
+}
+
static int mlx5e_handle_pflag(struct net_device *netdev,
u32 wanted_flags,
enum mlx5e_priv_flag flag,
err = mlx5e_handle_pflag(netdev, pflags,
MLX5E_PFLAG_RX_CQE_COMPRESS,
set_pflag_rx_cqe_compress);
+ if (err)
+ goto out;
+
+ err = mlx5e_handle_pflag(netdev, pflags,
+ MLX5E_PFLAG_RX_STRIDING_RQ,
+ set_pflag_rx_striding_rq);
out:
mutex_unlock(&priv->state_lock);
struct mlx5e_cq_param icosq_cq;
};
-static bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev)
+bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev)
{
return MLX5_CAP_GEN(mdev, striding_rq) &&
MLX5_CAP_GEN(mdev, umr_ptr_rlky) &&
MLX5_CAP_ETH(mdev, reg_umr_sq);
}
+u8 mlx5e_mpwqe_get_log_stride_size(struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params)
+{
+ return MLX5E_MPWQE_STRIDE_SZ(mdev,
+ MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_CQE_COMPRESS));
+}
+
+u8 mlx5e_mpwqe_get_log_num_strides(struct mlx5_core_dev *mdev,
+ struct mlx5e_params *params)
+{
+ return MLX5_MPWRQ_LOG_WQE_SZ -
+ mlx5e_mpwqe_get_log_stride_size(mdev, params);
+}
+
+static u16 mlx5e_get_rq_headroom(struct mlx5e_params *params)
+{
+ u16 linear_rq_headroom = params->xdp_prog ?
+ XDP_PACKET_HEADROOM : MLX5_RX_HEADROOM;
+
+ linear_rq_headroom += NET_IP_ALIGN;
+
+ if (params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST)
+ return linear_rq_headroom;
+
+ return 0;
+}
+
void mlx5e_init_rq_type_params(struct mlx5_core_dev *mdev,
- struct mlx5e_params *params, u8 rq_type)
+ struct mlx5e_params *params)
{
- params->rq_wq_type = rq_type;
params->lro_wqe_sz = MLX5E_PARAMS_DEFAULT_LRO_WQE_SZ;
switch (params->rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
params->log_rq_size = is_kdump_kernel() ?
MLX5E_PARAMS_MINIMUM_LOG_RQ_SIZE_MPW :
MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE_MPW;
- params->mpwqe_log_stride_sz = MLX5E_MPWQE_STRIDE_SZ(mdev,
- MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_CQE_COMPRESS));
- params->mpwqe_log_num_strides = MLX5_MPWRQ_LOG_WQE_SZ -
- params->mpwqe_log_stride_sz;
break;
default: /* MLX5_WQ_TYPE_LINKED_LIST */
params->log_rq_size = is_kdump_kernel() ?
MLX5E_PARAMS_MINIMUM_LOG_RQ_SIZE :
MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE;
- params->rq_headroom = params->xdp_prog ?
- XDP_PACKET_HEADROOM : MLX5_RX_HEADROOM;
- params->rq_headroom += NET_IP_ALIGN;
/* Extra room needed for build_skb */
- params->lro_wqe_sz -= params->rq_headroom +
+ params->lro_wqe_sz -= mlx5e_get_rq_headroom(params) +
SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
}
mlx5_core_info(mdev, "MLX5E: StrdRq(%d) RqSz(%ld) StrdSz(%ld) RxCqeCmprss(%d)\n",
params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ,
BIT(params->log_rq_size),
- BIT(params->mpwqe_log_stride_sz),
+ BIT(mlx5e_mpwqe_get_log_stride_size(mdev, params)),
MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_CQE_COMPRESS));
}
-static void mlx5e_set_rq_params(struct mlx5_core_dev *mdev,
+bool mlx5e_striding_rq_possible(struct mlx5_core_dev *mdev,
struct mlx5e_params *params)
{
- u8 rq_type = mlx5e_check_fragmented_striding_rq_cap(mdev) &&
- !params->xdp_prog && !MLX5_IPSEC_DEV(mdev) ?
- MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ :
- MLX5_WQ_TYPE_LINKED_LIST;
- mlx5e_init_rq_type_params(mdev, params, rq_type);
+ return mlx5e_check_fragmented_striding_rq_cap(mdev) &&
+ !params->xdp_prog && !MLX5_IPSEC_DEV(mdev);
+}
+
+void mlx5e_set_rq_type(struct mlx5_core_dev *mdev, struct mlx5e_params *params)
+{
+ params->rq_wq_type = mlx5e_striding_rq_possible(mdev, params) &&
+ MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ) ?
+ MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ :
+ MLX5_WQ_TYPE_LINKED_LIST;
}
static void mlx5e_update_carrier(struct mlx5e_priv *priv)
mutex_unlock(&priv->state_lock);
}
-static void mlx5e_tx_timeout_work(struct work_struct *work)
-{
- struct mlx5e_priv *priv = container_of(work, struct mlx5e_priv,
- tx_timeout_work);
- int err;
-
- rtnl_lock();
- mutex_lock(&priv->state_lock);
- if (!test_bit(MLX5E_STATE_OPENED, &priv->state))
- goto unlock;
- mlx5e_close_locked(priv->netdev);
- err = mlx5e_open_locked(priv->netdev);
- if (err)
- netdev_err(priv->netdev, "mlx5e_open_locked failed recovering from a tx_timeout, err(%d).\n",
- err);
-unlock:
- mutex_unlock(&priv->state_lock);
- rtnl_unlock();
-}
-
void mlx5e_update_stats(struct mlx5e_priv *priv)
{
int i;
goto err_rq_wq_destroy;
rq->buff.map_dir = rq->xdp_prog ? DMA_BIDIRECTIONAL : DMA_FROM_DEVICE;
- rq->buff.headroom = params->rq_headroom;
+ rq->buff.headroom = mlx5e_get_rq_headroom(params);
switch (rq->wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
goto err_rq_wq_destroy;
}
- rq->mpwqe.log_stride_sz = params->mpwqe_log_stride_sz;
- rq->mpwqe.num_strides = BIT(params->mpwqe_log_num_strides);
+ rq->mpwqe.log_stride_sz = mlx5e_mpwqe_get_log_stride_size(mdev, params);
+ rq->mpwqe.num_strides = BIT(mlx5e_mpwqe_get_log_num_strides(mdev, params));
byte_count = rq->mpwqe.num_strides << rq->mpwqe.log_stride_sz;
static int mlx5e_modify_rq_state(struct mlx5e_rq *rq, int curr_state,
int next_state)
{
- struct mlx5e_channel *c = rq->channel;
- struct mlx5_core_dev *mdev = c->mdev;
+ struct mlx5_core_dev *mdev = rq->mdev;
void *in;
void *rqc;
return 0;
}
+static void mlx5e_sq_recover(struct work_struct *work);
static int mlx5e_alloc_txqsq(struct mlx5e_channel *c,
int txq_ix,
struct mlx5e_params *params,
sq->channel = c;
sq->txq_ix = txq_ix;
sq->uar_map = mdev->mlx5e_res.bfreg.map;
- sq->max_inline = params->tx_max_inline;
sq->min_inline_mode = params->tx_min_inline_mode;
+ INIT_WORK(&sq->recover.recover_work, mlx5e_sq_recover);
if (MLX5_IPSEC_DEV(c->priv->mdev))
set_bit(MLX5E_SQ_STATE_IPSEC, &sq->state);
MLX5_SET(sqc, sqc, min_wqe_inline_mode, csp->min_inline_mode);
MLX5_SET(sqc, sqc, state, MLX5_SQC_STATE_RST);
+ MLX5_SET(sqc, sqc, flush_in_error_en, 1);
MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_CYCLIC);
MLX5_SET(wq, wq, uar_page, mdev->mlx5e_res.bfreg.index);
return err;
}
+static void mlx5e_reset_txqsq_cc_pc(struct mlx5e_txqsq *sq)
+{
+ WARN_ONCE(sq->cc != sq->pc,
+ "SQ 0x%x: cc (0x%x) != pc (0x%x)\n",
+ sq->sqn, sq->cc, sq->pc);
+ sq->cc = 0;
+ sq->dma_fifo_cc = 0;
+ sq->pc = 0;
+}
+
static void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
{
sq->txq = netdev_get_tx_queue(sq->channel->netdev, sq->txq_ix);
+ clear_bit(MLX5E_SQ_STATE_RECOVERING, &sq->state);
set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
netdev_tx_reset_queue(sq->txq);
netif_tx_start_queue(sq->txq);
mlx5e_free_txqsq(sq);
}
+static int mlx5e_wait_for_sq_flush(struct mlx5e_txqsq *sq)
+{
+ unsigned long exp_time = jiffies + msecs_to_jiffies(2000);
+
+ while (time_before(jiffies, exp_time)) {
+ if (sq->cc == sq->pc)
+ return 0;
+
+ msleep(20);
+ }
+
+ netdev_err(sq->channel->netdev,
+ "Wait for SQ 0x%x flush timeout (sq cc = 0x%x, sq pc = 0x%x)\n",
+ sq->sqn, sq->cc, sq->pc);
+
+ return -ETIMEDOUT;
+}
+
+static int mlx5e_sq_to_ready(struct mlx5e_txqsq *sq, int curr_state)
+{
+ struct mlx5_core_dev *mdev = sq->channel->mdev;
+ struct net_device *dev = sq->channel->netdev;
+ struct mlx5e_modify_sq_param msp = {0};
+ int err;
+
+ msp.curr_state = curr_state;
+ msp.next_state = MLX5_SQC_STATE_RST;
+
+ err = mlx5e_modify_sq(mdev, sq->sqn, &msp);
+ if (err) {
+ netdev_err(dev, "Failed to move sq 0x%x to reset\n", sq->sqn);
+ return err;
+ }
+
+ memset(&msp, 0, sizeof(msp));
+ msp.curr_state = MLX5_SQC_STATE_RST;
+ msp.next_state = MLX5_SQC_STATE_RDY;
+
+ err = mlx5e_modify_sq(mdev, sq->sqn, &msp);
+ if (err) {
+ netdev_err(dev, "Failed to move sq 0x%x to ready\n", sq->sqn);
+ return err;
+ }
+
+ return 0;
+}
+
+static void mlx5e_sq_recover(struct work_struct *work)
+{
+ struct mlx5e_txqsq_recover *recover =
+ container_of(work, struct mlx5e_txqsq_recover,
+ recover_work);
+ struct mlx5e_txqsq *sq = container_of(recover, struct mlx5e_txqsq,
+ recover);
+ struct mlx5_core_dev *mdev = sq->channel->mdev;
+ struct net_device *dev = sq->channel->netdev;
+ u8 state;
+ int err;
+
+ err = mlx5_core_query_sq_state(mdev, sq->sqn, &state);
+ if (err) {
+ netdev_err(dev, "Failed to query SQ 0x%x state. err = %d\n",
+ sq->sqn, err);
+ return;
+ }
+
+ if (state != MLX5_RQC_STATE_ERR) {
+ netdev_err(dev, "SQ 0x%x not in ERROR state\n", sq->sqn);
+ return;
+ }
+
+ netif_tx_disable_queue(sq->txq);
+
+ if (mlx5e_wait_for_sq_flush(sq))
+ return;
+
+ /* If the interval between two consecutive recovers per SQ is too
+ * short, don't recover to avoid infinite loop of ERR_CQE -> recover.
+ * If we reached this state, there is probably a bug that needs to be
+ * fixed. let's keep the queue close and let tx timeout cleanup.
+ */
+ if (jiffies_to_msecs(jiffies - recover->last_recover) <
+ MLX5E_SQ_RECOVER_MIN_INTERVAL) {
+ netdev_err(dev, "Recover SQ 0x%x canceled, too many error CQEs\n",
+ sq->sqn);
+ return;
+ }
+
+ /* At this point, no new packets will arrive from the stack as TXQ is
+ * marked with QUEUE_STATE_DRV_XOFF. In addition, NAPI cleared all
+ * pending WQEs. SQ can safely reset the SQ.
+ */
+ if (mlx5e_sq_to_ready(sq, state))
+ return;
+
+ mlx5e_reset_txqsq_cc_pc(sq);
+ sq->stats.recover++;
+ recover->last_recover = jiffies;
+ mlx5e_activate_txqsq(sq);
+}
+
static int mlx5e_open_icosq(struct mlx5e_channel *c,
struct mlx5e_params *params,
struct mlx5e_sq_param *param,
struct mlx5e_params *params,
struct mlx5e_rq_param *param)
{
+ struct mlx5_core_dev *mdev = priv->mdev;
void *rqc = param->rqc;
void *wq = MLX5_ADDR_OF(rqc, rqc, wq);
switch (params->rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
- MLX5_SET(wq, wq, log_wqe_num_of_strides, params->mpwqe_log_num_strides - 9);
- MLX5_SET(wq, wq, log_wqe_stride_size, params->mpwqe_log_stride_sz - 6);
+ MLX5_SET(wq, wq, log_wqe_num_of_strides,
+ mlx5e_mpwqe_get_log_num_strides(mdev, params) - 9);
+ MLX5_SET(wq, wq, log_wqe_stride_size,
+ mlx5e_mpwqe_get_log_stride_size(mdev, params) - 6);
MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ);
break;
default: /* MLX5_WQ_TYPE_LINKED_LIST */
MLX5_SET(wq, wq, end_padding_mode, MLX5_WQ_END_PAD_MODE_ALIGN);
MLX5_SET(wq, wq, log_wq_stride, ilog2(sizeof(struct mlx5e_rx_wqe)));
MLX5_SET(wq, wq, log_wq_sz, params->log_rq_size);
- MLX5_SET(wq, wq, pd, priv->mdev->mlx5e_res.pdn);
+ MLX5_SET(wq, wq, pd, mdev->mlx5e_res.pdn);
MLX5_SET(rqc, rqc, counter_set_id, priv->q_counter);
MLX5_SET(rqc, rqc, vsd, params->vlan_strip_disable);
MLX5_SET(rqc, rqc, scatter_fcs, params->scatter_fcs_en);
- param->wq.buf_numa_node = dev_to_node(&priv->mdev->pdev->dev);
+ param->wq.buf_numa_node = dev_to_node(&mdev->pdev->dev);
param->wq.linear = 1;
}
-static void mlx5e_build_drop_rq_param(struct mlx5_core_dev *mdev,
+static void mlx5e_build_drop_rq_param(struct mlx5e_priv *priv,
struct mlx5e_rq_param *param)
{
+ struct mlx5_core_dev *mdev = priv->mdev;
void *rqc = param->rqc;
void *wq = MLX5_ADDR_OF(rqc, rqc, wq);
MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_LINKED_LIST);
MLX5_SET(wq, wq, log_wq_stride, ilog2(sizeof(struct mlx5e_rx_wqe)));
+ MLX5_SET(rqc, rqc, counter_set_id, priv->drop_rq_q_counter);
param->wq.buf_numa_node = dev_to_node(&mdev->pdev->dev);
}
switch (params->rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
- log_cq_size = params->log_rq_size + params->mpwqe_log_num_strides;
+ log_cq_size = params->log_rq_size +
+ mlx5e_mpwqe_get_log_num_strides(priv->mdev, params);
break;
default: /* MLX5_WQ_TYPE_LINKED_LIST */
log_cq_size = params->log_rq_size;
return mlx5e_alloc_cq_common(mdev, param, cq);
}
-static int mlx5e_open_drop_rq(struct mlx5_core_dev *mdev,
+static int mlx5e_open_drop_rq(struct mlx5e_priv *priv,
struct mlx5e_rq *drop_rq)
{
+ struct mlx5_core_dev *mdev = priv->mdev;
struct mlx5e_cq_param cq_param = {};
struct mlx5e_rq_param rq_param = {};
struct mlx5e_cq *cq = &drop_rq->cq;
int err;
- mlx5e_build_drop_rq_param(mdev, &rq_param);
+ mlx5e_build_drop_rq_param(priv, &rq_param);
err = mlx5e_alloc_drop_cq(mdev, cq, &cq_param);
if (err)
if (err)
goto err_free_rq;
+ err = mlx5e_modify_rq_state(drop_rq, MLX5_RQC_STATE_RST, MLX5_RQC_STATE_RDY);
+ if (err)
+ mlx5_core_warn(priv->mdev, "modify_rq_state failed, rx_if_down_packets won't be counted %d\n", err);
+
return 0;
err_free_rq:
netdev_features_t features)
{
netdev_features_t oper_features = netdev->features;
- int err;
+ int err = 0;
- err = mlx5e_handle_feature(netdev, &oper_features, features,
- NETIF_F_LRO, set_feature_lro);
- err |= mlx5e_handle_feature(netdev, &oper_features, features,
- NETIF_F_HW_VLAN_CTAG_FILTER,
+#define MLX5E_HANDLE_FEATURE(feature, handler) \
+ mlx5e_handle_feature(netdev, &oper_features, features, feature, handler)
+
+ err |= MLX5E_HANDLE_FEATURE(NETIF_F_LRO, set_feature_lro);
+ err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_VLAN_CTAG_FILTER,
set_feature_cvlan_filter);
- err |= mlx5e_handle_feature(netdev, &oper_features, features,
- NETIF_F_HW_TC, set_feature_tc_num_filters);
- err |= mlx5e_handle_feature(netdev, &oper_features, features,
- NETIF_F_RXALL, set_feature_rx_all);
- err |= mlx5e_handle_feature(netdev, &oper_features, features,
- NETIF_F_RXFCS, set_feature_rx_fcs);
- err |= mlx5e_handle_feature(netdev, &oper_features, features,
- NETIF_F_HW_VLAN_CTAG_RX, set_feature_rx_vlan);
+ err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TC, set_feature_tc_num_filters);
+ err |= MLX5E_HANDLE_FEATURE(NETIF_F_RXALL, set_feature_rx_all);
+ err |= MLX5E_HANDLE_FEATURE(NETIF_F_RXFCS, set_feature_rx_fcs);
+ err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_VLAN_CTAG_RX, set_feature_rx_vlan);
#ifdef CONFIG_RFS_ACCEL
- err |= mlx5e_handle_feature(netdev, &oper_features, features,
- NETIF_F_NTUPLE, set_feature_arfs);
+ err |= MLX5E_HANDLE_FEATURE(NETIF_F_NTUPLE, set_feature_arfs);
#endif
if (err) {
return true;
}
-static void mlx5e_tx_timeout(struct net_device *dev)
+static void mlx5e_tx_timeout_work(struct work_struct *work)
{
- struct mlx5e_priv *priv = netdev_priv(dev);
+ struct mlx5e_priv *priv = container_of(work, struct mlx5e_priv,
+ tx_timeout_work);
+ struct net_device *dev = priv->netdev;
bool reopen_channels = false;
- int i;
+ int i, err;
- netdev_err(dev, "TX timeout detected\n");
+ rtnl_lock();
+ mutex_lock(&priv->state_lock);
+
+ if (!test_bit(MLX5E_STATE_OPENED, &priv->state))
+ goto unlock;
for (i = 0; i < priv->channels.num * priv->channels.params.num_tc; i++) {
struct netdev_queue *dev_queue = netdev_get_tx_queue(dev, i);
if (!netif_xmit_stopped(dev_queue))
continue;
- netdev_err(dev, "TX timeout on queue: %d, SQ: 0x%x, CQ: 0x%x, SQ Cons: 0x%x SQ Prod: 0x%x, usecs since last trans: %u\n",
+
+ netdev_err(dev,
+ "TX timeout on queue: %d, SQ: 0x%x, CQ: 0x%x, SQ Cons: 0x%x SQ Prod: 0x%x, usecs since last trans: %u\n",
i, sq->sqn, sq->cq.mcq.cqn, sq->cc, sq->pc,
jiffies_to_usecs(jiffies - dev_queue->trans_start));
}
}
- if (reopen_channels && test_bit(MLX5E_STATE_OPENED, &priv->state))
- schedule_work(&priv->tx_timeout_work);
+ if (!reopen_channels)
+ goto unlock;
+
+ mlx5e_close_locked(dev);
+ err = mlx5e_open_locked(dev);
+ if (err)
+ netdev_err(priv->netdev,
+ "mlx5e_open_locked failed recovering from a tx_timeout, err(%d).\n",
+ err);
+
+unlock:
+ mutex_unlock(&priv->state_lock);
+ rtnl_unlock();
+}
+
+static void mlx5e_tx_timeout(struct net_device *dev)
+{
+ struct mlx5e_priv *priv = netdev_priv(dev);
+
+ netdev_err(dev, "TX timeout detected\n");
+ queue_work(priv->wq, &priv->tx_timeout_work);
}
static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog)
bpf_prog_put(old_prog);
if (reset) /* change RQ type according to priv->xdp_prog */
- mlx5e_set_rq_params(priv->mdev, &priv->channels.params);
+ mlx5e_set_rq_type(priv->mdev, &priv->channels.params);
if (was_opened && reset)
mlx5e_open_locked(netdev);
return 0;
}
-u16 mlx5e_get_max_inline_cap(struct mlx5_core_dev *mdev)
-{
- int bf_buf_size = (1 << MLX5_CAP_GEN(mdev, log_bf_reg_size)) / 2;
-
- return bf_buf_size -
- sizeof(struct mlx5e_tx_wqe) +
- 2 /*sizeof(mlx5e_tx_wqe.inline_hdr_start)*/;
-}
-
void mlx5e_build_default_indir_rqt(u32 *indirection_rqt, int len,
int num_channels)
{
return 0;
}
-static bool cqe_compress_heuristic(u32 link_speed, u32 pci_bw)
+static bool slow_pci_heuristic(struct mlx5_core_dev *mdev)
{
- return (link_speed && pci_bw &&
- (pci_bw < 40000) && (pci_bw < link_speed));
-}
+ u32 link_speed = 0;
+ u32 pci_bw = 0;
-static bool hw_lro_heuristic(u32 link_speed, u32 pci_bw)
-{
- return !(link_speed && pci_bw &&
- (pci_bw <= 16000) && (pci_bw < link_speed));
+ mlx5e_get_max_linkspeed(mdev, &link_speed);
+ mlx5e_get_pci_bw(mdev, &pci_bw);
+ mlx5_core_dbg_once(mdev, "Max link speed = %d, PCI BW = %d\n",
+ link_speed, pci_bw);
+
+#define MLX5E_SLOW_PCI_RATIO (2)
+
+ return link_speed && pci_bw &&
+ link_speed > MLX5E_SLOW_PCI_RATIO * pci_bw;
}
void mlx5e_set_tx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode)
MLX5_CQ_PERIOD_MODE_START_FROM_CQE);
}
-u32 mlx5e_choose_lro_timeout(struct mlx5_core_dev *mdev, u32 wanted_timeout)
+static u32 mlx5e_choose_lro_timeout(struct mlx5_core_dev *mdev, u32 wanted_timeout)
{
int i;
u16 max_channels)
{
u8 cq_period_mode = 0;
- u32 link_speed = 0;
- u32 pci_bw = 0;
params->num_channels = max_channels;
params->num_tc = 1;
- mlx5e_get_max_linkspeed(mdev, &link_speed);
- mlx5e_get_pci_bw(mdev, &pci_bw);
- mlx5_core_dbg(mdev, "Max link speed = %d, PCI BW = %d\n",
- link_speed, pci_bw);
-
/* SQ */
params->log_sq_size = is_kdump_kernel() ?
MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE :
params->rx_cqe_compress_def = false;
if (MLX5_CAP_GEN(mdev, cqe_compression) &&
MLX5_CAP_GEN(mdev, vport_group_manager))
- params->rx_cqe_compress_def = cqe_compress_heuristic(link_speed, pci_bw);
+ params->rx_cqe_compress_def = slow_pci_heuristic(mdev);
MLX5E_SET_PFLAG(params, MLX5E_PFLAG_RX_CQE_COMPRESS, params->rx_cqe_compress_def);
/* RQ */
- mlx5e_set_rq_params(mdev, params);
+ if (mlx5e_striding_rq_possible(mdev, params))
+ MLX5E_SET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ,
+ !slow_pci_heuristic(mdev));
+ mlx5e_set_rq_type(mdev, params);
+ mlx5e_init_rq_type_params(mdev, params);
/* HW LRO */
/* TODO: && MLX5_CAP_ETH(mdev, lro_cap) */
if (params->rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ)
- params->lro_en = hw_lro_heuristic(link_speed, pci_bw);
+ params->lro_en = !slow_pci_heuristic(mdev);
params->lro_timeout = mlx5e_choose_lro_timeout(mdev, MLX5E_DEFAULT_LRO_TIMEOUT);
/* CQ moderation params */
mlx5e_set_tx_cq_mode_params(params, cq_period_mode);
/* TX inline */
- params->tx_max_inline = mlx5e_get_max_inline_cap(mdev);
params->tx_min_inline_mode = mlx5e_params_calculate_tx_min_inline(mdev);
/* RSS */
netdev->vlan_features |= NETIF_F_RXCSUM;
netdev->vlan_features |= NETIF_F_RXHASH;
+ netdev->hw_enc_features |= NETIF_F_HW_VLAN_CTAG_TX;
+ netdev->hw_enc_features |= NETIF_F_HW_VLAN_CTAG_RX;
+
if (!!MLX5_CAP_ETH(mdev, lro_cap))
netdev->vlan_features |= NETIF_F_LRO;
mlx5e_ipsec_build_netdev(priv);
}
-static void mlx5e_create_q_counter(struct mlx5e_priv *priv)
+static void mlx5e_create_q_counters(struct mlx5e_priv *priv)
{
struct mlx5_core_dev *mdev = priv->mdev;
int err;
mlx5_core_warn(mdev, "alloc queue counter failed, %d\n", err);
priv->q_counter = 0;
}
+
+ err = mlx5_core_alloc_q_counter(mdev, &priv->drop_rq_q_counter);
+ if (err) {
+ mlx5_core_warn(mdev, "alloc drop RQ counter failed, %d\n", err);
+ priv->drop_rq_q_counter = 0;
+ }
}
-static void mlx5e_destroy_q_counter(struct mlx5e_priv *priv)
+static void mlx5e_destroy_q_counters(struct mlx5e_priv *priv)
{
- if (!priv->q_counter)
- return;
+ if (priv->q_counter)
+ mlx5_core_dealloc_q_counter(priv->mdev, priv->q_counter);
- mlx5_core_dealloc_q_counter(priv->mdev, priv->q_counter);
+ if (priv->drop_rq_q_counter)
+ mlx5_core_dealloc_q_counter(priv->mdev, priv->drop_rq_q_counter);
}
static void mlx5e_nic_init(struct mlx5_core_dev *mdev,
if (err)
goto out;
- err = mlx5e_open_drop_rq(mdev, &priv->drop_rq);
+ mlx5e_create_q_counters(priv);
+
+ err = mlx5e_open_drop_rq(priv, &priv->drop_rq);
if (err) {
mlx5_core_err(mdev, "open drop rq failed, %d\n", err);
- goto err_cleanup_tx;
+ goto err_destroy_q_counters;
}
err = profile->init_rx(priv);
if (err)
goto err_close_drop_rq;
- mlx5e_create_q_counter(priv);
-
if (profile->enable)
profile->enable(priv);
err_close_drop_rq:
mlx5e_close_drop_rq(&priv->drop_rq);
-err_cleanup_tx:
+err_destroy_q_counters:
+ mlx5e_destroy_q_counters(priv);
profile->cleanup_tx(priv);
out:
profile->disable(priv);
flush_workqueue(priv->wq);
- mlx5e_destroy_q_counter(priv);
profile->cleanup_rx(priv);
mlx5e_close_drop_rq(&priv->drop_rq);
+ mlx5e_destroy_q_counters(priv);
profile->cleanup_tx(priv);
cancel_delayed_work_sync(&priv->update_stats_work);
}
params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation);
mlx5e_set_rx_cq_mode_params(params, cq_period_mode);
- params->tx_max_inline = mlx5e_get_max_inline_cap(mdev);
params->num_tc = 1;
params->lro_wqe_sz = MLX5E_PARAMS_DEFAULT_LRO_WQE_SZ;
len = ALIGN(headlen_pg, sizeof(long));
dma_sync_single_for_cpu(pdev, dma_info->addr + offset, len,
DMA_FROM_DEVICE);
- skb_copy_to_linear_data_offset(skb, 0,
- page_address(dma_info->page) + offset,
- len);
+ skb_copy_to_linear_data(skb, page_address(dma_info->page) + offset, len);
+
if (unlikely(offset + headlen > PAGE_SIZE)) {
dma_info++;
headlen_pg = len;
data = va + rx_headroom;
frag_size = MLX5_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);
- dma_sync_single_range_for_cpu(rq->pdev,
- di->addr + wi->offset,
- 0, frag_size,
- DMA_FROM_DEVICE);
+ dma_sync_single_range_for_cpu(rq->pdev, di->addr, wi->offset,
+ frag_size, DMA_FROM_DEVICE);
prefetch(data);
wi->offset += frag_size;
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_queue_wake) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_queue_dropped) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_xmit_more) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_cqe_err) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, tx_recover) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_wqe_err) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_mpwqe_filler) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_buff_alloc_err) },
s->tx_queue_stopped += sq_stats->stopped;
s->tx_queue_wake += sq_stats->wake;
s->tx_queue_dropped += sq_stats->dropped;
+ s->tx_cqe_err += sq_stats->cqe_err;
+ s->tx_recover += sq_stats->recover;
s->tx_xmit_more += sq_stats->xmit_more;
s->tx_csum_partial_inner += sq_stats->csum_partial_inner;
s->tx_csum_none += sq_stats->csum_none;
{ MLX5E_DECLARE_STAT(struct mlx5e_qcounter_stats, rx_out_of_buffer) },
};
+static const struct counter_desc drop_rq_stats_desc[] = {
+ { MLX5E_DECLARE_STAT(struct mlx5e_qcounter_stats, rx_if_down_packets) },
+};
+
#define NUM_Q_COUNTERS ARRAY_SIZE(q_stats_desc)
+#define NUM_DROP_RQ_COUNTERS ARRAY_SIZE(drop_rq_stats_desc)
static int mlx5e_grp_q_get_num_stats(struct mlx5e_priv *priv)
{
- return priv->q_counter ? NUM_Q_COUNTERS : 0;
+ int num_stats = 0;
+
+ if (priv->q_counter)
+ num_stats += NUM_Q_COUNTERS;
+
+ if (priv->drop_rq_q_counter)
+ num_stats += NUM_DROP_RQ_COUNTERS;
+
+ return num_stats;
}
static int mlx5e_grp_q_fill_strings(struct mlx5e_priv *priv, u8 *data, int idx)
int i;
for (i = 0; i < NUM_Q_COUNTERS && priv->q_counter; i++)
- strcpy(data + (idx++) * ETH_GSTRING_LEN, q_stats_desc[i].format);
+ strcpy(data + (idx++) * ETH_GSTRING_LEN,
+ q_stats_desc[i].format);
+
+ for (i = 0; i < NUM_DROP_RQ_COUNTERS && priv->drop_rq_q_counter; i++)
+ strcpy(data + (idx++) * ETH_GSTRING_LEN,
+ drop_rq_stats_desc[i].format);
+
return idx;
}
int i;
for (i = 0; i < NUM_Q_COUNTERS && priv->q_counter; i++)
- data[idx++] = MLX5E_READ_CTR32_CPU(&priv->stats.qcnt, q_stats_desc, i);
+ data[idx++] = MLX5E_READ_CTR32_CPU(&priv->stats.qcnt,
+ q_stats_desc, i);
+ for (i = 0; i < NUM_DROP_RQ_COUNTERS && priv->drop_rq_q_counter; i++)
+ data[idx++] = MLX5E_READ_CTR32_CPU(&priv->stats.qcnt,
+ drop_rq_stats_desc, i);
return idx;
}
{
struct mlx5e_qcounter_stats *qcnt = &priv->stats.qcnt;
u32 out[MLX5_ST_SZ_DW(query_q_counter_out)];
- int err;
- if (!priv->q_counter)
- return;
+ if (priv->q_counter &&
+ !mlx5_core_query_q_counter(priv->mdev, priv->q_counter, 0, out,
+ sizeof(out)))
+ qcnt->rx_out_of_buffer = MLX5_GET(query_q_counter_out,
+ out, out_of_buffer);
+ if (priv->drop_rq_q_counter &&
+ !mlx5_core_query_q_counter(priv->mdev, priv->drop_rq_q_counter, 0,
+ out, sizeof(out)))
+ qcnt->rx_if_down_packets = MLX5_GET(query_q_counter_out, out,
+ out_of_buffer);
+}
+
+#define VNIC_ENV_OFF(c) MLX5_BYTE_OFF(query_vnic_env_out, c)
+static const struct counter_desc vnic_env_stats_desc[] = {
+ { "rx_steer_missed_packets",
+ VNIC_ENV_OFF(vport_env.nic_receive_steering_discard) },
+};
+
+#define NUM_VNIC_ENV_COUNTERS ARRAY_SIZE(vnic_env_stats_desc)
+
+static int mlx5e_grp_vnic_env_get_num_stats(struct mlx5e_priv *priv)
+{
+ return MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard) ?
+ NUM_VNIC_ENV_COUNTERS : 0;
+}
+
+static int mlx5e_grp_vnic_env_fill_strings(struct mlx5e_priv *priv, u8 *data,
+ int idx)
+{
+ int i;
+
+ if (!MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard))
+ return idx;
+
+ for (i = 0; i < NUM_VNIC_ENV_COUNTERS; i++)
+ strcpy(data + (idx++) * ETH_GSTRING_LEN,
+ vnic_env_stats_desc[i].format);
+ return idx;
+}
+
+static int mlx5e_grp_vnic_env_fill_stats(struct mlx5e_priv *priv, u64 *data,
+ int idx)
+{
+ int i;
+
+ if (!MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard))
+ return idx;
- err = mlx5_core_query_q_counter(priv->mdev, priv->q_counter, 0, out, sizeof(out));
- if (err)
+ for (i = 0; i < NUM_VNIC_ENV_COUNTERS; i++)
+ data[idx++] = MLX5E_READ_CTR64_BE(priv->stats.vnic.query_vnic_env_out,
+ vnic_env_stats_desc, i);
+ return idx;
+}
+
+static void mlx5e_grp_vnic_env_update_stats(struct mlx5e_priv *priv)
+{
+ u32 *out = (u32 *)priv->stats.vnic.query_vnic_env_out;
+ int outlen = MLX5_ST_SZ_BYTES(query_vnic_env_out);
+ u32 in[MLX5_ST_SZ_DW(query_vnic_env_in)] = {0};
+ struct mlx5_core_dev *mdev = priv->mdev;
+
+ if (!MLX5_CAP_GEN(priv->mdev, nic_receive_steering_discard))
return;
- qcnt->rx_out_of_buffer = MLX5_GET(query_q_counter_out, out, out_of_buffer);
+ MLX5_SET(query_vnic_env_in, in, opcode,
+ MLX5_CMD_OP_QUERY_VNIC_ENV);
+ MLX5_SET(query_vnic_env_in, in, op_mod, 0);
+ MLX5_SET(query_vnic_env_in, in, other_vport, 0);
+ mlx5_cmd_exec(mdev, in, sizeof(in), out, outlen);
}
#define VPORT_COUNTER_OFF(c) MLX5_BYTE_OFF(query_vport_counter_out, c)
{ "rx_%s_pause_transition", PPORT_PER_PRIO_OFF(rx_pause_transition) },
};
+static const struct counter_desc pport_pfc_stall_stats_desc[] = {
+ { "tx_pause_storm_warning_events ", PPORT_PER_PRIO_OFF(device_stall_minor_watermark_cnt) },
+ { "tx_pause_storm_error_events", PPORT_PER_PRIO_OFF(device_stall_critical_watermark_cnt) },
+};
+
#define NUM_PPORT_PER_PRIO_PFC_COUNTERS ARRAY_SIZE(pport_per_prio_pfc_stats_desc)
+#define NUM_PPORT_PFC_STALL_COUNTERS(priv) (ARRAY_SIZE(pport_pfc_stall_stats_desc) * \
+ MLX5_CAP_PCAM_FEATURE((priv)->mdev, pfcc_mask) * \
+ MLX5_CAP_DEBUG((priv)->mdev, stall_detect))
static unsigned long mlx5e_query_pfc_combined(struct mlx5e_priv *priv)
{
{
return (mlx5e_query_global_pause_combined(priv) +
hweight8(mlx5e_query_pfc_combined(priv))) *
- NUM_PPORT_PER_PRIO_PFC_COUNTERS;
+ NUM_PPORT_PER_PRIO_PFC_COUNTERS +
+ NUM_PPORT_PFC_STALL_COUNTERS(priv);
}
static int mlx5e_grp_per_prio_pfc_fill_strings(struct mlx5e_priv *priv,
}
}
+ for (i = 0; i < NUM_PPORT_PFC_STALL_COUNTERS(priv); i++)
+ strcpy(data + (idx++) * ETH_GSTRING_LEN,
+ pport_pfc_stall_stats_desc[i].format);
+
return idx;
}
}
}
+ for (i = 0; i < NUM_PPORT_PFC_STALL_COUNTERS(priv); i++)
+ data[idx++] = MLX5E_READ_CTR64_BE(&priv->stats.pport.per_prio_counters[0],
+ pport_pfc_stall_stats_desc, i);
+
return idx;
}
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, wake) },
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, dropped) },
{ MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, xmit_more) },
+ { MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, cqe_err) },
+ { MLX5E_DECLARE_TX_STAT(struct mlx5e_sq_stats, recover) },
};
static const struct counter_desc ch_stats_desc[] = {
.update_stats_mask = MLX5E_NDO_UPDATE_STATS,
.update_stats = mlx5e_grp_q_update_stats,
},
+ {
+ .get_num_stats = mlx5e_grp_vnic_env_get_num_stats,
+ .fill_strings = mlx5e_grp_vnic_env_fill_strings,
+ .fill_stats = mlx5e_grp_vnic_env_fill_stats,
+ .update_stats = mlx5e_grp_vnic_env_update_stats,
+ },
{
.get_num_stats = mlx5e_grp_vport_get_num_stats,
.fill_strings = mlx5e_grp_vport_fill_strings,
u64 tx_queue_wake;
u64 tx_queue_dropped;
u64 tx_xmit_more;
+ u64 tx_cqe_err;
+ u64 tx_recover;
u64 rx_wqe_err;
u64 rx_mpwqe_filler;
u64 rx_buff_alloc_err;
struct mlx5e_qcounter_stats {
u32 rx_out_of_buffer;
+ u32 rx_if_down_packets;
+};
+
+struct mlx5e_vnic_env_stats {
+ __be64 query_vnic_env_out[MLX5_ST_SZ_QW(query_vnic_env_out)];
};
#define VPORT_COUNTER_GET(vstats, c) MLX5_GET64(query_vport_counter_out, \
u64 stopped;
u64 wake;
u64 dropped;
+ u64 cqe_err;
+ u64 recover;
};
struct mlx5e_ch_stats {
struct mlx5e_stats {
struct mlx5e_sw_stats sw;
struct mlx5e_qcounter_stats qcnt;
+ struct mlx5e_vnic_env_stats vnic;
struct mlx5e_vport_stats vport;
struct mlx5e_pport_stats pport;
struct rtnl_link_stats64 vf_vport;
if (tcf_vlan_action(a) == TCA_VLAN_ACT_POP) {
attr->action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_POP;
} else if (tcf_vlan_action(a) == TCA_VLAN_ACT_PUSH) {
- if (tcf_vlan_push_proto(a) != htons(ETH_P_8021Q) ||
- tcf_vlan_push_prio(a))
- return -EOPNOTSUPP;
-
attr->action |= MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH;
- attr->vlan = tcf_vlan_push_vid(a);
+ attr->vlan_vid = tcf_vlan_push_vid(a);
+ if (mlx5_eswitch_vlan_actions_supported(priv->mdev)) {
+ attr->vlan_prio = tcf_vlan_push_prio(a);
+ attr->vlan_proto = tcf_vlan_push_proto(a);
+ if (!attr->vlan_proto)
+ attr->vlan_proto = htons(ETH_P_8021Q);
+ } else if (tcf_vlan_push_proto(a) != htons(ETH_P_8021Q) ||
+ tcf_vlan_push_prio(a)) {
+ return -EOPNOTSUPP;
+ }
} else { /* action is TCA_VLAN_ACT_MODIFY */
return -EOPNOTSUPP;
}
return mlx5e_sq_xmit(sq, skb, wqe, pi);
}
+static void mlx5e_dump_error_cqe(struct mlx5e_txqsq *sq,
+ struct mlx5_err_cqe *err_cqe)
+{
+ u32 ci = mlx5_cqwq_get_ci(&sq->cq.wq);
+
+ netdev_err(sq->channel->netdev,
+ "Error cqe on cqn 0x%x, ci 0x%x, sqn 0x%x, syndrome 0x%x, vendor syndrome 0x%x\n",
+ sq->cq.mcq.cqn, ci, sq->sqn, err_cqe->syndrome,
+ err_cqe->vendor_err_synd);
+ mlx5_dump_err_cqe(sq->cq.mdev, err_cqe);
+}
+
bool mlx5e_poll_tx_cq(struct mlx5e_cq *cq, int napi_budget)
{
struct mlx5e_txqsq *sq;
wqe_counter = be16_to_cpu(cqe->wqe_counter);
+ if (unlikely(cqe->op_own >> 4 == MLX5_CQE_REQ_ERR)) {
+ if (!test_and_set_bit(MLX5E_SQ_STATE_RECOVERING,
+ &sq->state)) {
+ mlx5e_dump_error_cqe(sq,
+ (struct mlx5_err_cqe *)cqe);
+ queue_work(cq->channel->priv->wq,
+ &sq->recover.recover_work);
+ }
+ sq->stats.cqe_err++;
+ }
+
do {
struct mlx5e_tx_wqe_info *wi;
struct sk_buff *skb;
netdev_tx_completed_queue(sq->txq, npkts, nbytes);
if (netif_tx_queue_stopped(sq->txq) &&
- mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, MLX5E_SQ_STOP_ROOM)) {
+ mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc,
+ MLX5E_SQ_STOP_ROOM) &&
+ !test_bit(MLX5E_SQ_STATE_RECOVERING, &sq->state)) {
netif_tx_wake_queue(sq->txq);
sq->stats.wake++;
}
return err;
}
-static void mlx5_eswitch_query_vport_drop_stats(struct mlx5_core_dev *dev,
- int vport_idx,
- struct mlx5_vport_drop_stats *stats)
+static int mlx5_eswitch_query_vport_drop_stats(struct mlx5_core_dev *dev,
+ int vport_idx,
+ struct mlx5_vport_drop_stats *stats)
{
struct mlx5_eswitch *esw = dev->priv.eswitch;
struct mlx5_vport *vport = &esw->vports[vport_idx];
+ u64 rx_discard_vport_down, tx_discard_vport_down;
u64 bytes = 0;
u16 idx = 0;
+ int err = 0;
if (!vport->enabled || esw->mode != SRIOV_LEGACY)
- return;
+ return 0;
if (vport->egress.drop_counter) {
idx = vport->egress.drop_counter->id;
idx = vport->ingress.drop_counter->id;
mlx5_fc_query(dev, idx, &stats->tx_dropped, &bytes);
}
+
+ if (!MLX5_CAP_GEN(dev, receive_discard_vport_down) &&
+ !MLX5_CAP_GEN(dev, transmit_discard_vport_down))
+ return 0;
+
+ err = mlx5_query_vport_down_stats(dev, vport_idx,
+ &rx_discard_vport_down,
+ &tx_discard_vport_down);
+ if (err)
+ return err;
+
+ if (MLX5_CAP_GEN(dev, receive_discard_vport_down))
+ stats->rx_dropped += rx_discard_vport_down;
+ if (MLX5_CAP_GEN(dev, transmit_discard_vport_down))
+ stats->tx_dropped += tx_discard_vport_down;
+
+ return 0;
}
int mlx5_eswitch_get_vport_stats(struct mlx5_eswitch *esw,
vf_stats->broadcast =
MLX5_GET_CTR(out, received_eth_broadcast.packets);
- mlx5_eswitch_query_vport_drop_stats(esw->dev, vport, &stats);
+ err = mlx5_eswitch_query_vport_drop_stats(esw->dev, vport, &stats);
+ if (err)
+ goto free_out;
vf_stats->rx_dropped = stats.rx_dropped;
vf_stats->tx_dropped = stats.tx_dropped;
SET_VLAN_INSERT = BIT(1)
};
-#define MLX5_FLOW_CONTEXT_ACTION_VLAN_POP 0x4000
-#define MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH 0x8000
-
struct mlx5_esw_flow_attr {
struct mlx5_eswitch_rep *in_rep;
struct mlx5_eswitch_rep *out_rep;
int action;
- u16 vlan;
+ __be16 vlan_proto;
+ u16 vlan_vid;
+ u8 vlan_prio;
bool vlan_handled;
u32 encap_id;
u32 mod_hdr_id;
int __mlx5_eswitch_set_vport_vlan(struct mlx5_eswitch *esw,
int vport, u16 vlan, u8 qos, u8 set_flags);
+static inline bool mlx5_eswitch_vlan_actions_supported(struct mlx5_core_dev *dev)
+{
+ return MLX5_CAP_ESW_FLOWTABLE_FDB(dev, pop_vlan) &&
+ MLX5_CAP_ESW_FLOWTABLE_FDB(dev, push_vlan);
+}
+
#define MLX5_DEBUG_ESWITCH_MASK BIT(3)
#define esw_info(dev, format, ...) \
if (esw->mode != SRIOV_OFFLOADS)
return ERR_PTR(-EOPNOTSUPP);
- /* per flow vlan pop/push is emulated, don't set that into the firmware */
- flow_act.action = attr->action & ~(MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH | MLX5_FLOW_CONTEXT_ACTION_VLAN_POP);
+ flow_act.action = attr->action;
+ /* if per flow vlan pop/push is emulated, don't set that into the firmware */
+ if (!mlx5_eswitch_vlan_actions_supported(esw->dev))
+ flow_act.action &= ~(MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH |
+ MLX5_FLOW_CONTEXT_ACTION_VLAN_POP);
+ else if (flow_act.action & MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH) {
+ flow_act.vlan.ethtype = ntohs(attr->vlan_proto);
+ flow_act.vlan.vid = attr->vlan_vid;
+ flow_act.vlan.prio = attr->vlan_prio;
+ }
if (flow_act.action & MLX5_FLOW_CONTEXT_ACTION_FWD_DEST) {
dest[i].type = MLX5_FLOW_DESTINATION_TYPE_VPORT;
if (flow_act.action & MLX5_FLOW_CONTEXT_ACTION_DECAP)
spec->match_criteria_enable |= MLX5_MATCH_INNER_HEADERS;
- if (attr->action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)
+ if (flow_act.action & MLX5_FLOW_CONTEXT_ACTION_MOD_HDR)
flow_act.modify_id = attr->mod_hdr_id;
- if (attr->action & MLX5_FLOW_CONTEXT_ACTION_ENCAP)
+ if (flow_act.action & MLX5_FLOW_CONTEXT_ACTION_ENCAP)
flow_act.encap_id = attr->encap_id;
rule = mlx5_add_flow_rules((struct mlx5_flow_table *)esw->fdb_table.fdb,
/* protects against (1) setting rules with different vlans to push and
* (2) setting rules w.o vlans (attr->vlan = 0) && w. vlans to push (!= 0)
*/
- if (push && in_rep->vlan_refcount && (in_rep->vlan != attr->vlan))
+ if (push && in_rep->vlan_refcount && (in_rep->vlan != attr->vlan_vid))
goto out_notsupp;
return 0;
bool push, pop, fwd;
int err = 0;
+ /* nop if we're on the vlan push/pop non emulation mode */
+ if (mlx5_eswitch_vlan_actions_supported(esw->dev))
+ return 0;
+
push = !!(attr->action & MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH);
pop = !!(attr->action & MLX5_FLOW_CONTEXT_ACTION_VLAN_POP);
fwd = !!(attr->action & MLX5_FLOW_CONTEXT_ACTION_FWD_DEST);
if (vport->vlan_refcount)
goto skip_set_push;
- err = __mlx5_eswitch_set_vport_vlan(esw, vport->vport, attr->vlan, 0,
+ err = __mlx5_eswitch_set_vport_vlan(esw, vport->vport, attr->vlan_vid, 0,
SET_VLAN_INSERT | SET_VLAN_STRIP);
if (err)
goto out;
- vport->vlan = attr->vlan;
+ vport->vlan = attr->vlan_vid;
skip_set_push:
vport->vlan_refcount++;
}
bool push, pop, fwd;
int err = 0;
+ /* nop if we're on the vlan push/pop non emulation mode */
+ if (mlx5_eswitch_vlan_actions_supported(esw->dev))
+ return 0;
+
if (!attr->vlan_handled)
return 0;
rule->ctx = mlx5_fpga_ipsec_fs_create_sa_ctx(dev, fte, is_egress);
if (IS_ERR(rule->ctx)) {
+ int err = PTR_ERR(rule->ctx);
kfree(rule);
- return PTR_ERR(rule->ctx);
+ return err;
}
rule->fte = fte;
fte->dests_size * MLX5_ST_SZ_BYTES(dest_format_struct);
u32 out[MLX5_ST_SZ_DW(set_fte_out)] = {0};
struct mlx5_flow_rule *dst;
- void *in_flow_context;
+ void *in_flow_context, *vlan;
void *in_match_value;
void *in_dests;
u32 *in;
in_flow_context = MLX5_ADDR_OF(set_fte_in, in, flow_context);
MLX5_SET(flow_context, in_flow_context, group_id, group_id);
+
MLX5_SET(flow_context, in_flow_context, flow_tag, fte->action.flow_tag);
MLX5_SET(flow_context, in_flow_context, action, fte->action.action);
MLX5_SET(flow_context, in_flow_context, encap_id, fte->action.encap_id);
MLX5_SET(flow_context, in_flow_context, modify_header_id,
fte->action.modify_id);
+
+ vlan = MLX5_ADDR_OF(flow_context, in_flow_context, push_vlan);
+
+ MLX5_SET(vlan, vlan, ethtype, fte->action.vlan.ethtype);
+ MLX5_SET(vlan, vlan, vid, fte->action.vlan.vid);
+ MLX5_SET(vlan, vlan, prio, fte->action.vlan.prio);
+
in_match_value = MLX5_ADDR_OF(flow_context, in_flow_context,
match_value);
memcpy(in_match_value, &fte->val, sizeof(fte->val));
if (xored_actions & (MLX5_FLOW_CONTEXT_ACTION_DROP |
MLX5_FLOW_CONTEXT_ACTION_ENCAP |
MLX5_FLOW_CONTEXT_ACTION_DECAP |
- MLX5_FLOW_CONTEXT_ACTION_MOD_HDR))
+ MLX5_FLOW_CONTEXT_ACTION_MOD_HDR |
+ MLX5_FLOW_CONTEXT_ACTION_VLAN_POP |
+ MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH))
return true;
return false;
return err;
}
+ if (MLX5_CAP_GEN(dev, debug))
+ mlx5_core_get_caps(dev, MLX5_CAP_DEBUG);
+
if (MLX5_CAP_GEN(dev, pcam_reg))
mlx5_get_pcam_reg(dev);
trigger_cmd_completions(dev);
}
- mlx5_core_event(dev, MLX5_DEV_EVENT_SYS_ERROR, 0);
+ mlx5_core_event(dev, MLX5_DEV_EVENT_SYS_ERROR, 1);
mlx5_core_err(dev, "end\n");
unlock:
struct mlx5e_params *params)
{
/* Override RQ params as IPoIB supports only LINKED LIST RQ for now */
- mlx5e_init_rq_type_params(mdev, params, MLX5_WQ_TYPE_LINKED_LIST);
+ MLX5E_SET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ, false);
+ mlx5e_set_rq_type(mdev, params);
+ mlx5e_init_rq_type_params(mdev, params);
/* RQ size in ipoib by default is 512 */
params->log_rq_size = is_kdump_kernel() ?
__func__, __LINE__, current->pid, \
##__VA_ARGS__)
+#define mlx5_core_dbg_once(__dev, format, ...) \
+ dev_dbg_once(&(__dev)->pdev->dev, "%s:%d:(pid %d): " format, \
+ __func__, __LINE__, current->pid, \
+ ##__VA_ARGS__)
+
#define mlx5_core_dbg_mask(__dev, mask, format, ...) \
do { \
if ((mask) & mlx5_core_debug_mask) \
}
EXPORT_SYMBOL_GPL(mlx5_core_query_ib_ppcnt);
+static int mlx5_query_pfcc_reg(struct mlx5_core_dev *dev, u32 *out,
+ u32 out_size)
+{
+ u32 in[MLX5_ST_SZ_DW(pfcc_reg)] = {0};
+
+ MLX5_SET(pfcc_reg, in, local_port, 1);
+
+ return mlx5_core_access_reg(dev, in, sizeof(in), out,
+ out_size, MLX5_REG_PFCC, 0, 0);
+}
+
int mlx5_set_port_pause(struct mlx5_core_dev *dev, u32 rx_pause, u32 tx_pause)
{
u32 in[MLX5_ST_SZ_DW(pfcc_reg)] = {0};
int mlx5_query_port_pause(struct mlx5_core_dev *dev,
u32 *rx_pause, u32 *tx_pause)
{
- u32 in[MLX5_ST_SZ_DW(pfcc_reg)] = {0};
u32 out[MLX5_ST_SZ_DW(pfcc_reg)];
int err;
- MLX5_SET(pfcc_reg, in, local_port, 1);
- err = mlx5_core_access_reg(dev, in, sizeof(in), out,
- sizeof(out), MLX5_REG_PFCC, 0, 0);
+ err = mlx5_query_pfcc_reg(dev, out, sizeof(out));
if (err)
return err;
}
EXPORT_SYMBOL_GPL(mlx5_query_port_pause);
+int mlx5_set_port_stall_watermark(struct mlx5_core_dev *dev,
+ u16 stall_critical_watermark,
+ u16 stall_minor_watermark)
+{
+ u32 in[MLX5_ST_SZ_DW(pfcc_reg)] = {0};
+ u32 out[MLX5_ST_SZ_DW(pfcc_reg)];
+
+ MLX5_SET(pfcc_reg, in, local_port, 1);
+ MLX5_SET(pfcc_reg, in, pptx_mask_n, 1);
+ MLX5_SET(pfcc_reg, in, pprx_mask_n, 1);
+ MLX5_SET(pfcc_reg, in, ppan_mask_n, 1);
+ MLX5_SET(pfcc_reg, in, critical_stall_mask, 1);
+ MLX5_SET(pfcc_reg, in, minor_stall_mask, 1);
+ MLX5_SET(pfcc_reg, in, device_stall_critical_watermark,
+ stall_critical_watermark);
+ MLX5_SET(pfcc_reg, in, device_stall_minor_watermark, stall_minor_watermark);
+
+ return mlx5_core_access_reg(dev, in, sizeof(in), out,
+ sizeof(out), MLX5_REG_PFCC, 0, 1);
+}
+
+int mlx5_query_port_stall_watermark(struct mlx5_core_dev *dev,
+ u16 *stall_critical_watermark,
+ u16 *stall_minor_watermark)
+{
+ u32 out[MLX5_ST_SZ_DW(pfcc_reg)];
+ int err;
+
+ err = mlx5_query_pfcc_reg(dev, out, sizeof(out));
+ if (err)
+ return err;
+
+ if (stall_critical_watermark)
+ *stall_critical_watermark = MLX5_GET(pfcc_reg, out,
+ device_stall_critical_watermark);
+
+ if (stall_minor_watermark)
+ *stall_minor_watermark = MLX5_GET(pfcc_reg, out,
+ device_stall_minor_watermark);
+
+ return 0;
+}
+
int mlx5_set_port_pfc(struct mlx5_core_dev *dev, u8 pfc_en_tx, u8 pfc_en_rx)
{
u32 in[MLX5_ST_SZ_DW(pfcc_reg)] = {0};
int mlx5_query_port_pfc(struct mlx5_core_dev *dev, u8 *pfc_en_tx, u8 *pfc_en_rx)
{
- u32 in[MLX5_ST_SZ_DW(pfcc_reg)] = {0};
u32 out[MLX5_ST_SZ_DW(pfcc_reg)];
int err;
- MLX5_SET(pfcc_reg, in, local_port, 1);
- err = mlx5_core_access_reg(dev, in, sizeof(in), out,
- sizeof(out), MLX5_REG_PFCC, 0, 0);
+ err = mlx5_query_pfcc_reg(dev, out, sizeof(out));
if (err)
return err;
}
EXPORT_SYMBOL(mlx5_core_query_sq);
+int mlx5_core_query_sq_state(struct mlx5_core_dev *dev, u32 sqn, u8 *state)
+{
+ void *out;
+ void *sqc;
+ int inlen;
+ int err;
+
+ inlen = MLX5_ST_SZ_BYTES(query_sq_out);
+ out = kvzalloc(inlen, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ err = mlx5_core_query_sq(dev, sqn, out);
+ if (err)
+ goto out;
+
+ sqc = MLX5_ADDR_OF(query_sq_out, out, sq_context);
+ *state = MLX5_GET(sqc, sqc, state);
+
+out:
+ kvfree(out);
+ return err;
+}
+EXPORT_SYMBOL_GPL(mlx5_core_query_sq_state);
+
int mlx5_core_create_tir(struct mlx5_core_dev *dev, u32 *in, int inlen,
u32 *tirn)
{
}
EXPORT_SYMBOL_GPL(mlx5_core_query_vport_counter);
+int mlx5_query_vport_down_stats(struct mlx5_core_dev *mdev, u16 vport,
+ u64 *rx_discard_vport_down,
+ u64 *tx_discard_vport_down)
+{
+ u32 out[MLX5_ST_SZ_DW(query_vnic_env_out)] = {0};
+ u32 in[MLX5_ST_SZ_DW(query_vnic_env_in)] = {0};
+ int err;
+
+ MLX5_SET(query_vnic_env_in, in, opcode,
+ MLX5_CMD_OP_QUERY_VNIC_ENV);
+ MLX5_SET(query_vnic_env_in, in, op_mod, 0);
+ MLX5_SET(query_vnic_env_in, in, vport_number, vport);
+ if (vport)
+ MLX5_SET(query_vnic_env_in, in, other_vport, 1);
+
+ err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+ if (err)
+ return err;
+
+ *rx_discard_vport_down = MLX5_GET64(query_vnic_env_out, out,
+ vport_env.receive_discard_vport_down);
+ *tx_discard_vport_down = MLX5_GET64(query_vnic_env_out, out,
+ vport_env.transmit_discard_vport_down);
+ return 0;
+}
+
int mlx5_core_modify_hca_vport_context(struct mlx5_core_dev *dev,
u8 other_vport, u8 port_num,
int vf,
}
EXPORT_SYMBOL(mlxsw_afa_block_jump);
+int mlxsw_afa_block_terminate(struct mlxsw_afa_block *block)
+{
+ if (block->finished)
+ return -EINVAL;
+ mlxsw_afa_set_goto_set(block->cur_set,
+ MLXSW_AFA_SET_GOTO_BINDING_CMD_TERM, 0);
+ block->finished = true;
+ return 0;
+}
+EXPORT_SYMBOL(mlxsw_afa_block_terminate);
+
static struct mlxsw_afa_fwd_entry *
mlxsw_afa_fwd_entry_create(struct mlxsw_afa *mlxsw_afa, u8 local_port)
{
u32 mlxsw_afa_block_first_set_kvdl_index(struct mlxsw_afa_block *block);
int mlxsw_afa_block_continue(struct mlxsw_afa_block *block);
int mlxsw_afa_block_jump(struct mlxsw_afa_block *block, u16 group_id);
+int mlxsw_afa_block_terminate(struct mlxsw_afa_block *block);
int mlxsw_afa_block_append_drop(struct mlxsw_afa_block *block);
int mlxsw_afa_block_append_trap(struct mlxsw_afa_block *block, u16 trap_id);
int mlxsw_afa_block_append_trap_and_forward(struct mlxsw_afa_block *block,
switch (attr_type) {
case MLXSW_HWMON_ATTR_TYPE_TEMP:
mlxsw_hwmon_attr->dev_attr.show = mlxsw_hwmon_temp_show;
- mlxsw_hwmon_attr->dev_attr.attr.mode = S_IRUGO;
+ mlxsw_hwmon_attr->dev_attr.attr.mode = 0444;
snprintf(mlxsw_hwmon_attr->name, sizeof(mlxsw_hwmon_attr->name),
"temp%u_input", num + 1);
break;
case MLXSW_HWMON_ATTR_TYPE_TEMP_MAX:
mlxsw_hwmon_attr->dev_attr.show = mlxsw_hwmon_temp_max_show;
- mlxsw_hwmon_attr->dev_attr.attr.mode = S_IRUGO;
+ mlxsw_hwmon_attr->dev_attr.attr.mode = 0444;
snprintf(mlxsw_hwmon_attr->name, sizeof(mlxsw_hwmon_attr->name),
"temp%u_highest", num + 1);
break;
case MLXSW_HWMON_ATTR_TYPE_TEMP_RST:
mlxsw_hwmon_attr->dev_attr.store = mlxsw_hwmon_temp_rst_store;
- mlxsw_hwmon_attr->dev_attr.attr.mode = S_IWUSR;
+ mlxsw_hwmon_attr->dev_attr.attr.mode = 0200;
snprintf(mlxsw_hwmon_attr->name, sizeof(mlxsw_hwmon_attr->name),
"temp%u_reset_history", num + 1);
break;
case MLXSW_HWMON_ATTR_TYPE_FAN_RPM:
mlxsw_hwmon_attr->dev_attr.show = mlxsw_hwmon_fan_rpm_show;
- mlxsw_hwmon_attr->dev_attr.attr.mode = S_IRUGO;
+ mlxsw_hwmon_attr->dev_attr.attr.mode = 0444;
snprintf(mlxsw_hwmon_attr->name, sizeof(mlxsw_hwmon_attr->name),
"fan%u_input", num + 1);
break;
case MLXSW_HWMON_ATTR_TYPE_PWM:
mlxsw_hwmon_attr->dev_attr.show = mlxsw_hwmon_pwm_show;
mlxsw_hwmon_attr->dev_attr.store = mlxsw_hwmon_pwm_store;
- mlxsw_hwmon_attr->dev_attr.attr.mode = S_IWUSR | S_IRUGO;
+ mlxsw_hwmon_attr->dev_attr.attr.mode = 0644;
snprintf(mlxsw_hwmon_attr->name, sizeof(mlxsw_hwmon_attr->name),
"pwm%u", num + 1);
break;
MLXSW_REG_DEFINE(ptys, MLXSW_REG_PTYS_ID, MLXSW_REG_PTYS_LEN);
+/* an_disable_admin
+ * Auto negotiation disable administrative configuration
+ * 0 - Device doesn't support AN disable.
+ * 1 - Device supports AN disable.
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, ptys, an_disable_admin, 0x00, 30, 1);
+
/* reg_ptys_local_port
* Local port number.
* Access: Index
MLXSW_ITEM32(reg, ptys, eth_proto_lp_advertise, 0x30, 0, 32);
static inline void mlxsw_reg_ptys_eth_pack(char *payload, u8 local_port,
- u32 proto_admin)
+ u32 proto_admin, bool autoneg)
{
MLXSW_REG_ZERO(ptys, payload);
mlxsw_reg_ptys_local_port_set(payload, local_port);
mlxsw_reg_ptys_proto_mask_set(payload, MLXSW_REG_PTYS_PROTO_MASK_ETH);
mlxsw_reg_ptys_eth_proto_admin_set(payload, proto_admin);
+ mlxsw_reg_ptys_an_disable_admin_set(payload, !autoneg);
}
static inline void mlxsw_reg_ptys_eth_unpack(char *payload,
*/
MLXSW_ITEM32(reg, ritr, ipv4_mc, 0x00, 27, 1);
+/* reg_ritr_ipv6_mc
+ * IPv6 multicast routing enable.
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, ritr, ipv6_mc, 0x00, 26, 1);
+
enum mlxsw_reg_ritr_if_type {
/* VLAN interface. */
MLXSW_REG_RITR_VLAN_IF,
*/
MLXSW_ITEM32(reg, ritr, ipv4_mc_fe, 0x04, 27, 1);
+/* reg_ritr_ipv6_mc_fe
+ * IPv6 Multicast Forwarding Enable.
+ * When disabled, forwarding is blocked but local traffic (traps and IP to me)
+ * will be enabled.
+ * Access: RW
+ */
+MLXSW_ITEM32(reg, ritr, ipv6_mc_fe, 0x04, 26, 1);
+
/* reg_ritr_lb_en
* Loop-back filter enable for unicast packets.
* If the flag is set then loop-back filter for unicast packets is
mlxsw_reg_ritr_ipv4_set(payload, 1);
mlxsw_reg_ritr_ipv6_set(payload, 1);
mlxsw_reg_ritr_ipv4_mc_set(payload, 1);
+ mlxsw_reg_ritr_ipv6_mc_set(payload, 1);
mlxsw_reg_ritr_type_set(payload, type);
mlxsw_reg_ritr_op_set(payload, op);
mlxsw_reg_ritr_rif_set(payload, rif);
mlxsw_reg_ritr_ipv4_fe_set(payload, 1);
mlxsw_reg_ritr_ipv6_fe_set(payload, 1);
mlxsw_reg_ritr_ipv4_mc_fe_set(payload, 1);
+ mlxsw_reg_ritr_ipv6_mc_fe_set(payload, 1);
mlxsw_reg_ritr_lb_en_set(payload, 1);
mlxsw_reg_ritr_virtual_router_set(payload, vr_id);
mlxsw_reg_ritr_mtu_set(payload, mtu);
*/
MLXSW_ITEM32(reg, rmft2, irif, 0x08, 0, 16);
-/* reg_rmft2_dip4
- * Destination IPv4 address
+/* reg_rmft2_dip{4,6}
+ * Destination IPv4/6 address
* Access: RW
*/
+MLXSW_ITEM_BUF(reg, rmft2, dip6, 0x10, 16);
MLXSW_ITEM32(reg, rmft2, dip4, 0x1C, 0, 32);
-/* reg_rmft2_dip4_mask
+/* reg_rmft2_dip{4,6}_mask
* A bit that is set directs the TCAM to compare the corresponding bit in key. A
* bit that is clear directs the TCAM to ignore the corresponding bit in key.
* Access: RW
*/
+MLXSW_ITEM_BUF(reg, rmft2, dip6_mask, 0x20, 16);
MLXSW_ITEM32(reg, rmft2, dip4_mask, 0x2C, 0, 32);
-/* reg_rmft2_sip4
- * Source IPv4 address
+/* reg_rmft2_sip{4,6}
+ * Source IPv4/6 address
* Access: RW
*/
+MLXSW_ITEM_BUF(reg, rmft2, sip6, 0x30, 16);
MLXSW_ITEM32(reg, rmft2, sip4, 0x3C, 0, 32);
-/* reg_rmft2_sip4_mask
+/* reg_rmft2_sip{4,6}_mask
* A bit that is set directs the TCAM to compare the corresponding bit in key. A
* bit that is clear directs the TCAM to ignore the corresponding bit in key.
* Access: RW
*/
+MLXSW_ITEM_BUF(reg, rmft2, sip6_mask, 0x40, 16);
MLXSW_ITEM32(reg, rmft2, sip4_mask, 0x4C, 0, 32);
/* reg_rmft2_flexible_action_set
MLXSW_REG_FLEX_ACTION_SET_LEN);
static inline void
-mlxsw_reg_rmft2_ipv4_pack(char *payload, bool v, u16 offset, u16 virtual_router,
- enum mlxsw_reg_rmft2_irif_mask irif_mask, u16 irif,
- u32 dip4, u32 dip4_mask, u32 sip4, u32 sip4_mask,
- const char *flexible_action_set)
+mlxsw_reg_rmft2_common_pack(char *payload, bool v, u16 offset,
+ u16 virtual_router,
+ enum mlxsw_reg_rmft2_irif_mask irif_mask, u16 irif,
+ const char *flex_action_set)
{
MLXSW_REG_ZERO(rmft2, payload);
mlxsw_reg_rmft2_v_set(payload, v);
- mlxsw_reg_rmft2_type_set(payload, MLXSW_REG_RMFT2_TYPE_IPV4);
mlxsw_reg_rmft2_op_set(payload, MLXSW_REG_RMFT2_OP_READ_WRITE);
mlxsw_reg_rmft2_offset_set(payload, offset);
mlxsw_reg_rmft2_virtual_router_set(payload, virtual_router);
mlxsw_reg_rmft2_irif_mask_set(payload, irif_mask);
mlxsw_reg_rmft2_irif_set(payload, irif);
+ if (flex_action_set)
+ mlxsw_reg_rmft2_flexible_action_set_memcpy_to(payload,
+ flex_action_set);
+}
+
+static inline void
+mlxsw_reg_rmft2_ipv4_pack(char *payload, bool v, u16 offset, u16 virtual_router,
+ enum mlxsw_reg_rmft2_irif_mask irif_mask, u16 irif,
+ u32 dip4, u32 dip4_mask, u32 sip4, u32 sip4_mask,
+ const char *flexible_action_set)
+{
+ mlxsw_reg_rmft2_common_pack(payload, v, offset, virtual_router,
+ irif_mask, irif, flexible_action_set);
+ mlxsw_reg_rmft2_type_set(payload, MLXSW_REG_RMFT2_TYPE_IPV4);
mlxsw_reg_rmft2_dip4_set(payload, dip4);
mlxsw_reg_rmft2_dip4_mask_set(payload, dip4_mask);
mlxsw_reg_rmft2_sip4_set(payload, sip4);
mlxsw_reg_rmft2_sip4_mask_set(payload, sip4_mask);
- if (flexible_action_set)
- mlxsw_reg_rmft2_flexible_action_set_memcpy_to(payload,
- flexible_action_set);
+}
+
+static inline void
+mlxsw_reg_rmft2_ipv6_pack(char *payload, bool v, u16 offset, u16 virtual_router,
+ enum mlxsw_reg_rmft2_irif_mask irif_mask, u16 irif,
+ struct in6_addr dip6, struct in6_addr dip6_mask,
+ struct in6_addr sip6, struct in6_addr sip6_mask,
+ const char *flexible_action_set)
+{
+ mlxsw_reg_rmft2_common_pack(payload, v, offset, virtual_router,
+ irif_mask, irif, flexible_action_set);
+ mlxsw_reg_rmft2_type_set(payload, MLXSW_REG_RMFT2_TYPE_IPV6);
+ mlxsw_reg_rmft2_dip6_memcpy_to(payload, (void *)&dip6);
+ mlxsw_reg_rmft2_dip6_mask_memcpy_to(payload, (void *)&dip6_mask);
+ mlxsw_reg_rmft2_sip6_memcpy_to(payload, (void *)&sip6);
+ mlxsw_reg_rmft2_sip6_mask_memcpy_to(payload, (void *)&sip6_mask);
}
/* MFCR - Management Fan Control Register
#include "../mlxfw/mlxfw.h"
#define MLXSW_FWREV_MAJOR 13
-#define MLXSW_FWREV_MINOR 1530
-#define MLXSW_FWREV_SUBMINOR 152
+#define MLXSW_FWREV_MINOR 1620
+#define MLXSW_FWREV_SUBMINOR 192
#define MLXSW_FWREV_MINOR_TO_BRANCH(minor) ((minor) / 100)
#define MLXSW_SP_FW_FILENAME \
int err;
autoneg = mlxsw_sp_port->link.autoneg;
- mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sp_port->local_port, 0);
+ mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sp_port->local_port, 0, false);
err = mlxsw_reg_query(mlxsw_sp->core, MLXSW_REG(ptys), ptys_pl);
if (err)
return err;
bool autoneg;
int err;
- mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sp_port->local_port, 0);
+ mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sp_port->local_port, 0, false);
err = mlxsw_reg_query(mlxsw_sp->core, MLXSW_REG(ptys), ptys_pl);
if (err)
return err;
}
mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sp_port->local_port,
- eth_proto_new);
+ eth_proto_new, autoneg);
err = mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(ptys), ptys_pl);
if (err)
return err;
eth_proto_admin = mlxsw_sp_to_ptys_upper_speed(upper_speed);
mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sp_port->local_port,
- eth_proto_admin);
+ eth_proto_admin, mlxsw_sp_port->link.autoneg);
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(ptys), ptys_pl);
}
MLXSW_SP_RXL_NO_MARK(ACL0, TRAP_TO_CPU, IP2ME, false),
/* Multicast Router Traps */
MLXSW_SP_RXL_MARK(IPV4_PIM, TRAP_TO_CPU, PIM, false),
+ MLXSW_SP_RXL_MARK(IPV6_PIM, TRAP_TO_CPU, PIM, false),
MLXSW_SP_RXL_MARK(RPF, TRAP_TO_CPU, RPF, false),
MLXSW_SP_RXL_MARK(ACL1, TRAP_TO_CPU, MULTICAST, false),
MLXSW_SP_RXL_MR_MARK(ACL2, TRAP_TO_CPU, MULTICAST, false),
kvd_size = MLXSW_CORE_RES_GET(mlxsw_core, KVD_SIZE);
err = devlink_resource_register(devlink, MLXSW_SP_RESOURCE_NAME_KVD,
- true, kvd_size,
- MLXSW_SP_RESOURCE_KVD,
+ kvd_size, MLXSW_SP_RESOURCE_KVD,
DEVLINK_RESOURCE_ID_PARENT_TOP,
&kvd_size_params,
NULL);
linear_size = profile->kvd_linear_size;
err = devlink_resource_register(devlink, MLXSW_SP_RESOURCE_NAME_KVD_LINEAR,
- false, linear_size,
+ linear_size,
MLXSW_SP_RESOURCE_KVD_LINEAR,
MLXSW_SP_RESOURCE_KVD,
&linear_size_params,
profile->kvd_hash_single_parts;
double_size = rounddown(double_size, profile->kvd_hash_granularity);
err = devlink_resource_register(devlink, MLXSW_SP_RESOURCE_NAME_KVD_HASH_DOUBLE,
- false, double_size,
+ double_size,
MLXSW_SP_RESOURCE_KVD_HASH_DOUBLE,
MLXSW_SP_RESOURCE_KVD,
&hash_double_size_params,
single_size = kvd_size - double_size - linear_size;
err = devlink_resource_register(devlink, MLXSW_SP_RESOURCE_NAME_KVD_HASH_SINGLE,
- false, single_size,
+ single_size,
MLXSW_SP_RESOURCE_KVD_HASH_SINGLE,
MLXSW_SP_RESOURCE_KVD,
&hash_single_size_params,
int mlxsw_sp_acl_rulei_act_continue(struct mlxsw_sp_acl_rule_info *rulei);
int mlxsw_sp_acl_rulei_act_jump(struct mlxsw_sp_acl_rule_info *rulei,
u16 group_id);
+int mlxsw_sp_acl_rulei_act_terminate(struct mlxsw_sp_acl_rule_info *rulei);
int mlxsw_sp_acl_rulei_act_drop(struct mlxsw_sp_acl_rule_info *rulei);
int mlxsw_sp_acl_rulei_act_trap(struct mlxsw_sp_acl_rule_info *rulei);
int mlxsw_sp_acl_rulei_act_mirror(struct mlxsw_sp *mlxsw_sp,
return mlxsw_afa_block_jump(rulei->act_block, group_id);
}
+int mlxsw_sp_acl_rulei_act_terminate(struct mlxsw_sp_acl_rule_info *rulei)
+{
+ return mlxsw_afa_block_terminate(rulei->act_block);
+}
+
int mlxsw_sp_acl_rulei_act_drop(struct mlxsw_sp_acl_rule_info *rulei)
{
return mlxsw_afa_block_append_drop(rulei->act_block);
static const struct mlxsw_sp_sb_cm mlxsw_sp_cpu_port_sb_cms[] = {
MLXSW_SP_CPU_PORT_SB_CM,
+ MLXSW_SP_SB_CM(MLXSW_PORT_MAX_MTU, 0, 0),
+ MLXSW_SP_SB_CM(MLXSW_PORT_MAX_MTU, 0, 0),
+ MLXSW_SP_SB_CM(MLXSW_PORT_MAX_MTU, 0, 0),
+ MLXSW_SP_SB_CM(MLXSW_PORT_MAX_MTU, 0, 0),
+ MLXSW_SP_SB_CM(MLXSW_PORT_MAX_MTU, 0, 0),
MLXSW_SP_CPU_PORT_SB_CM,
- MLXSW_SP_CPU_PORT_SB_CM,
- MLXSW_SP_CPU_PORT_SB_CM,
- MLXSW_SP_CPU_PORT_SB_CM,
- MLXSW_SP_CPU_PORT_SB_CM,
- MLXSW_SP_CPU_PORT_SB_CM,
- MLXSW_SP_SB_CM(10000, 0, 0),
+ MLXSW_SP_SB_CM(MLXSW_PORT_MAX_MTU, 0, 0),
MLXSW_SP_CPU_PORT_SB_CM,
MLXSW_SP_CPU_PORT_SB_CM,
MLXSW_SP_CPU_PORT_SB_CM,
tcf_exts_to_list(exts, &actions);
list_for_each_entry(a, &actions, list) {
if (is_tcf_gact_ok(a)) {
- err = mlxsw_sp_acl_rulei_act_continue(rulei);
+ err = mlxsw_sp_acl_rulei_act_terminate(rulei);
if (err)
return err;
} else if (is_tcf_gact_shot(a)) {
mlxsw_sp_kvdl_resource_size_params_prepare(devlink);
err = devlink_resource_register(devlink, MLXSW_SP_RESOURCE_NAME_KVD_LINEAR_SINGLES,
- false, MLXSW_SP_KVDL_SINGLE_SIZE,
+ MLXSW_SP_KVDL_SINGLE_SIZE,
MLXSW_SP_RESOURCE_KVD_LINEAR_SINGLE,
MLXSW_SP_RESOURCE_KVD_LINEAR,
&mlxsw_sp_kvdl_single_size_params,
return err;
err = devlink_resource_register(devlink, MLXSW_SP_RESOURCE_NAME_KVD_LINEAR_CHUNKS,
- false, MLXSW_SP_KVDL_CHUNKS_SIZE,
+ MLXSW_SP_KVDL_CHUNKS_SIZE,
MLXSW_SP_RESOURCE_KVD_LINEAR_CHUNKS,
MLXSW_SP_RESOURCE_KVD_LINEAR,
&mlxsw_sp_kvdl_chunks_size_params,
return err;
err = devlink_resource_register(devlink, MLXSW_SP_RESOURCE_NAME_KVD_LINEAR_LARGE_CHUNKS,
- false, MLXSW_SP_KVDL_LARGE_CHUNKS_SIZE,
+ MLXSW_SP_KVDL_LARGE_CHUNKS_SIZE,
MLXSW_SP_RESOURCE_KVD_LINEAR_LARGE_CHUNKS,
MLXSW_SP_RESOURCE_KVD_LINEAR,
&mlxsw_sp_kvdl_large_chunks_size_params,
*/
#include <linux/rhashtable.h>
+#include <net/ipv6.h>
#include "spectrum_mr.h"
#include "spectrum_router.h"
/* priv has to be always the last item */
};
+struct mlxsw_sp_mr_vif;
+struct mlxsw_sp_mr_vif_ops {
+ bool (*is_regular)(const struct mlxsw_sp_mr_vif *vif);
+};
+
struct mlxsw_sp_mr_vif {
struct net_device *dev;
const struct mlxsw_sp_rif *rif;
* instance is used as an ingress VIF
*/
struct list_head route_ivif_list;
+
+ /* Protocol specific operations for a VIF */
+ const struct mlxsw_sp_mr_vif_ops *ops;
};
struct mlxsw_sp_mr_route_vif_entry {
struct mlxsw_sp_mr_route *mr_route;
};
+struct mlxsw_sp_mr_table;
+struct mlxsw_sp_mr_table_ops {
+ bool (*is_route_valid)(const struct mlxsw_sp_mr_table *mr_table,
+ const struct mr_mfc *mfc);
+ void (*key_create)(struct mlxsw_sp_mr_table *mr_table,
+ struct mlxsw_sp_mr_route_key *key,
+ struct mr_mfc *mfc);
+ bool (*is_route_starg)(const struct mlxsw_sp_mr_table *mr_table,
+ const struct mlxsw_sp_mr_route *mr_route);
+};
+
struct mlxsw_sp_mr_table {
struct list_head node;
enum mlxsw_sp_l3proto proto;
struct mlxsw_sp_mr_vif vifs[MAXVIFS];
struct list_head route_list;
struct rhashtable route_ht;
+ const struct mlxsw_sp_mr_table_ops *ops;
char catchall_route_priv[0];
/* catchall_route_priv has to be always the last item */
};
struct mlxsw_sp_mr_route_key key;
enum mlxsw_sp_mr_route_action route_action;
u16 min_mtu;
- struct mfc_cache *mfc4;
+ struct mr_mfc *mfc;
void *route_priv;
const struct mlxsw_sp_mr_table *mr_table;
/* A list of route_vif_entry structs that point to the egress VIFs */
.automatic_shrinking = true,
};
-static bool mlxsw_sp_mr_vif_regular(const struct mlxsw_sp_mr_vif *vif)
-{
- return !(vif->vif_flags & (VIFF_TUNNEL | VIFF_REGISTER));
-}
-
static bool mlxsw_sp_mr_vif_valid(const struct mlxsw_sp_mr_vif *vif)
{
- return mlxsw_sp_mr_vif_regular(vif) && vif->dev && vif->rif;
+ return vif->ops->is_regular(vif) && vif->dev && vif->rif;
}
static bool mlxsw_sp_mr_vif_exists(const struct mlxsw_sp_mr_vif *vif)
static bool
mlxsw_sp_mr_route_ivif_in_evifs(const struct mlxsw_sp_mr_route *mr_route)
{
- vifi_t ivif;
+ vifi_t ivif = mr_route->mfc->mfc_parent;
- switch (mr_route->mr_table->proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- ivif = mr_route->mfc4->_c.mfc_parent;
- return mr_route->mfc4->_c.mfc_un.res.ttls[ivif] != 255;
- case MLXSW_SP_L3_PROTO_IPV6:
- /* fall through */
- default:
- WARN_ON_ONCE(1);
- }
- return false;
+ return mr_route->mfc->mfc_un.res.ttls[ivif] != 255;
}
static int
return valid_evifs;
}
-static bool mlxsw_sp_mr_route_starg(const struct mlxsw_sp_mr_route *mr_route)
-{
- switch (mr_route->mr_table->proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- return mr_route->key.source_mask.addr4 == htonl(INADDR_ANY);
- case MLXSW_SP_L3_PROTO_IPV6:
- /* fall through */
- default:
- WARN_ON_ONCE(1);
- }
- return false;
-}
-
static enum mlxsw_sp_mr_route_action
mlxsw_sp_mr_route_action(const struct mlxsw_sp_mr_route *mr_route)
{
/* The kernel does not match a (*,G) route that the ingress interface is
* not one of the egress interfaces, so trap these kind of routes.
*/
- if (mlxsw_sp_mr_route_starg(mr_route) &&
+ if (mr_route->mr_table->ops->is_route_starg(mr_route->mr_table,
+ mr_route) &&
!mlxsw_sp_mr_route_ivif_in_evifs(mr_route))
return MLXSW_SP_MR_ROUTE_ACTION_TRAP;
static enum mlxsw_sp_mr_route_prio
mlxsw_sp_mr_route_prio(const struct mlxsw_sp_mr_route *mr_route)
{
- return mlxsw_sp_mr_route_starg(mr_route) ?
+ return mr_route->mr_table->ops->is_route_starg(mr_route->mr_table,
+ mr_route) ?
MLXSW_SP_MR_ROUTE_PRIO_STARG : MLXSW_SP_MR_ROUTE_PRIO_SG;
}
-static void mlxsw_sp_mr_route4_key(struct mlxsw_sp_mr_table *mr_table,
- struct mlxsw_sp_mr_route_key *key,
- const struct mfc_cache *mfc)
-{
- bool starg = (mfc->mfc_origin == htonl(INADDR_ANY));
-
- memset(key, 0, sizeof(*key));
- key->vrid = mr_table->vr_id;
- key->proto = mr_table->proto;
- key->group.addr4 = mfc->mfc_mcastgrp;
- key->group_mask.addr4 = htonl(0xffffffff);
- key->source.addr4 = mfc->mfc_origin;
- key->source_mask.addr4 = htonl(starg ? 0 : 0xffffffff);
-}
-
static int mlxsw_sp_mr_route_evif_link(struct mlxsw_sp_mr_route *mr_route,
struct mlxsw_sp_mr_vif *mr_vif)
{
}
static struct mlxsw_sp_mr_route *
-mlxsw_sp_mr_route4_create(struct mlxsw_sp_mr_table *mr_table,
- struct mfc_cache *mfc)
+mlxsw_sp_mr_route_create(struct mlxsw_sp_mr_table *mr_table,
+ struct mr_mfc *mfc)
{
struct mlxsw_sp_mr_route_vif_entry *rve, *tmp;
struct mlxsw_sp_mr_route *mr_route;
if (!mr_route)
return ERR_PTR(-ENOMEM);
INIT_LIST_HEAD(&mr_route->evif_list);
- mlxsw_sp_mr_route4_key(mr_table, &mr_route->key, mfc);
/* Find min_mtu and link iVIF and eVIFs */
mr_route->min_mtu = ETH_MAX_MTU;
- ipmr_cache_hold(mfc);
- mr_route->mfc4 = mfc;
+ mr_cache_hold(mfc);
+ mr_route->mfc = mfc;
+ mr_table->ops->key_create(mr_table, &mr_route->key, mr_route->mfc);
+
mr_route->mr_table = mr_table;
for (i = 0; i < MAXVIFS; i++) {
- if (mfc->_c.mfc_un.res.ttls[i] != 255) {
+ if (mfc->mfc_un.res.ttls[i] != 255) {
err = mlxsw_sp_mr_route_evif_link(mr_route,
&mr_table->vifs[i]);
if (err)
}
}
mlxsw_sp_mr_route_ivif_link(mr_route,
- &mr_table->vifs[mfc->_c.mfc_parent]);
+ &mr_table->vifs[mfc->mfc_parent]);
mr_route->route_action = mlxsw_sp_mr_route_action(mr_route);
return mr_route;
err:
- ipmr_cache_put(mfc);
+ mr_cache_put(mfc);
list_for_each_entry_safe(rve, tmp, &mr_route->evif_list, route_node)
mlxsw_sp_mr_route_evif_unlink(rve);
kfree(mr_route);
return ERR_PTR(err);
}
-static void mlxsw_sp_mr_route4_destroy(struct mlxsw_sp_mr_table *mr_table,
- struct mlxsw_sp_mr_route *mr_route)
+static void mlxsw_sp_mr_route_destroy(struct mlxsw_sp_mr_table *mr_table,
+ struct mlxsw_sp_mr_route *mr_route)
{
struct mlxsw_sp_mr_route_vif_entry *rve, *tmp;
mlxsw_sp_mr_route_ivif_unlink(mr_route);
- ipmr_cache_put(mr_route->mfc4);
+ mr_cache_put(mr_route->mfc);
list_for_each_entry_safe(rve, tmp, &mr_route->evif_list, route_node)
mlxsw_sp_mr_route_evif_unlink(rve);
kfree(mr_route);
}
-static void mlxsw_sp_mr_route_destroy(struct mlxsw_sp_mr_table *mr_table,
- struct mlxsw_sp_mr_route *mr_route)
-{
- switch (mr_table->proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- mlxsw_sp_mr_route4_destroy(mr_table, mr_route);
- break;
- case MLXSW_SP_L3_PROTO_IPV6:
- /* fall through */
- default:
- WARN_ON_ONCE(1);
- }
-}
-
static void mlxsw_sp_mr_mfc_offload_set(struct mlxsw_sp_mr_route *mr_route,
bool offload)
{
- switch (mr_route->mr_table->proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- if (offload)
- mr_route->mfc4->_c.mfc_flags |= MFC_OFFLOAD;
- else
- mr_route->mfc4->_c.mfc_flags &= ~MFC_OFFLOAD;
- break;
- case MLXSW_SP_L3_PROTO_IPV6:
- /* fall through */
- default:
- WARN_ON_ONCE(1);
- }
+ if (offload)
+ mr_route->mfc->mfc_flags |= MFC_OFFLOAD;
+ else
+ mr_route->mfc->mfc_flags &= ~MFC_OFFLOAD;
}
static void mlxsw_sp_mr_mfc_offload_update(struct mlxsw_sp_mr_route *mr_route)
mlxsw_sp_mr_route_destroy(mr_table, mr_route);
}
-int mlxsw_sp_mr_route4_add(struct mlxsw_sp_mr_table *mr_table,
- struct mfc_cache *mfc, bool replace)
+int mlxsw_sp_mr_route_add(struct mlxsw_sp_mr_table *mr_table,
+ struct mr_mfc *mfc, bool replace)
{
struct mlxsw_sp_mr_route *mr_orig_route = NULL;
struct mlxsw_sp_mr_route *mr_route;
int err;
- /* If the route is a (*,*) route, abort, as these kind of routes are
- * used for proxy routes.
- */
- if (mfc->mfc_origin == htonl(INADDR_ANY) &&
- mfc->mfc_mcastgrp == htonl(INADDR_ANY)) {
- dev_warn(mr_table->mlxsw_sp->bus_info->dev,
- "Offloading proxy routes is not supported.\n");
+ if (!mr_table->ops->is_route_valid(mr_table, mfc))
return -EINVAL;
- }
/* Create a new route */
- mr_route = mlxsw_sp_mr_route4_create(mr_table, mfc);
+ mr_route = mlxsw_sp_mr_route_create(mr_table, mfc);
if (IS_ERR(mr_route))
return PTR_ERR(mr_route);
&mr_orig_route->ht_node,
mlxsw_sp_mr_route_ht_params);
list_del(&mr_orig_route->node);
- mlxsw_sp_mr_route4_destroy(mr_table, mr_orig_route);
+ mlxsw_sp_mr_route_destroy(mr_table, mr_orig_route);
}
mlxsw_sp_mr_mfc_offload_update(mr_route);
list_del(&mr_route->node);
err_no_orig_route:
err_duplicate_route:
- mlxsw_sp_mr_route4_destroy(mr_table, mr_route);
+ mlxsw_sp_mr_route_destroy(mr_table, mr_route);
return err;
}
-void mlxsw_sp_mr_route4_del(struct mlxsw_sp_mr_table *mr_table,
- struct mfc_cache *mfc)
+void mlxsw_sp_mr_route_del(struct mlxsw_sp_mr_table *mr_table,
+ struct mr_mfc *mfc)
{
struct mlxsw_sp_mr_route *mr_route;
struct mlxsw_sp_mr_route_key key;
- mlxsw_sp_mr_route4_key(mr_table, &key, mfc);
+ mr_table->ops->key_create(mr_table, &key, mfc);
mr_route = rhashtable_lookup_fast(&mr_table->route_ht, &key,
mlxsw_sp_mr_route_ht_params);
if (mr_route)
}
}
+/* Protocol specific functions */
+static bool
+mlxsw_sp_mr_route4_validate(const struct mlxsw_sp_mr_table *mr_table,
+ const struct mr_mfc *c)
+{
+ struct mfc_cache *mfc = (struct mfc_cache *) c;
+
+ /* If the route is a (*,*) route, abort, as these kind of routes are
+ * used for proxy routes.
+ */
+ if (mfc->mfc_origin == htonl(INADDR_ANY) &&
+ mfc->mfc_mcastgrp == htonl(INADDR_ANY)) {
+ dev_warn(mr_table->mlxsw_sp->bus_info->dev,
+ "Offloading proxy routes is not supported.\n");
+ return false;
+ }
+ return true;
+}
+
+static void mlxsw_sp_mr_route4_key(struct mlxsw_sp_mr_table *mr_table,
+ struct mlxsw_sp_mr_route_key *key,
+ struct mr_mfc *c)
+{
+ const struct mfc_cache *mfc = (struct mfc_cache *) c;
+ bool starg;
+
+ starg = (mfc->mfc_origin == htonl(INADDR_ANY));
+
+ memset(key, 0, sizeof(*key));
+ key->vrid = mr_table->vr_id;
+ key->proto = MLXSW_SP_L3_PROTO_IPV4;
+ key->group.addr4 = mfc->mfc_mcastgrp;
+ key->group_mask.addr4 = htonl(0xffffffff);
+ key->source.addr4 = mfc->mfc_origin;
+ key->source_mask.addr4 = htonl(starg ? 0 : 0xffffffff);
+}
+
+static bool mlxsw_sp_mr_route4_starg(const struct mlxsw_sp_mr_table *mr_table,
+ const struct mlxsw_sp_mr_route *mr_route)
+{
+ return mr_route->key.source_mask.addr4 == htonl(INADDR_ANY);
+}
+
+static bool mlxsw_sp_mr_vif4_is_regular(const struct mlxsw_sp_mr_vif *vif)
+{
+ return !(vif->vif_flags & (VIFF_TUNNEL | VIFF_REGISTER));
+}
+
+static bool
+mlxsw_sp_mr_route6_validate(const struct mlxsw_sp_mr_table *mr_table,
+ const struct mr_mfc *c)
+{
+ struct mfc6_cache *mfc = (struct mfc6_cache *) c;
+
+ /* If the route is a (*,*) route, abort, as these kind of routes are
+ * used for proxy routes.
+ */
+ if (ipv6_addr_any(&mfc->mf6c_origin) &&
+ ipv6_addr_any(&mfc->mf6c_mcastgrp)) {
+ dev_warn(mr_table->mlxsw_sp->bus_info->dev,
+ "Offloading proxy routes is not supported.\n");
+ return false;
+ }
+ return true;
+}
+
+static void mlxsw_sp_mr_route6_key(struct mlxsw_sp_mr_table *mr_table,
+ struct mlxsw_sp_mr_route_key *key,
+ struct mr_mfc *c)
+{
+ const struct mfc6_cache *mfc = (struct mfc6_cache *) c;
+
+ memset(key, 0, sizeof(*key));
+ key->vrid = mr_table->vr_id;
+ key->proto = MLXSW_SP_L3_PROTO_IPV6;
+ key->group.addr6 = mfc->mf6c_mcastgrp;
+ memset(&key->group_mask.addr6, 0xff, sizeof(key->group_mask.addr6));
+ key->source.addr6 = mfc->mf6c_origin;
+ if (!ipv6_addr_any(&mfc->mf6c_origin))
+ memset(&key->source_mask.addr6, 0xff,
+ sizeof(key->source_mask.addr6));
+}
+
+static bool mlxsw_sp_mr_route6_starg(const struct mlxsw_sp_mr_table *mr_table,
+ const struct mlxsw_sp_mr_route *mr_route)
+{
+ return ipv6_addr_any(&mr_route->key.source_mask.addr6);
+}
+
+static bool mlxsw_sp_mr_vif6_is_regular(const struct mlxsw_sp_mr_vif *vif)
+{
+ return !(vif->vif_flags & MIFF_REGISTER);
+}
+
+static struct
+mlxsw_sp_mr_vif_ops mlxsw_sp_mr_vif_ops_arr[] = {
+ {
+ .is_regular = mlxsw_sp_mr_vif4_is_regular,
+ },
+ {
+ .is_regular = mlxsw_sp_mr_vif6_is_regular,
+ },
+};
+
+static struct
+mlxsw_sp_mr_table_ops mlxsw_sp_mr_table_ops_arr[] = {
+ {
+ .is_route_valid = mlxsw_sp_mr_route4_validate,
+ .key_create = mlxsw_sp_mr_route4_key,
+ .is_route_starg = mlxsw_sp_mr_route4_starg,
+ },
+ {
+ .is_route_valid = mlxsw_sp_mr_route6_validate,
+ .key_create = mlxsw_sp_mr_route6_key,
+ .is_route_starg = mlxsw_sp_mr_route6_starg,
+ },
+
+};
+
struct mlxsw_sp_mr_table *mlxsw_sp_mr_table_create(struct mlxsw_sp *mlxsw_sp,
u32 vr_id,
enum mlxsw_sp_l3proto proto)
.prio = MLXSW_SP_MR_ROUTE_PRIO_CATCHALL,
.key = {
.vrid = vr_id,
+ .proto = proto,
},
.value = {
.route_action = MLXSW_SP_MR_ROUTE_ACTION_TRAP,
mr_table->vr_id = vr_id;
mr_table->mlxsw_sp = mlxsw_sp;
mr_table->proto = proto;
+ mr_table->ops = &mlxsw_sp_mr_table_ops_arr[proto];
INIT_LIST_HEAD(&mr_table->route_list);
err = rhashtable_init(&mr_table->route_ht,
for (i = 0; i < MAXVIFS; i++) {
INIT_LIST_HEAD(&mr_table->vifs[i].route_evif_list);
INIT_LIST_HEAD(&mr_table->vifs[i].route_ivif_list);
+ mr_table->vifs[i].ops = &mlxsw_sp_mr_vif_ops_arr[proto];
}
err = mr->mr_ops->route_create(mlxsw_sp, mr->priv,
mr->mr_ops->route_stats(mlxsw_sp, mr_route->route_priv, &packets,
&bytes);
- switch (mr_route->mr_table->proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- if (mr_route->mfc4->_c.mfc_un.res.pkt != packets)
- mr_route->mfc4->_c.mfc_un.res.lastuse = jiffies;
- mr_route->mfc4->_c.mfc_un.res.pkt = packets;
- mr_route->mfc4->_c.mfc_un.res.bytes = bytes;
- break;
- case MLXSW_SP_L3_PROTO_IPV6:
- /* fall through */
- default:
- WARN_ON_ONCE(1);
- }
+ if (mr_route->mfc->mfc_un.res.pkt != packets)
+ mr_route->mfc->mfc_un.res.lastuse = jiffies;
+ mr_route->mfc->mfc_un.res.pkt = packets;
+ mr_route->mfc->mfc_un.res.bytes = bytes;
}
static void mlxsw_sp_mr_stats_update(struct work_struct *work)
#define _MLXSW_SPECTRUM_MCROUTER_H
#include <linux/mroute.h>
+#include <linux/mroute6.h>
#include "spectrum_router.h"
#include "spectrum.h"
int mlxsw_sp_mr_init(struct mlxsw_sp *mlxsw_sp,
const struct mlxsw_sp_mr_ops *mr_ops);
void mlxsw_sp_mr_fini(struct mlxsw_sp *mlxsw_sp);
-int mlxsw_sp_mr_route4_add(struct mlxsw_sp_mr_table *mr_table,
- struct mfc_cache *mfc, bool replace);
-void mlxsw_sp_mr_route4_del(struct mlxsw_sp_mr_table *mr_table,
- struct mfc_cache *mfc);
+int mlxsw_sp_mr_route_add(struct mlxsw_sp_mr_table *mr_table,
+ struct mr_mfc *mfc, bool replace);
+void mlxsw_sp_mr_route_del(struct mlxsw_sp_mr_table *mr_table,
+ struct mr_mfc *mfc);
int mlxsw_sp_mr_vif_add(struct mlxsw_sp_mr_table *mr_table,
struct net_device *dev, vifi_t vif_index,
unsigned long vif_flags,
};
struct mlxsw_sp_mr_tcam {
- struct mlxsw_sp_mr_tcam_region ipv4_tcam_region;
+ struct mlxsw_sp_mr_tcam_region tcam_regions[MLXSW_SP_L3_PROTO_MAX];
};
/* This struct maps to one RIGR2 register entry */
mlxsw_afa_block_first_set(afa_block));
break;
case MLXSW_SP_L3_PROTO_IPV6:
- default:
- WARN_ON_ONCE(1);
+ mlxsw_reg_rmft2_ipv6_pack(rmft2_pl, true, parman_item->index,
+ key->vrid,
+ MLXSW_REG_RMFT2_IRIF_MASK_IGNORE, 0,
+ key->group.addr6,
+ key->group_mask.addr6,
+ key->source.addr6,
+ key->source_mask.addr6,
+ mlxsw_afa_block_first_set(afa_block));
}
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(rmft2), rmft2_pl);
}
static int mlxsw_sp_mr_tcam_route_remove(struct mlxsw_sp *mlxsw_sp, int vrid,
+ struct mlxsw_sp_mr_route_key *key,
struct parman_item *parman_item)
{
+ struct in6_addr zero_addr = IN6ADDR_ANY_INIT;
char rmft2_pl[MLXSW_REG_RMFT2_LEN];
- mlxsw_reg_rmft2_ipv4_pack(rmft2_pl, false, parman_item->index, vrid,
- 0, 0, 0, 0, 0, 0, NULL);
+ switch (key->proto) {
+ case MLXSW_SP_L3_PROTO_IPV4:
+ mlxsw_reg_rmft2_ipv4_pack(rmft2_pl, false, parman_item->index,
+ vrid, 0, 0, 0, 0, 0, 0, NULL);
+ break;
+ case MLXSW_SP_L3_PROTO_IPV6:
+ mlxsw_reg_rmft2_ipv6_pack(rmft2_pl, false, parman_item->index,
+ vrid, 0, 0, zero_addr, zero_addr,
+ zero_addr, zero_addr, NULL);
+ break;
+ }
return mlxsw_reg_write(mlxsw_sp->core, MLXSW_REG(rmft2), rmft2_pl);
}
return 0;
}
+static struct mlxsw_sp_mr_tcam_region *
+mlxsw_sp_mr_tcam_protocol_region(struct mlxsw_sp_mr_tcam *mr_tcam,
+ enum mlxsw_sp_l3proto proto)
+{
+ return &mr_tcam->tcam_regions[proto];
+}
+
static int
mlxsw_sp_mr_tcam_route_parman_item_add(struct mlxsw_sp_mr_tcam *mr_tcam,
struct mlxsw_sp_mr_tcam_route *route,
enum mlxsw_sp_mr_route_prio prio)
{
- struct parman_prio *parman_prio = NULL;
+ struct mlxsw_sp_mr_tcam_region *tcam_region;
int err;
- switch (route->key.proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- parman_prio = &mr_tcam->ipv4_tcam_region.parman_prios[prio];
- err = parman_item_add(mr_tcam->ipv4_tcam_region.parman,
- parman_prio, &route->parman_item);
- if (err)
- return err;
- break;
- case MLXSW_SP_L3_PROTO_IPV6:
- default:
- WARN_ON_ONCE(1);
- }
- route->parman_prio = parman_prio;
+ tcam_region = mlxsw_sp_mr_tcam_protocol_region(mr_tcam,
+ route->key.proto);
+ err = parman_item_add(tcam_region->parman,
+ &tcam_region->parman_prios[prio],
+ &route->parman_item);
+ if (err)
+ return err;
+
+ route->parman_prio = &tcam_region->parman_prios[prio];
return 0;
}
mlxsw_sp_mr_tcam_route_parman_item_remove(struct mlxsw_sp_mr_tcam *mr_tcam,
struct mlxsw_sp_mr_tcam_route *route)
{
- switch (route->key.proto) {
- case MLXSW_SP_L3_PROTO_IPV4:
- parman_item_remove(mr_tcam->ipv4_tcam_region.parman,
- route->parman_prio, &route->parman_item);
- break;
- case MLXSW_SP_L3_PROTO_IPV6:
- default:
- WARN_ON_ONCE(1);
- }
+ struct mlxsw_sp_mr_tcam_region *tcam_region;
+
+ tcam_region = mlxsw_sp_mr_tcam_protocol_region(mr_tcam,
+ route->key.proto);
+
+ parman_item_remove(tcam_region->parman,
+ route->parman_prio, &route->parman_item);
}
static int
struct mlxsw_sp_mr_tcam *mr_tcam = priv;
mlxsw_sp_mr_tcam_route_remove(mlxsw_sp, route->key.vrid,
- &route->parman_item);
+ &route->key, &route->parman_item);
mlxsw_sp_mr_tcam_route_parman_item_remove(mr_tcam, route);
mlxsw_sp_mr_tcam_afa_block_destroy(route->afa_block);
mlxsw_sp_flow_counter_free(mlxsw_sp, route->counter_index);
static int mlxsw_sp_mr_tcam_init(struct mlxsw_sp *mlxsw_sp, void *priv)
{
struct mlxsw_sp_mr_tcam *mr_tcam = priv;
+ struct mlxsw_sp_mr_tcam_region *region = &mr_tcam->tcam_regions[0];
+ u32 rtar_key;
+ int err;
if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, MC_ERIF_LIST_ENTRIES) ||
!MLXSW_CORE_RES_VALID(mlxsw_sp->core, ACL_MAX_TCAM_RULES))
return -EIO;
- return mlxsw_sp_mr_tcam_region_init(mlxsw_sp,
- &mr_tcam->ipv4_tcam_region,
- MLXSW_REG_RTAR_KEY_TYPE_IPV4_MULTICAST);
+ rtar_key = MLXSW_REG_RTAR_KEY_TYPE_IPV4_MULTICAST;
+ err = mlxsw_sp_mr_tcam_region_init(mlxsw_sp,
+ ®ion[MLXSW_SP_L3_PROTO_IPV4],
+ rtar_key);
+ if (err)
+ return err;
+
+ rtar_key = MLXSW_REG_RTAR_KEY_TYPE_IPV6_MULTICAST;
+ err = mlxsw_sp_mr_tcam_region_init(mlxsw_sp,
+ ®ion[MLXSW_SP_L3_PROTO_IPV6],
+ rtar_key);
+ if (err)
+ goto err_ipv6_region_init;
+
+ return 0;
+
+err_ipv6_region_init:
+ mlxsw_sp_mr_tcam_region_fini(®ion[MLXSW_SP_L3_PROTO_IPV4]);
+ return err;
}
static void mlxsw_sp_mr_tcam_fini(void *priv)
{
struct mlxsw_sp_mr_tcam *mr_tcam = priv;
+ struct mlxsw_sp_mr_tcam_region *region = &mr_tcam->tcam_regions[0];
- mlxsw_sp_mr_tcam_region_fini(&mr_tcam->ipv4_tcam_region);
+ mlxsw_sp_mr_tcam_region_fini(®ion[MLXSW_SP_L3_PROTO_IPV6]);
+ mlxsw_sp_mr_tcam_region_fini(®ion[MLXSW_SP_L3_PROTO_IPV4]);
}
const struct mlxsw_sp_mr_ops mlxsw_sp_mr_tcam_ops = {
unsigned int rif_count;
struct mlxsw_sp_fib *fib4;
struct mlxsw_sp_fib *fib6;
- struct mlxsw_sp_mr_table *mr4_table;
+ struct mlxsw_sp_mr_table *mr_table[MLXSW_SP_L3_PROTO_MAX];
};
static const struct rhashtable_params mlxsw_sp_fib_ht_params;
static bool mlxsw_sp_vr_is_used(const struct mlxsw_sp_vr *vr)
{
- return !!vr->fib4 || !!vr->fib6 || !!vr->mr4_table;
+ return !!vr->fib4 || !!vr->fib6 ||
+ !!vr->mr_table[MLXSW_SP_L3_PROTO_IPV4] ||
+ !!vr->mr_table[MLXSW_SP_L3_PROTO_IPV6];
}
static struct mlxsw_sp_vr *mlxsw_sp_vr_find_unused(struct mlxsw_sp *mlxsw_sp)
u32 tb_id,
struct netlink_ext_ack *extack)
{
- struct mlxsw_sp_mr_table *mr4_table;
+ struct mlxsw_sp_mr_table *mr4_table, *mr6_table;
struct mlxsw_sp_fib *fib4;
struct mlxsw_sp_fib *fib6;
struct mlxsw_sp_vr *vr;
MLXSW_SP_L3_PROTO_IPV4);
if (IS_ERR(mr4_table)) {
err = PTR_ERR(mr4_table);
- goto err_mr_table_create;
+ goto err_mr4_table_create;
}
+ mr6_table = mlxsw_sp_mr_table_create(mlxsw_sp, vr->id,
+ MLXSW_SP_L3_PROTO_IPV6);
+ if (IS_ERR(mr6_table)) {
+ err = PTR_ERR(mr6_table);
+ goto err_mr6_table_create;
+ }
+
vr->fib4 = fib4;
vr->fib6 = fib6;
- vr->mr4_table = mr4_table;
+ vr->mr_table[MLXSW_SP_L3_PROTO_IPV4] = mr4_table;
+ vr->mr_table[MLXSW_SP_L3_PROTO_IPV6] = mr6_table;
vr->tb_id = tb_id;
return vr;
-err_mr_table_create:
+err_mr6_table_create:
+ mlxsw_sp_mr_table_destroy(mr4_table);
+err_mr4_table_create:
mlxsw_sp_fib_destroy(mlxsw_sp, fib6);
err_fib6_create:
mlxsw_sp_fib_destroy(mlxsw_sp, fib4);
static void mlxsw_sp_vr_destroy(struct mlxsw_sp *mlxsw_sp,
struct mlxsw_sp_vr *vr)
{
- mlxsw_sp_mr_table_destroy(vr->mr4_table);
- vr->mr4_table = NULL;
+ mlxsw_sp_mr_table_destroy(vr->mr_table[MLXSW_SP_L3_PROTO_IPV6]);
+ vr->mr_table[MLXSW_SP_L3_PROTO_IPV6] = NULL;
+ mlxsw_sp_mr_table_destroy(vr->mr_table[MLXSW_SP_L3_PROTO_IPV4]);
+ vr->mr_table[MLXSW_SP_L3_PROTO_IPV4] = NULL;
mlxsw_sp_fib_destroy(mlxsw_sp, vr->fib6);
vr->fib6 = NULL;
mlxsw_sp_fib_destroy(mlxsw_sp, vr->fib4);
{
if (!vr->rif_count && list_empty(&vr->fib4->node_list) &&
list_empty(&vr->fib6->node_list) &&
- mlxsw_sp_mr_table_empty(vr->mr4_table))
+ mlxsw_sp_mr_table_empty(vr->mr_table[MLXSW_SP_L3_PROTO_IPV4]) &&
+ mlxsw_sp_mr_table_empty(vr->mr_table[MLXSW_SP_L3_PROTO_IPV6]))
mlxsw_sp_vr_destroy(mlxsw_sp, vr);
}
return 0;
}
+static struct mlxsw_sp_mr_table *
+mlxsw_sp_router_fibmr_family_to_table(struct mlxsw_sp_vr *vr, int family)
+{
+ if (family == RTNL_FAMILY_IPMR)
+ return vr->mr_table[MLXSW_SP_L3_PROTO_IPV4];
+ else
+ return vr->mr_table[MLXSW_SP_L3_PROTO_IPV6];
+}
+
static int mlxsw_sp_router_fibmr_add(struct mlxsw_sp *mlxsw_sp,
struct mfc_entry_notifier_info *men_info,
bool replace)
{
+ struct mlxsw_sp_mr_table *mrt;
struct mlxsw_sp_vr *vr;
if (mlxsw_sp->router->aborted)
if (IS_ERR(vr))
return PTR_ERR(vr);
- return mlxsw_sp_mr_route4_add(vr->mr4_table, men_info->mfc, replace);
+ mrt = mlxsw_sp_router_fibmr_family_to_table(vr, men_info->info.family);
+ return mlxsw_sp_mr_route_add(mrt, men_info->mfc, replace);
}
static void mlxsw_sp_router_fibmr_del(struct mlxsw_sp *mlxsw_sp,
struct mfc_entry_notifier_info *men_info)
{
+ struct mlxsw_sp_mr_table *mrt;
struct mlxsw_sp_vr *vr;
if (mlxsw_sp->router->aborted)
if (WARN_ON(!vr))
return;
- mlxsw_sp_mr_route4_del(vr->mr4_table, men_info->mfc);
+ mrt = mlxsw_sp_router_fibmr_family_to_table(vr, men_info->info.family);
+ mlxsw_sp_mr_route_del(mrt, men_info->mfc);
mlxsw_sp_vr_put(mlxsw_sp, vr);
}
mlxsw_sp_router_fibmr_vif_add(struct mlxsw_sp *mlxsw_sp,
struct vif_entry_notifier_info *ven_info)
{
+ struct mlxsw_sp_mr_table *mrt;
struct mlxsw_sp_rif *rif;
struct mlxsw_sp_vr *vr;
if (IS_ERR(vr))
return PTR_ERR(vr);
+ mrt = mlxsw_sp_router_fibmr_family_to_table(vr, ven_info->info.family);
rif = mlxsw_sp_rif_find_by_dev(mlxsw_sp, ven_info->dev);
- return mlxsw_sp_mr_vif_add(vr->mr4_table, ven_info->dev,
+ return mlxsw_sp_mr_vif_add(mrt, ven_info->dev,
ven_info->vif_index,
ven_info->vif_flags, rif);
}
mlxsw_sp_router_fibmr_vif_del(struct mlxsw_sp *mlxsw_sp,
struct vif_entry_notifier_info *ven_info)
{
+ struct mlxsw_sp_mr_table *mrt;
struct mlxsw_sp_vr *vr;
if (mlxsw_sp->router->aborted)
if (WARN_ON(!vr))
return;
- mlxsw_sp_mr_vif_del(vr->mr4_table, ven_info->vif_index);
+ mrt = mlxsw_sp_router_fibmr_family_to_table(vr, ven_info->info.family);
+ mlxsw_sp_mr_vif_del(mrt, ven_info->vif_index);
mlxsw_sp_vr_put(mlxsw_sp, vr);
}
static void mlxsw_sp_router_fib_flush(struct mlxsw_sp *mlxsw_sp)
{
- int i;
+ int i, j;
for (i = 0; i < MLXSW_CORE_RES_GET(mlxsw_sp->core, MAX_VRS); i++) {
struct mlxsw_sp_vr *vr = &mlxsw_sp->router->vrs[i];
if (!mlxsw_sp_vr_is_used(vr))
continue;
- mlxsw_sp_mr_table_flush(vr->mr4_table);
+ for (j = 0; j < MLXSW_SP_L3_PROTO_MAX; j++)
+ mlxsw_sp_mr_table_flush(vr->mr_table[j]);
mlxsw_sp_vr_fib_flush(mlxsw_sp, vr, MLXSW_SP_L3_PROTO_IPV4);
/* If virtual router was only used for IPv4, then it's no
replace);
if (err)
mlxsw_sp_router_fib_abort(mlxsw_sp);
- ipmr_cache_put(fib_work->men_info.mfc);
+ mr_cache_put(fib_work->men_info.mfc);
break;
case FIB_EVENT_ENTRY_DEL:
mlxsw_sp_router_fibmr_del(mlxsw_sp, &fib_work->men_info);
- ipmr_cache_put(fib_work->men_info.mfc);
+ mr_cache_put(fib_work->men_info.mfc);
break;
case FIB_EVENT_VIF_ADD:
err = mlxsw_sp_router_fibmr_vif_add(mlxsw_sp,
case FIB_EVENT_ENTRY_ADD: /* fall through */
case FIB_EVENT_ENTRY_DEL:
memcpy(&fib_work->men_info, info, sizeof(fib_work->men_info));
- ipmr_cache_hold(fib_work->men_info.mfc);
+ mr_cache_hold(fib_work->men_info.mfc);
break;
case FIB_EVENT_VIF_ADD: /* fall through */
case FIB_EVENT_VIF_DEL:
if (!ipmr_rule_default(rule) && !rule->l3mdev)
err = -1;
break;
+ case RTNL_FAMILY_IP6MR:
+ if (!ip6mr_rule_default(rule) && !rule->l3mdev)
+ err = -1;
+ break;
}
if (err < 0)
if (!net_eq(info->net, &init_net) ||
(info->family != AF_INET && info->family != AF_INET6 &&
- info->family != RTNL_FAMILY_IPMR))
+ info->family != RTNL_FAMILY_IPMR &&
+ info->family != RTNL_FAMILY_IP6MR))
return NOTIFY_DONE;
router = container_of(nb, struct mlxsw_sp_router, fib_nb);
INIT_WORK(&fib_work->work, mlxsw_sp_router_fib6_event_work);
mlxsw_sp_router_fib6_event(fib_work, info);
break;
+ case RTNL_FAMILY_IP6MR:
case RTNL_FAMILY_IPMR:
INIT_WORK(&fib_work->work, mlxsw_sp_router_fibmr_event_work);
mlxsw_sp_router_fibmr_event(fib_work, info);
struct mlxsw_sp_rif *rif;
struct mlxsw_sp_vr *vr;
u16 rif_index;
- int err;
+ int i, err;
type = mlxsw_sp_dev_rif_type(mlxsw_sp, params->dev);
ops = mlxsw_sp->router->rif_ops_arr[type];
if (err)
goto err_configure;
- err = mlxsw_sp_mr_rif_add(vr->mr4_table, rif);
- if (err)
- goto err_mr_rif_add;
+ for (i = 0; i < MLXSW_SP_L3_PROTO_MAX; i++) {
+ err = mlxsw_sp_mr_rif_add(vr->mr_table[i], rif);
+ if (err)
+ goto err_mr_rif_add;
+ }
mlxsw_sp_rif_counters_alloc(rif);
mlxsw_sp->router->rifs[rif_index] = rif;
return rif;
err_mr_rif_add:
+ for (i--; i >= 0; i--)
+ mlxsw_sp_mr_rif_del(vr->mr_table[i], rif);
ops->deconfigure(rif);
err_configure:
if (fid)
struct mlxsw_sp *mlxsw_sp = rif->mlxsw_sp;
struct mlxsw_sp_fid *fid = rif->fid;
struct mlxsw_sp_vr *vr;
+ int i;
mlxsw_sp_router_rif_gone_sync(mlxsw_sp, rif);
vr = &mlxsw_sp->router->vrs[rif->vr_id];
mlxsw_sp->router->rifs[rif->rif_index] = NULL;
mlxsw_sp_rif_counters_free(rif);
- mlxsw_sp_mr_rif_del(vr->mr4_table, rif);
+ for (i = 0; i < MLXSW_SP_L3_PROTO_MAX; i++)
+ mlxsw_sp_mr_rif_del(vr->mr_table[i], rif);
ops->deconfigure(rif);
if (fid)
/* Loopback RIFs are not associated with a FID. */
if (rif->mtu != dev->mtu) {
struct mlxsw_sp_vr *vr;
+ int i;
/* The RIF is relevant only to its mr_table instance, as unlike
* unicast routing, in multicast routing a RIF cannot be shared
* between several multicast routing tables.
*/
vr = &mlxsw_sp->router->vrs[rif->vr_id];
- mlxsw_sp_mr_rif_mtu_update(vr->mr4_table, rif, dev->mtu);
+ for (i = 0; i < MLXSW_SP_L3_PROTO_MAX; i++)
+ mlxsw_sp_mr_rif_mtu_update(vr->mr_table[i],
+ rif, dev->mtu);
}
ether_addr_copy(rif->addr, dev->dev_addr);
enum mlxsw_sp_l3proto {
MLXSW_SP_L3_PROTO_IPV4,
MLXSW_SP_L3_PROTO_IPV6,
+#define MLXSW_SP_L3_PROTO_MAX (MLXSW_SP_L3_PROTO_IPV6 + 1)
};
union mlxsw_sp_l3addr {
}
static struct mlxsw_sp_span_inspected_port *
-mlxsw_sp_span_entry_bound_port_find(struct mlxsw_sp_port *port,
- struct mlxsw_sp_span_entry *span_entry)
+mlxsw_sp_span_entry_bound_port_find(struct mlxsw_sp_span_entry *span_entry,
+ enum mlxsw_sp_span_type type,
+ struct mlxsw_sp_port *port,
+ bool bind)
{
struct mlxsw_sp_span_inspected_port *p;
list_for_each_entry(p, &span_entry->bound_ports_list, list)
- if (port->local_port == p->local_port)
+ if (type == p->type &&
+ port->local_port == p->local_port &&
+ bind == p->bound)
return p;
return NULL;
}
struct mlxsw_sp_span_inspected_port *inspected_port;
struct mlxsw_sp *mlxsw_sp = port->mlxsw_sp;
char sbib_pl[MLXSW_REG_SBIB_LEN];
+ int i;
int err;
+ /* A given (source port, direction) can only be bound to one analyzer,
+ * so if a binding is requested, check for conflicts.
+ */
+ if (bind)
+ for (i = 0; i < mlxsw_sp->span.entries_count; i++) {
+ struct mlxsw_sp_span_entry *curr =
+ &mlxsw_sp->span.entries[i];
+
+ if (mlxsw_sp_span_entry_bound_port_find(curr, type,
+ port, bind))
+ return -EEXIST;
+ }
+
/* if it is an egress SPAN, bind a shared buffer to it */
if (type == MLXSW_SP_SPAN_EGRESS) {
u32 buffsize = mlxsw_sp_span_mtu_to_buffsize(mlxsw_sp,
}
inspected_port->local_port = port->local_port;
inspected_port->type = type;
+ inspected_port->bound = bind;
list_add_tail(&inspected_port->list, &span_entry->bound_ports_list);
return 0;
struct mlxsw_sp *mlxsw_sp = port->mlxsw_sp;
char sbib_pl[MLXSW_REG_SBIB_LEN];
- inspected_port = mlxsw_sp_span_entry_bound_port_find(port, span_entry);
+ inspected_port = mlxsw_sp_span_entry_bound_port_find(span_entry, type,
+ port, bind);
if (!inspected_port)
return;
{
struct mlxsw_sp *mlxsw_sp = from->mlxsw_sp;
const struct mlxsw_sp_span_entry_ops *ops;
- struct mlxsw_sp_span_parms sparms = {0};
+ struct mlxsw_sp_span_parms sparms = {NULL};
struct mlxsw_sp_span_entry *span_entry;
int err;
ASSERT_RTNL();
for (i = 0; i < mlxsw_sp->span.entries_count; i++) {
struct mlxsw_sp_span_entry *curr = &mlxsw_sp->span.entries[i];
- struct mlxsw_sp_span_parms sparms = {0};
+ struct mlxsw_sp_span_parms sparms = {NULL};
if (!curr->ref_count)
continue;
struct list_head list;
enum mlxsw_sp_span_type type;
u8 local_port;
+
+ /* Whether this is a directly bound mirror (port-to-port) or an ACL. */
+ bool bound;
};
struct mlxsw_sp_span_parms {
u32 supported, advertising, lp_advertising;
int err;
- mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sx_port->local_port, 0);
+ mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sx_port->local_port, 0, false);
err = mlxsw_reg_query(mlxsw_sx->core, MLXSW_REG(ptys), ptys_pl);
if (err) {
netdev_err(dev, "Failed to get proto");
mlxsw_sx_to_ptys_advert_link(advertising) :
mlxsw_sx_to_ptys_speed(speed);
- mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sx_port->local_port, 0);
+ mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sx_port->local_port, 0, false);
err = mlxsw_reg_query(mlxsw_sx->core, MLXSW_REG(ptys), ptys_pl);
if (err) {
netdev_err(dev, "Failed to get proto");
return 0;
mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sx_port->local_port,
- eth_proto_new);
+ eth_proto_new, true);
err = mlxsw_reg_write(mlxsw_sx->core, MLXSW_REG(ptys), ptys_pl);
if (err) {
netdev_err(dev, "Failed to set proto admin");
eth_proto_admin = mlxsw_sx_to_ptys_upper_speed(upper_speed);
mlxsw_reg_ptys_eth_pack(ptys_pl, mlxsw_sx_port->local_port,
- eth_proto_admin);
+ eth_proto_admin, true);
return mlxsw_reg_write(mlxsw_sx->core, MLXSW_REG(ptys), ptys_pl);
}
MLXSW_TRAP_ID_IPV6_DHCP = 0x69,
MLXSW_TRAP_ID_IPV6_ALL_ROUTERS_LINK = 0x6F,
MLXSW_TRAP_ID_RTR_INGRESS0 = 0x70,
+ MLXSW_TRAP_ID_IPV6_PIM = 0x79,
MLXSW_TRAP_ID_IPV4_BGP = 0x88,
MLXSW_TRAP_ID_IPV6_BGP = 0x89,
MLXSW_TRAP_ID_L3_IPV6_ROUTER_SOLICITATION = 0x8A,
/* Careful: must be accessed under kernel_param_lock() */
static char *myri10ge_fw_name = NULL;
-module_param(myri10ge_fw_name, charp, S_IRUGO | S_IWUSR);
+module_param(myri10ge_fw_name, charp, 0644);
MODULE_PARM_DESC(myri10ge_fw_name, "Firmware image name");
#define MYRI10GE_MAX_BOARDS 8
MODULE_PARM_DESC(myri10ge_fw_names, "Firmware image names per board");
static int myri10ge_ecrc_enable = 1;
-module_param(myri10ge_ecrc_enable, int, S_IRUGO);
+module_param(myri10ge_ecrc_enable, int, 0444);
MODULE_PARM_DESC(myri10ge_ecrc_enable, "Enable Extended CRC on PCI-E");
static int myri10ge_small_bytes = -1; /* -1 == auto */
-module_param(myri10ge_small_bytes, int, S_IRUGO | S_IWUSR);
+module_param(myri10ge_small_bytes, int, 0644);
MODULE_PARM_DESC(myri10ge_small_bytes, "Threshold of small packets");
static int myri10ge_msi = 1; /* enable msi by default */
-module_param(myri10ge_msi, int, S_IRUGO | S_IWUSR);
+module_param(myri10ge_msi, int, 0644);
MODULE_PARM_DESC(myri10ge_msi, "Enable Message Signalled Interrupts");
static int myri10ge_intr_coal_delay = 75;
-module_param(myri10ge_intr_coal_delay, int, S_IRUGO);
+module_param(myri10ge_intr_coal_delay, int, 0444);
MODULE_PARM_DESC(myri10ge_intr_coal_delay, "Interrupt coalescing delay");
static int myri10ge_flow_control = 1;
-module_param(myri10ge_flow_control, int, S_IRUGO);
+module_param(myri10ge_flow_control, int, 0444);
MODULE_PARM_DESC(myri10ge_flow_control, "Pause parameter");
static int myri10ge_deassert_wait = 1;
-module_param(myri10ge_deassert_wait, int, S_IRUGO | S_IWUSR);
+module_param(myri10ge_deassert_wait, int, 0644);
MODULE_PARM_DESC(myri10ge_deassert_wait,
"Wait when deasserting legacy interrupts");
static int myri10ge_force_firmware = 0;
-module_param(myri10ge_force_firmware, int, S_IRUGO);
+module_param(myri10ge_force_firmware, int, 0444);
MODULE_PARM_DESC(myri10ge_force_firmware,
"Force firmware to assume aligned completions");
static int myri10ge_initial_mtu = MYRI10GE_MAX_ETHER_MTU - ETH_HLEN;
-module_param(myri10ge_initial_mtu, int, S_IRUGO);
+module_param(myri10ge_initial_mtu, int, 0444);
MODULE_PARM_DESC(myri10ge_initial_mtu, "Initial MTU");
static int myri10ge_napi_weight = 64;
-module_param(myri10ge_napi_weight, int, S_IRUGO);
+module_param(myri10ge_napi_weight, int, 0444);
MODULE_PARM_DESC(myri10ge_napi_weight, "Set NAPI weight");
static int myri10ge_watchdog_timeout = 1;
-module_param(myri10ge_watchdog_timeout, int, S_IRUGO);
+module_param(myri10ge_watchdog_timeout, int, 0444);
MODULE_PARM_DESC(myri10ge_watchdog_timeout, "Set watchdog timeout");
static int myri10ge_max_irq_loops = 1048576;
-module_param(myri10ge_max_irq_loops, int, S_IRUGO);
+module_param(myri10ge_max_irq_loops, int, 0444);
MODULE_PARM_DESC(myri10ge_max_irq_loops,
"Set stuck legacy IRQ detection threshold");
MODULE_PARM_DESC(myri10ge_debug, "Debug level (0=none,...,16=all)");
static int myri10ge_fill_thresh = 256;
-module_param(myri10ge_fill_thresh, int, S_IRUGO | S_IWUSR);
+module_param(myri10ge_fill_thresh, int, 0644);
MODULE_PARM_DESC(myri10ge_fill_thresh, "Number of empty rx slots allowed");
static int myri10ge_reset_recover = 1;
static int myri10ge_max_slices = 1;
-module_param(myri10ge_max_slices, int, S_IRUGO);
+module_param(myri10ge_max_slices, int, 0444);
MODULE_PARM_DESC(myri10ge_max_slices, "Max tx/rx queues");
static int myri10ge_rss_hash = MXGEFW_RSS_HASH_TYPE_SRC_DST_PORT;
-module_param(myri10ge_rss_hash, int, S_IRUGO);
+module_param(myri10ge_rss_hash, int, 0444);
MODULE_PARM_DESC(myri10ge_rss_hash, "Type of RSS hashing to do");
static int myri10ge_dca = 1;
-module_param(myri10ge_dca, int, S_IRUGO);
+module_param(myri10ge_dca, int, 0444);
MODULE_PARM_DESC(myri10ge_dca, "Enable DCA if possible");
#define MYRI10GE_FW_OFFSET 1024*1024
#
-# National Semi-conductor device configuration
+# National Semiconductor device configuration
#
config NET_VENDOR_NATSEMI
- bool "National Semi-conductor devices"
+ bool "National Semiconductor devices"
default y
---help---
If you have a network (Ethernet) card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
kernel: saying N will just cause the configurator to skip all
- the questions about National Semi-conductor devices. If you say Y,
+ the questions about National Semiconductor devices. If you say Y,
you will be asked for your specific card in the following questions.
if NET_VENDOR_NATSEMI
# SPDX-License-Identifier: GPL-2.0
#
-# Makefile for the National Semi-conductor Sonic devices.
+# Makefile for the National Semiconductor Sonic devices.
#
obj-$(CONFIG_MACSONIC) += macsonic.o
#define NFP_FLOWER_MASK_MPLS_BOS BIT(8)
#define NFP_FLOWER_MASK_MPLS_Q BIT(0)
+#define NFP_FL_IP_FRAG_FIRST BIT(7)
+#define NFP_FL_IP_FRAGMENTED BIT(6)
+
/* Compressed HW representation of TCP Flags */
#define NFP_FL_TCP_FLAG_URG BIT(4)
#define NFP_FL_TCP_FLAG_PSH BIT(3)
__be16 port_dst;
};
+struct nfp_flower_ip_ext {
+ u8 tos;
+ u8 proto;
+ u8 ttl;
+ u8 flags;
+};
+
/* L3 IPv4 details (3W/12B)
* 3 2 1
* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
* +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
*/
struct nfp_flower_ipv4 {
- u8 tos;
- u8 proto;
- u8 ttl;
- u8 flags;
+ struct nfp_flower_ip_ext ip_ext;
__be32 ipv4_src;
__be32 ipv4_dst;
};
* 3 2 1
* 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
* +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
- * | DSCP |ECN| protocol | reserved |
+ * | DSCP |ECN| protocol | ttl | flags |
* +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
* | ipv6_exthdr | res | ipv6_flow_label |
* +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
* +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
*/
struct nfp_flower_ipv6 {
- u8 tos;
- u8 proto;
- u8 ttl;
- u8 reserved;
+ struct nfp_flower_ip_ext ip_ext;
__be32 ipv6_flow_label_exthdr;
struct in6_addr ipv6_src;
struct in6_addr ipv6_dst;
}
static void
-nfp_flower_compile_ipv4(struct nfp_flower_ipv4 *frame,
- struct tc_cls_flower_offload *flow,
- bool mask_version)
+nfp_flower_compile_ip_ext(struct nfp_flower_ip_ext *frame,
+ struct tc_cls_flower_offload *flow,
+ bool mask_version)
{
struct fl_flow_key *target = mask_version ? flow->mask : flow->key;
- struct flow_dissector_key_ipv4_addrs *addr;
- struct flow_dissector_key_basic *basic;
-
- memset(frame, 0, sizeof(struct nfp_flower_ipv4));
-
- if (dissector_uses_key(flow->dissector,
- FLOW_DISSECTOR_KEY_IPV4_ADDRS)) {
- addr = skb_flow_dissector_target(flow->dissector,
- FLOW_DISSECTOR_KEY_IPV4_ADDRS,
- target);
- frame->ipv4_src = addr->src;
- frame->ipv4_dst = addr->dst;
- }
if (dissector_uses_key(flow->dissector, FLOW_DISSECTOR_KEY_BASIC)) {
+ struct flow_dissector_key_basic *basic;
+
basic = skb_flow_dissector_target(flow->dissector,
FLOW_DISSECTOR_KEY_BASIC,
target);
if (tcp_flags & TCPHDR_URG)
frame->flags |= NFP_FL_TCP_FLAG_URG;
}
+
+ if (dissector_uses_key(flow->dissector, FLOW_DISSECTOR_KEY_CONTROL)) {
+ struct flow_dissector_key_control *key;
+
+ key = skb_flow_dissector_target(flow->dissector,
+ FLOW_DISSECTOR_KEY_CONTROL,
+ target);
+ if (key->flags & FLOW_DIS_IS_FRAGMENT)
+ frame->flags |= NFP_FL_IP_FRAGMENTED;
+ if (key->flags & FLOW_DIS_FIRST_FRAG)
+ frame->flags |= NFP_FL_IP_FRAG_FIRST;
+ }
+}
+
+static void
+nfp_flower_compile_ipv4(struct nfp_flower_ipv4 *frame,
+ struct tc_cls_flower_offload *flow,
+ bool mask_version)
+{
+ struct fl_flow_key *target = mask_version ? flow->mask : flow->key;
+ struct flow_dissector_key_ipv4_addrs *addr;
+
+ memset(frame, 0, sizeof(struct nfp_flower_ipv4));
+
+ if (dissector_uses_key(flow->dissector,
+ FLOW_DISSECTOR_KEY_IPV4_ADDRS)) {
+ addr = skb_flow_dissector_target(flow->dissector,
+ FLOW_DISSECTOR_KEY_IPV4_ADDRS,
+ target);
+ frame->ipv4_src = addr->src;
+ frame->ipv4_dst = addr->dst;
+ }
+
+ nfp_flower_compile_ip_ext(&frame->ip_ext, flow, mask_version);
}
static void
{
struct fl_flow_key *target = mask_version ? flow->mask : flow->key;
struct flow_dissector_key_ipv6_addrs *addr;
- struct flow_dissector_key_basic *basic;
memset(frame, 0, sizeof(struct nfp_flower_ipv6));
frame->ipv6_dst = addr->dst;
}
- if (dissector_uses_key(flow->dissector, FLOW_DISSECTOR_KEY_BASIC)) {
- basic = skb_flow_dissector_target(flow->dissector,
- FLOW_DISSECTOR_KEY_BASIC,
- target);
- frame->proto = basic->ip_proto;
- }
-
- if (dissector_uses_key(flow->dissector, FLOW_DISSECTOR_KEY_IP)) {
- struct flow_dissector_key_ip *flow_ip;
-
- flow_ip = skb_flow_dissector_target(flow->dissector,
- FLOW_DISSECTOR_KEY_IP,
- target);
- frame->tos = flow_ip->tos;
- frame->ttl = flow_ip->ttl;
- }
+ nfp_flower_compile_ip_ext(&frame->ip_ext, flow, mask_version);
}
static void
(TCPHDR_FIN | TCPHDR_SYN | TCPHDR_RST | \
TCPHDR_PSH | TCPHDR_URG)
+#define NFP_FLOWER_SUPPORTED_CTLFLAGS \
+ (FLOW_DIS_IS_FRAGMENT | \
+ FLOW_DIS_FIRST_FRAG)
+
#define NFP_FLOWER_WHITELIST_DISSECTOR \
(BIT(FLOW_DISSECTOR_KEY_CONTROL) | \
BIT(FLOW_DISSECTOR_KEY_BASIC) | \
}
}
+ if (dissector_uses_key(flow->dissector, FLOW_DISSECTOR_KEY_CONTROL)) {
+ struct flow_dissector_key_control *key_ctl;
+
+ key_ctl = skb_flow_dissector_target(flow->dissector,
+ FLOW_DISSECTOR_KEY_CONTROL,
+ flow->key);
+
+ if (key_ctl->flags & ~NFP_FLOWER_SUPPORTED_CTLFLAGS)
+ return -EOPNOTSUPP;
+ }
+
ret_key_ls->key_layer = key_layer;
ret_key_ls->key_layer_two = key_layer_two;
ret_key_ls->key_size = key_size;
for (i = 0; i < min(nn->max_rx_rings, nn->max_r_vecs); i++) {
sprintf(name, "%d", i);
- debugfs_create_file(name, S_IRUSR, rx,
+ debugfs_create_file(name, 0400, rx,
&nn->r_vecs[i], &nfp_rx_q_fops);
- debugfs_create_file(name, S_IRUSR, xdp,
+ debugfs_create_file(name, 0400, xdp,
&nn->r_vecs[i], &nfp_xdp_q_fops);
}
for (i = 0; i < min(nn->max_tx_rings, nn->max_r_vecs); i++) {
sprintf(name, "%d", i);
- debugfs_create_file(name, S_IRUSR, tx,
+ debugfs_create_file(name, 0400, tx,
&nn->r_vecs[i], &nfp_tx_q_fops);
}
}
--- /dev/null
+#
+# National Instuments network device configuration
+#
+
+config NET_VENDOR_NI
+ bool "National Instruments Devices"
+ default y
+ help
+ If you have a network (Ethernet) device belonging to this class, say Y.
+
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about National Instrument devices.
+ If you say Y, you will be asked for your specific device in the
+ following questions.
+
+if NET_VENDOR_NI
+
+config NI_XGE_MANAGEMENT_ENET
+ tristate "National Instruments XGE management enet support"
+ depends on ARCH_ZYNQ
+ select PHYLIB
+ help
+ Simple LAN device for debug or management purposes. Can
+ support either 10G or 1G PHYs via SFP+ ports.
+
+endif
--- /dev/null
+obj-$(CONFIG_NI_XGE_MANAGEMENT_ENET) += nixge.o
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2016-2017, National Instruments Corp.
+ *
+ * Author: Moritz Fischer <mdf@kernel.org>
+ */
+
+#include <linux/etherdevice.h>
+#include <linux/module.h>
+#include <linux/netdevice.h>
+#include <linux/of_address.h>
+#include <linux/of_mdio.h>
+#include <linux/of_net.h>
+#include <linux/of_platform.h>
+#include <linux/of_irq.h>
+#include <linux/skbuff.h>
+#include <linux/phy.h>
+#include <linux/mii.h>
+#include <linux/nvmem-consumer.h>
+#include <linux/ethtool.h>
+#include <linux/iopoll.h>
+
+#define TX_BD_NUM 64
+#define RX_BD_NUM 128
+
+/* Axi DMA Register definitions */
+#define XAXIDMA_TX_CR_OFFSET 0x00 /* Channel control */
+#define XAXIDMA_TX_SR_OFFSET 0x04 /* Status */
+#define XAXIDMA_TX_CDESC_OFFSET 0x08 /* Current descriptor pointer */
+#define XAXIDMA_TX_TDESC_OFFSET 0x10 /* Tail descriptor pointer */
+
+#define XAXIDMA_RX_CR_OFFSET 0x30 /* Channel control */
+#define XAXIDMA_RX_SR_OFFSET 0x34 /* Status */
+#define XAXIDMA_RX_CDESC_OFFSET 0x38 /* Current descriptor pointer */
+#define XAXIDMA_RX_TDESC_OFFSET 0x40 /* Tail descriptor pointer */
+
+#define XAXIDMA_CR_RUNSTOP_MASK 0x1 /* Start/stop DMA channel */
+#define XAXIDMA_CR_RESET_MASK 0x4 /* Reset DMA engine */
+
+#define XAXIDMA_BD_CTRL_LENGTH_MASK 0x007FFFFF /* Requested len */
+#define XAXIDMA_BD_CTRL_TXSOF_MASK 0x08000000 /* First tx packet */
+#define XAXIDMA_BD_CTRL_TXEOF_MASK 0x04000000 /* Last tx packet */
+#define XAXIDMA_BD_CTRL_ALL_MASK 0x0C000000 /* All control bits */
+
+#define XAXIDMA_DELAY_MASK 0xFF000000 /* Delay timeout counter */
+#define XAXIDMA_COALESCE_MASK 0x00FF0000 /* Coalesce counter */
+
+#define XAXIDMA_DELAY_SHIFT 24
+#define XAXIDMA_COALESCE_SHIFT 16
+
+#define XAXIDMA_IRQ_IOC_MASK 0x00001000 /* Completion intr */
+#define XAXIDMA_IRQ_DELAY_MASK 0x00002000 /* Delay interrupt */
+#define XAXIDMA_IRQ_ERROR_MASK 0x00004000 /* Error interrupt */
+#define XAXIDMA_IRQ_ALL_MASK 0x00007000 /* All interrupts */
+
+/* Default TX/RX Threshold and waitbound values for SGDMA mode */
+#define XAXIDMA_DFT_TX_THRESHOLD 24
+#define XAXIDMA_DFT_TX_WAITBOUND 254
+#define XAXIDMA_DFT_RX_THRESHOLD 24
+#define XAXIDMA_DFT_RX_WAITBOUND 254
+
+#define XAXIDMA_BD_STS_ACTUAL_LEN_MASK 0x007FFFFF /* Actual len */
+#define XAXIDMA_BD_STS_COMPLETE_MASK 0x80000000 /* Completed */
+#define XAXIDMA_BD_STS_DEC_ERR_MASK 0x40000000 /* Decode error */
+#define XAXIDMA_BD_STS_SLV_ERR_MASK 0x20000000 /* Slave error */
+#define XAXIDMA_BD_STS_INT_ERR_MASK 0x10000000 /* Internal err */
+#define XAXIDMA_BD_STS_ALL_ERR_MASK 0x70000000 /* All errors */
+#define XAXIDMA_BD_STS_RXSOF_MASK 0x08000000 /* First rx pkt */
+#define XAXIDMA_BD_STS_RXEOF_MASK 0x04000000 /* Last rx pkt */
+#define XAXIDMA_BD_STS_ALL_MASK 0xFC000000 /* All status bits */
+
+#define NIXGE_REG_CTRL_OFFSET 0x4000
+#define NIXGE_REG_INFO 0x00
+#define NIXGE_REG_MAC_CTL 0x04
+#define NIXGE_REG_PHY_CTL 0x08
+#define NIXGE_REG_LED_CTL 0x0c
+#define NIXGE_REG_MDIO_DATA 0x10
+#define NIXGE_REG_MDIO_ADDR 0x14
+#define NIXGE_REG_MDIO_OP 0x18
+#define NIXGE_REG_MDIO_CTRL 0x1c
+
+#define NIXGE_ID_LED_CTL_EN BIT(0)
+#define NIXGE_ID_LED_CTL_VAL BIT(1)
+
+#define NIXGE_MDIO_CLAUSE45 BIT(12)
+#define NIXGE_MDIO_CLAUSE22 0
+#define NIXGE_MDIO_OP(n) (((n) & 0x3) << 10)
+#define NIXGE_MDIO_OP_ADDRESS 0
+#define NIXGE_MDIO_C45_WRITE BIT(0)
+#define NIXGE_MDIO_C45_READ (BIT(1) | BIT(0))
+#define NIXGE_MDIO_C22_WRITE BIT(0)
+#define NIXGE_MDIO_C22_READ BIT(1)
+#define NIXGE_MDIO_ADDR(n) (((n) & 0x1f) << 5)
+#define NIXGE_MDIO_MMD(n) (((n) & 0x1f) << 0)
+
+#define NIXGE_REG_MAC_LSB 0x1000
+#define NIXGE_REG_MAC_MSB 0x1004
+
+/* Packet size info */
+#define NIXGE_HDR_SIZE 14 /* Size of Ethernet header */
+#define NIXGE_TRL_SIZE 4 /* Size of Ethernet trailer (FCS) */
+#define NIXGE_MTU 1500 /* Max MTU of an Ethernet frame */
+#define NIXGE_JUMBO_MTU 9000 /* Max MTU of a jumbo Eth. frame */
+
+#define NIXGE_MAX_FRAME_SIZE (NIXGE_MTU + NIXGE_HDR_SIZE + NIXGE_TRL_SIZE)
+#define NIXGE_MAX_JUMBO_FRAME_SIZE \
+ (NIXGE_JUMBO_MTU + NIXGE_HDR_SIZE + NIXGE_TRL_SIZE)
+
+struct nixge_hw_dma_bd {
+ u32 next;
+ u32 reserved1;
+ u32 phys;
+ u32 reserved2;
+ u32 reserved3;
+ u32 reserved4;
+ u32 cntrl;
+ u32 status;
+ u32 app0;
+ u32 app1;
+ u32 app2;
+ u32 app3;
+ u32 app4;
+ u32 sw_id_offset;
+ u32 reserved5;
+ u32 reserved6;
+};
+
+struct nixge_tx_skb {
+ struct sk_buff *skb;
+ dma_addr_t mapping;
+ size_t size;
+ bool mapped_as_page;
+};
+
+struct nixge_priv {
+ struct net_device *ndev;
+ struct napi_struct napi;
+ struct device *dev;
+
+ /* Connection to PHY device */
+ struct device_node *phy_node;
+ phy_interface_t phy_mode;
+
+ int link;
+ unsigned int speed;
+ unsigned int duplex;
+
+ /* MDIO bus data */
+ struct mii_bus *mii_bus; /* MII bus reference */
+
+ /* IO registers, dma functions and IRQs */
+ void __iomem *ctrl_regs;
+ void __iomem *dma_regs;
+
+ struct tasklet_struct dma_err_tasklet;
+
+ int tx_irq;
+ int rx_irq;
+ u32 last_link;
+
+ /* Buffer descriptors */
+ struct nixge_hw_dma_bd *tx_bd_v;
+ struct nixge_tx_skb *tx_skb;
+ dma_addr_t tx_bd_p;
+
+ struct nixge_hw_dma_bd *rx_bd_v;
+ dma_addr_t rx_bd_p;
+ u32 tx_bd_ci;
+ u32 tx_bd_tail;
+ u32 rx_bd_ci;
+
+ u32 coalesce_count_rx;
+ u32 coalesce_count_tx;
+};
+
+static void nixge_dma_write_reg(struct nixge_priv *priv, off_t offset, u32 val)
+{
+ writel(val, priv->dma_regs + offset);
+}
+
+static u32 nixge_dma_read_reg(const struct nixge_priv *priv, off_t offset)
+{
+ return readl(priv->dma_regs + offset);
+}
+
+static void nixge_ctrl_write_reg(struct nixge_priv *priv, off_t offset, u32 val)
+{
+ writel(val, priv->ctrl_regs + offset);
+}
+
+static u32 nixge_ctrl_read_reg(struct nixge_priv *priv, off_t offset)
+{
+ return readl(priv->ctrl_regs + offset);
+}
+
+#define nixge_ctrl_poll_timeout(priv, addr, val, cond, sleep_us, timeout_us) \
+ readl_poll_timeout((priv)->ctrl_regs + (addr), (val), (cond), \
+ (sleep_us), (timeout_us))
+
+#define nixge_dma_poll_timeout(priv, addr, val, cond, sleep_us, timeout_us) \
+ readl_poll_timeout((priv)->dma_regs + (addr), (val), (cond), \
+ (sleep_us), (timeout_us))
+
+static void nixge_hw_dma_bd_release(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ int i;
+
+ for (i = 0; i < RX_BD_NUM; i++) {
+ dma_unmap_single(ndev->dev.parent, priv->rx_bd_v[i].phys,
+ NIXGE_MAX_JUMBO_FRAME_SIZE, DMA_FROM_DEVICE);
+ dev_kfree_skb((struct sk_buff *)
+ (priv->rx_bd_v[i].sw_id_offset));
+ }
+
+ if (priv->rx_bd_v)
+ dma_free_coherent(ndev->dev.parent,
+ sizeof(*priv->rx_bd_v) * RX_BD_NUM,
+ priv->rx_bd_v,
+ priv->rx_bd_p);
+
+ if (priv->tx_skb)
+ devm_kfree(ndev->dev.parent, priv->tx_skb);
+
+ if (priv->tx_bd_v)
+ dma_free_coherent(ndev->dev.parent,
+ sizeof(*priv->tx_bd_v) * TX_BD_NUM,
+ priv->tx_bd_v,
+ priv->tx_bd_p);
+}
+
+static int nixge_hw_dma_bd_init(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ struct sk_buff *skb;
+ u32 cr;
+ int i;
+
+ /* Reset the indexes which are used for accessing the BDs */
+ priv->tx_bd_ci = 0;
+ priv->tx_bd_tail = 0;
+ priv->rx_bd_ci = 0;
+
+ /* Allocate the Tx and Rx buffer descriptors. */
+ priv->tx_bd_v = dma_zalloc_coherent(ndev->dev.parent,
+ sizeof(*priv->tx_bd_v) * TX_BD_NUM,
+ &priv->tx_bd_p, GFP_KERNEL);
+ if (!priv->tx_bd_v)
+ goto out;
+
+ priv->tx_skb = devm_kzalloc(ndev->dev.parent,
+ sizeof(*priv->tx_skb) *
+ TX_BD_NUM,
+ GFP_KERNEL);
+ if (!priv->tx_skb)
+ goto out;
+
+ priv->rx_bd_v = dma_zalloc_coherent(ndev->dev.parent,
+ sizeof(*priv->rx_bd_v) * RX_BD_NUM,
+ &priv->rx_bd_p, GFP_KERNEL);
+ if (!priv->rx_bd_v)
+ goto out;
+
+ for (i = 0; i < TX_BD_NUM; i++) {
+ priv->tx_bd_v[i].next = priv->tx_bd_p +
+ sizeof(*priv->tx_bd_v) *
+ ((i + 1) % TX_BD_NUM);
+ }
+
+ for (i = 0; i < RX_BD_NUM; i++) {
+ priv->rx_bd_v[i].next = priv->rx_bd_p +
+ sizeof(*priv->rx_bd_v) *
+ ((i + 1) % RX_BD_NUM);
+
+ skb = netdev_alloc_skb_ip_align(ndev,
+ NIXGE_MAX_JUMBO_FRAME_SIZE);
+ if (!skb)
+ goto out;
+
+ priv->rx_bd_v[i].sw_id_offset = (u32)skb;
+ priv->rx_bd_v[i].phys =
+ dma_map_single(ndev->dev.parent,
+ skb->data,
+ NIXGE_MAX_JUMBO_FRAME_SIZE,
+ DMA_FROM_DEVICE);
+ priv->rx_bd_v[i].cntrl = NIXGE_MAX_JUMBO_FRAME_SIZE;
+ }
+
+ /* Start updating the Rx channel control register */
+ cr = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ /* Update the interrupt coalesce count */
+ cr = ((cr & ~XAXIDMA_COALESCE_MASK) |
+ ((priv->coalesce_count_rx) << XAXIDMA_COALESCE_SHIFT));
+ /* Update the delay timer count */
+ cr = ((cr & ~XAXIDMA_DELAY_MASK) |
+ (XAXIDMA_DFT_RX_WAITBOUND << XAXIDMA_DELAY_SHIFT));
+ /* Enable coalesce, delay timer and error interrupts */
+ cr |= XAXIDMA_IRQ_ALL_MASK;
+ /* Write to the Rx channel control register */
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CR_OFFSET, cr);
+
+ /* Start updating the Tx channel control register */
+ cr = nixge_dma_read_reg(priv, XAXIDMA_TX_CR_OFFSET);
+ /* Update the interrupt coalesce count */
+ cr = (((cr & ~XAXIDMA_COALESCE_MASK)) |
+ ((priv->coalesce_count_tx) << XAXIDMA_COALESCE_SHIFT));
+ /* Update the delay timer count */
+ cr = (((cr & ~XAXIDMA_DELAY_MASK)) |
+ (XAXIDMA_DFT_TX_WAITBOUND << XAXIDMA_DELAY_SHIFT));
+ /* Enable coalesce, delay timer and error interrupts */
+ cr |= XAXIDMA_IRQ_ALL_MASK;
+ /* Write to the Tx channel control register */
+ nixge_dma_write_reg(priv, XAXIDMA_TX_CR_OFFSET, cr);
+
+ /* Populate the tail pointer and bring the Rx Axi DMA engine out of
+ * halted state. This will make the Rx side ready for reception.
+ */
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CDESC_OFFSET, priv->rx_bd_p);
+ cr = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CR_OFFSET,
+ cr | XAXIDMA_CR_RUNSTOP_MASK);
+ nixge_dma_write_reg(priv, XAXIDMA_RX_TDESC_OFFSET, priv->rx_bd_p +
+ (sizeof(*priv->rx_bd_v) * (RX_BD_NUM - 1)));
+
+ /* Write to the RS (Run-stop) bit in the Tx channel control register.
+ * Tx channel is now ready to run. But only after we write to the
+ * tail pointer register that the Tx channel will start transmitting.
+ */
+ nixge_dma_write_reg(priv, XAXIDMA_TX_CDESC_OFFSET, priv->tx_bd_p);
+ cr = nixge_dma_read_reg(priv, XAXIDMA_TX_CR_OFFSET);
+ nixge_dma_write_reg(priv, XAXIDMA_TX_CR_OFFSET,
+ cr | XAXIDMA_CR_RUNSTOP_MASK);
+
+ return 0;
+out:
+ nixge_hw_dma_bd_release(ndev);
+ return -ENOMEM;
+}
+
+static void __nixge_device_reset(struct nixge_priv *priv, off_t offset)
+{
+ u32 status;
+ int err;
+
+ /* Reset Axi DMA. This would reset NIXGE Ethernet core as well.
+ * The reset process of Axi DMA takes a while to complete as all
+ * pending commands/transfers will be flushed or completed during
+ * this reset process.
+ */
+ nixge_dma_write_reg(priv, offset, XAXIDMA_CR_RESET_MASK);
+ err = nixge_dma_poll_timeout(priv, offset, status,
+ !(status & XAXIDMA_CR_RESET_MASK), 10,
+ 1000);
+ if (err)
+ netdev_err(priv->ndev, "%s: DMA reset timeout!\n", __func__);
+}
+
+static void nixge_device_reset(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+
+ __nixge_device_reset(priv, XAXIDMA_TX_CR_OFFSET);
+ __nixge_device_reset(priv, XAXIDMA_RX_CR_OFFSET);
+
+ if (nixge_hw_dma_bd_init(ndev))
+ netdev_err(ndev, "%s: descriptor allocation failed\n",
+ __func__);
+
+ netif_trans_update(ndev);
+}
+
+static void nixge_handle_link_change(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ struct phy_device *phydev = ndev->phydev;
+
+ if (phydev->link != priv->link || phydev->speed != priv->speed ||
+ phydev->duplex != priv->duplex) {
+ priv->link = phydev->link;
+ priv->speed = phydev->speed;
+ priv->duplex = phydev->duplex;
+ phy_print_status(phydev);
+ }
+}
+
+static void nixge_tx_skb_unmap(struct nixge_priv *priv,
+ struct nixge_tx_skb *tx_skb)
+{
+ if (tx_skb->mapping) {
+ if (tx_skb->mapped_as_page)
+ dma_unmap_page(priv->ndev->dev.parent, tx_skb->mapping,
+ tx_skb->size, DMA_TO_DEVICE);
+ else
+ dma_unmap_single(priv->ndev->dev.parent,
+ tx_skb->mapping,
+ tx_skb->size, DMA_TO_DEVICE);
+ tx_skb->mapping = 0;
+ }
+
+ if (tx_skb->skb) {
+ dev_kfree_skb_any(tx_skb->skb);
+ tx_skb->skb = NULL;
+ }
+}
+
+static void nixge_start_xmit_done(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ struct nixge_hw_dma_bd *cur_p;
+ struct nixge_tx_skb *tx_skb;
+ unsigned int status = 0;
+ u32 packets = 0;
+ u32 size = 0;
+
+ cur_p = &priv->tx_bd_v[priv->tx_bd_ci];
+ tx_skb = &priv->tx_skb[priv->tx_bd_ci];
+
+ status = cur_p->status;
+
+ while (status & XAXIDMA_BD_STS_COMPLETE_MASK) {
+ nixge_tx_skb_unmap(priv, tx_skb);
+ cur_p->status = 0;
+
+ size += status & XAXIDMA_BD_STS_ACTUAL_LEN_MASK;
+ packets++;
+
+ ++priv->tx_bd_ci;
+ priv->tx_bd_ci %= TX_BD_NUM;
+ cur_p = &priv->tx_bd_v[priv->tx_bd_ci];
+ tx_skb = &priv->tx_skb[priv->tx_bd_ci];
+ status = cur_p->status;
+ }
+
+ ndev->stats.tx_packets += packets;
+ ndev->stats.tx_bytes += size;
+
+ if (packets)
+ netif_wake_queue(ndev);
+}
+
+static int nixge_check_tx_bd_space(struct nixge_priv *priv,
+ int num_frag)
+{
+ struct nixge_hw_dma_bd *cur_p;
+
+ cur_p = &priv->tx_bd_v[(priv->tx_bd_tail + num_frag) % TX_BD_NUM];
+ if (cur_p->status & XAXIDMA_BD_STS_ALL_MASK)
+ return NETDEV_TX_BUSY;
+ return 0;
+}
+
+static int nixge_start_xmit(struct sk_buff *skb, struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ struct nixge_hw_dma_bd *cur_p;
+ struct nixge_tx_skb *tx_skb;
+ dma_addr_t tail_p;
+ skb_frag_t *frag;
+ u32 num_frag;
+ u32 ii;
+
+ num_frag = skb_shinfo(skb)->nr_frags;
+ cur_p = &priv->tx_bd_v[priv->tx_bd_tail];
+ tx_skb = &priv->tx_skb[priv->tx_bd_tail];
+
+ if (nixge_check_tx_bd_space(priv, num_frag)) {
+ if (!netif_queue_stopped(ndev))
+ netif_stop_queue(ndev);
+ return NETDEV_TX_OK;
+ }
+
+ cur_p->phys = dma_map_single(ndev->dev.parent, skb->data,
+ skb_headlen(skb), DMA_TO_DEVICE);
+ if (dma_mapping_error(ndev->dev.parent, cur_p->phys))
+ goto drop;
+
+ cur_p->cntrl = skb_headlen(skb) | XAXIDMA_BD_CTRL_TXSOF_MASK;
+
+ tx_skb->skb = NULL;
+ tx_skb->mapping = cur_p->phys;
+ tx_skb->size = skb_headlen(skb);
+ tx_skb->mapped_as_page = false;
+
+ for (ii = 0; ii < num_frag; ii++) {
+ ++priv->tx_bd_tail;
+ priv->tx_bd_tail %= TX_BD_NUM;
+ cur_p = &priv->tx_bd_v[priv->tx_bd_tail];
+ tx_skb = &priv->tx_skb[priv->tx_bd_tail];
+ frag = &skb_shinfo(skb)->frags[ii];
+
+ cur_p->phys = skb_frag_dma_map(ndev->dev.parent, frag, 0,
+ skb_frag_size(frag),
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(ndev->dev.parent, cur_p->phys))
+ goto frag_err;
+
+ cur_p->cntrl = skb_frag_size(frag);
+
+ tx_skb->skb = NULL;
+ tx_skb->mapping = cur_p->phys;
+ tx_skb->size = skb_frag_size(frag);
+ tx_skb->mapped_as_page = true;
+ }
+
+ /* last buffer of the frame */
+ tx_skb->skb = skb;
+
+ cur_p->cntrl |= XAXIDMA_BD_CTRL_TXEOF_MASK;
+ cur_p->app4 = (unsigned long)skb;
+
+ tail_p = priv->tx_bd_p + sizeof(*priv->tx_bd_v) * priv->tx_bd_tail;
+ /* Start the transfer */
+ nixge_dma_write_reg(priv, XAXIDMA_TX_TDESC_OFFSET, tail_p);
+ ++priv->tx_bd_tail;
+ priv->tx_bd_tail %= TX_BD_NUM;
+
+ return NETDEV_TX_OK;
+frag_err:
+ for (; ii > 0; ii--) {
+ if (priv->tx_bd_tail)
+ priv->tx_bd_tail--;
+ else
+ priv->tx_bd_tail = TX_BD_NUM - 1;
+
+ tx_skb = &priv->tx_skb[priv->tx_bd_tail];
+ nixge_tx_skb_unmap(priv, tx_skb);
+
+ cur_p = &priv->tx_bd_v[priv->tx_bd_tail];
+ cur_p->status = 0;
+ }
+ dma_unmap_single(priv->ndev->dev.parent,
+ tx_skb->mapping,
+ tx_skb->size, DMA_TO_DEVICE);
+drop:
+ ndev->stats.tx_dropped++;
+ return NETDEV_TX_OK;
+}
+
+static int nixge_recv(struct net_device *ndev, int budget)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ struct sk_buff *skb, *new_skb;
+ struct nixge_hw_dma_bd *cur_p;
+ dma_addr_t tail_p = 0;
+ u32 packets = 0;
+ u32 length = 0;
+ u32 size = 0;
+
+ cur_p = &priv->rx_bd_v[priv->rx_bd_ci];
+
+ while ((cur_p->status & XAXIDMA_BD_STS_COMPLETE_MASK &&
+ budget > packets)) {
+ tail_p = priv->rx_bd_p + sizeof(*priv->rx_bd_v) *
+ priv->rx_bd_ci;
+
+ skb = (struct sk_buff *)(cur_p->sw_id_offset);
+
+ length = cur_p->status & XAXIDMA_BD_STS_ACTUAL_LEN_MASK;
+ if (length > NIXGE_MAX_JUMBO_FRAME_SIZE)
+ length = NIXGE_MAX_JUMBO_FRAME_SIZE;
+
+ dma_unmap_single(ndev->dev.parent, cur_p->phys,
+ NIXGE_MAX_JUMBO_FRAME_SIZE,
+ DMA_FROM_DEVICE);
+
+ skb_put(skb, length);
+
+ skb->protocol = eth_type_trans(skb, ndev);
+ skb_checksum_none_assert(skb);
+
+ /* For now mark them as CHECKSUM_NONE since
+ * we don't have offload capabilities
+ */
+ skb->ip_summed = CHECKSUM_NONE;
+
+ napi_gro_receive(&priv->napi, skb);
+
+ size += length;
+ packets++;
+
+ new_skb = netdev_alloc_skb_ip_align(ndev,
+ NIXGE_MAX_JUMBO_FRAME_SIZE);
+ if (!new_skb)
+ return packets;
+
+ cur_p->phys = dma_map_single(ndev->dev.parent, new_skb->data,
+ NIXGE_MAX_JUMBO_FRAME_SIZE,
+ DMA_FROM_DEVICE);
+ if (dma_mapping_error(ndev->dev.parent, cur_p->phys)) {
+ /* FIXME: bail out and clean up */
+ netdev_err(ndev, "Failed to map ...\n");
+ }
+ cur_p->cntrl = NIXGE_MAX_JUMBO_FRAME_SIZE;
+ cur_p->status = 0;
+ cur_p->sw_id_offset = (u32)new_skb;
+
+ ++priv->rx_bd_ci;
+ priv->rx_bd_ci %= RX_BD_NUM;
+ cur_p = &priv->rx_bd_v[priv->rx_bd_ci];
+ }
+
+ ndev->stats.rx_packets += packets;
+ ndev->stats.rx_bytes += size;
+
+ if (tail_p)
+ nixge_dma_write_reg(priv, XAXIDMA_RX_TDESC_OFFSET, tail_p);
+
+ return packets;
+}
+
+static int nixge_poll(struct napi_struct *napi, int budget)
+{
+ struct nixge_priv *priv = container_of(napi, struct nixge_priv, napi);
+ int work_done;
+ u32 status, cr;
+
+ work_done = 0;
+
+ work_done = nixge_recv(priv->ndev, budget);
+ if (work_done < budget) {
+ napi_complete_done(napi, work_done);
+ status = nixge_dma_read_reg(priv, XAXIDMA_RX_SR_OFFSET);
+
+ if (status & (XAXIDMA_IRQ_IOC_MASK | XAXIDMA_IRQ_DELAY_MASK)) {
+ /* If there's more, reschedule, but clear */
+ nixge_dma_write_reg(priv, XAXIDMA_RX_SR_OFFSET, status);
+ napi_reschedule(napi);
+ } else {
+ /* if not, turn on RX IRQs again ... */
+ cr = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ cr |= (XAXIDMA_IRQ_IOC_MASK | XAXIDMA_IRQ_DELAY_MASK);
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CR_OFFSET, cr);
+ }
+ }
+
+ return work_done;
+}
+
+static irqreturn_t nixge_tx_irq(int irq, void *_ndev)
+{
+ struct nixge_priv *priv = netdev_priv(_ndev);
+ struct net_device *ndev = _ndev;
+ unsigned int status;
+ u32 cr;
+
+ status = nixge_dma_read_reg(priv, XAXIDMA_TX_SR_OFFSET);
+ if (status & (XAXIDMA_IRQ_IOC_MASK | XAXIDMA_IRQ_DELAY_MASK)) {
+ nixge_dma_write_reg(priv, XAXIDMA_TX_SR_OFFSET, status);
+ nixge_start_xmit_done(priv->ndev);
+ goto out;
+ }
+ if (!(status & XAXIDMA_IRQ_ALL_MASK)) {
+ netdev_err(ndev, "No interrupts asserted in Tx path\n");
+ return IRQ_NONE;
+ }
+ if (status & XAXIDMA_IRQ_ERROR_MASK) {
+ netdev_err(ndev, "DMA Tx error 0x%x\n", status);
+ netdev_err(ndev, "Current BD is at: 0x%x\n",
+ (priv->tx_bd_v[priv->tx_bd_ci]).phys);
+
+ cr = nixge_dma_read_reg(priv, XAXIDMA_TX_CR_OFFSET);
+ /* Disable coalesce, delay timer and error interrupts */
+ cr &= (~XAXIDMA_IRQ_ALL_MASK);
+ /* Write to the Tx channel control register */
+ nixge_dma_write_reg(priv, XAXIDMA_TX_CR_OFFSET, cr);
+
+ cr = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ /* Disable coalesce, delay timer and error interrupts */
+ cr &= (~XAXIDMA_IRQ_ALL_MASK);
+ /* Write to the Rx channel control register */
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CR_OFFSET, cr);
+
+ tasklet_schedule(&priv->dma_err_tasklet);
+ nixge_dma_write_reg(priv, XAXIDMA_TX_SR_OFFSET, status);
+ }
+out:
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t nixge_rx_irq(int irq, void *_ndev)
+{
+ struct nixge_priv *priv = netdev_priv(_ndev);
+ struct net_device *ndev = _ndev;
+ unsigned int status;
+ u32 cr;
+
+ status = nixge_dma_read_reg(priv, XAXIDMA_RX_SR_OFFSET);
+ if (status & (XAXIDMA_IRQ_IOC_MASK | XAXIDMA_IRQ_DELAY_MASK)) {
+ /* Turn of IRQs because NAPI */
+ nixge_dma_write_reg(priv, XAXIDMA_RX_SR_OFFSET, status);
+ cr = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ cr &= ~(XAXIDMA_IRQ_IOC_MASK | XAXIDMA_IRQ_DELAY_MASK);
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CR_OFFSET, cr);
+
+ if (napi_schedule_prep(&priv->napi))
+ __napi_schedule(&priv->napi);
+ goto out;
+ }
+ if (!(status & XAXIDMA_IRQ_ALL_MASK)) {
+ netdev_err(ndev, "No interrupts asserted in Rx path\n");
+ return IRQ_NONE;
+ }
+ if (status & XAXIDMA_IRQ_ERROR_MASK) {
+ netdev_err(ndev, "DMA Rx error 0x%x\n", status);
+ netdev_err(ndev, "Current BD is at: 0x%x\n",
+ (priv->rx_bd_v[priv->rx_bd_ci]).phys);
+
+ cr = nixge_dma_read_reg(priv, XAXIDMA_TX_CR_OFFSET);
+ /* Disable coalesce, delay timer and error interrupts */
+ cr &= (~XAXIDMA_IRQ_ALL_MASK);
+ /* Finally write to the Tx channel control register */
+ nixge_dma_write_reg(priv, XAXIDMA_TX_CR_OFFSET, cr);
+
+ cr = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ /* Disable coalesce, delay timer and error interrupts */
+ cr &= (~XAXIDMA_IRQ_ALL_MASK);
+ /* write to the Rx channel control register */
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CR_OFFSET, cr);
+
+ tasklet_schedule(&priv->dma_err_tasklet);
+ nixge_dma_write_reg(priv, XAXIDMA_RX_SR_OFFSET, status);
+ }
+out:
+ return IRQ_HANDLED;
+}
+
+static void nixge_dma_err_handler(unsigned long data)
+{
+ struct nixge_priv *lp = (struct nixge_priv *)data;
+ struct nixge_hw_dma_bd *cur_p;
+ struct nixge_tx_skb *tx_skb;
+ u32 cr, i;
+
+ __nixge_device_reset(lp, XAXIDMA_TX_CR_OFFSET);
+ __nixge_device_reset(lp, XAXIDMA_RX_CR_OFFSET);
+
+ for (i = 0; i < TX_BD_NUM; i++) {
+ cur_p = &lp->tx_bd_v[i];
+ tx_skb = &lp->tx_skb[i];
+ nixge_tx_skb_unmap(lp, tx_skb);
+
+ cur_p->phys = 0;
+ cur_p->cntrl = 0;
+ cur_p->status = 0;
+ cur_p->app0 = 0;
+ cur_p->app1 = 0;
+ cur_p->app2 = 0;
+ cur_p->app3 = 0;
+ cur_p->app4 = 0;
+ cur_p->sw_id_offset = 0;
+ }
+
+ for (i = 0; i < RX_BD_NUM; i++) {
+ cur_p = &lp->rx_bd_v[i];
+ cur_p->status = 0;
+ cur_p->app0 = 0;
+ cur_p->app1 = 0;
+ cur_p->app2 = 0;
+ cur_p->app3 = 0;
+ cur_p->app4 = 0;
+ }
+
+ lp->tx_bd_ci = 0;
+ lp->tx_bd_tail = 0;
+ lp->rx_bd_ci = 0;
+
+ /* Start updating the Rx channel control register */
+ cr = nixge_dma_read_reg(lp, XAXIDMA_RX_CR_OFFSET);
+ /* Update the interrupt coalesce count */
+ cr = ((cr & ~XAXIDMA_COALESCE_MASK) |
+ (XAXIDMA_DFT_RX_THRESHOLD << XAXIDMA_COALESCE_SHIFT));
+ /* Update the delay timer count */
+ cr = ((cr & ~XAXIDMA_DELAY_MASK) |
+ (XAXIDMA_DFT_RX_WAITBOUND << XAXIDMA_DELAY_SHIFT));
+ /* Enable coalesce, delay timer and error interrupts */
+ cr |= XAXIDMA_IRQ_ALL_MASK;
+ /* Finally write to the Rx channel control register */
+ nixge_dma_write_reg(lp, XAXIDMA_RX_CR_OFFSET, cr);
+
+ /* Start updating the Tx channel control register */
+ cr = nixge_dma_read_reg(lp, XAXIDMA_TX_CR_OFFSET);
+ /* Update the interrupt coalesce count */
+ cr = (((cr & ~XAXIDMA_COALESCE_MASK)) |
+ (XAXIDMA_DFT_TX_THRESHOLD << XAXIDMA_COALESCE_SHIFT));
+ /* Update the delay timer count */
+ cr = (((cr & ~XAXIDMA_DELAY_MASK)) |
+ (XAXIDMA_DFT_TX_WAITBOUND << XAXIDMA_DELAY_SHIFT));
+ /* Enable coalesce, delay timer and error interrupts */
+ cr |= XAXIDMA_IRQ_ALL_MASK;
+ /* Finally write to the Tx channel control register */
+ nixge_dma_write_reg(lp, XAXIDMA_TX_CR_OFFSET, cr);
+
+ /* Populate the tail pointer and bring the Rx Axi DMA engine out of
+ * halted state. This will make the Rx side ready for reception.
+ */
+ nixge_dma_write_reg(lp, XAXIDMA_RX_CDESC_OFFSET, lp->rx_bd_p);
+ cr = nixge_dma_read_reg(lp, XAXIDMA_RX_CR_OFFSET);
+ nixge_dma_write_reg(lp, XAXIDMA_RX_CR_OFFSET,
+ cr | XAXIDMA_CR_RUNSTOP_MASK);
+ nixge_dma_write_reg(lp, XAXIDMA_RX_TDESC_OFFSET, lp->rx_bd_p +
+ (sizeof(*lp->rx_bd_v) * (RX_BD_NUM - 1)));
+
+ /* Write to the RS (Run-stop) bit in the Tx channel control register.
+ * Tx channel is now ready to run. But only after we write to the
+ * tail pointer register that the Tx channel will start transmitting
+ */
+ nixge_dma_write_reg(lp, XAXIDMA_TX_CDESC_OFFSET, lp->tx_bd_p);
+ cr = nixge_dma_read_reg(lp, XAXIDMA_TX_CR_OFFSET);
+ nixge_dma_write_reg(lp, XAXIDMA_TX_CR_OFFSET,
+ cr | XAXIDMA_CR_RUNSTOP_MASK);
+}
+
+static int nixge_open(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ struct phy_device *phy;
+ int ret;
+
+ nixge_device_reset(ndev);
+
+ phy = of_phy_connect(ndev, priv->phy_node,
+ &nixge_handle_link_change, 0, priv->phy_mode);
+ if (!phy)
+ return -ENODEV;
+
+ phy_start(phy);
+
+ /* Enable tasklets for Axi DMA error handling */
+ tasklet_init(&priv->dma_err_tasklet, nixge_dma_err_handler,
+ (unsigned long)priv);
+
+ napi_enable(&priv->napi);
+
+ /* Enable interrupts for Axi DMA Tx */
+ ret = request_irq(priv->tx_irq, nixge_tx_irq, 0, ndev->name, ndev);
+ if (ret)
+ goto err_tx_irq;
+ /* Enable interrupts for Axi DMA Rx */
+ ret = request_irq(priv->rx_irq, nixge_rx_irq, 0, ndev->name, ndev);
+ if (ret)
+ goto err_rx_irq;
+
+ netif_start_queue(ndev);
+
+ return 0;
+
+err_rx_irq:
+ free_irq(priv->tx_irq, ndev);
+err_tx_irq:
+ phy_stop(phy);
+ phy_disconnect(phy);
+ tasklet_kill(&priv->dma_err_tasklet);
+ netdev_err(ndev, "request_irq() failed\n");
+ return ret;
+}
+
+static int nixge_stop(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ u32 cr;
+
+ netif_stop_queue(ndev);
+ napi_disable(&priv->napi);
+
+ if (ndev->phydev) {
+ phy_stop(ndev->phydev);
+ phy_disconnect(ndev->phydev);
+ }
+
+ cr = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ nixge_dma_write_reg(priv, XAXIDMA_RX_CR_OFFSET,
+ cr & (~XAXIDMA_CR_RUNSTOP_MASK));
+ cr = nixge_dma_read_reg(priv, XAXIDMA_TX_CR_OFFSET);
+ nixge_dma_write_reg(priv, XAXIDMA_TX_CR_OFFSET,
+ cr & (~XAXIDMA_CR_RUNSTOP_MASK));
+
+ tasklet_kill(&priv->dma_err_tasklet);
+
+ free_irq(priv->tx_irq, ndev);
+ free_irq(priv->rx_irq, ndev);
+
+ nixge_hw_dma_bd_release(ndev);
+
+ return 0;
+}
+
+static int nixge_change_mtu(struct net_device *ndev, int new_mtu)
+{
+ if (netif_running(ndev))
+ return -EBUSY;
+
+ if ((new_mtu + NIXGE_HDR_SIZE + NIXGE_TRL_SIZE) >
+ NIXGE_MAX_JUMBO_FRAME_SIZE)
+ return -EINVAL;
+
+ ndev->mtu = new_mtu;
+
+ return 0;
+}
+
+static s32 __nixge_hw_set_mac_address(struct net_device *ndev)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MAC_LSB,
+ (ndev->dev_addr[2]) << 24 |
+ (ndev->dev_addr[3] << 16) |
+ (ndev->dev_addr[4] << 8) |
+ (ndev->dev_addr[5] << 0));
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MAC_MSB,
+ (ndev->dev_addr[1] | (ndev->dev_addr[0] << 8)));
+
+ return 0;
+}
+
+static int nixge_net_set_mac_address(struct net_device *ndev, void *p)
+{
+ int err;
+
+ err = eth_mac_addr(ndev, p);
+ if (!err)
+ __nixge_hw_set_mac_address(ndev);
+
+ return err;
+}
+
+static const struct net_device_ops nixge_netdev_ops = {
+ .ndo_open = nixge_open,
+ .ndo_stop = nixge_stop,
+ .ndo_start_xmit = nixge_start_xmit,
+ .ndo_change_mtu = nixge_change_mtu,
+ .ndo_set_mac_address = nixge_net_set_mac_address,
+ .ndo_validate_addr = eth_validate_addr,
+};
+
+static void nixge_ethtools_get_drvinfo(struct net_device *ndev,
+ struct ethtool_drvinfo *ed)
+{
+ strlcpy(ed->driver, "nixge", sizeof(ed->driver));
+ strlcpy(ed->bus_info, "platform", sizeof(ed->driver));
+}
+
+static int nixge_ethtools_get_coalesce(struct net_device *ndev,
+ struct ethtool_coalesce *ecoalesce)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ u32 regval = 0;
+
+ regval = nixge_dma_read_reg(priv, XAXIDMA_RX_CR_OFFSET);
+ ecoalesce->rx_max_coalesced_frames = (regval & XAXIDMA_COALESCE_MASK)
+ >> XAXIDMA_COALESCE_SHIFT;
+ regval = nixge_dma_read_reg(priv, XAXIDMA_TX_CR_OFFSET);
+ ecoalesce->tx_max_coalesced_frames = (regval & XAXIDMA_COALESCE_MASK)
+ >> XAXIDMA_COALESCE_SHIFT;
+ return 0;
+}
+
+static int nixge_ethtools_set_coalesce(struct net_device *ndev,
+ struct ethtool_coalesce *ecoalesce)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+
+ if (netif_running(ndev)) {
+ netdev_err(ndev,
+ "Please stop netif before applying configuration\n");
+ return -EBUSY;
+ }
+
+ if (ecoalesce->rx_coalesce_usecs ||
+ ecoalesce->rx_coalesce_usecs_irq ||
+ ecoalesce->rx_max_coalesced_frames_irq ||
+ ecoalesce->tx_coalesce_usecs ||
+ ecoalesce->tx_coalesce_usecs_irq ||
+ ecoalesce->tx_max_coalesced_frames_irq ||
+ ecoalesce->stats_block_coalesce_usecs ||
+ ecoalesce->use_adaptive_rx_coalesce ||
+ ecoalesce->use_adaptive_tx_coalesce ||
+ ecoalesce->pkt_rate_low ||
+ ecoalesce->rx_coalesce_usecs_low ||
+ ecoalesce->rx_max_coalesced_frames_low ||
+ ecoalesce->tx_coalesce_usecs_low ||
+ ecoalesce->tx_max_coalesced_frames_low ||
+ ecoalesce->pkt_rate_high ||
+ ecoalesce->rx_coalesce_usecs_high ||
+ ecoalesce->rx_max_coalesced_frames_high ||
+ ecoalesce->tx_coalesce_usecs_high ||
+ ecoalesce->tx_max_coalesced_frames_high ||
+ ecoalesce->rate_sample_interval)
+ return -EOPNOTSUPP;
+ if (ecoalesce->rx_max_coalesced_frames)
+ priv->coalesce_count_rx = ecoalesce->rx_max_coalesced_frames;
+ if (ecoalesce->tx_max_coalesced_frames)
+ priv->coalesce_count_tx = ecoalesce->tx_max_coalesced_frames;
+
+ return 0;
+}
+
+static int nixge_ethtools_set_phys_id(struct net_device *ndev,
+ enum ethtool_phys_id_state state)
+{
+ struct nixge_priv *priv = netdev_priv(ndev);
+ u32 ctrl;
+
+ ctrl = nixge_ctrl_read_reg(priv, NIXGE_REG_LED_CTL);
+ switch (state) {
+ case ETHTOOL_ID_ACTIVE:
+ ctrl |= NIXGE_ID_LED_CTL_EN;
+ /* Enable identification LED override*/
+ nixge_ctrl_write_reg(priv, NIXGE_REG_LED_CTL, ctrl);
+ return 2;
+
+ case ETHTOOL_ID_ON:
+ ctrl |= NIXGE_ID_LED_CTL_VAL;
+ nixge_ctrl_write_reg(priv, NIXGE_REG_LED_CTL, ctrl);
+ break;
+
+ case ETHTOOL_ID_OFF:
+ ctrl &= ~NIXGE_ID_LED_CTL_VAL;
+ nixge_ctrl_write_reg(priv, NIXGE_REG_LED_CTL, ctrl);
+ break;
+
+ case ETHTOOL_ID_INACTIVE:
+ /* Restore LED settings */
+ ctrl &= ~NIXGE_ID_LED_CTL_EN;
+ nixge_ctrl_write_reg(priv, NIXGE_REG_LED_CTL, ctrl);
+ break;
+ }
+
+ return 0;
+}
+
+static const struct ethtool_ops nixge_ethtool_ops = {
+ .get_drvinfo = nixge_ethtools_get_drvinfo,
+ .get_coalesce = nixge_ethtools_get_coalesce,
+ .set_coalesce = nixge_ethtools_set_coalesce,
+ .set_phys_id = nixge_ethtools_set_phys_id,
+ .get_link_ksettings = phy_ethtool_get_link_ksettings,
+ .set_link_ksettings = phy_ethtool_set_link_ksettings,
+ .get_link = ethtool_op_get_link,
+};
+
+static int nixge_mdio_read(struct mii_bus *bus, int phy_id, int reg)
+{
+ struct nixge_priv *priv = bus->priv;
+ u32 status, tmp;
+ int err;
+ u16 device;
+
+ if (reg & MII_ADDR_C45) {
+ device = (reg >> 16) & 0x1f;
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_ADDR, reg & 0xffff);
+
+ tmp = NIXGE_MDIO_CLAUSE45 | NIXGE_MDIO_OP(NIXGE_MDIO_OP_ADDRESS)
+ | NIXGE_MDIO_ADDR(phy_id) | NIXGE_MDIO_MMD(device);
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_OP, tmp);
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_CTRL, 1);
+
+ err = nixge_ctrl_poll_timeout(priv, NIXGE_REG_MDIO_CTRL, status,
+ !status, 10, 1000);
+ if (err) {
+ dev_err(priv->dev, "timeout setting address");
+ return err;
+ }
+
+ tmp = NIXGE_MDIO_CLAUSE45 | NIXGE_MDIO_OP(NIXGE_MDIO_C45_READ) |
+ NIXGE_MDIO_ADDR(phy_id) | NIXGE_MDIO_MMD(device);
+ } else {
+ device = reg & 0x1f;
+
+ tmp = NIXGE_MDIO_CLAUSE22 | NIXGE_MDIO_OP(NIXGE_MDIO_C22_READ) |
+ NIXGE_MDIO_ADDR(phy_id) | NIXGE_MDIO_MMD(device);
+ }
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_OP, tmp);
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_CTRL, 1);
+
+ err = nixge_ctrl_poll_timeout(priv, NIXGE_REG_MDIO_CTRL, status,
+ !status, 10, 1000);
+ if (err) {
+ dev_err(priv->dev, "timeout setting read command");
+ return err;
+ }
+
+ status = nixge_ctrl_read_reg(priv, NIXGE_REG_MDIO_DATA);
+
+ return status;
+}
+
+static int nixge_mdio_write(struct mii_bus *bus, int phy_id, int reg, u16 val)
+{
+ struct nixge_priv *priv = bus->priv;
+ u32 status, tmp;
+ u16 device;
+ int err;
+
+ if (reg & MII_ADDR_C45) {
+ device = (reg >> 16) & 0x1f;
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_ADDR, reg & 0xffff);
+
+ tmp = NIXGE_MDIO_CLAUSE45 | NIXGE_MDIO_OP(NIXGE_MDIO_OP_ADDRESS)
+ | NIXGE_MDIO_ADDR(phy_id) | NIXGE_MDIO_MMD(device);
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_OP, tmp);
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_CTRL, 1);
+
+ err = nixge_ctrl_poll_timeout(priv, NIXGE_REG_MDIO_CTRL, status,
+ !status, 10, 1000);
+ if (err) {
+ dev_err(priv->dev, "timeout setting address");
+ return err;
+ }
+
+ tmp = NIXGE_MDIO_CLAUSE45 | NIXGE_MDIO_OP(NIXGE_MDIO_C45_WRITE)
+ | NIXGE_MDIO_ADDR(phy_id) | NIXGE_MDIO_MMD(device);
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_DATA, val);
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_OP, tmp);
+ err = nixge_ctrl_poll_timeout(priv, NIXGE_REG_MDIO_CTRL, status,
+ !status, 10, 1000);
+ if (err)
+ dev_err(priv->dev, "timeout setting write command");
+ } else {
+ device = reg & 0x1f;
+
+ tmp = NIXGE_MDIO_CLAUSE22 |
+ NIXGE_MDIO_OP(NIXGE_MDIO_C22_WRITE) |
+ NIXGE_MDIO_ADDR(phy_id) | NIXGE_MDIO_MMD(device);
+
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_DATA, val);
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_OP, tmp);
+ nixge_ctrl_write_reg(priv, NIXGE_REG_MDIO_CTRL, 1);
+
+ err = nixge_ctrl_poll_timeout(priv, NIXGE_REG_MDIO_CTRL, status,
+ !status, 10, 1000);
+ if (err)
+ dev_err(priv->dev, "timeout setting write command");
+ }
+
+ return err;
+}
+
+static int nixge_mdio_setup(struct nixge_priv *priv, struct device_node *np)
+{
+ struct mii_bus *bus;
+
+ bus = devm_mdiobus_alloc(priv->dev);
+ if (!bus)
+ return -ENOMEM;
+
+ snprintf(bus->id, MII_BUS_ID_SIZE, "%s-mii", dev_name(priv->dev));
+ bus->priv = priv;
+ bus->name = "nixge_mii_bus";
+ bus->read = nixge_mdio_read;
+ bus->write = nixge_mdio_write;
+ bus->parent = priv->dev;
+
+ priv->mii_bus = bus;
+
+ return of_mdiobus_register(bus, np);
+}
+
+static void *nixge_get_nvmem_address(struct device *dev)
+{
+ struct nvmem_cell *cell;
+ size_t cell_size;
+ char *mac;
+
+ cell = nvmem_cell_get(dev, "address");
+ if (IS_ERR(cell))
+ return cell;
+
+ mac = nvmem_cell_read(cell, &cell_size);
+ nvmem_cell_put(cell);
+
+ return mac;
+}
+
+static int nixge_probe(struct platform_device *pdev)
+{
+ struct nixge_priv *priv;
+ struct net_device *ndev;
+ struct resource *dmares;
+ const char *mac_addr;
+ int err;
+
+ ndev = alloc_etherdev(sizeof(*priv));
+ if (!ndev)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, ndev);
+ SET_NETDEV_DEV(ndev, &pdev->dev);
+
+ ndev->features = NETIF_F_SG;
+ ndev->netdev_ops = &nixge_netdev_ops;
+ ndev->ethtool_ops = &nixge_ethtool_ops;
+
+ /* MTU range: 64 - 9000 */
+ ndev->min_mtu = 64;
+ ndev->max_mtu = NIXGE_JUMBO_MTU;
+
+ mac_addr = nixge_get_nvmem_address(&pdev->dev);
+ if (mac_addr && is_valid_ether_addr(mac_addr))
+ ether_addr_copy(ndev->dev_addr, mac_addr);
+ else
+ eth_hw_addr_random(ndev);
+
+ priv = netdev_priv(ndev);
+ priv->ndev = ndev;
+ priv->dev = &pdev->dev;
+
+ netif_napi_add(ndev, &priv->napi, nixge_poll, NAPI_POLL_WEIGHT);
+
+ dmares = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ priv->dma_regs = devm_ioremap_resource(&pdev->dev, dmares);
+ if (IS_ERR(priv->dma_regs)) {
+ netdev_err(ndev, "failed to map dma regs\n");
+ return PTR_ERR(priv->dma_regs);
+ }
+ priv->ctrl_regs = priv->dma_regs + NIXGE_REG_CTRL_OFFSET;
+ __nixge_hw_set_mac_address(ndev);
+
+ priv->tx_irq = platform_get_irq_byname(pdev, "tx");
+ if (priv->tx_irq < 0) {
+ netdev_err(ndev, "could not find 'tx' irq");
+ return priv->tx_irq;
+ }
+
+ priv->rx_irq = platform_get_irq_byname(pdev, "rx");
+ if (priv->rx_irq < 0) {
+ netdev_err(ndev, "could not find 'rx' irq");
+ return priv->rx_irq;
+ }
+
+ priv->coalesce_count_rx = XAXIDMA_DFT_RX_THRESHOLD;
+ priv->coalesce_count_tx = XAXIDMA_DFT_TX_THRESHOLD;
+
+ err = nixge_mdio_setup(priv, pdev->dev.of_node);
+ if (err) {
+ netdev_err(ndev, "error registering mdio bus");
+ goto free_netdev;
+ }
+
+ priv->phy_mode = of_get_phy_mode(pdev->dev.of_node);
+ if (priv->phy_mode < 0) {
+ netdev_err(ndev, "not find \"phy-mode\" property\n");
+ err = -EINVAL;
+ goto unregister_mdio;
+ }
+
+ priv->phy_node = of_parse_phandle(pdev->dev.of_node, "phy-handle", 0);
+ if (!priv->phy_node) {
+ netdev_err(ndev, "not find \"phy-handle\" property\n");
+ err = -EINVAL;
+ goto unregister_mdio;
+ }
+
+ err = register_netdev(priv->ndev);
+ if (err) {
+ netdev_err(ndev, "register_netdev() error (%i)\n", err);
+ goto unregister_mdio;
+ }
+
+ return 0;
+
+unregister_mdio:
+ mdiobus_unregister(priv->mii_bus);
+
+free_netdev:
+ free_netdev(ndev);
+
+ return err;
+}
+
+static int nixge_remove(struct platform_device *pdev)
+{
+ struct net_device *ndev = platform_get_drvdata(pdev);
+ struct nixge_priv *priv = netdev_priv(ndev);
+
+ unregister_netdev(ndev);
+
+ mdiobus_unregister(priv->mii_bus);
+
+ free_netdev(ndev);
+
+ return 0;
+}
+
+/* Match table for of_platform binding */
+static const struct of_device_id nixge_dt_ids[] = {
+ { .compatible = "ni,xge-enet-2.00", },
+ {},
+};
+MODULE_DEVICE_TABLE(of, nixge_dt_ids);
+
+static struct platform_driver nixge_driver = {
+ .probe = nixge_probe,
+ .remove = nixge_remove,
+ .driver = {
+ .name = "nixge",
+ .of_match_table = of_match_ptr(nixge_dt_ids),
+ },
+};
+module_platform_driver(nixge_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("National Instruments XGE Management MAC");
+MODULE_AUTHOR("Moritz Fischer <mdf@kernel.org>");
}
static const struct device_attribute dev_attr_bridged_mode = {
- .attr = {.name = "bridged_mode", .mode = (S_IRUGO | S_IWUSR)},
- .show = netxen_show_bridged_mode,
- .store = netxen_store_bridged_mode,
+ .attr = { .name = "bridged_mode", .mode = 0644 },
+ .show = netxen_show_bridged_mode,
+ .store = netxen_store_bridged_mode,
};
static ssize_t
}
static const struct device_attribute dev_attr_diag_mode = {
- .attr = {.name = "diag_mode", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "diag_mode", .mode = 0644 },
.show = netxen_show_diag_mode,
.store = netxen_store_diag_mode,
};
static const struct bin_attribute bin_attr_crb = {
- .attr = {.name = "crb", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "crb", .mode = 0644 },
.size = 0,
.read = netxen_sysfs_read_crb,
.write = netxen_sysfs_write_crb,
};
static const struct bin_attribute bin_attr_mem = {
- .attr = {.name = "mem", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "mem", .mode = 0644 },
.size = 0,
.read = netxen_sysfs_read_mem,
.write = netxen_sysfs_write_mem,
}
static const struct bin_attribute bin_attr_dimm = {
- .attr = { .name = "dimm", .mode = (S_IRUGO | S_IWUSR) },
+ .attr = { .name = "dimm", .mode = 0644 },
.size = sizeof(struct netxen_dimm_cfg),
.read = netxen_sysfs_read_dimm,
};
QED_COAL_MODE_ENABLE
};
+enum qed_nvm_cmd {
+ QED_PUT_FILE_BEGIN = DRV_MSG_CODE_NVM_PUT_FILE_BEGIN,
+ QED_PUT_FILE_DATA = DRV_MSG_CODE_NVM_PUT_FILE_DATA,
+ QED_NVM_WRITE_NVRAM = DRV_MSG_CODE_NVM_WRITE_NVRAM,
+ QED_GET_MCP_NVM_RESP = 0xFFFFFF00
+};
+
struct qed_eth_cb_ops;
struct qed_dev_info;
union qed_mcp_protocol_stats;
BAR_ID_1 /* Used for doorbells */
};
+struct qed_nvm_image_info {
+ u32 num_images;
+ struct bist_nvm_image_att *image_att;
+};
+
#define DRV_MODULE_VERSION \
__stringify(QED_MAJOR_VERSION) "." \
__stringify(QED_MINOR_VERSION) "." \
/* L2-related */
struct qed_l2_info *p_l2_info;
+ /* Nvm images number and attributes */
+ struct qed_nvm_image_info nvm_info;
+
struct qed_ptt *p_arfs_ptt;
struct qed_simd_fp_handler simd_proto_handler[64];
if (rc)
return rc;
- /* Free Task CXT */
+ /* Free Task CXT ( Intentionally RoCE as task-id is shared between
+ * RoCE and iWARP )
+ */
+ proto = PROTOCOLID_ROCE;
rc = qed_cxt_free_ilt_range(p_hwfn, QED_ELEM_TASK, 0,
qed_cxt_get_proto_tid_count(p_hwfn, proto));
if (rc)
u32 min;
u32 max;
bool is_preset;
+ bool is_persistent;
u32 exclude_all_preset_val;
u32 crash_preset_val;
};
static struct grc_param_defs s_grc_param_defs[] = {
/* DBG_GRC_PARAM_DUMP_TSTORM */
- {{1, 1, 1}, 0, 1, false, 1, 1},
+ {{1, 1, 1}, 0, 1, false, false, 1, 1},
/* DBG_GRC_PARAM_DUMP_MSTORM */
- {{1, 1, 1}, 0, 1, false, 1, 1},
+ {{1, 1, 1}, 0, 1, false, false, 1, 1},
/* DBG_GRC_PARAM_DUMP_USTORM */
- {{1, 1, 1}, 0, 1, false, 1, 1},
+ {{1, 1, 1}, 0, 1, false, false, 1, 1},
/* DBG_GRC_PARAM_DUMP_XSTORM */
- {{1, 1, 1}, 0, 1, false, 1, 1},
+ {{1, 1, 1}, 0, 1, false, false, 1, 1},
/* DBG_GRC_PARAM_DUMP_YSTORM */
- {{1, 1, 1}, 0, 1, false, 1, 1},
+ {{1, 1, 1}, 0, 1, false, false, 1, 1},
/* DBG_GRC_PARAM_DUMP_PSTORM */
- {{1, 1, 1}, 0, 1, false, 1, 1},
+ {{1, 1, 1}, 0, 1, false, false, 1, 1},
/* DBG_GRC_PARAM_DUMP_REGS */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_RAM */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_PBUF */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_IOR */
- {{0, 0, 0}, 0, 1, false, 0, 1},
+ {{0, 0, 0}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_VFC */
- {{0, 0, 0}, 0, 1, false, 0, 1},
+ {{0, 0, 0}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_CM_CTX */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_ILT */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_RSS */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_CAU */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_QM */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_MCP */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
- /* DBG_GRC_PARAM_RESERVED */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ /* DBG_GRC_PARAM_MCP_TRACE_META_SIZE */
+ {{1, 1, 1}, 1, 0xffffffff, false, true, 0, 1},
/* DBG_GRC_PARAM_DUMP_CFC */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_IGU */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_BRB */
- {{0, 0, 0}, 0, 1, false, 0, 1},
+ {{0, 0, 0}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_BTB */
- {{0, 0, 0}, 0, 1, false, 0, 1},
+ {{0, 0, 0}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_BMB */
- {{0, 0, 0}, 0, 1, false, 0, 1},
+ {{0, 0, 0}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_NIG */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_MULD */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_PRS */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_DMAE */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_TM */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_SDM */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_DIF */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_STATIC */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_UNSTALL */
- {{0, 0, 0}, 0, 1, false, 0, 0},
+ {{0, 0, 0}, 0, 1, false, false, 0, 0},
/* DBG_GRC_PARAM_NUM_LCIDS */
- {{MAX_LCIDS, MAX_LCIDS, MAX_LCIDS}, 1, MAX_LCIDS, false, MAX_LCIDS,
- MAX_LCIDS},
+ {{MAX_LCIDS, MAX_LCIDS, MAX_LCIDS}, 1, MAX_LCIDS, false, false,
+ MAX_LCIDS, MAX_LCIDS},
/* DBG_GRC_PARAM_NUM_LTIDS */
- {{MAX_LTIDS, MAX_LTIDS, MAX_LTIDS}, 1, MAX_LTIDS, false, MAX_LTIDS,
- MAX_LTIDS},
+ {{MAX_LTIDS, MAX_LTIDS, MAX_LTIDS}, 1, MAX_LTIDS, false, false,
+ MAX_LTIDS, MAX_LTIDS},
/* DBG_GRC_PARAM_EXCLUDE_ALL */
- {{0, 0, 0}, 0, 1, true, 0, 0},
+ {{0, 0, 0}, 0, 1, true, false, 0, 0},
/* DBG_GRC_PARAM_CRASH */
- {{0, 0, 0}, 0, 1, true, 0, 0},
+ {{0, 0, 0}, 0, 1, true, false, 0, 0},
/* DBG_GRC_PARAM_PARITY_SAFE */
- {{0, 0, 0}, 0, 1, false, 1, 0},
+ {{0, 0, 0}, 0, 1, false, false, 1, 0},
/* DBG_GRC_PARAM_DUMP_CM */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_DUMP_PHY */
- {{1, 1, 1}, 0, 1, false, 0, 1},
+ {{1, 1, 1}, 0, 1, false, false, 0, 1},
/* DBG_GRC_PARAM_NO_MCP */
- {{0, 0, 0}, 0, 1, false, 0, 0},
+ {{0, 0, 0}, 0, 1, false, false, 0, 0},
/* DBG_GRC_PARAM_NO_FW_VER */
- {{0, 0, 0}, 0, 1, false, 0, 0}
+ {{0, 0, 0}, 0, 1, false, false, 0, 0}
};
static struct rss_mem_defs s_rss_mem_defs[] = {
offset += qed_dump_section_hdr(dump_buf + offset,
dump, "mcp_trace_meta", 1);
- /* Read trace meta info (trace_meta_size_bytes is dword-aligned) */
- if (mcp_access) {
+ /* If MCP Trace meta size parameter was set, use it.
+ * Otherwise, read trace meta.
+ * trace_meta_size_bytes is dword-aligned.
+ */
+ trace_meta_size_bytes =
+ qed_grc_get_param(p_hwfn, DBG_GRC_PARAM_MCP_TRACE_META_SIZE);
+ if ((!trace_meta_size_bytes || dump) && mcp_access) {
status = qed_mcp_trace_get_meta_info(p_hwfn,
p_ptt,
trace_data_size_bytes,
u32 i;
for (i = 0; i < MAX_DBG_GRC_PARAMS; i++)
- dev_data->grc.param_val[i] =
- s_grc_param_defs[i].default_val[dev_data->chip_id];
+ if (!s_grc_param_defs[i].is_persistent)
+ dev_data->grc.param_val[i] =
+ s_grc_param_defs[i].default_val[dev_data->chip_id];
}
enum dbg_status qed_dbg_grc_get_dump_buf_size(struct qed_hwfn *p_hwfn,
/******************************** Variables **********************************/
-/* MCP Trace meta data - used in case the dump doesn't contain the meta data
- * (e.g. due to no NVRAM access).
+/* MCP Trace meta data array - used in case the dump doesn't contain the
+ * meta data (e.g. due to no NVRAM access).
*/
-static struct user_dbg_array s_mcp_trace_meta = { NULL, 0 };
+static struct user_dbg_array s_mcp_trace_meta_arr = { NULL, 0 };
+
+/* Parsed MCP Trace meta data info, based on MCP trace meta array */
+static struct mcp_trace_meta s_mcp_trace_meta;
+static bool s_mcp_trace_meta_valid;
/* Temporary buffer, used for print size calculations */
static char s_temp_buf[MAX_MSG_LEN];
val_ptr = (u8 *)&val;
+ /* Assume running on a LITTLE ENDIAN and the buffer is network order
+ * (BIG ENDIAN), as high order bytes are placed in lower memory address.
+ */
for (i = 0; i < num_bytes_to_read; i++) {
val_ptr[i] = bytes_buf[*offset];
*offset = qed_cyclic_add(*offset, 1, buf_size);
offset += 4;
}
- return offset / 4;
+ return (u32)offset / 4;
}
/* Reads a section header from the specified buffer.
{
u32 i;
+ s_mcp_trace_meta_valid = false;
+
/* Release modules */
if (meta->modules) {
for (i = 0; i < meta->modules_num; i++)
u8 *meta_buf_bytes = (u8 *)meta_buf;
u32 offset = 0, signature, i;
+ /* Free the previous meta before loading a new one. */
+ if (s_mcp_trace_meta_valid)
+ qed_mcp_trace_free_meta(p_hwfn, meta);
+
memset(meta, 0, sizeof(*meta));
/* Read first signature */
format_len, format_ptr->format_str);
}
+ s_mcp_trace_meta_valid = true;
return DBG_STATUS_OK;
}
+/* Parses an MCP trace buffer. If result_buf is not NULL, the MCP Trace results
+ * are printed to it. The parsing status is returned.
+ * Arguments:
+ * trace_buf - MCP trace cyclic buffer
+ * trace_buf_size - MCP trace cyclic buffer size in bytes
+ * data_offset - offset in bytes of the data to parse in the MCP trace cyclic
+ * buffer.
+ * data_size - size in bytes of data to parse.
+ * parsed_buf - destination buffer for parsed data.
+ * parsed_bytes - size of parsed data in bytes.
+ */
+static enum dbg_status qed_parse_mcp_trace_buf(u8 *trace_buf,
+ u32 trace_buf_size,
+ u32 data_offset,
+ u32 data_size,
+ char *parsed_buf,
+ u32 *parsed_bytes)
+{
+ u32 param_mask, param_shift;
+ enum dbg_status status;
+
+ *parsed_bytes = 0;
+
+ if (!s_mcp_trace_meta_valid)
+ return DBG_STATUS_MCP_TRACE_BAD_DATA;
+
+ status = DBG_STATUS_OK;
+
+ while (data_size) {
+ struct mcp_trace_format *format_ptr;
+ u8 format_level, format_module;
+ u32 params[3] = { 0, 0, 0 };
+ u32 header, format_idx, i;
+
+ if (data_size < MFW_TRACE_ENTRY_SIZE)
+ return DBG_STATUS_MCP_TRACE_BAD_DATA;
+
+ header = qed_read_from_cyclic_buf(trace_buf,
+ &data_offset,
+ trace_buf_size,
+ MFW_TRACE_ENTRY_SIZE);
+ data_size -= MFW_TRACE_ENTRY_SIZE;
+ format_idx = header & MFW_TRACE_EVENTID_MASK;
+
+ /* Skip message if its index doesn't exist in the meta data */
+ if (format_idx > s_mcp_trace_meta.formats_num) {
+ u8 format_size =
+ (u8)((header & MFW_TRACE_PRM_SIZE_MASK) >>
+ MFW_TRACE_PRM_SIZE_SHIFT);
+
+ if (data_size < format_size)
+ return DBG_STATUS_MCP_TRACE_BAD_DATA;
+
+ data_offset = qed_cyclic_add(data_offset,
+ format_size,
+ trace_buf_size);
+ data_size -= format_size;
+ continue;
+ }
+
+ format_ptr = &s_mcp_trace_meta.formats[format_idx];
+
+ for (i = 0,
+ param_mask = MCP_TRACE_FORMAT_P1_SIZE_MASK,
+ param_shift = MCP_TRACE_FORMAT_P1_SIZE_SHIFT;
+ i < MCP_TRACE_FORMAT_MAX_PARAMS;
+ i++,
+ param_mask <<= MCP_TRACE_FORMAT_PARAM_WIDTH,
+ param_shift += MCP_TRACE_FORMAT_PARAM_WIDTH) {
+ /* Extract param size (0..3) */
+ u8 param_size = (u8)((format_ptr->data & param_mask) >>
+ param_shift);
+
+ /* If the param size is zero, there are no other
+ * parameters.
+ */
+ if (!param_size)
+ break;
+
+ /* Size is encoded using 2 bits, where 3 is used to
+ * encode 4.
+ */
+ if (param_size == 3)
+ param_size = 4;
+
+ if (data_size < param_size)
+ return DBG_STATUS_MCP_TRACE_BAD_DATA;
+
+ params[i] = qed_read_from_cyclic_buf(trace_buf,
+ &data_offset,
+ trace_buf_size,
+ param_size);
+ data_size -= param_size;
+ }
+
+ format_level = (u8)((format_ptr->data &
+ MCP_TRACE_FORMAT_LEVEL_MASK) >>
+ MCP_TRACE_FORMAT_LEVEL_SHIFT);
+ format_module = (u8)((format_ptr->data &
+ MCP_TRACE_FORMAT_MODULE_MASK) >>
+ MCP_TRACE_FORMAT_MODULE_SHIFT);
+ if (format_level >= ARRAY_SIZE(s_mcp_trace_level_str))
+ return DBG_STATUS_MCP_TRACE_BAD_DATA;
+
+ /* Print current message to results buffer */
+ *parsed_bytes +=
+ sprintf(qed_get_buf_ptr(parsed_buf, *parsed_bytes),
+ "%s %-8s: ",
+ s_mcp_trace_level_str[format_level],
+ s_mcp_trace_meta.modules[format_module]);
+ *parsed_bytes +=
+ sprintf(qed_get_buf_ptr(parsed_buf, *parsed_bytes),
+ format_ptr->format_str,
+ params[0], params[1], params[2]);
+ }
+
+ /* Add string NULL terminator */
+ (*parsed_bytes)++;
+
+ return status;
+}
+
/* Parses an MCP Trace dump buffer.
* If result_buf is not NULL, the MCP Trace results are printed to it.
* In any case, the required results buffer size is assigned to
- * parsed_results_bytes.
+ * parsed_bytes.
* The parsing status is returned.
*/
static enum dbg_status qed_parse_mcp_trace_dump(struct qed_hwfn *p_hwfn,
u32 *dump_buf,
- char *results_buf,
- u32 *parsed_results_bytes)
+ char *parsed_buf,
+ u32 *parsed_bytes)
{
- u32 end_offset, bytes_left, trace_data_dwords, trace_meta_dwords;
- u32 param_mask, param_shift, param_num_val, num_section_params;
const char *section_name, *param_name, *param_str_val;
- u32 offset, results_offset = 0;
- struct mcp_trace_meta meta;
+ u32 data_size, trace_data_dwords, trace_meta_dwords;
+ u32 offset, results_offset, parsed_buf_bytes;
+ u32 param_num_val, num_section_params;
struct mcp_trace *trace;
enum dbg_status status;
const u32 *meta_buf;
u8 *trace_buf;
- *parsed_results_bytes = 0;
+ *parsed_bytes = 0;
/* Read global_params section */
dump_buf += qed_read_section_hdr(dump_buf,
/* Print global params */
dump_buf += qed_print_section_params(dump_buf,
num_section_params,
- results_buf, &results_offset);
+ parsed_buf, &results_offset);
/* Read trace_data section */
dump_buf += qed_read_section_hdr(dump_buf,
trace = (struct mcp_trace *)dump_buf;
trace_buf = (u8 *)dump_buf + sizeof(*trace);
offset = trace->trace_oldest;
- end_offset = trace->trace_prod;
- bytes_left = qed_cyclic_sub(end_offset, offset, trace->size);
+ data_size = qed_cyclic_sub(trace->trace_prod, offset, trace->size);
dump_buf += trace_data_dwords;
/* Read meta_data section */
/* Choose meta data buffer */
if (!trace_meta_dwords) {
/* Dump doesn't include meta data */
- if (!s_mcp_trace_meta.ptr)
+ if (!s_mcp_trace_meta_arr.ptr)
return DBG_STATUS_MCP_TRACE_NO_META;
- meta_buf = s_mcp_trace_meta.ptr;
+ meta_buf = s_mcp_trace_meta_arr.ptr;
} else {
/* Dump includes meta data */
meta_buf = dump_buf;
}
/* Allocate meta data memory */
- status = qed_mcp_trace_alloc_meta(p_hwfn, meta_buf, &meta);
+ status = qed_mcp_trace_alloc_meta(p_hwfn, meta_buf, &s_mcp_trace_meta);
if (status != DBG_STATUS_OK)
- goto free_mem;
-
- /* Ignore the level and modules masks - just print everything that is
- * already in the buffer.
- */
- while (bytes_left) {
- struct mcp_trace_format *format_ptr;
- u8 format_level, format_module;
- u32 params[3] = { 0, 0, 0 };
- u32 header, format_idx, i;
-
- if (bytes_left < MFW_TRACE_ENTRY_SIZE) {
- status = DBG_STATUS_MCP_TRACE_BAD_DATA;
- goto free_mem;
- }
-
- header = qed_read_from_cyclic_buf(trace_buf,
- &offset,
- trace->size,
- MFW_TRACE_ENTRY_SIZE);
- bytes_left -= MFW_TRACE_ENTRY_SIZE;
- format_idx = header & MFW_TRACE_EVENTID_MASK;
-
- /* Skip message if its index doesn't exist in the meta data */
- if (format_idx > meta.formats_num) {
- u8 format_size =
- (u8)((header &
- MFW_TRACE_PRM_SIZE_MASK) >>
- MFW_TRACE_PRM_SIZE_SHIFT);
-
- if (bytes_left < format_size) {
- status = DBG_STATUS_MCP_TRACE_BAD_DATA;
- goto free_mem;
- }
-
- offset = qed_cyclic_add(offset,
- format_size, trace->size);
- bytes_left -= format_size;
- continue;
- }
-
- format_ptr = &meta.formats[format_idx];
-
- for (i = 0,
- param_mask = MCP_TRACE_FORMAT_P1_SIZE_MASK, param_shift =
- MCP_TRACE_FORMAT_P1_SIZE_SHIFT;
- i < MCP_TRACE_FORMAT_MAX_PARAMS;
- i++, param_mask <<= MCP_TRACE_FORMAT_PARAM_WIDTH,
- param_shift += MCP_TRACE_FORMAT_PARAM_WIDTH) {
- /* Extract param size (0..3) */
- u8 param_size =
- (u8)((format_ptr->data &
- param_mask) >> param_shift);
-
- /* If the param size is zero, there are no other
- * parameters.
- */
- if (!param_size)
- break;
-
- /* Size is encoded using 2 bits, where 3 is used to
- * encode 4.
- */
- if (param_size == 3)
- param_size = 4;
-
- if (bytes_left < param_size) {
- status = DBG_STATUS_MCP_TRACE_BAD_DATA;
- goto free_mem;
- }
-
- params[i] = qed_read_from_cyclic_buf(trace_buf,
- &offset,
- trace->size,
- param_size);
-
- bytes_left -= param_size;
- }
+ return status;
- format_level =
- (u8)((format_ptr->data &
- MCP_TRACE_FORMAT_LEVEL_MASK) >>
- MCP_TRACE_FORMAT_LEVEL_SHIFT);
- format_module =
- (u8)((format_ptr->data &
- MCP_TRACE_FORMAT_MODULE_MASK) >>
- MCP_TRACE_FORMAT_MODULE_SHIFT);
- if (format_level >= ARRAY_SIZE(s_mcp_trace_level_str)) {
- status = DBG_STATUS_MCP_TRACE_BAD_DATA;
- goto free_mem;
- }
+ status = qed_parse_mcp_trace_buf(trace_buf,
+ trace->size,
+ offset,
+ data_size,
+ parsed_buf ?
+ parsed_buf + results_offset :
+ NULL,
+ &parsed_buf_bytes);
+ if (status != DBG_STATUS_OK)
+ return status;
- /* Print current message to results buffer */
- results_offset +=
- sprintf(qed_get_buf_ptr(results_buf,
- results_offset), "%s %-8s: ",
- s_mcp_trace_level_str[format_level],
- meta.modules[format_module]);
- results_offset +=
- sprintf(qed_get_buf_ptr(results_buf,
- results_offset),
- format_ptr->format_str, params[0], params[1],
- params[2]);
- }
+ *parsed_bytes = results_offset + parsed_buf_bytes;
-free_mem:
- *parsed_results_bytes = results_offset + 1;
- qed_mcp_trace_free_meta(p_hwfn, &meta);
- return status;
+ return DBG_STATUS_OK;
}
/* Parses a Reg FIFO dump buffer.
void qed_dbg_mcp_trace_set_meta_data(u32 *data, u32 size)
{
- s_mcp_trace_meta.ptr = data;
- s_mcp_trace_meta.size_in_dwords = size;
+ s_mcp_trace_meta_arr.ptr = data;
+ s_mcp_trace_meta_arr.size_in_dwords = size;
}
enum dbg_status qed_get_mcp_trace_results_buf_size(struct qed_hwfn *p_hwfn,
results_buf, &parsed_buf_size);
}
+enum dbg_status qed_print_mcp_trace_line(u8 *dump_buf,
+ u32 num_dumped_bytes,
+ char *results_buf)
+{
+ u32 parsed_bytes;
+
+ return qed_parse_mcp_trace_buf(dump_buf,
+ num_dumped_bytes,
+ 0,
+ num_dumped_bytes,
+ results_buf, &parsed_bytes);
+}
+
enum dbg_status qed_get_reg_fifo_results_buf_size(struct qed_hwfn *p_hwfn,
u32 *dump_buf,
u32 num_dumped_dwords,
}
}
+ qed_set_debug_engine(cdev, org_engine);
/* mcp_trace */
rc = qed_dbg_mcp_trace(cdev, (u8 *)buffer + offset +
REGDUMP_HEADER_SIZE, &feature_size);
DP_ERR(cdev, "qed_dbg_mcp_trace failed. rc = %d\n", rc);
}
- qed_set_debug_engine(cdev, org_engine);
-
return 0;
}
REGDUMP_HEADER_SIZE + qed_dbg_fw_asserts_size(cdev);
}
+ qed_set_debug_engine(cdev, org_engine);
+
/* Engine common */
regs_len += REGDUMP_HEADER_SIZE + qed_dbg_mcp_trace_size(cdev);
- qed_set_debug_engine(cdev, org_engine);
return regs_len;
}
qm_info->start_vport = (u8) RESC_START(p_hwfn, QED_VPORT);
/* rate limiting and weighted fair queueing are always enabled */
- qm_info->vport_rl_en = 1;
- qm_info->vport_wfq_en = 1;
+ qm_info->vport_rl_en = true;
+ qm_info->vport_wfq_en = true;
/* TC config is different for AH 4 port */
four_port = p_hwfn->cdev->num_ports_in_engine == MAX_NUM_PORTS_K2;
"pq overflow! pq %d, max pq %d\n", pq_idx, max_pq);
/* init pq params */
+ qm_info->qm_pq_params[pq_idx].port_id = p_hwfn->port_id;
qm_info->qm_pq_params[pq_idx].vport_id = qm_info->start_vport +
qm_info->num_vports;
qm_info->qm_pq_params[pq_idx].tc_id = tc;
pq = &(qm_info->qm_pq_params[i]);
DP_VERBOSE(p_hwfn,
NETIF_MSG_HW,
- "pq idx %d, vport_id %d, tc %d, wrr_grp %d, rl_valid %d\n",
+ "pq idx %d, port %d, vport_id %d, tc %d, wrr_grp %d, rl_valid %d\n",
qm_info->start_pq + i,
+ pq->port_id,
pq->vport_id,
pq->tc_id, pq->wrr_group, pq->rl_valid);
}
if (p_hwfn->mcp_info) {
if (p_hwfn->mcp_info->func_info.bandwidth_max)
- qm_info->pf_rl_en = 1;
+ qm_info->pf_rl_en = true;
if (p_hwfn->mcp_info->func_info.bandwidth_min)
- qm_info->pf_wfq_en = 1;
+ qm_info->pf_wfq_en = true;
}
memset(¶ms, 0, sizeof(params));
qed_vf_pf_tunnel_param_update(p_hwfn, p_params->p_tunn);
}
- p_hwfn->b_int_enabled = 1;
+ p_hwfn->b_int_enabled = true;
return 0;
}
return 0;
}
+static void qed_nvm_info_free(struct qed_hwfn *p_hwfn)
+{
+ kfree(p_hwfn->nvm_info.image_att);
+ p_hwfn->nvm_info.image_att = NULL;
+}
+
static int qed_hw_prepare_single(struct qed_hwfn *p_hwfn,
void __iomem *p_regview,
void __iomem *p_doorbells,
DP_NOTICE(p_hwfn, "Failed to initiate PF FLR\n");
}
+ /* NVRAM info initialization and population */
+ if (IS_LEAD_HWFN(p_hwfn)) {
+ rc = qed_mcp_nvm_info_populate(p_hwfn);
+ if (rc) {
+ DP_NOTICE(p_hwfn,
+ "Failed to populate nvm info shadow\n");
+ goto err2;
+ }
+ }
+
/* Allocate the init RT array and initialize the init-ops engine */
rc = qed_init_alloc(p_hwfn);
if (rc)
- goto err2;
+ goto err3;
return rc;
+err3:
+ if (IS_LEAD_HWFN(p_hwfn))
+ qed_nvm_info_free(p_hwfn);
err2:
if (IS_LEAD_HWFN(p_hwfn))
qed_iov_free_hw_info(p_hwfn->cdev);
if (rc) {
if (IS_PF(cdev)) {
qed_init_free(p_hwfn);
+ qed_nvm_info_free(p_hwfn);
qed_mcp_free(p_hwfn);
qed_hw_hwfn_free(p_hwfn);
}
}
qed_iov_free_hw_info(cdev);
+
+ qed_nvm_info_free(p_hwfn);
}
static void qed_chain_free_next_ptr(struct qed_dev *cdev,
__le16 reserved16;
__le16 tx_bd_cons;
__le16 tx_bd_or_spq_prod;
- __le16 word5;
+ __le16 updated_qm_pq_id;
__le16 conn_dpi;
u8 byte3;
u8 byte4;
enum gft_profile_type {
GFT_PROFILE_TYPE_4_TUPLE,
GFT_PROFILE_TYPE_L4_DST_PORT,
- GFT_PROFILE_TYPE_IP_DST_PORT,
+ GFT_PROFILE_TYPE_IP_DST_ADDR,
+ GFT_PROFILE_TYPE_IP_SRC_ADDR,
+ GFT_PROFILE_TYPE_TUNNEL_TYPE,
MAX_GFT_PROFILE_TYPE
};
u8 dcb_priority;
u8 dcb_tc;
u8 dscp_val;
- u8 reserved0;
+ u8 dcb_dont_add_vlan0;
};
/* Update tunnel configuration */
/* Attention bit mapping */
struct dbg_attn_bit_mapping {
- __le16 data;
+ u16 data;
#define DBG_ATTN_BIT_MAPPING_VAL_MASK 0x7FFF
#define DBG_ATTN_BIT_MAPPING_VAL_SHIFT 0
#define DBG_ATTN_BIT_MAPPING_IS_UNUSED_BIT_CNT_MASK 0x1
/* Attention block per-type data */
struct dbg_attn_block_type_data {
- __le16 names_offset;
- __le16 reserved1;
+ u16 names_offset;
+ u16 reserved1;
u8 num_regs;
u8 reserved2;
- __le16 regs_offset;
+ u16 regs_offset;
+
};
/* Block attentions */
/* Attention register result */
struct dbg_attn_reg_result {
- __le32 data;
+ u32 data;
#define DBG_ATTN_REG_RESULT_STS_ADDRESS_MASK 0xFFFFFF
#define DBG_ATTN_REG_RESULT_STS_ADDRESS_SHIFT 0
#define DBG_ATTN_REG_RESULT_NUM_REG_ATTN_MASK 0xFF
#define DBG_ATTN_REG_RESULT_NUM_REG_ATTN_SHIFT 24
- __le16 block_attn_offset;
- __le16 reserved;
- __le32 sts_val;
- __le32 mask_val;
+ u16 block_attn_offset;
+ u16 reserved;
+ u32 sts_val;
+ u32 mask_val;
};
/* Attention block result */
#define DBG_ATTN_BLOCK_RESULT_ATTN_TYPE_SHIFT 0
#define DBG_ATTN_BLOCK_RESULT_NUM_REGS_MASK 0x3F
#define DBG_ATTN_BLOCK_RESULT_NUM_REGS_SHIFT 2
- __le16 names_offset;
+ u16 names_offset;
struct dbg_attn_reg_result reg_results[15];
};
/* Mode header */
struct dbg_mode_hdr {
- __le16 data;
+ u16 data;
#define DBG_MODE_HDR_EVAL_MODE_MASK 0x1
#define DBG_MODE_HDR_EVAL_MODE_SHIFT 0
#define DBG_MODE_HDR_MODES_BUF_OFFSET_MASK 0x7FFF
/* Attention register */
struct dbg_attn_reg {
struct dbg_mode_hdr mode;
- __le16 block_attn_offset;
- __le32 data;
+ u16 block_attn_offset;
+ u32 data;
#define DBG_ATTN_REG_STS_ADDRESS_MASK 0xFFFFFF
#define DBG_ATTN_REG_STS_ADDRESS_SHIFT 0
#define DBG_ATTN_REG_NUM_REG_ATTN_MASK 0xFF
#define DBG_ATTN_REG_NUM_REG_ATTN_SHIFT 24
- __le32 sts_clr_address;
- __le32 mask_address;
+ u32 sts_clr_address;
+ u32 mask_address;
};
/* Attention types */
struct dbg_bus_block {
u8 num_of_lines;
u8 has_latency_events;
- __le16 lines_offset;
+ u16 lines_offset;
};
/* Debug Bus block user data */
struct dbg_bus_block_user_data {
u8 num_of_lines;
u8 has_latency_events;
- __le16 names_offset;
+ u16 names_offset;
};
/* Block Debug line data */
/* Memory data for registers dump */
struct dbg_dump_mem {
- __le32 dword0;
+ u32 dword0;
#define DBG_DUMP_MEM_ADDRESS_MASK 0xFFFFFF
#define DBG_DUMP_MEM_ADDRESS_SHIFT 0
#define DBG_DUMP_MEM_MEM_GROUP_ID_MASK 0xFF
#define DBG_DUMP_MEM_MEM_GROUP_ID_SHIFT 24
- __le32 dword1;
+ u32 dword1;
#define DBG_DUMP_MEM_LENGTH_MASK 0xFFFFFF
#define DBG_DUMP_MEM_LENGTH_SHIFT 0
#define DBG_DUMP_MEM_WIDE_BUS_MASK 0x1
/* Register data for registers dump */
struct dbg_dump_reg {
- __le32 data;
+ u32 data;
#define DBG_DUMP_REG_ADDRESS_MASK 0x7FFFFF
#define DBG_DUMP_REG_ADDRESS_SHIFT 0
#define DBG_DUMP_REG_WIDE_BUS_MASK 0x1
/* Split header for registers dump */
struct dbg_dump_split_hdr {
- __le32 hdr;
+ u32 hdr;
#define DBG_DUMP_SPLIT_HDR_DATA_SIZE_MASK 0xFFFFFF
#define DBG_DUMP_SPLIT_HDR_DATA_SIZE_SHIFT 0
#define DBG_DUMP_SPLIT_HDR_SPLIT_TYPE_ID_MASK 0xFF
/* Condition header for idle check */
struct dbg_idle_chk_cond_hdr {
struct dbg_mode_hdr mode; /* Mode header */
- __le16 data_size; /* size in dwords of the data following this header */
+ u16 data_size; /* size in dwords of the data following this header */
};
/* Idle Check condition register */
struct dbg_idle_chk_cond_reg {
- __le32 data;
+ u32 data;
#define DBG_IDLE_CHK_COND_REG_ADDRESS_MASK 0x7FFFFF
#define DBG_IDLE_CHK_COND_REG_ADDRESS_SHIFT 0
#define DBG_IDLE_CHK_COND_REG_WIDE_BUS_MASK 0x1
#define DBG_IDLE_CHK_COND_REG_WIDE_BUS_SHIFT 23
#define DBG_IDLE_CHK_COND_REG_BLOCK_ID_MASK 0xFF
#define DBG_IDLE_CHK_COND_REG_BLOCK_ID_SHIFT 24
- __le16 num_entries;
+ u16 num_entries;
u8 entry_size;
u8 start_entry;
};
/* Idle Check info register */
struct dbg_idle_chk_info_reg {
- __le32 data;
+ u32 data;
#define DBG_IDLE_CHK_INFO_REG_ADDRESS_MASK 0x7FFFFF
#define DBG_IDLE_CHK_INFO_REG_ADDRESS_SHIFT 0
#define DBG_IDLE_CHK_INFO_REG_WIDE_BUS_MASK 0x1
#define DBG_IDLE_CHK_INFO_REG_WIDE_BUS_SHIFT 23
#define DBG_IDLE_CHK_INFO_REG_BLOCK_ID_MASK 0xFF
#define DBG_IDLE_CHK_INFO_REG_BLOCK_ID_SHIFT 24
- __le16 size; /* register size in dwords */
+ u16 size; /* register size in dwords */
struct dbg_mode_hdr mode; /* Mode header */
};
/* Idle Check result header */
struct dbg_idle_chk_result_hdr {
- __le16 rule_id; /* Failing rule index */
- __le16 mem_entry_id; /* Failing memory entry index */
+ u16 rule_id; /* Failing rule index */
+ u16 mem_entry_id; /* Failing memory entry index */
u8 num_dumped_cond_regs; /* number of dumped condition registers */
u8 num_dumped_info_regs; /* number of dumped condition registers */
u8 severity; /* from dbg_idle_chk_severity_types enum */
#define DBG_IDLE_CHK_RESULT_REG_HDR_REG_ID_MASK 0x7F
#define DBG_IDLE_CHK_RESULT_REG_HDR_REG_ID_SHIFT 1
u8 start_entry; /* index of the first checked entry */
- __le16 size; /* register size in dwords */
+ u16 size; /* register size in dwords */
};
/* Idle Check rule */
struct dbg_idle_chk_rule {
- __le16 rule_id; /* Idle Check rule ID */
+ u16 rule_id; /* Idle Check rule ID */
u8 severity; /* value from dbg_idle_chk_severity_types enum */
u8 cond_id; /* Condition ID */
u8 num_cond_regs; /* number of condition registers */
u8 num_info_regs; /* number of info registers */
u8 num_imms; /* number of immediates in the condition */
u8 reserved1;
- __le16 reg_offset; /* offset of this rules registers in the idle check
- * register array (in dbg_idle_chk_reg units).
- */
- __le16 imm_offset; /* offset of this rules immediate values in the
- * immediate values array (in dwords).
- */
+ u16 reg_offset; /* offset of this rules registers in the idle check
+ * register array (in dbg_idle_chk_reg units).
+ */
+ u16 imm_offset; /* offset of this rules immediate values in the
+ * immediate values array (in dwords).
+ */
};
/* Idle Check rule parsing data */
struct dbg_idle_chk_rule_parsing_data {
- __le32 data;
+ u32 data;
#define DBG_IDLE_CHK_RULE_PARSING_DATA_HAS_FW_MSG_MASK 0x1
#define DBG_IDLE_CHK_RULE_PARSING_DATA_HAS_FW_MSG_SHIFT 0
#define DBG_IDLE_CHK_RULE_PARSING_DATA_STR_OFFSET_MASK 0x7FFFFFFF
/* Debug Bus block data */
struct dbg_bus_block_data {
- __le16 data;
+ u16 data;
#define DBG_BUS_BLOCK_DATA_ENABLE_MASK_MASK 0xF
#define DBG_BUS_BLOCK_DATA_ENABLE_MASK_SHIFT 0
#define DBG_BUS_BLOCK_DATA_RIGHT_SHIFT_MASK 0xF
/* Debug Bus memory address */
struct dbg_bus_mem_addr {
- __le32 lo;
- __le32 hi;
+ u32 lo;
+ u32 hi;
};
/* Debug Bus PCI buffer data */
struct dbg_bus_pci_buf_data {
struct dbg_bus_mem_addr phys_addr; /* PCI buffer physical address */
struct dbg_bus_mem_addr virt_addr; /* PCI buffer virtual address */
- __le32 size; /* PCI buffer size in bytes */
+ u32 size; /* PCI buffer size in bytes */
};
/* Debug Bus Storm EID range filter params */
u8 eid_range_not_mask;
u8 cid_filter_en;
union dbg_bus_storm_eid_params eid_filter_params;
- __le32 cid;
+ u32 cid;
};
/* Debug Bus data */
struct dbg_bus_data {
- __le32 app_version;
+ u32 app_version;
u8 state;
u8 hw_dwords;
- __le16 hw_id_mask;
+ u16 hw_id_mask;
u8 num_enabled_blocks;
u8 num_enabled_storms;
u8 target;
u8 adding_filter;
u8 filter_pre_trigger;
u8 filter_post_trigger;
- __le16 reserved;
+ u16 reserved;
u8 trigger_en;
struct dbg_bus_trigger_state_data trigger_states[3];
u8 next_trigger_state;
struct dbg_grc_data {
u8 params_initialized;
u8 reserved1;
- __le16 reserved2;
- __le32 param_val[48];
+ u16 reserved2;
+ u32 param_val[48];
};
/* Debug GRC params */
DBG_GRC_PARAM_DUMP_CAU,
DBG_GRC_PARAM_DUMP_QM,
DBG_GRC_PARAM_DUMP_MCP,
- DBG_GRC_PARAM_RESERVED,
+ DBG_GRC_PARAM_MCP_TRACE_META_SIZE,
DBG_GRC_PARAM_DUMP_CFC,
DBG_GRC_PARAM_DUMP_IGU,
DBG_GRC_PARAM_DUMP_BRB,
/* Idle Check data */
struct idle_chk_data {
- __le32 buf_size;
+ u32 buf_size;
u8 buf_size_set;
u8 reserved1;
- __le16 reserved2;
+ u16 reserved2;
};
/* Debug Tools data (per HW function) */
u8 platform_id;
u8 initialized;
u8 use_dmae;
- __le32 num_regs_read;
+ u32 num_regs_read;
};
/********************************/
/* BRB RAM init requirements */
struct init_brb_ram_req {
- __le32 guranteed_per_tc;
- __le32 headroom_per_tc;
- __le32 min_pkt_size;
- __le32 max_ports_per_engine;
+ u32 guranteed_per_tc;
+ u32 headroom_per_tc;
+ u32 min_pkt_size;
+ u32 max_ports_per_engine;
u8 num_active_tcs[MAX_NUM_PORTS];
};
struct init_ets_tc_req {
u8 use_sp;
u8 use_wfq;
- __le16 weight;
+ u16 weight;
};
/* ETS init requirements */
struct init_ets_req {
- __le32 mtu;
+ u32 mtu;
struct init_ets_tc_req tc_req[NUM_OF_TCS];
};
/* NIG LB RL init requirements */
struct init_nig_lb_rl_req {
- __le16 lb_mac_rate;
- __le16 lb_rate;
- __le32 mtu;
- __le16 tc_rate[NUM_OF_PHYS_TCS];
+ u16 lb_mac_rate;
+ u16 lb_rate;
+ u32 mtu;
+ u16 tc_rate[NUM_OF_PHYS_TCS];
};
/* NIG TC mapping for each priority */
struct init_qm_port_params {
u8 active;
u8 active_phys_tcs;
- __le16 num_pbf_cmd_lines;
- __le16 num_btb_blocks;
- __le16 reserved;
+ u16 num_pbf_cmd_lines;
+ u16 num_btb_blocks;
+ u16 reserved;
};
/* QM per-PQ init parameters */
u8 tc_id;
u8 wrr_group;
u8 rl_valid;
+ u8 port_id;
+ u8 reserved0;
+ u16 reserved1;
};
/* QM per-vport init parameters */
struct init_qm_vport_params {
- __le32 vport_rl;
- __le16 vport_wfq;
- __le16 first_tx_pq_id[NUM_OF_TCS];
+ u32 vport_rl;
+ u16 vport_wfq;
+ u16 first_tx_pq_id[NUM_OF_TCS];
};
/**************************************/
};
struct fw_asserts_ram_section {
- __le16 section_ram_line_offset;
- __le16 section_ram_line_size;
+ u16 section_ram_line_offset;
+ u16 section_ram_line_size;
u8 list_dword_offset;
u8 list_element_dword_size;
u8 list_num_elements;
/* Binary buffer header */
struct bin_buffer_hdr {
- __le32 offset;
- __le32 length;
+ u32 offset;
+ u32 length;
};
/* Binary init buffer types */
/* init array header: raw */
struct init_array_raw_hdr {
- __le32 data;
+ u32 data;
#define INIT_ARRAY_RAW_HDR_TYPE_MASK 0xF
#define INIT_ARRAY_RAW_HDR_TYPE_SHIFT 0
#define INIT_ARRAY_RAW_HDR_PARAMS_MASK 0xFFFFFFF
/* init array header: standard */
struct init_array_standard_hdr {
- __le32 data;
+ u32 data;
#define INIT_ARRAY_STANDARD_HDR_TYPE_MASK 0xF
#define INIT_ARRAY_STANDARD_HDR_TYPE_SHIFT 0
#define INIT_ARRAY_STANDARD_HDR_SIZE_MASK 0xFFFFFFF
/* init array header: zipped */
struct init_array_zipped_hdr {
- __le32 data;
+ u32 data;
#define INIT_ARRAY_ZIPPED_HDR_TYPE_MASK 0xF
#define INIT_ARRAY_ZIPPED_HDR_TYPE_SHIFT 0
#define INIT_ARRAY_ZIPPED_HDR_ZIPPED_SIZE_MASK 0xFFFFFFF
/* init array header: pattern */
struct init_array_pattern_hdr {
- __le32 data;
+ u32 data;
#define INIT_ARRAY_PATTERN_HDR_TYPE_MASK 0xF
#define INIT_ARRAY_PATTERN_HDR_TYPE_SHIFT 0
#define INIT_ARRAY_PATTERN_HDR_PATTERN_SIZE_MASK 0xF
/* init operation: callback */
struct init_callback_op {
- __le32 op_data;
+ u32 op_data;
#define INIT_CALLBACK_OP_OP_MASK 0xF
#define INIT_CALLBACK_OP_OP_SHIFT 0
#define INIT_CALLBACK_OP_RESERVED_MASK 0xFFFFFFF
#define INIT_CALLBACK_OP_RESERVED_SHIFT 4
- __le16 callback_id;
- __le16 block_id;
+ u16 callback_id;
+ u16 block_id;
};
/* init operation: delay */
struct init_delay_op {
- __le32 op_data;
+ u32 op_data;
#define INIT_DELAY_OP_OP_MASK 0xF
#define INIT_DELAY_OP_OP_SHIFT 0
#define INIT_DELAY_OP_RESERVED_MASK 0xFFFFFFF
#define INIT_DELAY_OP_RESERVED_SHIFT 4
- __le32 delay;
+ u32 delay;
};
/* init operation: if_mode */
struct init_if_mode_op {
- __le32 op_data;
+ u32 op_data;
#define INIT_IF_MODE_OP_OP_MASK 0xF
#define INIT_IF_MODE_OP_OP_SHIFT 0
#define INIT_IF_MODE_OP_RESERVED1_MASK 0xFFF
#define INIT_IF_MODE_OP_RESERVED1_SHIFT 4
#define INIT_IF_MODE_OP_CMD_OFFSET_MASK 0xFFFF
#define INIT_IF_MODE_OP_CMD_OFFSET_SHIFT 16
- __le16 reserved2;
- __le16 modes_buf_offset;
+ u16 reserved2;
+ u16 modes_buf_offset;
};
/* init operation: if_phase */
struct init_if_phase_op {
- __le32 op_data;
+ u32 op_data;
#define INIT_IF_PHASE_OP_OP_MASK 0xF
#define INIT_IF_PHASE_OP_OP_SHIFT 0
#define INIT_IF_PHASE_OP_DMAE_ENABLE_MASK 0x1
#define INIT_IF_PHASE_OP_RESERVED1_SHIFT 5
#define INIT_IF_PHASE_OP_CMD_OFFSET_MASK 0xFFFF
#define INIT_IF_PHASE_OP_CMD_OFFSET_SHIFT 16
- __le32 phase_data;
+ u32 phase_data;
#define INIT_IF_PHASE_OP_PHASE_MASK 0xFF
#define INIT_IF_PHASE_OP_PHASE_SHIFT 0
#define INIT_IF_PHASE_OP_RESERVED2_MASK 0xFF
/* init operation: raw */
struct init_raw_op {
- __le32 op_data;
+ u32 op_data;
#define INIT_RAW_OP_OP_MASK 0xF
#define INIT_RAW_OP_OP_SHIFT 0
#define INIT_RAW_OP_PARAM1_MASK 0xFFFFFFF
#define INIT_RAW_OP_PARAM1_SHIFT 4
- __le32 param2;
+ u32 param2;
};
/* init array params */
struct init_op_array_params {
- __le16 size;
- __le16 offset;
+ u16 size;
+ u16 offset;
};
/* Write init operation arguments */
union init_write_args {
- __le32 inline_val;
- __le32 zeros_count;
- __le32 array_offset;
+ u32 inline_val;
+ u32 zeros_count;
+ u32 array_offset;
struct init_op_array_params runtime;
};
/* init operation: write */
struct init_write_op {
- __le32 data;
+ u32 data;
#define INIT_WRITE_OP_OP_MASK 0xF
#define INIT_WRITE_OP_OP_SHIFT 0
#define INIT_WRITE_OP_SOURCE_MASK 0x7
/* init operation: read */
struct init_read_op {
- __le32 op_data;
+ u32 op_data;
#define INIT_READ_OP_OP_MASK 0xF
#define INIT_READ_OP_OP_SHIFT 0
#define INIT_READ_OP_POLL_TYPE_MASK 0xF
#define INIT_READ_OP_RESERVED_SHIFT 8
#define INIT_READ_OP_ADDRESS_MASK 0x7FFFFF
#define INIT_READ_OP_ADDRESS_SHIFT 9
- __le32 expected_val;
+ u32 expected_val;
};
/* Init operations union */
/* Internal RAM Offsets macro data */
struct iro {
- __le32 base;
- __le16 m1;
- __le16 m2;
- __le16 m3;
- __le16 size;
+ u32 base;
+ u16 m1;
+ u16 m2;
+ u16 m3;
+ u16 size;
};
/***************************** Public Functions *******************************/
u32 num_dumped_dwords,
char *results_buf);
+/**
+ * @brief print_mcp_trace_line - Prints MCP Trace results for a single line
+ *
+ * @param dump_buf - mcp trace dump buffer, starting from the header.
+ * @param num_dumped_bytes - number of bytes that were dumped.
+ * @param results_buf - buffer for printing the mcp trace results.
+ *
+ * @return error if the parsing fails, ok otherwise.
+ */
+enum dbg_status qed_print_mcp_trace_line(u8 *dump_buf,
+ u32 num_dumped_bytes,
+ char *results_buf);
+
/**
* @brief qed_get_reg_fifo_results_buf_size - Returns the required buffer size
* for reg_fifo results (in bytes).
struct qed_ptt *p_ptt,
bool eth_geneve_enable, bool ip_geneve_enable);
+void qed_set_vxlan_no_l2_enable(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt, bool enable);
+
/**
* @brief qed_gft_disable - Disable GFT
*
{0x80, 0x8, 0x0, 0x0, 0x4},
{0x84, 0x8, 0x0, 0x0, 0x2},
{0x4c48, 0x0, 0x0, 0x0, 0x78},
- {0x3e18, 0x0, 0x0, 0x0, 0x78},
- {0x2b58, 0x0, 0x0, 0x0, 0x78},
+ {0x3e38, 0x0, 0x0, 0x0, 0x78},
+ {0x2b78, 0x0, 0x0, 0x0, 0x78},
{0x4c40, 0x0, 0x0, 0x0, 0x78},
{0x4998, 0x0, 0x0, 0x0, 0x78},
{0x7f50, 0x0, 0x0, 0x0, 0x78},
{0x4ba8, 0x80, 0x0, 0x0, 0x20},
{0x8158, 0x40, 0x0, 0x0, 0x30},
{0xe770, 0x60, 0x0, 0x0, 0x60},
- {0x2cf0, 0x80, 0x0, 0x0, 0x38},
+ {0x2d10, 0x80, 0x0, 0x0, 0x38},
{0xf2b8, 0x78, 0x0, 0x0, 0x78},
{0x1f8, 0x4, 0x0, 0x0, 0x4},
{0xaf20, 0x0, 0x0, 0x0, 0xf0},
{0x10300, 0x18, 0x0, 0x0, 0x10},
{0xde48, 0x48, 0x0, 0x0, 0x38},
{0x10768, 0x20, 0x0, 0x0, 0x20},
- {0x2d28, 0x80, 0x0, 0x0, 0x10},
+ {0x2d48, 0x80, 0x0, 0x0, 0x10},
{0x5048, 0x10, 0x0, 0x0, 0x10},
{0xc9b8, 0x30, 0x0, 0x0, 0x10},
- {0xeee0, 0x10, 0x0, 0x0, 0x10},
+ {0xed90, 0x10, 0x0, 0x0, 0x10},
{0xa3a0, 0x10, 0x0, 0x0, 0x10},
{0x13108, 0x8, 0x0, 0x0, 0x8},
};
__le16 edpm_num_bds;
__le16 tx_bd_cons;
__le16 tx_bd_prod;
- __le16 tx_class;
+ __le16 updated_qm_pq_id;
__le16 conn_dpi;
u8 byte3;
u8 byte4;
#define ETH_VPORT_RX_MODE_BCAST_ACCEPT_ALL_SHIFT 5
#define ETH_VPORT_RX_MODE_RESERVED1_MASK 0x3FF
#define ETH_VPORT_RX_MODE_RESERVED1_SHIFT 6
- __le16 reserved2[3];
};
/* Command for setting tpa parameters */
#define ETH_VPORT_TX_MODE_BCAST_ACCEPT_ALL_SHIFT 4
#define ETH_VPORT_TX_MODE_RESERVED1_MASK 0x7FF
#define ETH_VPORT_TX_MODE_RESERVED1_SHIFT 5
- __le16 reserved2[3];
};
/* GFT filter update action type */
u8 complete_cqe_flg;
u8 complete_event_flg;
u8 vport_id;
- u8 reserved[4];
+ u8 set_default_rss_queue;
+ u8 reserved[3];
u8 reserved1;
u8 reserved2;
u8 reserved3;
u8 flow_id_valid;
u8 filter_action;
u8 assert_on_error;
- u8 reserved;
+ u8 inner_vlan_removal_en;
};
/* Ramrod data for rx queue start ramrod */
u8 zero_placement_offset;
u8 ctl_frame_mac_check_en;
u8 ctl_frame_ethtype_check_en;
- u8 reserved[5];
+ u8 reserved[1];
};
/* Ramrod data for vport stop ramrod */
struct eth_vport_rx_mode rx_mode;
struct eth_vport_tx_mode tx_mode;
+ __le32 reserved[3];
struct eth_vport_tpa_param tpa_param;
struct vport_update_ramrod_mcast approx_mcast;
struct eth_vport_rss_config rss_config;
__le16 edpm_num_bds;
__le16 tx_bd_cons;
__le16 tx_bd_prod;
- __le16 tx_class;
+ __le16 updated_qm_pq_id;
__le16 conn_dpi;
u8 byte3;
u8 byte4;
__le16 edpm_num_bds;
__le16 tx_bd_cons;
__le16 tx_bd_prod;
- __le16 tx_class;
+ __le16 updated_qm_pq_id;
__le16 conn_dpi;
};
#define E4_YSTORM_RDMA_TASK_AG_CTX_BIT1_SHIFT 5
#define E4_YSTORM_RDMA_TASK_AG_CTX_VALID_MASK 0x1
#define E4_YSTORM_RDMA_TASK_AG_CTX_VALID_SHIFT 6
-#define E4_YSTORM_RDMA_TASK_AG_CTX_BIT3_MASK 0x1
-#define E4_YSTORM_RDMA_TASK_AG_CTX_BIT3_SHIFT 7
+#define E4_YSTORM_RDMA_TASK_AG_CTX_DIF_FIRST_IO_MASK 0x1
+#define E4_YSTORM_RDMA_TASK_AG_CTX_DIF_FIRST_IO_SHIFT 7
u8 flags1;
#define E4_YSTORM_RDMA_TASK_AG_CTX_CF0_MASK 0x3
#define E4_YSTORM_RDMA_TASK_AG_CTX_CF0_SHIFT 0
#define E4_MSTORM_RDMA_TASK_AG_CTX_BIT1_SHIFT 5
#define E4_MSTORM_RDMA_TASK_AG_CTX_BIT2_MASK 0x1
#define E4_MSTORM_RDMA_TASK_AG_CTX_BIT2_SHIFT 6
-#define E4_MSTORM_RDMA_TASK_AG_CTX_BIT3_MASK 0x1
-#define E4_MSTORM_RDMA_TASK_AG_CTX_BIT3_SHIFT 7
+#define E4_MSTORM_RDMA_TASK_AG_CTX_DIF_FIRST_IO_MASK 0x1
+#define E4_MSTORM_RDMA_TASK_AG_CTX_DIF_FIRST_IO_SHIFT 7
u8 flags1;
#define E4_MSTORM_RDMA_TASK_AG_CTX_CF0_MASK 0x3
#define E4_MSTORM_RDMA_TASK_AG_CTX_CF0_SHIFT 0
struct e4_ustorm_rdma_task_ag_ctx {
u8 reserved;
- u8 byte1;
+ u8 state;
__le16 icid;
u8 flags0;
#define E4_USTORM_RDMA_TASK_AG_CTX_CONNECTION_TYPE_MASK 0xF
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_RESULT_TOGGLE_BIT_SHIFT 0
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_TX_IO_FLG_MASK 0x3
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_TX_IO_FLG_SHIFT 2
-#define E4_USTORM_RDMA_TASK_AG_CTX_CF3_MASK 0x3
-#define E4_USTORM_RDMA_TASK_AG_CTX_CF3_SHIFT 4
+#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_BLOCK_SIZE_MASK 0x3
+#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_BLOCK_SIZE_SHIFT 4
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_ERROR_CF_MASK 0x3
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_ERROR_CF_SHIFT 6
u8 flags2;
#define E4_USTORM_RDMA_TASK_AG_CTX_RESERVED2_SHIFT 1
#define E4_USTORM_RDMA_TASK_AG_CTX_RESERVED3_MASK 0x1
#define E4_USTORM_RDMA_TASK_AG_CTX_RESERVED3_SHIFT 2
-#define E4_USTORM_RDMA_TASK_AG_CTX_CF3EN_MASK 0x1
-#define E4_USTORM_RDMA_TASK_AG_CTX_CF3EN_SHIFT 3
+#define E4_USTORM_RDMA_TASK_AG_CTX_RESERVED4_MASK 0x1
+#define E4_USTORM_RDMA_TASK_AG_CTX_RESERVED4_SHIFT 3
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_ERROR_CF_EN_MASK 0x1
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_ERROR_CF_EN_SHIFT 4
#define E4_USTORM_RDMA_TASK_AG_CTX_RULE0EN_MASK 0x1
#define E4_USTORM_RDMA_TASK_AG_CTX_DIF_ERROR_TYPE_SHIFT 4
__le32 dif_err_intervals;
__le32 dif_error_1st_interval;
- __le32 reg2;
+ __le32 sq_cons;
__le32 dif_runt_value;
- __le32 reg4;
+ __le32 sge_index;
__le32 reg5;
+ u8 byte2;
+ u8 byte3;
+ __le16 word1;
+ __le16 word2;
+ __le16 word3;
+ __le32 reg6;
+ __le32 reg7;
};
/* RDMA task context */
u8 vf_id;
u8 vf_valid;
u8 relaxed_ordering;
- u8 reserved[2];
+ __le16 first_reg_srq_id;
+ __le32 reg_srq_base_addr;
+ __le32 reserved;
};
/* rdma function init ramrod data */
/* rdma create qp requester ramrod data */
struct rdma_srq_create_ramrod_data {
+ u8 flags;
+#define RDMA_SRQ_CREATE_RAMROD_DATA_XRC_FLAG_MASK 0x1
+#define RDMA_SRQ_CREATE_RAMROD_DATA_XRC_FLAG_SHIFT 0
+#define RDMA_SRQ_CREATE_RAMROD_DATA_RESERVED_KEY_EN_MASK 0x1
+#define RDMA_SRQ_CREATE_RAMROD_DATA_RESERVED_KEY_EN_SHIFT 1
+#define RDMA_SRQ_CREATE_RAMROD_DATA_RESERVED1_MASK 0x3F
+#define RDMA_SRQ_CREATE_RAMROD_DATA_RESERVED1_SHIFT 2
+ u8 reserved2;
+ __le16 xrc_domain;
+ __le32 xrc_srq_cq_cid;
struct regpair pbl_base_addr;
__le16 pages_in_srq_pbl;
__le16 pd_id;
struct rdma_srq_id srq_id;
__le16 page_size;
- __le16 reserved1;
- __le32 reserved2;
+ __le16 reserved3;
+ __le32 reserved4;
struct regpair producers_addr;
};
MAX_RDMA_TID_TYPE
};
-struct e4_xstorm_roce_conn_ag_ctx_dq_ext_ld_part {
+struct rdma_xrc_srq_context {
+ struct regpair temp[9];
+};
+
+struct e4_tstorm_rdma_task_ag_ctx {
+ u8 byte0;
+ u8 byte1;
+ __le16 word0;
+ u8 flags0;
+#define E4_TSTORM_RDMA_TASK_AG_CTX_NIBBLE0_MASK 0xF
+#define E4_TSTORM_RDMA_TASK_AG_CTX_NIBBLE0_SHIFT 0
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT0_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT0_SHIFT 4
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT1_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT1_SHIFT 5
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT2_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT2_SHIFT 6
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT3_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT3_SHIFT 7
+ u8 flags1;
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT4_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT4_SHIFT 0
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT5_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT5_SHIFT 1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0_SHIFT 2
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1_SHIFT 4
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2_SHIFT 6
+ u8 flags2;
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3_SHIFT 0
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4_SHIFT 2
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5_SHIFT 4
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6_SHIFT 6
+ u8 flags3;
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7_MASK 0x3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7_SHIFT 0
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0EN_SHIFT 2
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1EN_SHIFT 3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2EN_SHIFT 4
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3EN_SHIFT 5
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4EN_SHIFT 6
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5EN_SHIFT 7
+ u8 flags4;
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6EN_SHIFT 0
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7EN_SHIFT 1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE0EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE0EN_SHIFT 2
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE1EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE1EN_SHIFT 3
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE2EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE2EN_SHIFT 4
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE3EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE3EN_SHIFT 5
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE4EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE4EN_SHIFT 6
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE5EN_MASK 0x1
+#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE5EN_SHIFT 7
+ u8 byte2;
+ __le16 word1;
+ __le32 reg0;
+ u8 byte3;
+ u8 byte4;
+ __le16 word2;
+ __le16 word3;
+ __le16 word4;
+ __le32 reg1;
+ __le32 reg2;
+};
+
+struct e4_ustorm_rdma_conn_ag_ctx {
+ u8 reserved;
+ u8 byte1;
+ u8 flags0;
+#define E4_USTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_SHIFT 0
+#define E4_USTORM_RDMA_CONN_AG_CTX_DIF_ERROR_REPORTED_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_DIF_ERROR_REPORTED_SHIFT 1
+#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
+#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 2
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF1_MASK 0x3
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF1_SHIFT 4
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF2_MASK 0x3
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF2_SHIFT 6
+ u8 flags1;
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF3_MASK 0x3
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF3_SHIFT 0
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_MASK 0x3
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_SHIFT 2
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_MASK 0x3
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_SHIFT 4
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF6_MASK 0x3
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF6_SHIFT 6
+ u8 flags2;
+#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 0
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF1EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF1EN_SHIFT 1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF2EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF2EN_SHIFT 2
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF3EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF3EN_SHIFT 3
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_EN_SHIFT 4
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_EN_SHIFT 5
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF6EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CF6EN_SHIFT 6
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_SE_EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_SE_EN_SHIFT 7
+ u8 flags3;
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_EN_SHIFT 0
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE2EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE2EN_SHIFT 1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE3EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE3EN_SHIFT 2
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE4EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE4EN_SHIFT 3
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE5EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE5EN_SHIFT 4
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE6EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE6EN_SHIFT 5
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE7EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE7EN_SHIFT 6
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE8EN_MASK 0x1
+#define E4_USTORM_RDMA_CONN_AG_CTX_RULE8EN_SHIFT 7
+ u8 byte2;
+ u8 byte3;
+ __le16 conn_dpi;
+ __le16 word1;
+ __le32 cq_cons;
+ __le32 cq_se_prod;
+ __le32 cq_prod;
+ __le32 reg3;
+ __le16 int_timeout;
+ __le16 word3;
+};
+
+struct e4_xstorm_roce_conn_ag_ctx {
u8 reserved0;
u8 state;
u8 flags0;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM0_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM0_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT1_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT1_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT2_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT2_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM3_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM3_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT4_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT4_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT5_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT5_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT6_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT6_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT7_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT7_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_EXIST_IN_QM0_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_EXIST_IN_QM0_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT1_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT1_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT2_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT2_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_EXIST_IN_QM3_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_EXIST_IN_QM3_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT4_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT4_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT5_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT5_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT6_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT6_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT7_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT7_SHIFT 7
u8 flags1;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT8_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT8_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT9_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT9_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT10_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT10_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT11_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT11_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT12_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT12_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MSTORM_FLUSH_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MSTORM_FLUSH_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT14_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT14_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_YSTORM_FLUSH_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_YSTORM_FLUSH_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT8_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT8_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT9_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT9_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT10_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT10_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT11_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT11_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT12_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT12_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_MSEM_FLUSH_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_MSEM_FLUSH_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_MSDM_FLUSH_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_MSDM_FLUSH_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_YSTORM_FLUSH_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_YSTORM_FLUSH_SHIFT 7
u8 flags2;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF0_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF0_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF1_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF1_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF2_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF2_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF3_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF3_SHIFT 6
u8 flags3;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF4_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF4_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF5_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF5_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF6_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF6_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 6
u8 flags4;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF8_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF8_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF9_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF9_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF10_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF10_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF11_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF11_SHIFT 6
u8 flags5;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF12_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF12_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF13_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF13_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF14_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF14_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF15_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF15_SHIFT 6
u8 flags6;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF16_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF16_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF17_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF17_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF18_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF18_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF19_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF19_SHIFT 6
u8 flags7;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0EN_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1EN_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF20_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF20_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF21_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF21_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_SLOW_PATH_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_SLOW_PATH_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF0EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF0EN_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF1EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF1EN_SHIFT 7
u8 flags8;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2EN_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3EN_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4EN_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5EN_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6EN_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_EN_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8EN_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9EN_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF2EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF2EN_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF3EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF3EN_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF4EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF4EN_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF5EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF5EN_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF6EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF6EN_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF8EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF8EN_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF9EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF9EN_SHIFT 7
u8 flags9;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10EN_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11EN_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12EN_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13EN_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14EN_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15EN_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16EN_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17EN_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF10EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF10EN_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF11EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF11EN_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF12EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF12EN_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF13EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF13EN_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF14EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF14EN_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF15EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF15EN_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF16EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF16EN_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF17EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF17EN_SHIFT 7
u8 flags10;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18EN_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19EN_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20EN_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21EN_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_EN_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23EN_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE0EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE0EN_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE1EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE1EN_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF18EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF18EN_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF19EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF19EN_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF20EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF20EN_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF21EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF21EN_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_SLOW_PATH_EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_SLOW_PATH_EN_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF23EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF23EN_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE0EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE0EN_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE1EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE1EN_SHIFT 7
u8 flags11;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE2EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE2EN_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE3EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE3EN_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE4EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE4EN_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE5EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE5EN_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE6EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE6EN_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE7EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE7EN_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED1_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED1_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE9EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE9EN_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE2EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE2EN_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE3EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE3EN_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE4EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE4EN_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE5EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE5EN_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE6EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE6EN_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE7EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE7EN_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED1_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED1_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE9EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE9EN_SHIFT 7
u8 flags12;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE10EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE10EN_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE11EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE11EN_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED2_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED2_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED3_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED3_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE14EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE14EN_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE15EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE15EN_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE16EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE16EN_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE17EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE17EN_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE10EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE10EN_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE11EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE11EN_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED2_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED2_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED3_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED3_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE14EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE14EN_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE15EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE15EN_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE16EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE16EN_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE17EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE17EN_SHIFT 7
u8 flags13;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE18EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE18EN_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE19EN_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE19EN_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED4_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED4_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED5_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED5_SHIFT 3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED6_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED6_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED7_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED7_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED8_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED8_SHIFT 6
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED9_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED9_SHIFT 7
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE18EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE18EN_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE19EN_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RULE19EN_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED4_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED4_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED5_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED5_SHIFT 3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED6_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED6_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED7_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED7_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED8_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED8_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED9_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_A0_RESERVED9_SHIFT 7
u8 flags14;
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MIGRATION_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MIGRATION_SHIFT 0
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT17_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT17_SHIFT 1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_DPM_PORT_NUM_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_DPM_PORT_NUM_SHIFT 2
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RESERVED_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RESERVED_SHIFT 4
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_ROCE_EDPM_ENABLE_MASK 0x1
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_ROCE_EDPM_ENABLE_SHIFT 5
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23_MASK 0x3
-#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23_SHIFT 6
+#define E4_XSTORM_ROCE_CONN_AG_CTX_MIGRATION_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_MIGRATION_SHIFT 0
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT17_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_BIT17_SHIFT 1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_DPM_PORT_NUM_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_DPM_PORT_NUM_SHIFT 2
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RESERVED_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_RESERVED_SHIFT 4
+#define E4_XSTORM_ROCE_CONN_AG_CTX_ROCE_EDPM_ENABLE_MASK 0x1
+#define E4_XSTORM_ROCE_CONN_AG_CTX_ROCE_EDPM_ENABLE_SHIFT 5
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF23_MASK 0x3
+#define E4_XSTORM_ROCE_CONN_AG_CTX_CF23_SHIFT 6
u8 byte2;
__le16 physical_q0;
__le16 word1;
__le32 reg2;
__le32 snd_nxt_psn;
__le32 reg4;
+ __le32 reg5;
+ __le32 reg6;
};
-struct e4_mstorm_rdma_conn_ag_ctx {
- u8 byte0;
- u8 byte1;
- u8 flags0;
-#define E4_MSTORM_RDMA_CONN_AG_CTX_BIT0_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_BIT0_SHIFT 0
-#define E4_MSTORM_RDMA_CONN_AG_CTX_BIT1_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_BIT1_SHIFT 1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF0_MASK 0x3
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF0_SHIFT 2
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF1_MASK 0x3
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF1_SHIFT 4
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF2_MASK 0x3
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF2_SHIFT 6
- u8 flags1;
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF0EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF0EN_SHIFT 0
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF1EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF1EN_SHIFT 1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF2EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_CF2EN_SHIFT 2
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE0EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE0EN_SHIFT 3
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE1EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE1EN_SHIFT 4
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE2EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE2EN_SHIFT 5
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE3EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE3EN_SHIFT 6
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE4EN_MASK 0x1
-#define E4_MSTORM_RDMA_CONN_AG_CTX_RULE4EN_SHIFT 7
- __le16 word0;
- __le16 word1;
- __le32 reg0;
- __le32 reg1;
-};
-
-struct e4_tstorm_rdma_conn_ag_ctx {
+struct e4_tstorm_roce_conn_ag_ctx {
u8 reserved0;
u8 byte1;
u8 flags0;
-#define E4_TSTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_SHIFT 0
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT1_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT1_SHIFT 1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT2_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT2_SHIFT 2
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT3_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT3_SHIFT 3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT4_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT4_SHIFT 4
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT5_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_BIT5_SHIFT 5
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF0_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF0_SHIFT 6
+#define E4_TSTORM_ROCE_CONN_AG_CTX_EXIST_IN_QM0_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_EXIST_IN_QM0_SHIFT 0
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT1_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT1_SHIFT 1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT2_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT2_SHIFT 2
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT3_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT3_SHIFT 3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT4_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT4_SHIFT 4
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT5_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_BIT5_SHIFT 5
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF0_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF0_SHIFT 6
u8 flags1;
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF1_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF1_SHIFT 0
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF2_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF2_SHIFT 2
-#define E4_TSTORM_RDMA_CONN_AG_CTX_TIMER_STOP_ALL_CF_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_TIMER_STOP_ALL_CF_SHIFT 4
-#define E4_TSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 6
+#define E4_TSTORM_ROCE_CONN_AG_CTX_MSTORM_FLUSH_CF_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_MSTORM_FLUSH_CF_SHIFT 0
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF2_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF2_SHIFT 2
+#define E4_TSTORM_ROCE_CONN_AG_CTX_TIMER_STOP_ALL_CF_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_TIMER_STOP_ALL_CF_SHIFT 4
+#define E4_TSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 6
u8 flags2;
-#define E4_TSTORM_RDMA_CONN_AG_CTX_MSTORM_FLUSH_CF_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_MSTORM_FLUSH_CF_SHIFT 0
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF6_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF6_SHIFT 2
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF7_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF7_SHIFT 4
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF8_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF8_SHIFT 6
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF5_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF5_SHIFT 0
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF6_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF6_SHIFT 2
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF7_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF7_SHIFT 4
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF8_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF8_SHIFT 6
u8 flags3;
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF9_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF9_SHIFT 0
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF10_MASK 0x3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF10_SHIFT 2
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF0EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF0EN_SHIFT 4
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF1EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF1EN_SHIFT 5
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF2EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF2EN_SHIFT 6
-#define E4_TSTORM_RDMA_CONN_AG_CTX_TIMER_STOP_ALL_CF_EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_TIMER_STOP_ALL_CF_EN_SHIFT 7
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF9_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF9_SHIFT 0
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF10_MASK 0x3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF10_SHIFT 2
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF0EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF0EN_SHIFT 4
+#define E4_TSTORM_ROCE_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_SHIFT 5
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF2EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF2EN_SHIFT 6
+#define E4_TSTORM_ROCE_CONN_AG_CTX_TIMER_STOP_ALL_CF_EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_TIMER_STOP_ALL_CF_EN_SHIFT 7
u8 flags4;
-#define E4_TSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 0
-#define E4_TSTORM_RDMA_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_SHIFT 1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF6EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF6EN_SHIFT 2
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF7EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF7EN_SHIFT 3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF8EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF8EN_SHIFT 4
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF9EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF9EN_SHIFT 5
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF10EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_CF10EN_SHIFT 6
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE0EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE0EN_SHIFT 7
+#define E4_TSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 0
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF5EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF5EN_SHIFT 1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF6EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF6EN_SHIFT 2
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF7EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF7EN_SHIFT 3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF8EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF8EN_SHIFT 4
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF9EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF9EN_SHIFT 5
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF10EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_CF10EN_SHIFT 6
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE0EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE0EN_SHIFT 7
u8 flags5;
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE1EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE1EN_SHIFT 0
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE2EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE2EN_SHIFT 1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE3EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE3EN_SHIFT 2
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE4EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE4EN_SHIFT 3
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE5EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE5EN_SHIFT 4
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE6EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE6EN_SHIFT 5
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE7EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE7EN_SHIFT 6
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE8EN_MASK 0x1
-#define E4_TSTORM_RDMA_CONN_AG_CTX_RULE8EN_SHIFT 7
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE1EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE1EN_SHIFT 0
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE2EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE2EN_SHIFT 1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE3EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE3EN_SHIFT 2
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE4EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE4EN_SHIFT 3
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE5EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE5EN_SHIFT 4
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE6EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE6EN_SHIFT 5
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE7EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE7EN_SHIFT 6
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE8EN_MASK 0x1
+#define E4_TSTORM_ROCE_CONN_AG_CTX_RULE8EN_SHIFT 7
__le32 reg0;
__le32 reg1;
__le32 reg2;
__le32 reg10;
};
-struct e4_tstorm_rdma_task_ag_ctx {
- u8 byte0;
- u8 byte1;
- __le16 word0;
- u8 flags0;
-#define E4_TSTORM_RDMA_TASK_AG_CTX_NIBBLE0_MASK 0xF
-#define E4_TSTORM_RDMA_TASK_AG_CTX_NIBBLE0_SHIFT 0
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT0_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT0_SHIFT 4
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT1_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT1_SHIFT 5
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT2_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT2_SHIFT 6
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT3_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT3_SHIFT 7
- u8 flags1;
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT4_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT4_SHIFT 0
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT5_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_BIT5_SHIFT 1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0_SHIFT 2
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1_SHIFT 4
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2_SHIFT 6
- u8 flags2;
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3_SHIFT 0
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4_SHIFT 2
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5_SHIFT 4
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6_SHIFT 6
- u8 flags3;
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7_MASK 0x3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7_SHIFT 0
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF0EN_SHIFT 2
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF1EN_SHIFT 3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF2EN_SHIFT 4
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF3EN_SHIFT 5
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF4EN_SHIFT 6
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF5EN_SHIFT 7
- u8 flags4;
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF6EN_SHIFT 0
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_CF7EN_SHIFT 1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE0EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE0EN_SHIFT 2
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE1EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE1EN_SHIFT 3
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE2EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE2EN_SHIFT 4
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE3EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE3EN_SHIFT 5
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE4EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE4EN_SHIFT 6
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE5EN_MASK 0x1
-#define E4_TSTORM_RDMA_TASK_AG_CTX_RULE5EN_SHIFT 7
- u8 byte2;
- __le16 word1;
- __le32 reg0;
- u8 byte3;
- u8 byte4;
- __le16 word2;
- __le16 word3;
- __le16 word4;
- __le32 reg1;
- __le32 reg2;
-};
-
-struct e4_ustorm_rdma_conn_ag_ctx {
- u8 reserved;
- u8 byte1;
- u8 flags0;
-#define E4_USTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_SHIFT 0
-#define E4_USTORM_RDMA_CONN_AG_CTX_BIT1_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_BIT1_SHIFT 1
-#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
-#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 2
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF1_MASK 0x3
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF1_SHIFT 4
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF2_MASK 0x3
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF2_SHIFT 6
- u8 flags1;
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF3_MASK 0x3
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF3_SHIFT 0
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_MASK 0x3
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_SHIFT 2
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_MASK 0x3
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_SHIFT 4
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF6_MASK 0x3
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF6_SHIFT 6
- u8 flags2;
-#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 0
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF1EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF1EN_SHIFT 1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF2EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF2EN_SHIFT 2
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF3EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF3EN_SHIFT 3
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_SE_CF_EN_SHIFT 4
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_ARM_CF_EN_SHIFT 5
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF6EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CF6EN_SHIFT 6
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_SE_EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_SE_EN_SHIFT 7
- u8 flags3;
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_CQ_EN_SHIFT 0
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE2EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE2EN_SHIFT 1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE3EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE3EN_SHIFT 2
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE4EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE4EN_SHIFT 3
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE5EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE5EN_SHIFT 4
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE6EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE6EN_SHIFT 5
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE7EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE7EN_SHIFT 6
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE8EN_MASK 0x1
-#define E4_USTORM_RDMA_CONN_AG_CTX_RULE8EN_SHIFT 7
- u8 byte2;
- u8 byte3;
- __le16 conn_dpi;
- __le16 word1;
- __le32 cq_cons;
- __le32 cq_se_prod;
- __le32 cq_prod;
- __le32 reg3;
- __le16 int_timeout;
- __le16 word3;
-};
-
-struct e4_xstorm_rdma_conn_ag_ctx {
- u8 reserved0;
- u8 state;
- u8 flags0;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM0_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT1_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT1_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT2_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT2_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM3_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_EXIST_IN_QM3_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT4_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT4_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT5_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT5_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT6_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT6_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT7_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT7_SHIFT 7
- u8 flags1;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT8_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT8_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT9_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT9_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT10_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT10_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT11_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT11_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT12_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT12_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_MSTORM_FLUSH_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_MSTORM_FLUSH_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT14_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT14_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_YSTORM_FLUSH_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_YSTORM_FLUSH_SHIFT 7
- u8 flags2;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF0_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF0_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF1_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF1_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF2_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF2_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF3_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF3_SHIFT 6
- u8 flags3;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF4_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF4_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF5_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF5_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF6_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF6_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 6
- u8 flags4;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF8_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF8_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF9_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF9_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF10_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF10_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF11_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF11_SHIFT 6
- u8 flags5;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF12_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF12_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF13_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF13_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF14_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF14_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF15_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF15_SHIFT 6
- u8 flags6;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF16_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF16_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF17_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF17_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF18_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF18_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF19_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF19_SHIFT 6
- u8 flags7;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF20_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF20_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF21_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF21_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_SLOW_PATH_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_SLOW_PATH_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF0EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF0EN_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF1EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF1EN_SHIFT 7
- u8 flags8;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF2EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF2EN_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF3EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF3EN_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF4EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF4EN_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF5EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF5EN_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF6EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF6EN_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF8EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF8EN_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF9EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF9EN_SHIFT 7
- u8 flags9;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF10EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF10EN_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF11EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF11EN_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF12EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF12EN_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF13EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF13EN_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF14EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF14EN_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF15EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF15EN_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF16EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF16EN_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF17EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF17EN_SHIFT 7
- u8 flags10;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF18EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF18EN_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF19EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF19EN_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF20EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF20EN_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF21EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF21EN_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_SLOW_PATH_EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_SLOW_PATH_EN_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF23EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF23EN_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE0EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE0EN_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE1EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE1EN_SHIFT 7
- u8 flags11;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE2EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE2EN_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE3EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE3EN_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE4EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE4EN_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE5EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE5EN_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE6EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE6EN_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE7EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE7EN_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED1_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED1_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE9EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE9EN_SHIFT 7
- u8 flags12;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE10EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE10EN_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE11EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE11EN_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED2_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED2_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED3_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED3_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE14EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE14EN_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE15EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE15EN_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE16EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE16EN_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE17EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE17EN_SHIFT 7
- u8 flags13;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE18EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE18EN_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE19EN_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RULE19EN_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED4_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED4_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED5_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED5_SHIFT 3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED6_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED6_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED7_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED7_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED8_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED8_SHIFT 6
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED9_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_A0_RESERVED9_SHIFT 7
- u8 flags14;
-#define E4_XSTORM_RDMA_CONN_AG_CTX_MIGRATION_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_MIGRATION_SHIFT 0
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT17_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_BIT17_SHIFT 1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_DPM_PORT_NUM_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_DPM_PORT_NUM_SHIFT 2
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RESERVED_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_RESERVED_SHIFT 4
-#define E4_XSTORM_RDMA_CONN_AG_CTX_ROCE_EDPM_ENABLE_MASK 0x1
-#define E4_XSTORM_RDMA_CONN_AG_CTX_ROCE_EDPM_ENABLE_SHIFT 5
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF23_MASK 0x3
-#define E4_XSTORM_RDMA_CONN_AG_CTX_CF23_SHIFT 6
- u8 byte2;
- __le16 physical_q0;
- __le16 word1;
- __le16 word2;
- __le16 word3;
- __le16 word4;
- __le16 word5;
- __le16 conn_dpi;
- u8 byte3;
- u8 byte4;
- u8 byte5;
- u8 byte6;
- __le32 reg0;
- __le32 reg1;
- __le32 reg2;
- __le32 snd_nxt_psn;
- __le32 reg4;
- __le32 reg5;
- __le32 reg6;
-};
-
-struct e4_ystorm_rdma_conn_ag_ctx {
- u8 byte0;
- u8 byte1;
- u8 flags0;
-#define E4_YSTORM_RDMA_CONN_AG_CTX_BIT0_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_BIT0_SHIFT 0
-#define E4_YSTORM_RDMA_CONN_AG_CTX_BIT1_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_BIT1_SHIFT 1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF0_MASK 0x3
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF0_SHIFT 2
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF1_MASK 0x3
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF1_SHIFT 4
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF2_MASK 0x3
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF2_SHIFT 6
- u8 flags1;
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF0EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF0EN_SHIFT 0
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF1EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF1EN_SHIFT 1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF2EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_CF2EN_SHIFT 2
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE0EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE0EN_SHIFT 3
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE1EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE1EN_SHIFT 4
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE2EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE2EN_SHIFT 5
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE3EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE3EN_SHIFT 6
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE4EN_MASK 0x1
-#define E4_YSTORM_RDMA_CONN_AG_CTX_RULE4EN_SHIFT 7
- u8 byte2;
- u8 byte3;
- __le16 word0;
- __le32 reg0;
- __le32 reg1;
- __le16 word1;
- __le16 word2;
- __le16 word3;
- __le16 word4;
- __le32 reg2;
- __le32 reg3;
-};
-
/* The roce storm context of Ystorm */
struct ystorm_roce_conn_st_ctx {
struct regpair temp[2];
struct regpair ystorm_st_padding[2];
struct pstorm_roce_conn_st_ctx pstorm_st_context;
struct xstorm_roce_conn_st_ctx xstorm_st_context;
- struct regpair xstorm_st_padding[2];
- struct e4_xstorm_rdma_conn_ag_ctx xstorm_ag_context;
- struct e4_tstorm_rdma_conn_ag_ctx tstorm_ag_context;
+ struct e4_xstorm_roce_conn_ag_ctx xstorm_ag_context;
+ struct e4_tstorm_roce_conn_ag_ctx tstorm_ag_context;
struct timers_context timer_context;
struct e4_ustorm_rdma_conn_ag_ctx ustorm_ag_context;
struct tstorm_roce_conn_st_ctx tstorm_st_context;
+ struct regpair tstorm_st_padding[2];
struct mstorm_roce_conn_st_ctx mstorm_st_context;
+ struct regpair mstorm_st_padding[2];
struct ustorm_roce_conn_st_ctx ustorm_st_context;
- struct regpair ustorm_st_padding[2];
};
/* roce create qp requester ramrod data */
#define ROCE_CREATE_QP_REQ_RAMROD_DATA_SIGNALED_COMP_SHIFT 3
#define ROCE_CREATE_QP_REQ_RAMROD_DATA_PRI_MASK 0x7
#define ROCE_CREATE_QP_REQ_RAMROD_DATA_PRI_SHIFT 4
-#define ROCE_CREATE_QP_REQ_RAMROD_DATA_RESERVED_MASK 0x1
-#define ROCE_CREATE_QP_REQ_RAMROD_DATA_RESERVED_SHIFT 7
+#define ROCE_CREATE_QP_REQ_RAMROD_DATA_XRC_FLAG_MASK 0x1
+#define ROCE_CREATE_QP_REQ_RAMROD_DATA_XRC_FLAG_SHIFT 7
#define ROCE_CREATE_QP_REQ_RAMROD_DATA_ERR_RETRY_CNT_MASK 0xF
#define ROCE_CREATE_QP_REQ_RAMROD_DATA_ERR_RETRY_CNT_SHIFT 8
#define ROCE_CREATE_QP_REQ_RAMROD_DATA_RNR_NAK_CNT_MASK 0xF
__le16 udp_src_port;
__le32 src_gid[4];
__le32 dst_gid[4];
+ __le32 cq_cid;
struct regpair qp_handle_for_cqe;
struct regpair qp_handle_for_async;
u8 stats_counter_id;
u8 reserved3[7];
- __le32 cq_cid;
__le16 regular_latency_phy_queue;
__le16 dpi;
};
/* roce create qp responder ramrod data */
struct roce_create_qp_resp_ramrod_data {
- __le16 flags;
+ __le32 flags;
#define ROCE_CREATE_QP_RESP_RAMROD_DATA_ROCE_FLAVOR_MASK 0x3
#define ROCE_CREATE_QP_RESP_RAMROD_DATA_ROCE_FLAVOR_SHIFT 0
#define ROCE_CREATE_QP_RESP_RAMROD_DATA_RDMA_RD_EN_MASK 0x1
#define ROCE_CREATE_QP_RESP_RAMROD_DATA_PRI_SHIFT 8
#define ROCE_CREATE_QP_RESP_RAMROD_DATA_MIN_RNR_NAK_TIMER_MASK 0x1F
#define ROCE_CREATE_QP_RESP_RAMROD_DATA_MIN_RNR_NAK_TIMER_SHIFT 11
+#define ROCE_CREATE_QP_RESP_RAMROD_DATA_XRC_FLAG_MASK 0x1
+#define ROCE_CREATE_QP_RESP_RAMROD_DATA_XRC_FLAG_SHIFT 16
+#define ROCE_CREATE_QP_RESP_RAMROD_DATA_RESERVED_MASK 0x7FFF
+#define ROCE_CREATE_QP_RESP_RAMROD_DATA_RESERVED_SHIFT 17
+ __le16 xrc_domain;
u8 max_ird;
u8 traffic_class;
u8 hop_limit;
struct regpair qp_handle_for_cqe;
struct regpair qp_handle_for_async;
__le16 low_latency_phy_queue;
- u8 reserved2[6];
+ u8 reserved2[2];
__le32 cq_cid;
__le16 regular_latency_phy_queue;
__le16 dpi;
struct regpair output_params_addr;
};
-/* ROCE ramrod command IDs */
-enum roce_ramrod_cmd_id {
- ROCE_RAMROD_CREATE_QP = 11,
- ROCE_RAMROD_MODIFY_QP,
- ROCE_RAMROD_QUERY_QP,
- ROCE_RAMROD_DESTROY_QP,
- ROCE_RAMROD_CREATE_UD_QP,
- ROCE_RAMROD_DESTROY_UD_QP,
- MAX_ROCE_RAMROD_CMD_ID
+/* ROCE ramrod command IDs */
+enum roce_ramrod_cmd_id {
+ ROCE_RAMROD_CREATE_QP = 11,
+ ROCE_RAMROD_MODIFY_QP,
+ ROCE_RAMROD_QUERY_QP,
+ ROCE_RAMROD_DESTROY_QP,
+ ROCE_RAMROD_CREATE_UD_QP,
+ ROCE_RAMROD_DESTROY_UD_QP,
+ MAX_ROCE_RAMROD_CMD_ID
+};
+
+struct e4_xstorm_roce_conn_ag_ctx_dq_ext_ld_part {
+ u8 reserved0;
+ u8 state;
+ u8 flags0;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM0_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM0_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT1_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT1_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT2_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT2_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM3_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_EXIST_IN_QM3_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT4_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT4_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT5_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT5_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT6_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT6_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT7_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT7_SHIFT 7
+ u8 flags1;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT8_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT8_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT9_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT9_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT10_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT10_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT11_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT11_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT12_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT12_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MSEM_FLUSH_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MSEM_FLUSH_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MSDM_FLUSH_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MSDM_FLUSH_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_YSTORM_FLUSH_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_YSTORM_FLUSH_SHIFT 7
+ u8 flags2;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3_SHIFT 6
+ u8 flags3;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_SHIFT 6
+ u8 flags4;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11_SHIFT 6
+ u8 flags5;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15_SHIFT 6
+ u8 flags6;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19_SHIFT 6
+ u8 flags7;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF0EN_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF1EN_SHIFT 7
+ u8 flags8;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF2EN_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF3EN_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF4EN_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF5EN_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF6EN_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_FLUSH_Q0_CF_EN_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF8EN_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF9EN_SHIFT 7
+ u8 flags9;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF10EN_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF11EN_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF12EN_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF13EN_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF14EN_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF15EN_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF16EN_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF17EN_SHIFT 7
+ u8 flags10;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF18EN_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF19EN_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF20EN_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF21EN_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_SLOW_PATH_EN_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23EN_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE0EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE0EN_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE1EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE1EN_SHIFT 7
+ u8 flags11;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE2EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE2EN_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE3EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE3EN_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE4EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE4EN_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE5EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE5EN_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE6EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE6EN_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE7EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE7EN_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED1_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED1_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE9EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE9EN_SHIFT 7
+ u8 flags12;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE10EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE10EN_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE11EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE11EN_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED2_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED2_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED3_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED3_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE14EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE14EN_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE15EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE15EN_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE16EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE16EN_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE17EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE17EN_SHIFT 7
+ u8 flags13;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE18EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE18EN_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE19EN_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RULE19EN_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED4_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED4_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED5_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED5_SHIFT 3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED6_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED6_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED7_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED7_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED8_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED8_SHIFT 6
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED9_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_A0_RESERVED9_SHIFT 7
+ u8 flags14;
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MIGRATION_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_MIGRATION_SHIFT 0
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT17_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_BIT17_SHIFT 1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_DPM_PORT_NUM_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_DPM_PORT_NUM_SHIFT 2
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RESERVED_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_RESERVED_SHIFT 4
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_ROCE_EDPM_ENABLE_MASK 0x1
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_ROCE_EDPM_ENABLE_SHIFT 5
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23_MASK 0x3
+#define E4XSTORMROCECONNAGCTXDQEXTLDPART_CF23_SHIFT 6
+ u8 byte2;
+ __le16 physical_q0;
+ __le16 word1;
+ __le16 word2;
+ __le16 word3;
+ __le16 word4;
+ __le16 word5;
+ __le16 conn_dpi;
+ u8 byte3;
+ u8 byte4;
+ u8 byte5;
+ u8 byte6;
+ __le32 reg0;
+ __le32 reg1;
+ __le32 reg2;
+ __le32 snd_nxt_psn;
+ __le32 reg4;
+};
+
+struct e4_mstorm_roce_conn_ag_ctx {
+ u8 byte0;
+ u8 byte1;
+ u8 flags0;
+#define E4_MSTORM_ROCE_CONN_AG_CTX_BIT0_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_BIT0_SHIFT 0
+#define E4_MSTORM_ROCE_CONN_AG_CTX_BIT1_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_BIT1_SHIFT 1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF0_MASK 0x3
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF0_SHIFT 2
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF1_MASK 0x3
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF1_SHIFT 4
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF2_MASK 0x3
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF2_SHIFT 6
+ u8 flags1;
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF0EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF0EN_SHIFT 0
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF1EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF1EN_SHIFT 1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF2EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_CF2EN_SHIFT 2
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE0EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE0EN_SHIFT 3
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE1EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE1EN_SHIFT 4
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE2EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE2EN_SHIFT 5
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE3EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE3EN_SHIFT 6
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE4EN_MASK 0x1
+#define E4_MSTORM_ROCE_CONN_AG_CTX_RULE4EN_SHIFT 7
+ __le16 word0;
+ __le16 word1;
+ __le32 reg0;
+ __le32 reg1;
};
struct e4_mstorm_roce_req_conn_ag_ctx {
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TIMER_CF_MASK 0x3
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TIMER_CF_SHIFT 6
u8 flags1;
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_CF1_MASK 0x3
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_CF1_SHIFT 0
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_MASK 0x3
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_SHIFT 0
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_SQ_CF_MASK 0x3
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_SQ_CF_SHIFT 2
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TIMER_STOP_ALL_CF_MASK 0x3
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 6
u8 flags2;
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_MASK 0x3
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_SHIFT 0
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FORCE_COMP_CF_MASK 0x3
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FORCE_COMP_CF_SHIFT 0
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_SET_TIMER_CF_MASK 0x3
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_SET_TIMER_CF_SHIFT 2
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TX_ASYNC_ERROR_CF_MASK 0x3
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_SQ_DRAIN_COMPLETED_CF_SHIFT 2
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TIMER_CF_EN_MASK 0x1
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TIMER_CF_EN_SHIFT 4
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_CF1EN_MASK 0x1
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_CF1EN_SHIFT 5
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_MASK 0x1
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_SHIFT 5
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_SQ_CF_EN_MASK 0x1
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_SQ_CF_EN_SHIFT 6
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TIMER_STOP_ALL_CF_EN_MASK 0x1
u8 flags4;
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 0
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_MASK 0x1
-#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_SHIFT 1
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FORCE_COMP_CF_EN_MASK 0x1
+#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_FORCE_COMP_CF_EN_SHIFT 1
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_SET_TIMER_CF_EN_MASK 0x1
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_SET_TIMER_CF_EN_SHIFT 2
#define E4_TSTORM_ROCE_REQ_CONN_AG_CTX_TX_ASYNC_ERROR_CF_EN_MASK 0x1
u8 byte5;
__le16 snd_sq_cons;
__le16 conn_dpi;
- __le16 word3;
+ __le16 force_comp_cons;
__le32 reg9;
__le32 reg10;
};
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF0_MASK 0x3
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF0_SHIFT 6
u8 flags1;
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_MASK 0x3
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_SHIFT 0
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_MASK 0x3
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_SHIFT 0
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_TX_ERROR_CF_MASK 0x3
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_TX_ERROR_CF_SHIFT 2
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF3_MASK 0x3
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 6
u8 flags2;
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_MASK 0x3
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_SHIFT 0
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_MASK 0x3
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_SHIFT 0
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF6_MASK 0x3
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF6_SHIFT 2
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF7_MASK 0x3
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF10_SHIFT 2
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF0EN_MASK 0x1
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF0EN_SHIFT 4
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_EN_MASK 0x1
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_EN_SHIFT 5
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_MASK 0x1
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_SHIFT 5
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_TX_ERROR_CF_EN_MASK 0x1
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_TX_ERROR_CF_EN_SHIFT 6
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF3EN_MASK 0x1
u8 flags4;
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 0
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_MASK 0x1
-#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_MSTORM_FLUSH_CF_EN_SHIFT 1
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_EN_MASK 0x1
+#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_RX_ERROR_CF_EN_SHIFT 1
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF6EN_MASK 0x1
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF6EN_SHIFT 2
#define E4_TSTORM_ROCE_RESP_CONN_AG_CTX_CF7EN_MASK 0x1
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_MASK 0x3
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_SHIFT 6
u8 flags4;
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF8_MASK 0x3
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF8_SHIFT 0
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF9_MASK 0x3
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF9_SHIFT 2
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_DIF_ERROR_CF_MASK 0x3
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_DIF_ERROR_CF_SHIFT 0
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_SCAN_SQ_FOR_COMP_CF_MASK 0x3
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_SCAN_SQ_FOR_COMP_CF_SHIFT 2
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF10_MASK 0x3
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF10_SHIFT 4
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF11_MASK 0x3
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_SND_RXMIT_CF_EN_SHIFT 4
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_EN_MASK 0x1
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_FLUSH_Q0_CF_EN_SHIFT 5
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF8EN_MASK 0x1
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF8EN_SHIFT 6
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF9EN_MASK 0x1
-#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF9EN_SHIFT 7
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_DIF_ERROR_CF_EN_MASK 0x1
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_DIF_ERROR_CF_EN_SHIFT 6
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_SCAN_SQ_FOR_COMP_CF_EN_MASK 0x1
+#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_SCAN_SQ_FOR_COMP_CF_EN_SHIFT 7
u8 flags9;
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF10EN_MASK 0x1
#define E4_XSTORM_ROCE_REQ_CONN_AG_CTX_CF10EN_SHIFT 0
__le16 sq_cmp_cons;
__le16 sq_cons;
__le16 sq_prod;
- __le16 word5;
+ __le16 dif_error_first_sq_cons;
__le16 conn_dpi;
- u8 byte3;
+ u8 dif_error_sge_index;
u8 byte4;
u8 byte5;
u8 byte6;
__le32 ssn;
__le32 snd_una_psn;
__le32 snd_nxt_psn;
- __le32 reg4;
+ __le32 dif_error_offset;
__le32 orq_cons_th;
__le32 orq_cons;
};
__le32 msn_and_syndrome;
};
+struct e4_ystorm_roce_conn_ag_ctx {
+ u8 byte0;
+ u8 byte1;
+ u8 flags0;
+#define E4_YSTORM_ROCE_CONN_AG_CTX_BIT0_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_BIT0_SHIFT 0
+#define E4_YSTORM_ROCE_CONN_AG_CTX_BIT1_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_BIT1_SHIFT 1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF0_MASK 0x3
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF0_SHIFT 2
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF1_MASK 0x3
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF1_SHIFT 4
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF2_MASK 0x3
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF2_SHIFT 6
+ u8 flags1;
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF0EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF0EN_SHIFT 0
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF1EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF1EN_SHIFT 1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF2EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_CF2EN_SHIFT 2
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE0EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE0EN_SHIFT 3
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE1EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE1EN_SHIFT 4
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE2EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE2EN_SHIFT 5
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE3EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE3EN_SHIFT 6
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE4EN_MASK 0x1
+#define E4_YSTORM_ROCE_CONN_AG_CTX_RULE4EN_SHIFT 7
+ u8 byte2;
+ u8 byte3;
+ __le16 word0;
+ __le32 reg0;
+ __le32 reg1;
+ __le16 word1;
+ __le16 word2;
+ __le16 word3;
+ __le16 word4;
+ __le32 reg2;
+ __le32 reg3;
+};
+
struct e4_ystorm_roce_req_conn_ag_ctx {
u8 byte0;
u8 byte1;
/* The iwarp storm context of Xstorm */
struct xstorm_iwarp_conn_st_ctx {
- __le32 reserved[44];
+ __le32 reserved[48];
};
struct e4_xstorm_iwarp_conn_ag_ctx {
#define E4_XSTORM_IWARP_CONN_AG_CTX_FLUSH_Q1_EN_SHIFT 3
#define E4_XSTORM_IWARP_CONN_AG_CTX_SLOW_PATH_EN_MASK 0x1
#define E4_XSTORM_IWARP_CONN_AG_CTX_SLOW_PATH_EN_SHIFT 4
-#define E4_XSTORM_IWARP_CONN_AG_CTX_CF23EN_MASK 0x1
-#define E4_XSTORM_IWARP_CONN_AG_CTX_CF23EN_SHIFT 5
+#define E4_XSTORM_IWARP_CONN_AG_CTX_SEND_TERMINATE_CF_EN_MASK 0x1
+#define E4_XSTORM_IWARP_CONN_AG_CTX_SEND_TERMINATE_CF_EN_SHIFT 5
#define E4_XSTORM_IWARP_CONN_AG_CTX_RULE0EN_MASK 0x1
#define E4_XSTORM_IWARP_CONN_AG_CTX_RULE0EN_SHIFT 6
#define E4_XSTORM_IWARP_CONN_AG_CTX_MORE_TO_SEND_RULE_EN_MASK 0x1
#define E4_XSTORM_IWARP_CONN_AG_CTX_E5_RESERVED2_SHIFT 4
#define E4_XSTORM_IWARP_CONN_AG_CTX_E5_RESERVED3_MASK 0x1
#define E4_XSTORM_IWARP_CONN_AG_CTX_E5_RESERVED3_SHIFT 5
-#define E4_XSTORM_IWARP_CONN_AG_CTX_CF23_MASK 0x3
-#define E4_XSTORM_IWARP_CONN_AG_CTX_CF23_SHIFT 6
+#define E4_XSTORM_IWARP_CONN_AG_CTX_SEND_TERMINATE_CF_MASK 0x3
+#define E4_XSTORM_IWARP_CONN_AG_CTX_SEND_TERMINATE_CF_SHIFT 6
u8 byte2;
__le16 physical_q0;
__le16 physical_q1;
__le32 reg2;
__le32 more_to_send_seq;
__le32 reg4;
- __le32 rewinded_snd_max;
+ __le32 rewinded_snd_max_or_term_opcode;
__le32 rd_msn;
__le16 irq_prod_via_msdm;
__le16 irq_cons;
__le32 orq_cons;
__le32 orq_cons_th;
u8 byte7;
- u8 max_ord;
u8 wqe_data_pad_bytes;
+ u8 max_ord;
u8 former_hq_prod;
u8 irq_prod_via_msem;
u8 byte12;
#define E4_TSTORM_IWARP_CONN_AG_CTX_BIT1_SHIFT 1
#define E4_TSTORM_IWARP_CONN_AG_CTX_BIT2_MASK 0x1
#define E4_TSTORM_IWARP_CONN_AG_CTX_BIT2_SHIFT 2
-#define E4_TSTORM_IWARP_CONN_AG_CTX_MSTORM_FLUSH_MASK 0x1
-#define E4_TSTORM_IWARP_CONN_AG_CTX_MSTORM_FLUSH_SHIFT 3
+#define E4_TSTORM_IWARP_CONN_AG_CTX_MSTORM_FLUSH_OR_TERMINATE_SENT_MASK 0x1
+#define E4_TSTORM_IWARP_CONN_AG_CTX_MSTORM_FLUSH_OR_TERMINATE_SENT_SHIFT 3
#define E4_TSTORM_IWARP_CONN_AG_CTX_BIT4_MASK 0x1
#define E4_TSTORM_IWARP_CONN_AG_CTX_BIT4_SHIFT 4
#define E4_TSTORM_IWARP_CONN_AG_CTX_CACHED_ORQ_MASK 0x1
struct pstorm_iwarp_conn_st_ctx pstorm_st_context;
struct regpair pstorm_st_padding[2];
struct xstorm_iwarp_conn_st_ctx xstorm_st_context;
- struct regpair xstorm_st_padding[2];
struct e4_xstorm_iwarp_conn_ag_ctx xstorm_ag_context;
struct e4_tstorm_iwarp_conn_ag_ctx tstorm_ag_context;
struct timers_context timer_context;
#define IWARP_CREATE_QP_RAMROD_DATA_ATOMIC_EN_SHIFT 4
#define IWARP_CREATE_QP_RAMROD_DATA_SRQ_FLG_MASK 0x1
#define IWARP_CREATE_QP_RAMROD_DATA_SRQ_FLG_SHIFT 5
-#define IWARP_CREATE_QP_RAMROD_DATA_RESERVED0_MASK 0x3
-#define IWARP_CREATE_QP_RAMROD_DATA_RESERVED0_SHIFT 6
+#define IWARP_CREATE_QP_RAMROD_DATA_LOW_LATENCY_QUEUE_EN_MASK 0x1
+#define IWARP_CREATE_QP_RAMROD_DATA_LOW_LATENCY_QUEUE_EN_SHIFT 6
+#define IWARP_CREATE_QP_RAMROD_DATA_RESERVED0_MASK 0x1
+#define IWARP_CREATE_QP_RAMROD_DATA_RESERVED0_SHIFT 7
u8 reserved1;
__le16 pd;
__le16 sq_num_pages;
IWARP_EVENT_TYPE_QUERY_QP,
IWARP_EVENT_TYPE_MODIFY_QP,
IWARP_EVENT_TYPE_DESTROY_QP,
+ IWARP_EVENT_TYPE_ABORT_TCP_OFFLOAD,
MAX_IWARP_EQE_SYNC_OPCODE
};
IWARP_EXCEPTION_DETECTED_LLP_RESET,
IWARP_EXCEPTION_DETECTED_IRQ_FULL,
IWARP_EXCEPTION_DETECTED_RQ_EMPTY,
+ IWARP_EXCEPTION_DETECTED_SRQ_EMPTY,
+ IWARP_EXCEPTION_DETECTED_SRQ_LIMIT,
IWARP_EXCEPTION_DETECTED_LLP_TIMEOUT,
IWARP_EXCEPTION_DETECTED_REMOTE_PROTECTION_ERROR,
IWARP_EXCEPTION_DETECTED_CQ_OVERFLOW,
#define IWARP_MODIFY_QP_RAMROD_DATA_STATE_TRANS_EN_SHIFT 3
#define IWARP_MODIFY_QP_RAMROD_DATA_RDMA_OPS_EN_FLG_MASK 0x1
#define IWARP_MODIFY_QP_RAMROD_DATA_RDMA_OPS_EN_FLG_SHIFT 4
-#define IWARP_MODIFY_QP_RAMROD_DATA_RESERVED_MASK 0x7FF
-#define IWARP_MODIFY_QP_RAMROD_DATA_RESERVED_SHIFT 5
- __le32 reserved3[3];
- __le32 reserved4[8];
+#define IWARP_MODIFY_QP_RAMROD_DATA_PHYSICAL_QUEUE_FLG_MASK 0x1
+#define IWARP_MODIFY_QP_RAMROD_DATA_PHYSICAL_QUEUE_FLG_SHIFT 5
+#define IWARP_MODIFY_QP_RAMROD_DATA_RESERVED_MASK 0x3FF
+#define IWARP_MODIFY_QP_RAMROD_DATA_RESERVED_SHIFT 6
+ __le16 physical_q0;
+ __le16 physical_q1;
+ __le32 reserved1[10];
};
/* MPA params for Enhanced mode */
IWARP_RAMROD_CMD_ID_QUERY_QP,
IWARP_RAMROD_CMD_ID_MODIFY_QP,
IWARP_RAMROD_CMD_ID_DESTROY_QP,
+ IWARP_RAMROD_CMD_ID_ABORT_TCP_OFFLOAD,
MAX_IWARP_RAMROD_CMD_ID
};
#define E4_TSTORM_ISCSI_CONN_AG_CTX_RULE8EN_SHIFT 7
__le32 reg0;
__le32 reg1;
- __le32 reg2;
+ __le32 rx_tcp_checksum_err_cnt;
__le32 reg3;
__le32 reg4;
__le32 reg5;
#define DRV_MSG_CODE_VF_DISABLED_DONE 0xc0000000
#define DRV_MSG_CODE_CFG_VF_MSIX 0xc0010000
#define DRV_MSG_CODE_CFG_PF_VFS_MSIX 0xc0020000
+#define DRV_MSG_CODE_NVM_PUT_FILE_BEGIN 0x00010000
+#define DRV_MSG_CODE_NVM_PUT_FILE_DATA 0x00020000
#define DRV_MSG_CODE_NVM_GET_FILE_ATT 0x00030000
#define DRV_MSG_CODE_NVM_READ_NVRAM 0x00050000
+#define DRV_MSG_CODE_NVM_WRITE_NVRAM 0x00060000
#define DRV_MSG_CODE_MCP_RESET 0x00090000
#define DRV_MSG_CODE_SET_VERSION 0x000f0000
#define DRV_MSG_CODE_MCP_HALT 0x00100000
#define DRV_MSG_CODE_FEATURE_SUPPORT 0x00300000
#define DRV_MSG_CODE_GET_MFW_FEATURE_SUPPORT 0x00310000
-
#define DRV_MSG_SEQ_NUMBER_MASK 0x0000ffff
u32 drv_mb_param;
#define FW_MSG_CODE_DRV_CFG_VF_MSIX_DONE 0xb0010000
#define FW_MSG_CODE_NVM_OK 0x00010000
+#define FW_MSG_CODE_NVM_PUT_FILE_FINISH_OK 0x00400000
+#define FW_MSG_CODE_PHY_OK 0x00110000
#define FW_MSG_CODE_OK 0x00160000
+#define FW_MSG_CODE_ERROR 0x00170000
#define FW_MSG_CODE_OS_WOL_SUPPORTED 0x00800000
#define FW_MSG_CODE_OS_WOL_NOT_SUPPORTED 0x00810000
u16 *p_first_tx_pq_id;
ext_voq = qed_get_ext_voq(p_hwfn,
- p_params->port_id,
+ pq_params[i].port_id,
tc_id,
p_params->max_phys_tcs_per_port);
is_vf_pq = (i >= p_params->num_pf_pqs);
- rl_valid = pq_params[i].rl_valid &&
- pq_params[i].vport_id < max_qm_global_rls;
+ rl_valid = pq_params[i].rl_valid > 0;
/* Update first Tx PQ of VPORT/TC */
vport_id_in_pf = pq_params[i].vport_id - p_params->start_vport;
}
/* Check RL ID */
- if (pq_params[i].rl_valid && pq_params[i].vport_id >=
- max_qm_global_rls)
+ if (rl_valid && pq_params[i].vport_id >= max_qm_global_rls) {
DP_NOTICE(p_hwfn,
"Invalid VPORT ID for rate limiter configuration\n");
+ rl_valid = false;
+ }
/* Prepare PQ map entry */
QM_INIT_TX_PQ_MAP(p_hwfn,
pq_info = PQ_INFO_ELEMENT(*p_first_tx_pq_id,
p_params->pf_id,
tc_id,
- p_params->port_id,
+ pq_params[i].port_id,
rl_valid ? 1 : 0,
rl_valid ?
pq_params[i].vport_id : 0);
* Return -1 on error.
*/
static int qed_pf_wfq_rt_init(struct qed_hwfn *p_hwfn,
+
struct qed_qm_pf_rt_init_params *p_params)
{
u16 num_tx_pqs = p_params->num_pf_pqs + p_params->num_vf_pqs;
for (i = 0; i < num_tx_pqs; i++) {
ext_voq = qed_get_ext_voq(p_hwfn,
- p_params->port_id,
+ pq_params[i].port_id,
pq_params[i].tc_id,
p_params->max_phys_tcs_per_port);
crd_reg_offset =
*__p_var = (*__p_var & ~BIT(__offset)) | \
((enable) ? BIT(__offset) : 0); \
} while (0)
-#define PRS_ETH_TUNN_FIC_FORMAT -188897008
+#define PRS_ETH_TUNN_OUTPUT_FORMAT -188897008
+#define PRS_ETH_OUTPUT_FORMAT -46832
void qed_set_vxlan_dest_port(struct qed_hwfn *p_hwfn,
struct qed_ptt *p_ptt, u16 dest_port)
shift = PRS_REG_ENCAPSULATION_TYPE_EN_VXLAN_ENABLE_SHIFT;
SET_TUNNEL_TYPE_ENABLE_BIT(reg_val, shift, vxlan_enable);
qed_wr(p_hwfn, p_ptt, PRS_REG_ENCAPSULATION_TYPE_EN, reg_val);
- if (reg_val)
- qed_wr(p_hwfn,
- p_ptt,
- PRS_REG_OUTPUT_FORMAT_4_0_BB_K2,
- (u32)PRS_ETH_TUNN_FIC_FORMAT);
+ if (reg_val) {
+ reg_val =
+ qed_rd(p_hwfn, p_ptt, PRS_REG_OUTPUT_FORMAT_4_0_BB_K2);
+
+ /* Update output only if tunnel blocks not included. */
+ if (reg_val == (u32)PRS_ETH_OUTPUT_FORMAT)
+ qed_wr(p_hwfn, p_ptt, PRS_REG_OUTPUT_FORMAT_4_0_BB_K2,
+ (u32)PRS_ETH_TUNN_OUTPUT_FORMAT);
+ }
/* Update NIG register */
reg_val = qed_rd(p_hwfn, p_ptt, NIG_REG_ENC_TYPE_ENABLE);
shift = PRS_REG_ENCAPSULATION_TYPE_EN_IP_OVER_GRE_ENABLE_SHIFT;
SET_TUNNEL_TYPE_ENABLE_BIT(reg_val, shift, ip_gre_enable);
qed_wr(p_hwfn, p_ptt, PRS_REG_ENCAPSULATION_TYPE_EN, reg_val);
- if (reg_val)
- qed_wr(p_hwfn,
- p_ptt,
- PRS_REG_OUTPUT_FORMAT_4_0_BB_K2,
- (u32)PRS_ETH_TUNN_FIC_FORMAT);
+ if (reg_val) {
+ reg_val =
+ qed_rd(p_hwfn, p_ptt, PRS_REG_OUTPUT_FORMAT_4_0_BB_K2);
+
+ /* Update output only if tunnel blocks not included. */
+ if (reg_val == (u32)PRS_ETH_OUTPUT_FORMAT)
+ qed_wr(p_hwfn, p_ptt, PRS_REG_OUTPUT_FORMAT_4_0_BB_K2,
+ (u32)PRS_ETH_TUNN_OUTPUT_FORMAT);
+ }
/* Update NIG register */
reg_val = qed_rd(p_hwfn, p_ptt, NIG_REG_ENC_TYPE_ENABLE);
shift = PRS_REG_ENCAPSULATION_TYPE_EN_IP_OVER_GENEVE_ENABLE_SHIFT;
SET_TUNNEL_TYPE_ENABLE_BIT(reg_val, shift, ip_geneve_enable);
qed_wr(p_hwfn, p_ptt, PRS_REG_ENCAPSULATION_TYPE_EN, reg_val);
- if (reg_val)
- qed_wr(p_hwfn,
- p_ptt,
- PRS_REG_OUTPUT_FORMAT_4_0_BB_K2,
- (u32)PRS_ETH_TUNN_FIC_FORMAT);
+ if (reg_val) {
+ reg_val =
+ qed_rd(p_hwfn, p_ptt, PRS_REG_OUTPUT_FORMAT_4_0_BB_K2);
+
+ /* Update output only if tunnel blocks not included. */
+ if (reg_val == (u32)PRS_ETH_OUTPUT_FORMAT)
+ qed_wr(p_hwfn, p_ptt, PRS_REG_OUTPUT_FORMAT_4_0_BB_K2,
+ (u32)PRS_ETH_TUNN_OUTPUT_FORMAT);
+ }
/* Update NIG register */
qed_wr(p_hwfn, p_ptt, NIG_REG_NGE_ETH_ENABLE,
ip_geneve_enable ? 1 : 0);
}
+#define PRS_ETH_VXLAN_NO_L2_ENABLE_OFFSET 4
+#define PRS_ETH_VXLAN_NO_L2_OUTPUT_FORMAT -927094512
+
+void qed_set_vxlan_no_l2_enable(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt, bool enable)
+{
+ u32 reg_val, cfg_mask;
+
+ /* read PRS config register */
+ reg_val = qed_rd(p_hwfn, p_ptt, PRS_REG_MSG_INFO);
+
+ /* set VXLAN_NO_L2_ENABLE mask */
+ cfg_mask = BIT(PRS_ETH_VXLAN_NO_L2_ENABLE_OFFSET);
+
+ if (enable) {
+ /* set VXLAN_NO_L2_ENABLE flag */
+ reg_val |= cfg_mask;
+
+ /* update PRS FIC register */
+ qed_wr(p_hwfn,
+ p_ptt,
+ PRS_REG_OUTPUT_FORMAT_4_0_BB_K2,
+ (u32)PRS_ETH_VXLAN_NO_L2_OUTPUT_FORMAT);
+ } else {
+ /* clear VXLAN_NO_L2_ENABLE flag */
+ reg_val &= ~cfg_mask;
+ }
+
+ /* write PRS config register */
+ qed_wr(p_hwfn, p_ptt, PRS_REG_MSG_INFO, reg_val);
+}
+
#define T_ETH_PACKET_ACTION_GFT_EVENTID 23
#define PARSER_ETH_CONN_GFT_ACTION_CM_HDR 272
#define T_ETH_PACKET_MATCH_RFS_EVENTID 25
ram_line_lo = 0;
ram_line_hi = 0;
+ /* Tunnel type */
+ SET_FIELD(ram_line_lo, GFT_RAM_LINE_TUNNEL_DST_PORT, 1);
+ SET_FIELD(ram_line_lo, GFT_RAM_LINE_TUNNEL_OVER_IP_PROTOCOL, 1);
+
if (profile_type == GFT_PROFILE_TYPE_4_TUPLE) {
SET_FIELD(ram_line_hi, GFT_RAM_LINE_DST_IP, 1);
SET_FIELD(ram_line_hi, GFT_RAM_LINE_SRC_IP, 1);
SET_FIELD(ram_line_hi, GFT_RAM_LINE_OVER_IP_PROTOCOL, 1);
SET_FIELD(ram_line_lo, GFT_RAM_LINE_ETHERTYPE, 1);
SET_FIELD(ram_line_lo, GFT_RAM_LINE_DST_PORT, 1);
- } else if (profile_type == GFT_PROFILE_TYPE_IP_DST_PORT) {
+ } else if (profile_type == GFT_PROFILE_TYPE_IP_DST_ADDR) {
SET_FIELD(ram_line_hi, GFT_RAM_LINE_DST_IP, 1);
SET_FIELD(ram_line_lo, GFT_RAM_LINE_ETHERTYPE, 1);
+ } else if (profile_type == GFT_PROFILE_TYPE_IP_SRC_ADDR) {
+ SET_FIELD(ram_line_hi, GFT_RAM_LINE_SRC_IP, 1);
+ SET_FIELD(ram_line_lo, GFT_RAM_LINE_ETHERTYPE, 1);
+ } else if (profile_type == GFT_PROFILE_TYPE_TUNNEL_TYPE) {
+ SET_FIELD(ram_line_lo, GFT_RAM_LINE_TUNNEL_ETHERTYPE, 1);
}
qed_wr(p_hwfn,
iph = (struct iphdr *)((u8 *)(ethh) + eth_hlen);
if (eth_type == ETH_P_IP) {
+ if (iph->protocol != IPPROTO_TCP) {
+ DP_NOTICE(p_hwfn,
+ "Unexpected ip protocol on ll2 %x\n",
+ iph->protocol);
+ return -EINVAL;
+ }
+
cm_info->local_ip[0] = ntohl(iph->daddr);
cm_info->remote_ip[0] = ntohl(iph->saddr);
cm_info->ip_version = TCP_IPV4;
*payload_len = ntohs(iph->tot_len) - ip_hlen;
} else if (eth_type == ETH_P_IPV6) {
ip6h = (struct ipv6hdr *)iph;
+
+ if (ip6h->nexthdr != IPPROTO_TCP) {
+ DP_NOTICE(p_hwfn,
+ "Unexpected ip protocol on ll2 %x\n",
+ iph->protocol);
+ return -EINVAL;
+ }
+
for (i = 0; i < 4; i++) {
cm_info->local_ip[i] =
ntohl(ip6h->daddr.in6_u.u6_addr32[i]);
/* Missing lower byte is now available */
mpa_len = fpdu->fpdu_length | *mpa_data;
fpdu->fpdu_length = QED_IWARP_FPDU_LEN_WITH_PAD(mpa_len);
- fpdu->mpa_frag_len = fpdu->fpdu_length;
/* one byte of hdr */
+ fpdu->mpa_frag_len = 1;
fpdu->incomplete_bytes = fpdu->fpdu_length - 1;
DP_VERBOSE(p_hwfn,
QED_MSG_RDMA,
memset(&tx_pkt, 0, sizeof(tx_pkt));
tx_pkt.num_of_bds = 1;
- tx_pkt.vlan = data->vlan;
-
- if (GET_FIELD(data->parse_flags,
- PARSING_AND_ERR_FLAGS_TAG8021QEXIST))
- SET_FIELD(tx_pkt.bd_flags,
- CORE_TX_BD_DATA_VLAN_INSERTION, 1);
-
tx_pkt.l4_hdr_offset_w = (data->length.packet_length) >> 2;
tx_pkt.tx_dest = QED_LL2_TX_DEST_LB;
tx_pkt.first_frag = buf->data_phys_addr +
if (mode == QED_FILTER_CONFIG_MODE_5_TUPLE)
return GFT_PROFILE_TYPE_4_TUPLE;
if (mode == QED_FILTER_CONFIG_MODE_IP_DEST)
- return GFT_PROFILE_TYPE_IP_DST_PORT;
+ return GFT_PROFILE_TYPE_IP_DST_ADDR;
return GFT_PROFILE_TYPE_L4_DST_PORT;
}
}
}
-static u8 qed_ll2_convert_rx_parse_to_tx_flags(u16 parse_flags)
-{
- u8 bd_flags = 0;
-
- if (GET_FIELD(parse_flags, PARSING_AND_ERR_FLAGS_TAG8021QEXIST))
- SET_FIELD(bd_flags, CORE_TX_BD_DATA_VLAN_INSERTION, 1);
-
- return bd_flags;
-}
-
static int qed_ll2_lb_rxq_handler(struct qed_hwfn *p_hwfn,
struct qed_ll2_info *p_ll2_conn)
{
struct qed_ooo_buffer *p_buffer;
u16 l4_hdr_offset_w;
dma_addr_t first_frag;
- u16 parse_flags;
u8 bd_flags;
int rc;
first_frag = p_buffer->rx_buffer_phys_addr +
p_buffer->placement_offset;
- parse_flags = p_buffer->parse_flags;
- bd_flags = qed_ll2_convert_rx_parse_to_tx_flags(parse_flags);
SET_FIELD(bd_flags, CORE_TX_BD_DATA_FORCE_VLAN_MODE, 1);
SET_FIELD(bd_flags, CORE_TX_BD_DATA_L4_PROTOCOL, 1);
#include <linux/etherdevice.h>
#include <linux/vmalloc.h>
#include <linux/crash_dump.h>
+#include <linux/crc32.h>
#include <linux/qed/qed_if.h>
#include <linux/qed/qed_ll2_if.h>
return 0;
}
+static u32 qed_nvm_flash_image_access_crc(struct qed_dev *cdev,
+ struct qed_nvm_image_att *nvm_image,
+ u32 *crc)
+{
+ u8 *buf = NULL;
+ int rc, j;
+ u32 val;
+
+ /* Allocate a buffer for holding the nvram image */
+ buf = kzalloc(nvm_image->length, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ /* Read image into buffer */
+ rc = qed_mcp_nvm_read(cdev, nvm_image->start_addr,
+ buf, nvm_image->length);
+ if (rc) {
+ DP_ERR(cdev, "Failed reading image from nvm\n");
+ goto out;
+ }
+
+ /* Convert the buffer into big-endian format (excluding the
+ * closing 4 bytes of CRC).
+ */
+ for (j = 0; j < nvm_image->length - 4; j += 4) {
+ val = cpu_to_be32(*(u32 *)&buf[j]);
+ *(u32 *)&buf[j] = val;
+ }
+
+ /* Calc CRC for the "actual" image buffer, i.e. not including
+ * the last 4 CRC bytes.
+ */
+ *crc = (~cpu_to_be32(crc32(0xffffffff, buf, nvm_image->length - 4)));
+
+out:
+ kfree(buf);
+
+ return rc;
+}
+
+/* Binary file format -
+ * /----------------------------------------------------------------------\
+ * 0B | 0x4 [command index] |
+ * 4B | image_type | Options | Number of register settings |
+ * 8B | Value |
+ * 12B | Mask |
+ * 16B | Offset |
+ * \----------------------------------------------------------------------/
+ * There can be several Value-Mask-Offset sets as specified by 'Number of...'.
+ * Options - 0'b - Calculate & Update CRC for image
+ */
+static int qed_nvm_flash_image_access(struct qed_dev *cdev, const u8 **data,
+ bool *check_resp)
+{
+ struct qed_nvm_image_att nvm_image;
+ struct qed_hwfn *p_hwfn;
+ bool is_crc = false;
+ u32 image_type;
+ int rc = 0, i;
+ u16 len;
+
+ *data += 4;
+ image_type = **data;
+ p_hwfn = QED_LEADING_HWFN(cdev);
+ for (i = 0; i < p_hwfn->nvm_info.num_images; i++)
+ if (image_type == p_hwfn->nvm_info.image_att[i].image_type)
+ break;
+ if (i == p_hwfn->nvm_info.num_images) {
+ DP_ERR(cdev, "Failed to find nvram image of type %08x\n",
+ image_type);
+ return -ENOENT;
+ }
+
+ nvm_image.start_addr = p_hwfn->nvm_info.image_att[i].nvm_start_addr;
+ nvm_image.length = p_hwfn->nvm_info.image_att[i].len;
+
+ DP_VERBOSE(cdev, NETIF_MSG_DRV,
+ "Read image %02x; type = %08x; NVM [%08x,...,%08x]\n",
+ **data, image_type, nvm_image.start_addr,
+ nvm_image.start_addr + nvm_image.length - 1);
+ (*data)++;
+ is_crc = !!(**data & BIT(0));
+ (*data)++;
+ len = *((u16 *)*data);
+ *data += 2;
+ if (is_crc) {
+ u32 crc = 0;
+
+ rc = qed_nvm_flash_image_access_crc(cdev, &nvm_image, &crc);
+ if (rc) {
+ DP_ERR(cdev, "Failed calculating CRC, rc = %d\n", rc);
+ goto exit;
+ }
+
+ rc = qed_mcp_nvm_write(cdev, QED_NVM_WRITE_NVRAM,
+ (nvm_image.start_addr +
+ nvm_image.length - 4), (u8 *)&crc, 4);
+ if (rc)
+ DP_ERR(cdev, "Failed writing to %08x, rc = %d\n",
+ nvm_image.start_addr + nvm_image.length - 4, rc);
+ goto exit;
+ }
+
+ /* Iterate over the values for setting */
+ while (len) {
+ u32 offset, mask, value, cur_value;
+ u8 buf[4];
+
+ value = *((u32 *)*data);
+ *data += 4;
+ mask = *((u32 *)*data);
+ *data += 4;
+ offset = *((u32 *)*data);
+ *data += 4;
+
+ rc = qed_mcp_nvm_read(cdev, nvm_image.start_addr + offset, buf,
+ 4);
+ if (rc) {
+ DP_ERR(cdev, "Failed reading from %08x\n",
+ nvm_image.start_addr + offset);
+ goto exit;
+ }
+
+ cur_value = le32_to_cpu(*((__le32 *)buf));
+ DP_VERBOSE(cdev, NETIF_MSG_DRV,
+ "NVM %08x: %08x -> %08x [Value %08x Mask %08x]\n",
+ nvm_image.start_addr + offset, cur_value,
+ (cur_value & ~mask) | (value & mask), value, mask);
+ value = (value & mask) | (cur_value & ~mask);
+ rc = qed_mcp_nvm_write(cdev, QED_NVM_WRITE_NVRAM,
+ nvm_image.start_addr + offset,
+ (u8 *)&value, 4);
+ if (rc) {
+ DP_ERR(cdev, "Failed writing to %08x\n",
+ nvm_image.start_addr + offset);
+ goto exit;
+ }
+
+ len--;
+ }
+exit:
+ return rc;
+}
+
+/* Binary file format -
+ * /----------------------------------------------------------------------\
+ * 0B | 0x3 [command index] |
+ * 4B | b'0: check_response? | b'1-31 reserved |
+ * 8B | File-type | reserved |
+ * \----------------------------------------------------------------------/
+ * Start a new file of the provided type
+ */
+static int qed_nvm_flash_image_file_start(struct qed_dev *cdev,
+ const u8 **data, bool *check_resp)
+{
+ int rc;
+
+ *data += 4;
+ *check_resp = !!(**data & BIT(0));
+ *data += 4;
+
+ DP_VERBOSE(cdev, NETIF_MSG_DRV,
+ "About to start a new file of type %02x\n", **data);
+ rc = qed_mcp_nvm_put_file_begin(cdev, **data);
+ *data += 4;
+
+ return rc;
+}
+
+/* Binary file format -
+ * /----------------------------------------------------------------------\
+ * 0B | 0x2 [command index] |
+ * 4B | Length in bytes |
+ * 8B | b'0: check_response? | b'1-31 reserved |
+ * 12B | Offset in bytes |
+ * 16B | Data ... |
+ * \----------------------------------------------------------------------/
+ * Write data as part of a file that was previously started. Data should be
+ * of length equal to that provided in the message
+ */
+static int qed_nvm_flash_image_file_data(struct qed_dev *cdev,
+ const u8 **data, bool *check_resp)
+{
+ u32 offset, len;
+ int rc;
+
+ *data += 4;
+ len = *((u32 *)(*data));
+ *data += 4;
+ *check_resp = !!(**data & BIT(0));
+ *data += 4;
+ offset = *((u32 *)(*data));
+ *data += 4;
+
+ DP_VERBOSE(cdev, NETIF_MSG_DRV,
+ "About to write File-data: %08x bytes to offset %08x\n",
+ len, offset);
+
+ rc = qed_mcp_nvm_write(cdev, QED_PUT_FILE_DATA, offset,
+ (char *)(*data), len);
+ *data += len;
+
+ return rc;
+}
+
+/* Binary file format [General header] -
+ * /----------------------------------------------------------------------\
+ * 0B | QED_NVM_SIGNATURE |
+ * 4B | Length in bytes |
+ * 8B | Highest command in this batchfile | Reserved |
+ * \----------------------------------------------------------------------/
+ */
+static int qed_nvm_flash_image_validate(struct qed_dev *cdev,
+ const struct firmware *image,
+ const u8 **data)
+{
+ u32 signature, len;
+
+ /* Check minimum size */
+ if (image->size < 12) {
+ DP_ERR(cdev, "Image is too short [%08x]\n", (u32)image->size);
+ return -EINVAL;
+ }
+
+ /* Check signature */
+ signature = *((u32 *)(*data));
+ if (signature != QED_NVM_SIGNATURE) {
+ DP_ERR(cdev, "Wrong signature '%08x'\n", signature);
+ return -EINVAL;
+ }
+
+ *data += 4;
+ /* Validate internal size equals the image-size */
+ len = *((u32 *)(*data));
+ if (len != image->size) {
+ DP_ERR(cdev, "Size mismatch: internal = %08x image = %08x\n",
+ len, (u32)image->size);
+ return -EINVAL;
+ }
+
+ *data += 4;
+ /* Make sure driver familiar with all commands necessary for this */
+ if (*((u16 *)(*data)) >= QED_NVM_FLASH_CMD_NVM_MAX) {
+ DP_ERR(cdev, "File contains unsupported commands [Need %04x]\n",
+ *((u16 *)(*data)));
+ return -EINVAL;
+ }
+
+ *data += 4;
+
+ return 0;
+}
+
+static int qed_nvm_flash(struct qed_dev *cdev, const char *name)
+{
+ const struct firmware *image;
+ const u8 *data, *data_end;
+ u32 cmd_type;
+ int rc;
+
+ rc = request_firmware(&image, name, &cdev->pdev->dev);
+ if (rc) {
+ DP_ERR(cdev, "Failed to find '%s'\n", name);
+ return rc;
+ }
+
+ DP_VERBOSE(cdev, NETIF_MSG_DRV,
+ "Flashing '%s' - firmware's data at %p, size is %08x\n",
+ name, image->data, (u32)image->size);
+ data = image->data;
+ data_end = data + image->size;
+
+ rc = qed_nvm_flash_image_validate(cdev, image, &data);
+ if (rc)
+ goto exit;
+
+ while (data < data_end) {
+ bool check_resp = false;
+
+ /* Parse the actual command */
+ cmd_type = *((u32 *)data);
+ switch (cmd_type) {
+ case QED_NVM_FLASH_CMD_FILE_DATA:
+ rc = qed_nvm_flash_image_file_data(cdev, &data,
+ &check_resp);
+ break;
+ case QED_NVM_FLASH_CMD_FILE_START:
+ rc = qed_nvm_flash_image_file_start(cdev, &data,
+ &check_resp);
+ break;
+ case QED_NVM_FLASH_CMD_NVM_CHANGE:
+ rc = qed_nvm_flash_image_access(cdev, &data,
+ &check_resp);
+ break;
+ default:
+ DP_ERR(cdev, "Unknown command %08x\n", cmd_type);
+ rc = -EINVAL;
+ goto exit;
+ }
+
+ if (rc) {
+ DP_ERR(cdev, "Command %08x failed\n", cmd_type);
+ goto exit;
+ }
+
+ /* Check response if needed */
+ if (check_resp) {
+ u32 mcp_response = 0;
+
+ if (qed_mcp_nvm_resp(cdev, (u8 *)&mcp_response)) {
+ DP_ERR(cdev, "Failed getting MCP response\n");
+ rc = -EINVAL;
+ goto exit;
+ }
+
+ switch (mcp_response & FW_MSG_CODE_MASK) {
+ case FW_MSG_CODE_OK:
+ case FW_MSG_CODE_NVM_OK:
+ case FW_MSG_CODE_NVM_PUT_FILE_FINISH_OK:
+ case FW_MSG_CODE_PHY_OK:
+ break;
+ default:
+ DP_ERR(cdev, "MFW returns error: %08x\n",
+ mcp_response);
+ rc = -EINVAL;
+ goto exit;
+ }
+ }
+ }
+
+exit:
+ release_firmware(image);
+
+ return rc;
+}
+
static int qed_nvm_get_image(struct qed_dev *cdev, enum qed_nvm_images type,
u8 *buf, u16 len)
{
.dbg_all_data_size = &qed_dbg_all_data_size,
.chain_alloc = &qed_chain_alloc,
.chain_free = &qed_chain_free,
+ .nvm_flash = &qed_nvm_flash,
.nvm_get_image = &qed_nvm_get_image,
.set_coalesce = &qed_set_coalesce,
.set_led = &qed_set_led,
return 0;
}
+int qed_mcp_nvm_wr_cmd(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt,
+ u32 cmd,
+ u32 param,
+ u32 *o_mcp_resp,
+ u32 *o_mcp_param, u32 i_txn_size, u32 *i_buf)
+{
+ struct qed_mcp_mb_params mb_params;
+ int rc;
+
+ memset(&mb_params, 0, sizeof(mb_params));
+ mb_params.cmd = cmd;
+ mb_params.param = param;
+ mb_params.p_data_src = i_buf;
+ mb_params.data_src_size = (u8)i_txn_size;
+ rc = qed_mcp_cmd_and_union(p_hwfn, p_ptt, &mb_params);
+ if (rc)
+ return rc;
+
+ *o_mcp_resp = mb_params.mcp_resp;
+ *o_mcp_param = mb_params.mcp_param;
+
+ return 0;
+}
+
int qed_mcp_nvm_rd_cmd(struct qed_hwfn *p_hwfn,
struct qed_ptt *p_ptt,
u32 cmd,
return rc;
}
+int qed_mcp_nvm_resp(struct qed_dev *cdev, u8 *p_buf)
+{
+ struct qed_hwfn *p_hwfn = QED_LEADING_HWFN(cdev);
+ struct qed_ptt *p_ptt;
+
+ p_ptt = qed_ptt_acquire(p_hwfn);
+ if (!p_ptt)
+ return -EBUSY;
+
+ memcpy(p_buf, &cdev->mcp_nvm_resp, sizeof(cdev->mcp_nvm_resp));
+ qed_ptt_release(p_hwfn, p_ptt);
+
+ return 0;
+}
+
+int qed_mcp_nvm_put_file_begin(struct qed_dev *cdev, u32 addr)
+{
+ struct qed_hwfn *p_hwfn = QED_LEADING_HWFN(cdev);
+ struct qed_ptt *p_ptt;
+ u32 resp, param;
+ int rc;
+
+ p_ptt = qed_ptt_acquire(p_hwfn);
+ if (!p_ptt)
+ return -EBUSY;
+ rc = qed_mcp_cmd(p_hwfn, p_ptt, DRV_MSG_CODE_NVM_PUT_FILE_BEGIN, addr,
+ &resp, ¶m);
+ cdev->mcp_nvm_resp = resp;
+ qed_ptt_release(p_hwfn, p_ptt);
+
+ return rc;
+}
+
+int qed_mcp_nvm_write(struct qed_dev *cdev,
+ u32 cmd, u32 addr, u8 *p_buf, u32 len)
+{
+ u32 buf_idx = 0, buf_size, nvm_cmd, nvm_offset, resp = 0, param;
+ struct qed_hwfn *p_hwfn = QED_LEADING_HWFN(cdev);
+ struct qed_ptt *p_ptt;
+ int rc = -EINVAL;
+
+ p_ptt = qed_ptt_acquire(p_hwfn);
+ if (!p_ptt)
+ return -EBUSY;
+
+ switch (cmd) {
+ case QED_PUT_FILE_DATA:
+ nvm_cmd = DRV_MSG_CODE_NVM_PUT_FILE_DATA;
+ break;
+ case QED_NVM_WRITE_NVRAM:
+ nvm_cmd = DRV_MSG_CODE_NVM_WRITE_NVRAM;
+ break;
+ default:
+ DP_NOTICE(p_hwfn, "Invalid nvm write command 0x%x\n", cmd);
+ rc = -EINVAL;
+ goto out;
+ }
+
+ while (buf_idx < len) {
+ buf_size = min_t(u32, (len - buf_idx), MCP_DRV_NVM_BUF_LEN);
+ nvm_offset = ((buf_size << DRV_MB_PARAM_NVM_LEN_OFFSET) |
+ addr) + buf_idx;
+ rc = qed_mcp_nvm_wr_cmd(p_hwfn, p_ptt, nvm_cmd, nvm_offset,
+ &resp, ¶m, buf_size,
+ (u32 *)&p_buf[buf_idx]);
+ if (rc) {
+ DP_NOTICE(cdev, "nvm write failed, rc = %d\n", rc);
+ resp = FW_MSG_CODE_ERROR;
+ break;
+ }
+
+ if (resp != FW_MSG_CODE_OK &&
+ resp != FW_MSG_CODE_NVM_OK &&
+ resp != FW_MSG_CODE_NVM_PUT_FILE_FINISH_OK) {
+ DP_NOTICE(cdev,
+ "nvm write failed, resp = 0x%08x\n", resp);
+ rc = -EINVAL;
+ break;
+ }
+
+ /* This can be a lengthy process, and it's possible scheduler
+ * isn't pre-emptable. Sleep a bit to prevent CPU hogging.
+ */
+ if (buf_idx % 0x1000 > (buf_idx + buf_size) % 0x1000)
+ usleep_range(1000, 2000);
+
+ buf_idx += buf_size;
+ }
+
+ cdev->mcp_nvm_resp = resp;
+out:
+ qed_ptt_release(p_hwfn, p_ptt);
+
+ return rc;
+}
+
int qed_mcp_bist_register_test(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt)
{
u32 drv_mb_param = 0, rsp, param;
return rc;
}
-int qed_mcp_bist_nvm_test_get_num_images(struct qed_hwfn *p_hwfn,
- struct qed_ptt *p_ptt,
- u32 *num_images)
+int qed_mcp_bist_nvm_get_num_images(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt,
+ u32 *num_images)
{
u32 drv_mb_param = 0, rsp;
int rc = 0;
return rc;
}
-int qed_mcp_bist_nvm_test_get_image_att(struct qed_hwfn *p_hwfn,
- struct qed_ptt *p_ptt,
- struct bist_nvm_image_att *p_image_att,
- u32 image_index)
+int qed_mcp_bist_nvm_get_image_att(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt,
+ struct bist_nvm_image_att *p_image_att,
+ u32 image_index)
{
u32 buf_size = 0, param, resp = 0, resp_param = 0;
int rc;
return rc;
}
+int qed_mcp_nvm_info_populate(struct qed_hwfn *p_hwfn)
+{
+ struct qed_nvm_image_info *nvm_info = &p_hwfn->nvm_info;
+ struct qed_ptt *p_ptt;
+ int rc;
+ u32 i;
+
+ p_ptt = qed_ptt_acquire(p_hwfn);
+ if (!p_ptt) {
+ DP_ERR(p_hwfn, "failed to acquire ptt\n");
+ return -EBUSY;
+ }
+
+ /* Acquire from MFW the amount of available images */
+ nvm_info->num_images = 0;
+ rc = qed_mcp_bist_nvm_get_num_images(p_hwfn,
+ p_ptt, &nvm_info->num_images);
+ if (rc == -EOPNOTSUPP) {
+ DP_INFO(p_hwfn, "DRV_MSG_CODE_BIST_TEST is not supported\n");
+ goto out;
+ } else if (rc || !nvm_info->num_images) {
+ DP_ERR(p_hwfn, "Failed getting number of images\n");
+ goto err0;
+ }
+
+ nvm_info->image_att = kmalloc(nvm_info->num_images *
+ sizeof(struct bist_nvm_image_att),
+ GFP_KERNEL);
+ if (!nvm_info->image_att) {
+ rc = -ENOMEM;
+ goto err0;
+ }
+
+ /* Iterate over images and get their attributes */
+ for (i = 0; i < nvm_info->num_images; i++) {
+ rc = qed_mcp_bist_nvm_get_image_att(p_hwfn, p_ptt,
+ &nvm_info->image_att[i], i);
+ if (rc) {
+ DP_ERR(p_hwfn,
+ "Failed getting image index %d attributes\n", i);
+ goto err1;
+ }
+
+ DP_VERBOSE(p_hwfn, QED_MSG_SP, "image index %d, size %x\n", i,
+ nvm_info->image_att[i].len);
+ }
+out:
+ qed_ptt_release(p_hwfn, p_ptt);
+ return 0;
+
+err1:
+ kfree(nvm_info->image_att);
+err0:
+ qed_ptt_release(p_hwfn, p_ptt);
+ return rc;
+}
+
static int
qed_mcp_get_nvm_image_att(struct qed_hwfn *p_hwfn,
struct qed_ptt *p_ptt,
enum qed_nvm_images image_id,
struct qed_nvm_image_att *p_image_att)
{
- struct bist_nvm_image_att mfw_image_att;
enum nvm_image_type type;
- u32 num_images, i;
- int rc;
+ u32 i;
/* Translate image_id into MFW definitions */
switch (image_id) {
return -EINVAL;
}
- /* Learn number of images, then traverse and see if one fits */
- rc = qed_mcp_bist_nvm_test_get_num_images(p_hwfn, p_ptt, &num_images);
- if (rc || !num_images)
- return -EINVAL;
-
- for (i = 0; i < num_images; i++) {
- rc = qed_mcp_bist_nvm_test_get_image_att(p_hwfn, p_ptt,
- &mfw_image_att, i);
- if (rc)
- return rc;
-
- if (type == mfw_image_att.image_type)
+ for (i = 0; i < p_hwfn->nvm_info.num_images; i++)
+ if (type == p_hwfn->nvm_info.image_att[i].image_type)
break;
- }
- if (i == num_images) {
+ if (i == p_hwfn->nvm_info.num_images) {
DP_VERBOSE(p_hwfn, QED_MSG_STORAGE,
"Failed to find nvram image of type %08x\n",
image_id);
- return -EINVAL;
+ return -ENOENT;
}
- p_image_att->start_addr = mfw_image_att.nvm_start_addr;
- p_image_att->length = mfw_image_att.len;
+ p_image_att->start_addr = p_hwfn->nvm_info.image_att[i].nvm_start_addr;
+ p_image_att->length = p_hwfn->nvm_info.image_att[i].len;
return 0;
}
*/
int qed_mcp_nvm_read(struct qed_dev *cdev, u32 addr, u8 *p_buf, u32 len);
+/**
+ * @brief Write to nvm
+ *
+ * @param cdev
+ * @param addr - nvm offset
+ * @param cmd - nvm command
+ * @param p_buf - nvm write buffer
+ * @param len - buffer len
+ *
+ * @return int - 0 - operation was successful.
+ */
+int qed_mcp_nvm_write(struct qed_dev *cdev,
+ u32 cmd, u32 addr, u8 *p_buf, u32 len);
+
+/**
+ * @brief Put file begin
+ *
+ * @param cdev
+ * @param addr - nvm offset
+ *
+ * @return int - 0 - operation was successful.
+ */
+int qed_mcp_nvm_put_file_begin(struct qed_dev *cdev, u32 addr);
+
+/**
+ * @brief Check latest response
+ *
+ * @param cdev
+ * @param p_buf - nvm write buffer
+ *
+ * @return int - 0 - operation was successful.
+ */
+int qed_mcp_nvm_resp(struct qed_dev *cdev, u8 *p_buf);
+
struct qed_nvm_image_att {
u32 start_addr;
u32 length;
*
* @return int - 0 - operation was successful.
*/
-int qed_mcp_bist_nvm_test_get_num_images(struct qed_hwfn *p_hwfn,
- struct qed_ptt *p_ptt,
- u32 *num_images);
+int qed_mcp_bist_nvm_get_num_images(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt,
+ u32 *num_images);
/**
* @brief Bist nvm test - get image attributes by index
*
* @return int - 0 - operation was successful.
*/
-int qed_mcp_bist_nvm_test_get_image_att(struct qed_hwfn *p_hwfn,
- struct qed_ptt *p_ptt,
- struct bist_nvm_image_att *p_image_att,
- u32 image_index);
+int qed_mcp_bist_nvm_get_image_att(struct qed_hwfn *p_hwfn,
+ struct qed_ptt *p_ptt,
+ struct bist_nvm_image_att *p_image_att,
+ u32 image_index);
/* Using hwfn number (and not pf_num) is required since in CMT mode,
* same pf_num may be used by two different hwfn
* @param p_ptt
*/
int qed_mcp_set_capabilities(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt);
+
+/**
+ * @brief Populate the nvm info shadow in the given hardware function
+ *
+ * @param p_hwfn
+ */
+int qed_mcp_nvm_info_populate(struct qed_hwfn *p_hwfn);
+
#endif
DP_VERBOSE(p_hwfn, QED_MSG_RDMA, "Freeing RDMA\n");
qed_rdma_free_reserved_lkey(p_hwfn);
+ qed_cxt_free_proto_ilt(p_hwfn, p_hwfn->p_rdma_info->proto);
qed_rdma_resc_free(p_hwfn);
}
}
/* Acquire from MFW the amount of available images */
- rc = qed_mcp_bist_nvm_test_get_num_images(p_hwfn, p_ptt, &num_images);
+ rc = qed_mcp_bist_nvm_get_num_images(p_hwfn, p_ptt, &num_images);
if (rc || !num_images) {
DP_ERR(p_hwfn, "Failed getting number of images\n");
- return -EINVAL;
+ rc = -EINVAL;
+ goto err0;
}
/* Iterate over images and validate CRC */
/* This mailbox returns information about the image required for
* reading it.
*/
- rc = qed_mcp_bist_nvm_test_get_image_att(p_hwfn, p_ptt,
- &image_att, i);
+ rc = qed_mcp_bist_nvm_get_image_att(p_hwfn, p_ptt,
+ &image_att, i);
if (rc) {
DP_ERR(p_hwfn,
"Failed getting image index %d attributes\n",
return current_link.link_up;
}
+static int qede_flash_device(struct net_device *dev,
+ struct ethtool_flash *flash)
+{
+ struct qede_dev *edev = netdev_priv(dev);
+
+ return edev->ops->common->nvm_flash(edev->cdev, flash->data);
+}
+
static int qede_get_coalesce(struct net_device *dev,
struct ethtool_coalesce *coal)
{
.get_tunable = qede_get_tunable,
.set_tunable = qede_set_tunable,
+ .flash_device = qede_flash_device,
};
static const struct ethtool_ops qede_vf_ethtool_ops = {
}
/* Must register notifier before pci ops, since we might miss
- * interface rename after pci probe and netdev registeration.
+ * interface rename after pci probe and netdev registration.
*/
ret = register_netdevice_notifier(&qede_netdev_notifier);
if (ret) {
if (rc)
goto err3;
- /* Prepare the lock prior to the registeration of the netdev,
+ /* Prepare the lock prior to the registration of the netdev,
* as once it's registered we might reach flows requiring it
* [it's even possible to reach a flow needing it directly
* from there, although it's unlikely].
link_params.link_up = true;
edev->ops->common->set_link(edev->cdev, &link_params);
- qede_rdma_dev_event_open(edev);
-
edev->state = QEDE_STATE_OPEN;
DP_INFO(edev, "Ending successfully qede load\n");
DP_NOTICE(edev, "Link is up\n");
netif_tx_start_all_queues(edev->ndev);
netif_carrier_on(edev->ndev);
+ qede_rdma_dev_event_open(edev);
}
} else {
if (netif_carrier_ok(edev->ndev)) {
DP_NOTICE(edev, "Link is down\n");
netif_tx_disable(edev->ndev);
netif_carrier_off(edev->ndev);
+ qede_rdma_dev_event_close(edev);
}
}
}
ptp->clock = ptp_clock_register(&ptp->clock_info, &edev->pdev->dev);
if (IS_ERR(ptp->clock)) {
rc = -EINVAL;
- DP_ERR(edev, "PTP clock registeration failed\n");
+ DP_ERR(edev, "PTP clock registration failed\n");
goto err2;
}
qdev->small_buf_release_cnt -= 8;
}
wmb();
- writel(qdev->small_buf_q_producer_index,
- &port_regs->CommonRegs.rxSmallQProducerIndex);
+ writel_relaxed(qdev->small_buf_q_producer_index,
+ &port_regs->CommonRegs.rxSmallQProducerIndex);
+ mmiowb();
}
}
wmb();
/* clear the interrupt trigger control register */
- writel(0, adapter->isr_int_vec);
+ writel_relaxed(0, adapter->isr_int_vec);
intr_val = readl(adapter->isr_int_vec);
do {
intr_val = readl(adapter->tgt_status_reg);
}
static const struct device_attribute dev_attr_bridged_mode = {
- .attr = {.name = "bridged_mode", .mode = (S_IRUGO | S_IWUSR)},
- .show = qlcnic_show_bridged_mode,
- .store = qlcnic_store_bridged_mode,
+ .attr = { .name = "bridged_mode", .mode = 0644 },
+ .show = qlcnic_show_bridged_mode,
+ .store = qlcnic_store_bridged_mode,
};
static const struct device_attribute dev_attr_diag_mode = {
- .attr = {.name = "diag_mode", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "diag_mode", .mode = 0644 },
.show = qlcnic_show_diag_mode,
.store = qlcnic_store_diag_mode,
};
static const struct device_attribute dev_attr_beacon = {
- .attr = {.name = "beacon", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "beacon", .mode = 0644 },
.show = qlcnic_show_beacon,
.store = qlcnic_store_beacon,
};
static const struct bin_attribute bin_attr_crb = {
- .attr = {.name = "crb", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "crb", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_read_crb,
.write = qlcnic_sysfs_write_crb,
};
static const struct bin_attribute bin_attr_mem = {
- .attr = {.name = "mem", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "mem", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_read_mem,
.write = qlcnic_sysfs_write_mem,
};
static const struct bin_attribute bin_attr_npar_config = {
- .attr = {.name = "npar_config", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "npar_config", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_read_npar_config,
.write = qlcnic_sysfs_write_npar_config,
};
static const struct bin_attribute bin_attr_pci_config = {
- .attr = {.name = "pci_config", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "pci_config", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_read_pci_config,
.write = NULL,
};
static const struct bin_attribute bin_attr_port_stats = {
- .attr = {.name = "port_stats", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "port_stats", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_get_port_stats,
.write = qlcnic_sysfs_clear_port_stats,
};
static const struct bin_attribute bin_attr_esw_stats = {
- .attr = {.name = "esw_stats", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "esw_stats", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_get_esw_stats,
.write = qlcnic_sysfs_clear_esw_stats,
};
static const struct bin_attribute bin_attr_esw_config = {
- .attr = {.name = "esw_config", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "esw_config", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_read_esw_config,
.write = qlcnic_sysfs_write_esw_config,
};
static const struct bin_attribute bin_attr_pm_config = {
- .attr = {.name = "pm_config", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "pm_config", .mode = 0644 },
.size = 0,
.read = qlcnic_sysfs_read_pm_config,
.write = qlcnic_sysfs_write_pm_config,
};
static const struct bin_attribute bin_attr_flash = {
- .attr = {.name = "flash", .mode = (S_IRUGO | S_IWUSR)},
+ .attr = { .name = "flash", .mode = 0644 },
.size = 0,
.read = qlcnic_83xx_sysfs_flash_read_handler,
.write = qlcnic_83xx_sysfs_flash_write_handler,
}
/* hwmon-sysfs attributes */
-static SENSOR_DEVICE_ATTR(temp1_input, S_IRUGO,
+static SENSOR_DEVICE_ATTR(temp1_input, 0444,
qlcnic_hwmon_show_temp, NULL, 1);
static struct attribute *qlcnic_hwmon_attrs[] = {
mmiowb();
}
+/*
+ * Doorbell Registers:
+ * Doorbell registers are virtual registers in the PCI memory space.
+ * The space is allocated by the chip during PCI initialization. The
+ * device driver finds the doorbell address in BAR 3 in PCI config space.
+ * The registers are used to control outbound and inbound queues. For
+ * example, the producer index for an outbound queue. Each queue uses
+ * 1 4k chunk of memory. The lower half of the space is for outbound
+ * queues. The upper half is for inbound queues.
+ * Caller has to guarantee ordering.
+ */
+static inline void ql_write_db_reg_relaxed(u32 val, void __iomem *addr)
+{
+ writel_relaxed(val, addr);
+}
+
/*
* Shadow Registers:
* Outbound queues have a consumer index that is maintained by the chip.
tx_ring->prod_idx = 0;
wmb();
- ql_write_db_reg(tx_ring->prod_idx, tx_ring->prod_idx_db_reg);
+ ql_write_db_reg_relaxed(tx_ring->prod_idx, tx_ring->prod_idx_db_reg);
+ mmiowb();
netif_printk(qdev, tx_queued, KERN_DEBUG, qdev->ndev,
"tx queued, slot %d, len %d\n",
tx_ring->prod_idx, skb->len);
while (tx_q->tpd.consume_idx != hw_consume_idx) {
tpbuf = GET_TPD_BUFFER(tx_q, tx_q->tpd.consume_idx);
if (tpbuf->dma_addr) {
- dma_unmap_single(adpt->netdev->dev.parent,
- tpbuf->dma_addr, tpbuf->length,
- DMA_TO_DEVICE);
+ dma_unmap_page(adpt->netdev->dev.parent,
+ tpbuf->dma_addr, tpbuf->length,
+ DMA_TO_DEVICE);
tpbuf->dma_addr = 0;
}
tpbuf = GET_TPD_BUFFER(tx_q, tx_q->tpd.produce_idx);
tpbuf->length = mapped_len;
- tpbuf->dma_addr = dma_map_single(adpt->netdev->dev.parent,
- skb->data, tpbuf->length,
- DMA_TO_DEVICE);
+ tpbuf->dma_addr = dma_map_page(adpt->netdev->dev.parent,
+ virt_to_page(skb->data),
+ offset_in_page(skb->data),
+ tpbuf->length,
+ DMA_TO_DEVICE);
ret = dma_mapping_error(adpt->netdev->dev.parent,
tpbuf->dma_addr);
if (ret)
if (mapped_len < len) {
tpbuf = GET_TPD_BUFFER(tx_q, tx_q->tpd.produce_idx);
tpbuf->length = len - mapped_len;
- tpbuf->dma_addr = dma_map_single(adpt->netdev->dev.parent,
- skb->data + mapped_len,
- tpbuf->length, DMA_TO_DEVICE);
+ tpbuf->dma_addr = dma_map_page(adpt->netdev->dev.parent,
+ virt_to_page(skb->data +
+ mapped_len),
+ offset_in_page(skb->data +
+ mapped_len),
+ tpbuf->length, DMA_TO_DEVICE);
ret = dma_mapping_error(adpt->netdev->dev.parent,
tpbuf->dma_addr);
if (ret)
dev_name(&qca->net_dev->dev));
return;
}
- debugfs_create_file("info", S_IFREG | S_IRUGO, device_root, qca,
+ debugfs_create_file("info", S_IFREG | 0444, device_root, qca,
&qcaspi_info_ops);
}
-/* Copyright (c) 2013-2017, The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
/* Local Definitions and Declarations */
+static const struct nla_policy rmnet_policy[IFLA_RMNET_MAX + 1] = {
+ [IFLA_RMNET_MUX_ID] = { .type = NLA_U16 },
+ [IFLA_RMNET_FLAGS] = { .len = sizeof(struct ifla_rmnet_flags) },
+};
+
static int rmnet_is_real_dev_registered(const struct net_device *real_dev)
{
return rcu_access_pointer(real_dev->rx_handler) == rmnet_rx_handler;
struct nlattr *tb[], struct nlattr *data[],
struct netlink_ext_ack *extack)
{
- u32 data_format = RMNET_INGRESS_FORMAT_DEAGGREGATION;
+ u32 data_format = RMNET_FLAGS_INGRESS_DEAGGREGATION;
struct net_device *real_dev;
int mode = RMNET_EPMODE_VND;
struct rmnet_endpoint *ep;
if (!real_dev || !dev)
return -ENODEV;
- if (!data[IFLA_VLAN_ID])
+ if (!data[IFLA_RMNET_MUX_ID])
return -EINVAL;
ep = kzalloc(sizeof(*ep), GFP_ATOMIC);
if (!ep)
return -ENOMEM;
- mux_id = nla_get_u16(data[IFLA_VLAN_ID]);
+ mux_id = nla_get_u16(data[IFLA_RMNET_MUX_ID]);
err = rmnet_register_real_device(real_dev);
if (err)
hlist_add_head_rcu(&ep->hlnode, &port->muxed_ep[mux_id]);
- if (data[IFLA_VLAN_FLAGS]) {
- struct ifla_vlan_flags *flags;
+ if (data[IFLA_RMNET_FLAGS]) {
+ struct ifla_rmnet_flags *flags;
- flags = nla_data(data[IFLA_VLAN_FLAGS]);
+ flags = nla_data(data[IFLA_RMNET_FLAGS]);
data_format = flags->flags & flags->mask;
}
{
u16 mux_id;
- if (!data || !data[IFLA_VLAN_ID])
+ if (!data || !data[IFLA_RMNET_MUX_ID])
return -EINVAL;
- mux_id = nla_get_u16(data[IFLA_VLAN_ID]);
+ mux_id = nla_get_u16(data[IFLA_RMNET_MUX_ID]);
if (mux_id > (RMNET_MAX_LOGICAL_EP - 1))
return -ERANGE;
port = rmnet_get_port_rtnl(real_dev);
- if (data[IFLA_VLAN_ID]) {
- mux_id = nla_get_u16(data[IFLA_VLAN_ID]);
+ if (data[IFLA_RMNET_MUX_ID]) {
+ mux_id = nla_get_u16(data[IFLA_RMNET_MUX_ID]);
ep = rmnet_get_endpoint(port, priv->mux_id);
+ if (!ep)
+ return -ENODEV;
hlist_del_init_rcu(&ep->hlnode);
hlist_add_head_rcu(&ep->hlnode, &port->muxed_ep[mux_id]);
priv->mux_id = mux_id;
}
- if (data[IFLA_VLAN_FLAGS]) {
- struct ifla_vlan_flags *flags;
+ if (data[IFLA_RMNET_FLAGS]) {
+ struct ifla_rmnet_flags *flags;
- flags = nla_data(data[IFLA_VLAN_FLAGS]);
+ flags = nla_data(data[IFLA_RMNET_FLAGS]);
port->data_format = flags->flags & flags->mask;
}
static size_t rmnet_get_size(const struct net_device *dev)
{
- return nla_total_size(2) /* IFLA_VLAN_ID */ +
- nla_total_size(sizeof(struct ifla_vlan_flags)); /* IFLA_VLAN_FLAGS */
+ return
+ /* IFLA_RMNET_MUX_ID */
+ nla_total_size(2) +
+ /* IFLA_RMNET_FLAGS */
+ nla_total_size(sizeof(struct ifla_rmnet_flags));
+}
+
+static int rmnet_fill_info(struct sk_buff *skb, const struct net_device *dev)
+{
+ struct rmnet_priv *priv = netdev_priv(dev);
+ struct net_device *real_dev;
+ struct ifla_rmnet_flags f;
+ struct rmnet_port *port;
+
+ real_dev = priv->real_dev;
+
+ if (!rmnet_is_real_dev_registered(real_dev))
+ return -ENODEV;
+
+ if (nla_put_u16(skb, IFLA_RMNET_MUX_ID, priv->mux_id))
+ goto nla_put_failure;
+
+ port = rmnet_get_port_rtnl(real_dev);
+
+ f.flags = port->data_format;
+ f.mask = ~0;
+
+ if (nla_put(skb, IFLA_RMNET_FLAGS, sizeof(f), &f))
+ goto nla_put_failure;
+
+ return 0;
+
+nla_put_failure:
+ return -EMSGSIZE;
}
struct rtnl_link_ops rmnet_link_ops __read_mostly = {
.kind = "rmnet",
- .maxtype = __IFLA_VLAN_MAX,
+ .maxtype = __IFLA_RMNET_MAX,
.priv_size = sizeof(struct rmnet_priv),
.setup = rmnet_vnd_setup,
.validate = rmnet_rtnl_validate,
.dellink = rmnet_dellink,
.get_size = rmnet_get_size,
.changelink = rmnet_changelink,
+ .policy = rmnet_policy,
+ .fill_info = rmnet_fill_info,
};
/* Needs either rcu_read_lock() or rtnl lock */
-/* Copyright (c) 2013-2014, 2016-2017 The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2014, 2016-2018 The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
-/* Copyright (c) 2013-2017, The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
u8 mux_id;
if (RMNET_MAP_GET_CD_BIT(skb)) {
- if (port->data_format & RMNET_INGRESS_FORMAT_MAP_COMMANDS)
+ if (port->data_format & RMNET_FLAGS_INGRESS_MAP_COMMANDS)
return rmnet_map_command(skb, port);
goto free_skb;
skb_pull(skb, sizeof(struct rmnet_map_header));
rmnet_set_skb_proto(skb);
- if (port->data_format & RMNET_INGRESS_FORMAT_MAP_CKSUMV4) {
+ if (port->data_format & RMNET_FLAGS_INGRESS_MAP_CKSUMV4) {
if (!rmnet_map_checksum_downlink_packet(skb, len + pad))
skb->ip_summed = CHECKSUM_UNNECESSARY;
}
skb_push(skb, ETH_HLEN);
}
- if (port->data_format & RMNET_INGRESS_FORMAT_DEAGGREGATION) {
+ if (port->data_format & RMNET_FLAGS_INGRESS_DEAGGREGATION) {
while ((skbn = rmnet_map_deaggregate(skb, port)) != NULL)
__rmnet_map_ingress_handler(skbn, port);
additional_header_len = 0;
required_headroom = sizeof(struct rmnet_map_header);
- if (port->data_format & RMNET_EGRESS_FORMAT_MAP_CKSUMV4) {
+ if (port->data_format & RMNET_FLAGS_EGRESS_MAP_CKSUMV4) {
additional_header_len = sizeof(struct rmnet_map_ul_csum_header);
required_headroom += additional_header_len;
}
goto fail;
}
- if (port->data_format & RMNET_EGRESS_FORMAT_MAP_CKSUMV4)
+ if (port->data_format & RMNET_FLAGS_EGRESS_MAP_CKSUMV4)
rmnet_map_checksum_uplink_packet(skb, orig_dev);
map_header = rmnet_map_add_map_header(skb, additional_header_len, 0);
-/* Copyright (c) 2013-2017, The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
struct {
u16 ip_family:2;
u16 reserved:14;
- u16 flow_control_seq_num;
- u32 qos_id;
+ __be16 flow_control_seq_num;
+ __be32 qos_id;
} flow_control;
u8 data[0];
};
u8 reserved_bit:1;
u8 cd_bit:1;
u8 mux_id;
- u16 pkt_len;
+ __be16 pkt_len;
} __aligned(1);
struct rmnet_map_dl_csum_trailer {
-/* Copyright (c) 2013-2017, The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
struct rmnet_map_control_command *cmd;
int xmit_status;
- if (port->data_format & RMNET_INGRESS_FORMAT_MAP_CKSUMV4) {
+ if (port->data_format & RMNET_FLAGS_INGRESS_MAP_CKSUMV4) {
if (skb->len < sizeof(struct rmnet_map_header) +
RMNET_MAP_GET_LENGTH(skb) +
sizeof(struct rmnet_map_dl_csum_trailer)) {
-/* Copyright (c) 2013-2017, The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
maph = (struct rmnet_map_header *)skb->data;
packet_len = ntohs(maph->pkt_len) + sizeof(struct rmnet_map_header);
- if (port->data_format & RMNET_INGRESS_FORMAT_MAP_CKSUMV4)
+ if (port->data_format & RMNET_FLAGS_INGRESS_MAP_CKSUMV4)
packet_len += sizeof(struct rmnet_map_dl_csum_trailer);
if (((int)skb->len - (int)packet_len) < 0)
if (!skbn)
return NULL;
- skbn->dev = skb->dev;
skb_reserve(skbn, RMNET_MAP_DEAGGR_HEADROOM);
skb_put(skbn, packet_len);
memcpy(skbn->data, skb->data, packet_len);
-/* Copyright (c) 2013-2014, 2016-2017 The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2014, 2016-2018 The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
#define RMNET_NEEDED_HEADROOM 16
#define RMNET_TX_QUEUE_LEN 1000
-/* Constants */
-#define RMNET_INGRESS_FORMAT_DEAGGREGATION BIT(0)
-#define RMNET_INGRESS_FORMAT_MAP_COMMANDS BIT(1)
-#define RMNET_INGRESS_FORMAT_MAP_CKSUMV4 BIT(2)
-#define RMNET_EGRESS_FORMAT_MAP_CKSUMV4 BIT(3)
-
/* Replace skb->dev to a virtual rmnet device and pass up the stack */
#define RMNET_EPMODE_VND (1)
/* Pass the frame directly to another device with dev_queue_xmit() */
-/* Copyright (c) 2013-2017, The Linux Foundation. All rights reserved.
+/* Copyright (c) 2013-2018, The Linux Foundation. All rights reserved.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 and
MODULE_FIRMWARE(FIRMWARE_8107E_1);
MODULE_FIRMWARE(FIRMWARE_8107E_2);
+static inline struct device *tp_to_dev(struct rtl8169_private *tp)
+{
+ return &tp->pci_dev->dev;
+}
+
static void rtl_lock_work(struct rtl8169_private *tp)
{
mutex_lock(&tp->wk.mutex);
mutex_unlock(&tp->wk.mutex);
}
-static void rtl_tx_performance_tweak(struct pci_dev *pdev, u16 force)
+static void rtl_tx_performance_tweak(struct rtl8169_private *tp, u16 force)
{
- pcie_capability_clear_and_set_word(pdev, PCI_EXP_DEVCTL,
+ pcie_capability_clear_and_set_word(tp->pci_dev, PCI_EXP_DEVCTL,
PCI_EXP_DEVCTL_READRQ, force);
}
static void rtl8169_check_link_status(struct net_device *dev,
struct rtl8169_private *tp)
{
+ struct device *d = tp_to_dev(tp);
+
if (tp->link_ok(tp)) {
rtl_link_chg_patch(tp);
/* This is to cancel a scheduled suspend if there's one. */
- pm_request_resume(&tp->pci_dev->dev);
+ pm_request_resume(d);
netif_carrier_on(dev);
if (net_ratelimit())
netif_info(tp, ifup, dev, "link up\n");
} else {
netif_carrier_off(dev);
netif_info(tp, ifdown, dev, "link down\n");
- pm_runtime_idle(&tp->pci_dev->dev);
+ pm_runtime_idle(d);
}
}
static void rtl8169_get_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
{
struct rtl8169_private *tp = netdev_priv(dev);
- struct device *d = &tp->pci_dev->dev;
+ struct device *d = tp_to_dev(tp);
pm_runtime_get_noresume(d);
static int rtl8169_set_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
{
struct rtl8169_private *tp = netdev_priv(dev);
- struct device *d = &tp->pci_dev->dev;
+ struct device *d = tp_to_dev(tp);
pm_runtime_get_noresume(d);
rtl_unlock_work(tp);
- device_set_wakeup_enable(&tp->pci_dev->dev, wol->wolopts);
+ device_set_wakeup_enable(d, wol->wolopts);
pm_runtime_put_noidle(d);
struct ethtool_stats *stats, u64 *data)
{
struct rtl8169_private *tp = netdev_priv(dev);
- struct device *d = &tp->pci_dev->dev;
+ struct device *d = tp_to_dev(tp);
struct rtl8169_counters *counters = tp->counters;
ASSERT_RTNL();
static int rtl_set_mac_address(struct net_device *dev, void *p)
{
struct rtl8169_private *tp = netdev_priv(dev);
- struct device *d = &tp->pci_dev->dev;
- struct sockaddr *addr = p;
-
- if (!is_valid_ether_addr(addr->sa_data))
- return -EADDRNOTAVAIL;
+ struct device *d = tp_to_dev(tp);
+ int ret;
- memcpy(dev->dev_addr, addr->sa_data, dev->addr_len);
+ ret = eth_mac_addr(dev, p);
+ if (ret)
+ return ret;
pm_runtime_get_noresume(d);
{
RTL_W8(tp, Config3, RTL_R8(tp, Config3) | Jumbo_En0);
RTL_W8(tp, Config4, RTL_R8(tp, Config4) | Jumbo_En1);
- rtl_tx_performance_tweak(tp->pci_dev, PCI_EXP_DEVCTL_READRQ_512B);
+ rtl_tx_performance_tweak(tp, PCI_EXP_DEVCTL_READRQ_512B);
}
static void r8168c_hw_jumbo_disable(struct rtl8169_private *tp)
{
RTL_W8(tp, Config3, RTL_R8(tp, Config3) & ~Jumbo_En0);
RTL_W8(tp, Config4, RTL_R8(tp, Config4) & ~Jumbo_En1);
- rtl_tx_performance_tweak(tp->pci_dev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
}
static void r8168dp_hw_jumbo_enable(struct rtl8169_private *tp)
RTL_W8(tp, MaxTxPacketSize, 0x3f);
RTL_W8(tp, Config3, RTL_R8(tp, Config3) | Jumbo_En0);
RTL_W8(tp, Config4, RTL_R8(tp, Config4) | 0x01);
- rtl_tx_performance_tweak(tp->pci_dev, PCI_EXP_DEVCTL_READRQ_512B);
+ rtl_tx_performance_tweak(tp, PCI_EXP_DEVCTL_READRQ_512B);
}
static void r8168e_hw_jumbo_disable(struct rtl8169_private *tp)
RTL_W8(tp, MaxTxPacketSize, 0x0c);
RTL_W8(tp, Config3, RTL_R8(tp, Config3) & ~Jumbo_En0);
RTL_W8(tp, Config4, RTL_R8(tp, Config4) & ~0x01);
- rtl_tx_performance_tweak(tp->pci_dev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
}
static void r8168b_0_hw_jumbo_enable(struct rtl8169_private *tp)
{
- rtl_tx_performance_tweak(tp->pci_dev,
+ rtl_tx_performance_tweak(tp,
PCI_EXP_DEVCTL_READRQ_512B | PCI_EXP_DEVCTL_NOSNOOP_EN);
}
static void r8168b_0_hw_jumbo_disable(struct rtl8169_private *tp)
{
- rtl_tx_performance_tweak(tp->pci_dev,
+ rtl_tx_performance_tweak(tp,
(0x5 << MAX_READ_REQUEST_SHIFT) | PCI_EXP_DEVCTL_NOSNOOP_EN);
}
if (!rtl_fw)
goto err_warn;
- rc = request_firmware(&rtl_fw->fw, name, &tp->pci_dev->dev);
+ rc = request_firmware(&rtl_fw->fw, name, tp_to_dev(tp));
if (rc < 0)
goto err_free;
}
}
-static void rtl_disable_clock_request(struct pci_dev *pdev)
+static void rtl_disable_clock_request(struct rtl8169_private *tp)
{
- pcie_capability_clear_word(pdev, PCI_EXP_LNKCTL,
+ pcie_capability_clear_word(tp->pci_dev, PCI_EXP_LNKCTL,
PCI_EXP_LNKCTL_CLKREQ_EN);
}
-static void rtl_enable_clock_request(struct pci_dev *pdev)
+static void rtl_enable_clock_request(struct rtl8169_private *tp)
{
- pcie_capability_set_word(pdev, PCI_EXP_LNKCTL,
+ pcie_capability_set_word(tp->pci_dev, PCI_EXP_LNKCTL,
PCI_EXP_LNKCTL_CLKREQ_EN);
}
static void rtl_hw_start_8168bb(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
RTL_W8(tp, Config3, RTL_R8(tp, Config3) & ~Beacon_en);
RTL_W16(tp, CPlusCmd, RTL_R16(tp, CPlusCmd) & ~R8168_CPCMD_QUIRK_MASK);
if (tp->dev->mtu <= ETH_DATA_LEN) {
- rtl_tx_performance_tweak(pdev, (0x5 << MAX_READ_REQUEST_SHIFT) |
+ rtl_tx_performance_tweak(tp, (0x5 << MAX_READ_REQUEST_SHIFT) |
PCI_EXP_DEVCTL_NOSNOOP_EN);
}
}
static void __rtl_hw_start_8168cp(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
RTL_W8(tp, Config1, RTL_R8(tp, Config1) | Speed_down);
RTL_W8(tp, Config3, RTL_R8(tp, Config3) & ~Beacon_en);
if (tp->dev->mtu <= ETH_DATA_LEN)
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
- rtl_disable_clock_request(pdev);
+ rtl_disable_clock_request(tp);
RTL_W16(tp, CPlusCmd, RTL_R16(tp, CPlusCmd) & ~R8168_CPCMD_QUIRK_MASK);
}
static void rtl_hw_start_8168cp_2(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
rtl_csi_access_enable_2(tp);
RTL_W8(tp, Config3, RTL_R8(tp, Config3) & ~Beacon_en);
if (tp->dev->mtu <= ETH_DATA_LEN)
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W16(tp, CPlusCmd, RTL_R16(tp, CPlusCmd) & ~R8168_CPCMD_QUIRK_MASK);
}
static void rtl_hw_start_8168cp_3(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
rtl_csi_access_enable_2(tp);
RTL_W8(tp, Config3, RTL_R8(tp, Config3) & ~Beacon_en);
RTL_W8(tp, MaxTxPacketSize, TxPacketMax);
if (tp->dev->mtu <= ETH_DATA_LEN)
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W16(tp, CPlusCmd, RTL_R16(tp, CPlusCmd) & ~R8168_CPCMD_QUIRK_MASK);
}
static void rtl_hw_start_8168d(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
rtl_csi_access_enable_2(tp);
- rtl_disable_clock_request(pdev);
+ rtl_disable_clock_request(tp);
RTL_W8(tp, MaxTxPacketSize, TxPacketMax);
if (tp->dev->mtu <= ETH_DATA_LEN)
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W16(tp, CPlusCmd, RTL_R16(tp, CPlusCmd) & ~R8168_CPCMD_QUIRK_MASK);
}
static void rtl_hw_start_8168dp(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
rtl_csi_access_enable_1(tp);
if (tp->dev->mtu <= ETH_DATA_LEN)
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W8(tp, MaxTxPacketSize, TxPacketMax);
- rtl_disable_clock_request(pdev);
+ rtl_disable_clock_request(tp);
}
static void rtl_hw_start_8168d_4(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
static const struct ephy_info e_info_8168d_4[] = {
{ 0x0b, 0x0000, 0x0048 },
{ 0x19, 0x0020, 0x0050 },
rtl_csi_access_enable_1(tp);
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W8(tp, MaxTxPacketSize, TxPacketMax);
rtl_ephy_init(tp, e_info_8168d_4, ARRAY_SIZE(e_info_8168d_4));
- rtl_enable_clock_request(pdev);
+ rtl_enable_clock_request(tp);
}
static void rtl_hw_start_8168e_1(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
static const struct ephy_info e_info_8168e_1[] = {
{ 0x00, 0x0200, 0x0100 },
{ 0x00, 0x0000, 0x0004 },
rtl_ephy_init(tp, e_info_8168e_1, ARRAY_SIZE(e_info_8168e_1));
if (tp->dev->mtu <= ETH_DATA_LEN)
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W8(tp, MaxTxPacketSize, TxPacketMax);
- rtl_disable_clock_request(pdev);
+ rtl_disable_clock_request(tp);
/* Reset tx FIFO pointer */
RTL_W32(tp, MISC, RTL_R32(tp, MISC) | TXPLA_RST);
static void rtl_hw_start_8168e_2(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
static const struct ephy_info e_info_8168e_2[] = {
{ 0x09, 0x0000, 0x0080 },
{ 0x19, 0x0000, 0x0224 }
rtl_ephy_init(tp, e_info_8168e_2, ARRAY_SIZE(e_info_8168e_2));
if (tp->dev->mtu <= ETH_DATA_LEN)
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
rtl_eri_write(tp, 0xc0, ERIAR_MASK_0011, 0x0000, ERIAR_EXGMAC);
rtl_eri_write(tp, 0xb8, ERIAR_MASK_0011, 0x0000, ERIAR_EXGMAC);
RTL_W8(tp, MaxTxPacketSize, EarlySize);
- rtl_disable_clock_request(pdev);
+ rtl_disable_clock_request(tp);
RTL_W32(tp, TxConfig, RTL_R32(tp, TxConfig) | TXCFG_AUTO_FIFO);
RTL_W8(tp, MCU, RTL_R8(tp, MCU) & ~NOW_IS_OOB);
static void rtl_hw_start_8168f(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
rtl_csi_access_enable_2(tp);
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
rtl_eri_write(tp, 0xc0, ERIAR_MASK_0011, 0x0000, ERIAR_EXGMAC);
rtl_eri_write(tp, 0xb8, ERIAR_MASK_0011, 0x0000, ERIAR_EXGMAC);
RTL_W8(tp, MaxTxPacketSize, EarlySize);
- rtl_disable_clock_request(pdev);
+ rtl_disable_clock_request(tp);
RTL_W32(tp, TxConfig, RTL_R32(tp, TxConfig) | TXCFG_AUTO_FIFO);
RTL_W8(tp, MCU, RTL_R8(tp, MCU) & ~NOW_IS_OOB);
static void rtl_hw_start_8168g(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
RTL_W32(tp, TxConfig, RTL_R32(tp, TxConfig) | TXCFG_AUTO_FIFO);
rtl_eri_write(tp, 0xc8, ERIAR_MASK_0101, 0x080002, ERIAR_EXGMAC);
rtl_csi_access_enable_1(tp);
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
rtl_w0w1_eri(tp, 0xdc, ERIAR_MASK_0001, 0x00, 0x01, ERIAR_EXGMAC);
rtl_w0w1_eri(tp, 0xdc, ERIAR_MASK_0001, 0x01, 0x00, ERIAR_EXGMAC);
static void rtl_hw_start_8168h_1(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
int rg_saw_cnt;
u32 data;
static const struct ephy_info e_info_8168h_1[] = {
rtl_csi_access_enable_1(tp);
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
rtl_w0w1_eri(tp, 0xdc, ERIAR_MASK_0001, 0x00, 0x01, ERIAR_EXGMAC);
rtl_w0w1_eri(tp, 0xdc, ERIAR_MASK_0001, 0x01, 0x00, ERIAR_EXGMAC);
static void rtl_hw_start_8168ep(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
rtl8168ep_stop_cmac(tp);
RTL_W32(tp, TxConfig, RTL_R32(tp, TxConfig) | TXCFG_AUTO_FIFO);
rtl_csi_access_enable_1(tp);
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
rtl_w0w1_eri(tp, 0xdc, ERIAR_MASK_0001, 0x00, 0x01, ERIAR_EXGMAC);
rtl_w0w1_eri(tp, 0xdc, ERIAR_MASK_0001, 0x01, 0x00, ERIAR_EXGMAC);
static void rtl_hw_start_8102e_1(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
static const struct ephy_info e_info_8102e_1[] = {
{ 0x01, 0, 0x6e65 },
{ 0x02, 0, 0x091f },
RTL_W8(tp, DBG_REG, FIX_NAK_1);
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W8(tp, Config1,
LEDS1 | LEDS0 | Speed_down | MEMMAP | IOMAP | VPD | PMEnable);
static void rtl_hw_start_8102e_2(struct rtl8169_private *tp)
{
- struct pci_dev *pdev = tp->pci_dev;
-
rtl_csi_access_enable_2(tp);
- rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
RTL_W8(tp, Config1, MEMMAP | IOMAP | VPD | PMEnable);
RTL_W8(tp, Config3, RTL_R8(tp, Config3) & ~Beacon_en);
rtl_ephy_init(tp, e_info_8402, ARRAY_SIZE(e_info_8402));
- rtl_tx_performance_tweak(tp->pci_dev, 0x5 << MAX_READ_REQUEST_SHIFT);
+ rtl_tx_performance_tweak(tp, 0x5 << MAX_READ_REQUEST_SHIFT);
rtl_eri_write(tp, 0xc8, ERIAR_MASK_1111, 0x00000002, ERIAR_EXGMAC);
rtl_eri_write(tp, 0xe8, ERIAR_MASK_1111, 0x00000006, ERIAR_EXGMAC);
static void rtl8169_free_rx_databuff(struct rtl8169_private *tp,
void **data_buff, struct RxDesc *desc)
{
- dma_unmap_single(&tp->pci_dev->dev, le64_to_cpu(desc->addr), rx_buf_sz,
+ dma_unmap_single(tp_to_dev(tp), le64_to_cpu(desc->addr), rx_buf_sz,
DMA_FROM_DEVICE);
kfree(*data_buff);
{
void *data;
dma_addr_t mapping;
- struct device *d = &tp->pci_dev->dev;
+ struct device *d = tp_to_dev(tp);
struct net_device *dev = tp->dev;
int node = dev->dev.parent ? dev_to_node(dev->dev.parent) : -1;
if (len) {
struct sk_buff *skb = tx_skb->skb;
- rtl8169_unmap_tx_skb(&tp->pci_dev->dev, tx_skb,
+ rtl8169_unmap_tx_skb(tp_to_dev(tp), tx_skb,
tp->TxDescArray + entry);
if (skb) {
dev_consume_skb_any(skb);
struct skb_shared_info *info = skb_shinfo(skb);
unsigned int cur_frag, entry;
struct TxDesc *uninitialized_var(txd);
- struct device *d = &tp->pci_dev->dev;
+ struct device *d = tp_to_dev(tp);
entry = tp->cur_tx;
for (cur_frag = 0; cur_frag < info->nr_frags; cur_frag++) {
struct rtl8169_private *tp = netdev_priv(dev);
unsigned int entry = tp->cur_tx % NUM_TX_DESC;
struct TxDesc *txd = tp->TxDescArray + entry;
- struct device *d = &tp->pci_dev->dev;
+ struct device *d = tp_to_dev(tp);
dma_addr_t mapping;
u32 status, len;
u32 opts[2];
*/
dma_rmb();
- rtl8169_unmap_tx_skb(&tp->pci_dev->dev, tx_skb,
+ rtl8169_unmap_tx_skb(tp_to_dev(tp), tx_skb,
tp->TxDescArray + entry);
if (status & LastFrag) {
u64_stats_update_begin(&tp->tx_stats.syncp);
dma_addr_t addr)
{
struct sk_buff *skb;
- struct device *d = &tp->pci_dev->dev;
+ struct device *d = tp_to_dev(tp);
data = rtl8169_align(data);
dma_sync_single_for_cpu(d, addr, pkt_size, DMA_FROM_DEVICE);
return ioread32(mdp->tsu_addr + mdp->reg_offset[enum_index]);
}
-static bool sh_eth_is_gether(struct sh_eth_private *mdp)
-{
- return mdp->reg_offset == sh_eth_offset_gigabit;
-}
-
-static bool sh_eth_is_rz_fast_ether(struct sh_eth_private *mdp)
-{
- return mdp->reg_offset == sh_eth_offset_fast_rz;
-}
-
static void sh_eth_select_mii(struct net_device *ndev)
{
struct sh_eth_private *mdp = netdev_priv(ndev);
mdelay(1);
}
+static int sh_eth_soft_reset(struct net_device *ndev)
+{
+ sh_eth_modify(ndev, EDMR, EDMR_SRST_ETHER, EDMR_SRST_ETHER);
+ mdelay(3);
+ sh_eth_modify(ndev, EDMR, EDMR_SRST_ETHER, 0);
+
+ return 0;
+}
+
+static int sh_eth_check_soft_reset(struct net_device *ndev)
+{
+ int cnt;
+
+ for (cnt = 100; cnt > 0; cnt--) {
+ if (!(sh_eth_read(ndev, EDMR) & EDMR_SRST_GETHER))
+ return 0;
+ mdelay(1);
+ }
+
+ netdev_err(ndev, "Device reset failed\n");
+ return -ETIMEDOUT;
+}
+
+static int sh_eth_soft_reset_gether(struct net_device *ndev)
+{
+ struct sh_eth_private *mdp = netdev_priv(ndev);
+ int ret;
+
+ sh_eth_write(ndev, EDSR_ENALL, EDSR);
+ sh_eth_modify(ndev, EDMR, EDMR_SRST_GETHER, EDMR_SRST_GETHER);
+
+ ret = sh_eth_check_soft_reset(ndev);
+ if (ret)
+ return ret;
+
+ /* Table Init */
+ sh_eth_write(ndev, 0, TDLAR);
+ sh_eth_write(ndev, 0, TDFAR);
+ sh_eth_write(ndev, 0, TDFXR);
+ sh_eth_write(ndev, 0, TDFFR);
+ sh_eth_write(ndev, 0, RDLAR);
+ sh_eth_write(ndev, 0, RDFAR);
+ sh_eth_write(ndev, 0, RDFXR);
+ sh_eth_write(ndev, 0, RDFFR);
+
+ /* Reset HW CRC register */
+ if (mdp->cd->hw_checksum)
+ sh_eth_write(ndev, 0, CSMR);
+
+ /* Select MII mode */
+ if (mdp->cd->select_mii)
+ sh_eth_select_mii(ndev);
+
+ return ret;
+}
+
static void sh_eth_set_rate_gether(struct net_device *ndev)
{
struct sh_eth_private *mdp = netdev_priv(ndev);
#ifdef CONFIG_OF
/* R7S72100 */
static struct sh_eth_cpu_data r7s72100_data = {
+ .soft_reset = sh_eth_soft_reset_gether,
+
.chip_reset = sh_eth_chip_reset,
.set_duplex = sh_eth_set_duplex,
.register_type = SH_ETH_REG_FAST_RZ,
+ .edtrr_trns = EDTRR_TRNS_GETHER,
.ecsr_value = ECSR_ICD,
.ecsipr_value = ECSIPR_ICDIP,
.eesipr_value = EESIPR_TWB1IP | EESIPR_TWBIP | EESIPR_TC1IP |
.rpadir_value = 2 << 16,
.no_trimd = 1,
.no_ade = 1,
+ .xdfar_rw = 1,
.hw_checksum = 1,
.tsu = 1,
+ .no_tx_cntrs = 1,
};
static void sh_eth_chip_reset_r8a7740(struct net_device *ndev)
/* R8A7740 */
static struct sh_eth_cpu_data r8a7740_data = {
+ .soft_reset = sh_eth_soft_reset_gether,
+
.chip_reset = sh_eth_chip_reset_r8a7740,
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_gether,
.register_type = SH_ETH_REG_GIGABIT,
+ .edtrr_trns = EDTRR_TRNS_GETHER,
.ecsr_value = ECSR_ICD | ECSR_MPD,
.ecsipr_value = ECSIPR_LCHNGIP | ECSIPR_ICDIP | ECSIPR_MPDIP,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ECIIP |
.rpadir_value = 2 << 16,
.no_trimd = 1,
.no_ade = 1,
+ .xdfar_rw = 1,
.hw_checksum = 1,
.tsu = 1,
.select_mii = 1,
.magic = 1,
+ .cexcr = 1,
};
/* There is CPU dependent code */
/* R-Car Gen1 */
static struct sh_eth_cpu_data rcar_gen1_data = {
+ .soft_reset = sh_eth_soft_reset,
+
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_rcar,
.register_type = SH_ETH_REG_FAST_RCAR,
+ .edtrr_trns = EDTRR_TRNS_ETHER,
.ecsr_value = ECSR_PSRTO | ECSR_LCHNG | ECSR_ICD,
.ecsipr_value = ECSIPR_PSRTOIP | ECSIPR_LCHNGIP | ECSIPR_ICDIP,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ADEIP | EESIPR_ECIIP |
/* R-Car Gen2 and RZ/G1 */
static struct sh_eth_cpu_data rcar_gen2_data = {
+ .soft_reset = sh_eth_soft_reset,
+
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_rcar,
.register_type = SH_ETH_REG_FAST_RCAR,
+ .edtrr_trns = EDTRR_TRNS_ETHER,
.ecsr_value = ECSR_PSRTO | ECSR_LCHNG | ECSR_ICD | ECSR_MPD,
.ecsipr_value = ECSIPR_PSRTOIP | ECSIPR_LCHNGIP | ECSIPR_ICDIP |
ECSIPR_MPDIP,
/* SH7724 */
static struct sh_eth_cpu_data sh7724_data = {
+ .soft_reset = sh_eth_soft_reset,
+
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_sh7724,
.register_type = SH_ETH_REG_FAST_SH4,
+ .edtrr_trns = EDTRR_TRNS_ETHER,
.ecsr_value = ECSR_PSRTO | ECSR_LCHNG | ECSR_ICD,
.ecsipr_value = ECSIPR_PSRTOIP | ECSIPR_LCHNGIP | ECSIPR_ICDIP,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ADEIP | EESIPR_ECIIP |
/* SH7757 */
static struct sh_eth_cpu_data sh7757_data = {
+ .soft_reset = sh_eth_soft_reset,
+
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_sh7757,
.register_type = SH_ETH_REG_FAST_SH4,
+ .edtrr_trns = EDTRR_TRNS_ETHER,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ECIIP |
EESIPR_FTCIP | EESIPR_TDEIP | EESIPR_TFUFIP |
EESIPR_FRIP | EESIPR_RDEIP | EESIPR_RFOFIP |
/* SH7757(GETHERC) */
static struct sh_eth_cpu_data sh7757_data_giga = {
+ .soft_reset = sh_eth_soft_reset_gether,
+
.chip_reset = sh_eth_chip_reset_giga,
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_giga,
.register_type = SH_ETH_REG_GIGABIT,
+ .edtrr_trns = EDTRR_TRNS_GETHER,
.ecsr_value = ECSR_ICD | ECSR_MPD,
.ecsipr_value = ECSIPR_LCHNGIP | ECSIPR_ICDIP | ECSIPR_MPDIP,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ECIIP |
.rpadir_value = 2 << 16,
.no_trimd = 1,
.no_ade = 1,
+ .xdfar_rw = 1,
.tsu = 1,
+ .cexcr = 1,
.dual_port = 1,
};
/* SH7734 */
static struct sh_eth_cpu_data sh7734_data = {
+ .soft_reset = sh_eth_soft_reset_gether,
+
.chip_reset = sh_eth_chip_reset,
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_gether,
.register_type = SH_ETH_REG_GIGABIT,
+ .edtrr_trns = EDTRR_TRNS_GETHER,
.ecsr_value = ECSR_ICD | ECSR_MPD,
.ecsipr_value = ECSIPR_LCHNGIP | ECSIPR_ICDIP | ECSIPR_MPDIP,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ECIIP |
.hw_swap = 1,
.no_trimd = 1,
.no_ade = 1,
+ .xdfar_rw = 1,
.tsu = 1,
.hw_checksum = 1,
.select_mii = 1,
.magic = 1,
+ .cexcr = 1,
};
/* SH7763 */
static struct sh_eth_cpu_data sh7763_data = {
+ .soft_reset = sh_eth_soft_reset_gether,
+
.chip_reset = sh_eth_chip_reset,
.set_duplex = sh_eth_set_duplex,
.set_rate = sh_eth_set_rate_gether,
.register_type = SH_ETH_REG_GIGABIT,
+ .edtrr_trns = EDTRR_TRNS_GETHER,
.ecsr_value = ECSR_ICD | ECSR_MPD,
.ecsipr_value = ECSIPR_LCHNGIP | ECSIPR_ICDIP | ECSIPR_MPDIP,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ECIIP |
.hw_swap = 1,
.no_trimd = 1,
.no_ade = 1,
+ .xdfar_rw = 1,
.tsu = 1,
.irq_flags = IRQF_SHARED,
.magic = 1,
+ .cexcr = 1,
.dual_port = 1,
};
static struct sh_eth_cpu_data sh7619_data = {
+ .soft_reset = sh_eth_soft_reset,
+
.register_type = SH_ETH_REG_FAST_SH3_SH2,
+ .edtrr_trns = EDTRR_TRNS_ETHER,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ECIIP |
EESIPR_FTCIP | EESIPR_TDEIP | EESIPR_TFUFIP |
EESIPR_FRIP | EESIPR_RDEIP | EESIPR_RFOFIP |
};
static struct sh_eth_cpu_data sh771x_data = {
+ .soft_reset = sh_eth_soft_reset,
+
.register_type = SH_ETH_REG_FAST_SH3_SH2,
+ .edtrr_trns = EDTRR_TRNS_ETHER,
.eesipr_value = EESIPR_RFCOFIP | EESIPR_ECIIP |
EESIPR_FTCIP | EESIPR_TDEIP | EESIPR_TFUFIP |
EESIPR_FRIP | EESIPR_RDEIP | EESIPR_RFOFIP |
cd->trscer_err_mask = DEFAULT_TRSCER_ERR_MASK;
}
-static int sh_eth_check_reset(struct net_device *ndev)
-{
- int cnt;
-
- for (cnt = 100; cnt > 0; cnt--) {
- if (!(sh_eth_read(ndev, EDMR) & EDMR_SRST_GETHER))
- return 0;
- mdelay(1);
- }
-
- netdev_err(ndev, "Device reset failed\n");
- return -ETIMEDOUT;
-}
-
-static int sh_eth_reset(struct net_device *ndev)
-{
- struct sh_eth_private *mdp = netdev_priv(ndev);
- int ret = 0;
-
- if (sh_eth_is_gether(mdp) || sh_eth_is_rz_fast_ether(mdp)) {
- sh_eth_write(ndev, EDSR_ENALL, EDSR);
- sh_eth_modify(ndev, EDMR, EDMR_SRST_GETHER, EDMR_SRST_GETHER);
-
- ret = sh_eth_check_reset(ndev);
- if (ret)
- return ret;
-
- /* Table Init */
- sh_eth_write(ndev, 0x0, TDLAR);
- sh_eth_write(ndev, 0x0, TDFAR);
- sh_eth_write(ndev, 0x0, TDFXR);
- sh_eth_write(ndev, 0x0, TDFFR);
- sh_eth_write(ndev, 0x0, RDLAR);
- sh_eth_write(ndev, 0x0, RDFAR);
- sh_eth_write(ndev, 0x0, RDFXR);
- sh_eth_write(ndev, 0x0, RDFFR);
-
- /* Reset HW CRC register */
- if (mdp->cd->hw_checksum)
- sh_eth_write(ndev, 0x0, CSMR);
-
- /* Select MII mode */
- if (mdp->cd->select_mii)
- sh_eth_select_mii(ndev);
- } else {
- sh_eth_modify(ndev, EDMR, EDMR_SRST_ETHER, EDMR_SRST_ETHER);
- mdelay(3);
- sh_eth_modify(ndev, EDMR, EDMR_SRST_ETHER, 0);
- }
-
- return ret;
-}
-
static void sh_eth_set_receive_align(struct sk_buff *skb)
{
uintptr_t reserve = (uintptr_t)skb->data & (SH_ETH_RX_ALIGN - 1);
}
}
-static u32 sh_eth_get_edtrr_trns(struct sh_eth_private *mdp)
-{
- if (sh_eth_is_gether(mdp) || sh_eth_is_rz_fast_ether(mdp))
- return EDTRR_TRNS_GETHER;
- else
- return EDTRR_TRNS_ETHER;
-}
-
struct bb_info {
void (*set_gate)(void *addr);
struct mdiobb_ctrl ctrl;
/* Rx descriptor address set */
if (i == 0) {
sh_eth_write(ndev, mdp->rx_desc_dma, RDLAR);
- if (sh_eth_is_gether(mdp) ||
- sh_eth_is_rz_fast_ether(mdp))
+ if (mdp->cd->xdfar_rw)
sh_eth_write(ndev, mdp->rx_desc_dma, RDFAR);
}
}
if (i == 0) {
/* Tx descriptor address set */
sh_eth_write(ndev, mdp->tx_desc_dma, TDLAR);
- if (sh_eth_is_gether(mdp) ||
- sh_eth_is_rz_fast_ether(mdp))
+ if (mdp->cd->xdfar_rw)
sh_eth_write(ndev, mdp->tx_desc_dma, TDFAR);
}
}
int ret;
/* Soft Reset */
- ret = sh_eth_reset(ndev);
+ ret = mdp->cd->soft_reset(ndev);
if (ret)
return ret;
*/
msleep(2); /* max frame time at 10 Mbps < 1250 us */
sh_eth_get_stats(ndev);
- sh_eth_reset(ndev);
+ mdp->cd->soft_reset(ndev);
/* Set MAC address again */
update_mac_address(ndev);
sh_eth_tx_free(ndev, true);
/* SH7712 BUG */
- if (edtrr ^ sh_eth_get_edtrr_trns(mdp)) {
+ if (edtrr ^ mdp->cd->edtrr_trns) {
/* tx dma start */
- sh_eth_write(ndev, sh_eth_get_edtrr_trns(mdp), EDTRR);
+ sh_eth_write(ndev, mdp->cd->edtrr_trns, EDTRR);
}
/* wakeup */
netif_wake_queue(ndev);
mdp->cur_tx++;
- if (!(sh_eth_read(ndev, EDTRR) & sh_eth_get_edtrr_trns(mdp)))
- sh_eth_write(ndev, sh_eth_get_edtrr_trns(mdp), EDTRR);
+ if (!(sh_eth_read(ndev, EDTRR) & mdp->cd->edtrr_trns))
+ sh_eth_write(ndev, mdp->cd->edtrr_trns, EDTRR);
return NETDEV_TX_OK;
}
{
struct sh_eth_private *mdp = netdev_priv(ndev);
- if (sh_eth_is_rz_fast_ether(mdp))
+ if (mdp->cd->no_tx_cntrs)
return &ndev->stats;
if (!mdp->is_opened)
sh_eth_update_stat(ndev, &ndev->stats.collisions, CDCR);
sh_eth_update_stat(ndev, &ndev->stats.tx_carrier_errors, LCCR);
- if (sh_eth_is_gether(mdp)) {
+ if (mdp->cd->cexcr) {
sh_eth_update_stat(ndev, &ndev->stats.tx_carrier_errors,
CERCR);
sh_eth_update_stat(ndev, &ndev->stats.tx_carrier_errors,
/* This structure is used by each CPU dependency handling. */
struct sh_eth_cpu_data {
+ /* mandatory functions */
+ int (*soft_reset)(struct net_device *ndev);
+
/* optional functions */
void (*chip_reset)(struct net_device *ndev);
void (*set_duplex)(struct net_device *ndev);
/* mandatory initialize value */
int register_type;
+ u32 edtrr_trns;
u32 eesipr_value;
/* optional initialize value */
unsigned rpadir:1; /* E-DMAC have RPADIR */
unsigned no_trimd:1; /* E-DMAC DO NOT have TRIMD */
unsigned no_ade:1; /* E-DMAC DO NOT have ADE bit in EESR */
+ unsigned xdfar_rw:1; /* E-DMAC has writeable RDFAR/TDFAR */
unsigned hw_checksum:1; /* E-DMAC has CSMR */
unsigned select_mii:1; /* EtherC have RMII_MII (MII select register) */
unsigned rmiimode:1; /* EtherC has RMIIMODE register */
unsigned rtrate:1; /* EtherC has RTRATE register */
unsigned magic:1; /* EtherC has ECMR.MPDE and ECSR.MPD */
+ unsigned no_tx_cntrs:1; /* EtherC DOES NOT have TX error counters */
+ unsigned cexcr:1; /* EtherC has CERCR/CEECR */
unsigned dual_port:1; /* Dual EtherC/E-DMAC */
};
static int debug = -1;
static int eee_timer = SXGBE_DEFAULT_LPI_TIMER;
-module_param(eee_timer, int, S_IRUGO | S_IWUSR);
+module_param(eee_timer, int, 0644);
-module_param(debug, int, S_IRUGO | S_IWUSR);
+module_param(debug, int, 0644);
static const u32 default_msg_level = (NETIF_MSG_DRV | NETIF_MSG_PROBE |
NETIF_MSG_LINK | NETIF_MSG_IFUP |
NETIF_MSG_IFDOWN | NETIF_MSG_TIMER);
MC_CMD_GET_PARSER_DISP_INFO_OUT_SUPPORTED_MATCHES_MAXNUM * 2];
unsigned int rx_match_count;
+ struct rw_semaphore lock; /* Protects entries */
struct {
unsigned long spec; /* pointer to spec plus flag bits */
-/* BUSY flag indicates that an update is in progress. AUTO_OLD is
- * used to mark and sweep MAC filters for the device address lists.
- */
-#define EFX_EF10_FILTER_FLAG_BUSY 1UL
+/* AUTO_OLD is used to mark and sweep MAC filters for the device address lists. */
+/* unused flag 1UL */
#define EFX_EF10_FILTER_FLAG_AUTO_OLD 2UL
#define EFX_EF10_FILTER_FLAGS 3UL
u64 handle; /* firmware handle */
} *entry;
- wait_queue_head_t waitq;
/* Shadow of net_device address lists, guarded by mac_lock */
struct efx_ef10_dev_addr dev_uc_list[EFX_EF10_FILTER_DEV_UC_MAX];
struct efx_ef10_dev_addr dev_mc_list[EFX_EF10_FILTER_DEV_MC_MAX];
/* All our allocations have been reset */
nic_data->must_realloc_vis = true;
+ nic_data->must_restore_rss_contexts = true;
nic_data->must_restore_filters = true;
nic_data->must_restore_piobufs = true;
efx_ef10_forget_old_piobufs(efx);
{
int rc;
+ WARN_ON(!mutex_is_locked(&efx->rss_lock));
+
if (ctx->context_id == EFX_EF10_RSS_CONTEXT_INVALID) {
rc = efx_ef10_alloc_rss_context(efx, true, ctx, NULL);
if (rc)
size_t outlen;
int rc, i;
+ WARN_ON(!mutex_is_locked(&efx->rss_lock));
+
BUILD_BUG_ON(MC_CMD_RSS_CONTEXT_GET_TABLE_IN_LEN !=
MC_CMD_RSS_CONTEXT_GET_KEY_IN_LEN);
static int efx_ef10_rx_pull_rss_config(struct efx_nic *efx)
{
- return efx_ef10_rx_pull_rss_context_config(efx, &efx->rss_context);
+ int rc;
+
+ mutex_lock(&efx->rss_lock);
+ rc = efx_ef10_rx_pull_rss_context_config(efx, &efx->rss_context);
+ mutex_unlock(&efx->rss_lock);
+ return rc;
}
static void efx_ef10_rx_restore_rss_contexts(struct efx_nic *efx)
{
+ struct efx_ef10_nic_data *nic_data = efx->nic_data;
struct efx_rss_context *ctx;
int rc;
+ WARN_ON(!mutex_is_locked(&efx->rss_lock));
+
+ if (!nic_data->must_restore_rss_contexts)
+ return;
+
list_for_each_entry(ctx, &efx->rss_context.list, list) {
/* previous NIC RSS context is gone */
ctx->context_id = EFX_EF10_RSS_CONTEXT_INVALID;
"; RSS filters may fail to be applied\n",
ctx->user_id, rc);
}
+ nic_data->must_restore_rss_contexts = false;
}
static int efx_ef10_pf_rx_push_rss_config(struct efx_nic *efx, bool user,
struct efx_filter_spec *spec,
bool replace_equal)
{
- struct efx_ef10_filter_table *table = efx->filter_state;
DECLARE_BITMAP(mc_rem_map, EFX_EF10_FILTER_SEARCH_LIMIT);
+ struct efx_ef10_nic_data *nic_data = efx->nic_data;
+ struct efx_ef10_filter_table *table;
struct efx_filter_spec *saved_spec;
struct efx_rss_context *ctx = NULL;
unsigned int match_pri, hash;
unsigned int priv_flags;
+ bool rss_locked = false;
bool replacing = false;
+ unsigned int depth, i;
int ins_index = -1;
DEFINE_WAIT(wait);
bool is_mc_recip;
s32 rc;
+ down_read(&efx->filter_sem);
+ table = efx->filter_state;
+ down_write(&table->lock);
+
/* For now, only support RX filters */
if ((spec->flags & (EFX_FILTER_FLAG_RX | EFX_FILTER_FLAG_TX)) !=
- EFX_FILTER_FLAG_RX)
- return -EINVAL;
+ EFX_FILTER_FLAG_RX) {
+ rc = -EINVAL;
+ goto out_unlock;
+ }
rc = efx_ef10_filter_pri(table, spec);
if (rc < 0)
- return rc;
+ goto out_unlock;
match_pri = rc;
hash = efx_ef10_filter_hash(spec);
bitmap_zero(mc_rem_map, EFX_EF10_FILTER_SEARCH_LIMIT);
if (spec->flags & EFX_FILTER_FLAG_RX_RSS) {
+ mutex_lock(&efx->rss_lock);
+ rss_locked = true;
if (spec->rss_context)
- ctx = efx_find_rss_context_entry(spec->rss_context,
- &efx->rss_context.list);
+ ctx = efx_find_rss_context_entry(efx, spec->rss_context);
else
ctx = &efx->rss_context;
- if (!ctx)
- return -ENOENT;
- if (ctx->context_id == EFX_EF10_RSS_CONTEXT_INVALID)
- return -EOPNOTSUPP;
+ if (!ctx) {
+ rc = -ENOENT;
+ goto out_unlock;
+ }
+ if (ctx->context_id == EFX_EF10_RSS_CONTEXT_INVALID) {
+ rc = -EOPNOTSUPP;
+ goto out_unlock;
+ }
}
/* Find any existing filters with the same match tuple or
- * else a free slot to insert at. If any of them are busy,
- * we have to wait and retry.
+ * else a free slot to insert at.
*/
- for (;;) {
- unsigned int depth = 1;
- unsigned int i;
-
- spin_lock_bh(&efx->filter_lock);
-
- for (;;) {
- i = (hash + depth) & (HUNT_FILTER_TBL_ROWS - 1);
- saved_spec = efx_ef10_filter_entry_spec(table, i);
+ for (depth = 1; depth < EFX_EF10_FILTER_SEARCH_LIMIT; depth++) {
+ i = (hash + depth) & (HUNT_FILTER_TBL_ROWS - 1);
+ saved_spec = efx_ef10_filter_entry_spec(table, i);
- if (!saved_spec) {
- if (ins_index < 0)
- ins_index = i;
- } else if (efx_ef10_filter_equal(spec, saved_spec)) {
- if (table->entry[i].spec &
- EFX_EF10_FILTER_FLAG_BUSY)
- break;
- if (spec->priority < saved_spec->priority &&
- spec->priority != EFX_FILTER_PRI_AUTO) {
- rc = -EPERM;
- goto out_unlock;
- }
- if (!is_mc_recip) {
- /* This is the only one */
- if (spec->priority ==
- saved_spec->priority &&
- !replace_equal) {
- rc = -EEXIST;
- goto out_unlock;
- }
- ins_index = i;
- goto found;
- } else if (spec->priority >
- saved_spec->priority ||
- (spec->priority ==
- saved_spec->priority &&
- replace_equal)) {
- if (ins_index < 0)
- ins_index = i;
- else
- __set_bit(depth, mc_rem_map);
- }
+ if (!saved_spec) {
+ if (ins_index < 0)
+ ins_index = i;
+ } else if (efx_ef10_filter_equal(spec, saved_spec)) {
+ if (spec->priority < saved_spec->priority &&
+ spec->priority != EFX_FILTER_PRI_AUTO) {
+ rc = -EPERM;
+ goto out_unlock;
}
-
- /* Once we reach the maximum search depth, use
- * the first suitable slot or return -EBUSY if
- * there was none
- */
- if (depth == EFX_EF10_FILTER_SEARCH_LIMIT) {
- if (ins_index < 0) {
- rc = -EBUSY;
+ if (!is_mc_recip) {
+ /* This is the only one */
+ if (spec->priority ==
+ saved_spec->priority &&
+ !replace_equal) {
+ rc = -EEXIST;
goto out_unlock;
}
- goto found;
+ ins_index = i;
+ break;
+ } else if (spec->priority >
+ saved_spec->priority ||
+ (spec->priority ==
+ saved_spec->priority &&
+ replace_equal)) {
+ if (ins_index < 0)
+ ins_index = i;
+ else
+ __set_bit(depth, mc_rem_map);
}
-
- ++depth;
}
-
- prepare_to_wait(&table->waitq, &wait, TASK_UNINTERRUPTIBLE);
- spin_unlock_bh(&efx->filter_lock);
- schedule();
}
-found:
- /* Create a software table entry if necessary, and mark it
- * busy. We might yet fail to insert, but any attempt to
- * insert a conflicting filter while we're waiting for the
- * firmware must find the busy entry.
+ /* Once we reach the maximum search depth, use the first suitable
+ * slot, or return -EBUSY if there was none
*/
+ if (ins_index < 0) {
+ rc = -EBUSY;
+ goto out_unlock;
+ }
+
+ /* Create a software table entry if necessary. */
saved_spec = efx_ef10_filter_entry_spec(table, ins_index);
if (saved_spec) {
if (spec->priority == EFX_FILTER_PRI_AUTO &&
*saved_spec = *spec;
priv_flags = 0;
}
- efx_ef10_filter_set_entry(table, ins_index, saved_spec,
- priv_flags | EFX_EF10_FILTER_FLAG_BUSY);
-
- /* Mark lower-priority multicast recipients busy prior to removal */
- if (is_mc_recip) {
- unsigned int depth, i;
-
- for (depth = 0; depth < EFX_EF10_FILTER_SEARCH_LIMIT; depth++) {
- i = (hash + depth) & (HUNT_FILTER_TBL_ROWS - 1);
- if (test_bit(depth, mc_rem_map))
- table->entry[i].spec |=
- EFX_EF10_FILTER_FLAG_BUSY;
- }
- }
-
- spin_unlock_bh(&efx->filter_lock);
+ efx_ef10_filter_set_entry(table, ins_index, saved_spec, priv_flags);
+ /* Actually insert the filter on the HW */
rc = efx_ef10_filter_push(efx, spec, &table->entry[ins_index].handle,
ctx, replacing);
+ if (rc == -EINVAL && nic_data->must_realloc_vis)
+ /* The MC rebooted under us, causing it to reject our filter
+ * insertion as pointing to an invalid VI (spec->dmaq_id).
+ */
+ rc = -EAGAIN;
+
/* Finalise the software table entry */
- spin_lock_bh(&efx->filter_lock);
if (rc == 0) {
if (replacing) {
/* Update the fields that may differ */
} else if (!replacing) {
kfree(saved_spec);
saved_spec = NULL;
+ } else {
+ /* We failed to replace, so the old filter is still present.
+ * Roll back the software table to reflect this. In fact the
+ * efx_ef10_filter_set_entry() call below will do the right
+ * thing, so nothing extra is needed here.
+ */
}
efx_ef10_filter_set_entry(table, ins_index, saved_spec, priv_flags);
priv_flags = efx_ef10_filter_entry_flags(table, i);
if (rc == 0) {
- spin_unlock_bh(&efx->filter_lock);
MCDI_SET_DWORD(inbuf, FILTER_OP_IN_OP,
MC_CMD_FILTER_OP_IN_OP_UNSUBSCRIBE);
MCDI_SET_QWORD(inbuf, FILTER_OP_IN_HANDLE,
rc = efx_mcdi_rpc(efx, MC_CMD_FILTER_OP,
inbuf, sizeof(inbuf),
NULL, 0, NULL);
- spin_lock_bh(&efx->filter_lock);
}
if (rc == 0) {
kfree(saved_spec);
saved_spec = NULL;
priv_flags = 0;
- } else {
- priv_flags &= ~EFX_EF10_FILTER_FLAG_BUSY;
}
efx_ef10_filter_set_entry(table, i, saved_spec,
priv_flags);
if (rc == 0)
rc = efx_ef10_make_filter_id(match_pri, ins_index);
- wake_up_all(&table->waitq);
out_unlock:
- spin_unlock_bh(&efx->filter_lock);
- finish_wait(&table->waitq, &wait);
+ if (rss_locked)
+ mutex_unlock(&efx->rss_lock);
+ up_write(&table->lock);
+ up_read(&efx->filter_sem);
return rc;
}
* If !by_index, remove by ID
* If by_index, remove by index
* Filter ID may come from userland and must be range-checked.
+ * Caller must hold efx->filter_sem for read, and efx->filter_state->lock
+ * for write.
*/
static int efx_ef10_filter_remove_internal(struct efx_nic *efx,
unsigned int priority_mask,
DEFINE_WAIT(wait);
int rc;
- /* Find the software table entry and mark it busy. Don't
- * remove it yet; any attempt to update while we're waiting
- * for the firmware must find the busy entry.
- */
- for (;;) {
- spin_lock_bh(&efx->filter_lock);
- if (!(table->entry[filter_idx].spec &
- EFX_EF10_FILTER_FLAG_BUSY))
- break;
- prepare_to_wait(&table->waitq, &wait, TASK_UNINTERRUPTIBLE);
- spin_unlock_bh(&efx->filter_lock);
- schedule();
- }
-
spec = efx_ef10_filter_entry_spec(table, filter_idx);
if (!spec ||
(!by_index &&
efx_ef10_filter_pri(table, spec) !=
- efx_ef10_filter_get_unsafe_pri(filter_id))) {
- rc = -ENOENT;
- goto out_unlock;
- }
+ efx_ef10_filter_get_unsafe_pri(filter_id)))
+ return -ENOENT;
if (spec->flags & EFX_FILTER_FLAG_RX_OVER_AUTO &&
priority_mask == (1U << EFX_FILTER_PRI_AUTO)) {
/* Just remove flags */
spec->flags &= ~EFX_FILTER_FLAG_RX_OVER_AUTO;
table->entry[filter_idx].spec &= ~EFX_EF10_FILTER_FLAG_AUTO_OLD;
- rc = 0;
- goto out_unlock;
- }
-
- if (!(priority_mask & (1U << spec->priority))) {
- rc = -ENOENT;
- goto out_unlock;
+ return 0;
}
- table->entry[filter_idx].spec |= EFX_EF10_FILTER_FLAG_BUSY;
- spin_unlock_bh(&efx->filter_lock);
+ if (!(priority_mask & (1U << spec->priority)))
+ return -ENOENT;
if (spec->flags & EFX_FILTER_FLAG_RX_OVER_AUTO) {
/* Reset to an automatic filter */
&efx->rss_context,
true);
- spin_lock_bh(&efx->filter_lock);
if (rc == 0)
*spec = new_spec;
} else {
rc = efx_mcdi_rpc_quiet(efx, MC_CMD_FILTER_OP,
inbuf, sizeof(inbuf), NULL, 0, NULL);
- spin_lock_bh(&efx->filter_lock);
if ((rc == 0) || (rc == -ENOENT)) {
/* Filter removed OK or didn't actually exist */
kfree(spec);
}
}
- table->entry[filter_idx].spec &= ~EFX_EF10_FILTER_FLAG_BUSY;
- wake_up_all(&table->waitq);
-out_unlock:
- spin_unlock_bh(&efx->filter_lock);
- finish_wait(&table->waitq, &wait);
return rc;
}
enum efx_filter_priority priority,
u32 filter_id)
{
- return efx_ef10_filter_remove_internal(efx, 1U << priority,
- filter_id, false);
+ struct efx_ef10_filter_table *table;
+ int rc;
+
+ down_read(&efx->filter_sem);
+ table = efx->filter_state;
+ down_write(&table->lock);
+ rc = efx_ef10_filter_remove_internal(efx, 1U << priority, filter_id,
+ false);
+ up_write(&table->lock);
+ up_read(&efx->filter_sem);
+ return rc;
}
+/* Caller must hold efx->filter_sem for read */
static void efx_ef10_filter_remove_unsafe(struct efx_nic *efx,
enum efx_filter_priority priority,
u32 filter_id)
{
+ struct efx_ef10_filter_table *table = efx->filter_state;
+
if (filter_id == EFX_EF10_FILTER_ID_INVALID)
return;
- efx_ef10_filter_remove_internal(efx, 1U << priority, filter_id, true);
+
+ down_write(&table->lock);
+ efx_ef10_filter_remove_internal(efx, 1U << priority, filter_id,
+ true);
+ up_write(&table->lock);
}
static int efx_ef10_filter_get_safe(struct efx_nic *efx,
u32 filter_id, struct efx_filter_spec *spec)
{
unsigned int filter_idx = efx_ef10_filter_get_unsafe_id(filter_id);
- struct efx_ef10_filter_table *table = efx->filter_state;
const struct efx_filter_spec *saved_spec;
+ struct efx_ef10_filter_table *table;
int rc;
- spin_lock_bh(&efx->filter_lock);
+ down_read(&efx->filter_sem);
+ table = efx->filter_state;
+ down_read(&table->lock);
saved_spec = efx_ef10_filter_entry_spec(table, filter_idx);
if (saved_spec && saved_spec->priority == priority &&
efx_ef10_filter_pri(table, saved_spec) ==
} else {
rc = -ENOENT;
}
- spin_unlock_bh(&efx->filter_lock);
+ up_read(&table->lock);
+ up_read(&efx->filter_sem);
return rc;
}
static int efx_ef10_filter_clear_rx(struct efx_nic *efx,
- enum efx_filter_priority priority)
+ enum efx_filter_priority priority)
{
+ struct efx_ef10_filter_table *table;
unsigned int priority_mask;
unsigned int i;
int rc;
priority_mask = (((1U << (priority + 1)) - 1) &
~(1U << EFX_FILTER_PRI_AUTO));
+ down_read(&efx->filter_sem);
+ table = efx->filter_state;
+ down_write(&table->lock);
for (i = 0; i < HUNT_FILTER_TBL_ROWS; i++) {
rc = efx_ef10_filter_remove_internal(efx, priority_mask,
i, true);
if (rc && rc != -ENOENT)
- return rc;
+ break;
+ rc = 0;
}
- return 0;
+ up_write(&table->lock);
+ up_read(&efx->filter_sem);
+ return rc;
}
static u32 efx_ef10_filter_count_rx_used(struct efx_nic *efx,
enum efx_filter_priority priority)
{
- struct efx_ef10_filter_table *table = efx->filter_state;
+ struct efx_ef10_filter_table *table;
unsigned int filter_idx;
s32 count = 0;
- spin_lock_bh(&efx->filter_lock);
+ down_read(&efx->filter_sem);
+ table = efx->filter_state;
+ down_read(&table->lock);
for (filter_idx = 0; filter_idx < HUNT_FILTER_TBL_ROWS; filter_idx++) {
if (table->entry[filter_idx].spec &&
efx_ef10_filter_entry_spec(table, filter_idx)->priority ==
priority)
++count;
}
- spin_unlock_bh(&efx->filter_lock);
+ up_read(&table->lock);
+ up_read(&efx->filter_sem);
return count;
}
enum efx_filter_priority priority,
u32 *buf, u32 size)
{
- struct efx_ef10_filter_table *table = efx->filter_state;
+ struct efx_ef10_filter_table *table;
struct efx_filter_spec *spec;
unsigned int filter_idx;
s32 count = 0;
- spin_lock_bh(&efx->filter_lock);
+ down_read(&efx->filter_sem);
+ table = efx->filter_state;
+ down_read(&table->lock);
+
for (filter_idx = 0; filter_idx < HUNT_FILTER_TBL_ROWS; filter_idx++) {
spec = efx_ef10_filter_entry_spec(table, filter_idx);
if (spec && spec->priority == priority) {
filter_idx);
}
}
- spin_unlock_bh(&efx->filter_lock);
+ up_read(&table->lock);
+ up_read(&efx->filter_sem);
return count;
}
#ifdef CONFIG_RFS_ACCEL
-static efx_mcdi_async_completer efx_ef10_filter_rfs_insert_complete;
-
-static s32 efx_ef10_filter_rfs_insert(struct efx_nic *efx,
- struct efx_filter_spec *spec)
-{
- struct efx_ef10_filter_table *table = efx->filter_state;
- MCDI_DECLARE_BUF(inbuf, MC_CMD_FILTER_OP_EXT_IN_LEN);
- struct efx_filter_spec *saved_spec;
- unsigned int hash, i, depth = 1;
- bool replacing = false;
- int ins_index = -1;
- u64 cookie;
- s32 rc;
-
- /* Must be an RX filter without RSS and not for a multicast
- * destination address (RFS only works for connected sockets).
- * These restrictions allow us to pass only a tiny amount of
- * data through to the completion function.
- */
- EFX_WARN_ON_PARANOID(spec->flags !=
- (EFX_FILTER_FLAG_RX | EFX_FILTER_FLAG_RX_SCATTER));
- EFX_WARN_ON_PARANOID(spec->priority != EFX_FILTER_PRI_HINT);
- EFX_WARN_ON_PARANOID(efx_filter_is_mc_recipient(spec));
-
- hash = efx_ef10_filter_hash(spec);
-
- spin_lock_bh(&efx->filter_lock);
-
- /* Find any existing filter with the same match tuple or else
- * a free slot to insert at. If an existing filter is busy,
- * we have to give up.
- */
- for (;;) {
- i = (hash + depth) & (HUNT_FILTER_TBL_ROWS - 1);
- saved_spec = efx_ef10_filter_entry_spec(table, i);
-
- if (!saved_spec) {
- if (ins_index < 0)
- ins_index = i;
- } else if (efx_ef10_filter_equal(spec, saved_spec)) {
- if (table->entry[i].spec & EFX_EF10_FILTER_FLAG_BUSY) {
- rc = -EBUSY;
- goto fail_unlock;
- }
- if (spec->priority < saved_spec->priority) {
- rc = -EPERM;
- goto fail_unlock;
- }
- ins_index = i;
- break;
- }
-
- /* Once we reach the maximum search depth, use the
- * first suitable slot or return -EBUSY if there was
- * none
- */
- if (depth == EFX_EF10_FILTER_SEARCH_LIMIT) {
- if (ins_index < 0) {
- rc = -EBUSY;
- goto fail_unlock;
- }
- break;
- }
-
- ++depth;
- }
-
- /* Create a software table entry if necessary, and mark it
- * busy. We might yet fail to insert, but any attempt to
- * insert a conflicting filter while we're waiting for the
- * firmware must find the busy entry.
- */
- saved_spec = efx_ef10_filter_entry_spec(table, ins_index);
- if (saved_spec) {
- replacing = true;
- } else {
- saved_spec = kmalloc(sizeof(*spec), GFP_ATOMIC);
- if (!saved_spec) {
- rc = -ENOMEM;
- goto fail_unlock;
- }
- *saved_spec = *spec;
- }
- efx_ef10_filter_set_entry(table, ins_index, saved_spec,
- EFX_EF10_FILTER_FLAG_BUSY);
-
- spin_unlock_bh(&efx->filter_lock);
-
- /* Pack up the variables needed on completion */
- cookie = replacing << 31 | ins_index << 16 | spec->dmaq_id;
-
- efx_ef10_filter_push_prep(efx, spec, inbuf,
- table->entry[ins_index].handle, NULL,
- replacing);
- efx_mcdi_rpc_async(efx, MC_CMD_FILTER_OP, inbuf, sizeof(inbuf),
- MC_CMD_FILTER_OP_OUT_LEN,
- efx_ef10_filter_rfs_insert_complete, cookie);
-
- return ins_index;
-
-fail_unlock:
- spin_unlock_bh(&efx->filter_lock);
- return rc;
-}
-
-static void
-efx_ef10_filter_rfs_insert_complete(struct efx_nic *efx, unsigned long cookie,
- int rc, efx_dword_t *outbuf,
- size_t outlen_actual)
-{
- struct efx_ef10_filter_table *table = efx->filter_state;
- unsigned int ins_index, dmaq_id;
- struct efx_filter_spec *spec;
- bool replacing;
-
- /* Unpack the cookie */
- replacing = cookie >> 31;
- ins_index = (cookie >> 16) & (HUNT_FILTER_TBL_ROWS - 1);
- dmaq_id = cookie & 0xffff;
-
- spin_lock_bh(&efx->filter_lock);
- spec = efx_ef10_filter_entry_spec(table, ins_index);
- if (rc == 0) {
- table->entry[ins_index].handle =
- MCDI_QWORD(outbuf, FILTER_OP_OUT_HANDLE);
- if (replacing)
- spec->dmaq_id = dmaq_id;
- } else if (!replacing) {
- kfree(spec);
- spec = NULL;
- }
- efx_ef10_filter_set_entry(table, ins_index, spec, 0);
- spin_unlock_bh(&efx->filter_lock);
-
- wake_up_all(&table->waitq);
-}
-
-static void
-efx_ef10_filter_rfs_expire_complete(struct efx_nic *efx,
- unsigned long filter_idx,
- int rc, efx_dword_t *outbuf,
- size_t outlen_actual);
-
static bool efx_ef10_filter_rfs_expire_one(struct efx_nic *efx, u32 flow_id,
unsigned int filter_idx)
{
- struct efx_ef10_filter_table *table = efx->filter_state;
- struct efx_filter_spec *spec =
- efx_ef10_filter_entry_spec(table, filter_idx);
- MCDI_DECLARE_BUF(inbuf,
- MC_CMD_FILTER_OP_IN_HANDLE_OFST +
- MC_CMD_FILTER_OP_IN_HANDLE_LEN);
-
- if (!spec ||
- (table->entry[filter_idx].spec & EFX_EF10_FILTER_FLAG_BUSY) ||
- spec->priority != EFX_FILTER_PRI_HINT ||
- !rps_may_expire_flow(efx->net_dev, spec->dmaq_id,
- flow_id, filter_idx))
- return false;
-
- MCDI_SET_DWORD(inbuf, FILTER_OP_IN_OP,
- MC_CMD_FILTER_OP_IN_OP_REMOVE);
- MCDI_SET_QWORD(inbuf, FILTER_OP_IN_HANDLE,
- table->entry[filter_idx].handle);
- if (efx_mcdi_rpc_async(efx, MC_CMD_FILTER_OP, inbuf, sizeof(inbuf), 0,
- efx_ef10_filter_rfs_expire_complete, filter_idx))
- return false;
+ struct efx_ef10_filter_table *table;
+ struct efx_filter_spec *spec;
+ bool ret;
- table->entry[filter_idx].spec |= EFX_EF10_FILTER_FLAG_BUSY;
- return true;
-}
+ down_read(&efx->filter_sem);
+ table = efx->filter_state;
+ down_write(&table->lock);
+ spec = efx_ef10_filter_entry_spec(table, filter_idx);
-static void
-efx_ef10_filter_rfs_expire_complete(struct efx_nic *efx,
- unsigned long filter_idx,
- int rc, efx_dword_t *outbuf,
- size_t outlen_actual)
-{
- struct efx_ef10_filter_table *table = efx->filter_state;
- struct efx_filter_spec *spec =
- efx_ef10_filter_entry_spec(table, filter_idx);
+ if (!spec || spec->priority != EFX_FILTER_PRI_HINT) {
+ ret = true;
+ goto out_unlock;
+ }
- spin_lock_bh(&efx->filter_lock);
- if (rc == 0) {
- kfree(spec);
- efx_ef10_filter_set_entry(table, filter_idx, NULL, 0);
+ if (!rps_may_expire_flow(efx->net_dev, spec->dmaq_id,
+ flow_id, filter_idx)) {
+ ret = false;
+ goto out_unlock;
}
- table->entry[filter_idx].spec &= ~EFX_EF10_FILTER_FLAG_BUSY;
- wake_up_all(&table->waitq);
- spin_unlock_bh(&efx->filter_lock);
+
+ ret = efx_ef10_filter_remove_internal(efx, 1U << spec->priority,
+ filter_idx, true) == 0;
+out_unlock:
+ up_write(&table->lock);
+ up_read(&efx->filter_sem);
+ return ret;
}
#endif /* CONFIG_RFS_ACCEL */
table->vlan_filter =
!!(efx->net_dev->features & NETIF_F_HW_VLAN_CTAG_FILTER);
INIT_LIST_HEAD(&table->vlan_list);
+ init_rwsem(&table->lock);
efx->filter_state = table;
- init_waitqueue_head(&table->waitq);
list_for_each_entry(vlan, &nic_data->vlan_list, list) {
rc = efx_ef10_filter_add_vlan(efx, vlan->vid);
if (!table)
return;
- spin_lock_bh(&efx->filter_lock);
+ down_write(&table->lock);
+ mutex_lock(&efx->rss_lock);
for (filter_idx = 0; filter_idx < HUNT_FILTER_TBL_ROWS; filter_idx++) {
spec = efx_ef10_filter_entry_spec(table, filter_idx);
goto not_restored;
}
if (spec->rss_context)
- ctx = efx_find_rss_context_entry(spec->rss_context,
- &efx->rss_context.list);
+ ctx = efx_find_rss_context_entry(efx, spec->rss_context);
else
ctx = &efx->rss_context;
if (spec->flags & EFX_FILTER_FLAG_RX_RSS) {
}
}
- table->entry[filter_idx].spec |= EFX_EF10_FILTER_FLAG_BUSY;
- spin_unlock_bh(&efx->filter_lock);
-
rc = efx_ef10_filter_push(efx, spec,
&table->entry[filter_idx].handle,
ctx, false);
if (rc)
failed++;
- spin_lock_bh(&efx->filter_lock);
if (rc) {
not_restored:
kfree(spec);
efx_ef10_filter_set_entry(table, filter_idx, NULL, 0);
- } else {
- table->entry[filter_idx].spec &=
- ~EFX_EF10_FILTER_FLAG_BUSY;
}
}
- spin_unlock_bh(&efx->filter_lock);
+ mutex_unlock(&efx->rss_lock);
+ up_write(&table->lock);
/* This can happen validly if the MC's capabilities have changed, so
* is not an error.
struct efx_ef10_filter_table *table = efx->filter_state;
unsigned int filter_idx;
+ efx_rwsem_assert_write_locked(&table->lock);
+
if (*id != EFX_EF10_FILTER_ID_INVALID) {
filter_idx = efx_ef10_filter_get_unsafe_id(*id);
if (!table->entry[filter_idx].spec)
struct efx_ef10_filter_table *table = efx->filter_state;
struct efx_ef10_filter_vlan *vlan;
- spin_lock_bh(&efx->filter_lock);
+ down_write(&table->lock);
list_for_each_entry(vlan, &table->vlan_list, list)
_efx_ef10_filter_vlan_mark_old(efx, vlan);
- spin_unlock_bh(&efx->filter_lock);
+ up_write(&table->lock);
}
static void efx_ef10_filter_uc_addr_list(struct efx_nic *efx)
return rc;
}
-/* Remove filters that weren't renewed. Since nothing else changes the AUTO_OLD
- * flag or removes these filters, we don't need to hold the filter_lock while
- * scanning for these filters.
- */
+/* Remove filters that weren't renewed. */
static void efx_ef10_filter_remove_old(struct efx_nic *efx)
{
struct efx_ef10_filter_table *table = efx->filter_state;
int rc;
int i;
+ down_write(&table->lock);
for (i = 0; i < HUNT_FILTER_TBL_ROWS; i++) {
if (READ_ONCE(table->entry[i].spec) &
EFX_EF10_FILTER_FLAG_AUTO_OLD) {
remove_failed++;
}
}
+ up_write(&table->lock);
if (remove_failed)
netif_info(efx, drv, efx->net_dev,
.filter_get_rx_id_limit = efx_ef10_filter_get_rx_id_limit,
.filter_get_rx_ids = efx_ef10_filter_get_rx_ids,
#ifdef CONFIG_RFS_ACCEL
- .filter_rfs_insert = efx_ef10_filter_rfs_insert,
.filter_rfs_expire_one = efx_ef10_filter_rfs_expire_one,
#endif
#ifdef CONFIG_SFC_MTD
.filter_get_rx_id_limit = efx_ef10_filter_get_rx_id_limit,
.filter_get_rx_ids = efx_ef10_filter_get_rx_ids,
#ifdef CONFIG_RFS_ACCEL
- .filter_rfs_insert = efx_ef10_filter_rfs_insert,
.filter_rfs_expire_one = efx_ef10_filter_rfs_expire_one,
#endif
#ifdef CONFIG_SFC_MTD
efx_update_irq_mod(efx, channel);
}
- efx_filter_rfs_expire(channel);
+#ifdef CONFIG_RFS_ACCEL
+ /* Perhaps expire some ARFS filters */
+ schedule_work(&channel->filter_work);
+#endif
/* There is no race here; although napi_disable() will
* only wait for napi_complete(), this isn't a problem
tx_queue->channel = channel;
}
+#ifdef CONFIG_RFS_ACCEL
+ INIT_WORK(&channel->filter_work, efx_filter_rfs_expire);
+#endif
+
rx_queue = &channel->rx_queue;
rx_queue->efx = efx;
timer_setup(&rx_queue->slow_fill, efx_rx_slow_fill, 0);
rx_queue->buffer = NULL;
memset(&rx_queue->rxd, 0, sizeof(rx_queue->rxd));
timer_setup(&rx_queue->slow_fill, efx_rx_slow_fill, 0);
+#ifdef CONFIG_RFS_ACCEL
+ INIT_WORK(&channel->filter_work, efx_filter_rfs_expire);
+#endif
return channel;
}
{
int rc;
- spin_lock_init(&efx->filter_lock);
init_rwsem(&efx->filter_sem);
mutex_lock(&efx->mac_lock);
down_write(&efx->filter_sem);
efx_disable_interrupts(efx);
mutex_lock(&efx->mac_lock);
+ mutex_lock(&efx->rss_lock);
if (efx->port_initialized && method != RESET_TYPE_INVISIBLE &&
method != RESET_TYPE_DATAPATH)
efx->phy_op->fini(efx);
if (efx->type->rx_restore_rss_contexts)
efx->type->rx_restore_rss_contexts(efx);
+ mutex_unlock(&efx->rss_lock);
down_read(&efx->filter_sem);
efx_restore_filters(efx);
up_read(&efx->filter_sem);
fail:
efx->port_initialized = false;
+ mutex_unlock(&efx->rss_lock);
mutex_unlock(&efx->mac_lock);
return rc;
efx->rx_packet_ts_offset =
efx->type->rx_ts_offset - efx->type->rx_prefix_size;
INIT_LIST_HEAD(&efx->rss_context.list);
+ mutex_init(&efx->rss_lock);
spin_lock_init(&efx->stats_lock);
efx->vi_stride = EFX_DEFAULT_VI_STRIDE;
efx->num_mac_stats = MC_CMD_MAC_NSTATS;
BUILD_BUG_ON(MC_CMD_MAC_NSTATS - 1 != MC_CMD_MAC_GENERATION_END);
mutex_init(&efx->mac_lock);
+#ifdef CONFIG_RFS_ACCEL
+ mutex_init(&efx->rps_mutex);
+#endif
efx->phy_op = &efx_dummy_phy_operations;
efx->mdio.dev = net_dev;
INIT_WORK(&efx->mac_work, efx_mac_work);
/* RSS contexts. We're using linked lists and crappy O(n) algorithms, because
* (a) this is an infrequent control-plane operation and (b) n is small (max 64)
*/
-struct efx_rss_context *efx_alloc_rss_context_entry(struct list_head *head)
+struct efx_rss_context *efx_alloc_rss_context_entry(struct efx_nic *efx)
{
+ struct list_head *head = &efx->rss_context.list;
struct efx_rss_context *ctx, *new;
u32 id = 1; /* Don't use zero, that refers to the master RSS context */
+ WARN_ON(!mutex_is_locked(&efx->rss_lock));
+
/* Search for first gap in the numbering */
list_for_each_entry(ctx, head, list) {
if (ctx->user_id != id)
return new;
}
-struct efx_rss_context *efx_find_rss_context_entry(u32 id, struct list_head *head)
+struct efx_rss_context *efx_find_rss_context_entry(struct efx_nic *efx, u32 id)
{
+ struct list_head *head = &efx->rss_context.list;
struct efx_rss_context *ctx;
+ WARN_ON(!mutex_is_locked(&efx->rss_lock));
+
list_for_each_entry(ctx, head, list)
if (ctx->user_id == id)
return ctx;
int efx_filter_rfs(struct net_device *net_dev, const struct sk_buff *skb,
u16 rxq_index, u32 flow_id);
bool __efx_filter_rfs_expire(struct efx_nic *efx, unsigned quota);
-static inline void efx_filter_rfs_expire(struct efx_channel *channel)
+static inline void efx_filter_rfs_expire(struct work_struct *data)
{
+ struct efx_channel *channel = container_of(data, struct efx_channel,
+ filter_work);
+
if (channel->rfs_filters_added >= 60 &&
__efx_filter_rfs_expire(channel->efx, 100))
channel->rfs_filters_added -= 60;
}
#define efx_filter_rfs_enabled() 1
#else
-static inline void efx_filter_rfs_expire(struct efx_channel *channel) {}
+static inline void efx_filter_rfs_expire(struct work_struct *data) {}
#define efx_filter_rfs_enabled() 0
#endif
bool efx_filter_is_mc_recipient(const struct efx_filter_spec *spec);
/* RSS contexts */
-struct efx_rss_context *efx_alloc_rss_context_entry(struct list_head *list);
-struct efx_rss_context *efx_find_rss_context_entry(u32 id, struct list_head *list);
+struct efx_rss_context *efx_alloc_rss_context_entry(struct efx_nic *efx);
+struct efx_rss_context *efx_find_rss_context_entry(struct efx_nic *efx, u32 id);
void efx_free_rss_context_entry(struct efx_rss_context *ctx);
static inline bool efx_rss_active(struct efx_rss_context *ctx)
{
{
struct efx_nic *efx = netdev_priv(net_dev);
u32 rss_context = 0;
- s32 rc;
+ s32 rc = 0;
switch (info->cmd) {
case ETHTOOL_GRXRINGS:
case ETHTOOL_GRXFH: {
struct efx_rss_context *ctx = &efx->rss_context;
+ mutex_lock(&efx->rss_lock);
if (info->flow_type & FLOW_RSS && info->rss_context) {
- ctx = efx_find_rss_context_entry(info->rss_context,
- &efx->rss_context.list);
- if (!ctx)
- return -ENOENT;
+ ctx = efx_find_rss_context_entry(efx, info->rss_context);
+ if (!ctx) {
+ rc = -ENOENT;
+ goto out_unlock;
+ }
}
info->data = 0;
if (!efx_rss_active(ctx)) /* No RSS */
- return 0;
+ goto out_unlock;
switch (info->flow_type & ~FLOW_RSS) {
case UDP_V4_FLOW:
if (ctx->rx_hash_udp_4tuple)
default:
break;
}
- return 0;
+out_unlock:
+ mutex_unlock(&efx->rss_lock);
+ return rc;
}
case ETHTOOL_GRXCLSRLCNT:
struct ethtool_tcpip6_spec *ip6_mask = &rule->m_u.tcp_ip6_spec;
struct ethtool_usrip6_spec *uip6_entry = &rule->h_u.usr_ip6_spec;
struct ethtool_usrip6_spec *uip6_mask = &rule->m_u.usr_ip6_spec;
+ u32 flow_type = rule->flow_type & ~(FLOW_EXT | FLOW_RSS);
struct ethhdr *mac_entry = &rule->h_u.ether_spec;
struct ethhdr *mac_mask = &rule->m_u.ether_spec;
enum efx_filter_flags flags = 0;
if (rule->flow_type & FLOW_RSS)
spec.rss_context = rss_context;
- switch (rule->flow_type & ~(FLOW_EXT | FLOW_RSS)) {
+ switch (flow_type) {
case TCP_V4_FLOW:
case UDP_V4_FLOW:
spec.match_flags = (EFX_FILTER_MATCH_ETHER_TYPE |
EFX_FILTER_MATCH_IP_PROTO);
spec.ether_type = htons(ETH_P_IP);
- spec.ip_proto = ((rule->flow_type & ~FLOW_EXT) == TCP_V4_FLOW ?
- IPPROTO_TCP : IPPROTO_UDP);
+ spec.ip_proto = flow_type == TCP_V4_FLOW ? IPPROTO_TCP
+ : IPPROTO_UDP;
if (ip_mask->ip4dst) {
if (ip_mask->ip4dst != IP4_ADDR_FULL_MASK)
return -EINVAL;
spec.match_flags = (EFX_FILTER_MATCH_ETHER_TYPE |
EFX_FILTER_MATCH_IP_PROTO);
spec.ether_type = htons(ETH_P_IPV6);
- spec.ip_proto = ((rule->flow_type & ~FLOW_EXT) == TCP_V6_FLOW ?
- IPPROTO_TCP : IPPROTO_UDP);
+ spec.ip_proto = flow_type == TCP_V6_FLOW ? IPPROTO_TCP
+ : IPPROTO_UDP;
if (!ip6_mask_is_empty(ip6_mask->ip6dst)) {
if (!ip6_mask_is_full(ip6_mask->ip6dst))
return -EINVAL;
{
struct efx_nic *efx = netdev_priv(net_dev);
struct efx_rss_context *ctx;
- int rc;
+ int rc = 0;
if (!efx->type->rx_pull_rss_context_config)
return -EOPNOTSUPP;
- ctx = efx_find_rss_context_entry(rss_context, &efx->rss_context.list);
- if (!ctx)
- return -ENOENT;
+
+ mutex_lock(&efx->rss_lock);
+ ctx = efx_find_rss_context_entry(efx, rss_context);
+ if (!ctx) {
+ rc = -ENOENT;
+ goto out_unlock;
+ }
rc = efx->type->rx_pull_rss_context_config(efx, ctx);
if (rc)
- return rc;
+ goto out_unlock;
if (hfunc)
*hfunc = ETH_RSS_HASH_TOP;
memcpy(indir, ctx->rx_indir_table, sizeof(ctx->rx_indir_table));
if (key)
memcpy(key, ctx->rx_hash_key, efx->type->rx_hash_key_size);
- return 0;
+out_unlock:
+ mutex_unlock(&efx->rss_lock);
+ return rc;
}
static int efx_ethtool_set_rxfh_context(struct net_device *net_dev,
/* Hash function is Toeplitz, cannot be changed */
if (hfunc != ETH_RSS_HASH_NO_CHANGE && hfunc != ETH_RSS_HASH_TOP)
return -EOPNOTSUPP;
+
+ mutex_lock(&efx->rss_lock);
+
if (*rss_context == ETH_RXFH_CONTEXT_ALLOC) {
- if (delete)
+ if (delete) {
/* alloc + delete == Nothing to do */
- return -EINVAL;
- ctx = efx_alloc_rss_context_entry(&efx->rss_context.list);
- if (!ctx)
- return -ENOMEM;
+ rc = -EINVAL;
+ goto out_unlock;
+ }
+ ctx = efx_alloc_rss_context_entry(efx);
+ if (!ctx) {
+ rc = -ENOMEM;
+ goto out_unlock;
+ }
ctx->context_id = EFX_EF10_RSS_CONTEXT_INVALID;
/* Initialise indir table and key to defaults */
efx_set_default_rx_indir_table(efx, ctx);
netdev_rss_key_fill(ctx->rx_hash_key, sizeof(ctx->rx_hash_key));
allocated = true;
} else {
- ctx = efx_find_rss_context_entry(*rss_context,
- &efx->rss_context.list);
- if (!ctx)
- return -ENOENT;
+ ctx = efx_find_rss_context_entry(efx, *rss_context);
+ if (!ctx) {
+ rc = -ENOENT;
+ goto out_unlock;
+ }
}
if (delete) {
rc = efx->type->rx_push_rss_context_config(efx, ctx, NULL, NULL);
if (!rc)
efx_free_rss_context_entry(ctx);
- return rc;
+ goto out_unlock;
}
if (!key)
efx_free_rss_context_entry(ctx);
else
*rss_context = ctx->user_id;
+out_unlock:
+ mutex_unlock(&efx->rss_lock);
return rc;
}
};
struct efx_farch_filter_state {
+ struct rw_semaphore lock; /* Protects table contents */
struct efx_farch_filter_table table[EFX_FARCH_FILTER_TABLE_COUNT];
};
if (rc)
return rc;
+ down_write(&state->lock);
+
table = &state->table[efx_farch_filter_spec_table_id(&spec)];
- if (table->size == 0)
- return -EINVAL;
+ if (table->size == 0) {
+ rc = -EINVAL;
+ goto out_unlock;
+ }
netif_vdbg(efx, hw, efx->net_dev,
"%s: type %d search_limit=%d", __func__, spec.type,
EFX_FARCH_FILTER_MC_DEF - EFX_FARCH_FILTER_UC_DEF);
rep_index = spec.type - EFX_FARCH_FILTER_UC_DEF;
ins_index = rep_index;
-
- spin_lock_bh(&efx->filter_lock);
} else {
/* Search concurrently for
* (1) a filter to be replaced (rep_index): any filter
ins_index = -1;
depth = 1;
- spin_lock_bh(&efx->filter_lock);
-
for (;;) {
if (!test_bit(i, table->used_bitmap)) {
if (ins_index < 0)
/* Case (b) */
if (ins_index < 0) {
rc = -EBUSY;
- goto out;
+ goto out_unlock;
}
rep_index = -1;
break;
if (spec.priority == saved_spec->priority && !replace_equal) {
rc = -EEXIST;
- goto out;
+ goto out_unlock;
}
if (spec.priority < saved_spec->priority) {
rc = -EPERM;
- goto out;
+ goto out_unlock;
}
if (saved_spec->priority == EFX_FILTER_PRI_AUTO ||
saved_spec->flags & EFX_FILTER_FLAG_RX_OVER_AUTO)
__func__, spec.type, ins_index, spec.dmaq_id);
rc = efx_farch_filter_make_id(&spec, ins_index);
-out:
- spin_unlock_bh(&efx->filter_lock);
+out_unlock:
+ up_write(&state->lock);
return rc;
}
filter_idx = efx_farch_filter_id_index(filter_id);
if (filter_idx >= table->size)
return -ENOENT;
+ down_write(&state->lock);
spec = &table->spec[filter_idx];
- spin_lock_bh(&efx->filter_lock);
rc = efx_farch_filter_remove(efx, table, filter_idx, priority);
- spin_unlock_bh(&efx->filter_lock);
+ up_write(&state->lock);
return rc;
}
struct efx_farch_filter_table *table;
struct efx_farch_filter_spec *spec;
unsigned int filter_idx;
- int rc;
+ int rc = -ENOENT;
+
+ down_read(&state->lock);
table_id = efx_farch_filter_id_table_id(filter_id);
if ((unsigned int)table_id >= EFX_FARCH_FILTER_TABLE_COUNT)
- return -ENOENT;
+ goto out_unlock;
table = &state->table[table_id];
filter_idx = efx_farch_filter_id_index(filter_id);
if (filter_idx >= table->size)
- return -ENOENT;
+ goto out_unlock;
spec = &table->spec[filter_idx];
- spin_lock_bh(&efx->filter_lock);
-
if (test_bit(filter_idx, table->used_bitmap) &&
spec->priority == priority) {
efx_farch_filter_to_gen_spec(spec_buf, spec);
rc = 0;
- } else {
- rc = -ENOENT;
}
- spin_unlock_bh(&efx->filter_lock);
-
+out_unlock:
+ up_read(&state->lock);
return rc;
}
struct efx_farch_filter_table *table = &state->table[table_id];
unsigned int filter_idx;
- spin_lock_bh(&efx->filter_lock);
+ down_write(&state->lock);
for (filter_idx = 0; filter_idx < table->size; ++filter_idx) {
if (table->spec[filter_idx].priority != EFX_FILTER_PRI_AUTO)
efx_farch_filter_remove(efx, table,
filter_idx, priority);
}
- spin_unlock_bh(&efx->filter_lock);
+ up_write(&state->lock);
}
int efx_farch_filter_clear_rx(struct efx_nic *efx,
unsigned int filter_idx;
u32 count = 0;
- spin_lock_bh(&efx->filter_lock);
+ down_read(&state->lock);
for (table_id = EFX_FARCH_FILTER_TABLE_RX_IP;
table_id <= EFX_FARCH_FILTER_TABLE_RX_DEF;
}
}
- spin_unlock_bh(&efx->filter_lock);
+ up_read(&state->lock);
return count;
}
unsigned int filter_idx;
s32 count = 0;
- spin_lock_bh(&efx->filter_lock);
+ down_read(&state->lock);
for (table_id = EFX_FARCH_FILTER_TABLE_RX_IP;
table_id <= EFX_FARCH_FILTER_TABLE_RX_DEF;
}
}
out:
- spin_unlock_bh(&efx->filter_lock);
+ up_read(&state->lock);
return count;
}
efx_oword_t filter;
unsigned int filter_idx;
- spin_lock_bh(&efx->filter_lock);
+ down_write(&state->lock);
for (table_id = 0; table_id < EFX_FARCH_FILTER_TABLE_COUNT; table_id++) {
table = &state->table[table_id];
efx_farch_filter_push_rx_config(efx);
efx_farch_filter_push_tx_limits(efx);
- spin_unlock_bh(&efx->filter_lock);
+ up_write(&state->lock);
}
void efx_farch_filter_table_remove(struct efx_nic *efx)
efx_oword_t filter;
unsigned int filter_idx;
- spin_lock_bh(&efx->filter_lock);
+ down_write(&state->lock);
for (table_id = EFX_FARCH_FILTER_TABLE_RX_IP;
table_id <= EFX_FARCH_FILTER_TABLE_RX_DEF;
efx_farch_filter_push_rx_config(efx);
- spin_unlock_bh(&efx->filter_lock);
+ up_write(&state->lock);
}
#ifdef CONFIG_RFS_ACCEL
-s32 efx_farch_filter_rfs_insert(struct efx_nic *efx,
- struct efx_filter_spec *gen_spec)
-{
- return efx_farch_filter_insert(efx, gen_spec, true);
-}
-
bool efx_farch_filter_rfs_expire_one(struct efx_nic *efx, u32 flow_id,
unsigned int index)
{
struct efx_farch_filter_state *state = efx->filter_state;
- struct efx_farch_filter_table *table =
- &state->table[EFX_FARCH_FILTER_TABLE_RX_IP];
+ struct efx_farch_filter_table *table;
+ bool ret = false;
+ down_write(&state->lock);
+ table = &state->table[EFX_FARCH_FILTER_TABLE_RX_IP];
if (test_bit(index, table->used_bitmap) &&
table->spec[index].priority == EFX_FILTER_PRI_HINT &&
rps_may_expire_flow(efx->net_dev, table->spec[index].dmaq_id,
flow_id, index)) {
efx_farch_filter_table_clear_entry(efx, table, index);
- return true;
+ ret = true;
}
- return false;
+ up_write(&state->lock);
+ return ret;
}
#endif /* CONFIG_RFS_ACCEL */
attr->limit_value = limit_value;
sysfs_attr_init(&attr->dev_attr.attr);
attr->dev_attr.attr.name = attr->name;
- attr->dev_attr.attr.mode = S_IRUGO;
+ attr->dev_attr.attr.mode = 0444;
attr->dev_attr.show = reader;
hwmon->group.attrs[hwmon->n_attrs++] = &attr->dev_attr.attr;
}
* @event_test_cpu: Last CPU to handle interrupt or test event for this channel
* @irq_count: Number of IRQs since last adaptive moderation decision
* @irq_mod_score: IRQ moderation score
+ * @filter_work: Work item for efx_filter_rfs_expire()
* @rps_flow_id: Flow IDs of filters allocated for accelerated RFS,
* indexed by filter ID
* @n_rx_tobe_disc: Count of RX_TOBE_DISC errors
unsigned int irq_mod_score;
#ifdef CONFIG_RFS_ACCEL
unsigned int rfs_filters_added;
+ struct work_struct filter_work;
#define RPS_FLOW_ID_INVALID 0xFFFFFFFF
u32 *rps_flow_id;
#endif
* @rx_scatter: Scatter mode enabled for receives
* @rss_context: Main RSS context. Its @list member is the head of the list of
* RSS contexts created by user requests
+ * @rss_lock: Protects custom RSS context software state in @rss_context.list
* @int_error_count: Number of internal errors seen recently
* @int_error_expire: Time at which error count will be expired
* @irq_soft_enabled: Are IRQs soft-enabled? If not, IRQ handler will
* @loopback_mode: Loopback status
* @loopback_modes: Supported loopback mode bitmask
* @loopback_selftest: Offline self-test private state
- * @filter_sem: Filter table rw_semaphore, for freeing the table
- * @filter_lock: Filter table lock, for mere content changes
+ * @filter_sem: Filter table rw_semaphore, protects existence of @filter_state
* @filter_state: Architecture-dependent filter table state
+ * @rps_mutex: Protects RPS state of all channels
* @rps_expire_channel: Next channel to check for expiry
* @rps_expire_index: Next index to check for expiry in
* @rps_expire_channel's @rps_flow_id
int rx_packet_ts_offset;
bool rx_scatter;
struct efx_rss_context rss_context;
+ struct mutex rss_lock;
unsigned int_error_count;
unsigned long int_error_expire;
void *loopback_selftest;
struct rw_semaphore filter_sem;
- spinlock_t filter_lock;
void *filter_state;
#ifdef CONFIG_RFS_ACCEL
+ struct mutex rps_mutex;
unsigned int rps_expire_channel;
unsigned int rps_expire_index;
#endif
* @filter_count_rx_used: Get the number of filters in use at a given priority
* @filter_get_rx_id_limit: Get maximum value of a filter id, plus 1
* @filter_get_rx_ids: Get list of RX filters at a given priority
- * @filter_rfs_insert: Add or replace a filter for RFS. This must be
- * atomic. The hardware change may be asynchronous but should
- * not be delayed for long. It may fail if this can't be done
- * atomically.
* @filter_rfs_expire_one: Consider expiring a filter inserted for RFS.
* This must check whether the specified table entry is used by RFS
* and that rps_may_expire_flow() returns true for it.
enum efx_filter_priority priority,
u32 *buf, u32 size);
#ifdef CONFIG_RFS_ACCEL
- s32 (*filter_rfs_insert)(struct efx_nic *efx,
- struct efx_filter_spec *spec);
bool (*filter_rfs_expire_one)(struct efx_nic *efx, u32 flow_id,
unsigned int index);
#endif
* @vi_base: Absolute index of first VI in this function
* @n_allocated_vis: Number of VIs allocated to this function
* @must_realloc_vis: Flag: VIs have yet to be reallocated after MC reboot
+ * @must_restore_rss_contexts: Flag: RSS contexts have yet to be restored after
+ * MC reboot
* @must_restore_filters: Flag: filters have yet to be restored after MC reboot
* @n_piobufs: Number of PIO buffers allocated to this function
* @wc_membase: Base address of write-combining mapping of the memory BAR
unsigned int vi_base;
unsigned int n_allocated_vis;
bool must_realloc_vis;
+ bool must_restore_rss_contexts;
bool must_restore_filters;
unsigned int n_piobufs;
void __iomem *wc_membase, *pio_write_base;
enum efx_filter_priority priority, u32 *buf,
u32 size);
#ifdef CONFIG_RFS_ACCEL
-s32 efx_farch_filter_rfs_insert(struct efx_nic *efx,
- struct efx_filter_spec *spec);
bool efx_farch_filter_rfs_expire_one(struct efx_nic *efx, u32 flow_id,
unsigned int index);
#endif
#ifdef CONFIG_RFS_ACCEL
+/**
+ * struct efx_async_filter_insertion - Request to asynchronously insert a filter
+ * @net_dev: Reference to the netdevice
+ * @spec: The filter to insert
+ * @work: Workitem for this request
+ * @rxq_index: Identifies the channel for which this request was made
+ * @flow_id: Identifies the kernel-side flow for which this request was made
+ */
+struct efx_async_filter_insertion {
+ struct net_device *net_dev;
+ struct efx_filter_spec spec;
+ struct work_struct work;
+ u16 rxq_index;
+ u32 flow_id;
+};
+
+static void efx_filter_rfs_work(struct work_struct *data)
+{
+ struct efx_async_filter_insertion *req = container_of(data, struct efx_async_filter_insertion,
+ work);
+ struct efx_nic *efx = netdev_priv(req->net_dev);
+ struct efx_channel *channel = efx_get_channel(efx, req->rxq_index);
+ int rc;
+
+ rc = efx->type->filter_insert(efx, &req->spec, false);
+ if (rc >= 0) {
+ /* Remember this so we can check whether to expire the filter
+ * later.
+ */
+ mutex_lock(&efx->rps_mutex);
+ channel->rps_flow_id[rc] = req->flow_id;
+ ++channel->rfs_filters_added;
+ mutex_unlock(&efx->rps_mutex);
+
+ if (req->spec.ether_type == htons(ETH_P_IP))
+ netif_info(efx, rx_status, efx->net_dev,
+ "steering %s %pI4:%u:%pI4:%u to queue %u [flow %u filter %d]\n",
+ (req->spec.ip_proto == IPPROTO_TCP) ? "TCP" : "UDP",
+ req->spec.rem_host, ntohs(req->spec.rem_port),
+ req->spec.loc_host, ntohs(req->spec.loc_port),
+ req->rxq_index, req->flow_id, rc);
+ else
+ netif_info(efx, rx_status, efx->net_dev,
+ "steering %s [%pI6]:%u:[%pI6]:%u to queue %u [flow %u filter %d]\n",
+ (req->spec.ip_proto == IPPROTO_TCP) ? "TCP" : "UDP",
+ req->spec.rem_host, ntohs(req->spec.rem_port),
+ req->spec.loc_host, ntohs(req->spec.loc_port),
+ req->rxq_index, req->flow_id, rc);
+ }
+
+ /* Release references */
+ dev_put(req->net_dev);
+ kfree(req);
+}
+
int efx_filter_rfs(struct net_device *net_dev, const struct sk_buff *skb,
u16 rxq_index, u32 flow_id)
{
struct efx_nic *efx = netdev_priv(net_dev);
- struct efx_channel *channel;
- struct efx_filter_spec spec;
+ struct efx_async_filter_insertion *req;
struct flow_keys fk;
- int rc;
if (flow_id == RPS_FLOW_ID_INVALID)
return -EINVAL;
if (fk.control.flags & FLOW_DIS_IS_FRAGMENT)
return -EPROTONOSUPPORT;
- efx_filter_init_rx(&spec, EFX_FILTER_PRI_HINT,
+ req = kmalloc(sizeof(*req), GFP_ATOMIC);
+ if (!req)
+ return -ENOMEM;
+
+ efx_filter_init_rx(&req->spec, EFX_FILTER_PRI_HINT,
efx->rx_scatter ? EFX_FILTER_FLAG_RX_SCATTER : 0,
rxq_index);
- spec.match_flags =
+ req->spec.match_flags =
EFX_FILTER_MATCH_ETHER_TYPE | EFX_FILTER_MATCH_IP_PROTO |
EFX_FILTER_MATCH_LOC_HOST | EFX_FILTER_MATCH_LOC_PORT |
EFX_FILTER_MATCH_REM_HOST | EFX_FILTER_MATCH_REM_PORT;
- spec.ether_type = fk.basic.n_proto;
- spec.ip_proto = fk.basic.ip_proto;
+ req->spec.ether_type = fk.basic.n_proto;
+ req->spec.ip_proto = fk.basic.ip_proto;
if (fk.basic.n_proto == htons(ETH_P_IP)) {
- spec.rem_host[0] = fk.addrs.v4addrs.src;
- spec.loc_host[0] = fk.addrs.v4addrs.dst;
+ req->spec.rem_host[0] = fk.addrs.v4addrs.src;
+ req->spec.loc_host[0] = fk.addrs.v4addrs.dst;
} else {
- memcpy(spec.rem_host, &fk.addrs.v6addrs.src, sizeof(struct in6_addr));
- memcpy(spec.loc_host, &fk.addrs.v6addrs.dst, sizeof(struct in6_addr));
+ memcpy(req->spec.rem_host, &fk.addrs.v6addrs.src,
+ sizeof(struct in6_addr));
+ memcpy(req->spec.loc_host, &fk.addrs.v6addrs.dst,
+ sizeof(struct in6_addr));
}
- spec.rem_port = fk.ports.src;
- spec.loc_port = fk.ports.dst;
-
- rc = efx->type->filter_rfs_insert(efx, &spec);
- if (rc < 0)
- return rc;
+ req->spec.rem_port = fk.ports.src;
+ req->spec.loc_port = fk.ports.dst;
- /* Remember this so we can check whether to expire the filter later */
- channel = efx_get_channel(efx, rxq_index);
- channel->rps_flow_id[rc] = flow_id;
- ++channel->rfs_filters_added;
-
- if (spec.ether_type == htons(ETH_P_IP))
- netif_info(efx, rx_status, efx->net_dev,
- "steering %s %pI4:%u:%pI4:%u to queue %u [flow %u filter %d]\n",
- (spec.ip_proto == IPPROTO_TCP) ? "TCP" : "UDP",
- spec.rem_host, ntohs(spec.rem_port), spec.loc_host,
- ntohs(spec.loc_port), rxq_index, flow_id, rc);
- else
- netif_info(efx, rx_status, efx->net_dev,
- "steering %s [%pI6]:%u:[%pI6]:%u to queue %u [flow %u filter %d]\n",
- (spec.ip_proto == IPPROTO_TCP) ? "TCP" : "UDP",
- spec.rem_host, ntohs(spec.rem_port), spec.loc_host,
- ntohs(spec.loc_port), rxq_index, flow_id, rc);
-
- return rc;
+ dev_hold(req->net_dev = net_dev);
+ INIT_WORK(&req->work, efx_filter_rfs_work);
+ req->rxq_index = rxq_index;
+ req->flow_id = flow_id;
+ schedule_work(&req->work);
+ return 0;
}
bool __efx_filter_rfs_expire(struct efx_nic *efx, unsigned int quota)
unsigned int channel_idx, index, size;
u32 flow_id;
- if (!spin_trylock_bh(&efx->filter_lock))
+ if (!mutex_trylock(&efx->rps_mutex))
return false;
-
expire_one = efx->type->filter_rfs_expire_one;
channel_idx = efx->rps_expire_channel;
index = efx->rps_expire_index;
efx->rps_expire_channel = channel_idx;
efx->rps_expire_index = index;
- spin_unlock_bh(&efx->filter_lock);
+ mutex_unlock(&efx->rps_mutex);
return true;
}
.filter_get_rx_id_limit = efx_farch_filter_get_rx_id_limit,
.filter_get_rx_ids = efx_farch_filter_get_rx_ids,
#ifdef CONFIG_RFS_ACCEL
- .filter_rfs_insert = efx_farch_filter_rfs_insert,
.filter_rfs_expire_one = efx_farch_filter_rfs_expire_one,
#endif
#ifdef CONFIG_SFC_MTD
pdata = netdev_priv(dev);
BUG_ON(!pdata);
BUG_ON(!pdata->ioaddr);
- WARN_ON(dev->phydev);
SMSC_TRACE(pdata, ifdown, "Stopping driver");
+ unregister_netdev(dev);
+
mdiobus_unregister(pdata->mii_bus);
mdiobus_free(pdata->mii_bus);
- unregister_netdev(dev);
res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
"smsc911x-memory");
if (!res)
val |= AVE_IIRQC_EN0 | (AVE_INTM_COUNT << 16);
writel(val, priv->base + AVE_IIRQC);
- val = AVE_GI_RXIINT | AVE_GI_RXOVF | AVE_GI_TX;
+ val = AVE_GI_RXIINT | AVE_GI_RXOVF | AVE_GI_TX | AVE_GI_RXDROP;
ave_irq_restore(ndev, val);
napi_enable(&priv->napi_rx);
.is_desc_64bit = true,
};
+static const struct ave_soc_data ave_pxs3_data = {
+ .is_desc_64bit = false,
+};
+
static const struct of_device_id of_ave_match[] = {
{
.compatible = "socionext,uniphier-pro4-ave4",
.compatible = "socionext,uniphier-ld20-ave4",
.data = &ave_ld20_data,
},
+ {
+ .compatible = "socionext,uniphier-pxs3-ave4",
+ .data = &ave_pxs3_data,
+ },
{ /* Sentinel */ }
};
MODULE_DEVICE_TABLE(of, of_ave_match);
/* Module parameters */
#define TX_TIMEO 5000
static int watchdog = TX_TIMEO;
-module_param(watchdog, int, S_IRUGO | S_IWUSR);
+module_param(watchdog, int, 0644);
MODULE_PARM_DESC(watchdog, "Transmit timeout in milliseconds (default 5s)");
static int debug = -1;
-module_param(debug, int, S_IRUGO | S_IWUSR);
+module_param(debug, int, 0644);
MODULE_PARM_DESC(debug, "Message Level (-1: default, 0: no output, 16: all)");
static int phyaddr = -1;
-module_param(phyaddr, int, S_IRUGO);
+module_param(phyaddr, int, 0444);
MODULE_PARM_DESC(phyaddr, "Physical device address");
#define STMMAC_TX_THRESH (DMA_TX_SIZE / 4)
#define STMMAC_RX_THRESH (DMA_RX_SIZE / 4)
static int flow_ctrl = FLOW_OFF;
-module_param(flow_ctrl, int, S_IRUGO | S_IWUSR);
+module_param(flow_ctrl, int, 0644);
MODULE_PARM_DESC(flow_ctrl, "Flow control ability [on/off]");
static int pause = PAUSE_TIME;
-module_param(pause, int, S_IRUGO | S_IWUSR);
+module_param(pause, int, 0644);
MODULE_PARM_DESC(pause, "Flow Control Pause Time");
#define TC_DEFAULT 64
static int tc = TC_DEFAULT;
-module_param(tc, int, S_IRUGO | S_IWUSR);
+module_param(tc, int, 0644);
MODULE_PARM_DESC(tc, "DMA threshold control value");
#define DEFAULT_BUFSIZE 1536
static int buf_sz = DEFAULT_BUFSIZE;
-module_param(buf_sz, int, S_IRUGO | S_IWUSR);
+module_param(buf_sz, int, 0644);
MODULE_PARM_DESC(buf_sz, "DMA buffer size");
#define STMMAC_RX_COPYBREAK 256
#define STMMAC_DEFAULT_LPI_TIMER 1000
static int eee_timer = STMMAC_DEFAULT_LPI_TIMER;
-module_param(eee_timer, int, S_IRUGO | S_IWUSR);
+module_param(eee_timer, int, 0644);
MODULE_PARM_DESC(eee_timer, "LPI tx expiration time in msec");
#define STMMAC_LPI_T(x) (jiffies + msecs_to_jiffies(x))
* but allow user to force to use the chain instead of the ring
*/
static unsigned int chain_mode;
-module_param(chain_mode, int, S_IRUGO);
+module_param(chain_mode, int, 0444);
MODULE_PARM_DESC(chain_mode, "To use chain instead of ring mode");
static irqreturn_t stmmac_interrupt(int irq, void *dev_id);
/* Entry to report DMA RX/TX rings */
priv->dbgfs_rings_status =
- debugfs_create_file("descriptors_status", S_IRUGO,
+ debugfs_create_file("descriptors_status", 0444,
priv->dbgfs_dir, dev,
&stmmac_rings_status_fops);
}
/* Entry to report the DMA HW features */
- priv->dbgfs_dma_cap = debugfs_create_file("dma_cap", S_IRUGO,
- priv->dbgfs_dir,
- dev, &stmmac_dma_cap_fops);
+ priv->dbgfs_dma_cap = debugfs_create_file("dma_cap", 0444,
+ priv->dbgfs_dir,
+ dev, &stmmac_dma_cap_fops);
if (!priv->dbgfs_dma_cap || IS_ERR(priv->dbgfs_dma_cap)) {
netdev_err(priv->dev, "ERROR creating stmmac MMC debugfs file\n");
}
static struct device_attribute niu_parent_attributes[] = {
- __ATTR(port_phy, S_IRUGO, show_port_phy, NULL),
- __ATTR(plat_type, S_IRUGO, show_plat_type, NULL),
- __ATTR(rxchan_per_port, S_IRUGO, show_rxchan_per_port, NULL),
- __ATTR(txchan_per_port, S_IRUGO, show_txchan_per_port, NULL),
- __ATTR(num_ports, S_IRUGO, show_num_ports, NULL),
+ __ATTR(port_phy, 0444, show_port_phy, NULL),
+ __ATTR(plat_type, 0444, show_plat_type, NULL),
+ __ATTR(rxchan_per_port, 0444, show_rxchan_per_port, NULL),
+ __ATTR(txchan_per_port, 0444, show_txchan_per_port, NULL),
+ __ATTR(num_ports, 0444, show_num_ports, NULL),
{}
};
dev->ethtool_ops = &vnet_ethtool_ops;
dev->watchdog_timeo = VNET_TX_TIMEOUT;
- dev->hw_features = NETIF_F_TSO | NETIF_F_GSO | NETIF_F_GSO_SOFTWARE |
+ dev->hw_features = NETIF_F_TSO | NETIF_F_GSO | NETIF_F_ALL_TSO |
NETIF_F_HW_CSUM | NETIF_F_SG;
dev->features = dev->hw_features;
/* set speed_in input in case RMII mode is used in 100Mbps */
if (phy->speed == 100)
mac_control |= BIT(15);
- else if (phy->speed == 10)
+ /* in band mode only works in 10Mbps RGMII mode */
+ else if ((phy->speed == 10) && phy_interface_is_rgmii(phy))
mac_control |= BIT(18); /* In Band mode */
if (priv->rx_pause)
.exit_batch = geneve_exit_batch_net,
.id = &geneve_net_id,
.size = sizeof(struct geneve_net),
- .async = true,
};
static int __init geneve_init_module(void)
.exit = gtp_net_exit,
.id = >p_net_id,
.size = sizeof(struct gtp_net),
- .async = true,
};
static int __init gtp_init(void)
static int __init bpq_init_driver(void)
{
#ifdef CONFIG_PROC_FS
- if (!proc_create("bpqether", S_IRUGO, init_net.proc_net,
- &bpq_info_fops)) {
+ if (!proc_create("bpqether", 0444, init_net.proc_net, &bpq_info_fops)) {
printk(KERN_ERR
"bpq: cannot create /proc/net/bpqether entry.\n");
return -ENOENT;
yam_timer.expires = jiffies + HZ / 100;
add_timer(&yam_timer);
- proc_create("yam", S_IRUGO, init_net.proc_net, &yam_info_fops);
+ proc_create("yam", 0444, init_net.proc_net, &yam_info_fops);
return 0;
error:
while (--i >= 0) {
struct list_head req_list;
struct work_struct mcast_work;
+ u32 filter;
bool link_state; /* 0 - link up, 1 - link down */
int netvsc_poll(struct napi_struct *napi, int budget);
void rndis_set_subchannel(struct work_struct *w);
-bool rndis_filter_opened(const struct netvsc_device *nvdev);
int rndis_filter_open(struct netvsc_device *nvdev);
int rndis_filter_close(struct netvsc_device *nvdev);
struct netvsc_device *rndis_filter_device_add(struct hv_device *dev,
/* Receive buffer allocated by us but manages by NetVSP */
void *recv_buf;
+ u32 recv_buf_size; /* allocated bytes */
u32 recv_buf_gpadl_handle;
u32 recv_section_cnt;
u32 recv_section_size;
= container_of(head, struct netvsc_device, rcu);
int i;
+ kfree(nvdev->extension);
+ vfree(nvdev->recv_buf);
+ vfree(nvdev->send_buf);
+ kfree(nvdev->send_section_map);
+
for (i = 0; i < VRSS_CHANNEL_MAX; i++)
vfree(nvdev->chan_table[i].mrc.slots);
net_device->recv_buf_gpadl_handle = 0;
}
- if (net_device->recv_buf) {
- /* Free up the receive buffer */
- vfree(net_device->recv_buf);
- net_device->recv_buf = NULL;
- }
-
if (net_device->send_buf_gpadl_handle) {
ret = vmbus_teardown_gpadl(device->channel,
net_device->send_buf_gpadl_handle);
}
net_device->send_buf_gpadl_handle = 0;
}
- if (net_device->send_buf) {
- /* Free up the send buffer */
- vfree(net_device->send_buf);
- net_device->send_buf = NULL;
- }
- kfree(net_device->send_section_map);
}
int netvsc_alloc_recv_comp_ring(struct netvsc_device *net_device, u32 q_idx)
goto cleanup;
}
+ net_device->recv_buf_size = buf_size;
+
/*
* Establish the gpadl handle for this buffer on this
* channel. Note: This call uses the vmbus connection rather
= rtnl_dereference(net_device_ctx->nvdev);
int i;
- cancel_work_sync(&net_device->subchan_work);
-
netvsc_revoke_buf(device, net_device);
RCU_INIT_POINTER(net_device_ctx->nvdev, NULL);
+ /* And disassociate NAPI context from device */
+ for (i = 0; i < net_device->num_chn; i++)
+ netif_napi_del(&net_device->chan_table[i].napi);
+
/*
* At this point, no one should be accessing net_device
* except in here
*/
netdev_dbg(ndev, "net device safe to remove\n");
+ /* older versions require that buffer be revoked before close */
+ if (net_device->nvsp_version < NVSP_PROTOCOL_VERSION_4)
+ netvsc_teardown_gpadl(device, net_device);
+
/* Now, we can close the channel safely */
vmbus_close(device->channel);
- netvsc_teardown_gpadl(device, net_device);
-
- /* And dissassociate NAPI context from device */
- for (i = 0; i < net_device->num_chn; i++)
- netif_napi_del(&net_device->chan_table[i].napi);
+ if (net_device->nvsp_version >= NVSP_PROTOCOL_VERSION_4)
+ netvsc_teardown_gpadl(device, net_device);
/* Release all resources */
free_netvsc_device_rcu(net_device);
queue_sends =
atomic_dec_return(&net_device->chan_table[q_idx].queue_sends);
- if (net_device->destroy && queue_sends == 0)
- wake_up(&net_device->wait_drain);
+ if (unlikely(net_device->destroy)) {
+ if (queue_sends == 0)
+ wake_up(&net_device->wait_drain);
+ } else {
+ struct netdev_queue *txq = netdev_get_tx_queue(ndev, q_idx);
- if (netif_tx_queue_stopped(netdev_get_tx_queue(ndev, q_idx)) &&
- (hv_ringbuf_avail_percent(&channel->outbound) > RING_AVAIL_PERCENT_HIWATER ||
- queue_sends < 1)) {
- netif_tx_wake_queue(netdev_get_tx_queue(ndev, q_idx));
- ndev_ctx->eth_stats.wake_queue++;
+ if (netif_tx_queue_stopped(txq) &&
+ (hv_ringbuf_avail_percent(&channel->outbound) > RING_AVAIL_PERCENT_HIWATER ||
+ queue_sends < 1)) {
+ netif_tx_wake_queue(txq);
+ ndev_ctx->eth_stats.wake_queue++;
+ }
}
}
/* Each range represents 1 RNDIS pkt that contains 1 ethernet frame */
for (i = 0; i < count; i++) {
- void *data = recv_buf
- + vmxferpage_packet->ranges[i].byte_offset;
+ u32 offset = vmxferpage_packet->ranges[i].byte_offset;
u32 buflen = vmxferpage_packet->ranges[i].byte_count;
+ void *data;
+ int ret;
+
+ if (unlikely(offset + buflen > net_device->recv_buf_size)) {
+ status = NVSP_STAT_FAIL;
+ netif_err(net_device_ctx, rx_err, ndev,
+ "Packet offset:%u + len:%u too big\n",
+ offset, buflen);
+
+ continue;
+ }
+
+ data = recv_buf + offset;
trace_rndis_recv(ndev, q_idx, data);
/* Pass it to the upper layer */
- status = rndis_filter_receive(ndev, net_device,
- channel, data, buflen);
+ ret = rndis_filter_receive(ndev, net_device,
+ channel, data, buflen);
+
+ if (unlikely(ret != NVSP_STAT_SUCCESS))
+ status = NVSP_STAT_FAIL;
}
enq_receive_complete(ndev, net_device, q_idx,
#include "hyperv_net.h"
-#define RING_SIZE_MIN 64
+#define RING_SIZE_MIN 64
+#define RETRY_US_LO 5000
+#define RETRY_US_HI 10000
+#define RETRY_MAX 2000 /* >10 sec */
#define LINKCHANGE_INT (2 * HZ)
#define VF_TAKEOVER_INT (HZ / 10)
static unsigned int ring_size __ro_after_init = 128;
-module_param(ring_size, uint, S_IRUGO);
+module_param(ring_size, uint, 0444);
MODULE_PARM_DESC(ring_size, "Ring buffer size (# of pages)");
unsigned int netvsc_ring_bytes __ro_after_init;
struct reciprocal_value netvsc_ring_reciprocal __ro_after_init;
NETIF_MSG_TX_ERR;
static int debug = -1;
-module_param(debug, int, S_IRUGO);
+module_param(debug, int, 0444);
MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)");
static void netvsc_change_rx_flags(struct net_device *net, int change)
static void netvsc_set_rx_mode(struct net_device *net)
{
struct net_device_context *ndev_ctx = netdev_priv(net);
- struct net_device *vf_netdev = rtnl_dereference(ndev_ctx->vf_netdev);
- struct netvsc_device *nvdev = rtnl_dereference(ndev_ctx->nvdev);
+ struct net_device *vf_netdev;
+ struct netvsc_device *nvdev;
+ rcu_read_lock();
+ vf_netdev = rcu_dereference(ndev_ctx->vf_netdev);
if (vf_netdev) {
dev_uc_sync(vf_netdev, net);
dev_mc_sync(vf_netdev, net);
}
- rndis_filter_update(nvdev);
+ nvdev = rcu_dereference(ndev_ctx->nvdev);
+ if (nvdev)
+ rndis_filter_update(nvdev);
+ rcu_read_unlock();
}
static int netvsc_open(struct net_device *net)
}
rdev = nvdev->extension;
- if (!rdev->link_state) {
+ if (!rdev->link_state)
netif_carrier_on(net);
- netif_tx_wake_all_queues(net);
- }
if (vf_netdev) {
/* Setting synthetic device up transparently sets
return 0;
}
-static int netvsc_close(struct net_device *net)
+static int netvsc_wait_until_empty(struct netvsc_device *nvdev)
{
- struct net_device_context *net_device_ctx = netdev_priv(net);
- struct net_device *vf_netdev
- = rtnl_dereference(net_device_ctx->vf_netdev);
- struct netvsc_device *nvdev = rtnl_dereference(net_device_ctx->nvdev);
- int ret = 0;
- u32 aread, i, msec = 10, retry = 0, retry_max = 20;
- struct vmbus_channel *chn;
-
- netif_tx_disable(net);
-
- /* No need to close rndis filter if it is removed already */
- if (!nvdev)
- goto out;
-
- ret = rndis_filter_close(nvdev);
- if (ret != 0) {
- netdev_err(net, "unable to close device (ret %d).\n", ret);
- return ret;
- }
+ unsigned int retry = 0;
+ int i;
/* Ensure pending bytes in ring are read */
- while (true) {
- aread = 0;
+ for (;;) {
+ u32 aread = 0;
+
for (i = 0; i < nvdev->num_chn; i++) {
- chn = nvdev->chan_table[i].channel;
+ struct vmbus_channel *chn
+ = nvdev->chan_table[i].channel;
+
if (!chn)
continue;
+ /* make sure receive not running now */
+ napi_synchronize(&nvdev->chan_table[i].napi);
+
aread = hv_get_bytes_to_read(&chn->inbound);
if (aread)
break;
break;
}
- retry++;
- if (retry > retry_max || aread == 0)
- break;
+ if (aread == 0)
+ return 0;
- msleep(msec);
+ if (++retry > RETRY_MAX)
+ return -ETIMEDOUT;
- if (msec < 1000)
- msec *= 2;
+ usleep_range(RETRY_US_LO, RETRY_US_HI);
}
+}
- if (aread) {
- netdev_err(net, "Ring buffer not empty after closing rndis\n");
- ret = -ETIMEDOUT;
+static int netvsc_close(struct net_device *net)
+{
+ struct net_device_context *net_device_ctx = netdev_priv(net);
+ struct net_device *vf_netdev
+ = rtnl_dereference(net_device_ctx->vf_netdev);
+ struct netvsc_device *nvdev = rtnl_dereference(net_device_ctx->nvdev);
+ int ret;
+
+ netif_tx_disable(net);
+
+ /* No need to close rndis filter if it is removed already */
+ if (!nvdev)
+ return 0;
+
+ ret = rndis_filter_close(nvdev);
+ if (ret != 0) {
+ netdev_err(net, "unable to close device (ret %d).\n", ret);
+ return ret;
}
-out:
+ ret = netvsc_wait_until_empty(nvdev);
+ if (ret)
+ netdev_err(net, "Ring buffer not empty after closing rndis\n");
+
if (vf_netdev)
dev_close(vf_netdev);
u64_stats_update_end(&rx_stats->syncp);
napi_gro_receive(&nvchan->napi, skb);
- return 0;
+ return NVSP_STAT_SUCCESS;
}
static void netvsc_get_drvinfo(struct net_device *net,
}
}
+static int netvsc_detach(struct net_device *ndev,
+ struct netvsc_device *nvdev)
+{
+ struct net_device_context *ndev_ctx = netdev_priv(ndev);
+ struct hv_device *hdev = ndev_ctx->device_ctx;
+ int ret;
+
+ /* Don't try continuing to try and setup sub channels */
+ if (cancel_work_sync(&nvdev->subchan_work))
+ nvdev->num_chn = 1;
+
+ /* If device was up (receiving) then shutdown */
+ if (netif_running(ndev)) {
+ netif_tx_disable(ndev);
+
+ ret = rndis_filter_close(nvdev);
+ if (ret) {
+ netdev_err(ndev,
+ "unable to close device (ret %d).\n", ret);
+ return ret;
+ }
+
+ ret = netvsc_wait_until_empty(nvdev);
+ if (ret) {
+ netdev_err(ndev,
+ "Ring buffer not empty after closing rndis\n");
+ return ret;
+ }
+ }
+
+ netif_device_detach(ndev);
+
+ rndis_filter_device_remove(hdev, nvdev);
+
+ return 0;
+}
+
+static int netvsc_attach(struct net_device *ndev,
+ struct netvsc_device_info *dev_info)
+{
+ struct net_device_context *ndev_ctx = netdev_priv(ndev);
+ struct hv_device *hdev = ndev_ctx->device_ctx;
+ struct netvsc_device *nvdev;
+ struct rndis_device *rdev;
+ int ret;
+
+ nvdev = rndis_filter_device_add(hdev, dev_info);
+ if (IS_ERR(nvdev))
+ return PTR_ERR(nvdev);
+
+ /* Note: enable and attach happen when sub-channels setup */
+
+ netif_carrier_off(ndev);
+
+ if (netif_running(ndev)) {
+ ret = rndis_filter_open(nvdev);
+ if (ret)
+ return ret;
+
+ rdev = nvdev->extension;
+ if (!rdev->link_state)
+ netif_carrier_on(ndev);
+ }
+
+ return 0;
+}
+
static int netvsc_set_channels(struct net_device *net,
struct ethtool_channels *channels)
{
struct net_device_context *net_device_ctx = netdev_priv(net);
- struct hv_device *dev = net_device_ctx->device_ctx;
struct netvsc_device *nvdev = rtnl_dereference(net_device_ctx->nvdev);
unsigned int orig, count = channels->combined_count;
struct netvsc_device_info device_info;
- bool was_opened;
- int ret = 0;
+ int ret;
/* We do not support separate count for rx, tx, or other */
if (count == 0 ||
return -EINVAL;
orig = nvdev->num_chn;
- was_opened = rndis_filter_opened(nvdev);
- if (was_opened)
- rndis_filter_close(nvdev);
memset(&device_info, 0, sizeof(device_info));
device_info.num_chn = count;
device_info.recv_sections = nvdev->recv_section_cnt;
device_info.recv_section_size = nvdev->recv_section_size;
- rndis_filter_device_remove(dev, nvdev);
+ ret = netvsc_detach(net, nvdev);
+ if (ret)
+ return ret;
- nvdev = rndis_filter_device_add(dev, &device_info);
- if (IS_ERR(nvdev)) {
- ret = PTR_ERR(nvdev);
+ ret = netvsc_attach(net, &device_info);
+ if (ret) {
device_info.num_chn = orig;
- nvdev = rndis_filter_device_add(dev, &device_info);
-
- if (IS_ERR(nvdev)) {
- netdev_err(net, "restoring channel setting failed: %ld\n",
- PTR_ERR(nvdev));
- return ret;
- }
+ if (netvsc_attach(net, &device_info))
+ netdev_err(net, "restoring channel setting failed\n");
}
- if (was_opened)
- rndis_filter_open(nvdev);
-
- /* We may have missed link change notifications */
- net_device_ctx->last_reconfig = 0;
- schedule_delayed_work(&net_device_ctx->dwork, 0);
-
return ret;
}
struct net_device_context *ndevctx = netdev_priv(ndev);
struct net_device *vf_netdev = rtnl_dereference(ndevctx->vf_netdev);
struct netvsc_device *nvdev = rtnl_dereference(ndevctx->nvdev);
- struct hv_device *hdev = ndevctx->device_ctx;
int orig_mtu = ndev->mtu;
struct netvsc_device_info device_info;
- bool was_opened;
int ret = 0;
if (!nvdev || nvdev->destroy)
return ret;
}
- netif_device_detach(ndev);
- was_opened = rndis_filter_opened(nvdev);
- if (was_opened)
- rndis_filter_close(nvdev);
-
memset(&device_info, 0, sizeof(device_info));
device_info.num_chn = nvdev->num_chn;
device_info.send_sections = nvdev->send_section_cnt;
device_info.recv_sections = nvdev->recv_section_cnt;
device_info.recv_section_size = nvdev->recv_section_size;
- rndis_filter_device_remove(hdev, nvdev);
+ ret = netvsc_detach(ndev, nvdev);
+ if (ret)
+ goto rollback_vf;
ndev->mtu = mtu;
- nvdev = rndis_filter_device_add(hdev, &device_info);
- if (IS_ERR(nvdev)) {
- ret = PTR_ERR(nvdev);
-
- /* Attempt rollback to original MTU */
- ndev->mtu = orig_mtu;
- nvdev = rndis_filter_device_add(hdev, &device_info);
-
- if (vf_netdev)
- dev_set_mtu(vf_netdev, orig_mtu);
-
- if (IS_ERR(nvdev)) {
- netdev_err(ndev, "restoring mtu failed: %ld\n",
- PTR_ERR(nvdev));
- return ret;
- }
- }
+ ret = netvsc_attach(ndev, &device_info);
+ if (ret)
+ goto rollback;
- if (was_opened)
- rndis_filter_open(nvdev);
+ return 0;
- netif_device_attach(ndev);
+rollback:
+ /* Attempt rollback to original MTU */
+ ndev->mtu = orig_mtu;
- /* We may have missed link change notifications */
- schedule_delayed_work(&ndevctx->dwork, 0);
+ if (netvsc_attach(ndev, &device_info))
+ netdev_err(ndev, "restoring mtu failed\n");
+rollback_vf:
+ if (vf_netdev)
+ dev_set_mtu(vf_netdev, orig_mtu);
return ret;
}
{
struct net_device_context *ndevctx = netdev_priv(ndev);
struct netvsc_device *nvdev = rtnl_dereference(ndevctx->nvdev);
- struct hv_device *hdev = ndevctx->device_ctx;
struct netvsc_device_info device_info;
struct ethtool_ringparam orig;
u32 new_tx, new_rx;
- bool was_opened;
int ret = 0;
if (!nvdev || nvdev->destroy)
device_info.recv_sections = new_rx;
device_info.recv_section_size = nvdev->recv_section_size;
- netif_device_detach(ndev);
- was_opened = rndis_filter_opened(nvdev);
- if (was_opened)
- rndis_filter_close(nvdev);
-
- rndis_filter_device_remove(hdev, nvdev);
-
- nvdev = rndis_filter_device_add(hdev, &device_info);
- if (IS_ERR(nvdev)) {
- ret = PTR_ERR(nvdev);
+ ret = netvsc_detach(ndev, nvdev);
+ if (ret)
+ return ret;
+ ret = netvsc_attach(ndev, &device_info);
+ if (ret) {
device_info.send_sections = orig.tx_pending;
device_info.recv_sections = orig.rx_pending;
- nvdev = rndis_filter_device_add(hdev, &device_info);
- if (IS_ERR(nvdev)) {
- netdev_err(ndev, "restoring ringparam failed: %ld\n",
- PTR_ERR(nvdev));
- return ret;
- }
- }
-
- if (was_opened)
- rndis_filter_open(nvdev);
- netif_device_attach(ndev);
- /* We may have missed link change notifications */
- ndevctx->last_reconfig = 0;
- schedule_delayed_work(&ndevctx->dwork, 0);
+ if (netvsc_attach(ndev, &device_info))
+ netdev_err(ndev, "restoring ringparam failed");
+ }
return ret;
}
/* set multicast etc flags on VF */
dev_change_flags(vf_netdev, ndev->flags | IFF_SLAVE);
+
+ /* sync address list from ndev to VF */
+ netif_addr_lock_bh(ndev);
dev_uc_sync(vf_netdev, ndev);
dev_mc_sync(vf_netdev, ndev);
+ netif_addr_unlock_bh(ndev);
if (netif_running(ndev)) {
ret = dev_open(vf_netdev);
static int netvsc_remove(struct hv_device *dev)
{
struct net_device_context *ndev_ctx;
- struct net_device *vf_netdev;
- struct net_device *net;
+ struct net_device *vf_netdev, *net;
+ struct netvsc_device *nvdev;
net = hv_get_drvdata(dev);
if (net == NULL) {
ndev_ctx = netdev_priv(net);
- netif_device_detach(net);
-
cancel_delayed_work_sync(&ndev_ctx->dwork);
+ rcu_read_lock();
+ nvdev = rcu_dereference(ndev_ctx->nvdev);
+
+ if (nvdev)
+ cancel_work_sync(&nvdev->subchan_work);
+
/*
* Call to the vsc driver to let it know that the device is being
* removed. Also blocks mtu and channel changes.
if (vf_netdev)
netvsc_unregister_vf(vf_netdev);
+ if (nvdev)
+ rndis_filter_device_remove(dev, nvdev);
+
unregister_netdevice(net);
- rndis_filter_device_remove(dev,
- rtnl_dereference(ndev_ctx->nvdev));
rtnl_unlock();
+ rcu_read_unlock();
hv_set_drvdata(dev, NULL);
}
}
-static void rndis_filter_receive_response(struct rndis_device *dev,
- struct rndis_message *resp)
+static void rndis_filter_receive_response(struct net_device *ndev,
+ struct netvsc_device *nvdev,
+ const struct rndis_message *resp)
{
+ struct rndis_device *dev = nvdev->extension;
struct rndis_request *request = NULL;
bool found = false;
unsigned long flags;
- struct net_device *ndev = dev->ndev;
+
+ /* This should never happen, it means control message
+ * response received after device removed.
+ */
+ if (dev->state == RNDIS_DEV_UNINITIALIZED) {
+ netdev_err(ndev,
+ "got rndis message uninitialized\n");
+ return;
+ }
spin_lock_irqsave(&dev->request_lock, flags);
list_for_each_entry(request, &dev->req_list, list_ent) {
static int rndis_filter_receive_data(struct net_device *ndev,
struct netvsc_device *nvdev,
- struct rndis_device *dev,
struct rndis_message *msg,
struct vmbus_channel *channel,
void *data, u32 data_buflen)
* should be the data packet size plus the trailer padding size
*/
if (unlikely(data_buflen < rndis_pkt->data_len)) {
- netdev_err(dev->ndev, "rndis message buffer "
+ netdev_err(ndev, "rndis message buffer "
"overflow detected (got %u, min %u)"
"...dropping this message!\n",
data_buflen, rndis_pkt->data_len);
void *data, u32 buflen)
{
struct net_device_context *net_device_ctx = netdev_priv(ndev);
- struct rndis_device *rndis_dev = net_dev->extension;
struct rndis_message *rndis_msg = data;
- /* Make sure the rndis device state is initialized */
- if (unlikely(!rndis_dev)) {
- netif_dbg(net_device_ctx, rx_err, ndev,
- "got rndis message but no rndis device!\n");
- return NVSP_STAT_FAIL;
- }
-
- if (unlikely(rndis_dev->state == RNDIS_DEV_UNINITIALIZED)) {
- netif_dbg(net_device_ctx, rx_err, ndev,
- "got rndis message uninitialized\n");
- return NVSP_STAT_FAIL;
- }
-
if (netif_msg_rx_status(net_device_ctx))
dump_rndis_message(ndev, rndis_msg);
switch (rndis_msg->ndis_msg_type) {
case RNDIS_MSG_PACKET:
- return rndis_filter_receive_data(ndev, net_dev,
- rndis_dev, rndis_msg,
+ return rndis_filter_receive_data(ndev, net_dev, rndis_msg,
channel, data, buflen);
case RNDIS_MSG_INIT_C:
case RNDIS_MSG_QUERY_C:
case RNDIS_MSG_SET_C:
/* completion msgs */
- rndis_filter_receive_response(rndis_dev, rndis_msg);
+ rndis_filter_receive_response(ndev, net_dev, rndis_msg);
break;
case RNDIS_MSG_INDICATE:
"unhandled rndis message (type %u len %u)\n",
rndis_msg->ndis_msg_type,
rndis_msg->msg_len);
- break;
+ return NVSP_STAT_FAIL;
}
- return 0;
+ return NVSP_STAT_SUCCESS;
}
static int rndis_filter_query_device(struct rndis_device *dev,
struct rndis_set_request *set;
int ret;
+ if (dev->filter == new_filter)
+ return 0;
+
request = get_rndis_request(dev, RNDIS_MSG_SET,
RNDIS_MESSAGE_SIZE(struct rndis_set_request) +
sizeof(u32));
if (!request)
return -ENOMEM;
-
/* Setup the rndis set */
set = &request->request_msg.msg.set_req;
set->oid = RNDIS_OID_GEN_CURRENT_PACKET_FILTER;
&new_filter, sizeof(u32));
ret = rndis_filter_send_request(dev, request);
- if (ret == 0)
+ if (ret == 0) {
wait_for_completion(&request->wait_event);
+ dev->filter = new_filter;
+ }
put_rndis_request(dev, request);
filter = NDIS_PACKET_TYPE_PROMISCUOUS;
} else {
if (flags & IFF_ALLMULTI)
- flags |= NDIS_PACKET_TYPE_ALL_MULTICAST;
+ filter |= NDIS_PACKET_TYPE_ALL_MULTICAST;
if (flags & IFF_BROADCAST)
- flags |= NDIS_PACKET_TYPE_BROADCAST;
+ filter |= NDIS_PACKET_TYPE_BROADCAST;
}
rndis_filter_set_packet_filter(rdev, filter);
for (i = 0; i < VRSS_SEND_TAB_SIZE; i++)
ndev_ctx->tx_table[i] = i % nvdev->num_chn;
+ netif_device_attach(ndev);
rtnl_unlock();
return;
nvdev->max_chn = 1;
nvdev->num_chn = 1;
+
+ netif_device_attach(ndev);
unlock:
rtnl_unlock();
}
net_device->num_chn = 1;
}
+ /* No sub channels, device is ready */
+ if (net_device->num_chn == 1)
+ netif_device_attach(net);
+
return net_device;
err_dev_remv:
{
struct rndis_device *rndis_dev = net_dev->extension;
- /* Don't try and setup sub channels if about to halt */
- cancel_work_sync(&net_dev->subchan_work);
-
/* Halt and release the rndis device */
rndis_filter_halt_device(net_dev, rndis_dev);
net_dev->extension = NULL;
netvsc_device_remove(dev);
- kfree(rndis_dev);
}
int rndis_filter_open(struct netvsc_device *nvdev)
return rndis_filter_close_device(nvdev->extension);
}
-
-bool rndis_filter_opened(const struct netvsc_device *nvdev)
-{
- const struct rndis_device *dev = nvdev->extension;
-
- return dev->state == RNDIS_DEV_DATAINITIALIZED;
-}
if (!at86rf230_debugfs_root)
return -ENOMEM;
- stats = debugfs_create_file("trac_stats", S_IRUGO,
+ stats = debugfs_create_file("trac_stats", 0444,
at86rf230_debugfs_root, lp,
&at86rf230_stats_fops);
if (!stats)
.id = &ipvlan_netid,
.size = sizeof(struct ipvlan_netns),
.exit = ipvlan_ns_exit,
- .async = true,
};
static int __init ipvlan_init_module(void)
/* Registered in net/core/dev.c */
struct pernet_operations __net_initdata loopback_net_ops = {
.init = loopback_net_init,
- .async = true,
};
err = netdev_upper_dev_link(real_dev, dev, extack);
if (err < 0)
- goto unregister;
+ goto put_dev;
/* need to be already registered so that ->init has run and
* the MAC addr is set
macsec_del_dev(macsec);
unlink:
netdev_upper_dev_unlink(real_dev, dev);
-unregister:
+put_dev:
+ dev_put(real_dev);
unregister_netdevice(dev);
return err;
}
lowerdev_features &= (features | ~NETIF_F_LRO);
features = netdev_increment_features(lowerdev_features, features, mask);
features |= ALWAYS_ON_FEATURES;
- features &= ~NETIF_F_NETNS_LOCAL;
+ features &= (ALWAYS_ON_FEATURES | MACVLAN_FEATURES);
return features;
}
netdevsim-objs += \
bpf.o
endif
+
+ifneq ($(CONFIG_NET_DEVLINK),)
+netdevsim-objs += devlink.o fib.o
+endif
--- /dev/null
+/*
+ * Copyright (c) 2018 Cumulus Networks. All rights reserved.
+ * Copyright (c) 2018 David Ahern <dsa@cumulusnetworks.com>
+ *
+ * This software is licensed under the GNU General License Version 2,
+ * June 1991 as shown in the file COPYING in the top-level directory of this
+ * source tree.
+ *
+ * THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS"
+ * WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,
+ * BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE
+ * OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME
+ * THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+ */
+
+#include <linux/device.h>
+#include <net/devlink.h>
+#include <net/netns/generic.h>
+
+#include "netdevsim.h"
+
+static unsigned int nsim_devlink_id;
+
+/* place holder until devlink and namespaces is sorted out */
+static struct net *nsim_devlink_net(struct devlink *devlink)
+{
+ return &init_net;
+}
+
+/* IPv4
+ */
+static u64 nsim_ipv4_fib_resource_occ_get(struct devlink *devlink)
+{
+ struct net *net = nsim_devlink_net(devlink);
+
+ return nsim_fib_get_val(net, NSIM_RESOURCE_IPV4_FIB, false);
+}
+
+static struct devlink_resource_ops nsim_ipv4_fib_res_ops = {
+ .occ_get = nsim_ipv4_fib_resource_occ_get,
+};
+
+static u64 nsim_ipv4_fib_rules_res_occ_get(struct devlink *devlink)
+{
+ struct net *net = nsim_devlink_net(devlink);
+
+ return nsim_fib_get_val(net, NSIM_RESOURCE_IPV4_FIB_RULES, false);
+}
+
+static struct devlink_resource_ops nsim_ipv4_fib_rules_res_ops = {
+ .occ_get = nsim_ipv4_fib_rules_res_occ_get,
+};
+
+/* IPv6
+ */
+static u64 nsim_ipv6_fib_resource_occ_get(struct devlink *devlink)
+{
+ struct net *net = nsim_devlink_net(devlink);
+
+ return nsim_fib_get_val(net, NSIM_RESOURCE_IPV6_FIB, false);
+}
+
+static struct devlink_resource_ops nsim_ipv6_fib_res_ops = {
+ .occ_get = nsim_ipv6_fib_resource_occ_get,
+};
+
+static u64 nsim_ipv6_fib_rules_res_occ_get(struct devlink *devlink)
+{
+ struct net *net = nsim_devlink_net(devlink);
+
+ return nsim_fib_get_val(net, NSIM_RESOURCE_IPV6_FIB_RULES, false);
+}
+
+static struct devlink_resource_ops nsim_ipv6_fib_rules_res_ops = {
+ .occ_get = nsim_ipv6_fib_rules_res_occ_get,
+};
+
+static int devlink_resources_register(struct devlink *devlink)
+{
+ struct devlink_resource_size_params params = {
+ .size_max = (u64)-1,
+ .size_granularity = 1,
+ .unit = DEVLINK_RESOURCE_UNIT_ENTRY
+ };
+ struct net *net = nsim_devlink_net(devlink);
+ int err;
+ u64 n;
+
+ /* Resources for IPv4 */
+ err = devlink_resource_register(devlink, "IPv4", (u64)-1,
+ NSIM_RESOURCE_IPV4,
+ DEVLINK_RESOURCE_ID_PARENT_TOP,
+ ¶ms, NULL);
+ if (err) {
+ pr_err("Failed to register IPv4 top resource\n");
+ goto out;
+ }
+
+ n = nsim_fib_get_val(net, NSIM_RESOURCE_IPV4_FIB, true);
+ err = devlink_resource_register(devlink, "fib", n,
+ NSIM_RESOURCE_IPV4_FIB,
+ NSIM_RESOURCE_IPV4,
+ ¶ms, &nsim_ipv4_fib_res_ops);
+ if (err) {
+ pr_err("Failed to register IPv4 FIB resource\n");
+ return err;
+ }
+
+ n = nsim_fib_get_val(net, NSIM_RESOURCE_IPV4_FIB_RULES, true);
+ err = devlink_resource_register(devlink, "fib-rules", n,
+ NSIM_RESOURCE_IPV4_FIB_RULES,
+ NSIM_RESOURCE_IPV4,
+ ¶ms, &nsim_ipv4_fib_rules_res_ops);
+ if (err) {
+ pr_err("Failed to register IPv4 FIB rules resource\n");
+ return err;
+ }
+
+ /* Resources for IPv6 */
+ err = devlink_resource_register(devlink, "IPv6", (u64)-1,
+ NSIM_RESOURCE_IPV6,
+ DEVLINK_RESOURCE_ID_PARENT_TOP,
+ ¶ms, NULL);
+ if (err) {
+ pr_err("Failed to register IPv6 top resource\n");
+ goto out;
+ }
+
+ n = nsim_fib_get_val(net, NSIM_RESOURCE_IPV6_FIB, true);
+ err = devlink_resource_register(devlink, "fib", n,
+ NSIM_RESOURCE_IPV6_FIB,
+ NSIM_RESOURCE_IPV6,
+ ¶ms, &nsim_ipv6_fib_res_ops);
+ if (err) {
+ pr_err("Failed to register IPv6 FIB resource\n");
+ return err;
+ }
+
+ n = nsim_fib_get_val(net, NSIM_RESOURCE_IPV6_FIB_RULES, true);
+ err = devlink_resource_register(devlink, "fib-rules", n,
+ NSIM_RESOURCE_IPV6_FIB_RULES,
+ NSIM_RESOURCE_IPV6,
+ ¶ms, &nsim_ipv6_fib_rules_res_ops);
+ if (err) {
+ pr_err("Failed to register IPv6 FIB rules resource\n");
+ return err;
+ }
+out:
+ return err;
+}
+
+static int nsim_devlink_reload(struct devlink *devlink)
+{
+ enum nsim_resource_id res_ids[] = {
+ NSIM_RESOURCE_IPV4_FIB, NSIM_RESOURCE_IPV4_FIB_RULES,
+ NSIM_RESOURCE_IPV6_FIB, NSIM_RESOURCE_IPV6_FIB_RULES
+ };
+ struct net *net = nsim_devlink_net(devlink);
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(res_ids); ++i) {
+ int err;
+ u64 val;
+
+ err = devlink_resource_size_get(devlink, res_ids[i], &val);
+ if (!err) {
+ err = nsim_fib_set_max(net, res_ids[i], val);
+ if (err)
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static void nsim_devlink_net_reset(struct net *net)
+{
+ enum nsim_resource_id res_ids[] = {
+ NSIM_RESOURCE_IPV4_FIB, NSIM_RESOURCE_IPV4_FIB_RULES,
+ NSIM_RESOURCE_IPV6_FIB, NSIM_RESOURCE_IPV6_FIB_RULES
+ };
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(res_ids); ++i) {
+ if (nsim_fib_set_max(net, res_ids[i], (u64)-1)) {
+ pr_err("Failed to reset limit for resource %u\n",
+ res_ids[i]);
+ }
+ }
+}
+
+static const struct devlink_ops nsim_devlink_ops = {
+ .reload = nsim_devlink_reload,
+};
+
+/* once devlink / namespace issues are sorted out
+ * this needs to be net in which a devlink instance
+ * is to be created. e.g., dev_net(ns->netdev)
+ */
+static struct net *nsim_to_net(struct netdevsim *ns)
+{
+ return &init_net;
+}
+
+void nsim_devlink_teardown(struct netdevsim *ns)
+{
+ if (ns->devlink) {
+ struct net *net = nsim_to_net(ns);
+ bool *reg_devlink = net_generic(net, nsim_devlink_id);
+
+ devlink_unregister(ns->devlink);
+ devlink_free(ns->devlink);
+ ns->devlink = NULL;
+
+ nsim_devlink_net_reset(net);
+ *reg_devlink = true;
+ }
+}
+
+void nsim_devlink_setup(struct netdevsim *ns)
+{
+ struct net *net = nsim_to_net(ns);
+ bool *reg_devlink = net_generic(net, nsim_devlink_id);
+ struct devlink *devlink;
+ int err = -ENOMEM;
+
+ /* only one device per namespace controls devlink */
+ if (!*reg_devlink) {
+ ns->devlink = NULL;
+ return;
+ }
+
+ devlink = devlink_alloc(&nsim_devlink_ops, 0);
+ if (!devlink)
+ return;
+
+ err = devlink_register(devlink, &ns->dev);
+ if (err)
+ goto err_devlink_free;
+
+ err = devlink_resources_register(devlink);
+ if (err)
+ goto err_dl_unregister;
+
+ ns->devlink = devlink;
+
+ *reg_devlink = false;
+
+ return;
+
+err_dl_unregister:
+ devlink_unregister(devlink);
+err_devlink_free:
+ devlink_free(devlink);
+}
+
+/* Initialize per network namespace state */
+static int __net_init nsim_devlink_netns_init(struct net *net)
+{
+ bool *reg_devlink = net_generic(net, nsim_devlink_id);
+
+ *reg_devlink = true;
+
+ return 0;
+}
+
+static struct pernet_operations nsim_devlink_net_ops __net_initdata = {
+ .init = nsim_devlink_netns_init,
+ .id = &nsim_devlink_id,
+ .size = sizeof(bool),
+};
+
+void nsim_devlink_exit(void)
+{
+ unregister_pernet_subsys(&nsim_devlink_net_ops);
+ nsim_fib_exit();
+}
+
+int nsim_devlink_init(void)
+{
+ int err;
+
+ err = nsim_fib_init();
+ if (err)
+ goto err_out;
+
+ err = register_pernet_subsys(&nsim_devlink_net_ops);
+ if (err)
+ nsim_fib_exit();
+
+err_out:
+ return err;
+}
--- /dev/null
+/*
+ * Copyright (c) 2018 Cumulus Networks. All rights reserved.
+ * Copyright (c) 2018 David Ahern <dsa@cumulusnetworks.com>
+ *
+ * This software is licensed under the GNU General License Version 2,
+ * June 1991 as shown in the file COPYING in the top-level directory of this
+ * source tree.
+ *
+ * THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS"
+ * WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING,
+ * BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE
+ * OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME
+ * THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+ */
+
+#include <net/fib_notifier.h>
+#include <net/ip_fib.h>
+#include <net/ip6_fib.h>
+#include <net/fib_rules.h>
+#include <net/netns/generic.h>
+
+#include "netdevsim.h"
+
+struct nsim_fib_entry {
+ u64 max;
+ u64 num;
+};
+
+struct nsim_per_fib_data {
+ struct nsim_fib_entry fib;
+ struct nsim_fib_entry rules;
+};
+
+struct nsim_fib_data {
+ struct nsim_per_fib_data ipv4;
+ struct nsim_per_fib_data ipv6;
+};
+
+static unsigned int nsim_fib_net_id;
+
+u64 nsim_fib_get_val(struct net *net, enum nsim_resource_id res_id, bool max)
+{
+ struct nsim_fib_data *fib_data = net_generic(net, nsim_fib_net_id);
+ struct nsim_fib_entry *entry;
+
+ switch (res_id) {
+ case NSIM_RESOURCE_IPV4_FIB:
+ entry = &fib_data->ipv4.fib;
+ break;
+ case NSIM_RESOURCE_IPV4_FIB_RULES:
+ entry = &fib_data->ipv4.rules;
+ break;
+ case NSIM_RESOURCE_IPV6_FIB:
+ entry = &fib_data->ipv6.fib;
+ break;
+ case NSIM_RESOURCE_IPV6_FIB_RULES:
+ entry = &fib_data->ipv6.rules;
+ break;
+ default:
+ return 0;
+ }
+
+ return max ? entry->max : entry->num;
+}
+
+int nsim_fib_set_max(struct net *net, enum nsim_resource_id res_id, u64 val)
+{
+ struct nsim_fib_data *fib_data = net_generic(net, nsim_fib_net_id);
+ struct nsim_fib_entry *entry;
+ int err = 0;
+
+ switch (res_id) {
+ case NSIM_RESOURCE_IPV4_FIB:
+ entry = &fib_data->ipv4.fib;
+ break;
+ case NSIM_RESOURCE_IPV4_FIB_RULES:
+ entry = &fib_data->ipv4.rules;
+ break;
+ case NSIM_RESOURCE_IPV6_FIB:
+ entry = &fib_data->ipv6.fib;
+ break;
+ case NSIM_RESOURCE_IPV6_FIB_RULES:
+ entry = &fib_data->ipv6.rules;
+ break;
+ default:
+ return 0;
+ }
+
+ /* not allowing a new max to be less than curren occupancy
+ * --> no means of evicting entries
+ */
+ if (val < entry->num)
+ err = -EINVAL;
+ else
+ entry->max = val;
+
+ return err;
+}
+
+static int nsim_fib_rule_account(struct nsim_fib_entry *entry, bool add,
+ struct netlink_ext_ack *extack)
+{
+ int err = 0;
+
+ if (add) {
+ if (entry->num < entry->max) {
+ entry->num++;
+ } else {
+ err = -ENOSPC;
+ NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported fib rule entries");
+ }
+ } else {
+ entry->num--;
+ }
+
+ return err;
+}
+
+static int nsim_fib_rule_event(struct fib_notifier_info *info, bool add)
+{
+ struct nsim_fib_data *data = net_generic(info->net, nsim_fib_net_id);
+ struct netlink_ext_ack *extack = info->extack;
+ int err = 0;
+
+ switch (info->family) {
+ case AF_INET:
+ err = nsim_fib_rule_account(&data->ipv4.rules, add, extack);
+ break;
+ case AF_INET6:
+ err = nsim_fib_rule_account(&data->ipv6.rules, add, extack);
+ break;
+ }
+
+ return err;
+}
+
+static int nsim_fib_account(struct nsim_fib_entry *entry, bool add,
+ struct netlink_ext_ack *extack)
+{
+ int err = 0;
+
+ if (add) {
+ if (entry->num < entry->max) {
+ entry->num++;
+ } else {
+ err = -ENOSPC;
+ NL_SET_ERR_MSG_MOD(extack, "Exceeded number of supported fib entries");
+ }
+ } else {
+ entry->num--;
+ }
+
+ return err;
+}
+
+static int nsim_fib_event(struct fib_notifier_info *info, bool add)
+{
+ struct nsim_fib_data *data = net_generic(info->net, nsim_fib_net_id);
+ struct netlink_ext_ack *extack = info->extack;
+ int err = 0;
+
+ switch (info->family) {
+ case AF_INET:
+ err = nsim_fib_account(&data->ipv4.fib, add, extack);
+ break;
+ case AF_INET6:
+ err = nsim_fib_account(&data->ipv6.fib, add, extack);
+ break;
+ }
+
+ return err;
+}
+
+static int nsim_fib_event_nb(struct notifier_block *nb, unsigned long event,
+ void *ptr)
+{
+ struct fib_notifier_info *info = ptr;
+ int err = 0;
+
+ switch (event) {
+ case FIB_EVENT_RULE_ADD: /* fall through */
+ case FIB_EVENT_RULE_DEL:
+ err = nsim_fib_rule_event(info, event == FIB_EVENT_RULE_ADD);
+ break;
+
+ case FIB_EVENT_ENTRY_ADD: /* fall through */
+ case FIB_EVENT_ENTRY_DEL:
+ err = nsim_fib_event(info, event == FIB_EVENT_ENTRY_ADD);
+ break;
+ }
+
+ return notifier_from_errno(err);
+}
+
+/* inconsistent dump, trying again */
+static void nsim_fib_dump_inconsistent(struct notifier_block *nb)
+{
+ struct nsim_fib_data *data;
+ struct net *net;
+
+ rcu_read_lock();
+ for_each_net_rcu(net) {
+ data = net_generic(net, nsim_fib_net_id);
+
+ data->ipv4.fib.num = 0ULL;
+ data->ipv4.rules.num = 0ULL;
+
+ data->ipv6.fib.num = 0ULL;
+ data->ipv6.rules.num = 0ULL;
+ }
+ rcu_read_unlock();
+}
+
+static struct notifier_block nsim_fib_nb = {
+ .notifier_call = nsim_fib_event_nb,
+};
+
+/* Initialize per network namespace state */
+static int __net_init nsim_fib_netns_init(struct net *net)
+{
+ struct nsim_fib_data *data = net_generic(net, nsim_fib_net_id);
+
+ data->ipv4.fib.max = (u64)-1;
+ data->ipv4.rules.max = (u64)-1;
+
+ data->ipv6.fib.max = (u64)-1;
+ data->ipv6.rules.max = (u64)-1;
+
+ return 0;
+}
+
+static struct pernet_operations nsim_fib_net_ops __net_initdata = {
+ .init = nsim_fib_netns_init,
+ .id = &nsim_fib_net_id,
+ .size = sizeof(struct nsim_fib_data),
+};
+
+void nsim_fib_exit(void)
+{
+ unregister_pernet_subsys(&nsim_fib_net_ops);
+ unregister_fib_notifier(&nsim_fib_nb);
+}
+
+int nsim_fib_init(void)
+{
+ int err;
+
+ err = register_pernet_subsys(&nsim_fib_net_ops);
+ if (err < 0) {
+ pr_err("Failed to register pernet subsystem\n");
+ goto err_out;
+ }
+
+ err = register_fib_notifier(&nsim_fib_nb, nsim_fib_dump_inconsistent);
+ if (err < 0) {
+ pr_err("Failed to register fib notifier\n");
+ goto err_out;
+ }
+
+err_out:
+ return err;
+}
SET_NETDEV_DEV(dev, &ns->dev);
+ nsim_devlink_setup(ns);
+
return 0;
err_bpf_uninit:
{
struct netdevsim *ns = netdev_priv(dev);
+ nsim_devlink_teardown(ns);
debugfs_remove_recursive(ns->ddir);
nsim_bpf_uninit(ns);
}
if (err)
goto err_debugfs_destroy;
- err = rtnl_link_register(&nsim_link_ops);
+ err = nsim_devlink_init();
if (err)
goto err_unreg_bus;
+ err = rtnl_link_register(&nsim_link_ops);
+ if (err)
+ goto err_dl_fini;
+
return 0;
+err_dl_fini:
+ nsim_devlink_exit();
err_unreg_bus:
bus_unregister(&nsim_bus);
err_debugfs_destroy:
static void __exit nsim_module_exit(void)
{
rtnl_link_unregister(&nsim_link_ops);
+ nsim_devlink_exit();
bus_unregister(&nsim_bus);
debugfs_remove_recursive(nsim_ddir);
}
bool bpf_map_accept;
struct list_head bpf_bound_maps;
+#if IS_ENABLED(CONFIG_NET_DEVLINK)
+ struct devlink *devlink;
+#endif
};
extern struct dentry *nsim_ddir;
}
#endif
+#if IS_ENABLED(CONFIG_NET_DEVLINK)
+enum nsim_resource_id {
+ NSIM_RESOURCE_NONE, /* DEVLINK_RESOURCE_ID_PARENT_TOP */
+ NSIM_RESOURCE_IPV4,
+ NSIM_RESOURCE_IPV4_FIB,
+ NSIM_RESOURCE_IPV4_FIB_RULES,
+ NSIM_RESOURCE_IPV6,
+ NSIM_RESOURCE_IPV6_FIB,
+ NSIM_RESOURCE_IPV6_FIB_RULES,
+};
+
+void nsim_devlink_setup(struct netdevsim *ns);
+void nsim_devlink_teardown(struct netdevsim *ns);
+
+int nsim_devlink_init(void);
+void nsim_devlink_exit(void);
+
+int nsim_fib_init(void);
+void nsim_fib_exit(void);
+u64 nsim_fib_get_val(struct net *net, enum nsim_resource_id res_id, bool max);
+int nsim_fib_set_max(struct net *net, enum nsim_resource_id res_id, u64 val);
+#else
+static inline void nsim_devlink_setup(struct netdevsim *ns)
+{
+}
+
+static inline void nsim_devlink_teardown(struct netdevsim *ns)
+{
+}
+
+static inline int nsim_devlink_init(void)
+{
+ return 0;
+}
+
+static inline void nsim_devlink_exit(void)
+{
+}
+#endif
+
static inline struct netdevsim *to_nsim(struct device *ptr)
{
return container_of(ptr, struct netdevsim, dev);
unsigned int i;
for (i = 0; i < ARRAY_SIZE(bcm_phy_hw_stats); i++)
- memcpy(data + i * ETH_GSTRING_LEN,
- bcm_phy_hw_stats[i].string, ETH_GSTRING_LEN);
+ strlcpy(data + i * ETH_GSTRING_LEN,
+ bcm_phy_hw_stats[i].string, ETH_GSTRING_LEN);
}
EXPORT_SYMBOL_GPL(bcm_phy_get_strings);
#define PHY_ID_PHY22F_1_4 0xD565A410
#define PHY_ID_PHY11G_1_5 0xD565A401
#define PHY_ID_PHY22F_1_5 0xD565A411
-#define PHY_ID_PHY11G_VR9 0xD565A409
-#define PHY_ID_PHY22F_VR9 0xD565A419
+#define PHY_ID_PHY11G_VR9_1_1 0xD565A408
+#define PHY_ID_PHY22F_VR9_1_1 0xD565A418
+#define PHY_ID_PHY11G_VR9_1_2 0xD565A409
+#define PHY_ID_PHY22F_VR9_1_2 0xD565A419
static int xway_gphy_config_init(struct phy_device *phydev)
{
.suspend = genphy_suspend,
.resume = genphy_resume,
}, {
- .phy_id = PHY_ID_PHY11G_VR9,
+ .phy_id = PHY_ID_PHY11G_VR9_1_1,
.phy_id_mask = 0xffffffff,
- .name = "Intel XWAY PHY11G (xRX integrated)",
+ .name = "Intel XWAY PHY11G (xRX v1.1 integrated)",
.features = PHY_GBIT_FEATURES,
.flags = PHY_HAS_INTERRUPT,
.config_init = xway_gphy_config_init,
.suspend = genphy_suspend,
.resume = genphy_resume,
}, {
- .phy_id = PHY_ID_PHY22F_VR9,
+ .phy_id = PHY_ID_PHY22F_VR9_1_1,
.phy_id_mask = 0xffffffff,
- .name = "Intel XWAY PHY22F (xRX integrated)",
+ .name = "Intel XWAY PHY22F (xRX v1.1 integrated)",
+ .features = PHY_BASIC_FEATURES,
+ .flags = PHY_HAS_INTERRUPT,
+ .config_init = xway_gphy_config_init,
+ .ack_interrupt = xway_gphy_ack_interrupt,
+ .did_interrupt = xway_gphy_did_interrupt,
+ .config_intr = xway_gphy_config_intr,
+ .suspend = genphy_suspend,
+ .resume = genphy_resume,
+ }, {
+ .phy_id = PHY_ID_PHY11G_VR9_1_2,
+ .phy_id_mask = 0xffffffff,
+ .name = "Intel XWAY PHY11G (xRX v1.2 integrated)",
+ .features = PHY_GBIT_FEATURES,
+ .flags = PHY_HAS_INTERRUPT,
+ .config_init = xway_gphy_config_init,
+ .ack_interrupt = xway_gphy_ack_interrupt,
+ .did_interrupt = xway_gphy_did_interrupt,
+ .config_intr = xway_gphy_config_intr,
+ .suspend = genphy_suspend,
+ .resume = genphy_resume,
+ }, {
+ .phy_id = PHY_ID_PHY22F_VR9_1_2,
+ .phy_id_mask = 0xffffffff,
+ .name = "Intel XWAY PHY22F (xRX v1.2 integrated)",
.features = PHY_BASIC_FEATURES,
.flags = PHY_HAS_INTERRUPT,
.config_init = xway_gphy_config_init,
{ PHY_ID_PHY22F_1_4, 0xffffffff },
{ PHY_ID_PHY11G_1_5, 0xffffffff },
{ PHY_ID_PHY22F_1_5, 0xffffffff },
- { PHY_ID_PHY11G_VR9, 0xffffffff },
- { PHY_ID_PHY22F_VR9, 0xffffffff },
+ { PHY_ID_PHY11G_VR9_1_1, 0xffffffff },
+ { PHY_ID_PHY22F_VR9_1_1, 0xffffffff },
+ { PHY_ID_PHY11G_VR9_1_2, 0xffffffff },
+ { PHY_ID_PHY22F_VR9_1_2, 0xffffffff },
{ }
};
MODULE_DEVICE_TABLE(mdio, xway_gphy_tbl);
int i;
for (i = 0; i < ARRAY_SIZE(marvell_hw_stats); i++) {
- memcpy(data + i * ETH_GSTRING_LEN,
- marvell_hw_stats[i].string, ETH_GSTRING_LEN);
+ strlcpy(data + i * ETH_GSTRING_LEN,
+ marvell_hw_stats[i].string, ETH_GSTRING_LEN);
}
}
return 0;
}
-/* This routine returns -1 as an indication to the caller that the
- * Micrel ksz9021 10/100/1000 PHY does not support standard IEEE
- * MMD extended PHY registers.
- */
-static int
-ksz9021_rd_mmd_phyreg(struct phy_device *phydev, int devad, u16 regnum)
-{
- return -1;
-}
-
-/* This routine does nothing since the Micrel ksz9021 does not support
- * standard IEEE MMD extended PHY registers.
- */
-static int
-ksz9021_wr_mmd_phyreg(struct phy_device *phydev, int devad, u16 regnum, u16 val)
-{
- return -1;
-}
-
static int kszphy_get_sset_count(struct phy_device *phydev)
{
return ARRAY_SIZE(kszphy_hw_stats);
int i;
for (i = 0; i < ARRAY_SIZE(kszphy_hw_stats); i++) {
- memcpy(data + i * ETH_GSTRING_LEN,
- kszphy_hw_stats[i].string, ETH_GSTRING_LEN);
+ strlcpy(data + i * ETH_GSTRING_LEN,
+ kszphy_hw_stats[i].string, ETH_GSTRING_LEN);
}
}
.get_stats = kszphy_get_stats,
.suspend = genphy_suspend,
.resume = genphy_resume,
- .read_mmd = ksz9021_rd_mmd_phyreg,
- .write_mmd = ksz9021_wr_mmd_phyreg,
+ .read_mmd = genphy_read_mmd_unsupported,
+ .write_mmd = genphy_write_mmd_unsupported,
}, {
.phy_id = PHY_ID_KSZ9031,
.phy_id_mask = MICREL_PHY_ID_MASK,
phy_trigger_machine(phydev, false);
}
+/**
+ * phy_disable_interrupts - Disable the PHY interrupts from the PHY side
+ * @phydev: target phy_device struct
+ */
+static int phy_disable_interrupts(struct phy_device *phydev)
+{
+ int err;
+
+ /* Disable PHY interrupts */
+ err = phy_config_interrupt(phydev, PHY_INTERRUPT_DISABLED);
+ if (err)
+ return err;
+
+ /* Clear the interrupt */
+ return phy_clear_interrupt(phydev);
+}
+
+/**
+ * phy_change - Called by the phy_interrupt to handle PHY changes
+ * @phydev: phy_device struct that interrupted
+ */
+static irqreturn_t phy_change(struct phy_device *phydev)
+{
+ if (phy_interrupt_is_valid(phydev)) {
+ if (phydev->drv->did_interrupt &&
+ !phydev->drv->did_interrupt(phydev))
+ return IRQ_NONE;
+
+ if (phydev->state == PHY_HALTED)
+ if (phy_disable_interrupts(phydev))
+ goto phy_err;
+ }
+
+ mutex_lock(&phydev->lock);
+ if ((PHY_RUNNING == phydev->state) || (PHY_NOLINK == phydev->state))
+ phydev->state = PHY_CHANGELINK;
+ mutex_unlock(&phydev->lock);
+
+ /* reschedule state queue work to run as soon as possible */
+ phy_trigger_machine(phydev, true);
+
+ if (phy_interrupt_is_valid(phydev) && phy_clear_interrupt(phydev))
+ goto phy_err;
+ return IRQ_HANDLED;
+
+phy_err:
+ phy_error(phydev);
+ return IRQ_NONE;
+}
+
+/**
+ * phy_change_work - Scheduled by the phy_mac_interrupt to handle PHY changes
+ * @work: work_struct that describes the work to be done
+ */
+void phy_change_work(struct work_struct *work)
+{
+ struct phy_device *phydev =
+ container_of(work, struct phy_device, phy_queue);
+
+ phy_change(phydev);
+}
+
/**
* phy_interrupt - PHY interrupt handler
* @irq: interrupt line
if (PHY_HALTED == phydev->state)
return IRQ_NONE; /* It can't be ours. */
- phy_change(phydev);
-
- return IRQ_HANDLED;
+ return phy_change(phydev);
}
/**
return phy_config_interrupt(phydev, PHY_INTERRUPT_ENABLED);
}
-/**
- * phy_disable_interrupts - Disable the PHY interrupts from the PHY side
- * @phydev: target phy_device struct
- */
-static int phy_disable_interrupts(struct phy_device *phydev)
-{
- int err;
-
- /* Disable PHY interrupts */
- err = phy_config_interrupt(phydev, PHY_INTERRUPT_DISABLED);
- if (err)
- return err;
-
- /* Clear the interrupt */
- return phy_clear_interrupt(phydev);
-}
-
/**
* phy_start_interrupts - request and enable interrupts for a PHY device
* @phydev: target phy_device struct
}
EXPORT_SYMBOL(phy_stop_interrupts);
-/**
- * phy_change - Called by the phy_interrupt to handle PHY changes
- * @phydev: phy_device struct that interrupted
- */
-void phy_change(struct phy_device *phydev)
-{
- if (phy_interrupt_is_valid(phydev)) {
- if (phydev->drv->did_interrupt &&
- !phydev->drv->did_interrupt(phydev))
- return;
-
- if (phydev->state == PHY_HALTED)
- if (phy_disable_interrupts(phydev))
- goto phy_err;
- }
-
- mutex_lock(&phydev->lock);
- if ((PHY_RUNNING == phydev->state) || (PHY_NOLINK == phydev->state))
- phydev->state = PHY_CHANGELINK;
- mutex_unlock(&phydev->lock);
-
- /* reschedule state queue work to run as soon as possible */
- phy_trigger_machine(phydev, true);
-
- if (phy_interrupt_is_valid(phydev) && phy_clear_interrupt(phydev))
- goto phy_err;
- return;
-
-phy_err:
- phy_error(phydev);
-}
-
-/**
- * phy_change_work - Scheduled by the phy_mac_interrupt to handle PHY changes
- * @work: work_struct that describes the work to be done
- */
-void phy_change_work(struct work_struct *work)
-{
- struct phy_device *phydev =
- container_of(work, struct phy_device, phy_queue);
-
- phy_change(phydev);
-}
-
/**
* phy_stop - Bring down the PHY link, and stop checking the status
* @phydev: target phy_device struct
err = sysfs_create_link(&phydev->mdio.dev.kobj, &dev->dev.kobj,
"attached_dev");
if (!err) {
- err = sysfs_create_link(&dev->dev.kobj, &phydev->mdio.dev.kobj,
- "phydev");
- if (err)
- goto error;
+ err = sysfs_create_link_nowarn(&dev->dev.kobj,
+ &phydev->mdio.dev.kobj,
+ "phydev");
+ if (err) {
+ dev_err(&dev->dev, "could not add device link to %s err %d\n",
+ kobject_name(&phydev->mdio.dev.kobj),
+ err);
+ /* non-fatal - some net drivers can use one netdevice
+ * with more then one phy
+ */
+ }
phydev->sysfs_links = true;
}
}
EXPORT_SYMBOL(genphy_config_init);
+/* This is used for the phy device which doesn't support the MMD extended
+ * register access, but it does have side effect when we are trying to access
+ * the MMD register via indirect method.
+ */
+int genphy_read_mmd_unsupported(struct phy_device *phdev, int devad, u16 regnum)
+{
+ return -EOPNOTSUPP;
+}
+EXPORT_SYMBOL(genphy_read_mmd_unsupported);
+
+int genphy_write_mmd_unsupported(struct phy_device *phdev, int devnum,
+ u16 regnum, u16 val)
+{
+ return -EOPNOTSUPP;
+}
+EXPORT_SYMBOL(genphy_write_mmd_unsupported);
+
int genphy_suspend(struct phy_device *phydev)
{
return phy_set_bits(phydev, MII_BMCR, BMCR_PDOWN);
.flags = PHY_HAS_INTERRUPT,
.ack_interrupt = &rtl821x_ack_interrupt,
.config_intr = &rtl8211b_config_intr,
+ .read_mmd = &genphy_read_mmd_unsupported,
+ .write_mmd = &genphy_write_mmd_unsupported,
}, {
.phy_id = 0x001cc914,
.name = "RTL8211DN Gigabit Ethernet",
{
/* SFP module inserted - read I2C data */
struct sfp_eeprom_id id;
+ bool cotsworks;
u8 check;
int ret;
return -EAGAIN;
}
+ /* Cotsworks do not seem to update the checksums when they
+ * do the final programming with the final module part number,
+ * serial number and date code.
+ */
+ cotsworks = !memcmp(id.base.vendor_name, "COTSWORKS ", 16);
+
/* Validate the checksum over the base structure */
check = sfp_check(&id.base, sizeof(id.base) - 1);
if (check != id.base.cc_base) {
- dev_err(sfp->dev,
- "EEPROM base structure checksum failure: 0x%02x\n",
- check);
- print_hex_dump(KERN_ERR, "sfp EE: ", DUMP_PREFIX_OFFSET,
- 16, 1, &id, sizeof(id.base) - 1, true);
- return -EINVAL;
+ if (cotsworks) {
+ dev_warn(sfp->dev,
+ "EEPROM base structure checksum failure (0x%02x != 0x%02x)\n",
+ check, id.base.cc_base);
+ } else {
+ dev_err(sfp->dev,
+ "EEPROM base structure checksum failure: 0x%02x != 0x%02x\n",
+ check, id.base.cc_base);
+ print_hex_dump(KERN_ERR, "sfp EE: ", DUMP_PREFIX_OFFSET,
+ 16, 1, &id, sizeof(id), true);
+ return -EINVAL;
+ }
}
check = sfp_check(&id.ext, sizeof(id.ext) - 1);
if (check != id.ext.cc_ext) {
- dev_err(sfp->dev,
- "EEPROM extended structure checksum failure: 0x%02x\n",
- check);
- memset(&id.ext, 0, sizeof(id.ext));
+ if (cotsworks) {
+ dev_warn(sfp->dev,
+ "EEPROM extended structure checksum failure (0x%02x != 0x%02x)\n",
+ check, id.ext.cc_ext);
+ } else {
+ dev_err(sfp->dev,
+ "EEPROM extended structure checksum failure: 0x%02x != 0x%02x\n",
+ check, id.ext.cc_ext);
+ print_hex_dump(KERN_ERR, "sfp EE: ", DUMP_PREFIX_OFFSET,
+ 16, 1, &id, sizeof(id), true);
+ memset(&id.ext, 0, sizeof(id.ext));
+ }
}
sfp->id = id;
static const struct bin_attribute ks8995_registers_attr = {
.attr = {
.name = "registers",
- .mode = S_IRUSR | S_IWUSR,
+ .mode = 0600,
},
.size = KS8995_REGS_SIZE,
.read = ks8995_registers_read,
/* Prototypes. */
static int ppp_unattached_ioctl(struct net *net, struct ppp_file *pf,
struct file *file, unsigned int cmd, unsigned long arg);
-static void ppp_xmit_process(struct ppp *ppp);
+static void ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb);
static void ppp_send_frame(struct ppp *ppp, struct sk_buff *skb);
static void ppp_push(struct ppp *ppp);
static void ppp_channel_push(struct channel *pch);
goto out;
}
- skb_queue_tail(&pf->xq, skb);
-
switch (pf->kind) {
case INTERFACE:
- ppp_xmit_process(PF_TO_PPP(pf));
+ ppp_xmit_process(PF_TO_PPP(pf), skb);
break;
case CHANNEL:
+ skb_queue_tail(&pf->xq, skb);
ppp_channel_push(PF_TO_CHANNEL(pf));
break;
}
.exit = ppp_exit_net,
.id = &ppp_net_id,
.size = sizeof(struct ppp_net),
- .async = true,
};
static int ppp_unit_register(struct ppp *ppp, int unit, bool ifname_is_set)
put_unaligned_be16(proto, pp);
skb_scrub_packet(skb, !net_eq(ppp->ppp_net, dev_net(dev)));
- skb_queue_tail(&ppp->file.xq, skb);
- ppp_xmit_process(ppp);
+ ppp_xmit_process(ppp, skb);
+
return NETDEV_TX_OK;
outf:
*/
/* Called to do any work queued up on the transmit side that can now be done */
-static void __ppp_xmit_process(struct ppp *ppp)
+static void __ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb)
{
- struct sk_buff *skb;
-
ppp_xmit_lock(ppp);
if (!ppp->closing) {
ppp_push(ppp);
+
+ if (skb)
+ skb_queue_tail(&ppp->file.xq, skb);
while (!ppp->xmit_pending &&
(skb = skb_dequeue(&ppp->file.xq)))
ppp_send_frame(ppp, skb);
ppp_xmit_unlock(ppp);
}
-static void ppp_xmit_process(struct ppp *ppp)
+static void ppp_xmit_process(struct ppp *ppp, struct sk_buff *skb)
{
local_bh_disable();
goto err;
(*this_cpu_ptr(ppp->xmit_recursion))++;
- __ppp_xmit_process(ppp);
+ __ppp_xmit_process(ppp, skb);
(*this_cpu_ptr(ppp->xmit_recursion))--;
local_bh_enable();
err:
local_bh_enable();
+ kfree_skb(skb);
+
if (net_ratelimit())
netdev_err(ppp->dev, "recursion detected\n");
}
#ifdef CONFIG_PPP_MULTILINK
static bool mp_protocol_compress __read_mostly = true;
-module_param(mp_protocol_compress, bool, S_IRUGO | S_IWUSR);
+module_param(mp_protocol_compress, bool, 0644);
MODULE_PARM_DESC(mp_protocol_compress,
"compress protocol id in multilink fragments");
if (skb_queue_empty(&pch->file.xq)) {
ppp = pch->ppp;
if (ppp)
- __ppp_xmit_process(ppp);
+ __ppp_xmit_process(ppp, NULL);
}
}
rwlock_init(&pn->hash_lock);
- pde = proc_create("pppoe", S_IRUGO, net->proc_net, &pppoe_seq_fops);
+ pde = proc_create("pppoe", 0444, net->proc_net, &pppoe_seq_fops);
#ifdef CONFIG_PROC_FS
if (!pde)
return -ENOMEM;
.exit = pppoe_exit_net,
.id = &pppoe_net_id,
.size = sizeof(struct pppoe_net),
- .async = true,
};
static int __init pppoe_init(void)
if (!nlh) {
err = __send_and_alloc_skb(&skb, team, portid, send_func);
if (err)
- goto errout;
+ return err;
goto send_done;
}
if (!nlh) {
err = __send_and_alloc_skb(&skb, team, portid, send_func);
if (err)
- goto errout;
+ return err;
goto send_done;
}
return tun;
}
-static void tun_ptr_free(void *ptr)
+void tun_ptr_free(void *ptr)
{
if (!ptr)
return;
__skb_array_destroy_skb(ptr);
}
}
+EXPORT_SYMBOL_GPL(tun_ptr_free);
static void tun_queue_purge(struct tun_file *tfile)
{
#else
static bool prefer_mbim;
#endif
-module_param(prefer_mbim, bool, S_IRUGO | S_IWUSR);
+module_param(prefer_mbim, bool, 0644);
MODULE_PARM_DESC(prefer_mbim, "Prefer MBIM setting on dual NCM/MBIM functions");
static void cdc_ncm_txpath_bh(unsigned long param);
return len;
}
-static DEVICE_ATTR(min_tx_pkt, S_IRUGO | S_IWUSR, cdc_ncm_show_min_tx_pkt, cdc_ncm_store_min_tx_pkt);
-static DEVICE_ATTR(rx_max, S_IRUGO | S_IWUSR, cdc_ncm_show_rx_max, cdc_ncm_store_rx_max);
-static DEVICE_ATTR(tx_max, S_IRUGO | S_IWUSR, cdc_ncm_show_tx_max, cdc_ncm_store_tx_max);
-static DEVICE_ATTR(tx_timer_usecs, S_IRUGO | S_IWUSR, cdc_ncm_show_tx_timer_usecs, cdc_ncm_store_tx_timer_usecs);
+static DEVICE_ATTR(min_tx_pkt, 0644, cdc_ncm_show_min_tx_pkt, cdc_ncm_store_min_tx_pkt);
+static DEVICE_ATTR(rx_max, 0644, cdc_ncm_show_rx_max, cdc_ncm_store_rx_max);
+static DEVICE_ATTR(tx_max, 0644, cdc_ncm_show_tx_max, cdc_ncm_store_tx_max);
+static DEVICE_ATTR(tx_timer_usecs, 0644, cdc_ncm_show_tx_timer_usecs, cdc_ncm_store_tx_timer_usecs);
static ssize_t ndp_to_end_show(struct device *d, struct device_attribute *attr, char *buf)
{
struct cdc_ncm_ctx *ctx = (struct cdc_ncm_ctx *)dev->data[0]; \
return sprintf(buf, format "\n", tocpu(ctx->ncm_parm.name)); \
} \
-static DEVICE_ATTR(name, S_IRUGO, cdc_ncm_show_##name, NULL)
+static DEVICE_ATTR(name, 0444, cdc_ncm_show_##name, NULL)
NCM_PARM_ATTR(bmNtbFormatsSupported, "0x%04x", le16_to_cpu);
NCM_PARM_ATTR(dwNtbInMaxSize, "%u", le32_to_cpu);
return sprintf(buf, "%s\n", port_name);
}
-static DEVICE_ATTR(hsotype, S_IRUGO, hso_sysfs_show_porttype, NULL);
+static DEVICE_ATTR(hsotype, 0444, hso_sysfs_show_porttype, NULL);
static struct attribute *hso_serial_dev_attrs[] = {
&dev_attr_hsotype.attr,
/* change the debug level (eg: insmod hso.ko debug=0x04) */
MODULE_PARM_DESC(debug, "debug level mask [0x01 | 0x02 | 0x04 | 0x08 | 0x10]");
-module_param(debug, int, S_IRUGO | S_IWUSR);
+module_param(debug, int, 0644);
/* set the major tty number (eg: insmod hso.ko tty_major=245) */
MODULE_PARM_DESC(tty_major, "Set the major tty number");
-module_param(tty_major, int, S_IRUGO | S_IWUSR);
+module_param(tty_major, int, 0644);
/* disable network interface (eg: insmod hso.ko disable_net=1) */
MODULE_PARM_DESC(disable_net, "Disable the network interface");
-module_param(disable_net, int, S_IRUGO | S_IWUSR);
+module_param(disable_net, int, 0644);
void usbnet_skb_return (struct usbnet *dev, struct sk_buff *skb)
{
struct pcpu_sw_netstats *stats64 = this_cpu_ptr(dev->stats64);
+ unsigned long flags;
int status;
if (test_bit(EVENT_RX_PAUSED, &dev->flags)) {
if (skb->protocol == 0)
skb->protocol = eth_type_trans (skb, dev->net);
- u64_stats_update_begin(&stats64->syncp);
+ flags = u64_stats_update_begin_irqsave(&stats64->syncp);
stats64->rx_packets++;
stats64->rx_bytes += skb->len;
- u64_stats_update_end(&stats64->syncp);
+ u64_stats_update_end_irqrestore(&stats64->syncp, flags);
netif_dbg(dev, rx_status, dev->net, "< rx, len %zu, type 0x%x\n",
skb->len + sizeof (struct ethhdr), skb->protocol);
if (urb->status == 0) {
struct pcpu_sw_netstats *stats64 = this_cpu_ptr(dev->stats64);
+ unsigned long flags;
- u64_stats_update_begin(&stats64->syncp);
+ flags = u64_stats_update_begin_irqsave(&stats64->syncp);
stats64->tx_packets += entry->packets;
stats64->tx_bytes += entry->length;
- u64_stats_update_end(&stats64->syncp);
+ u64_stats_update_end_irqrestore(&stats64->syncp, flags);
} else {
dev->net->stats.tx_errors++;
/* Assume link up if device can't report link status,
otherwise get link status from config. */
+ netif_carrier_off(dev);
if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_STATUS)) {
- netif_carrier_off(dev);
schedule_work(&vi->config_work);
} else {
vi->status = VIRTIO_NET_S_LINK_UP;
{
int ret;
u32 count;
+ int num_pkts;
+ int tx_num_deferred;
unsigned long flags;
struct vmxnet3_tx_ctx ctx;
union Vmxnet3_GenericDesc *gdesc;
#else
gdesc = ctx.sop_txd;
#endif
+ tx_num_deferred = le32_to_cpu(tq->shared->txNumDeferred);
if (ctx.mss) {
gdesc->txd.hlen = ctx.eth_ip_hdr_size + ctx.l4_hdr_size;
gdesc->txd.om = VMXNET3_OM_TSO;
gdesc->txd.msscof = ctx.mss;
- le32_add_cpu(&tq->shared->txNumDeferred, (skb->len -
- gdesc->txd.hlen + ctx.mss - 1) / ctx.mss);
+ num_pkts = (skb->len - gdesc->txd.hlen + ctx.mss - 1) / ctx.mss;
} else {
if (skb->ip_summed == CHECKSUM_PARTIAL) {
gdesc->txd.hlen = ctx.eth_ip_hdr_size;
gdesc->txd.om = 0;
gdesc->txd.msscof = 0;
}
- le32_add_cpu(&tq->shared->txNumDeferred, 1);
+ num_pkts = 1;
}
+ le32_add_cpu(&tq->shared->txNumDeferred, num_pkts);
+ tx_num_deferred += num_pkts;
if (skb_vlan_tag_present(skb)) {
gdesc->txd.ti = 1;
spin_unlock_irqrestore(&tq->tx_lock, flags);
- if (le32_to_cpu(tq->shared->txNumDeferred) >=
- le32_to_cpu(tq->shared->txThreshold)) {
+ if (tx_num_deferred >= le32_to_cpu(tq->shared->txThreshold)) {
tq->shared->txNumDeferred = 0;
VMXNET3_WRITE_BAR0_REG(adapter,
VMXNET3_REG_TXPROD + tq->qid * 8,
vmxnet3_rx_csum(adapter, skb,
(union Vmxnet3_GenericDesc *)rcd);
skb->protocol = eth_type_trans(skb, adapter->netdev);
- if (!rcd->tcp || !adapter->lro)
+ if (!rcd->tcp ||
+ !(adapter->netdev->features & NETIF_F_LRO))
goto not_lro;
if (segCnt != 0 && mss != 0) {
/*
* Version numbers
*/
-#define VMXNET3_DRIVER_VERSION_STRING "1.4.11.0-k"
+#define VMXNET3_DRIVER_VERSION_STRING "1.4.13.0-k"
/* a 32-bit int, each byte encode a verion number in VMXNET3_DRIVER_VERSION */
-#define VMXNET3_DRIVER_VERSION_NUM 0x01040b00
+#define VMXNET3_DRIVER_VERSION_NUM 0x01040d00
#if defined(CONFIG_PCI_MSI)
/* RSS only makes sense if MSI-X is supported. */
u8 __iomem *hw_addr1; /* for BAR 1 */
u8 version;
- bool rxcsum;
- bool lro;
-
#ifdef VMXNET3_RSS
struct UPT1_RSSConf *rss_conf;
bool rss;
.init = vrf_netns_init,
.id = &vrf_net_id,
.size = sizeof(bool),
- .async = true,
};
static int __init vrf_init_module(void)
.exit_batch = vxlan_exit_batch_net,
.id = &vxlan_net_id,
.size = sizeof(struct vxlan_net),
- .async = true,
};
static int __init vxlan_init_module(void)
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_ADMTEK
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
For more information and documentation on this module you can visit:
ieee80211_hw_set(hw, SPECTRUM_MGMT);
ieee80211_hw_set(hw, SIGNAL_DBM);
ieee80211_hw_set(hw, AMPDU_AGGREGATION);
+ ieee80211_hw_set(hw, DOESNT_SUPPORT_QOS_NDP);
if (ath9k_ps_enable)
ieee80211_hw_set(hw, SUPPORTS_PS);
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_ATMEL
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_BROADCOM
/* Configure the "global" bus state used by upper layers */
void brcmf_bus_change_state(struct brcmf_bus *bus, enum brcmf_bus_state state);
-int brcmf_bus_started(struct device *dev);
s32 brcmf_iovar_data_set(struct device *dev, char *name, void *data, u32 len);
void brcmf_bus_add_txhdrlen(struct device *dev, uint len);
if (WARN_ON(ifp->vif->profile.use_fwsup != BRCMF_PROFILE_FWSUP_1X))
return -EINVAL;
+ if (conf->pmk_len > BRCMF_WSEC_MAX_PSK_LEN)
+ return -ERANGE;
+
return brcmf_set_pmk(ifp, conf->pmk, conf->pmk_len);
}
/* Enable tx beamforming, errors can be ignored (not supported) */
(void)brcmf_fil_iovar_int_set(ifp, "txbf", 1);
-
- /* do bus specific preinit here */
- err = brcmf_bus_preinit(ifp->drvr->bus_if);
done:
return err;
}
}
#endif
-int brcmf_attach(struct device *dev, struct brcmf_mp_device *settings)
-{
- struct brcmf_pub *drvr = NULL;
- int ret = 0;
- int i;
-
- brcmf_dbg(TRACE, "Enter\n");
-
- /* Allocate primary brcmf_info */
- drvr = kzalloc(sizeof(struct brcmf_pub), GFP_ATOMIC);
- if (!drvr)
- return -ENOMEM;
-
- for (i = 0; i < ARRAY_SIZE(drvr->if2bss); i++)
- drvr->if2bss[i] = BRCMF_BSSIDX_INVALID;
-
- mutex_init(&drvr->proto_block);
-
- /* Link to bus module */
- drvr->hdrlen = 0;
- drvr->bus_if = dev_get_drvdata(dev);
- drvr->bus_if->drvr = drvr;
- drvr->settings = settings;
-
- /* attach debug facilities */
- brcmf_debug_attach(drvr);
-
- /* Attach and link in the protocol */
- ret = brcmf_proto_attach(drvr);
- if (ret != 0) {
- brcmf_err("brcmf_prot_attach failed\n");
- goto fail;
- }
-
- /* Attach to events important for core code */
- brcmf_fweh_register(drvr, BRCMF_E_PSM_WATCHDOG,
- brcmf_psm_watchdog_notify);
-
- /* attach firmware event handler */
- brcmf_fweh_attach(drvr);
-
- return ret;
-
-fail:
- brcmf_detach(dev);
-
- return ret;
-}
-
static int brcmf_revinfo_read(struct seq_file *s, void *data)
{
struct brcmf_bus *bus_if = dev_get_drvdata(s->private);
return 0;
}
-int brcmf_bus_started(struct device *dev)
+static int brcmf_bus_started(struct brcmf_pub *drvr)
{
int ret = -1;
- struct brcmf_bus *bus_if = dev_get_drvdata(dev);
- struct brcmf_pub *drvr = bus_if->drvr;
+ struct brcmf_bus *bus_if = drvr->bus_if;
struct brcmf_if *ifp;
struct brcmf_if *p2p_ifp;
/* signal bus ready */
brcmf_bus_change_state(bus_if, BRCMF_BUS_UP);
+ /* do bus specific preinit here */
+ ret = brcmf_bus_preinit(bus_if);
+ if (ret < 0)
+ goto fail;
+
/* Bus is ready, do any initialization */
ret = brcmf_c_preinit_dcmds(ifp);
if (ret < 0)
return ret;
}
+int brcmf_attach(struct device *dev, struct brcmf_mp_device *settings)
+{
+ struct brcmf_pub *drvr = NULL;
+ int ret = 0;
+ int i;
+
+ brcmf_dbg(TRACE, "Enter\n");
+
+ /* Allocate primary brcmf_info */
+ drvr = kzalloc(sizeof(*drvr), GFP_ATOMIC);
+ if (!drvr)
+ return -ENOMEM;
+
+ for (i = 0; i < ARRAY_SIZE(drvr->if2bss); i++)
+ drvr->if2bss[i] = BRCMF_BSSIDX_INVALID;
+
+ mutex_init(&drvr->proto_block);
+
+ /* Link to bus module */
+ drvr->hdrlen = 0;
+ drvr->bus_if = dev_get_drvdata(dev);
+ drvr->bus_if->drvr = drvr;
+ drvr->settings = settings;
+
+ /* attach debug facilities */
+ brcmf_debug_attach(drvr);
+
+ /* Attach and link in the protocol */
+ ret = brcmf_proto_attach(drvr);
+ if (ret != 0) {
+ brcmf_err("brcmf_prot_attach failed\n");
+ goto fail;
+ }
+
+ /* Attach to events important for core code */
+ brcmf_fweh_register(drvr, BRCMF_E_PSM_WATCHDOG,
+ brcmf_psm_watchdog_notify);
+
+ /* attach firmware event handler */
+ brcmf_fweh_attach(drvr);
+
+ ret = brcmf_bus_started(drvr);
+ if (ret != 0) {
+ brcmf_err("dongle is not responding: err=%d\n", ret);
+ goto fail;
+ }
+ return 0;
+
+fail:
+ brcmf_detach(dev);
+
+ return ret;
+}
+
void brcmf_bus_add_txhdrlen(struct device *dev, uint len)
{
struct brcmf_bus *bus_if = dev_get_drvdata(dev);
int ifidx;
brcmf_dbg(TRACE, "%d -> %d\n", bus->state, state);
+
+ if (!drvr) {
+ brcmf_dbg(INFO, "ignoring transition, bus not attached yet\n");
+ return;
+ }
+
bus->state = state;
if (state == BRCMF_BUS_UP) {
* @netif_stop_lock: spinlock for update netif_stop from multiple sources.
* @pend_8021x_cnt: tracks outstanding number of 802.1x frames.
* @pend_8021x_wait: used for signalling change in count.
+ * @fwil_fwerr: flag indicating fwil layer should return firmware error codes.
*/
struct brcmf_if {
struct brcmf_pub *drvr;
wait_queue_head_t pend_8021x_wait;
struct in6_addr ipv6_addr_tbl[NDOL_MAX_ENTRIES];
u8 ipv6addr_idx;
+ bool fwil_fwerr;
};
int brcmf_netdev_wait_pend8021x(struct brcmf_if *ifp);
u32 data;
int err;
+ /* we need to know firmware error */
+ ifp->fwil_fwerr = true;
+
err = brcmf_fil_iovar_int_get(ifp, name, &data);
if (err == 0) {
brcmf_dbg(INFO, "enabling feature: %s\n", brcmf_feat_names[id]);
brcmf_dbg(TRACE, "%s feature check failed: %d\n",
brcmf_feat_names[id], err);
}
+
+ ifp->fwil_fwerr = false;
}
static void brcmf_feat_iovar_data_set(struct brcmf_if *ifp,
{
int err;
+ /* we need to know firmware error */
+ ifp->fwil_fwerr = true;
+
err = brcmf_fil_iovar_data_set(ifp, name, data, len);
if (err != -BRCMF_FW_UNSUPPORTED) {
brcmf_dbg(INFO, "enabling feature: %s\n", brcmf_feat_names[id]);
brcmf_dbg(TRACE, "%s feature check failed: %d\n",
brcmf_feat_names[id], err);
}
+
+ ifp->fwil_fwerr = false;
}
#define MAX_CAPS_BUFFER_SIZE 512
brcmf_fil_get_errstr((u32)(-fwerr)), fwerr);
err = -EBADE;
}
+ if (ifp->fwil_fwerr)
+ return fwerr;
+
return err;
}
* @dev_addr: optional device address.
*
* P2P needs mac addresses for P2P device and interface. If no device
- * address it specified, these are derived from the primary net device, ie.
- * the permanent ethernet address of the device.
+ * address it specified, these are derived from a random ethernet
+ * address.
*/
static void brcmf_p2p_generate_bss_mac(struct brcmf_p2p_info *p2p, u8 *dev_addr)
{
- struct brcmf_if *pri_ifp = p2p->bss_idx[P2PAPI_BSSCFG_PRIMARY].vif->ifp;
- bool local_admin = false;
+ bool random_addr = false;
- if (!dev_addr || is_zero_ether_addr(dev_addr)) {
- dev_addr = pri_ifp->mac_addr;
- local_admin = true;
- }
+ if (!dev_addr || is_zero_ether_addr(dev_addr))
+ random_addr = true;
- /* Generate the P2P Device Address. This consists of the device's
- * primary MAC address with the locally administered bit set.
+ /* Generate the P2P Device Address obtaining a random ethernet
+ * address with the locally administered bit set.
*/
- memcpy(p2p->dev_addr, dev_addr, ETH_ALEN);
- if (local_admin)
- p2p->dev_addr[0] |= 0x02;
+ if (random_addr)
+ eth_random_addr(p2p->dev_addr);
+ else
+ memcpy(p2p->dev_addr, dev_addr, ETH_ALEN);
/* Generate the P2P Interface Address. If the discovery and connection
* BSSCFGs need to simultaneously co-exist, then this address must be
}
-static int brcmf_pcie_attach_bus(struct brcmf_pciedev_info *devinfo)
-{
- int ret;
-
- /* Attach to the common driver interface */
- ret = brcmf_attach(&devinfo->pdev->dev, devinfo->settings);
- if (ret) {
- brcmf_err("brcmf_attach failed\n");
- } else {
- ret = brcmf_bus_started(&devinfo->pdev->dev);
- if (ret)
- brcmf_err("dongle is not responding\n");
- }
-
- return ret;
-}
-
-
static u32 brcmf_pcie_buscore_prep_addr(const struct pci_dev *pdev, u32 addr)
{
u32 ret_addr;
init_waitqueue_head(&devinfo->mbdata_resp_wait);
brcmf_pcie_intr_enable(devinfo);
- if (brcmf_pcie_attach_bus(devinfo) == 0)
+ if (brcmf_attach(&devinfo->pdev->dev, devinfo->settings) == 0)
return;
brcmf_pcie_bus_console_read(devinfo);
u8 *buf = NULL, *rbuf;
int sdret;
- brcmf_dbg(TRACE, "Enter\n");
-
+ brcmf_dbg(SDIO, "Enter\n");
if (bus->rxblen)
buf = vzalloc(bus->rxblen);
if (!buf)
struct brcmf_sdio_hdrinfo *rd = &bus->cur_read, rd_new;
u8 head_read = 0;
- brcmf_dbg(TRACE, "Enter\n");
+ brcmf_dbg(SDIO, "Enter\n");
/* Not finished unless we encounter no more frames indication */
bus->rxpending = true;
struct brcmf_sdio_hdrinfo hd_info = {0};
int ret;
- brcmf_dbg(TRACE, "Enter\n");
+ brcmf_dbg(SDIO, "Enter\n");
/* Back the pointer to make room for bus header */
frame -= bus->tx_hdrlen;
uint framecnt; /* Temporary counter of tx/rx frames */
int err = 0;
- brcmf_dbg(TRACE, "Enter\n");
+ brcmf_dbg(SDIO, "Enter\n");
sdio_claim_host(bus->sdiodev->func1);
/* Would be active due to wake-wlan in gSPI */
if (intstatus & I_CHIPACTIVE) {
- brcmf_dbg(INFO, "Dongle reports CHIPACTIVE\n");
+ brcmf_dbg(SDIO, "Dongle reports CHIPACTIVE\n");
intstatus &= ~I_CHIPACTIVE;
}
u32 value;
int err;
+ /* maxctl provided by common layer */
+ if (WARN_ON(!bus_if->maxctl))
+ return -EINVAL;
+
+ /* Allocate control receive buffer */
+ bus_if->maxctl += bus->roundup;
+ value = roundup((bus_if->maxctl + SDPCM_HDRLEN), ALIGNMENT);
+ value += bus->head_align;
+ bus->rxbuf = kmalloc(value, GFP_ATOMIC);
+ if (bus->rxbuf)
+ bus->rxblen = value;
+
+ brcmf_sdio_debugfs_create(bus);
+
/* the commands below use the terms tx and rx from
* a device perspective, ie. bus:txglom affects the
* bus transfers from device to host.
void *nvram, u32 nvram_len)
{
struct brcmf_bus *bus_if = dev_get_drvdata(dev);
- struct brcmf_sdio_dev *sdiodev = bus_if->bus_priv.sdio;
- struct brcmf_sdio *bus = sdiodev->bus;
- struct brcmf_sdio_dev *sdiod = bus->sdiodev;
+ struct brcmf_sdio_dev *sdiod = bus_if->bus_priv.sdio;
+ struct brcmf_sdio *bus = sdiod->bus;
struct brcmf_core *core = bus->sdio_core;
u8 saveclk;
if (err)
goto fail;
- if (!bus_if->drvr)
- return;
-
/* try to download image and nvram to the dongle */
bus->alp_only = true;
err = brcmf_sdio_download_firmware(bus, code, nvram, nvram_len);
bus->sdcnt.tickcnt = 0;
brcmf_sdio_wd_timer(bus, true);
- sdio_claim_host(sdiodev->func1);
+ sdio_claim_host(sdiod->func1);
/* Make sure backplane clock is on, needed to generate F2 interrupt */
brcmf_sdio_clkctl(bus, CLK_AVAIL, false);
goto release;
/* Force clocks on backplane to be sure F2 interrupt propagates */
- saveclk = brcmf_sdiod_readb(sdiodev, SBSDIO_FUNC1_CHIPCLKCSR, &err);
+ saveclk = brcmf_sdiod_readb(sdiod, SBSDIO_FUNC1_CHIPCLKCSR, &err);
if (!err) {
- brcmf_sdiod_writeb(sdiodev, SBSDIO_FUNC1_CHIPCLKCSR,
+ brcmf_sdiod_writeb(sdiod, SBSDIO_FUNC1_CHIPCLKCSR,
(saveclk | SBSDIO_FORCE_HT), &err);
}
if (err) {
brcmf_sdiod_writel(sdiod, core->base + SD_REG(tosbmailboxdata),
SDPCM_PROT_VERSION << SMB_DATA_VERSION_SHIFT, NULL);
- err = sdio_enable_func(sdiodev->func2);
+ err = sdio_enable_func(sdiod->func2);
brcmf_dbg(INFO, "enable F2: err=%d\n", err);
bus->hostintmask, NULL);
- brcmf_sdiod_writeb(sdiodev, SBSDIO_WATERMARK, 8, &err);
+ brcmf_sdiod_writeb(sdiod, SBSDIO_WATERMARK, 8, &err);
} else {
/* Disable F2 again */
- sdio_disable_func(sdiodev->func2);
+ sdio_disable_func(sdiod->func2);
goto release;
}
brcmf_sdio_sr_init(bus);
} else {
/* Restore previous clock setting */
- brcmf_sdiod_writeb(sdiodev, SBSDIO_FUNC1_CHIPCLKCSR,
+ brcmf_sdiod_writeb(sdiod, SBSDIO_FUNC1_CHIPCLKCSR,
saveclk, &err);
}
/* Allow full data communication using DPC from now on. */
brcmf_sdiod_change_state(bus->sdiodev, BRCMF_SDIOD_DATA);
- err = brcmf_sdiod_intr_register(sdiodev);
+ err = brcmf_sdiod_intr_register(sdiod);
if (err != 0)
brcmf_err("intr register failed:%d\n", err);
}
if (err != 0)
brcmf_sdio_clkctl(bus, CLK_NONE, false);
- sdio_release_host(sdiodev->func1);
+ sdio_release_host(sdiod->func1);
- err = brcmf_bus_started(dev);
+ /* Assign bus interface call back */
+ sdiod->bus_if->dev = sdiod->dev;
+ sdiod->bus_if->ops = &brcmf_sdio_bus_ops;
+ sdiod->bus_if->chip = bus->ci->chip;
+ sdiod->bus_if->chiprev = bus->ci->chiprev;
+
+ /* Attach to the common layer, reserve hdr space */
+ err = brcmf_attach(sdiod->dev, sdiod->settings);
if (err != 0) {
- brcmf_err("dongle is not responding\n");
+ brcmf_err("brcmf_attach failed\n");
goto fail;
}
+
+ /* ready */
return;
release:
- sdio_release_host(sdiodev->func1);
+ sdio_release_host(sdiod->func1);
fail:
brcmf_dbg(TRACE, "failed: dev=%s, err=%d\n", dev_name(dev), err);
- device_release_driver(&sdiodev->func2->dev);
+ device_release_driver(&sdiod->func2->dev);
device_release_driver(dev);
}
bus->dpc_triggered = false;
bus->dpc_running = false;
- /* Assign bus interface call back */
- bus->sdiodev->bus_if->dev = bus->sdiodev->dev;
- bus->sdiodev->bus_if->ops = &brcmf_sdio_bus_ops;
- bus->sdiodev->bus_if->chip = bus->ci->chip;
- bus->sdiodev->bus_if->chiprev = bus->ci->chiprev;
-
/* default sdio bus header length for tx packet */
bus->tx_hdrlen = SDPCM_HWHDR_LEN + SDPCM_SWHDR_LEN;
- /* Attach to the common layer, reserve hdr space */
- ret = brcmf_attach(bus->sdiodev->dev, bus->sdiodev->settings);
- if (ret != 0) {
- brcmf_err("brcmf_attach failed\n");
- goto fail;
- }
-
/* Query the F2 block size, set roundup accordingly */
bus->blocksize = bus->sdiodev->func2->cur_blksize;
bus->roundup = min(max_roundup, bus->blocksize);
- /* Allocate buffers */
- if (bus->sdiodev->bus_if->maxctl) {
- bus->sdiodev->bus_if->maxctl += bus->roundup;
- bus->rxblen =
- roundup((bus->sdiodev->bus_if->maxctl + SDPCM_HDRLEN),
- ALIGNMENT) + bus->head_align;
- bus->rxbuf = kmalloc(bus->rxblen, GFP_ATOMIC);
- if (!(bus->rxbuf)) {
- brcmf_err("rxbuf allocation failed\n");
- goto fail;
- }
- }
-
sdio_claim_host(bus->sdiodev->func1);
/* Disable F2 to clear any intermediate frame state on the dongle */
/* SR state */
bus->sr_enabled = false;
- brcmf_sdio_debugfs_create(bus);
brcmf_dbg(INFO, "completed!!\n");
ret = brcmf_fw_map_chip_to_name(bus->ci->chip, bus->ci->chiprev,
}
static const struct brcmf_bus_ops brcmf_usb_bus_ops = {
- .txdata = brcmf_usb_tx,
+ .preinit = brcmf_usb_up,
.stop = brcmf_usb_down,
+ .txdata = brcmf_usb_tx,
.txctl = brcmf_usb_tx_ctlpkt,
.rxctl = brcmf_usb_rx_ctlpkt,
.wowl_config = brcmf_usb_wowl_config,
.get_fwname = brcmf_usb_get_fwname,
};
-static int brcmf_usb_bus_setup(struct brcmf_usbdev_info *devinfo)
-{
- int ret;
-
- /* Attach to the common driver interface */
- ret = brcmf_attach(devinfo->dev, devinfo->settings);
- if (ret) {
- brcmf_err("brcmf_attach failed\n");
- return ret;
- }
-
- ret = brcmf_usb_up(devinfo->dev);
- if (ret)
- goto fail;
-
- ret = brcmf_bus_started(devinfo->dev);
- if (ret)
- goto fail;
-
- return 0;
-fail:
- brcmf_detach(devinfo->dev);
- return ret;
-}
-
static void brcmf_usb_probe_phase2(struct device *dev, int ret,
const struct firmware *fw,
void *nvram, u32 nvlen)
if (ret)
goto error;
- ret = brcmf_usb_bus_setup(devinfo);
+ /* Attach to the common driver interface */
+ ret = brcmf_attach(devinfo->dev, devinfo->settings);
if (ret)
goto error;
}
if (!brcmf_usb_dlneeded(devinfo)) {
- ret = brcmf_usb_bus_setup(devinfo);
+ ret = brcmf_attach(devinfo->dev, devinfo->settings);
if (ret)
goto fail;
/* we are done */
brcmf_dbg(USB, "Enter\n");
if (!devinfo->wowl_enabled)
- return brcmf_usb_bus_setup(devinfo);
+ return brcmf_attach(devinfo->dev, devinfo->settings);
devinfo->bus_pub.state = BRCMFMAC_USB_STATE_UP;
brcmf_usb_rx_fill_all(devinfo);
*/
if (!(ch->flags & IEEE80211_CHAN_DISABLED))
ch->flags |= IEEE80211_CHAN_RADAR |
- IEEE80211_CHAN_NO_IR |
IEEE80211_CHAN_NO_IR;
}
}
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_CISCO
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_INTEL
config IWLWIFI_PCIE_RTPM
bool "Enable runtime power management mode for PCIe devices"
depends on IWLMVM && PM && EXPERT
- default false
help
Say Y here to enable runtime power management for PCIe
devices. If enabled, the device will go into low power mode
* @TE_V2_NOTIF_HOST_FRAG_END:request/receive notification on frag end
* @TE_V2_NOTIF_INTERNAL_FRAG_START: internal FW use.
* @TE_V2_NOTIF_INTERNAL_FRAG_END: internal FW use.
- * @T2_V2_START_IMMEDIATELY: start time event immediately
+ * @TE_V2_START_IMMEDIATELY: start time event immediately
* @TE_V2_DEP_OTHER: depends on another time event
* @TE_V2_DEP_TSF: depends on a specific time
* @TE_V2_EVENT_SOCIOPATHIC: can't co-exist with other events of tha same MAC
TE_V2_NOTIF_HOST_FRAG_END = BIT(5),
TE_V2_NOTIF_INTERNAL_FRAG_START = BIT(6),
TE_V2_NOTIF_INTERNAL_FRAG_END = BIT(7),
- T2_V2_START_IMMEDIATELY = BIT(11),
+ TE_V2_START_IMMEDIATELY = BIT(11),
/* placement characteristics */
TE_V2_DEP_OTHER = BIT(TE_V2_PLACEMENT_POS),
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2015 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2015 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
out:
iwl_fw_free_dump_desc(fwrt);
- fwrt->dump.trig = NULL;
clear_bit(IWL_FWRT_STATUS_DUMPING, &fwrt->status);
IWL_DEBUG_INFO(fwrt, "WRT dump done\n");
}
fwrt->ops->dump_start(fwrt->ops_ctx))
return;
+ if (fwrt->ops && fwrt->ops->fw_running &&
+ !fwrt->ops->fw_running(fwrt->ops_ctx)) {
+ IWL_ERR(fwrt, "Firmware not running - cannot dump error\n");
+ iwl_fw_free_dump_desc(fwrt);
+ clear_bit(IWL_FWRT_STATUS_DUMPING, &fwrt->status);
+ goto out;
+ }
+
if (fwrt->trans->cfg->device_family == IWL_DEVICE_FAMILY_7000) {
/* stop recording */
iwl_fw_dbg_stop_recording(fwrt);
iwl_write_prph(fwrt->trans, DBGC_OUT_CTRL, out_ctrl);
}
}
-
+out:
if (fwrt->ops && fwrt->ops->dump_end)
fwrt->ops->dump_end(fwrt->ops_ctx);
}
* Copyright(c) 2008 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2015 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* Copyright(c) 2005 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2015 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
if (fwrt->dump.desc != &iwl_dump_desc_assert)
kfree(fwrt->dump.desc);
fwrt->dump.desc = NULL;
+ fwrt->dump.trig = NULL;
}
void iwl_fw_error_dump(struct iwl_fw_runtime *fwrt);
cancel_delayed_work_sync(&fwrt->timestamp.wk);
}
+static inline void iwl_fw_suspend_timestamp(struct iwl_fw_runtime *fwrt)
+{
+ cancel_delayed_work_sync(&fwrt->timestamp.wk);
+}
+
+static inline void iwl_fw_resume_timestamp(struct iwl_fw_runtime *fwrt)
+{
+ if (!fwrt->timestamp.delay)
+ return;
+
+ schedule_delayed_work(&fwrt->timestamp.wk,
+ round_jiffies_relative(fwrt->timestamp.delay));
+}
+
#else
static inline int iwl_fwrt_dbgfs_register(struct iwl_fw_runtime *fwrt,
struct dentry *dbgfs_dir)
static inline void iwl_fw_cancel_timestamp(struct iwl_fw_runtime *fwrt) {}
+static inline void iwl_fw_suspend_timestamp(struct iwl_fw_runtime *fwrt) {}
+
+static inline void iwl_fw_resume_timestamp(struct iwl_fw_runtime *fwrt) {}
+
#endif /* CONFIG_IWLWIFI_DEBUGFS */
}
IWL_EXPORT_SYMBOL(iwl_fw_runtime_init);
-void iwl_fw_runtime_exit(struct iwl_fw_runtime *fwrt)
+void iwl_fw_runtime_suspend(struct iwl_fw_runtime *fwrt)
{
- iwl_fw_cancel_timestamp(fwrt);
+ iwl_fw_suspend_timestamp(fwrt);
}
-IWL_EXPORT_SYMBOL(iwl_fw_runtime_exit);
+IWL_EXPORT_SYMBOL(iwl_fw_runtime_suspend);
+
+void iwl_fw_runtime_resume(struct iwl_fw_runtime *fwrt)
+{
+ iwl_fw_resume_timestamp(fwrt);
+}
+IWL_EXPORT_SYMBOL(iwl_fw_runtime_resume);
* GPL LICENSE SUMMARY
*
* Copyright(c) 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* BSD LICENSE
*
* Copyright(c) 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
struct iwl_fw_runtime_ops {
int (*dump_start)(void *ctx);
void (*dump_end)(void *ctx);
+ bool (*fw_running)(void *ctx);
};
#define MAX_NUM_LMAC 2
void iwl_fw_runtime_exit(struct iwl_fw_runtime *fwrt);
+void iwl_fw_runtime_suspend(struct iwl_fw_runtime *fwrt);
+
+void iwl_fw_runtime_resume(struct iwl_fw_runtime *fwrt);
+
static inline void iwl_fw_set_current_image(struct iwl_fw_runtime *fwrt,
enum iwl_ucode_type cur_fw_img)
{
/* make sure the d0i3 exit work is not pending */
flush_work(&mvm->d0i3_exit_work);
+ iwl_fw_runtime_suspend(&mvm->fwrt);
+
ret = iwl_trans_suspend(trans);
if (ret)
return ret;
mvm->trans->system_pm_mode = IWL_PLAT_PM_MODE_DISABLED;
+ iwl_fw_runtime_resume(&mvm->fwrt);
+
return ret;
}
mvm->trans->system_pm_mode = IWL_PLAT_PM_MODE_D3;
+ iwl_fw_runtime_suspend(&mvm->fwrt);
+
/* start pseudo D3 */
rtnl_lock();
err = __iwl_mvm_suspend(mvm->hw, mvm->hw->wiphy->wowlan_config, true);
__iwl_mvm_resume(mvm, true);
rtnl_unlock();
+ iwl_fw_runtime_resume(&mvm->fwrt);
+
mvm->trans->system_pm_mode = IWL_PLAT_PM_MODE_DISABLED;
iwl_abort_notification_waits(&mvm->notif_wait);
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
{
int ret;
- if (!iwl_mvm_firmware_running(mvm))
- return -EIO;
-
ret = iwl_mvm_ref_sync(mvm, IWL_MVM_REF_PRPH_WRITE);
if (ret)
return ret;
}
/* Allocate the CAB queue for softAP and GO interfaces */
- if (vif->type == NL80211_IFTYPE_AP) {
+ if (vif->type == NL80211_IFTYPE_AP ||
+ vif->type == NL80211_IFTYPE_ADHOC) {
/*
* For TVQM this will be overwritten later with the FW assigned
* queue value (when queue is enabled).
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
if (ret)
goto out_remove;
- ret = iwl_mvm_add_mcast_sta(mvm, vif);
- if (ret)
- goto out_unbind;
-
- /* Send the bcast station. At this stage the TBTT and DTIM time events
- * are added and applied to the scheduler */
- ret = iwl_mvm_send_add_bcast_sta(mvm, vif);
- if (ret)
- goto out_rm_mcast;
+ /*
+ * This is not very nice, but the simplest:
+ * For older FWs adding the mcast sta before the bcast station may
+ * cause assert 0x2b00.
+ * This is fixed in later FW so make the order of removal depend on
+ * the TLV
+ */
+ if (fw_has_api(&mvm->fw->ucode_capa, IWL_UCODE_TLV_API_STA_TYPE)) {
+ ret = iwl_mvm_add_mcast_sta(mvm, vif);
+ if (ret)
+ goto out_unbind;
+ /*
+ * Send the bcast station. At this stage the TBTT and DTIM time
+ * events are added and applied to the scheduler
+ */
+ ret = iwl_mvm_send_add_bcast_sta(mvm, vif);
+ if (ret) {
+ iwl_mvm_rm_mcast_sta(mvm, vif);
+ goto out_unbind;
+ }
+ } else {
+ /*
+ * Send the bcast station. At this stage the TBTT and DTIM time
+ * events are added and applied to the scheduler
+ */
+ iwl_mvm_send_add_bcast_sta(mvm, vif);
+ if (ret)
+ goto out_unbind;
+ iwl_mvm_add_mcast_sta(mvm, vif);
+ if (ret) {
+ iwl_mvm_send_rm_bcast_sta(mvm, vif);
+ goto out_unbind;
+ }
+ }
/* must be set before quota calculations */
mvmvif->ap_ibss_active = true;
iwl_mvm_power_update_mac(mvm);
mvmvif->ap_ibss_active = false;
iwl_mvm_send_rm_bcast_sta(mvm, vif);
-out_rm_mcast:
iwl_mvm_rm_mcast_sta(mvm, vif);
out_unbind:
iwl_mvm_binding_remove_vif(mvm, vif);
/* enable beacon filtering */
WARN_ON(iwl_mvm_enable_beacon_filter(mvm, vif, 0));
+
+ iwl_mvm_rs_rate_init(mvm, sta, mvmvif->phy_ctxt->channel->band,
+ false);
+
ret = 0;
} else if (old_state == IEEE80211_STA_AUTHORIZED &&
new_state == IEEE80211_STA_ASSOC) {
#include "fw/runtime.h"
#include "fw/dbg.h"
#include "fw/acpi.h"
+#include "fw/debugfs.h"
#define IWL_MVM_MAX_ADDRESSES 5
/* RSSI offset for WkP */
static inline void iwl_mvm_stop_device(struct iwl_mvm *mvm)
{
+ iwl_fw_cancel_timestamp(&mvm->fwrt);
iwl_free_fw_paging(&mvm->fwrt);
clear_bit(IWL_MVM_STATUS_FIRMWARE_RUNNING, &mvm->status);
iwl_fw_dump_conf_clear(&mvm->fwrt);
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of version 2 of the GNU General Public License as
* Copyright(c) 2012 - 2014 Intel Corporation. All rights reserved.
* Copyright(c) 2013 - 2015 Intel Mobile Communications GmbH
* Copyright(c) 2016 - 2017 Intel Deutschland GmbH
+ * Copyright(c) 2018 Intel Corporation
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
iwl_mvm_unref(mvm, IWL_MVM_REF_FW_DBG_COLLECT);
}
+static bool iwl_mvm_fwrt_fw_running(void *ctx)
+{
+ return iwl_mvm_firmware_running(ctx);
+}
+
static const struct iwl_fw_runtime_ops iwl_mvm_fwrt_ops = {
.dump_start = iwl_mvm_fwrt_dump_start,
.dump_end = iwl_mvm_fwrt_dump_end,
+ .fw_running = iwl_mvm_fwrt_fw_running,
};
static struct iwl_op_mode *
iwl_mvm_leds_exit(mvm);
iwl_mvm_thermal_exit(mvm);
out_free:
- iwl_fw_runtime_exit(&mvm->fwrt);
iwl_fw_flush_dump(&mvm->fwrt);
if (iwlmvm_mod_params.init_dbg)
#if defined(CONFIG_PM_SLEEP) && defined(CONFIG_IWLWIFI_DEBUGFS)
kfree(mvm->d3_resume_sram);
#endif
- iwl_fw_runtime_exit(&mvm->fwrt);
iwl_trans_op_mode_leave(mvm->trans);
iwl_phy_db_free(mvm->phy_db);
struct ieee80211_sta *sta,
struct iwl_lq_sta *lq_sta,
enum nl80211_band band,
- struct rs_rate *rate)
+ struct rs_rate *rate,
+ bool init)
{
int i, nentries;
unsigned long active_rate;
*/
if (sta->vht_cap.vht_supported &&
best_rssi > IWL_RS_LOW_RSSI_THRESHOLD) {
- switch (sta->bandwidth) {
- case IEEE80211_STA_RX_BW_160:
- case IEEE80211_STA_RX_BW_80:
- case IEEE80211_STA_RX_BW_40:
+ /*
+ * In AP mode, when a new station associates, rs is initialized
+ * immediately upon association completion, before the phy
+ * context is updated with the association parameters, so the
+ * sta bandwidth might be wider than the phy context allows.
+ * To avoid this issue, always initialize rs with 20mhz
+ * bandwidth rate, and after authorization, when the phy context
+ * is already up-to-date, re-init rs with the correct bw.
+ */
+ u32 bw = init ? RATE_MCS_CHAN_WIDTH_20 : rs_bw_from_sta_bw(sta);
+
+ switch (bw) {
+ case RATE_MCS_CHAN_WIDTH_40:
+ case RATE_MCS_CHAN_WIDTH_80:
+ case RATE_MCS_CHAN_WIDTH_160:
initial_rates = rs_optimal_rates_vht;
nentries = ARRAY_SIZE(rs_optimal_rates_vht);
break;
- case IEEE80211_STA_RX_BW_20:
+ case RATE_MCS_CHAN_WIDTH_20:
initial_rates = rs_optimal_rates_vht_20mhz;
nentries = ARRAY_SIZE(rs_optimal_rates_vht_20mhz);
break;
active_rate = lq_sta->active_siso_rate;
rate->type = LQ_VHT_SISO;
- rate->bw = rs_bw_from_sta_bw(sta);
+ rate->bw = bw;
} else if (sta->ht_cap.ht_supported &&
best_rssi > IWL_RS_LOW_RSSI_THRESHOLD) {
initial_rates = rs_optimal_rates_ht;
tbl = &(lq_sta->lq_info[active_tbl]);
rate = &tbl->rate;
- rs_get_initial_rate(mvm, sta, lq_sta, band, rate);
+ rs_get_initial_rate(mvm, sta, lq_sta, band, rate, init);
rs_init_optimal_rate(mvm, sta, lq_sta);
WARN_ONCE(rate->ant != ANT_A && rate->ant != ANT_B,
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
struct ieee80211_rx_status *stats = IEEE80211_SKB_RXCB(skb);
struct iwl_mvm_key_pn *ptk_pn;
+ int res;
u8 tid, keyidx;
u8 pn[IEEE80211_CCMP_PN_LEN];
u8 *extiv;
pn[4] = extiv[1];
pn[5] = extiv[0];
- if (memcmp(pn, ptk_pn->q[queue].pn[tid],
- IEEE80211_CCMP_PN_LEN) <= 0)
+ res = memcmp(pn, ptk_pn->q[queue].pn[tid], IEEE80211_CCMP_PN_LEN);
+ if (res < 0)
+ return -1;
+ if (!res && !(stats->flag & RX_FLAG_ALLOW_SAME_PN))
return -1;
- if (!(stats->flag & RX_FLAG_AMSDU_MORE))
- memcpy(ptk_pn->q[queue].pn[tid], pn, IEEE80211_CCMP_PN_LEN);
+ memcpy(ptk_pn->q[queue].pn[tid], pn, IEEE80211_CCMP_PN_LEN);
stats->flag |= RX_FLAG_PN_VALIDATED;
return 0;
}
/*
- * returns true if a packet outside BA session is a duplicate and
- * should be dropped
+ * returns true if a packet is a duplicate and should be dropped.
+ * Updates AMSDU PN tracking info
*/
-static bool iwl_mvm_is_nonagg_dup(struct ieee80211_sta *sta, int queue,
- struct ieee80211_rx_status *rx_status,
- struct ieee80211_hdr *hdr,
- struct iwl_rx_mpdu_desc *desc)
+static bool iwl_mvm_is_dup(struct ieee80211_sta *sta, int queue,
+ struct ieee80211_rx_status *rx_status,
+ struct ieee80211_hdr *hdr,
+ struct iwl_rx_mpdu_desc *desc)
{
struct iwl_mvm_sta *mvm_sta;
struct iwl_mvm_rxq_dup_data *dup_data;
- u8 baid, tid, sub_frame_idx;
+ u8 tid, sub_frame_idx;
if (WARN_ON(IS_ERR_OR_NULL(sta)))
return false;
- baid = (le32_to_cpu(desc->reorder_data) &
- IWL_RX_MPDU_REORDER_BAID_MASK) >>
- IWL_RX_MPDU_REORDER_BAID_SHIFT;
-
- if (baid != IWL_RX_REORDER_DATA_INVALID_BAID)
- return false;
-
mvm_sta = iwl_mvm_sta_from_mac80211(sta);
dup_data = &mvm_sta->dup_data[queue];
dup_data->last_sub_frame[tid] >= sub_frame_idx))
return true;
+ /* Allow same PN as the first subframe for following sub frames */
+ if (dup_data->last_seq[tid] == hdr->seq_ctrl &&
+ sub_frame_idx > dup_data->last_sub_frame[tid] &&
+ desc->mac_flags2 & IWL_RX_MPDU_MFLG2_AMSDU)
+ rx_status->flag |= RX_FLAG_ALLOW_SAME_PN;
+
dup_data->last_seq[tid] = hdr->seq_ctrl;
dup_data->last_sub_frame[tid] = sub_frame_idx;
if (ieee80211_is_data(hdr->frame_control))
iwl_mvm_rx_csum(sta, skb, desc);
- if (iwl_mvm_is_nonagg_dup(sta, queue, rx_status, hdr, desc)) {
+ if (iwl_mvm_is_dup(sta, queue, rx_status, hdr, desc)) {
kfree_skb(skb);
goto out;
}
struct iwl_trans_txq_scd_cfg cfg = {
.fifo = IWL_MVM_TX_FIFO_MCAST,
.sta_id = msta->sta_id,
- .tid = IWL_MAX_TID_COUNT,
+ .tid = 0,
.aggregate = false,
.frame_limit = IWL_FRAME_LIMIT,
};
vif->type != NL80211_IFTYPE_ADHOC))
return -ENOTSUPP;
+ /*
+ * In IBSS, ieee80211_check_queues() sets the cab_queue to be
+ * invalid, so make sure we use the queue we want.
+ * Note that this is done here as we want to avoid making DQA
+ * changes in mac80211 layer.
+ */
+ if (vif->type == NL80211_IFTYPE_ADHOC) {
+ vif->cab_queue = IWL_MVM_DQA_GCAST_QUEUE;
+ mvmvif->cab_queue = vif->cab_queue;
+ }
+
/*
* While in previous FWs we had to exclude cab queue from TFD queue
* mask, now it is needed as any other queue.
if (iwl_mvm_has_new_tx_api(mvm)) {
int queue = iwl_mvm_tvqm_enable_txq(mvm, vif->cab_queue,
msta->sta_id,
- IWL_MAX_TID_COUNT,
+ 0,
timeout);
mvmvif->cab_queue = queue;
} else if (!fw_has_api(&mvm->fw->ucode_capa,
- IWL_UCODE_TLV_API_STA_TYPE)) {
- /*
- * In IBSS, ieee80211_check_queues() sets the cab_queue to be
- * invalid, so make sure we use the queue we want.
- * Note that this is done here as we want to avoid making DQA
- * changes in mac80211 layer.
- */
- if (vif->type == NL80211_IFTYPE_ADHOC) {
- vif->cab_queue = IWL_MVM_DQA_GCAST_QUEUE;
- mvmvif->cab_queue = vif->cab_queue;
- }
+ IWL_UCODE_TLV_API_STA_TYPE))
iwl_mvm_enable_txq(mvm, vif->cab_queue, vif->cab_queue, 0,
&cfg, timeout);
- }
return 0;
}
iwl_mvm_flush_sta(mvm, &mvmvif->mcast_sta, true, 0);
iwl_mvm_disable_txq(mvm, mvmvif->cab_queue, vif->cab_queue,
- IWL_MAX_TID_COUNT, 0);
+ 0, 0);
ret = iwl_mvm_rm_sta_common(mvm, mvmvif->mcast_sta.sta_id);
if (ret)
int ret, size;
u32 status;
+ /* This is a valid situation for GTK removal */
if (sta_id == IWL_MVM_INVALID_STA)
- return -EINVAL;
+ return 0;
key_flags = cpu_to_le16((keyconf->keyidx << STA_KEY_FLG_KEYID_POS) &
STA_KEY_FLG_KEYID_MSK);
time_cmd.repeat = 1;
time_cmd.policy = cpu_to_le16(TE_V2_NOTIF_HOST_EVENT_START |
TE_V2_NOTIF_HOST_EVENT_END |
- T2_V2_START_IMMEDIATELY);
+ TE_V2_START_IMMEDIATELY);
if (!wait_for_notif) {
iwl_mvm_time_event_send_add(mvm, vif, te_data, &time_cmd);
time_cmd.repeat = 1;
time_cmd.policy = cpu_to_le16(TE_V2_NOTIF_HOST_EVENT_START |
TE_V2_NOTIF_HOST_EVENT_END |
- T2_V2_START_IMMEDIATELY);
+ TE_V2_START_IMMEDIATELY);
return iwl_mvm_time_event_send_add(mvm, vif, te_data, &time_cmd);
}
time_cmd.interval = cpu_to_le32(1);
time_cmd.policy = cpu_to_le16(TE_V2_NOTIF_HOST_EVENT_START |
TE_V2_ABSENCE);
+ if (!apply_time)
+ time_cmd.policy |= cpu_to_le16(TE_V2_START_IMMEDIATELY);
return iwl_mvm_time_event_send_add(mvm, vif, te_data, &time_cmd);
}
{
struct ieee80211_key_conf *keyconf = info->control.hw_key;
u8 *crypto_hdr = skb_frag->data + hdrlen;
+ enum iwl_tx_cmd_sec_ctrl type = TX_CMD_SEC_CCM;
u64 pn;
switch (keyconf->cipher) {
case WLAN_CIPHER_SUITE_CCMP:
- case WLAN_CIPHER_SUITE_CCMP_256:
iwl_mvm_set_tx_cmd_ccmp(info, tx_cmd);
iwl_mvm_set_tx_cmd_pn(info, crypto_hdr);
break;
break;
case WLAN_CIPHER_SUITE_GCMP:
case WLAN_CIPHER_SUITE_GCMP_256:
+ type = TX_CMD_SEC_GCMP;
+ /* Fall through */
+ case WLAN_CIPHER_SUITE_CCMP_256:
/* TODO: Taking the key from the table might introduce a race
* when PTK rekeying is done, having an old packets with a PN
* based on the old key but the message encrypted with a new
* one.
* Need to handle this.
*/
- tx_cmd->sec_ctl |= TX_CMD_SEC_GCMP | TX_CMD_SEC_KEY_FROM_TABLE;
+ tx_cmd->sec_ctl |= type | TX_CMD_SEC_KEY_FROM_TABLE;
tx_cmd->key[0] = keyconf->hw_key_idx;
iwl_mvm_set_tx_cmd_pn(info, crypto_hdr);
break;
if (info.control.vif->type == NL80211_IFTYPE_P2P_DEVICE ||
info.control.vif->type == NL80211_IFTYPE_AP ||
info.control.vif->type == NL80211_IFTYPE_ADHOC) {
- sta_id = mvmvif->bcast_sta.sta_id;
+ if (info.control.vif->type == NL80211_IFTYPE_P2P_DEVICE)
+ sta_id = mvmvif->bcast_sta.sta_id;
+ else
+ sta_id = mvmvif->mcast_sta.sta_id;
+
queue = iwl_mvm_get_ctrl_vif_queue(mvm, &info,
hdr->frame_control);
if (queue < 0)
/* Sanity check on number of chunks */
num_tbs = iwl_pcie_gen2_get_num_tbs(trans, tfd);
- if (num_tbs >= trans_pcie->max_tbs) {
+ if (num_tbs > trans_pcie->max_tbs) {
IWL_ERR(trans, "Too many chunks: %i\n", num_tbs);
return;
}
/* Sanity check on number of chunks */
num_tbs = iwl_pcie_tfd_get_num_tbs(trans, tfd);
- if (num_tbs >= trans_pcie->max_tbs) {
+ if (num_tbs > trans_pcie->max_tbs) {
IWL_ERR(trans, "Too many chunks: %i\n", num_tbs);
/* @todo issue fatal error, it is quite serious situation */
return;
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_INTERSIL
mutex_init(&data->mutex);
data->netgroup = hwsim_net_get_netgroup(net);
+ data->wmediumd = hwsim_net_get_wmediumd(net);
/* Enable frame retransmissions for lossy channels */
hw->max_rates = 4;
.exit = hwsim_exit_net,
.id = &hwsim_net_id,
.size = sizeof(struct hwsim_net),
- .async = true,
};
static void hwsim_exit_netlink(void)
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_MARVELL
le16_to_cpu(ht_cap->header.len));
mwifiex_fill_cap_info(priv, radio_type, &ht_cap->ht_cap);
+ /* Update HT40 capability from current channel information */
+ if (bss_desc->bcn_ht_oper) {
+ u8 ht_param = bss_desc->bcn_ht_oper->ht_param;
+ u8 radio =
+ mwifiex_band_to_radio_type(bss_desc->bss_band);
+ int freq =
+ ieee80211_channel_to_frequency(bss_desc->channel,
+ radio);
+ struct ieee80211_channel *chan =
+ ieee80211_get_channel(priv->adapter->wiphy, freq);
+
+ switch (ht_param & IEEE80211_HT_PARAM_CHA_SEC_OFFSET) {
+ case IEEE80211_HT_PARAM_CHA_SEC_ABOVE:
+ if (chan->flags & IEEE80211_CHAN_NO_HT40PLUS) {
+ ht_cap->ht_cap.cap_info &=
+ ~IEEE80211_HT_CAP_SUP_WIDTH_20_40;
+ ht_cap->ht_cap.cap_info &=
+ ~IEEE80211_HT_CAP_SGI_40;
+ }
+ break;
+ case IEEE80211_HT_PARAM_CHA_SEC_BELOW:
+ if (chan->flags & IEEE80211_CHAN_NO_HT40MINUS) {
+ ht_cap->ht_cap.cap_info &=
+ ~IEEE80211_HT_CAP_SUP_WIDTH_20_40;
+ ht_cap->ht_cap.cap_info &=
+ ~IEEE80211_HT_CAP_SGI_40;
+ }
+ break;
+ }
+ }
*buffer += sizeof(struct mwifiex_ie_types_htcap);
ret_len += sizeof(struct mwifiex_ie_types_htcap);
/* This function maps IEEE HT secondary channel type to NL80211 channel type
*/
-u8 mwifiex_sec_chan_offset_to_chan_type(u8 second_chan_offset)
+u8 mwifiex_get_chan_type(struct mwifiex_private *priv)
{
- switch (second_chan_offset) {
- case IEEE80211_HT_PARAM_CHA_SEC_NONE:
- return NL80211_CHAN_HT20;
- case IEEE80211_HT_PARAM_CHA_SEC_ABOVE:
- return NL80211_CHAN_HT40PLUS;
- case IEEE80211_HT_PARAM_CHA_SEC_BELOW:
- return NL80211_CHAN_HT40MINUS;
- default:
- return NL80211_CHAN_HT20;
+ struct mwifiex_channel_band channel_band;
+ int ret;
+
+ ret = mwifiex_get_chan_info(priv, &channel_band);
+
+ if (!ret) {
+ switch (channel_band.band_config.chan_width) {
+ case CHAN_BW_20MHZ:
+ if (IS_11N_ENABLED(priv))
+ return NL80211_CHAN_HT20;
+ else
+ return NL80211_CHAN_NO_HT;
+ case CHAN_BW_40MHZ:
+ if (channel_band.band_config.chan2_offset ==
+ SEC_CHAN_ABOVE)
+ return NL80211_CHAN_HT40PLUS;
+ else
+ return NL80211_CHAN_HT40MINUS;
+ default:
+ return NL80211_CHAN_HT20;
+ }
}
+
+ return NL80211_CHAN_HT20;
}
/*
struct mwifiex_private *priv = mwifiex_netdev_get_priv(wdev->netdev);
struct mwifiex_bssdescriptor *curr_bss;
struct ieee80211_channel *chan;
- u8 second_chan_offset;
enum nl80211_channel_type chan_type;
enum nl80211_band band;
int freq;
chan = ieee80211_get_channel(wiphy, freq);
if (priv->ht_param_present) {
- second_chan_offset = priv->assoc_resp_ht_param &
- IEEE80211_HT_PARAM_CHA_SEC_OFFSET;
- chan_type = mwifiex_sec_chan_offset_to_chan_type
- (second_chan_offset);
+ chan_type = mwifiex_get_chan_type(priv);
cfg80211_chandef_create(chandef, chan, chan_type);
} else {
cfg80211_chandef_create(chandef, chan,
adapter->fw_release_number = le32_to_cpu(hw_spec->fw_release_number);
adapter->fw_api_ver = (adapter->fw_release_number >> 16) & 0xff;
- adapter->number_of_antenna = le16_to_cpu(hw_spec->number_of_antenna);
+ adapter->number_of_antenna =
+ le16_to_cpu(hw_spec->number_of_antenna) & 0xf;
if (le32_to_cpu(hw_spec->dot_11ac_dev_cap)) {
adapter->is_hw_11ac_capable = true;
RDWR_STATUS_DONE = 2
};
+enum mwifiex_chan_width {
+ CHAN_BW_20MHZ = 0,
+ CHAN_BW_10MHZ,
+ CHAN_BW_40MHZ,
+ CHAN_BW_80MHZ,
+ CHAN_BW_8080MHZ,
+ CHAN_BW_160MHZ,
+ CHAN_BW_5MHZ,
+};
+
+enum mwifiex_chan_offset {
+ SEC_CHAN_NONE = 0,
+ SEC_CHAN_ABOVE = 1,
+ SEC_CHAN_5MHZ = 2,
+ SEC_CHAN_BELOW = 3
+};
+
#endif /* !_MWIFIEX_DECL_H_ */
#define HostCmd_CMD_TDLS_OPER 0x0122
#define HostCmd_CMD_FW_DUMP_EVENT 0x0125
#define HostCmd_CMD_SDIO_SP_RX_AGGR_CFG 0x0223
+#define HostCmd_CMD_STA_CONFIGURE 0x023f
#define HostCmd_CMD_CHAN_REGION_CFG 0x0242
#define HostCmd_CMD_PACKET_AGGR_CTRL 0x0251
__le16 tx_aggr_align;
} __packed;
+struct host_cmd_ds_sta_configure {
+ __le16 action;
+ u8 tlv_buffer[0];
+} __packed;
+
struct host_cmd_ds_command {
__le16 command;
__le16 size;
struct host_cmd_ds_gtk_rekey_params rekey;
struct host_cmd_ds_chan_region_cfg reg_cfg;
struct host_cmd_ds_pkt_aggr_ctrl pkt_aggr_ctrl;
+ struct host_cmd_ds_sta_configure sta_cfg;
} params;
} __packed;
struct net_device *dev)
{
int ret;
- u64 mac_addr;
+ u64 mac_addr, old_mac_addr;
- if (priv->bss_type != MWIFIEX_BSS_TYPE_P2P)
- goto done;
+ if (priv->bss_type == MWIFIEX_BSS_TYPE_ANY)
+ return -ENOTSUPP;
mac_addr = ether_addr_to_u64(priv->curr_addr);
- mac_addr |= BIT_ULL(MWIFIEX_MAC_LOCAL_ADMIN_BIT);
+ old_mac_addr = mac_addr;
+
+ if (priv->bss_type == MWIFIEX_BSS_TYPE_P2P)
+ mac_addr |= BIT_ULL(MWIFIEX_MAC_LOCAL_ADMIN_BIT);
+
+ if (mwifiex_get_intf_num(priv->adapter, priv->bss_type) > 1) {
+ /* Set mac address based on bss_type/bss_num */
+ mac_addr ^= BIT_ULL(priv->bss_type + 8);
+ mac_addr += priv->bss_num;
+ }
+
+ if (mac_addr == old_mac_addr)
+ goto done;
+
u64_to_ether_addr(mac_addr, priv->curr_addr);
/* Send request to firmware */
HostCmd_ACT_GEN_SET, 0, NULL, true);
if (ret) {
+ u64_to_ether_addr(old_mac_addr, priv->curr_addr);
mwifiex_dbg(priv->adapter, ERROR,
"set mac address failed: ret=%d\n", ret);
return ret;
}
done:
- memcpy(dev->dev_addr, priv->curr_addr, ETH_ALEN);
+ ether_addr_copy(dev->dev_addr, priv->curr_addr);
return 0;
}
MWIFIEX_IFACE_WORK_CARD_RESET,
};
+struct mwifiex_band_config {
+ u8 chan_band:2;
+ u8 chan_width:2;
+ u8 chan2_offset:2;
+ u8 scan_mode:2;
+} __packed;
+
+struct mwifiex_channel_band {
+ struct mwifiex_band_config band_config;
+ u8 channel;
+};
+
struct mwifiex_private {
struct mwifiex_adapter *adapter;
u8 bss_type;
return pos;
}
+/* This function return interface number with the same bss_type.
+ */
+static inline u8
+mwifiex_get_intf_num(struct mwifiex_adapter *adapter, u8 bss_type)
+{
+ u8 i, num = 0;
+
+ for (i = 0; i < adapter->priv_num; i++)
+ if (adapter->priv[i] && adapter->priv[i]->bss_type == bss_type)
+ num++;
+ return num;
+}
+
/*
* This function returns the correct private structure pointer based
* upon the BSS type and BSS number.
struct mwifiex_bssdescriptor *bss_desc);
u8 mwifiex_chan_type_to_sec_chan_offset(enum nl80211_channel_type chan_type);
-u8 mwifiex_sec_chan_offset_to_chan_type(u8 second_chan_offset);
+u8 mwifiex_get_chan_type(struct mwifiex_private *priv);
struct wireless_dev *mwifiex_add_virtual_intf(struct wiphy *wiphy,
const char *name,
int mwifiex_get_wakeup_reason(struct mwifiex_private *priv, u16 action,
int cmd_type,
struct mwifiex_ds_wakeup_reason *wakeup_reason);
+int mwifiex_get_chan_info(struct mwifiex_private *priv,
+ struct mwifiex_channel_band *channel_band);
int mwifiex_ret_wakeup_reason(struct mwifiex_private *priv,
struct host_cmd_ds_command *resp,
struct host_cmd_ds_wakeup_reason *wakeup_reason);
return 0;
}
+static int mwifiex_cmd_get_chan_info(struct host_cmd_ds_command *cmd,
+ u16 cmd_action)
+{
+ struct host_cmd_ds_sta_configure *sta_cfg_cmd = &cmd->params.sta_cfg;
+ struct host_cmd_tlv_channel_band *tlv_band_channel =
+ (struct host_cmd_tlv_channel_band *)sta_cfg_cmd->tlv_buffer;
+
+ cmd->command = cpu_to_le16(HostCmd_CMD_STA_CONFIGURE);
+ cmd->size = cpu_to_le16(sizeof(*sta_cfg_cmd) +
+ sizeof(*tlv_band_channel) + S_DS_GEN);
+ sta_cfg_cmd->action = cpu_to_le16(cmd_action);
+ memset(tlv_band_channel, 0, sizeof(*tlv_band_channel));
+ tlv_band_channel->header.type = cpu_to_le16(TLV_TYPE_CHANNELBANDLIST);
+ tlv_band_channel->header.len = cpu_to_le16(sizeof(*tlv_band_channel) -
+ sizeof(struct mwifiex_ie_types_header));
+
+ return 0;
+}
+
/* This function check if the command is supported by firmware */
static int mwifiex_is_cmd_supported(struct mwifiex_private *priv, u16 cmd_no)
{
cmd_ptr->command = cpu_to_le16(cmd_no);
cmd_ptr->size = cpu_to_le16(S_DS_GEN);
break;
+ case HostCmd_CMD_STA_CONFIGURE:
+ ret = mwifiex_cmd_get_chan_info(cmd_ptr, cmd_action);
+ break;
default:
mwifiex_dbg(priv->adapter, ERROR,
"PREP_CMD: unknown cmd- %#x\n", cmd_no);
return 0;
}
+static int mwifiex_ret_get_chan_info(struct mwifiex_private *priv,
+ struct host_cmd_ds_command *resp,
+ struct mwifiex_channel_band *channel_band)
+{
+ struct host_cmd_ds_sta_configure *sta_cfg_cmd = &resp->params.sta_cfg;
+ struct host_cmd_tlv_channel_band *tlv_band_channel;
+
+ tlv_band_channel =
+ (struct host_cmd_tlv_channel_band *)sta_cfg_cmd->tlv_buffer;
+ memcpy(&channel_band->band_config, &tlv_band_channel->band_config,
+ sizeof(struct mwifiex_band_config));
+ channel_band->channel = tlv_band_channel->channel;
+
+ return 0;
+}
+
/*
* This function handles the command responses.
*
case HostCmd_CMD_CHAN_REGION_CFG:
ret = mwifiex_ret_chan_region_cfg(priv, resp);
break;
+ case HostCmd_CMD_STA_CONFIGURE:
+ ret = mwifiex_ret_get_chan_info(priv, resp, data_buf);
+ break;
default:
mwifiex_dbg(adapter, ERROR,
"CMD_RESP: unknown cmd response %#x\n",
size_t beacon_ie_len;
struct mwifiex_bss_priv *bss_priv = (void *)bss->priv;
const struct cfg80211_bss_ies *ies;
- int ret;
rcu_read_lock();
ies = rcu_dereference(bss->ies);
if (bss_desc->cap_info_bitmap & WLAN_CAPABILITY_SPECTRUM_MGMT)
bss_desc->sensed_11h = true;
- ret = mwifiex_update_bss_desc_with_ie(priv->adapter, bss_desc);
- if (ret)
- return ret;
-
- /* Update HT40 capability based on current channel information */
- if (bss_desc->bcn_ht_oper && bss_desc->bcn_ht_cap) {
- u8 ht_param = bss_desc->bcn_ht_oper->ht_param;
- u8 radio = mwifiex_band_to_radio_type(bss_desc->bss_band);
- struct ieee80211_supported_band *sband =
- priv->wdev.wiphy->bands[radio];
- int freq = ieee80211_channel_to_frequency(bss_desc->channel,
- radio);
- struct ieee80211_channel *chan =
- ieee80211_get_channel(priv->adapter->wiphy, freq);
-
- switch (ht_param & IEEE80211_HT_PARAM_CHA_SEC_OFFSET) {
- case IEEE80211_HT_PARAM_CHA_SEC_ABOVE:
- if (chan->flags & IEEE80211_CHAN_NO_HT40PLUS) {
- sband->ht_cap.cap &=
- ~IEEE80211_HT_CAP_SUP_WIDTH_20_40;
- sband->ht_cap.cap &= ~IEEE80211_HT_CAP_SGI_40;
- } else {
- sband->ht_cap.cap |=
- IEEE80211_HT_CAP_SUP_WIDTH_20_40 |
- IEEE80211_HT_CAP_SGI_40;
- }
- break;
- case IEEE80211_HT_PARAM_CHA_SEC_BELOW:
- if (chan->flags & IEEE80211_CHAN_NO_HT40MINUS) {
- sband->ht_cap.cap &=
- ~IEEE80211_HT_CAP_SUP_WIDTH_20_40;
- sband->ht_cap.cap &= ~IEEE80211_HT_CAP_SGI_40;
- } else {
- sband->ht_cap.cap |=
- IEEE80211_HT_CAP_SUP_WIDTH_20_40 |
- IEEE80211_HT_CAP_SGI_40;
- }
- break;
- }
- }
-
- return 0;
+ return mwifiex_update_bss_desc_with_ie(priv->adapter, bss_desc);
}
void mwifiex_dnld_txpwr_table(struct mwifiex_private *priv)
return status;
}
+
+int mwifiex_get_chan_info(struct mwifiex_private *priv,
+ struct mwifiex_channel_band *channel_band)
+{
+ int status = 0;
+
+ status = mwifiex_send_cmd(priv, HostCmd_CMD_STA_CONFIGURE,
+ HostCmd_ACT_GEN_GET, 0, channel_band,
+ MWIFIEX_SYNC_CMD);
+
+ return status;
+}
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_MEDIATEK
return devm_led_classdev_register(dev->dev, &dev->led_cdev);
}
+static void mt76_init_stream_cap(struct mt76_dev *dev,
+ struct ieee80211_supported_band *sband,
+ bool vht)
+{
+ struct ieee80211_sta_ht_cap *ht_cap = &sband->ht_cap;
+ int i, nstream = __sw_hweight8(dev->antenna_mask);
+ struct ieee80211_sta_vht_cap *vht_cap;
+ u16 mcs_map = 0;
+
+ if (nstream > 1)
+ ht_cap->cap |= IEEE80211_HT_CAP_TX_STBC;
+ else
+ ht_cap->cap &= ~IEEE80211_HT_CAP_TX_STBC;
+
+ for (i = 0; i < IEEE80211_HT_MCS_MASK_LEN; i++)
+ ht_cap->mcs.rx_mask[i] = i < nstream ? 0xff : 0;
+
+ if (!vht)
+ return;
+
+ vht_cap = &sband->vht_cap;
+ if (nstream > 1)
+ vht_cap->cap |= IEEE80211_VHT_CAP_TXSTBC;
+ else
+ vht_cap->cap &= ~IEEE80211_VHT_CAP_TXSTBC;
+
+ for (i = 0; i < 8; i++) {
+ if (i < nstream)
+ mcs_map |= (IEEE80211_VHT_MCS_SUPPORT_0_9 << (i * 2));
+ else
+ mcs_map |=
+ (IEEE80211_VHT_MCS_NOT_SUPPORTED << (i * 2));
+ }
+ vht_cap->vht_mcs.rx_mcs_map = cpu_to_le16(mcs_map);
+ vht_cap->vht_mcs.tx_mcs_map = cpu_to_le16(mcs_map);
+}
+
+void mt76_set_stream_caps(struct mt76_dev *dev, bool vht)
+{
+ if (dev->cap.has_2ghz)
+ mt76_init_stream_cap(dev, &dev->sband_2g.sband, false);
+ if (dev->cap.has_5ghz)
+ mt76_init_stream_cap(dev, &dev->sband_5g.sband, vht);
+}
+EXPORT_SYMBOL_GPL(mt76_set_stream_caps);
+
static int
mt76_init_sband(struct mt76_dev *dev, struct mt76_sband *msband,
const struct ieee80211_channel *chan, int n_chan,
struct ieee80211_sta_ht_cap *ht_cap;
struct ieee80211_sta_vht_cap *vht_cap;
void *chanlist;
- u16 mcs_map;
int size;
size = n_chan * sizeof(*chan);
IEEE80211_HT_CAP_GRN_FLD |
IEEE80211_HT_CAP_SGI_20 |
IEEE80211_HT_CAP_SGI_40 |
- IEEE80211_HT_CAP_TX_STBC |
(1 << IEEE80211_HT_CAP_RX_STBC_SHIFT);
- ht_cap->mcs.rx_mask[0] = 0xff;
- ht_cap->mcs.rx_mask[1] = 0xff;
ht_cap->mcs.tx_params = IEEE80211_HT_MCS_TX_DEFINED;
ht_cap->ampdu_factor = IEEE80211_HT_MAX_AMPDU_64K;
ht_cap->ampdu_density = IEEE80211_HT_MPDU_DENSITY_4;
+ mt76_init_stream_cap(dev, sband, vht);
+
if (!vht)
return 0;
vht_cap = &sband->vht_cap;
vht_cap->vht_supported = true;
-
- mcs_map = (IEEE80211_VHT_MCS_SUPPORT_0_9 << (0 * 2)) |
- (IEEE80211_VHT_MCS_SUPPORT_0_9 << (1 * 2)) |
- (IEEE80211_VHT_MCS_NOT_SUPPORTED << (2 * 2)) |
- (IEEE80211_VHT_MCS_NOT_SUPPORTED << (3 * 2)) |
- (IEEE80211_VHT_MCS_NOT_SUPPORTED << (4 * 2)) |
- (IEEE80211_VHT_MCS_NOT_SUPPORTED << (5 * 2)) |
- (IEEE80211_VHT_MCS_NOT_SUPPORTED << (6 * 2)) |
- (IEEE80211_VHT_MCS_NOT_SUPPORTED << (7 * 2));
-
- vht_cap->vht_mcs.rx_mcs_map = cpu_to_le16(mcs_map);
- vht_cap->vht_mcs.tx_mcs_map = cpu_to_le16(mcs_map);
vht_cap->cap |= IEEE80211_VHT_CAP_RXLDPC |
- IEEE80211_VHT_CAP_TXSTBC |
IEEE80211_VHT_CAP_RXSTBC_1 |
IEEE80211_VHT_CAP_SHORT_GI_80;
wiphy->features |= NL80211_FEATURE_ACTIVE_MONITOR;
+ wiphy->available_antennas_tx = dev->antenna_mask;
+ wiphy->available_antennas_rx = dev->antenna_mask;
+
hw->txq_data_size = sizeof(struct mt76_txq);
hw->max_tx_fragments = 16;
u32 rev;
unsigned long state;
+ u8 antenna_mask;
+
struct mt76_sband sband_2g;
struct mt76_sband sband_5g;
struct debugfs_blob_wrapper eeprom;
void mt76_set_channel(struct mt76_dev *dev);
int mt76_get_survey(struct ieee80211_hw *hw, int idx,
struct survey_info *survey);
+void mt76_set_stream_caps(struct mt76_dev *dev, bool vht);
int mt76_rx_aggr_start(struct mt76_dev *dev, struct mt76_wcid *wcid, u8 tid,
u16 ssn, u8 size);
int mt76x2_apply_calibration_data(struct mt76x2_dev *dev, int channel);
void mt76x2_set_tx_ackto(struct mt76x2_dev *dev);
+void mt76x2_phy_set_antenna(struct mt76x2_dev *dev);
int mt76x2_phy_start(struct mt76x2_dev *dev);
int mt76x2_phy_set_channel(struct mt76x2_dev *dev,
struct cfg80211_chan_def *chandef);
mt76x2_eeprom_load(struct mt76x2_dev *dev)
{
void *efuse;
- int len = MT7662_EEPROM_SIZE;
bool found;
int ret;
- ret = mt76_eeprom_init(&dev->mt76, len);
+ ret = mt76_eeprom_init(&dev->mt76, MT7662_EEPROM_SIZE);
if (ret < 0)
return ret;
if (found)
found = !mt76x2_check_eeprom(dev);
- dev->mt76.otp.data = devm_kzalloc(dev->mt76.dev, len, GFP_KERNEL);
- dev->mt76.otp.size = len;
+ dev->mt76.otp.data = devm_kzalloc(dev->mt76.dev, MT7662_EEPROM_SIZE,
+ GFP_KERNEL);
+ dev->mt76.otp.size = MT7662_EEPROM_SIZE;
if (!dev->mt76.otp.data)
return -ENOMEM;
efuse = dev->mt76.otp.data;
- if (mt76x2_get_efuse_data(dev, efuse, len))
+ if (mt76x2_get_efuse_data(dev, efuse, MT7662_EEPROM_SIZE))
goto out;
if (found) {
} else {
/* FIXME: check if efuse data is complete */
found = true;
- memcpy(dev->mt76.eeprom.data, efuse, len);
+ memcpy(dev->mt76.eeprom.data, efuse, MT7662_EEPROM_SIZE);
}
out:
dev->mt76.led_cdev.brightness_set = mt76x2_led_set_brightness;
dev->mt76.led_cdev.blink_set = mt76x2_led_set_blink;
+ /* init antenna configuration */
+ dev->mt76.antenna_mask = 3;
+
ret = mt76_register_device(&dev->mt76, true, mt76x2_rates,
ARRAY_SIZE(mt76x2_rates));
if (ret)
ccmp_pn[5] = pn >> 24;
ccmp_pn[6] = pn >> 32;
ccmp_pn[7] = pn >> 40;
- txwi->iv = *((u32 *) &ccmp_pn[0]);
- txwi->eiv = *((u32 *) &ccmp_pn[1]);
+ txwi->iv = *((__le32 *)&ccmp_pn[0]);
+ txwi->eiv = *((__le32 *)&ccmp_pn[1]);
}
spin_lock_bh(&dev->mt76.lock);
return 0;
}
+static int mt76x2_set_antenna(struct ieee80211_hw *hw, u32 tx_ant,
+ u32 rx_ant)
+{
+ struct mt76x2_dev *dev = hw->priv;
+
+ if (!tx_ant || tx_ant > 3 || tx_ant != rx_ant)
+ return -EINVAL;
+
+ mutex_lock(&dev->mutex);
+
+ dev->chainmask = (tx_ant == 3) ? 0x202 : 0x101;
+ dev->mt76.antenna_mask = tx_ant;
+
+ mt76_set_stream_caps(&dev->mt76, true);
+ mt76x2_phy_set_antenna(dev);
+
+ mutex_unlock(&dev->mutex);
+
+ return 0;
+}
+
+static int mt76x2_get_antenna(struct ieee80211_hw *hw, u32 *tx_ant,
+ u32 *rx_ant)
+{
+ struct mt76x2_dev *dev = hw->priv;
+
+ mutex_lock(&dev->mutex);
+ *tx_ant = dev->mt76.antenna_mask;
+ *rx_ant = dev->mt76.antenna_mask;
+ mutex_unlock(&dev->mutex);
+
+ return 0;
+}
+
const struct ieee80211_ops mt76x2_ops = {
.tx = mt76x2_tx,
.start = mt76x2_start,
.set_coverage_class = mt76x2_set_coverage_class,
.get_survey = mt76_get_survey,
.set_tim = mt76x2_set_tim,
+ .set_antenna = mt76x2_set_antenna,
+ .get_antenna = mt76x2_get_antenna,
};
primary_upper);
}
-static void
-mt76x2_set_rx_chains(struct mt76x2_dev *dev)
+void mt76x2_phy_set_antenna(struct mt76x2_dev *dev)
{
u32 val;
val = mt76_rr(dev, MT_BBP(AGC, 0));
- val &= ~(BIT(3) | BIT(4));
+ val &= ~(BIT(4) | BIT(1));
+ switch (dev->mt76.antenna_mask) {
+ case 1:
+ /* disable mac DAC control */
+ mt76_clear(dev, MT_BBP(IBI, 9), BIT(11));
+ mt76_clear(dev, MT_BBP(TXBE, 5), 3);
+ mt76_rmw_field(dev, MT_TX_PIN_CFG, MT_TX_PIN_CFG_TXANT, 0x3);
+ mt76_rmw_field(dev, MT_BBP(CORE, 32), GENMASK(21, 20), 2);
+ /* disable DAC 1 */
+ mt76_rmw_field(dev, MT_BBP(CORE, 33), GENMASK(12, 9), 4);
- if (dev->chainmask & BIT(1))
- val |= BIT(3);
+ val &= ~(BIT(3) | BIT(0));
+ break;
+ case 2:
+ /* disable mac DAC control */
+ mt76_clear(dev, MT_BBP(IBI, 9), BIT(11));
+ mt76_rmw_field(dev, MT_BBP(TXBE, 5), 3, 1);
+ mt76_rmw_field(dev, MT_TX_PIN_CFG, MT_TX_PIN_CFG_TXANT, 0xc);
+ mt76_rmw_field(dev, MT_BBP(CORE, 32), GENMASK(21, 20), 1);
+ /* disable DAC 0 */
+ mt76_rmw_field(dev, MT_BBP(CORE, 33), GENMASK(12, 9), 1);
+
+ val &= ~BIT(3);
+ val |= BIT(0);
+ break;
+ case 3:
+ default:
+ /* enable mac DAC control */
+ mt76_set(dev, MT_BBP(IBI, 9), BIT(11));
+ mt76_set(dev, MT_BBP(TXBE, 5), 3);
+ mt76_rmw_field(dev, MT_TX_PIN_CFG, MT_TX_PIN_CFG_TXANT, 0xf);
+ mt76_clear(dev, MT_BBP(CORE, 32), GENMASK(21, 20));
+ mt76_clear(dev, MT_BBP(CORE, 33), GENMASK(12, 9));
+ val &= ~BIT(0);
+ val |= BIT(3);
+ break;
+ }
mt76_wr(dev, MT_BBP(AGC, 0), val);
}
-static void
-mt76x2_set_tx_dac(struct mt76x2_dev *dev)
-{
- if (dev->chainmask & BIT(1))
- mt76_set(dev, MT_BBP(TXBE, 5), 3);
- else
- mt76_clear(dev, MT_BBP(TXBE, 5), 3);
-}
-
static void
mt76x2_get_agc_gain(struct mt76x2_dev *dev, u8 *dest)
{
mt76x2_configure_tx_delay(dev, band, bw);
mt76x2_phy_set_txpower(dev);
- mt76x2_set_rx_chains(dev);
mt76x2_phy_set_band(dev, chan->band, ch_group_index & 1);
mt76x2_phy_set_bw(dev, chandef->width, ch_group_index);
- mt76x2_set_tx_dac(dev);
mt76_rmw(dev, MT_EXT_CCA_CFG,
(MT_EXT_CCA_CFG_CCA0 |
mt76x2_mcu_init_gain(dev, channel, dev->cal.rx.mcu_gain, true);
+ mt76x2_phy_set_antenna(dev);
+
/* Enable LDPC Rx */
if (mt76xx_rev(dev) >= MT76XX_REV_E3)
mt76_set(dev, MT_BBP(RXO, 13), BIT(10));
#define MT_TX_PWR_CFG_2 0x131c
#define MT_TX_PWR_CFG_3 0x1320
#define MT_TX_PWR_CFG_4 0x1324
+#define MT_TX_PIN_CFG 0x1328
+#define MT_TX_PIN_CFG_TXANT GENMASK(3, 0)
#define MT_TX_BAND_CFG 0x132c
#define MT_TX_BAND_CFG_UPPER_40M BIT(0)
#include <asm/unaligned.h>
#include "mt7601u.h"
#include "eeprom.h"
+#include "mac.h"
static bool
field_valid(u8 val)
mt7601u_efuse_physical_size_check(struct mt7601u_dev *dev)
{
const int map_reads = DIV_ROUND_UP(MT_EFUSE_USAGE_MAP_SIZE, 16);
- u8 data[map_reads * 16];
+ u8 data[round_up(MT_EFUSE_USAGE_MAP_SIZE, 16)];
int ret, i;
u32 start = 0, end = 0, cnt_free;
"Error: device has more than 1 RX/TX stream!\n");
}
-static int
-mt7601u_set_macaddr(struct mt7601u_dev *dev, const u8 *eeprom)
-{
- const void *src = eeprom + MT_EE_MAC_ADDR;
-
- ether_addr_copy(dev->macaddr, src);
-
- if (!is_valid_ether_addr(dev->macaddr)) {
- eth_random_addr(dev->macaddr);
- dev_info(dev->dev,
- "Invalid MAC address, using random address %pM\n",
- dev->macaddr);
- }
-
- mt76_wr(dev, MT_MAC_ADDR_DW0, get_unaligned_le32(dev->macaddr));
- mt76_wr(dev, MT_MAC_ADDR_DW1, get_unaligned_le16(dev->macaddr + 4) |
- FIELD_PREP(MT_MAC_ADDR_DW1_U2ME_MASK, 0xff));
-
- return 0;
-}
-
static void mt7601u_set_channel_target_power(struct mt7601u_dev *dev,
u8 *eeprom, u8 max_pwr)
{
dev_info(dev->dev, "EEPROM ver:%02hhx fae:%02hhx\n",
eeprom[MT_EE_VERSION_EE], eeprom[MT_EE_VERSION_FAE]);
- mt7601u_set_macaddr(dev, eeprom);
+ mt7601u_set_macaddr(dev, eeprom + MT_EE_MAC_ADDR);
mt7601u_set_chip_cap(dev, eeprom);
mt7601u_set_channel_power(dev, eeprom);
mt7601u_set_country_reg(dev, eeprom);
{ MT_TXOP_HLDR_ET, 0x00000002 },
{ MT_XIFS_TIME_CFG, 0x33a41010 },
{ MT_PWR_PIN_CFG, 0x00000000 },
+ { MT_PN_PAD_MODE, 0x00000001 },
};
static const struct mt76_reg_pair mac_chip_vals[] = {
#include "trace.h"
#include <linux/etherdevice.h>
+void mt7601u_set_macaddr(struct mt7601u_dev *dev, const u8 *addr)
+{
+ ether_addr_copy(dev->macaddr, addr);
+
+ if (!is_valid_ether_addr(dev->macaddr)) {
+ eth_random_addr(dev->macaddr);
+ dev_info(dev->dev,
+ "Invalid MAC address, using random address %pM\n",
+ dev->macaddr);
+ }
+
+ mt76_wr(dev, MT_MAC_ADDR_DW0, get_unaligned_le32(dev->macaddr));
+ mt76_wr(dev, MT_MAC_ADDR_DW1, get_unaligned_le16(dev->macaddr + 4) |
+ FIELD_PREP(MT_MAC_ADDR_DW1_U2ME_MASK, 0xff));
+}
+
static void
mt76_mac_process_tx_rate(struct ieee80211_tx_rate *txrate, u16 rate)
{
if (rxwi->rxinfo & cpu_to_le32(MT_RXINFO_DECRYPT)) {
status->flag |= RX_FLAG_DECRYPTED;
- status->flag |= RX_FLAG_IV_STRIPPED | RX_FLAG_MMIC_STRIPPED;
+ status->flag |= RX_FLAG_MMIC_STRIPPED;
+ status->flag |= RX_FLAG_MIC_STRIPPED;
+ status->flag |= RX_FLAG_ICV_STRIPPED;
+ status->flag |= RX_FLAG_IV_STRIPPED;
}
+ /* let mac80211 take care of PN validation since apparently
+ * the hardware does not support it
+ */
+ if (rxwi->rxinfo & cpu_to_le32(MT_RXINFO_PN_LEN))
+ status->flag &= ~RX_FLAG_IV_STRIPPED;
status->chains = BIT(0);
rssi = mt7601u_phy_get_rssi(dev, rxwi, rate);
struct mt76_tx_status
mt7601u_mac_fetch_tx_status(struct mt7601u_dev *dev);
void mt76_send_tx_status(struct mt7601u_dev *dev, struct mt76_tx_status *stat);
+void mt7601u_set_macaddr(struct mt7601u_dev *dev, const u8 *addr);
#endif
*/
mvif->idx = idx;
+ if (!ether_addr_equal(dev->macaddr, vif->addr))
+ mt7601u_set_macaddr(dev, vif->addr);
+
if (dev->wcid_mask[wcid / BITS_PER_LONG] & BIT(wcid % BITS_PER_LONG))
return -ENOSPC;
dev->wcid_mask[wcid / BITS_PER_LONG] |= BIT(wcid % BITS_PER_LONG);
trace_mt_mcu_msg_send(dev, skb, csum, need_resp);
}
-static struct sk_buff *
-mt7601u_mcu_msg_alloc(struct mt7601u_dev *dev, const void *data, int len)
+static struct sk_buff *mt7601u_mcu_msg_alloc(const void *data, int len)
{
struct sk_buff *skb;
.value = cpu_to_le32(val),
};
- skb = mt7601u_mcu_msg_alloc(dev, &msg, sizeof(msg));
+ skb = mt7601u_mcu_msg_alloc(&msg, sizeof(msg));
if (!skb)
return -ENOMEM;
return mt7601u_mcu_msg_send(dev, skb, CMD_FUN_SET_OP, func == 5);
.value = cpu_to_le32(val),
};
- skb = mt7601u_mcu_msg_alloc(dev, &msg, sizeof(msg));
+ skb = mt7601u_mcu_msg_alloc(&msg, sizeof(msg));
if (!skb)
return -ENOMEM;
return mt7601u_mcu_msg_send(dev, skb, CMD_CALIBRATION_OP, true);
* @rx_lock: protects @rx_q.
* @con_mon_lock: protects @ap_bssid, @bcn_*, @avg_rssi.
* @mutex: ensures exclusive access from mac80211 callbacks.
- * @vendor_req_mutex: protects @vend_buf, ensures atomicity of split writes.
+ * @vendor_req_mutex: protects @vend_buf, ensures atomicity of read/write
+ * accesses
* @reg_atomic_mutex: ensures atomicity of indirect register accesses
* (accesses to RF and BBP).
* @hw_atomic_mutex: ensures exclusive access to HW during critical
MT_VEND_DEV_MODE_RESET, 0, NULL, 0);
}
-u32 mt7601u_rr(struct mt7601u_dev *dev, u32 offset)
+/* should be called with vendor_req_mutex held */
+static u32 __mt7601u_rr(struct mt7601u_dev *dev, u32 offset)
{
int ret;
u32 val = ~0;
WARN_ONCE(offset > USHRT_MAX, "read high off:%08x", offset);
- mutex_lock(&dev->vendor_req_mutex);
-
ret = mt7601u_vendor_request(dev, MT_VEND_MULTI_READ, USB_DIR_IN,
0, offset, dev->vend_buf, MT_VEND_BUF);
if (ret == MT_VEND_BUF)
dev_err(dev->dev, "Error: wrong size read:%d off:%08x\n",
ret, offset);
- mutex_unlock(&dev->vendor_req_mutex);
-
trace_reg_read(dev, offset, val);
return val;
}
-int mt7601u_vendor_single_wr(struct mt7601u_dev *dev, const u8 req,
- const u16 offset, const u32 val)
+u32 mt7601u_rr(struct mt7601u_dev *dev, u32 offset)
{
- int ret;
+ u32 ret;
mutex_lock(&dev->vendor_req_mutex);
+ ret = __mt7601u_rr(dev, offset);
+ mutex_unlock(&dev->vendor_req_mutex);
- ret = mt7601u_vendor_request(dev, req, USB_DIR_OUT,
- val & 0xffff, offset, NULL, 0);
+ return ret;
+}
+
+/* should be called with vendor_req_mutex held */
+static int __mt7601u_vendor_single_wr(struct mt7601u_dev *dev, const u8 req,
+ const u16 offset, const u32 val)
+{
+ int ret = mt7601u_vendor_request(dev, req, USB_DIR_OUT,
+ val & 0xffff, offset, NULL, 0);
if (!ret)
ret = mt7601u_vendor_request(dev, req, USB_DIR_OUT,
val >> 16, offset + 2, NULL, 0);
+ trace_reg_write(dev, offset, val);
+ return ret;
+}
+
+int mt7601u_vendor_single_wr(struct mt7601u_dev *dev, const u8 req,
+ const u16 offset, const u32 val)
+{
+ int ret;
+ mutex_lock(&dev->vendor_req_mutex);
+ ret = __mt7601u_vendor_single_wr(dev, req, offset, val);
mutex_unlock(&dev->vendor_req_mutex);
return ret;
WARN_ONCE(offset > USHRT_MAX, "write high off:%08x", offset);
mt7601u_vendor_single_wr(dev, MT_VEND_WRITE, offset, val);
- trace_reg_write(dev, offset, val);
}
u32 mt7601u_rmw(struct mt7601u_dev *dev, u32 offset, u32 mask, u32 val)
{
- val |= mt7601u_rr(dev, offset) & ~mask;
- mt7601u_wr(dev, offset, val);
+ mutex_lock(&dev->vendor_req_mutex);
+ val |= __mt7601u_rr(dev, offset) & ~mask;
+ __mt7601u_vendor_single_wr(dev, MT_VEND_WRITE, offset, val);
+ mutex_unlock(&dev->vendor_req_mutex);
+
return val;
}
u32 mt7601u_rmc(struct mt7601u_dev *dev, u32 offset, u32 mask, u32 val)
{
- u32 reg = mt7601u_rr(dev, offset);
+ u32 reg;
+ mutex_lock(&dev->vendor_req_mutex);
+ reg = __mt7601u_rr(dev, offset);
val |= reg & ~mask;
if (reg != val)
- mt7601u_wr(dev, offset, val);
+ __mt7601u_vendor_single_wr(dev, MT_VEND_WRITE,
+ offset, val);
+ mutex_unlock(&dev->vendor_req_mutex);
+
return val;
}
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_QUANTENNA
char fwname[32];
struct napi_struct mux_napi;
struct net_device mux_dev;
- struct completion request_firmware_complete;
+ struct completion firmware_init_complete;
struct workqueue_struct *workqueue;
+ struct work_struct fw_work;
struct work_struct event_work;
struct mutex bus_lock; /* lock during command/event processing */
struct dentry *dbg_dir;
spin_unlock_irqrestore(&priv->irq_lock, flags);
}
-static int qtnf_pcie_init_irq(struct qtnf_pcie_bus_priv *priv)
+static void qtnf_pcie_init_irq(struct qtnf_pcie_bus_priv *priv)
{
struct pci_dev *pdev = priv->pdev;
pr_warn("legacy PCIE interrupts enabled\n");
pci_intx(pdev, 1);
}
-
- return 0;
}
static void qtnf_deassert_intx(struct qtnf_pcie_bus_priv *priv)
qtnf_non_posted_write(cfg, reg);
}
+static void qtnf_reset_card(struct qtnf_pcie_bus_priv *priv)
+{
+ const u32 data = QTN_PEARL_IPC_IRQ_WORD(QTN_PEARL_LHOST_EP_RESET);
+ void __iomem *reg = priv->sysctl_bar +
+ QTN_PEARL_SYSCTL_LHOST_IRQ_OFFSET;
+
+ qtnf_non_posted_write(data, reg);
+ msleep(QTN_EP_RESET_WAIT_MS);
+ pci_restore_state(priv->pdev);
+}
+
static void qtnf_ipc_gen_ep_int(void *arg)
{
const struct qtnf_pcie_bus_priv *priv = arg;
}
/* all rx/tx activity should have ceased before calling this function */
-static void free_xfer_buffers(void *data)
+static void qtnf_free_xfer_buffers(struct qtnf_pcie_bus_priv *priv)
{
- struct qtnf_pcie_bus_priv *priv = (struct qtnf_pcie_bus_priv *)data;
+ struct qtnf_tx_bd *txbd;
struct qtnf_rx_bd *rxbd;
+ struct sk_buff *skb;
dma_addr_t paddr;
int i;
for (i = 0; i < priv->rx_bd_num; i++) {
if (priv->rx_skb && priv->rx_skb[i]) {
rxbd = &priv->rx_bd_vbase[i];
+ skb = priv->rx_skb[i];
paddr = QTN_HOST_ADDR(le32_to_cpu(rxbd->addr_h),
le32_to_cpu(rxbd->addr));
pci_unmap_single(priv->pdev, paddr, SKB_BUF_SIZE,
PCI_DMA_FROMDEVICE);
-
- dev_kfree_skb_any(priv->rx_skb[i]);
+ dev_kfree_skb_any(skb);
+ priv->rx_skb[i] = NULL;
}
}
/* free tx buffers */
for (i = 0; i < priv->tx_bd_num; i++) {
if (priv->tx_skb && priv->tx_skb[i]) {
- dev_kfree_skb_any(priv->tx_skb[i]);
+ txbd = &priv->tx_bd_vbase[i];
+ skb = priv->tx_skb[i];
+ paddr = QTN_HOST_ADDR(le32_to_cpu(txbd->addr_h),
+ le32_to_cpu(txbd->addr));
+ pci_unmap_single(priv->pdev, paddr, skb->len,
+ PCI_DMA_TODEVICE);
+ dev_kfree_skb_any(skb);
priv->tx_skb[i] = NULL;
}
}
.data_rx_stop = qtnf_pcie_data_rx_stop,
};
+static int qtnf_dbg_mps_show(struct seq_file *s, void *data)
+{
+ struct qtnf_bus *bus = dev_get_drvdata(s->private);
+ struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
+
+ seq_printf(s, "%d\n", priv->mps);
+
+ return 0;
+}
+
+static int qtnf_dbg_msi_show(struct seq_file *s, void *data)
+{
+ struct qtnf_bus *bus = dev_get_drvdata(s->private);
+ struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
+
+ seq_printf(s, "%u\n", priv->msi_enabled);
+
+ return 0;
+}
+
+static int qtnf_dbg_irq_stats(struct seq_file *s, void *data)
+{
+ struct qtnf_bus *bus = dev_get_drvdata(s->private);
+ struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
+ u32 reg = readl(PCIE_HDP_INT_EN(priv->pcie_reg_base));
+ u32 status;
+
+ seq_printf(s, "pcie_irq_count(%u)\n", priv->pcie_irq_count);
+ seq_printf(s, "pcie_irq_tx_count(%u)\n", priv->pcie_irq_tx_count);
+ status = reg & PCIE_HDP_INT_TX_BITS;
+ seq_printf(s, "pcie_irq_tx_status(%s)\n",
+ (status == PCIE_HDP_INT_TX_BITS) ? "EN" : "DIS");
+ seq_printf(s, "pcie_irq_rx_count(%u)\n", priv->pcie_irq_rx_count);
+ status = reg & PCIE_HDP_INT_RX_BITS;
+ seq_printf(s, "pcie_irq_rx_status(%s)\n",
+ (status == PCIE_HDP_INT_RX_BITS) ? "EN" : "DIS");
+ seq_printf(s, "pcie_irq_uf_count(%u)\n", priv->pcie_irq_uf_count);
+ status = reg & PCIE_HDP_INT_HHBM_UF;
+ seq_printf(s, "pcie_irq_hhbm_uf_status(%s)\n",
+ (status == PCIE_HDP_INT_HHBM_UF) ? "EN" : "DIS");
+
+ return 0;
+}
+
+static int qtnf_dbg_hdp_stats(struct seq_file *s, void *data)
+{
+ struct qtnf_bus *bus = dev_get_drvdata(s->private);
+ struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
+
+ seq_printf(s, "tx_full_count(%u)\n", priv->tx_full_count);
+ seq_printf(s, "tx_done_count(%u)\n", priv->tx_done_count);
+ seq_printf(s, "tx_reclaim_done(%u)\n", priv->tx_reclaim_done);
+ seq_printf(s, "tx_reclaim_req(%u)\n", priv->tx_reclaim_req);
+
+ seq_printf(s, "tx_bd_r_index(%u)\n", priv->tx_bd_r_index);
+ seq_printf(s, "tx_bd_p_index(%u)\n",
+ readl(PCIE_HDP_RX0DMA_CNT(priv->pcie_reg_base))
+ & (priv->tx_bd_num - 1));
+ seq_printf(s, "tx_bd_w_index(%u)\n", priv->tx_bd_w_index);
+ seq_printf(s, "tx queue len(%u)\n",
+ CIRC_CNT(priv->tx_bd_w_index, priv->tx_bd_r_index,
+ priv->tx_bd_num));
+
+ seq_printf(s, "rx_bd_r_index(%u)\n", priv->rx_bd_r_index);
+ seq_printf(s, "rx_bd_p_index(%u)\n",
+ readl(PCIE_HDP_TX0DMA_CNT(priv->pcie_reg_base))
+ & (priv->rx_bd_num - 1));
+ seq_printf(s, "rx_bd_w_index(%u)\n", priv->rx_bd_w_index);
+ seq_printf(s, "rx alloc queue len(%u)\n",
+ CIRC_SPACE(priv->rx_bd_w_index, priv->rx_bd_r_index,
+ priv->rx_bd_num));
+
+ return 0;
+}
+
+static int qtnf_dbg_shm_stats(struct seq_file *s, void *data)
+{
+ struct qtnf_bus *bus = dev_get_drvdata(s->private);
+ struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
+
+ seq_printf(s, "shm_ipc_ep_in.tx_packet_count(%zu)\n",
+ priv->shm_ipc_ep_in.tx_packet_count);
+ seq_printf(s, "shm_ipc_ep_in.rx_packet_count(%zu)\n",
+ priv->shm_ipc_ep_in.rx_packet_count);
+ seq_printf(s, "shm_ipc_ep_out.tx_packet_count(%zu)\n",
+ priv->shm_ipc_ep_out.tx_timeout_count);
+ seq_printf(s, "shm_ipc_ep_out.rx_packet_count(%zu)\n",
+ priv->shm_ipc_ep_out.rx_packet_count);
+
+ return 0;
+}
+
static int qtnf_ep_fw_send(struct qtnf_pcie_bus_priv *priv, uint32_t size,
int blk, const u8 *pblk, const u8 *fw)
{
return 0;
}
-static void qtnf_firmware_load(const struct firmware *fw, void *context)
-{
- struct qtnf_pcie_bus_priv *priv = (void *)context;
- struct pci_dev *pdev = priv->pdev;
- struct qtnf_bus *bus = pci_get_drvdata(pdev);
- int ret;
-
- if (!fw) {
- pr_err("failed to get firmware %s\n", bus->fwname);
- goto fw_load_err;
- }
-
- ret = qtnf_ep_fw_load(priv, fw->data, fw->size);
- if (ret) {
- pr_err("FW upload error\n");
- goto fw_load_err;
- }
-
- if (qtnf_poll_state(&priv->bda->bda_ep_state, QTN_EP_FW_DONE,
- QTN_FW_DL_TIMEOUT_MS)) {
- pr_err("FW bringup timed out\n");
- goto fw_load_err;
- }
-
- bus->fw_state = QTNF_FW_STATE_FW_DNLD_DONE;
- pr_info("firmware is up and running\n");
-
-fw_load_err:
-
- if (fw)
- release_firmware(fw);
-
- complete(&bus->request_firmware_complete);
-}
-
-static int qtnf_bringup_fw(struct qtnf_bus *bus)
+static void qtnf_fw_work_handler(struct work_struct *work)
{
+ struct qtnf_bus *bus = container_of(work, struct qtnf_bus, fw_work);
struct qtnf_pcie_bus_priv *priv = (void *)get_bus_priv(bus);
struct pci_dev *pdev = priv->pdev;
+ const struct firmware *fw;
int ret;
u32 state = QTN_RC_FW_LOADRDY | QTN_RC_FW_QLINK;
- if (flashboot)
+ if (flashboot) {
state |= QTN_RC_FW_FLASHBOOT;
+ } else {
+ ret = request_firmware(&fw, bus->fwname, &pdev->dev);
+ if (ret < 0) {
+ pr_err("failed to get firmware %s\n", bus->fwname);
+ goto fw_load_fail;
+ }
+ }
qtnf_set_state(&priv->bda->bda_rc_state, state);
if (qtnf_poll_state(&priv->bda->bda_ep_state, QTN_EP_FW_LOADRDY,
QTN_FW_DL_TIMEOUT_MS)) {
pr_err("card is not ready\n");
- return -ETIMEDOUT;
+ goto fw_load_fail;
}
qtnf_clear_state(&priv->bda->bda_ep_state, QTN_EP_FW_LOADRDY);
if (flashboot) {
- pr_info("Booting FW from flash\n");
-
- if (!qtnf_poll_state(&priv->bda->bda_ep_state, QTN_EP_FW_DONE,
- QTN_FW_DL_TIMEOUT_MS))
- bus->fw_state = QTNF_FW_STATE_FW_DNLD_DONE;
+ pr_info("booting firmware from flash\n");
+ } else {
+ pr_info("starting firmware upload: %s\n", bus->fwname);
- return 0;
+ ret = qtnf_ep_fw_load(priv, fw->data, fw->size);
+ release_firmware(fw);
+ if (ret) {
+ pr_err("firmware upload error\n");
+ goto fw_load_fail;
+ }
}
- pr_info("starting firmware upload: %s\n", bus->fwname);
-
- ret = request_firmware_nowait(THIS_MODULE, 1, bus->fwname, &pdev->dev,
- GFP_KERNEL, priv, qtnf_firmware_load);
- if (ret < 0)
- pr_err("request_firmware_nowait error %d\n", ret);
- else
- ret = 1;
-
- return ret;
-}
-
-static void qtnf_reclaim_tasklet_fn(unsigned long data)
-{
- struct qtnf_pcie_bus_priv *priv = (void *)data;
-
- qtnf_pcie_data_tx_reclaim(priv);
- qtnf_en_txdone_irq(priv);
-}
-
-static int qtnf_dbg_mps_show(struct seq_file *s, void *data)
-{
- struct qtnf_bus *bus = dev_get_drvdata(s->private);
- struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
+ if (qtnf_poll_state(&priv->bda->bda_ep_state, QTN_EP_FW_DONE,
+ QTN_FW_DL_TIMEOUT_MS)) {
+ pr_err("firmware bringup timed out\n");
+ goto fw_load_fail;
+ }
- seq_printf(s, "%d\n", priv->mps);
+ bus->fw_state = QTNF_FW_STATE_FW_DNLD_DONE;
+ pr_info("firmware is up and running\n");
- return 0;
-}
+ if (qtnf_poll_state(&priv->bda->bda_ep_state,
+ QTN_EP_FW_QLINK_DONE, QTN_FW_QLINK_TIMEOUT_MS)) {
+ pr_err("firmware runtime failure\n");
+ goto fw_load_fail;
+ }
-static int qtnf_dbg_msi_show(struct seq_file *s, void *data)
-{
- struct qtnf_bus *bus = dev_get_drvdata(s->private);
- struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
+ ret = qtnf_core_attach(bus);
+ if (ret) {
+ pr_err("failed to attach core\n");
+ goto fw_load_fail;
+ }
- seq_printf(s, "%u\n", priv->msi_enabled);
+ qtnf_debugfs_init(bus, DRV_NAME);
+ qtnf_debugfs_add_entry(bus, "mps", qtnf_dbg_mps_show);
+ qtnf_debugfs_add_entry(bus, "msi_enabled", qtnf_dbg_msi_show);
+ qtnf_debugfs_add_entry(bus, "hdp_stats", qtnf_dbg_hdp_stats);
+ qtnf_debugfs_add_entry(bus, "irq_stats", qtnf_dbg_irq_stats);
+ qtnf_debugfs_add_entry(bus, "shm_stats", qtnf_dbg_shm_stats);
- return 0;
-}
+ goto fw_load_exit;
-static int qtnf_dbg_irq_stats(struct seq_file *s, void *data)
-{
- struct qtnf_bus *bus = dev_get_drvdata(s->private);
- struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
- u32 reg = readl(PCIE_HDP_INT_EN(priv->pcie_reg_base));
- u32 status;
+fw_load_fail:
+ bus->fw_state = QTNF_FW_STATE_DEAD;
- seq_printf(s, "pcie_irq_count(%u)\n", priv->pcie_irq_count);
- seq_printf(s, "pcie_irq_tx_count(%u)\n", priv->pcie_irq_tx_count);
- status = reg & PCIE_HDP_INT_TX_BITS;
- seq_printf(s, "pcie_irq_tx_status(%s)\n",
- (status == PCIE_HDP_INT_TX_BITS) ? "EN" : "DIS");
- seq_printf(s, "pcie_irq_rx_count(%u)\n", priv->pcie_irq_rx_count);
- status = reg & PCIE_HDP_INT_RX_BITS;
- seq_printf(s, "pcie_irq_rx_status(%s)\n",
- (status == PCIE_HDP_INT_RX_BITS) ? "EN" : "DIS");
- seq_printf(s, "pcie_irq_uf_count(%u)\n", priv->pcie_irq_uf_count);
- status = reg & PCIE_HDP_INT_HHBM_UF;
- seq_printf(s, "pcie_irq_hhbm_uf_status(%s)\n",
- (status == PCIE_HDP_INT_HHBM_UF) ? "EN" : "DIS");
-
- return 0;
+fw_load_exit:
+ complete(&bus->firmware_init_complete);
+ put_device(&pdev->dev);
}
-static int qtnf_dbg_hdp_stats(struct seq_file *s, void *data)
+static void qtnf_bringup_fw_async(struct qtnf_bus *bus)
{
- struct qtnf_bus *bus = dev_get_drvdata(s->private);
- struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
-
- seq_printf(s, "tx_full_count(%u)\n", priv->tx_full_count);
- seq_printf(s, "tx_done_count(%u)\n", priv->tx_done_count);
- seq_printf(s, "tx_reclaim_done(%u)\n", priv->tx_reclaim_done);
- seq_printf(s, "tx_reclaim_req(%u)\n", priv->tx_reclaim_req);
-
- seq_printf(s, "tx_bd_r_index(%u)\n", priv->tx_bd_r_index);
- seq_printf(s, "tx_bd_p_index(%u)\n",
- readl(PCIE_HDP_RX0DMA_CNT(priv->pcie_reg_base))
- & (priv->tx_bd_num - 1));
- seq_printf(s, "tx_bd_w_index(%u)\n", priv->tx_bd_w_index);
- seq_printf(s, "tx queue len(%u)\n",
- CIRC_CNT(priv->tx_bd_w_index, priv->tx_bd_r_index,
- priv->tx_bd_num));
-
- seq_printf(s, "rx_bd_r_index(%u)\n", priv->rx_bd_r_index);
- seq_printf(s, "rx_bd_p_index(%u)\n",
- readl(PCIE_HDP_TX0DMA_CNT(priv->pcie_reg_base))
- & (priv->rx_bd_num - 1));
- seq_printf(s, "rx_bd_w_index(%u)\n", priv->rx_bd_w_index);
- seq_printf(s, "rx alloc queue len(%u)\n",
- CIRC_SPACE(priv->rx_bd_w_index, priv->rx_bd_r_index,
- priv->rx_bd_num));
+ struct qtnf_pcie_bus_priv *priv = (void *)get_bus_priv(bus);
+ struct pci_dev *pdev = priv->pdev;
- return 0;
+ get_device(&pdev->dev);
+ INIT_WORK(&bus->fw_work, qtnf_fw_work_handler);
+ schedule_work(&bus->fw_work);
}
-static int qtnf_dbg_shm_stats(struct seq_file *s, void *data)
+static void qtnf_reclaim_tasklet_fn(unsigned long data)
{
- struct qtnf_bus *bus = dev_get_drvdata(s->private);
- struct qtnf_pcie_bus_priv *priv = get_bus_priv(bus);
-
- seq_printf(s, "shm_ipc_ep_in.tx_packet_count(%zu)\n",
- priv->shm_ipc_ep_in.tx_packet_count);
- seq_printf(s, "shm_ipc_ep_in.rx_packet_count(%zu)\n",
- priv->shm_ipc_ep_in.rx_packet_count);
- seq_printf(s, "shm_ipc_ep_out.tx_packet_count(%zu)\n",
- priv->shm_ipc_ep_out.tx_timeout_count);
- seq_printf(s, "shm_ipc_ep_out.rx_packet_count(%zu)\n",
- priv->shm_ipc_ep_out.rx_packet_count);
+ struct qtnf_pcie_bus_priv *priv = (void *)data;
- return 0;
+ qtnf_pcie_data_tx_reclaim(priv);
+ qtnf_en_txdone_irq(priv);
}
static int qtnf_pcie_probe(struct pci_dev *pdev, const struct pci_device_id *id)
bus = devm_kzalloc(&pdev->dev,
sizeof(*bus) + sizeof(*pcie_priv), GFP_KERNEL);
- if (!bus) {
- ret = -ENOMEM;
- goto err_init;
- }
+ if (!bus)
+ return -ENOMEM;
pcie_priv = get_bus_priv(bus);
pcie_priv->pdev = pdev;
strcpy(bus->fwname, QTN_PCI_PEARL_FW_NAME);
- init_completion(&bus->request_firmware_complete);
+ init_completion(&bus->firmware_init_complete);
mutex_init(&bus->bus_lock);
spin_lock_init(&pcie_priv->tx0_lock);
spin_lock_init(&pcie_priv->irq_lock);
pcie_priv->tx_reclaim_done = 0;
pcie_priv->tx_reclaim_req = 0;
+ tasklet_init(&pcie_priv->reclaim_tq, qtnf_reclaim_tasklet_fn,
+ (unsigned long)pcie_priv);
+
+ init_dummy_netdev(&bus->mux_dev);
+ netif_napi_add(&bus->mux_dev, &bus->mux_napi,
+ qtnf_rx_poll, 10);
+
pcie_priv->workqueue = create_singlethread_workqueue("QTNF_PEARL_PCIE");
if (!pcie_priv->workqueue) {
pr_err("failed to alloc bus workqueue\n");
ret = -ENODEV;
- goto err_priv;
+ goto err_init;
}
if (!pci_is_pcie(pdev)) {
goto err_base;
}
- pcim_pin_device(pdev);
pci_set_master(pdev);
-
- ret = qtnf_pcie_init_irq(pcie_priv);
- if (ret < 0) {
- pr_err("irq init failed\n");
- goto err_base;
- }
+ qtnf_pcie_init_irq(pcie_priv);
ret = qtnf_pcie_init_memory(pcie_priv);
if (ret < 0) {
goto err_base;
}
+ pci_save_state(pdev);
+
ret = qtnf_pcie_init_shm_ipc(pcie_priv);
if (ret < 0) {
pr_err("PCIE SHM IPC init failed\n");
goto err_base;
}
- ret = devm_add_action(&pdev->dev, free_xfer_buffers, (void *)pcie_priv);
- if (ret) {
- pr_err("custom release callback init failed\n");
- goto err_base;
- }
-
ret = qtnf_pcie_init_xfer(pcie_priv);
if (ret) {
pr_err("PCIE xfer init failed\n");
- goto err_base;
+ goto err_ipc;
}
/* init default irq settings */
"qtnf_pcie_irq", (void *)bus);
if (ret) {
pr_err("failed to request pcie irq %d\n", pdev->irq);
- goto err_base;
- }
-
- tasklet_init(&pcie_priv->reclaim_tq, qtnf_reclaim_tasklet_fn,
- (unsigned long)pcie_priv);
- init_dummy_netdev(&bus->mux_dev);
- netif_napi_add(&bus->mux_dev, &bus->mux_napi,
- qtnf_rx_poll, 10);
-
- ret = qtnf_bringup_fw(bus);
- if (ret < 0)
- goto err_bringup_fw;
- else if (ret)
- wait_for_completion(&bus->request_firmware_complete);
-
- if (bus->fw_state != QTNF_FW_STATE_FW_DNLD_DONE) {
- pr_err("failed to start FW\n");
- goto err_bringup_fw;
- }
-
- if (qtnf_poll_state(&pcie_priv->bda->bda_ep_state, QTN_EP_FW_QLINK_DONE,
- QTN_FW_QLINK_TIMEOUT_MS)) {
- pr_err("FW runtime failure\n");
- goto err_bringup_fw;
+ goto err_xfer;
}
- ret = qtnf_core_attach(bus);
- if (ret) {
- pr_err("failed to attach core\n");
- goto err_bringup_fw;
- }
-
- qtnf_debugfs_init(bus, DRV_NAME);
- qtnf_debugfs_add_entry(bus, "mps", qtnf_dbg_mps_show);
- qtnf_debugfs_add_entry(bus, "msi_enabled", qtnf_dbg_msi_show);
- qtnf_debugfs_add_entry(bus, "hdp_stats", qtnf_dbg_hdp_stats);
- qtnf_debugfs_add_entry(bus, "irq_stats", qtnf_dbg_irq_stats);
- qtnf_debugfs_add_entry(bus, "shm_stats", qtnf_dbg_shm_stats);
+ qtnf_bringup_fw_async(bus);
return 0;
-err_bringup_fw:
- netif_napi_del(&bus->mux_napi);
+err_xfer:
+ qtnf_free_xfer_buffers(pcie_priv);
+
+err_ipc:
+ qtnf_pcie_free_shm_ipc(pcie_priv);
err_base:
flush_workqueue(pcie_priv->workqueue);
destroy_workqueue(pcie_priv->workqueue);
+ netif_napi_del(&bus->mux_napi);
-err_priv:
+err_init:
+ tasklet_kill(&pcie_priv->reclaim_tq);
pci_set_drvdata(pdev, NULL);
-err_init:
return ret;
}
if (!bus)
return;
+ wait_for_completion(&bus->firmware_init_complete);
+
+ if (bus->fw_state == QTNF_FW_STATE_ACTIVE)
+ qtnf_core_detach(bus);
+
priv = get_bus_priv(bus);
- qtnf_core_detach(bus);
netif_napi_del(&bus->mux_napi);
-
flush_workqueue(priv->workqueue);
destroy_workqueue(priv->workqueue);
tasklet_kill(&priv->reclaim_tq);
+ qtnf_free_xfer_buffers(priv);
qtnf_debugfs_remove(bus);
qtnf_pcie_free_shm_ipc(priv);
+ qtnf_reset_card(priv);
}
#ifdef CONFIG_PM_SLEEP
/* state transition timeouts */
#define QTN_FW_DL_TIMEOUT_MS 3000
#define QTN_FW_QLINK_TIMEOUT_MS 30000
+#define QTN_EP_RESET_WAIT_MS 1000
#define PCIE_HDP_INT_RX_BITS (0 \
| PCIE_HDP_INT_EP_TXDMA \
#define QTN_PEARL_IPC_IRQ_WORD(irq) (BIT(irq) | BIT(irq + 16))
#define QTN_PEARL_LHOST_IPC_IRQ (6)
+#define QTN_PEARL_LHOST_EP_RESET (7)
#endif /* __PEARL_PCIE_H */
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_RALINK
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_REALTEK
goto err_free_dev;
}
mutex_init(&priv->io_mutex);
+ mutex_init(&priv->conf_mutex);
SET_IEEE80211_DEV(dev, &intf->dev);
usb_set_intfdata(intf, dev);
printk(KERN_ERR "rtl8187: Cannot register device\n");
goto err_free_dmabuf;
}
- mutex_init(&priv->conf_mutex);
skb_queue_head_init(&priv->b_tx_status.queue);
wiphy_info(dev->wiphy, "hwaddr %pM, %s V%d + %s, rfkill mask %d\n",
struct rtl_priv *rtlpriv = rtl_priv(hw);
struct rtl_hal *rtlhal = rtl_hal(rtlpriv);
+ if (!(rtlpriv->cfg->spec_ver & RTL_SPEC_SUPPORT_VHT))
+ return;
+
if (rtlhal->hw_type == HARDWARE_TYPE_RTL8812AE ||
rtlhal->hw_type == HARDWARE_TYPE_RTL8822BE) {
u16 mcs_map;
ieee80211_hw_set(hw, MFP_CAPABLE);
ieee80211_hw_set(hw, REPORTS_TX_ACK_STATUS);
ieee80211_hw_set(hw, SUPPORTS_AMSDU_IN_AMPDU);
+ ieee80211_hw_set(hw, SUPPORT_FAST_XMIT);
/* swlps or hwlps has been set in diff chip in init_sw_vars */
if (rtlpriv->psc.swctrl_lps) {
tcb_desc->packet_bw = HT_CHANNEL_WIDTH_20_40;
- if (rtlpriv->rtlhal.hw_type == HARDWARE_TYPE_RTL8812AE ||
- rtlpriv->rtlhal.hw_type == HARDWARE_TYPE_RTL8821AE) {
+ if (rtlpriv->cfg->spec_ver & RTL_SPEC_SUPPORT_VHT) {
if (mac->opmode == NL80211_IFTYPE_AP ||
mac->opmode == NL80211_IFTYPE_ADHOC ||
mac->opmode == NL80211_IFTYPE_MESH_POINT) {
struct rtl_tx_report *tx_report = &rtlpriv->tx_report;
u16 sn;
- sn = atomic_inc_return(&tx_report->sn) & 0x0FFF;
+ /* SW_DEFINE[11:8] are reserved (driver fills zeros)
+ * SW_DEFINE[7:2] are used by driver
+ * SW_DEFINE[1:0] are reserved for firmware (driver fills zeros)
+ */
+ sn = (atomic_inc_return(&tx_report->sn) & 0x003F) << 2;
tx_report->last_sent_sn = sn;
tx_report->last_sent_time = jiffies;
struct rtl_priv *rtlpriv = rtl_priv(hw);
struct rtl_tx_report *tx_report = &rtlpriv->tx_report;
u16 sn;
+ u8 st, retry;
- sn = ((tmp_buf[7] & 0x0F) << 8) | tmp_buf[6];
+ if (rtlpriv->cfg->spec_ver & RTL_SPEC_EXT_C2H) {
+ sn = GET_TX_REPORT_SN_V2(tmp_buf);
+ st = GET_TX_REPORT_ST_V2(tmp_buf);
+ retry = GET_TX_REPORT_RETRY_V2(tmp_buf);
+ } else {
+ sn = GET_TX_REPORT_SN_V1(tmp_buf);
+ st = GET_TX_REPORT_ST_V1(tmp_buf);
+ retry = GET_TX_REPORT_RETRY_V1(tmp_buf);
+ }
tx_report->last_recv_sn = sn;
RT_TRACE(rtlpriv, COMP_TX_REPORT, DBG_DMESG,
"Recv TX-Report st=0x%02X sn=0x%X retry=0x%X\n",
- tmp_buf[0], sn, tmp_buf[2]);
+ st, sn, retry);
}
EXPORT_SYMBOL_GPL(rtl_tx_report_handler);
if (time_before(tx_report->last_sent_time + 3 * HZ, jiffies)) {
RT_TRACE(rtlpriv, COMP_TX_REPORT, DBG_WARNING,
- "Check TX-Report timeout!!\n");
+ "Check TX-Report timeout!! s_sn=0x%X r_sn=0x%X\n",
+ tx_report->last_sent_sn, tx_report->last_recv_sn);
return true; /* 3 sec. (timeout) seen as acked */
}
static int __init rtl_core_module_init(void)
{
+ BUILD_BUG_ON(TX_PWR_BY_RATE_NUM_RATE < TX_PWR_BY_RATE_NUM_SECTION);
+ BUILD_BUG_ON(MAX_RATE_SECTION_NUM != MAX_RATE_SECTION);
+ BUILD_BUG_ON(MAX_BASE_NUM_IN_PHY_REG_PG_24G != MAX_RATE_SECTION);
+ BUILD_BUG_ON(MAX_BASE_NUM_IN_PHY_REG_PG_5G != (MAX_RATE_SECTION - 1));
+
if (rtl_rate_control_register())
pr_err("rtl: Unable to register rtl_rc, use default RC !!\n");
}
if ((type == 1) || (type == 2) || (type == 9) || (type == 11) ||
- (type == 101) || (type == 102) || (type == 109) || (type == 101)) {
+ (type == 101) || (type == 102) || (type == 109) || (type == 111)) {
if (!coex_sta->force_lps_on) {
/* Native power save TDMA, only for A2DP-only case
* 1/2/9/11 while wifi noisy threshold > 30
--- /dev/null
+/******************************************************************************
+ *
+ * Copyright(c) 2016-2017 Realtek Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ *****************************************************************************/
+#include "halbt_precomp.h"
+
+void ex_hal8822b_wifi_only_hw_config(struct wifi_only_cfg *wifionlycfg)
+{
+ /*BB control*/
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0x4c, 0x01800000, 0x2);
+ /*SW control*/
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0xcb4, 0xff, 0x77);
+ /*antenna mux switch */
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0x974, 0x300, 0x3);
+
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0x1990, 0x300, 0x0);
+
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0xcbc, 0x80000, 0x0);
+ /*switch to WL side controller and gnt_wl gnt_bt debug signal */
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0x70, 0xff000000, 0x0e);
+ /*gnt_wl=1 , gnt_bt=0*/
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0x1704, 0xffffffff, 0x7700);
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0x1700, 0xffffffff, 0xc00f0038);
+}
+
+void ex_hal8822b_wifi_only_scannotify(struct wifi_only_cfg *wifionlycfg,
+ u8 is_5g)
+{
+ hal8822b_wifi_only_switch_antenna(wifionlycfg, is_5g);
+}
+
+void ex_hal8822b_wifi_only_switchbandnotify(struct wifi_only_cfg *wifionlycfg,
+ u8 is_5g)
+{
+ hal8822b_wifi_only_switch_antenna(wifionlycfg, is_5g);
+}
+
+void hal8822b_wifi_only_switch_antenna(struct wifi_only_cfg *wifionlycfg,
+ u8 is_5g)
+{
+ if (is_5g)
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0xcbc, 0x300, 0x1);
+ else
+ halwifionly_phy_set_bb_reg(wifionlycfg, 0xcbc, 0x300, 0x2);
+}
--- /dev/null
+/******************************************************************************
+ *
+ * Copyright(c) 2016-2017 Realtek Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ *****************************************************************************/
+#ifndef __INC_HAL8822BWIFIONLYHWCFG_H
+#define __INC_HAL8822BWIFIONLYHWCFG_H
+
+void ex_hal8822b_wifi_only_hw_config(struct wifi_only_cfg *wifionlycfg);
+void ex_hal8822b_wifi_only_scannotify(struct wifi_only_cfg *wifionlycfg,
+ u8 is_5g);
+void ex_hal8822b_wifi_only_switchbandnotify(struct wifi_only_cfg *wifionlycfg,
+ u8 is_5g);
+void hal8822b_wifi_only_switch_antenna(struct wifi_only_cfg *wifionlycfg,
+ u8 is_5g);
+#endif
cmd_len, cmd_buf);
}
+void halbtc_send_wifi_port_id_cmd(void *bt_context)
+{
+ struct btc_coexist *btcoexist = (struct btc_coexist *)bt_context;
+ struct rtl_priv *rtlpriv = btcoexist->adapter;
+ u8 cmd_buf[1] = {0}; /* port id [2:0] = 0 */
+
+ rtlpriv->cfg->ops->fill_h2c_cmd(rtlpriv->mac80211.hw, H2C_BT_PORT_ID,
+ 1, cmd_buf);
+}
+
+void halbtc_set_default_port_id_cmd(void *bt_context)
+{
+ struct btc_coexist *btcoexist = (struct btc_coexist *)bt_context;
+ struct rtl_priv *rtlpriv = btcoexist->adapter;
+ struct ieee80211_hw *hw = rtlpriv->mac80211.hw;
+
+ if (!rtlpriv->cfg->ops->set_default_port_id_cmd)
+ return;
+
+ rtlpriv->cfg->ops->set_default_port_id_cmd(hw);
+}
+
static
void halbtc_set_bt_reg(void *btc_context, u8 reg_type, u32 offset, u32 set_val)
{
void exhalbtc_low_wifi_traffic_notify(struct btc_coexist *btcoexist);
void exhalbtc_set_single_ant_path(struct btc_coexist *btcoexist,
u8 single_ant_path);
+void halbtc_send_wifi_port_id_cmd(void *bt_context);
+void halbtc_set_default_port_id_cmd(void *bt_context);
/* The following are used by wifi_only case */
enum wifionly_chip_interface {
{11, 0, 0, 28}
};
+static const struct rtl_efuse_ops efuse_ops = {
+ .efuse_onebyte_read = efuse_one_byte_read,
+ .efuse_logical_map_read = efuse_shadow_read,
+};
+
static void efuse_shadow_read_1byte(struct ieee80211_hw *hw, u16 offset,
u8 *value);
static void efuse_shadow_read_2byte(struct ieee80211_hw *hw, u16 offset,
*pfwlen = fwlen;
}
EXPORT_SYMBOL_GPL(rtl_fill_dummy);
+
+void rtl_efuse_ops_init(struct ieee80211_hw *hw)
+{
+ struct rtl_priv *rtlpriv = rtl_priv(hw);
+
+ rtlpriv->efuse.efuse_ops = &efuse_ops;
+}
+EXPORT_SYMBOL_GPL(rtl_efuse_ops_init);
void rtl_fw_page_write(struct ieee80211_hw *hw, u32 page, const u8 *buffer,
u32 size);
void rtl_fw_block_write(struct ieee80211_hw *hw, const u8 *buffer, u32 size);
-
+void rtl_efuse_ops_init(struct ieee80211_hw *hw);
#endif
rtlpriv->cfg = (struct rtl_hal_cfg *)(id->driver_data);
rtlpriv->intf_ops = &rtl_pci_ops;
rtlpriv->glb_var = &rtl_global_var;
+ rtl_efuse_ops_init(hw);
/* MEM map */
err = pci_request_regions(pdev, KBUILD_MODNAME);
struct rtl_phy *rtlphy = &(rtlpriv->phy);
struct rtl_sta_info *sta_entry = NULL;
u16 wireless_mode = 0;
+ u8 nss;
+ struct ieee80211_tx_rate rate;
+
+ switch (get_rf_type(rtlphy)) {
+ case RF_4T4R:
+ nss = 4;
+ break;
+ case RF_3T3R:
+ nss = 3;
+ break;
+ case RF_2T2R:
+ nss = 2;
+ break;
+ default:
+ nss = 1;
+ break;
+ }
/*
*this rate is no use for true rate, firmware
} else if (wireless_mode == WIRELESS_MODE_G) {
return G_MODE_MAX_RIX;
} else if (wireless_mode == WIRELESS_MODE_N_24G) {
- if (get_rf_type(rtlphy) != RF_2T2R)
+ if (nss == 1)
return N_MODE_MCS7_RIX;
else
return N_MODE_MCS15_RIX;
} else if (wireless_mode == WIRELESS_MODE_AC_24G) {
- return AC_MODE_MCS9_RIX;
+ if (sta->bandwidth == IEEE80211_STA_RX_BW_20) {
+ ieee80211_rate_set_vht(&rate,
+ AC_MODE_MCS8_RIX,
+ nss);
+ goto out;
+ } else {
+ ieee80211_rate_set_vht(&rate,
+ AC_MODE_MCS9_RIX,
+ nss);
+ goto out;
+ }
}
return 0;
} else {
if (wireless_mode == WIRELESS_MODE_A) {
return A_MODE_MAX_RIX;
} else if (wireless_mode == WIRELESS_MODE_N_5G) {
- if (get_rf_type(rtlphy) != RF_2T2R)
+ if (nss == 1)
return N_MODE_MCS7_RIX;
else
return N_MODE_MCS15_RIX;
} else if (wireless_mode == WIRELESS_MODE_AC_5G) {
- return AC_MODE_MCS9_RIX;
+ if (sta->bandwidth == IEEE80211_STA_RX_BW_20) {
+ ieee80211_rate_set_vht(&rate,
+ AC_MODE_MCS8_RIX,
+ nss);
+ goto out;
+ } else {
+ ieee80211_rate_set_vht(&rate,
+ AC_MODE_MCS9_RIX,
+ nss);
+ goto out;
+ }
}
return 0;
}
}
+
+out:
+ return rate.idx;
}
static void _rtl_rc_rate_set_series(struct rtl_priv *rtlpriv,
}
rate->count = tries;
rate->idx = rix >= 0x00 ? rix : 0x00;
- if (rtlpriv->rtlhal.hw_type == HARDWARE_TYPE_RTL8812AE &&
- wireless_mode == WIRELESS_MODE_AC_5G)
- rate->idx += 0x10;/*2NSS for 8812AE*/
if (!not_data) {
if (txrc->short_preamble)
if (sta && sta->vht_cap.vht_supported)
rate->flags |= IEEE80211_TX_RC_80_MHZ_WIDTH;
} else {
- if (mac->bw_40)
- rate->flags |= IEEE80211_TX_RC_40_MHZ_WIDTH;
if (mac->bw_80)
rate->flags |= IEEE80211_TX_RC_80_MHZ_WIDTH;
+ else if (mac->bw_40)
+ rate->flags |= IEEE80211_TX_RC_40_MHZ_WIDTH;
}
if (sgi_20 || sgi_40 || sgi_80)
writeVal = 0x00000000;
if (rtlpriv->dm.dynamic_txhighpower_lvl == TXHIGHPWRLEVEL_BT1)
writeVal = writeVal - 0x06060606;
- else if (rtlpriv->dm.dynamic_txhighpower_lvl ==
- TXHIGHPWRLEVEL_BT2)
- writeVal = writeVal;
*(p_outwriteval + rf) = writeVal;
}
}
(u32)hdr->addr1[0], (u32)hdr->addr1[1],
(u32)hdr->addr1[2], (u32)hdr->addr1[3],
(u32)hdr->addr1[4], (u32)hdr->addr1[5]);
- memcpy(IEEE80211_SKB_RXCB(skb), rx_status, sizeof(*rx_status));
ieee80211_rx(hw, skb);
}
/* Configuration Space offset 0x70f BIT7 is used to control L0S */
tmp8 = _rtl8723be_dbi_read(rtlpriv, 0x70f);
- _rtl8723be_dbi_write(rtlpriv, 0x70f, tmp8 | BIT(7));
+ _rtl8723be_dbi_write(rtlpriv, 0x70f, tmp8 | BIT(7) |
+ ASPM_L1_LATENCY << 3);
/* Configuration Space offset 0x719 Bit3 is for L1
* BIT4 is for clock request
.alt_fw_name = "rtlwifi/rtl8821aefw.bin",
.ops = &rtl8821ae_hal_ops,
.mod_params = &rtl8821ae_mod_params,
+ .spec_ver = RTL_SPEC_SUPPORT_VHT,
.maps[SYS_ISO_CTRL] = REG_SYS_ISO_CTRL,
.maps[SYS_FUNC_EN] = REG_SYS_FUNC_EN,
.maps[SYS_CLK] = REG_SYS_CLKR,
MAX_H2CCMD
};
+enum {
+ H2C_BT_PORT_ID = 0x71,
+};
+
+#define GET_TX_REPORT_SN_V1(c2h) (c2h[6])
+#define GET_TX_REPORT_ST_V1(c2h) (c2h[0] & 0xC0)
+#define GET_TX_REPORT_RETRY_V1(c2h) (c2h[2] & 0x3F)
+#define GET_TX_REPORT_SN_V2(c2h) (c2h[6])
+#define GET_TX_REPORT_ST_V2(c2h) (c2h[7] & 0xC0)
+#define GET_TX_REPORT_RETRY_V2(c2h) (c2h[8] & 0x3F)
+
#define MAX_TX_COUNT 4
#define MAX_REGULATION_NUM 4
#define MAX_RF_PATH_NUM 4
-#define MAX_RATE_SECTION_NUM 6
+#define MAX_RATE_SECTION_NUM 6 /* = MAX_RATE_SECTION */
#define MAX_2_4G_BANDWIDTH_NUM 4
#define MAX_5G_BANDWIDTH_NUM 4
#define MAX_RF_PATH 4
#define TX_PWR_BY_RATE_NUM_BAND 2
#define TX_PWR_BY_RATE_NUM_RF 4
#define TX_PWR_BY_RATE_NUM_SECTION 12
-#define MAX_BASE_NUM_IN_PHY_REG_PG_24G 6
-#define MAX_BASE_NUM_IN_PHY_REG_PG_5G 5
+#define TX_PWR_BY_RATE_NUM_RATE 84 /* >= TX_PWR_BY_RATE_NUM_SECTION */
+#define MAX_BASE_NUM_IN_PHY_REG_PG_24G 6 /* MAX_RATE_SECTION */
+#define MAX_BASE_NUM_IN_PHY_REG_PG_5G 5 /* MAX_RATE_SECTION -1 */
#define BUFDESC_SEG_NUM 1 /* 0:2 seg, 1: 4 seg, 2: 8 seg */
HT_MCS8_MCS15,
VHT_1SSMCS0_1SSMCS9,
VHT_2SSMCS0_2SSMCS9,
+ MAX_RATE_SECTION,
};
enum intf_type {
RF90_PATH_D = 3,
};
+enum radio_mask {
+ RF_MASK_A = BIT(0),
+ RF_MASK_B = BIT(1),
+ RF_MASK_C = BIT(2),
+ RF_MASK_D = BIT(3),
+};
+
enum regulation_txpwr_lmt {
TXPWR_LMT_FCC = 0,
TXPWR_LMT_MKK = 1,
HT_CHANNEL_WIDTH_20 = 0,
HT_CHANNEL_WIDTH_20_40 = 1,
HT_CHANNEL_WIDTH_80 = 2,
+ HT_CHANNEL_WIDTH_MAX,
};
/* Ref: 802.11i sepc D10.0 7.3.2.25.1
enum rtl_spec_ver {
RTL_SPEC_NEW_RATEID = BIT(0), /* use ratr_table_mode_new */
+ RTL_SPEC_SUPPORT_VHT = BIT(1), /* support VHT */
+ RTL_SPEC_EXT_C2H = BIT(2), /* extend FW C2H (e.g. TX REPORT) */
};
struct octet_string {
u32 tx_power_by_rate_offset[TX_PWR_BY_RATE_NUM_BAND]
[TX_PWR_BY_RATE_NUM_RF]
[TX_PWR_BY_RATE_NUM_RF]
- [TX_PWR_BY_RATE_NUM_SECTION];
+ [TX_PWR_BY_RATE_NUM_RATE];
u8 txpwr_by_rate_base_24g[TX_PWR_BY_RATE_NUM_RF]
[TX_PWR_BY_RATE_NUM_RF]
[MAX_BASE_NUM_IN_PHY_REG_PG_24G];
#define EFUSE_MAX_LOGICAL_SIZE 512
struct rtl_efuse {
+ const struct rtl_efuse_ops *efuse_ops;
bool autoLoad_ok;
bool bootfromefuse;
u16 max_physical_size;
u8 channel_plan;
};
+struct rtl_efuse_ops {
+ int (*efuse_onebyte_read)(struct ieee80211_hw *hw, u16 addr, u8 *data);
+ void (*efuse_logical_map_read)(struct ieee80211_hw *hw, u8 type,
+ u16 offset, u32 *value);
+};
+
struct rtl_tx_report {
atomic_t sn;
u16 last_sent_sn;
void (*bt_coex_off_before_lps) (struct ieee80211_hw *hw);
void (*fill_h2c_cmd) (struct ieee80211_hw *hw, u8 element_id,
u32 cmd_len, u8 *p_cmdbuffer);
+ void (*set_default_port_id_cmd)(struct ieee80211_hw *hw);
bool (*get_btc_status) (void);
bool (*is_fw_header)(struct rtlwifi_firmware_header *hdr);
u32 (*rx_command_packet)(struct ieee80211_hw *hw,
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_RSI
This option enables the USB bus support in rsi drivers.
Select M (recommended), if you have a RSI 1x1 wireless module.
+config RSI_COEX
+ bool "Redpine Signals WLAN BT Coexistence support"
+ depends on BT_HCIRSI && RSI_91X
+ default y
+ ---help---
+ This option enables the WLAN BT coex support in rsi drivers.
+ Select M (recommended), if you have want to use this feature
+ and you have RS9113 module.
+
endif # WLAN_VENDOR_RSI
rsi_91x-y += rsi_91x_mgmt.o
rsi_91x-y += rsi_91x_hal.o
rsi_91x-y += rsi_91x_ps.o
+rsi_91x-$(CONFIG_RSI_COEX) += rsi_91x_coex.o
rsi_91x-$(CONFIG_RSI_DEBUGFS) += rsi_91x_debugfs.o
rsi_usb-y += rsi_91x_usb.o rsi_91x_usb_ops.o
--- /dev/null
+/**
+ * Copyright (c) 2018 Redpine Signals Inc.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include "rsi_main.h"
+#include "rsi_coex.h"
+#include "rsi_mgmt.h"
+#include "rsi_hal.h"
+
+static enum rsi_coex_queues rsi_coex_determine_coex_q
+ (struct rsi_coex_ctrl_block *coex_cb)
+{
+ enum rsi_coex_queues q_num = RSI_COEX_Q_INVALID;
+
+ if (skb_queue_len(&coex_cb->coex_tx_qs[RSI_COEX_Q_COMMON]) > 0)
+ q_num = RSI_COEX_Q_COMMON;
+ if (skb_queue_len(&coex_cb->coex_tx_qs[RSI_COEX_Q_BT]) > 0)
+ q_num = RSI_COEX_Q_BT;
+ if (skb_queue_len(&coex_cb->coex_tx_qs[RSI_COEX_Q_WLAN]) > 0)
+ q_num = RSI_COEX_Q_WLAN;
+
+ return q_num;
+}
+
+static void rsi_coex_sched_tx_pkts(struct rsi_coex_ctrl_block *coex_cb)
+{
+ enum rsi_coex_queues coex_q = RSI_COEX_Q_INVALID;
+ struct sk_buff *skb;
+
+ do {
+ coex_q = rsi_coex_determine_coex_q(coex_cb);
+ rsi_dbg(INFO_ZONE, "queue = %d\n", coex_q);
+
+ if (coex_q == RSI_COEX_Q_BT) {
+ skb = skb_dequeue(&coex_cb->coex_tx_qs[RSI_COEX_Q_BT]);
+ rsi_send_bt_pkt(coex_cb->priv, skb);
+ }
+ } while (coex_q != RSI_COEX_Q_INVALID);
+}
+
+static void rsi_coex_scheduler_thread(struct rsi_common *common)
+{
+ struct rsi_coex_ctrl_block *coex_cb =
+ (struct rsi_coex_ctrl_block *)common->coex_cb;
+ u32 timeout = EVENT_WAIT_FOREVER;
+
+ do {
+ rsi_wait_event(&coex_cb->coex_tx_thread.event, timeout);
+ rsi_reset_event(&coex_cb->coex_tx_thread.event);
+
+ rsi_coex_sched_tx_pkts(coex_cb);
+ } while (atomic_read(&coex_cb->coex_tx_thread.thread_done) == 0);
+
+ complete_and_exit(&coex_cb->coex_tx_thread.completion, 0);
+}
+
+int rsi_coex_recv_pkt(struct rsi_common *common, u8 *msg)
+{
+ u8 msg_type = msg[RSI_RX_DESC_MSG_TYPE_OFFSET];
+
+ switch (msg_type) {
+ case COMMON_CARD_READY_IND:
+ rsi_dbg(INFO_ZONE, "common card ready received\n");
+ rsi_handle_card_ready(common, msg);
+ break;
+ case SLEEP_NOTIFY_IND:
+ rsi_dbg(INFO_ZONE, "sleep notify received\n");
+ rsi_mgmt_pkt_recv(common, msg);
+ break;
+ }
+
+ return 0;
+}
+
+static inline int rsi_map_coex_q(u8 hal_queue)
+{
+ switch (hal_queue) {
+ case RSI_COEX_Q:
+ return RSI_COEX_Q_COMMON;
+ case RSI_WLAN_Q:
+ return RSI_COEX_Q_WLAN;
+ case RSI_BT_Q:
+ return RSI_COEX_Q_BT;
+ }
+ return RSI_COEX_Q_INVALID;
+}
+
+int rsi_coex_send_pkt(void *priv, struct sk_buff *skb, u8 hal_queue)
+{
+ struct rsi_common *common = (struct rsi_common *)priv;
+ struct rsi_coex_ctrl_block *coex_cb =
+ (struct rsi_coex_ctrl_block *)common->coex_cb;
+ struct skb_info *tx_params = NULL;
+ enum rsi_coex_queues coex_q;
+ int status;
+
+ coex_q = rsi_map_coex_q(hal_queue);
+ if (coex_q == RSI_COEX_Q_INVALID) {
+ rsi_dbg(ERR_ZONE, "Invalid coex queue\n");
+ return -EINVAL;
+ }
+ if (coex_q != RSI_COEX_Q_COMMON &&
+ coex_q != RSI_COEX_Q_WLAN) {
+ skb_queue_tail(&coex_cb->coex_tx_qs[coex_q], skb);
+ rsi_set_event(&coex_cb->coex_tx_thread.event);
+ return 0;
+ }
+ if (common->iface_down) {
+ tx_params =
+ (struct skb_info *)&IEEE80211_SKB_CB(skb)->driver_data;
+
+ if (!(tx_params->flags & INTERNAL_MGMT_PKT)) {
+ rsi_indicate_tx_status(common->priv, skb, -EINVAL);
+ return 0;
+ }
+ }
+
+ /* Send packet to hal */
+ if (skb->priority == MGMT_SOFT_Q)
+ status = rsi_send_mgmt_pkt(common, skb);
+ else
+ status = rsi_send_data_pkt(common, skb);
+
+ return status;
+}
+
+int rsi_coex_attach(struct rsi_common *common)
+{
+ struct rsi_coex_ctrl_block *coex_cb;
+ int cnt;
+
+ coex_cb = kzalloc(sizeof(*coex_cb), GFP_KERNEL);
+ if (!coex_cb)
+ return -ENOMEM;
+
+ common->coex_cb = (void *)coex_cb;
+ coex_cb->priv = common;
+
+ /* Initialize co-ex queues */
+ for (cnt = 0; cnt < NUM_COEX_TX_QUEUES; cnt++)
+ skb_queue_head_init(&coex_cb->coex_tx_qs[cnt]);
+ rsi_init_event(&coex_cb->coex_tx_thread.event);
+
+ /* Initialize co-ex thread */
+ if (rsi_create_kthread(common,
+ &coex_cb->coex_tx_thread,
+ rsi_coex_scheduler_thread,
+ "Coex-Tx-Thread")) {
+ rsi_dbg(ERR_ZONE, "%s: Unable to init tx thrd\n", __func__);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+void rsi_coex_detach(struct rsi_common *common)
+{
+ struct rsi_coex_ctrl_block *coex_cb =
+ (struct rsi_coex_ctrl_block *)common->coex_cb;
+ int cnt;
+
+ rsi_kill_thread(&coex_cb->coex_tx_thread);
+
+ for (cnt = 0; cnt < NUM_COEX_TX_QUEUES; cnt++)
+ skb_queue_purge(&coex_cb->coex_tx_qs[cnt]);
+
+ kfree(coex_cb);
+}
#include "rsi_mgmt.h"
#include "rsi_common.h"
#include "rsi_hal.h"
+#include "rsi_coex.h"
/**
* rsi_determine_min_weight_queue() - This function determines the queue with
mutex_unlock(&common->tx_lock);
break;
}
-
- if (q_num == MGMT_SOFT_Q) {
- status = rsi_send_mgmt_pkt(common, skb);
- } else if (q_num == MGMT_BEACON_Q) {
+ if (q_num == MGMT_BEACON_Q) {
status = rsi_send_pkt_to_bus(common, skb);
dev_kfree_skb(skb);
} else {
- status = rsi_send_data_pkt(common, skb);
+#ifdef CONFIG_RSI_COEX
+ if (common->coex_mode > 1) {
+ status = rsi_coex_send_pkt(common, skb,
+ RSI_WLAN_Q);
+ } else {
+#endif
+ if (q_num == MGMT_SOFT_Q)
+ status = rsi_send_mgmt_pkt(common, skb);
+ else
+ status = rsi_send_data_pkt(common, skb);
+#ifdef CONFIG_RSI_COEX
+ }
+#endif
}
if (status) {
*/
#include <linux/firmware.h>
+#include <net/bluetooth/bluetooth.h>
#include "rsi_mgmt.h"
#include "rsi_hal.h"
#include "rsi_sdio.h"
static struct ta_metadata metadata_flash_content[] = {
{"flash_content", 0x00010000},
{"rsi/rs9113_wlan_qspi.rps", 0x00010000},
+ {"rsi/rs9113_wlan_bt_dual_mode.rps", 0x00010000},
};
int rsi_send_pkt_to_bus(struct rsi_common *common, struct sk_buff *skb)
struct rsi_hw *adapter = common->priv;
int status;
+ if (common->coex_mode > 1)
+ mutex_lock(&common->tx_bus_mutex);
+
status = adapter->host_intf_ops->write_pkt(common->priv,
skb->data, skb->len);
+
+ if (common->coex_mode > 1)
+ mutex_unlock(&common->tx_bus_mutex);
+
return status;
}
if (status)
goto err;
- status = adapter->host_intf_ops->write_pkt(common->priv, skb->data,
- skb->len);
+ status = rsi_send_pkt_to_bus(common, skb);
if (status)
rsi_dbg(ERR_ZONE, "%s: Failed to write pkt\n", __func__);
goto err;
rsi_prepare_mgmt_desc(common, skb);
- status = adapter->host_intf_ops->write_pkt(common->priv,
- (u8 *)skb->data, skb->len);
+ status = rsi_send_pkt_to_bus(common, skb);
if (status)
rsi_dbg(ERR_ZONE, "%s: Failed to write the packet\n", __func__);
return status;
}
+int rsi_send_bt_pkt(struct rsi_common *common, struct sk_buff *skb)
+{
+ int status = -EINVAL;
+ u8 header_size = 0;
+ struct rsi_bt_desc *bt_desc;
+ u8 queueno = ((skb->data[1] >> 4) & 0xf);
+
+ if (queueno == RSI_BT_MGMT_Q) {
+ status = rsi_send_pkt_to_bus(common, skb);
+ if (status)
+ rsi_dbg(ERR_ZONE, "%s: Failed to write bt mgmt pkt\n",
+ __func__);
+ goto out;
+ }
+ header_size = FRAME_DESC_SZ;
+ if (header_size > skb_headroom(skb)) {
+ rsi_dbg(ERR_ZONE, "%s: Not enough headroom\n", __func__);
+ status = -ENOSPC;
+ goto out;
+ }
+ skb_push(skb, header_size);
+ memset(skb->data, 0, header_size);
+ bt_desc = (struct rsi_bt_desc *)skb->data;
+
+ rsi_set_len_qno(&bt_desc->len_qno, (skb->len - FRAME_DESC_SZ),
+ RSI_BT_DATA_Q);
+ bt_desc->bt_pkt_type = cpu_to_le16(bt_cb(skb)->pkt_type);
+
+ status = rsi_send_pkt_to_bus(common, skb);
+ if (status)
+ rsi_dbg(ERR_ZONE, "%s: Failed to write bt pkt\n", __func__);
+
+out:
+ dev_kfree_skb(skb);
+ return status;
+}
+
int rsi_prepare_beacon(struct rsi_common *common, struct sk_buff *skb)
{
struct rsi_hw *adapter = (struct rsi_hw *)common->priv;
{
struct rsi_common *common = adapter->priv;
- common->coex_mode = RSI_DEV_COEX_MODE_WIFI_ALONE;
- common->oper_mode = RSI_DEV_OPMODE_WIFI_ALONE;
- adapter->device_model = RSI_DEV_9113;
-
switch (adapter->device_model) {
case RSI_DEV_9113:
if (rsi_load_firmware(adapter)) {
#include <linux/module.h>
#include <linux/firmware.h>
+#include <net/rsi_91x.h>
#include "rsi_mgmt.h"
#include "rsi_common.h"
+#include "rsi_coex.h"
#include "rsi_hal.h"
u32 rsi_zone_enabled = /* INFO_ZONE |
0;
EXPORT_SYMBOL_GPL(rsi_zone_enabled);
+#ifdef CONFIG_RSI_COEX
+static struct rsi_proto_ops g_proto_ops = {
+ .coex_send_pkt = rsi_coex_send_pkt,
+ .get_host_intf = rsi_get_host_intf,
+ .set_bt_context = rsi_set_bt_context,
+};
+#endif
+
/**
* rsi_dbg() - This function outputs informational messages.
* @zone: Zone of interest for output message.
static char *opmode_str(int oper_mode)
{
switch (oper_mode) {
- case RSI_DEV_OPMODE_WIFI_ALONE:
+ case DEV_OPMODE_WIFI_ALONE:
return "Wi-Fi alone";
+ case DEV_OPMODE_BT_ALONE:
+ return "BT EDR alone";
+ case DEV_OPMODE_BT_LE_ALONE:
+ return "BT LE alone";
+ case DEV_OPMODE_BT_DUAL:
+ return "BT Dual";
+ case DEV_OPMODE_STA_BT:
+ return "Wi-Fi STA + BT EDR";
+ case DEV_OPMODE_STA_BT_LE:
+ return "Wi-Fi STA + BT LE";
+ case DEV_OPMODE_STA_BT_DUAL:
+ return "Wi-Fi STA + BT DUAL";
+ case DEV_OPMODE_AP_BT:
+ return "Wi-Fi AP + BT EDR";
+ case DEV_OPMODE_AP_BT_DUAL:
+ return "Wi-Fi AP + BT DUAL";
}
return "Unknown";
*
* Return: 0 on success, -1 on failure.
*/
-int rsi_read_pkt(struct rsi_common *common, s32 rcv_pkt_len)
+int rsi_read_pkt(struct rsi_common *common, u8 *rx_pkt, s32 rcv_pkt_len)
{
u8 *frame_desc = NULL, extended_desc = 0;
u32 index, length = 0, queueno = 0;
u16 actual_length = 0, offset;
struct sk_buff *skb = NULL;
+#ifdef CONFIG_RSI_COEX
+ u8 bt_pkt_type;
+#endif
index = 0;
do {
- frame_desc = &common->rx_data_pkt[index];
+ frame_desc = &rx_pkt[index];
actual_length = *(u16 *)&frame_desc[0];
offset = *(u16 *)&frame_desc[2];
switch (queueno) {
case RSI_COEX_Q:
- rsi_mgmt_pkt_recv(common, (frame_desc + offset));
+#ifdef CONFIG_RSI_COEX
+ if (common->coex_mode > 1)
+ rsi_coex_recv_pkt(common, frame_desc + offset);
+ else
+#endif
+ rsi_mgmt_pkt_recv(common,
+ (frame_desc + offset));
break;
+
case RSI_WIFI_DATA_Q:
skb = rsi_prepare_skb(common,
(frame_desc + offset),
rsi_mgmt_pkt_recv(common, (frame_desc + offset));
break;
+#ifdef CONFIG_RSI_COEX
+ case RSI_BT_MGMT_Q:
+ case RSI_BT_DATA_Q:
+#define BT_RX_PKT_TYPE_OFST 14
+#define BT_CARD_READY_IND 0x89
+ bt_pkt_type = frame_desc[offset + BT_RX_PKT_TYPE_OFST];
+ if (bt_pkt_type == BT_CARD_READY_IND) {
+ rsi_dbg(INFO_ZONE, "BT Card ready recvd\n");
+ if (rsi_bt_ops.attach(common, &g_proto_ops))
+ rsi_dbg(ERR_ZONE,
+ "Failed to attach BT module\n");
+ } else {
+ if (common->bt_adapter)
+ rsi_bt_ops.recv_pkt(common->bt_adapter,
+ frame_desc + offset);
+ }
+ break;
+#endif
+
default:
rsi_dbg(ERR_ZONE, "%s: pkt from invalid queue: %d\n",
__func__, queueno);
complete_and_exit(&common->tx_thread.completion, 0);
}
+#ifdef CONFIG_RSI_COEX
+enum rsi_host_intf rsi_get_host_intf(void *priv)
+{
+ struct rsi_common *common = (struct rsi_common *)priv;
+
+ return common->priv->rsi_host_intf;
+}
+
+void rsi_set_bt_context(void *priv, void *bt_context)
+{
+ struct rsi_common *common = (struct rsi_common *)priv;
+
+ common->bt_adapter = bt_context;
+}
+#endif
+
/**
* rsi_91x_init() - This function initializes os interface operations.
* @void: Void.
*
* Return: Pointer to the adapter structure on success, NULL on failure .
*/
-struct rsi_hw *rsi_91x_init(void)
+struct rsi_hw *rsi_91x_init(u16 oper_mode)
{
struct rsi_hw *adapter = NULL;
struct rsi_common *common = NULL;
mutex_init(&common->mutex);
mutex_init(&common->tx_lock);
mutex_init(&common->rx_lock);
+ mutex_init(&common->tx_bus_mutex);
if (rsi_create_kthread(common,
&common->tx_thread,
timer_setup(&common->roc_timer, rsi_roc_timeout, 0);
init_completion(&common->wlan_init_completion);
common->init_done = true;
+ adapter->device_model = RSI_DEV_9113;
+ common->oper_mode = oper_mode;
+
+ /* Determine coex mode */
+ switch (common->oper_mode) {
+ case DEV_OPMODE_STA_BT_DUAL:
+ case DEV_OPMODE_STA_BT:
+ case DEV_OPMODE_STA_BT_LE:
+ case DEV_OPMODE_BT_ALONE:
+ case DEV_OPMODE_BT_LE_ALONE:
+ case DEV_OPMODE_BT_DUAL:
+ common->coex_mode = 2;
+ break;
+ case DEV_OPMODE_AP_BT_DUAL:
+ case DEV_OPMODE_AP_BT:
+ common->coex_mode = 4;
+ break;
+ case DEV_OPMODE_WIFI_ALONE:
+ common->coex_mode = 1;
+ break;
+ default:
+ common->oper_mode = 1;
+ common->coex_mode = 1;
+ }
+ rsi_dbg(INFO_ZONE, "%s: oper_mode = %d, coex_mode = %d\n",
+ __func__, common->oper_mode, common->coex_mode);
+
+ adapter->device_model = RSI_DEV_9113;
+#ifdef CONFIG_RSI_COEX
+ if (common->coex_mode > 1) {
+ if (rsi_coex_attach(common)) {
+ rsi_dbg(ERR_ZONE, "Failed to init coex module\n");
+ goto err;
+ }
+ }
+#endif
+
return adapter;
err:
for (ii = 0; ii < NUM_SOFT_QUEUES; ii++)
skb_queue_purge(&common->tx_queue[ii]);
+#ifdef CONFIG_RSI_COEX
+ if (common->coex_mode > 1) {
+ if (common->bt_adapter) {
+ rsi_bt_ops.detach(common->bt_adapter);
+ common->bt_adapter = NULL;
+ }
+ rsi_coex_detach(common);
+ }
+#endif
+
common->init_done = false;
kfree(common);
return -EINVAL;
}
-static int rsi_handle_card_ready(struct rsi_common *common, u8 *msg)
+int rsi_handle_card_ready(struct rsi_common *common, u8 *msg)
{
switch (common->fsm_state) {
case FSM_CARD_NOT_READY:
#include <linux/module.h>
#include "rsi_sdio.h"
#include "rsi_common.h"
+#include "rsi_coex.h"
#include "rsi_hal.h"
+/* Default operating mode is wlan STA + BT */
+static u16 dev_oper_mode = DEV_OPMODE_STA_BT_DUAL;
+module_param(dev_oper_mode, ushort, 0444);
+MODULE_PARM_DESC(dev_oper_mode,
+ "1[Wi-Fi], 4[BT], 8[BT LE], 5[Wi-Fi STA + BT classic]\n"
+ "9[Wi-Fi STA + BT LE], 13[Wi-Fi STA + BT classic + BT LE]\n"
+ "6[AP + BT classic], 14[AP + BT classic + BT LE]");
+
/**
* rsi_sdio_set_cmd52_arg() - This function prepares cmd 52 read/write arg.
* @rw: Read/write
int status;
queueno = ((pkt[1] >> 4) & 0xf);
+ if (queueno == RSI_BT_MGMT_Q || queueno == RSI_BT_DATA_Q)
+ queueno = RSI_BT_Q;
num_blocks = len / block_size;
const struct sdio_device_id *id)
{
struct rsi_hw *adapter;
+ struct rsi_91x_sdiodev *sdev;
+ int status;
rsi_dbg(INIT_ZONE, "%s: Init function called\n", __func__);
- adapter = rsi_91x_init();
+ adapter = rsi_91x_init(dev_oper_mode);
if (!adapter) {
rsi_dbg(ERR_ZONE, "%s: Failed to init os intf ops\n",
__func__);
- return 1;
+ return -EINVAL;
}
adapter->rsi_host_intf = RSI_HOST_INTF_SDIO;
adapter->host_intf_ops = &sdio_host_intf_ops;
if (rsi_init_sdio_interface(adapter, pfunction)) {
rsi_dbg(ERR_ZONE, "%s: Failed to init sdio interface\n",
__func__);
- goto fail;
+ status = -EIO;
+ goto fail_free_adapter;
+ }
+ sdev = (struct rsi_91x_sdiodev *)adapter->rsi_dev;
+ rsi_init_event(&sdev->rx_thread.event);
+ status = rsi_create_kthread(adapter->priv, &sdev->rx_thread,
+ rsi_sdio_rx_thread, "SDIO-RX-Thread");
+ if (status) {
+ rsi_dbg(ERR_ZONE, "%s: Unable to init rx thrd\n", __func__);
+ goto fail_free_adapter;
}
+ skb_queue_head_init(&sdev->rx_q.head);
+ sdev->rx_q.num_rx_pkts = 0;
+
sdio_claim_host(pfunction);
if (sdio_claim_irq(pfunction, rsi_handle_interrupt)) {
rsi_dbg(ERR_ZONE, "%s: Failed to request IRQ\n", __func__);
sdio_release_host(pfunction);
- goto fail;
+ status = -EIO;
+ goto fail_kill_thread;
}
sdio_release_host(pfunction);
rsi_dbg(INIT_ZONE, "%s: Registered Interrupt handler\n", __func__);
if (rsi_hal_device_init(adapter)) {
rsi_dbg(ERR_ZONE, "%s: Failed in device init\n", __func__);
- sdio_claim_host(pfunction);
- sdio_release_irq(pfunction);
- sdio_disable_func(pfunction);
- sdio_release_host(pfunction);
- goto fail;
+ status = -EINVAL;
+ goto fail_kill_thread;
}
rsi_dbg(INFO_ZONE, "===> RSI Device Init Done <===\n");
if (rsi_sdio_master_access_msword(adapter, MISC_CFG_BASE_ADDR)) {
rsi_dbg(ERR_ZONE, "%s: Unable to set ms word reg\n", __func__);
- return -EIO;
+ status = -EIO;
+ goto fail_dev_init;
}
adapter->priv->hibernate_resume = false;
adapter->priv->reinit_hw = false;
return 0;
-fail:
+
+fail_dev_init:
+ sdio_claim_host(pfunction);
+ sdio_release_irq(pfunction);
+ sdio_disable_func(pfunction);
+ sdio_release_host(pfunction);
+fail_kill_thread:
+ rsi_kill_thread(&sdev->rx_thread);
+fail_free_adapter:
rsi_91x_deinit(adapter);
rsi_dbg(ERR_ZONE, "%s: Failed in probe...Exiting\n", __func__);
- return 1;
+ return status;
}
static void ulp_read_write(struct rsi_hw *adapter, u16 addr, u32 data,
return;
dev = (struct rsi_91x_sdiodev *)adapter->rsi_dev;
+
+ rsi_kill_thread(&dev->rx_thread);
sdio_claim_host(pfunction);
sdio_release_irq(pfunction);
sdio_release_host(pfunction);
*/
#include <linux/firmware.h>
+#include <net/rsi_91x.h>
#include "rsi_sdio.h"
#include "rsi_common.h"
return status;
}
+void rsi_sdio_rx_thread(struct rsi_common *common)
+{
+ struct rsi_hw *adapter = common->priv;
+ struct rsi_91x_sdiodev *sdev = adapter->rsi_dev;
+ struct sk_buff *skb;
+ int status;
+
+ do {
+ rsi_wait_event(&sdev->rx_thread.event, EVENT_WAIT_FOREVER);
+ rsi_reset_event(&sdev->rx_thread.event);
+
+ while (true) {
+ if (atomic_read(&sdev->rx_thread.thread_done))
+ goto out;
+
+ skb = skb_dequeue(&sdev->rx_q.head);
+ if (!skb)
+ break;
+ if (sdev->rx_q.num_rx_pkts > 0)
+ sdev->rx_q.num_rx_pkts--;
+ status = rsi_read_pkt(common, skb->data, skb->len);
+ if (status) {
+ rsi_dbg(ERR_ZONE, "Failed to read the packet\n");
+ dev_kfree_skb(skb);
+ break;
+ }
+ dev_kfree_skb(skb);
+ }
+ } while (1);
+
+out:
+ rsi_dbg(INFO_ZONE, "%s: Terminated SDIO RX thread\n", __func__);
+ skb_queue_purge(&sdev->rx_q.head);
+ atomic_inc(&sdev->rx_thread.thread_done);
+ complete_and_exit(&sdev->rx_thread.completion, 0);
+}
+
/**
* rsi_process_pkt() - This Function reads rx_blocks register and figures out
* the size of the rx pkt.
u32 rcv_pkt_len = 0;
int status = 0;
u8 value = 0;
+ struct sk_buff *skb;
+
+ if (dev->rx_q.num_rx_pkts >= RSI_MAX_RX_PKTS)
+ return 0;
num_blks = ((adapter->interrupt_status & 1) |
((adapter->interrupt_status >> RECV_NUM_BLOCKS) << 1));
rcv_pkt_len = (num_blks * 256);
- common->rx_data_pkt = kmalloc(rcv_pkt_len, GFP_KERNEL);
- if (!common->rx_data_pkt) {
- rsi_dbg(ERR_ZONE, "%s: Failed in memory allocation\n",
- __func__);
+ skb = dev_alloc_skb(rcv_pkt_len);
+ if (!skb)
return -ENOMEM;
- }
- status = rsi_sdio_host_intf_read_pkt(adapter,
- common->rx_data_pkt,
- rcv_pkt_len);
+ status = rsi_sdio_host_intf_read_pkt(adapter, skb->data, rcv_pkt_len);
if (status) {
rsi_dbg(ERR_ZONE, "%s: Failed to read packet from card\n",
__func__);
- goto fail;
+ dev_kfree_skb(skb);
+ return status;
}
+ skb_put(skb, rcv_pkt_len);
+ skb_queue_tail(&dev->rx_q.head, skb);
+ dev->rx_q.num_rx_pkts++;
- status = rsi_read_pkt(common, rcv_pkt_len);
+ rsi_set_event(&dev->rx_thread.event);
-fail:
- kfree(common->rx_data_pkt);
- return status;
+ return 0;
}
/**
*/
#include <linux/module.h>
+#include <net/rsi_91x.h>
#include "rsi_usb.h"
#include "rsi_hal.h"
+#include "rsi_coex.h"
+
+/* Default operating mode is wlan STA + BT */
+static u16 dev_oper_mode = DEV_OPMODE_STA_BT_DUAL;
+module_param(dev_oper_mode, ushort, 0444);
+MODULE_PARM_DESC(dev_oper_mode,
+ "1[Wi-Fi], 4[BT], 8[BT LE], 5[Wi-Fi STA + BT classic]\n"
+ "9[Wi-Fi STA + BT LE], 13[Wi-Fi STA + BT classic + BT LE]\n"
+ "6[AP + BT classic], 14[AP + BT classic + BT LE]");
+
+static int rsi_rx_urb_submit(struct rsi_hw *adapter, u8 ep_num);
/**
* rsi_usb_card_write() - This function writes to the USB Card.
struct usb_host_interface *iface_desc;
struct usb_endpoint_descriptor *endpoint;
__le16 buffer_size;
- int ii, bep_found = 0;
+ int ii, bin_found = 0, bout_found = 0;
iface_desc = &(interface->altsetting[0]);
for (ii = 0; ii < iface_desc->desc.bNumEndpoints; ++ii) {
endpoint = &(iface_desc->endpoint[ii].desc);
- if ((!(dev->bulkin_endpoint_addr)) &&
+ if (!dev->bulkin_endpoint_addr[bin_found] &&
(endpoint->bEndpointAddress & USB_DIR_IN) &&
- ((endpoint->bmAttributes &
- USB_ENDPOINT_XFERTYPE_MASK) ==
+ ((endpoint->bmAttributes & USB_ENDPOINT_XFERTYPE_MASK) ==
USB_ENDPOINT_XFER_BULK)) {
buffer_size = endpoint->wMaxPacketSize;
- dev->bulkin_size = buffer_size;
- dev->bulkin_endpoint_addr =
+ dev->bulkin_size[bin_found] = buffer_size;
+ dev->bulkin_endpoint_addr[bin_found] =
endpoint->bEndpointAddress;
+ bin_found++;
}
- if (!dev->bulkout_endpoint_addr[bep_found] &&
+ if (!dev->bulkout_endpoint_addr[bout_found] &&
!(endpoint->bEndpointAddress & USB_DIR_IN) &&
((endpoint->bmAttributes & USB_ENDPOINT_XFERTYPE_MASK) ==
- USB_ENDPOINT_XFER_BULK)) {
- dev->bulkout_endpoint_addr[bep_found] =
+ USB_ENDPOINT_XFER_BULK)) {
+ buffer_size = endpoint->wMaxPacketSize;
+ dev->bulkout_endpoint_addr[bout_found] =
endpoint->bEndpointAddress;
buffer_size = endpoint->wMaxPacketSize;
- dev->bulkout_size[bep_found] = buffer_size;
- bep_found++;
+ dev->bulkout_size[bout_found] = buffer_size;
+ bout_found++;
}
- if (bep_found >= MAX_BULK_EP)
+ if (bin_found >= MAX_BULK_EP || bout_found >= MAX_BULK_EP)
break;
}
- if (!(dev->bulkin_endpoint_addr) &&
- (dev->bulkout_endpoint_addr[0]))
+ if (!(dev->bulkin_endpoint_addr[0]) &&
+ dev->bulkout_endpoint_addr[0])
return -EINVAL;
return 0;
*/
static void rsi_rx_done_handler(struct urb *urb)
{
- struct rsi_hw *adapter = urb->context;
- struct rsi_91x_usbdev *dev = (struct rsi_91x_usbdev *)adapter->rsi_dev;
+ struct rx_usb_ctrl_block *rx_cb = urb->context;
+ struct rsi_91x_usbdev *dev = (struct rsi_91x_usbdev *)rx_cb->data;
+ int status = -EINVAL;
if (urb->status)
- return;
+ goto out;
+
+ if (urb->actual_length <= 0) {
+ rsi_dbg(INFO_ZONE, "%s: Zero length packet\n", __func__);
+ goto out;
+ }
+ if (skb_queue_len(&dev->rx_q) >= RSI_MAX_RX_PKTS) {
+ rsi_dbg(INFO_ZONE, "Max RX packets reached\n");
+ goto out;
+ }
+ skb_put(rx_cb->rx_skb, urb->actual_length);
+ skb_queue_tail(&dev->rx_q, rx_cb->rx_skb);
rsi_set_event(&dev->rx_thread.event);
+ status = 0;
+
+out:
+ if (rsi_rx_urb_submit(dev->priv, rx_cb->ep_num))
+ rsi_dbg(ERR_ZONE, "%s: Failed in urb submission", __func__);
+
+ if (status)
+ dev_kfree_skb(rx_cb->rx_skb);
}
/**
*
* Return: 0 on success, a negative error code on failure.
*/
-static int rsi_rx_urb_submit(struct rsi_hw *adapter)
+static int rsi_rx_urb_submit(struct rsi_hw *adapter, u8 ep_num)
{
struct rsi_91x_usbdev *dev = (struct rsi_91x_usbdev *)adapter->rsi_dev;
- struct urb *urb = dev->rx_usb_urb[0];
+ struct rx_usb_ctrl_block *rx_cb = &dev->rx_cb[ep_num - 1];
+ struct urb *urb = rx_cb->rx_urb;
int status;
+ struct sk_buff *skb;
+ u8 dword_align_bytes = 0;
+
+#define RSI_MAX_RX_USB_PKT_SIZE 3000
+ skb = dev_alloc_skb(RSI_MAX_RX_USB_PKT_SIZE);
+ if (!skb)
+ return -ENOMEM;
+ skb_reserve(skb, MAX_DWORD_ALIGN_BYTES);
+ dword_align_bytes = (unsigned long)skb->data & 0x3f;
+ if (dword_align_bytes > 0)
+ skb_push(skb, dword_align_bytes);
+ urb->transfer_buffer = skb->data;
+ rx_cb->rx_skb = skb;
usb_fill_bulk_urb(urb,
dev->usbdev,
usb_rcvbulkpipe(dev->usbdev,
- dev->bulkin_endpoint_addr),
+ dev->bulkin_endpoint_addr[ep_num - 1]),
urb->transfer_buffer,
- 3000,
+ RSI_MAX_RX_USB_PKT_SIZE,
rsi_rx_done_handler,
- adapter);
+ rx_cb);
status = usb_submit_urb(urb, GFP_KERNEL);
if (status)
struct rsi_91x_usbdev *dev = (struct rsi_91x_usbdev *)adapter->rsi_dev;
rsi_kill_thread(&dev->rx_thread);
- usb_free_urb(dev->rx_usb_urb[0]);
- kfree(adapter->priv->rx_data_pkt);
+
+ usb_free_urb(dev->rx_cb[0].rx_urb);
+ if (adapter->priv->coex_mode > 1)
+ usb_free_urb(dev->rx_cb[1].rx_urb);
+
kfree(dev->tx_buffer);
}
+static int rsi_usb_init_rx(struct rsi_hw *adapter)
+{
+ struct rsi_91x_usbdev *dev = (struct rsi_91x_usbdev *)adapter->rsi_dev;
+ struct rx_usb_ctrl_block *rx_cb;
+ u8 idx, num_rx_cb;
+
+ num_rx_cb = (adapter->priv->coex_mode > 1 ? 2 : 1);
+
+ for (idx = 0; idx < num_rx_cb; idx++) {
+ rx_cb = &dev->rx_cb[idx];
+
+ rx_cb->rx_urb = usb_alloc_urb(0, GFP_KERNEL);
+ if (!rx_cb->rx_urb) {
+ rsi_dbg(ERR_ZONE, "Failed alloc rx urb[%d]\n", idx);
+ goto err;
+ }
+ rx_cb->ep_num = idx + 1;
+ rx_cb->data = (void *)dev;
+ }
+ skb_queue_head_init(&dev->rx_q);
+ rsi_init_event(&dev->rx_thread.event);
+ if (rsi_create_kthread(adapter->priv, &dev->rx_thread,
+ rsi_usb_rx_thread, "RX-Thread")) {
+ rsi_dbg(ERR_ZONE, "%s: Unable to init rx thrd\n", __func__);
+ goto err;
+ }
+
+ return 0;
+
+err:
+ usb_free_urb(dev->rx_cb[0].rx_urb);
+ if (adapter->priv->coex_mode > 1)
+ usb_free_urb(dev->rx_cb[1].rx_urb);
+
+ return -1;
+}
+
/**
* rsi_init_usb_interface() - This function initializes the usb interface.
* @adapter: Pointer to the adapter structure.
struct usb_interface *pfunction)
{
struct rsi_91x_usbdev *rsi_dev;
- struct rsi_common *common = adapter->priv;
int status;
rsi_dev = kzalloc(sizeof(*rsi_dev), GFP_KERNEL);
adapter->rsi_dev = rsi_dev;
rsi_dev->usbdev = interface_to_usbdev(pfunction);
+ rsi_dev->priv = (void *)adapter;
- if (rsi_find_bulk_in_and_out_endpoints(pfunction, adapter))
- return -EINVAL;
+ if (rsi_find_bulk_in_and_out_endpoints(pfunction, adapter)) {
+ status = -EINVAL;
+ goto fail_eps;
+ }
adapter->device = &pfunction->dev;
usb_set_intfdata(pfunction, adapter);
- common->rx_data_pkt = kmalloc(2048, GFP_KERNEL);
- if (!common->rx_data_pkt) {
- rsi_dbg(ERR_ZONE, "%s: Failed to allocate memory\n",
- __func__);
- return -ENOMEM;
- }
-
rsi_dev->tx_buffer = kmalloc(2048, GFP_KERNEL);
if (!rsi_dev->tx_buffer) {
status = -ENOMEM;
- goto fail_tx;
+ goto fail_eps;
}
- rsi_dev->rx_usb_urb[0] = usb_alloc_urb(0, GFP_KERNEL);
- if (!rsi_dev->rx_usb_urb[0]) {
+
+ if (rsi_usb_init_rx(adapter)) {
+ rsi_dbg(ERR_ZONE, "Failed to init RX handle\n");
status = -ENOMEM;
goto fail_rx;
}
- rsi_dev->rx_usb_urb[0]->transfer_buffer = adapter->priv->rx_data_pkt;
+
rsi_dev->tx_blk_size = 252;
adapter->block_size = rsi_dev->tx_blk_size;
/* Initializing function callbacks */
- adapter->rx_urb_submit = rsi_rx_urb_submit;
adapter->check_hw_queue_status = rsi_usb_check_queue_status;
adapter->determine_event_timeout = rsi_usb_event_timeout;
adapter->rsi_host_intf = RSI_HOST_INTF_USB;
adapter->host_intf_ops = &usb_host_intf_ops;
- rsi_init_event(&rsi_dev->rx_thread.event);
- status = rsi_create_kthread(common, &rsi_dev->rx_thread,
- rsi_usb_rx_thread, "RX-Thread");
- if (status) {
- rsi_dbg(ERR_ZONE, "%s: Unable to init rx thrd\n", __func__);
- goto fail_thread;
- }
-
#ifdef CONFIG_RSI_DEBUGFS
/* In USB, one less than the MAX_DEBUGFS_ENTRIES entries is required */
adapter->num_debugfs_entries = (MAX_DEBUGFS_ENTRIES - 1);
rsi_dbg(INIT_ZONE, "%s: Enabled the interface\n", __func__);
return 0;
-fail_thread:
- usb_free_urb(rsi_dev->rx_usb_urb[0]);
fail_rx:
kfree(rsi_dev->tx_buffer);
-fail_tx:
- kfree(common->rx_data_pkt);
+
+fail_eps:
+ kfree(rsi_dev);
+
return status;
}
rsi_dbg(INIT_ZONE, "%s: Init function called\n", __func__);
- adapter = rsi_91x_init();
+ adapter = rsi_91x_init(dev_oper_mode);
if (!adapter) {
rsi_dbg(ERR_ZONE, "%s: Failed to init os intf ops\n",
__func__);
rsi_dbg(INIT_ZONE, "%s: Device Init Done\n", __func__);
}
- status = rsi_rx_urb_submit(adapter);
+ status = rsi_rx_urb_submit(adapter, WLAN_EP);
if (status)
goto err1;
+ if (adapter->priv->coex_mode > 1) {
+ status = rsi_rx_urb_submit(adapter, BT_EP);
+ if (status)
+ goto err1;
+ }
+
return 0;
err1:
rsi_deinit_usb_interface(adapter);
struct rsi_hw *adapter = common->priv;
struct rsi_91x_usbdev *dev = (struct rsi_91x_usbdev *)adapter->rsi_dev;
int status;
+ struct sk_buff *skb;
do {
rsi_wait_event(&dev->rx_thread.event, EVENT_WAIT_FOREVER);
+ rsi_reset_event(&dev->rx_thread.event);
- if (atomic_read(&dev->rx_thread.thread_done))
- goto out;
+ while (true) {
+ if (atomic_read(&dev->rx_thread.thread_done))
+ goto out;
- mutex_lock(&common->rx_lock);
- status = rsi_read_pkt(common, 0);
- if (status) {
- rsi_dbg(ERR_ZONE, "%s: Failed To read data", __func__);
- mutex_unlock(&common->rx_lock);
- return;
- }
- mutex_unlock(&common->rx_lock);
- rsi_reset_event(&dev->rx_thread.event);
- if (adapter->rx_urb_submit(adapter)) {
- rsi_dbg(ERR_ZONE,
- "%s: Failed in urb submission", __func__);
- return;
+ skb = skb_dequeue(&dev->rx_q);
+ if (!skb)
+ break;
+ status = rsi_read_pkt(common, skb->data, 0);
+ if (status) {
+ rsi_dbg(ERR_ZONE, "%s: Failed To read data",
+ __func__);
+ break;
+ }
+ dev_kfree_skb(skb);
}
} while (1);
out:
rsi_dbg(INFO_ZONE, "%s: Terminated thread\n", __func__);
+ skb_queue_purge(&dev->rx_q);
complete_and_exit(&dev->rx_thread.completion, 0);
}
--- /dev/null
+/**
+ * Copyright (c) 2018 Redpine Signals Inc.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#ifndef __RSI_COEX_H__
+#define __RSI_COEX_H__
+
+#include "rsi_common.h"
+
+#ifdef CONFIG_RSI_COEX
+#define COMMON_CARD_READY_IND 0
+#define NUM_COEX_TX_QUEUES 5
+
+struct rsi_coex_ctrl_block {
+ struct rsi_common *priv;
+ struct sk_buff_head coex_tx_qs[NUM_COEX_TX_QUEUES];
+ struct rsi_thread coex_tx_thread;
+};
+
+int rsi_coex_attach(struct rsi_common *common);
+void rsi_coex_detach(struct rsi_common *common);
+int rsi_coex_send_pkt(void *priv, struct sk_buff *skb, u8 proto_type);
+int rsi_coex_recv_pkt(struct rsi_common *common, u8 *msg);
+#endif
+#endif
u8 *name)
{
init_completion(&thread->completion);
+ atomic_set(&thread->thread_done, 0);
thread->task = kthread_run(func_ptr, common, "%s", name);
if (IS_ERR(thread->task))
return (int)PTR_ERR(thread->task);
void rsi_mac80211_detach(struct rsi_hw *hw);
u16 rsi_get_connected_channel(struct ieee80211_vif *vif);
-struct rsi_hw *rsi_91x_init(void);
+struct rsi_hw *rsi_91x_init(u16 oper_mode);
void rsi_91x_deinit(struct rsi_hw *adapter);
-int rsi_read_pkt(struct rsi_common *common, s32 rcv_pkt_len);
+int rsi_read_pkt(struct rsi_common *common, u8 *rx_pkt, s32 rcv_pkt_len);
#ifdef CONFIG_PM
int rsi_config_wowlan(struct rsi_hw *adapter, struct cfg80211_wowlan *wowlan);
#endif
#ifndef __RSI_HAL_H__
#define __RSI_HAL_H__
+/* Device Operating modes */
+#define DEV_OPMODE_WIFI_ALONE 1
+#define DEV_OPMODE_BT_ALONE 4
+#define DEV_OPMODE_BT_LE_ALONE 8
+#define DEV_OPMODE_BT_DUAL 12
+#define DEV_OPMODE_STA_BT 5
+#define DEV_OPMODE_STA_BT_LE 9
+#define DEV_OPMODE_STA_BT_DUAL 13
+#define DEV_OPMODE_AP_BT 6
+#define DEV_OPMODE_AP_BT_DUAL 14
+
#define FLASH_WRITE_CHUNK_SIZE (4 * 1024)
#define FLASH_SECTOR_SIZE (4 * 1024)
#define FW_FLASH_OFFSET 0x820
#define LMAC_VER_OFFSET (FW_FLASH_OFFSET + 0x200)
+#define MAX_DWORD_ALIGN_BYTES 64
struct bl_header {
__le32 flags;
u8 sta_id;
} __packed;
+struct rsi_bt_desc {
+ __le16 len_qno;
+ __le16 reserved1;
+ __le32 reserved2;
+ __le32 reserved3;
+ __le16 reserved4;
+ __le16 bt_pkt_type;
+} __packed;
+
int rsi_hal_device_init(struct rsi_hw *adapter);
int rsi_prepare_beacon(struct rsi_common *common, struct sk_buff *skb);
int rsi_send_pkt_to_bus(struct rsi_common *common, struct sk_buff *skb);
+int rsi_send_bt_pkt(struct rsi_common *common, struct sk_buff *skb);
#endif
#include <linux/string.h>
#include <linux/skbuff.h>
#include <net/mac80211.h>
+#include <net/rsi_91x.h>
struct rsi_sta {
struct ieee80211_sta *sta;
#define MGMT_HW_Q 10
#define BEACON_HW_Q 11
-/* Queue information */
-#define RSI_COEX_Q 0x0
-#define RSI_WIFI_MGMT_Q 0x4
-#define RSI_WIFI_DATA_Q 0x5
#define IEEE80211_MGMT_FRAME 0x00
#define IEEE80211_CTL_FRAME 0x04
#define RSI_WOW_NO_CONNECTION BIT(1)
#define RSI_DEV_9113 1
+#define RSI_MAX_RX_PKTS 64
struct version_info {
u16 major;
struct rsi_hw *priv;
struct vif_priv vif_info[RSI_MAX_VIFS];
+ void *coex_cb;
bool mgmt_q_block;
struct version_info lmac_ver;
u8 obm_ant_sel_val;
int tx_power;
u8 ant_in_use;
+ /* Mutex used for writing packet to bus */
+ struct mutex tx_bus_mutex;
bool hibernate_resume;
bool reinit_hw;
u8 wow_flags;
bool p2p_enabled;
struct timer_list roc_timer;
struct ieee80211_vif *roc_vif;
-};
-enum host_intf {
- RSI_HOST_INTF_SDIO = 0,
- RSI_HOST_INTF_USB
+ void *bt_adapter;
};
struct eepromrw_info {
struct device *device;
u8 sc_nvifs;
- enum host_intf rsi_host_intf;
+ enum rsi_host_intf rsi_host_intf;
u16 block_size;
enum ps_state ps_state;
struct rsi_ps_info ps_info;
void *rsi_dev;
struct rsi_host_intf_ops *host_intf_ops;
int (*check_hw_queue_status)(struct rsi_hw *adapter, u8 q_num);
- int (*rx_urb_submit)(struct rsi_hw *adapter);
int (*determine_event_timeout)(struct rsi_hw *adapter);
};
u8 *fw);
int (*reinit_device)(struct rsi_hw *adapter);
};
+
+enum rsi_host_intf rsi_get_host_intf(void *priv);
+void rsi_set_bt_context(void *priv, void *bt_context);
+
#endif
#define WOW_PATTERN_SIZE 256
/* Receive Frame Types */
+#define RSI_RX_DESC_MSG_TYPE_OFFSET 2
#define TA_CONFIRM_TYPE 0x01
#define RX_DOT11_MGMT 0x02
#define TX_STATUS_IND 0x04
#define BEACON_EVENT_IND 0x08
#define PROBEREQ_CONFIRM 2
#define CARD_READY_IND 0x00
+#define SLEEP_NOTIFY_IND 0x06
#define RSI_DELETE_PEER 0x0
#define RSI_ADD_PEER 0x1
*addr = cpu_to_le16(len | ((qno & 7) << 12));
}
+int rsi_handle_card_ready(struct rsi_common *common, u8 *msg);
int rsi_mgmt_pkt_recv(struct rsi_common *common, u8 *msg);
int rsi_set_vap_capabilities(struct rsi_common *common, enum opmode mode,
u8 *mac_addr, u8 vap_id, u8 vap_status);
u32 buf_available_counter;
};
+struct rsi_sdio_rx_q {
+ u8 num_rx_pkts;
+ struct sk_buff_head head;
+};
+
struct rsi_91x_sdiodev {
struct sdio_func *pfunction;
struct task_struct *sdio_irq_task;
u16 tx_blk_size;
u8 write_fail;
bool buff_status_updated;
+ struct rsi_sdio_rx_q rx_q;
+ struct rsi_thread rx_thread;
};
void rsi_interrupt_handler(struct rsi_hw *adapter);
void rsi_sdio_ack_intr(struct rsi_hw *adapter, u8 int_bit);
int rsi_sdio_determine_event_timeout(struct rsi_hw *adapter);
int rsi_sdio_check_buffer_status(struct rsi_hw *adapter, u8 q_num);
+void rsi_sdio_rx_thread(struct rsi_common *common);
#endif
#define USB_VENDOR_REGISTER_WRITE 0x16
#define RSI_USB_TX_HEAD_ROOM 128
-#define MAX_RX_URBS 1
+#define MAX_RX_URBS 2
#define MAX_BULK_EP 8
#define WLAN_EP 1
#define BT_EP 2
#define RSI_USB_BUF_SIZE 4096
#define RSI_USB_CTRL_BUF_SIZE 0x04
+struct rx_usb_ctrl_block {
+ u8 *data;
+ struct urb *rx_urb;
+ struct sk_buff *rx_skb;
+ u8 ep_num;
+};
+
struct rsi_91x_usbdev {
+ void *priv;
struct rsi_thread rx_thread;
u8 endpoint;
struct usb_device *usbdev;
struct usb_interface *pfunction;
- struct urb *rx_usb_urb[MAX_RX_URBS];
+ struct rx_usb_ctrl_block rx_cb[MAX_RX_URBS];
u8 *tx_buffer;
- __le16 bulkin_size;
- u8 bulkin_endpoint_addr;
+ __le16 bulkin_size[MAX_BULK_EP];
+ u8 bulkin_endpoint_addr[MAX_BULK_EP];
__le16 bulkout_size[MAX_BULK_EP];
u8 bulkout_endpoint_addr[MAX_BULK_EP];
u32 tx_blk_size;
u8 write_fail;
+ struct sk_buff_head rx_q;
};
static inline int rsi_usb_check_queue_status(struct rsi_hw *adapter, u8 q_num)
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_ST
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_TI
goto out;
}
- wl->nvs_len = fw->size;
- wl->nvs = kmemdup(fw->data, wl->nvs_len, GFP_KERNEL);
+ wl->nvs = kmemdup(fw->data, fw->size, GFP_KERNEL);
if (!wl->nvs) {
wl1251_error("could not allocate memory for the nvs file");
goto out;
}
+ wl->nvs_len = fw->size;
+
ret = 0;
out:
goto out;
}
- if (wl->nvs == NULL && !wl->use_eeprom) {
- /* No NVS from netlink, try to get it from the filesystem */
- ret = wl1251_fetch_nvs(wl);
- if (ret < 0)
- goto out;
- }
-
out:
return ret;
}
return 0;
}
+#define NVS_OFF_MAC_LEN 0x19
+#define NVS_OFF_MAC_ADDR_LO 0x1a
+#define NVS_OFF_MAC_ADDR_HI 0x1b
+#define NVS_OFF_MAC_DATA 0x1c
+
+static int wl1251_check_nvs_mac(struct wl1251 *wl)
+{
+ if (wl->nvs_len < 0x24)
+ return -ENODATA;
+
+ /* length is 2 and data address is 0x546c (ANDed with 0xfffe) */
+ if (wl->nvs[NVS_OFF_MAC_LEN] != 2 ||
+ wl->nvs[NVS_OFF_MAC_ADDR_LO] != 0x6d ||
+ wl->nvs[NVS_OFF_MAC_ADDR_HI] != 0x54)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int wl1251_read_nvs_mac(struct wl1251 *wl)
+{
+ u8 mac[ETH_ALEN];
+ int i, ret;
+
+ ret = wl1251_check_nvs_mac(wl);
+ if (ret)
+ return ret;
+
+ /* MAC is stored in reverse order */
+ for (i = 0; i < ETH_ALEN; i++)
+ mac[i] = wl->nvs[NVS_OFF_MAC_DATA + ETH_ALEN - i - 1];
+
+ /* 00:00:20:07:03:09 is in example file wl1251-nvs.bin, so invalid */
+ if (ether_addr_equal_unaligned(mac, "\x00\x00\x20\x07\x03\x09"))
+ return -EINVAL;
+
+ memcpy(wl->mac_addr, mac, ETH_ALEN);
+ return 0;
+}
+
+static int wl1251_write_nvs_mac(struct wl1251 *wl)
+{
+ int i, ret;
+
+ ret = wl1251_check_nvs_mac(wl);
+ if (ret)
+ return ret;
+
+ /* MAC is stored in reverse order */
+ for (i = 0; i < ETH_ALEN; i++)
+ wl->nvs[NVS_OFF_MAC_DATA + i] = wl->mac_addr[ETH_ALEN - i - 1];
+
+ return 0;
+}
+
static int wl1251_register_hw(struct wl1251 *wl)
{
int ret;
wl->hw->queues = 4;
+ if (wl->nvs == NULL && !wl->use_eeprom) {
+ ret = wl1251_fetch_nvs(wl);
+ if (ret < 0)
+ goto out;
+ }
+
if (wl->use_eeprom)
- wl1251_read_eeprom_mac(wl);
+ ret = wl1251_read_eeprom_mac(wl);
+ else
+ ret = wl1251_read_nvs_mac(wl);
+
+ if (ret == 0 && !is_valid_ether_addr(wl->mac_addr))
+ ret = -EINVAL;
+
+ if (ret < 0) {
+ /*
+ * In case our MAC address is not correctly set,
+ * we use a random but Nokia MAC.
+ */
+ static const u8 nokia_oui[3] = {0x00, 0x1f, 0xdf};
+ memcpy(wl->mac_addr, nokia_oui, 3);
+ get_random_bytes(wl->mac_addr + 3, 3);
+ if (!wl->use_eeprom)
+ wl1251_write_nvs_mac(wl);
+ wl1251_warning("MAC address in eeprom or nvs data is not valid");
+ wl1251_warning("Setting random MAC address: %pM", wl->mac_addr);
+ }
ret = wl1251_register_hw(wl);
if (ret)
struct ieee80211_hw *hw;
struct wl1251 *wl;
int i;
- static const u8 nokia_oui[3] = {0x00, 0x1f, 0xdf};
hw = ieee80211_alloc_hw(sizeof(*wl), &wl1251_ops);
if (!hw) {
INIT_WORK(&wl->irq_work, wl1251_irq_work);
INIT_WORK(&wl->tx_work, wl1251_tx_work);
- /*
- * In case our MAC address is not correctly set,
- * we use a random but Nokia MAC.
- */
- memcpy(wl->mac_addr, nokia_oui, 3);
- get_random_bytes(wl->mac_addr + 3, 3);
-
wl->state = WL1251_STATE_OFF;
mutex_init(&wl->mutex);
spin_lock_init(&wl->wl_lock);
If you have a wireless card belonging to this class, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the questions about cards. If you say Y, you will be asked for
+ kernel: saying N will just cause the configurator to skip all the
+ questions about these cards. If you say Y, you will be asked for
your specific card in the following questions.
if WLAN_VENDOR_ZYDAS
int found = 0;
int i, position = 0;
- q = &mac->ack_wait_queue;
spin_lock_irqsave(&q->lock, flags);
skb_queue_walk(q, skb) {
snprintf(filename, sizeof(filename), "io_ring_q%d", i);
pfile = debugfs_create_file(filename,
- S_IRUSR | S_IWUSR,
+ 0600,
vif->xenvif_dbg_root,
&vif->queues[i],
&xenvif_dbg_io_ring_ops_fops);
if (vif->ctrl_irq) {
pfile = debugfs_create_file("ctrl",
- S_IRUSR,
+ 0400,
vif->xenvif_dbg_root,
vif,
&xenvif_dbg_ctrl_ops_fops);
return len;
}
-static DEVICE_ATTR(rxbuf_min, S_IRUGO|S_IWUSR, show_rxbuf, store_rxbuf);
-static DEVICE_ATTR(rxbuf_max, S_IRUGO|S_IWUSR, show_rxbuf, store_rxbuf);
-static DEVICE_ATTR(rxbuf_cur, S_IRUGO, show_rxbuf, NULL);
+static DEVICE_ATTR(rxbuf_min, 0644, show_rxbuf, store_rxbuf);
+static DEVICE_ATTR(rxbuf_max, 0644, show_rxbuf, store_rxbuf);
+static DEVICE_ATTR(rxbuf_cur, 0444, show_rxbuf, NULL);
static struct attribute *xennet_dev_attrs[] = {
&dev_attr_rxbuf_min.attr,
disk->queue = q;
disk->flags = GENHD_FL_EXT_DEVT;
nvdimm_namespace_disk_name(&nsblk->common, disk->disk_name);
- set_capacity(disk, 0);
- device_add_disk(dev, disk);
if (devm_add_action_or_reset(dev, nd_blk_release_disk, disk))
return -ENOMEM;
}
set_capacity(disk, available_disk_size >> SECTOR_SHIFT);
+ device_add_disk(dev, disk);
revalidate_disk(disk);
return 0;
}
queue_flag_set_unlocked(QUEUE_FLAG_NONROT, btt->btt_queue);
btt->btt_queue->queuedata = btt;
- set_capacity(btt->btt_disk, 0);
- device_add_disk(&btt->nd_btt->dev, btt->btt_disk);
if (btt_meta_size(btt)) {
int rc = nd_integrity_init(btt->btt_disk, btt_meta_size(btt));
}
}
set_capacity(btt->btt_disk, btt->nlba * btt->sector_size >> 9);
+ device_add_disk(&btt->nd_btt->dev, btt->btt_disk);
btt->nd_btt->size = btt->nlba * (u64)btt->sector_size;
revalidate_disk(btt->btt_disk);
struct device *nd_pfn_devinit(struct nd_pfn *nd_pfn,
struct nd_namespace_common *ndns)
{
- struct device *dev = &nd_pfn->dev;
+ struct device *dev;
if (!nd_pfn)
return NULL;
struct device_attribute *attr, char *buf)
{
struct nd_region *nd_region = to_nd_region(dev);
- unsigned long flags = nd_region->flags;
- return sprintf(buf, "%s%s\n",
- flags & BIT(ND_REGION_PERSIST_CACHE) ? "cpu_cache " : "",
- flags & BIT(ND_REGION_PERSIST_MEMCTRL) ? "memory_controller " : "");
+ if (test_bit(ND_REGION_PERSIST_CACHE, &nd_region->flags))
+ return sprintf(buf, "cpu_cache\n");
+ else if (test_bit(ND_REGION_PERSIST_MEMCTRL, &nd_region->flags))
+ return sprintf(buf, "memory_controller\n");
+ else
+ return sprintf(buf, "\n");
}
static DEVICE_ATTR_RO(persistence_domain);
return 0;
}
+ if (a == &dev_attr_persistence_domain.attr) {
+ if ((nd_region->flags & (BIT(ND_REGION_PERSIST_CACHE)
+ | BIT(ND_REGION_PERSIST_MEMCTRL))) == 0)
+ return 0;
+ return a->mode;
+ }
+
if (a != &dev_attr_set_cookie.attr
&& a != &dev_attr_available_size.attr)
return a->mode;
ns->disk->disk_name);
nvme_mpath_add_disk(ns->head);
- nvme_mpath_add_disk_links(ns);
return;
out_unlink_ns:
mutex_lock(&ctrl->subsys->lock);
return;
if (ns->disk && ns->disk->flags & GENHD_FL_UP) {
- nvme_mpath_remove_disk_links(ns);
sysfs_remove_group(&disk_to_dev(ns->disk)->kobj,
&nvme_ns_id_attr_group);
if (ns->ndev)
ret = -EINVAL;
goto out;
}
+ if (opts->discovery_nqn) {
+ pr_debug("Ignoring nr_io_queues value for discovery controller\n");
+ break;
+ }
+
opts->nr_io_queues = min_t(unsigned int,
num_online_cpus(), token);
break;
sizeof(struct fcnvme_lsdesc_cr_assoc_cmd));
assoc_rqst->assoc_cmd.ersp_ratio = cpu_to_be16(ersp_ratio);
- assoc_rqst->assoc_cmd.sqsize = cpu_to_be16(qsize);
+ assoc_rqst->assoc_cmd.sqsize = cpu_to_be16(qsize - 1);
/* Linux supports only Dynamic controllers */
assoc_rqst->assoc_cmd.cntlid = cpu_to_be16(0xffff);
uuid_copy(&assoc_rqst->assoc_cmd.hostid, &ctrl->ctrl.opts->host->id);
sizeof(struct fcnvme_lsdesc_cr_conn_cmd));
conn_rqst->connect_cmd.ersp_ratio = cpu_to_be16(ersp_ratio);
conn_rqst->connect_cmd.qid = cpu_to_be16(queue->qnum);
- conn_rqst->connect_cmd.sqsize = cpu_to_be16(qsize);
+ conn_rqst->connect_cmd.sqsize = cpu_to_be16(qsize - 1);
lsop->queue = queue;
lsreq->rqstaddr = conn_rqst;
goto out_free_tag_set;
}
- ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.opts->queue_size);
+ ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
if (ret)
goto out_cleanup_blk_queue;
- ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.opts->queue_size);
+ ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
if (ret)
goto out_delete_hw_queues;
if (ret)
goto out_free_io_queues;
- ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.opts->queue_size);
+ ret = nvme_fc_create_hw_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
if (ret)
goto out_free_io_queues;
- ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.opts->queue_size);
+ ret = nvme_fc_connect_io_queues(ctrl, ctrl->ctrl.sqsize + 1);
if (ret)
goto out_delete_hw_queues;
nvme_fc_init_queue(ctrl, 0);
ret = __nvme_fc_create_hw_queue(ctrl, &ctrl->queues[0], 0,
- NVME_AQ_BLK_MQ_DEPTH);
+ NVME_AQ_DEPTH);
if (ret)
goto out_free_queue;
ret = nvme_fc_connect_admin_queue(ctrl, &ctrl->queues[0],
- NVME_AQ_BLK_MQ_DEPTH,
- (NVME_AQ_BLK_MQ_DEPTH / 4));
+ NVME_AQ_DEPTH, (NVME_AQ_DEPTH / 4));
if (ret)
goto out_delete_hw_queue;
}
ctrl->ctrl.sqsize =
- min_t(int, NVME_CAP_MQES(ctrl->ctrl.cap) + 1, ctrl->ctrl.sqsize);
+ min_t(int, NVME_CAP_MQES(ctrl->ctrl.cap), ctrl->ctrl.sqsize);
ret = nvme_enable_ctrl(&ctrl->ctrl, ctrl->ctrl.cap);
if (ret)
opts->queue_size = ctrl->ctrl.maxcmd;
}
+ if (opts->queue_size > ctrl->ctrl.sqsize + 1) {
+ /* warn if sqsize is lower than queue_size */
+ dev_warn(ctrl->ctrl.device,
+ "queue_size %zu > ctrl sqsize %u, clamping down\n",
+ opts->queue_size, ctrl->ctrl.sqsize + 1);
+ opts->queue_size = ctrl->ctrl.sqsize + 1;
+ }
+
ret = nvme_fc_init_aen_ops(ctrl);
if (ret)
goto out_term_aen_ops;
mutex_unlock(&head->subsys->lock);
}
-void nvme_mpath_add_disk_links(struct nvme_ns *ns)
-{
- struct kobject *slave_disk_kobj, *holder_disk_kobj;
-
- if (!ns->head->disk)
- return;
-
- slave_disk_kobj = &disk_to_dev(ns->disk)->kobj;
- if (sysfs_create_link(ns->head->disk->slave_dir, slave_disk_kobj,
- kobject_name(slave_disk_kobj)))
- return;
-
- holder_disk_kobj = &disk_to_dev(ns->head->disk)->kobj;
- if (sysfs_create_link(ns->disk->part0.holder_dir, holder_disk_kobj,
- kobject_name(holder_disk_kobj)))
- sysfs_remove_link(ns->head->disk->slave_dir,
- kobject_name(slave_disk_kobj));
-}
-
void nvme_mpath_remove_disk(struct nvme_ns_head *head)
{
if (!head->disk)
blk_cleanup_queue(head->disk->queue);
put_disk(head->disk);
}
-
-void nvme_mpath_remove_disk_links(struct nvme_ns *ns)
-{
- if (!ns->head->disk)
- return;
-
- sysfs_remove_link(ns->disk->part0.holder_dir,
- kobject_name(&disk_to_dev(ns->head->disk)->kobj));
- sysfs_remove_link(ns->head->disk->slave_dir,
- kobject_name(&disk_to_dev(ns->disk)->kobj));
-}
void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl);
int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl,struct nvme_ns_head *head);
void nvme_mpath_add_disk(struct nvme_ns_head *head);
-void nvme_mpath_add_disk_links(struct nvme_ns *ns);
void nvme_mpath_remove_disk(struct nvme_ns_head *head);
-void nvme_mpath_remove_disk_links(struct nvme_ns *ns);
static inline void nvme_mpath_clear_current_path(struct nvme_ns *ns)
{
static inline void nvme_mpath_remove_disk(struct nvme_ns_head *head)
{
}
-static inline void nvme_mpath_add_disk_links(struct nvme_ns *ns)
-{
-}
-static inline void nvme_mpath_remove_disk_links(struct nvme_ns *ns)
-{
-}
static inline void nvme_mpath_clear_current_path(struct nvme_ns *ns)
{
}
if (!(csts & NVME_CSTS_CFS) && !nssro)
return false;
- /* If PCI error recovery process is happening, we cannot reset or
- * the recovery mechanism will surely fail.
- */
- if (pci_channel_offline(to_pci_dev(dev->dev)))
- return false;
-
return true;
}
struct nvme_command cmd;
u32 csts = readl(dev->bar + NVME_REG_CSTS);
+ /* If PCI error recovery process is happening, we cannot reset or
+ * the recovery mechanism will surely fail.
+ */
+ mb();
+ if (pci_channel_offline(to_pci_dev(dev->dev)))
+ return BLK_EH_RESET_TIMER;
+
/*
* Reset immediately if the controller is failed
*/
int result, nr_io_queues;
unsigned long size;
- nr_io_queues = num_present_cpus();
+ nr_io_queues = num_possible_cpus();
result = nvme_set_queue_count(&dev->ctrl, &nr_io_queues);
if (result < 0)
return result;
/* setup bus numbers */
val = dw_pcie_readl_dbi(pci, PCI_PRIMARY_BUS);
val &= 0xff000000;
- val |= 0x00010100;
+ val |= 0x00ff0100;
dw_pcie_writel_dbi(pci, PCI_PRIMARY_BUS, val);
/* setup command register */
quirk_dma_func1_alias);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_TTI, 0x0642,
quirk_dma_func1_alias);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_TTI, 0x0645,
+ quirk_dma_func1_alias);
/* https://bugs.gentoo.org/show_bug.cgi?id=497630 */
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_JMICRON,
PCI_DEVICE_ID_JMICRON_JMB388_ESD,
if (irq_is_percpu_devid(irq))
disable_percpu_irq(irq);
else
- disable_irq(irq);
+ disable_irq_nosync(irq);
}
per_cpu(cpu_armpmu, cpu) = NULL;
return 0;
}
EXPORT_SYMBOL_GPL(ufs_qcom_phy_power_off);
+
+MODULE_AUTHOR("Yaniv Gardi <ygardi@codeaurora.org>");
+MODULE_AUTHOR("Vivek Gautam <vivek.gautam@codeaurora.org>");
+MODULE_DESCRIPTION("Universal Flash Storage (UFS) QCOM PHY");
+MODULE_LICENSE("GPL v2");
return ret;
}
-static const struct chromeos_laptop samsung_series_5_550 = {
+static struct chromeos_laptop samsung_series_5_550 = {
.i2c_peripherals = {
/* Touchpad. */
{ .add = setup_cyapa_tp, I2C_ADAPTER_SMBUS },
},
};
-static const struct chromeos_laptop samsung_series_5 = {
+static struct chromeos_laptop samsung_series_5 = {
.i2c_peripherals = {
/* Light Sensor. */
{ .add = setup_tsl2583_als, I2C_ADAPTER_SMBUS },
},
};
-static const struct chromeos_laptop chromebook_pixel = {
+static struct chromeos_laptop chromebook_pixel = {
.i2c_peripherals = {
/* Touch Screen. */
{ .add = setup_atmel_1664s_ts, I2C_ADAPTER_PANEL },
},
};
-static const struct chromeos_laptop hp_chromebook_14 = {
+static struct chromeos_laptop hp_chromebook_14 = {
.i2c_peripherals = {
/* Touchpad. */
{ .add = setup_cyapa_tp, I2C_ADAPTER_DESIGNWARE_0 },
},
};
-static const struct chromeos_laptop dell_chromebook_11 = {
+static struct chromeos_laptop dell_chromebook_11 = {
.i2c_peripherals = {
/* Touchpad. */
{ .add = setup_cyapa_tp, I2C_ADAPTER_DESIGNWARE_0 },
},
};
-static const struct chromeos_laptop toshiba_cb35 = {
+static struct chromeos_laptop toshiba_cb35 = {
.i2c_peripherals = {
/* Touchpad. */
{ .add = setup_cyapa_tp, I2C_ADAPTER_DESIGNWARE_0 },
},
};
-static const struct chromeos_laptop acer_c7_chromebook = {
+static struct chromeos_laptop acer_c7_chromebook = {
.i2c_peripherals = {
/* Touchpad. */
{ .add = setup_cyapa_tp, I2C_ADAPTER_SMBUS },
},
};
-static const struct chromeos_laptop acer_ac700 = {
+static struct chromeos_laptop acer_ac700 = {
.i2c_peripherals = {
/* Light Sensor. */
{ .add = setup_tsl2563_als, I2C_ADAPTER_SMBUS },
},
};
-static const struct chromeos_laptop acer_c720 = {
+static struct chromeos_laptop acer_c720 = {
.i2c_peripherals = {
/* Touchscreen. */
{ .add = setup_atmel_1664s_ts, I2C_ADAPTER_DESIGNWARE_1 },
},
};
-static const struct chromeos_laptop hp_pavilion_14_chromebook = {
+static struct chromeos_laptop hp_pavilion_14_chromebook = {
.i2c_peripherals = {
/* Touchpad. */
{ .add = setup_cyapa_tp, I2C_ADAPTER_SMBUS },
},
};
-static const struct chromeos_laptop cr48 = {
+static struct chromeos_laptop cr48 = {
.i2c_peripherals = {
/* Light Sensor. */
{ .add = setup_tsl2563_als, I2C_ADAPTER_SMBUS },
If you have an ACPI-compatible ASUS laptop, say Y or M here.
+#
+# The DELL_SMBIOS driver depends on ACPI_WMI and/or DCDBAS if those
+# backends are selected. The "depends" line prevents a configuration
+# where DELL_SMBIOS=y while either of those dependencies =m.
+#
config DELL_SMBIOS
- tristate
+ tristate "Dell SMBIOS driver"
+ depends on DCDBAS || DCDBAS=n
+ depends on ACPI_WMI || ACPI_WMI=n
+ ---help---
+ This provides support for the Dell SMBIOS calling interface.
+ If you have a Dell computer you should enable this option.
+
+ Be sure to select at least one backend for it to work properly.
config DELL_SMBIOS_WMI
- tristate "Dell SMBIOS calling interface (WMI implementation)"
+ bool "Dell SMBIOS driver WMI backend"
+ default y
depends on ACPI_WMI
select DELL_WMI_DESCRIPTOR
- select DELL_SMBIOS
+ depends on DELL_SMBIOS
---help---
This provides an implementation for the Dell SMBIOS calling interface
communicated over ACPI-WMI.
- If you have a Dell computer from >2007 you should say Y or M here.
+ If you have a Dell computer from >2007 you should say Y here.
If you aren't sure and this module doesn't work for your computer
it just won't load.
config DELL_SMBIOS_SMM
- tristate "Dell SMBIOS calling interface (SMM implementation)"
+ bool "Dell SMBIOS driver SMM backend"
+ default y
depends on DCDBAS
- select DELL_SMBIOS
+ depends on DELL_SMBIOS
---help---
This provides an implementation for the Dell SMBIOS calling interface
communicated over SMI/SMM.
- If you have a Dell computer from <=2017 you should say Y or M here.
+ If you have a Dell computer from <=2017 you should say Y here.
If you aren't sure and this module doesn't work for your computer
it just won't load.
obj-$(CONFIG_ACPI_CMPC) += classmate-laptop.o
obj-$(CONFIG_COMPAL_LAPTOP) += compal-laptop.o
obj-$(CONFIG_DELL_SMBIOS) += dell-smbios.o
-obj-$(CONFIG_DELL_SMBIOS_WMI) += dell-smbios-wmi.o
-obj-$(CONFIG_DELL_SMBIOS_SMM) += dell-smbios-smm.o
+dell-smbios-objs := dell-smbios-base.o
+dell-smbios-$(CONFIG_DELL_SMBIOS_WMI) += dell-smbios-wmi.o
+dell-smbios-$(CONFIG_DELL_SMBIOS_SMM) += dell-smbios-smm.o
obj-$(CONFIG_DELL_LAPTOP) += dell-laptop.o
obj-$(CONFIG_DELL_WMI) += dell-wmi.o
obj-$(CONFIG_DELL_WMI_DESCRIPTOR) += dell-wmi-descriptor.o
--- /dev/null
+/*
+ * Common functions for kernel modules using Dell SMBIOS
+ *
+ * Copyright (c) Red Hat <mjg@redhat.com>
+ * Copyright (c) 2014 Gabriele Mazzotta <gabriele.mzt@gmail.com>
+ * Copyright (c) 2014 Pali Rohár <pali.rohar@gmail.com>
+ *
+ * Based on documentation in the libsmbios package:
+ * Copyright (C) 2005-2014 Dell Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/capability.h>
+#include <linux/dmi.h>
+#include <linux/err.h>
+#include <linux/mutex.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include "dell-smbios.h"
+
+static u32 da_supported_commands;
+static int da_num_tokens;
+static struct platform_device *platform_device;
+static struct calling_interface_token *da_tokens;
+static struct device_attribute *token_location_attrs;
+static struct device_attribute *token_value_attrs;
+static struct attribute **token_attrs;
+static DEFINE_MUTEX(smbios_mutex);
+
+struct smbios_device {
+ struct list_head list;
+ struct device *device;
+ int (*call_fn)(struct calling_interface_buffer *arg);
+};
+
+struct smbios_call {
+ u32 need_capability;
+ int cmd_class;
+ int cmd_select;
+};
+
+/* calls that are whitelisted for given capabilities */
+static struct smbios_call call_whitelist[] = {
+ /* generally tokens are allowed, but may be further filtered or
+ * restricted by token blacklist or whitelist
+ */
+ {CAP_SYS_ADMIN, CLASS_TOKEN_READ, SELECT_TOKEN_STD},
+ {CAP_SYS_ADMIN, CLASS_TOKEN_READ, SELECT_TOKEN_AC},
+ {CAP_SYS_ADMIN, CLASS_TOKEN_READ, SELECT_TOKEN_BAT},
+ {CAP_SYS_ADMIN, CLASS_TOKEN_WRITE, SELECT_TOKEN_STD},
+ {CAP_SYS_ADMIN, CLASS_TOKEN_WRITE, SELECT_TOKEN_AC},
+ {CAP_SYS_ADMIN, CLASS_TOKEN_WRITE, SELECT_TOKEN_BAT},
+ /* used by userspace: fwupdate */
+ {CAP_SYS_ADMIN, CLASS_ADMIN_PROP, SELECT_ADMIN_PROP},
+ /* used by userspace: fwupd */
+ {CAP_SYS_ADMIN, CLASS_INFO, SELECT_DOCK},
+ {CAP_SYS_ADMIN, CLASS_FLASH_INTERFACE, SELECT_FLASH_INTERFACE},
+};
+
+/* calls that are explicitly blacklisted */
+static struct smbios_call call_blacklist[] = {
+ {0x0000, 1, 7}, /* manufacturing use */
+ {0x0000, 6, 5}, /* manufacturing use */
+ {0x0000, 11, 3}, /* write once */
+ {0x0000, 11, 7}, /* write once */
+ {0x0000, 11, 11}, /* write once */
+ {0x0000, 19, -1}, /* diagnostics */
+ /* handled by kernel: dell-laptop */
+ {0x0000, CLASS_INFO, SELECT_RFKILL},
+ {0x0000, CLASS_KBD_BACKLIGHT, SELECT_KBD_BACKLIGHT},
+};
+
+struct token_range {
+ u32 need_capability;
+ u16 min;
+ u16 max;
+};
+
+/* tokens that are whitelisted for given capabilities */
+static struct token_range token_whitelist[] = {
+ /* used by userspace: fwupdate */
+ {CAP_SYS_ADMIN, CAPSULE_EN_TOKEN, CAPSULE_DIS_TOKEN},
+ /* can indicate to userspace that WMI is needed */
+ {0x0000, WSMT_EN_TOKEN, WSMT_DIS_TOKEN}
+};
+
+/* tokens that are explicitly blacklisted */
+static struct token_range token_blacklist[] = {
+ {0x0000, 0x0058, 0x0059}, /* ME use */
+ {0x0000, 0x00CD, 0x00D0}, /* raid shadow copy */
+ {0x0000, 0x013A, 0x01FF}, /* sata shadow copy */
+ {0x0000, 0x0175, 0x0176}, /* write once */
+ {0x0000, 0x0195, 0x0197}, /* diagnostics */
+ {0x0000, 0x01DC, 0x01DD}, /* manufacturing use */
+ {0x0000, 0x027D, 0x0284}, /* diagnostics */
+ {0x0000, 0x02E3, 0x02E3}, /* manufacturing use */
+ {0x0000, 0x02FF, 0x02FF}, /* manufacturing use */
+ {0x0000, 0x0300, 0x0302}, /* manufacturing use */
+ {0x0000, 0x0325, 0x0326}, /* manufacturing use */
+ {0x0000, 0x0332, 0x0335}, /* fan control */
+ {0x0000, 0x0350, 0x0350}, /* manufacturing use */
+ {0x0000, 0x0363, 0x0363}, /* manufacturing use */
+ {0x0000, 0x0368, 0x0368}, /* manufacturing use */
+ {0x0000, 0x03F6, 0x03F7}, /* manufacturing use */
+ {0x0000, 0x049E, 0x049F}, /* manufacturing use */
+ {0x0000, 0x04A0, 0x04A3}, /* disagnostics */
+ {0x0000, 0x04E6, 0x04E7}, /* manufacturing use */
+ {0x0000, 0x4000, 0x7FFF}, /* internal BIOS use */
+ {0x0000, 0x9000, 0x9001}, /* internal BIOS use */
+ {0x0000, 0xA000, 0xBFFF}, /* write only */
+ {0x0000, 0xEFF0, 0xEFFF}, /* internal BIOS use */
+ /* handled by kernel: dell-laptop */
+ {0x0000, BRIGHTNESS_TOKEN, BRIGHTNESS_TOKEN},
+ {0x0000, KBD_LED_OFF_TOKEN, KBD_LED_AUTO_TOKEN},
+ {0x0000, KBD_LED_AC_TOKEN, KBD_LED_AC_TOKEN},
+ {0x0000, KBD_LED_AUTO_25_TOKEN, KBD_LED_AUTO_75_TOKEN},
+ {0x0000, KBD_LED_AUTO_100_TOKEN, KBD_LED_AUTO_100_TOKEN},
+ {0x0000, GLOBAL_MIC_MUTE_ENABLE, GLOBAL_MIC_MUTE_DISABLE},
+};
+
+static LIST_HEAD(smbios_device_list);
+
+int dell_smbios_error(int value)
+{
+ switch (value) {
+ case 0: /* Completed successfully */
+ return 0;
+ case -1: /* Completed with error */
+ return -EIO;
+ case -2: /* Function not supported */
+ return -ENXIO;
+ default: /* Unknown error */
+ return -EINVAL;
+ }
+}
+EXPORT_SYMBOL_GPL(dell_smbios_error);
+
+int dell_smbios_register_device(struct device *d, void *call_fn)
+{
+ struct smbios_device *priv;
+
+ priv = devm_kzalloc(d, sizeof(struct smbios_device), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+ get_device(d);
+ priv->device = d;
+ priv->call_fn = call_fn;
+ mutex_lock(&smbios_mutex);
+ list_add_tail(&priv->list, &smbios_device_list);
+ mutex_unlock(&smbios_mutex);
+ dev_dbg(d, "Added device: %s\n", d->driver->name);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(dell_smbios_register_device);
+
+void dell_smbios_unregister_device(struct device *d)
+{
+ struct smbios_device *priv;
+
+ mutex_lock(&smbios_mutex);
+ list_for_each_entry(priv, &smbios_device_list, list) {
+ if (priv->device == d) {
+ list_del(&priv->list);
+ put_device(d);
+ break;
+ }
+ }
+ mutex_unlock(&smbios_mutex);
+ dev_dbg(d, "Remove device: %s\n", d->driver->name);
+}
+EXPORT_SYMBOL_GPL(dell_smbios_unregister_device);
+
+int dell_smbios_call_filter(struct device *d,
+ struct calling_interface_buffer *buffer)
+{
+ u16 t = 0;
+ int i;
+
+ /* can't make calls over 30 */
+ if (buffer->cmd_class > 30) {
+ dev_dbg(d, "class too big: %u\n", buffer->cmd_class);
+ return -EINVAL;
+ }
+
+ /* supported calls on the particular system */
+ if (!(da_supported_commands & (1 << buffer->cmd_class))) {
+ dev_dbg(d, "invalid command, supported commands: 0x%8x\n",
+ da_supported_commands);
+ return -EINVAL;
+ }
+
+ /* match against call blacklist */
+ for (i = 0; i < ARRAY_SIZE(call_blacklist); i++) {
+ if (buffer->cmd_class != call_blacklist[i].cmd_class)
+ continue;
+ if (buffer->cmd_select != call_blacklist[i].cmd_select &&
+ call_blacklist[i].cmd_select != -1)
+ continue;
+ dev_dbg(d, "blacklisted command: %u/%u\n",
+ buffer->cmd_class, buffer->cmd_select);
+ return -EINVAL;
+ }
+
+ /* if a token call, find token ID */
+
+ if ((buffer->cmd_class == CLASS_TOKEN_READ ||
+ buffer->cmd_class == CLASS_TOKEN_WRITE) &&
+ buffer->cmd_select < 3) {
+ /* find the matching token ID */
+ for (i = 0; i < da_num_tokens; i++) {
+ if (da_tokens[i].location != buffer->input[0])
+ continue;
+ t = da_tokens[i].tokenID;
+ break;
+ }
+
+ /* token call; but token didn't exist */
+ if (!t) {
+ dev_dbg(d, "token at location %04x doesn't exist\n",
+ buffer->input[0]);
+ return -EINVAL;
+ }
+
+ /* match against token blacklist */
+ for (i = 0; i < ARRAY_SIZE(token_blacklist); i++) {
+ if (!token_blacklist[i].min || !token_blacklist[i].max)
+ continue;
+ if (t >= token_blacklist[i].min &&
+ t <= token_blacklist[i].max)
+ return -EINVAL;
+ }
+
+ /* match against token whitelist */
+ for (i = 0; i < ARRAY_SIZE(token_whitelist); i++) {
+ if (!token_whitelist[i].min || !token_whitelist[i].max)
+ continue;
+ if (t < token_whitelist[i].min ||
+ t > token_whitelist[i].max)
+ continue;
+ if (!token_whitelist[i].need_capability ||
+ capable(token_whitelist[i].need_capability)) {
+ dev_dbg(d, "whitelisted token: %x\n", t);
+ return 0;
+ }
+
+ }
+ }
+ /* match against call whitelist */
+ for (i = 0; i < ARRAY_SIZE(call_whitelist); i++) {
+ if (buffer->cmd_class != call_whitelist[i].cmd_class)
+ continue;
+ if (buffer->cmd_select != call_whitelist[i].cmd_select)
+ continue;
+ if (!call_whitelist[i].need_capability ||
+ capable(call_whitelist[i].need_capability)) {
+ dev_dbg(d, "whitelisted capable command: %u/%u\n",
+ buffer->cmd_class, buffer->cmd_select);
+ return 0;
+ }
+ dev_dbg(d, "missing capability %d for %u/%u\n",
+ call_whitelist[i].need_capability,
+ buffer->cmd_class, buffer->cmd_select);
+
+ }
+
+ /* not in a whitelist, only allow processes with capabilities */
+ if (capable(CAP_SYS_RAWIO)) {
+ dev_dbg(d, "Allowing %u/%u due to CAP_SYS_RAWIO\n",
+ buffer->cmd_class, buffer->cmd_select);
+ return 0;
+ }
+
+ return -EACCES;
+}
+EXPORT_SYMBOL_GPL(dell_smbios_call_filter);
+
+int dell_smbios_call(struct calling_interface_buffer *buffer)
+{
+ int (*call_fn)(struct calling_interface_buffer *) = NULL;
+ struct device *selected_dev = NULL;
+ struct smbios_device *priv;
+ int ret;
+
+ mutex_lock(&smbios_mutex);
+ list_for_each_entry(priv, &smbios_device_list, list) {
+ if (!selected_dev || priv->device->id >= selected_dev->id) {
+ dev_dbg(priv->device, "Trying device ID: %d\n",
+ priv->device->id);
+ call_fn = priv->call_fn;
+ selected_dev = priv->device;
+ }
+ }
+
+ if (!selected_dev) {
+ ret = -ENODEV;
+ pr_err("No dell-smbios drivers are loaded\n");
+ goto out_smbios_call;
+ }
+
+ ret = call_fn(buffer);
+
+out_smbios_call:
+ mutex_unlock(&smbios_mutex);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dell_smbios_call);
+
+struct calling_interface_token *dell_smbios_find_token(int tokenid)
+{
+ int i;
+
+ for (i = 0; i < da_num_tokens; i++) {
+ if (da_tokens[i].tokenID == tokenid)
+ return &da_tokens[i];
+ }
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(dell_smbios_find_token);
+
+static BLOCKING_NOTIFIER_HEAD(dell_laptop_chain_head);
+
+int dell_laptop_register_notifier(struct notifier_block *nb)
+{
+ return blocking_notifier_chain_register(&dell_laptop_chain_head, nb);
+}
+EXPORT_SYMBOL_GPL(dell_laptop_register_notifier);
+
+int dell_laptop_unregister_notifier(struct notifier_block *nb)
+{
+ return blocking_notifier_chain_unregister(&dell_laptop_chain_head, nb);
+}
+EXPORT_SYMBOL_GPL(dell_laptop_unregister_notifier);
+
+void dell_laptop_call_notifier(unsigned long action, void *data)
+{
+ blocking_notifier_call_chain(&dell_laptop_chain_head, action, data);
+}
+EXPORT_SYMBOL_GPL(dell_laptop_call_notifier);
+
+static void __init parse_da_table(const struct dmi_header *dm)
+{
+ /* Final token is a terminator, so we don't want to copy it */
+ int tokens = (dm->length-11)/sizeof(struct calling_interface_token)-1;
+ struct calling_interface_token *new_da_tokens;
+ struct calling_interface_structure *table =
+ container_of(dm, struct calling_interface_structure, header);
+
+ /*
+ * 4 bytes of table header, plus 7 bytes of Dell header
+ * plus at least 6 bytes of entry
+ */
+
+ if (dm->length < 17)
+ return;
+
+ da_supported_commands = table->supportedCmds;
+
+ new_da_tokens = krealloc(da_tokens, (da_num_tokens + tokens) *
+ sizeof(struct calling_interface_token),
+ GFP_KERNEL);
+
+ if (!new_da_tokens)
+ return;
+ da_tokens = new_da_tokens;
+
+ memcpy(da_tokens+da_num_tokens, table->tokens,
+ sizeof(struct calling_interface_token) * tokens);
+
+ da_num_tokens += tokens;
+}
+
+static void zero_duplicates(struct device *dev)
+{
+ int i, j;
+
+ for (i = 0; i < da_num_tokens; i++) {
+ if (da_tokens[i].tokenID == 0)
+ continue;
+ for (j = i+1; j < da_num_tokens; j++) {
+ if (da_tokens[j].tokenID == 0)
+ continue;
+ if (da_tokens[i].tokenID == da_tokens[j].tokenID) {
+ dev_dbg(dev, "Zeroing dup token ID %x(%x/%x)\n",
+ da_tokens[j].tokenID,
+ da_tokens[j].location,
+ da_tokens[j].value);
+ da_tokens[j].tokenID = 0;
+ }
+ }
+ }
+}
+
+static void __init find_tokens(const struct dmi_header *dm, void *dummy)
+{
+ switch (dm->type) {
+ case 0xd4: /* Indexed IO */
+ case 0xd5: /* Protected Area Type 1 */
+ case 0xd6: /* Protected Area Type 2 */
+ break;
+ case 0xda: /* Calling interface */
+ parse_da_table(dm);
+ break;
+ }
+}
+
+static int match_attribute(struct device *dev,
+ struct device_attribute *attr)
+{
+ int i;
+
+ for (i = 0; i < da_num_tokens * 2; i++) {
+ if (!token_attrs[i])
+ continue;
+ if (strcmp(token_attrs[i]->name, attr->attr.name) == 0)
+ return i/2;
+ }
+ dev_dbg(dev, "couldn't match: %s\n", attr->attr.name);
+ return -EINVAL;
+}
+
+static ssize_t location_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int i;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ i = match_attribute(dev, attr);
+ if (i > 0)
+ return scnprintf(buf, PAGE_SIZE, "%08x", da_tokens[i].location);
+ return 0;
+}
+
+static ssize_t value_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ int i;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ i = match_attribute(dev, attr);
+ if (i > 0)
+ return scnprintf(buf, PAGE_SIZE, "%08x", da_tokens[i].value);
+ return 0;
+}
+
+static struct attribute_group smbios_attribute_group = {
+ .name = "tokens"
+};
+
+static struct platform_driver platform_driver = {
+ .driver = {
+ .name = "dell-smbios",
+ },
+};
+
+static int build_tokens_sysfs(struct platform_device *dev)
+{
+ char *location_name;
+ char *value_name;
+ size_t size;
+ int ret;
+ int i, j;
+
+ /* (number of tokens + 1 for null terminated */
+ size = sizeof(struct device_attribute) * (da_num_tokens + 1);
+ token_location_attrs = kzalloc(size, GFP_KERNEL);
+ if (!token_location_attrs)
+ return -ENOMEM;
+ token_value_attrs = kzalloc(size, GFP_KERNEL);
+ if (!token_value_attrs)
+ goto out_allocate_value;
+
+ /* need to store both location and value + terminator*/
+ size = sizeof(struct attribute *) * ((2 * da_num_tokens) + 1);
+ token_attrs = kzalloc(size, GFP_KERNEL);
+ if (!token_attrs)
+ goto out_allocate_attrs;
+
+ for (i = 0, j = 0; i < da_num_tokens; i++) {
+ /* skip empty */
+ if (da_tokens[i].tokenID == 0)
+ continue;
+ /* add location */
+ location_name = kasprintf(GFP_KERNEL, "%04x_location",
+ da_tokens[i].tokenID);
+ if (location_name == NULL)
+ goto out_unwind_strings;
+ sysfs_attr_init(&token_location_attrs[i].attr);
+ token_location_attrs[i].attr.name = location_name;
+ token_location_attrs[i].attr.mode = 0444;
+ token_location_attrs[i].show = location_show;
+ token_attrs[j++] = &token_location_attrs[i].attr;
+
+ /* add value */
+ value_name = kasprintf(GFP_KERNEL, "%04x_value",
+ da_tokens[i].tokenID);
+ if (value_name == NULL)
+ goto loop_fail_create_value;
+ sysfs_attr_init(&token_value_attrs[i].attr);
+ token_value_attrs[i].attr.name = value_name;
+ token_value_attrs[i].attr.mode = 0444;
+ token_value_attrs[i].show = value_show;
+ token_attrs[j++] = &token_value_attrs[i].attr;
+ continue;
+
+loop_fail_create_value:
+ kfree(value_name);
+ goto out_unwind_strings;
+ }
+ smbios_attribute_group.attrs = token_attrs;
+
+ ret = sysfs_create_group(&dev->dev.kobj, &smbios_attribute_group);
+ if (ret)
+ goto out_unwind_strings;
+ return 0;
+
+out_unwind_strings:
+ for (i = i-1; i > 0; i--) {
+ kfree(token_location_attrs[i].attr.name);
+ kfree(token_value_attrs[i].attr.name);
+ }
+ kfree(token_attrs);
+out_allocate_attrs:
+ kfree(token_value_attrs);
+out_allocate_value:
+ kfree(token_location_attrs);
+
+ return -ENOMEM;
+}
+
+static void free_group(struct platform_device *pdev)
+{
+ int i;
+
+ sysfs_remove_group(&pdev->dev.kobj,
+ &smbios_attribute_group);
+ for (i = 0; i < da_num_tokens; i++) {
+ kfree(token_location_attrs[i].attr.name);
+ kfree(token_value_attrs[i].attr.name);
+ }
+ kfree(token_attrs);
+ kfree(token_value_attrs);
+ kfree(token_location_attrs);
+}
+
+static int __init dell_smbios_init(void)
+{
+ const struct dmi_device *valid;
+ int ret, wmi, smm;
+
+ valid = dmi_find_device(DMI_DEV_TYPE_OEM_STRING, "Dell System", NULL);
+ if (!valid) {
+ pr_err("Unable to run on non-Dell system\n");
+ return -ENODEV;
+ }
+
+ dmi_walk(find_tokens, NULL);
+
+ if (!da_tokens) {
+ pr_info("Unable to find dmi tokens\n");
+ return -ENODEV;
+ }
+
+ ret = platform_driver_register(&platform_driver);
+ if (ret)
+ goto fail_platform_driver;
+
+ platform_device = platform_device_alloc("dell-smbios", 0);
+ if (!platform_device) {
+ ret = -ENOMEM;
+ goto fail_platform_device_alloc;
+ }
+ ret = platform_device_add(platform_device);
+ if (ret)
+ goto fail_platform_device_add;
+
+ /* duplicate tokens will cause problems building sysfs files */
+ zero_duplicates(&platform_device->dev);
+
+ ret = build_tokens_sysfs(platform_device);
+ if (ret)
+ goto fail_create_group;
+
+ /* register backends */
+ wmi = init_dell_smbios_wmi();
+ if (wmi)
+ pr_debug("Failed to initialize WMI backend: %d\n", wmi);
+ smm = init_dell_smbios_smm();
+ if (smm)
+ pr_debug("Failed to initialize SMM backend: %d\n", smm);
+ if (wmi && smm) {
+ pr_err("No SMBIOS backends available (wmi: %d, smm: %d)\n",
+ wmi, smm);
+ goto fail_sysfs;
+ }
+
+ return 0;
+
+fail_sysfs:
+ free_group(platform_device);
+
+fail_create_group:
+ platform_device_del(platform_device);
+
+fail_platform_device_add:
+ platform_device_put(platform_device);
+
+fail_platform_device_alloc:
+ platform_driver_unregister(&platform_driver);
+
+fail_platform_driver:
+ kfree(da_tokens);
+ return ret;
+}
+
+static void __exit dell_smbios_exit(void)
+{
+ exit_dell_smbios_wmi();
+ exit_dell_smbios_smm();
+ mutex_lock(&smbios_mutex);
+ if (platform_device) {
+ free_group(platform_device);
+ platform_device_unregister(platform_device);
+ platform_driver_unregister(&platform_driver);
+ }
+ kfree(da_tokens);
+ mutex_unlock(&smbios_mutex);
+}
+
+module_init(dell_smbios_init);
+module_exit(dell_smbios_exit);
+
+MODULE_AUTHOR("Matthew Garrett <mjg@redhat.com>");
+MODULE_AUTHOR("Gabriele Mazzotta <gabriele.mzt@gmail.com>");
+MODULE_AUTHOR("Pali Rohár <pali.rohar@gmail.com>");
+MODULE_AUTHOR("Mario Limonciello <mario.limonciello@dell.com>");
+MODULE_DESCRIPTION("Common functions for kernel modules using Dell SMBIOS");
+MODULE_LICENSE("GPL");
};
MODULE_DEVICE_TABLE(dmi, dell_device_table);
-static void __init parse_da_table(const struct dmi_header *dm)
+static void parse_da_table(const struct dmi_header *dm)
{
struct calling_interface_structure *table =
container_of(dm, struct calling_interface_structure, header);
da_command_code = table->cmdIOCode;
}
-static void __init find_cmd_address(const struct dmi_header *dm, void *dummy)
+static void find_cmd_address(const struct dmi_header *dm, void *dummy)
{
switch (dm->type) {
case 0xda: /* Calling interface */
return false;
}
-static int __init dell_smbios_smm_init(void)
+int init_dell_smbios_smm(void)
{
int ret;
/*
return ret;
}
-static void __exit dell_smbios_smm_exit(void)
+void exit_dell_smbios_smm(void)
{
if (platform_device) {
dell_smbios_unregister_device(&platform_device->dev);
free_page((unsigned long)buffer);
}
}
-
-subsys_initcall(dell_smbios_smm_init);
-module_exit(dell_smbios_smm_exit);
-
-MODULE_AUTHOR("Matthew Garrett <mjg@redhat.com>");
-MODULE_AUTHOR("Gabriele Mazzotta <gabriele.mzt@gmail.com>");
-MODULE_AUTHOR("Pali Rohár <pali.rohar@gmail.com>");
-MODULE_AUTHOR("Mario Limonciello <mario.limonciello@dell.com>");
-MODULE_DESCRIPTION("Dell SMBIOS communications over SMI");
-MODULE_LICENSE("GPL");
{ },
};
-static void __init parse_b1_table(const struct dmi_header *dm)
+static void parse_b1_table(const struct dmi_header *dm)
{
struct misc_bios_flags_structure *flags =
container_of(dm, struct misc_bios_flags_structure, header);
wmi_supported = 1;
}
-static void __init find_b1(const struct dmi_header *dm, void *dummy)
+static void find_b1(const struct dmi_header *dm, void *dummy)
{
switch (dm->type) {
case 0xb1: /* misc bios flags */
.filter_callback = dell_smbios_wmi_filter,
};
-static int __init init_dell_smbios_wmi(void)
+int init_dell_smbios_wmi(void)
{
dmi_walk(find_b1, NULL);
return wmi_driver_register(&dell_smbios_wmi_driver);
}
-static void __exit exit_dell_smbios_wmi(void)
+void exit_dell_smbios_wmi(void)
{
wmi_driver_unregister(&dell_smbios_wmi_driver);
}
-module_init(init_dell_smbios_wmi);
-module_exit(exit_dell_smbios_wmi);
-
MODULE_ALIAS("wmi:" DELL_WMI_SMBIOS_GUID);
-MODULE_AUTHOR("Mario Limonciello <mario.limonciello@dell.com>");
-MODULE_DESCRIPTION("Dell SMBIOS communications over WMI");
-MODULE_LICENSE("GPL");
+++ /dev/null
-/*
- * Common functions for kernel modules using Dell SMBIOS
- *
- * Copyright (c) Red Hat <mjg@redhat.com>
- * Copyright (c) 2014 Gabriele Mazzotta <gabriele.mzt@gmail.com>
- * Copyright (c) 2014 Pali Rohár <pali.rohar@gmail.com>
- *
- * Based on documentation in the libsmbios package:
- * Copyright (C) 2005-2014 Dell Inc.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/capability.h>
-#include <linux/dmi.h>
-#include <linux/err.h>
-#include <linux/mutex.h>
-#include <linux/platform_device.h>
-#include <linux/slab.h>
-#include "dell-smbios.h"
-
-static u32 da_supported_commands;
-static int da_num_tokens;
-static struct platform_device *platform_device;
-static struct calling_interface_token *da_tokens;
-static struct device_attribute *token_location_attrs;
-static struct device_attribute *token_value_attrs;
-static struct attribute **token_attrs;
-static DEFINE_MUTEX(smbios_mutex);
-
-struct smbios_device {
- struct list_head list;
- struct device *device;
- int (*call_fn)(struct calling_interface_buffer *);
-};
-
-struct smbios_call {
- u32 need_capability;
- int cmd_class;
- int cmd_select;
-};
-
-/* calls that are whitelisted for given capabilities */
-static struct smbios_call call_whitelist[] = {
- /* generally tokens are allowed, but may be further filtered or
- * restricted by token blacklist or whitelist
- */
- {CAP_SYS_ADMIN, CLASS_TOKEN_READ, SELECT_TOKEN_STD},
- {CAP_SYS_ADMIN, CLASS_TOKEN_READ, SELECT_TOKEN_AC},
- {CAP_SYS_ADMIN, CLASS_TOKEN_READ, SELECT_TOKEN_BAT},
- {CAP_SYS_ADMIN, CLASS_TOKEN_WRITE, SELECT_TOKEN_STD},
- {CAP_SYS_ADMIN, CLASS_TOKEN_WRITE, SELECT_TOKEN_AC},
- {CAP_SYS_ADMIN, CLASS_TOKEN_WRITE, SELECT_TOKEN_BAT},
- /* used by userspace: fwupdate */
- {CAP_SYS_ADMIN, CLASS_ADMIN_PROP, SELECT_ADMIN_PROP},
- /* used by userspace: fwupd */
- {CAP_SYS_ADMIN, CLASS_INFO, SELECT_DOCK},
- {CAP_SYS_ADMIN, CLASS_FLASH_INTERFACE, SELECT_FLASH_INTERFACE},
-};
-
-/* calls that are explicitly blacklisted */
-static struct smbios_call call_blacklist[] = {
- {0x0000, 1, 7}, /* manufacturing use */
- {0x0000, 6, 5}, /* manufacturing use */
- {0x0000, 11, 3}, /* write once */
- {0x0000, 11, 7}, /* write once */
- {0x0000, 11, 11}, /* write once */
- {0x0000, 19, -1}, /* diagnostics */
- /* handled by kernel: dell-laptop */
- {0x0000, CLASS_INFO, SELECT_RFKILL},
- {0x0000, CLASS_KBD_BACKLIGHT, SELECT_KBD_BACKLIGHT},
-};
-
-struct token_range {
- u32 need_capability;
- u16 min;
- u16 max;
-};
-
-/* tokens that are whitelisted for given capabilities */
-static struct token_range token_whitelist[] = {
- /* used by userspace: fwupdate */
- {CAP_SYS_ADMIN, CAPSULE_EN_TOKEN, CAPSULE_DIS_TOKEN},
- /* can indicate to userspace that WMI is needed */
- {0x0000, WSMT_EN_TOKEN, WSMT_DIS_TOKEN}
-};
-
-/* tokens that are explicitly blacklisted */
-static struct token_range token_blacklist[] = {
- {0x0000, 0x0058, 0x0059}, /* ME use */
- {0x0000, 0x00CD, 0x00D0}, /* raid shadow copy */
- {0x0000, 0x013A, 0x01FF}, /* sata shadow copy */
- {0x0000, 0x0175, 0x0176}, /* write once */
- {0x0000, 0x0195, 0x0197}, /* diagnostics */
- {0x0000, 0x01DC, 0x01DD}, /* manufacturing use */
- {0x0000, 0x027D, 0x0284}, /* diagnostics */
- {0x0000, 0x02E3, 0x02E3}, /* manufacturing use */
- {0x0000, 0x02FF, 0x02FF}, /* manufacturing use */
- {0x0000, 0x0300, 0x0302}, /* manufacturing use */
- {0x0000, 0x0325, 0x0326}, /* manufacturing use */
- {0x0000, 0x0332, 0x0335}, /* fan control */
- {0x0000, 0x0350, 0x0350}, /* manufacturing use */
- {0x0000, 0x0363, 0x0363}, /* manufacturing use */
- {0x0000, 0x0368, 0x0368}, /* manufacturing use */
- {0x0000, 0x03F6, 0x03F7}, /* manufacturing use */
- {0x0000, 0x049E, 0x049F}, /* manufacturing use */
- {0x0000, 0x04A0, 0x04A3}, /* disagnostics */
- {0x0000, 0x04E6, 0x04E7}, /* manufacturing use */
- {0x0000, 0x4000, 0x7FFF}, /* internal BIOS use */
- {0x0000, 0x9000, 0x9001}, /* internal BIOS use */
- {0x0000, 0xA000, 0xBFFF}, /* write only */
- {0x0000, 0xEFF0, 0xEFFF}, /* internal BIOS use */
- /* handled by kernel: dell-laptop */
- {0x0000, BRIGHTNESS_TOKEN, BRIGHTNESS_TOKEN},
- {0x0000, KBD_LED_OFF_TOKEN, KBD_LED_AUTO_TOKEN},
- {0x0000, KBD_LED_AC_TOKEN, KBD_LED_AC_TOKEN},
- {0x0000, KBD_LED_AUTO_25_TOKEN, KBD_LED_AUTO_75_TOKEN},
- {0x0000, KBD_LED_AUTO_100_TOKEN, KBD_LED_AUTO_100_TOKEN},
- {0x0000, GLOBAL_MIC_MUTE_ENABLE, GLOBAL_MIC_MUTE_DISABLE},
-};
-
-static LIST_HEAD(smbios_device_list);
-
-int dell_smbios_error(int value)
-{
- switch (value) {
- case 0: /* Completed successfully */
- return 0;
- case -1: /* Completed with error */
- return -EIO;
- case -2: /* Function not supported */
- return -ENXIO;
- default: /* Unknown error */
- return -EINVAL;
- }
-}
-EXPORT_SYMBOL_GPL(dell_smbios_error);
-
-int dell_smbios_register_device(struct device *d, void *call_fn)
-{
- struct smbios_device *priv;
-
- priv = devm_kzalloc(d, sizeof(struct smbios_device), GFP_KERNEL);
- if (!priv)
- return -ENOMEM;
- get_device(d);
- priv->device = d;
- priv->call_fn = call_fn;
- mutex_lock(&smbios_mutex);
- list_add_tail(&priv->list, &smbios_device_list);
- mutex_unlock(&smbios_mutex);
- dev_dbg(d, "Added device: %s\n", d->driver->name);
- return 0;
-}
-EXPORT_SYMBOL_GPL(dell_smbios_register_device);
-
-void dell_smbios_unregister_device(struct device *d)
-{
- struct smbios_device *priv;
-
- mutex_lock(&smbios_mutex);
- list_for_each_entry(priv, &smbios_device_list, list) {
- if (priv->device == d) {
- list_del(&priv->list);
- put_device(d);
- break;
- }
- }
- mutex_unlock(&smbios_mutex);
- dev_dbg(d, "Remove device: %s\n", d->driver->name);
-}
-EXPORT_SYMBOL_GPL(dell_smbios_unregister_device);
-
-int dell_smbios_call_filter(struct device *d,
- struct calling_interface_buffer *buffer)
-{
- u16 t = 0;
- int i;
-
- /* can't make calls over 30 */
- if (buffer->cmd_class > 30) {
- dev_dbg(d, "class too big: %u\n", buffer->cmd_class);
- return -EINVAL;
- }
-
- /* supported calls on the particular system */
- if (!(da_supported_commands & (1 << buffer->cmd_class))) {
- dev_dbg(d, "invalid command, supported commands: 0x%8x\n",
- da_supported_commands);
- return -EINVAL;
- }
-
- /* match against call blacklist */
- for (i = 0; i < ARRAY_SIZE(call_blacklist); i++) {
- if (buffer->cmd_class != call_blacklist[i].cmd_class)
- continue;
- if (buffer->cmd_select != call_blacklist[i].cmd_select &&
- call_blacklist[i].cmd_select != -1)
- continue;
- dev_dbg(d, "blacklisted command: %u/%u\n",
- buffer->cmd_class, buffer->cmd_select);
- return -EINVAL;
- }
-
- /* if a token call, find token ID */
-
- if ((buffer->cmd_class == CLASS_TOKEN_READ ||
- buffer->cmd_class == CLASS_TOKEN_WRITE) &&
- buffer->cmd_select < 3) {
- /* find the matching token ID */
- for (i = 0; i < da_num_tokens; i++) {
- if (da_tokens[i].location != buffer->input[0])
- continue;
- t = da_tokens[i].tokenID;
- break;
- }
-
- /* token call; but token didn't exist */
- if (!t) {
- dev_dbg(d, "token at location %04x doesn't exist\n",
- buffer->input[0]);
- return -EINVAL;
- }
-
- /* match against token blacklist */
- for (i = 0; i < ARRAY_SIZE(token_blacklist); i++) {
- if (!token_blacklist[i].min || !token_blacklist[i].max)
- continue;
- if (t >= token_blacklist[i].min &&
- t <= token_blacklist[i].max)
- return -EINVAL;
- }
-
- /* match against token whitelist */
- for (i = 0; i < ARRAY_SIZE(token_whitelist); i++) {
- if (!token_whitelist[i].min || !token_whitelist[i].max)
- continue;
- if (t < token_whitelist[i].min ||
- t > token_whitelist[i].max)
- continue;
- if (!token_whitelist[i].need_capability ||
- capable(token_whitelist[i].need_capability)) {
- dev_dbg(d, "whitelisted token: %x\n", t);
- return 0;
- }
-
- }
- }
- /* match against call whitelist */
- for (i = 0; i < ARRAY_SIZE(call_whitelist); i++) {
- if (buffer->cmd_class != call_whitelist[i].cmd_class)
- continue;
- if (buffer->cmd_select != call_whitelist[i].cmd_select)
- continue;
- if (!call_whitelist[i].need_capability ||
- capable(call_whitelist[i].need_capability)) {
- dev_dbg(d, "whitelisted capable command: %u/%u\n",
- buffer->cmd_class, buffer->cmd_select);
- return 0;
- }
- dev_dbg(d, "missing capability %d for %u/%u\n",
- call_whitelist[i].need_capability,
- buffer->cmd_class, buffer->cmd_select);
-
- }
-
- /* not in a whitelist, only allow processes with capabilities */
- if (capable(CAP_SYS_RAWIO)) {
- dev_dbg(d, "Allowing %u/%u due to CAP_SYS_RAWIO\n",
- buffer->cmd_class, buffer->cmd_select);
- return 0;
- }
-
- return -EACCES;
-}
-EXPORT_SYMBOL_GPL(dell_smbios_call_filter);
-
-int dell_smbios_call(struct calling_interface_buffer *buffer)
-{
- int (*call_fn)(struct calling_interface_buffer *) = NULL;
- struct device *selected_dev = NULL;
- struct smbios_device *priv;
- int ret;
-
- mutex_lock(&smbios_mutex);
- list_for_each_entry(priv, &smbios_device_list, list) {
- if (!selected_dev || priv->device->id >= selected_dev->id) {
- dev_dbg(priv->device, "Trying device ID: %d\n",
- priv->device->id);
- call_fn = priv->call_fn;
- selected_dev = priv->device;
- }
- }
-
- if (!selected_dev) {
- ret = -ENODEV;
- pr_err("No dell-smbios drivers are loaded\n");
- goto out_smbios_call;
- }
-
- ret = call_fn(buffer);
-
-out_smbios_call:
- mutex_unlock(&smbios_mutex);
- return ret;
-}
-EXPORT_SYMBOL_GPL(dell_smbios_call);
-
-struct calling_interface_token *dell_smbios_find_token(int tokenid)
-{
- int i;
-
- for (i = 0; i < da_num_tokens; i++) {
- if (da_tokens[i].tokenID == tokenid)
- return &da_tokens[i];
- }
-
- return NULL;
-}
-EXPORT_SYMBOL_GPL(dell_smbios_find_token);
-
-static BLOCKING_NOTIFIER_HEAD(dell_laptop_chain_head);
-
-int dell_laptop_register_notifier(struct notifier_block *nb)
-{
- return blocking_notifier_chain_register(&dell_laptop_chain_head, nb);
-}
-EXPORT_SYMBOL_GPL(dell_laptop_register_notifier);
-
-int dell_laptop_unregister_notifier(struct notifier_block *nb)
-{
- return blocking_notifier_chain_unregister(&dell_laptop_chain_head, nb);
-}
-EXPORT_SYMBOL_GPL(dell_laptop_unregister_notifier);
-
-void dell_laptop_call_notifier(unsigned long action, void *data)
-{
- blocking_notifier_call_chain(&dell_laptop_chain_head, action, data);
-}
-EXPORT_SYMBOL_GPL(dell_laptop_call_notifier);
-
-static void __init parse_da_table(const struct dmi_header *dm)
-{
- /* Final token is a terminator, so we don't want to copy it */
- int tokens = (dm->length-11)/sizeof(struct calling_interface_token)-1;
- struct calling_interface_token *new_da_tokens;
- struct calling_interface_structure *table =
- container_of(dm, struct calling_interface_structure, header);
-
- /* 4 bytes of table header, plus 7 bytes of Dell header, plus at least
- 6 bytes of entry */
-
- if (dm->length < 17)
- return;
-
- da_supported_commands = table->supportedCmds;
-
- new_da_tokens = krealloc(da_tokens, (da_num_tokens + tokens) *
- sizeof(struct calling_interface_token),
- GFP_KERNEL);
-
- if (!new_da_tokens)
- return;
- da_tokens = new_da_tokens;
-
- memcpy(da_tokens+da_num_tokens, table->tokens,
- sizeof(struct calling_interface_token) * tokens);
-
- da_num_tokens += tokens;
-}
-
-static void zero_duplicates(struct device *dev)
-{
- int i, j;
-
- for (i = 0; i < da_num_tokens; i++) {
- if (da_tokens[i].tokenID == 0)
- continue;
- for (j = i+1; j < da_num_tokens; j++) {
- if (da_tokens[j].tokenID == 0)
- continue;
- if (da_tokens[i].tokenID == da_tokens[j].tokenID) {
- dev_dbg(dev, "Zeroing dup token ID %x(%x/%x)\n",
- da_tokens[j].tokenID,
- da_tokens[j].location,
- da_tokens[j].value);
- da_tokens[j].tokenID = 0;
- }
- }
- }
-}
-
-static void __init find_tokens(const struct dmi_header *dm, void *dummy)
-{
- switch (dm->type) {
- case 0xd4: /* Indexed IO */
- case 0xd5: /* Protected Area Type 1 */
- case 0xd6: /* Protected Area Type 2 */
- break;
- case 0xda: /* Calling interface */
- parse_da_table(dm);
- break;
- }
-}
-
-static int match_attribute(struct device *dev,
- struct device_attribute *attr)
-{
- int i;
-
- for (i = 0; i < da_num_tokens * 2; i++) {
- if (!token_attrs[i])
- continue;
- if (strcmp(token_attrs[i]->name, attr->attr.name) == 0)
- return i/2;
- }
- dev_dbg(dev, "couldn't match: %s\n", attr->attr.name);
- return -EINVAL;
-}
-
-static ssize_t location_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- int i;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- i = match_attribute(dev, attr);
- if (i > 0)
- return scnprintf(buf, PAGE_SIZE, "%08x", da_tokens[i].location);
- return 0;
-}
-
-static ssize_t value_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- int i;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- i = match_attribute(dev, attr);
- if (i > 0)
- return scnprintf(buf, PAGE_SIZE, "%08x", da_tokens[i].value);
- return 0;
-}
-
-static struct attribute_group smbios_attribute_group = {
- .name = "tokens"
-};
-
-static struct platform_driver platform_driver = {
- .driver = {
- .name = "dell-smbios",
- },
-};
-
-static int build_tokens_sysfs(struct platform_device *dev)
-{
- char *location_name;
- char *value_name;
- size_t size;
- int ret;
- int i, j;
-
- /* (number of tokens + 1 for null terminated */
- size = sizeof(struct device_attribute) * (da_num_tokens + 1);
- token_location_attrs = kzalloc(size, GFP_KERNEL);
- if (!token_location_attrs)
- return -ENOMEM;
- token_value_attrs = kzalloc(size, GFP_KERNEL);
- if (!token_value_attrs)
- goto out_allocate_value;
-
- /* need to store both location and value + terminator*/
- size = sizeof(struct attribute *) * ((2 * da_num_tokens) + 1);
- token_attrs = kzalloc(size, GFP_KERNEL);
- if (!token_attrs)
- goto out_allocate_attrs;
-
- for (i = 0, j = 0; i < da_num_tokens; i++) {
- /* skip empty */
- if (da_tokens[i].tokenID == 0)
- continue;
- /* add location */
- location_name = kasprintf(GFP_KERNEL, "%04x_location",
- da_tokens[i].tokenID);
- if (location_name == NULL)
- goto out_unwind_strings;
- sysfs_attr_init(&token_location_attrs[i].attr);
- token_location_attrs[i].attr.name = location_name;
- token_location_attrs[i].attr.mode = 0444;
- token_location_attrs[i].show = location_show;
- token_attrs[j++] = &token_location_attrs[i].attr;
-
- /* add value */
- value_name = kasprintf(GFP_KERNEL, "%04x_value",
- da_tokens[i].tokenID);
- if (value_name == NULL)
- goto loop_fail_create_value;
- sysfs_attr_init(&token_value_attrs[i].attr);
- token_value_attrs[i].attr.name = value_name;
- token_value_attrs[i].attr.mode = 0444;
- token_value_attrs[i].show = value_show;
- token_attrs[j++] = &token_value_attrs[i].attr;
- continue;
-
-loop_fail_create_value:
- kfree(value_name);
- goto out_unwind_strings;
- }
- smbios_attribute_group.attrs = token_attrs;
-
- ret = sysfs_create_group(&dev->dev.kobj, &smbios_attribute_group);
- if (ret)
- goto out_unwind_strings;
- return 0;
-
-out_unwind_strings:
- for (i = i-1; i > 0; i--) {
- kfree(token_location_attrs[i].attr.name);
- kfree(token_value_attrs[i].attr.name);
- }
- kfree(token_attrs);
-out_allocate_attrs:
- kfree(token_value_attrs);
-out_allocate_value:
- kfree(token_location_attrs);
-
- return -ENOMEM;
-}
-
-static void free_group(struct platform_device *pdev)
-{
- int i;
-
- sysfs_remove_group(&pdev->dev.kobj,
- &smbios_attribute_group);
- for (i = 0; i < da_num_tokens; i++) {
- kfree(token_location_attrs[i].attr.name);
- kfree(token_value_attrs[i].attr.name);
- }
- kfree(token_attrs);
- kfree(token_value_attrs);
- kfree(token_location_attrs);
-}
-
-static int __init dell_smbios_init(void)
-{
- const struct dmi_device *valid;
- int ret;
-
- valid = dmi_find_device(DMI_DEV_TYPE_OEM_STRING, "Dell System", NULL);
- if (!valid) {
- pr_err("Unable to run on non-Dell system\n");
- return -ENODEV;
- }
-
- dmi_walk(find_tokens, NULL);
-
- if (!da_tokens) {
- pr_info("Unable to find dmi tokens\n");
- return -ENODEV;
- }
-
- ret = platform_driver_register(&platform_driver);
- if (ret)
- goto fail_platform_driver;
-
- platform_device = platform_device_alloc("dell-smbios", 0);
- if (!platform_device) {
- ret = -ENOMEM;
- goto fail_platform_device_alloc;
- }
- ret = platform_device_add(platform_device);
- if (ret)
- goto fail_platform_device_add;
-
- /* duplicate tokens will cause problems building sysfs files */
- zero_duplicates(&platform_device->dev);
-
- ret = build_tokens_sysfs(platform_device);
- if (ret)
- goto fail_create_group;
-
- return 0;
-
-fail_create_group:
- platform_device_del(platform_device);
-
-fail_platform_device_add:
- platform_device_put(platform_device);
-
-fail_platform_device_alloc:
- platform_driver_unregister(&platform_driver);
-
-fail_platform_driver:
- kfree(da_tokens);
- return ret;
-}
-
-static void __exit dell_smbios_exit(void)
-{
- mutex_lock(&smbios_mutex);
- if (platform_device) {
- free_group(platform_device);
- platform_device_unregister(platform_device);
- platform_driver_unregister(&platform_driver);
- }
- kfree(da_tokens);
- mutex_unlock(&smbios_mutex);
-}
-
-subsys_initcall(dell_smbios_init);
-module_exit(dell_smbios_exit);
-
-MODULE_AUTHOR("Matthew Garrett <mjg@redhat.com>");
-MODULE_AUTHOR("Gabriele Mazzotta <gabriele.mzt@gmail.com>");
-MODULE_AUTHOR("Pali Rohár <pali.rohar@gmail.com>");
-MODULE_DESCRIPTION("Common functions for kernel modules using Dell SMBIOS");
-MODULE_LICENSE("GPL");
int dell_laptop_unregister_notifier(struct notifier_block *nb);
void dell_laptop_call_notifier(unsigned long action, void *data);
-#endif
+/* for the supported backends */
+#ifdef CONFIG_DELL_SMBIOS_WMI
+int init_dell_smbios_wmi(void);
+void exit_dell_smbios_wmi(void);
+#else /* CONFIG_DELL_SMBIOS_WMI */
+static inline int init_dell_smbios_wmi(void)
+{
+ return -ENODEV;
+}
+static inline void exit_dell_smbios_wmi(void)
+{}
+#endif /* CONFIG_DELL_SMBIOS_WMI */
+
+#ifdef CONFIG_DELL_SMBIOS_SMM
+int init_dell_smbios_smm(void);
+void exit_dell_smbios_smm(void);
+#else /* CONFIG_DELL_SMBIOS_SMM */
+static inline int init_dell_smbios_smm(void)
+{
+ return -ENODEV;
+}
+static inline void exit_dell_smbios_smm(void)
+{}
+#endif /* CONFIG_DELL_SMBIOS_SMM */
+
+#endif /* _DELL_SMBIOS_H_ */
return wmi_driver_register(&dell_wmi_driver);
}
-module_init(dell_wmi_init);
+late_initcall(dell_wmi_init);
static void __exit dell_wmi_exit(void)
{
rstate = regulator_get_suspend_state(rdev, *state);
if (rstate == NULL)
- return -EINVAL;
+ return 0;
mutex_lock(&rdev->mutex);
* arbitrary timeout.
*/
ret = readl_poll_timeout(priv->base + STM32_VREFBUF_CSR, val,
- !(val & STM32_VRR), 650, 10000);
+ val & STM32_VRR, 650, 10000);
if (ret) {
dev_err(&rdev->dev, "stm32 vrefbuf timed out!\n");
val = readl_relaxed(priv->base + STM32_VREFBUF_CSR);
case DASD_CQR_QUEUED:
/* request was not started - just set to cleared */
cqr->status = DASD_CQR_CLEARED;
- if (cqr->callback_data == DASD_SLEEPON_START_TAG)
- cqr->callback_data = DASD_SLEEPON_END_TAG;
break;
case DASD_CQR_IN_IO:
/* request in IO - terminate IO and release again */
wait_event(dasd_flush_wq,
(cqr->status != DASD_CQR_CLEAR_PENDING));
- /* mark sleepon requests as ended */
- if (cqr->callback_data == DASD_SLEEPON_START_TAG)
- cqr->callback_data = DASD_SLEEPON_END_TAG;
+ /*
+ * requeue requests to blocklayer will only work
+ * for block device requests
+ */
+ if (_dasd_requeue_request(cqr))
+ continue;
/* remove requests from device and block queue */
list_del_init(&cqr->devlist);
cqr = refers;
}
- /*
- * requeue requests to blocklayer will only work
- * for block device requests
- */
- if (_dasd_requeue_request(cqr))
- continue;
-
if (cqr->block)
list_del_init(&cqr->blocklist);
cqr->block->base->discipline->free_cp(
list_splice_tail(&requeue_queue, &device->ccw_queue);
spin_unlock_irq(get_ccwdev_lock(device->cdev));
}
- /* wake up generic waitqueue for eventually ended sleepon requests */
- wake_up(&generic_waitq);
+ dasd_schedule_device_bh(device);
return rc;
}
ccw_device_set_timeout(cdev, 0);
cdev->private->iretry = 255;
+ cdev->private->async_kill_io_rc = -ETIMEDOUT;
ret = ccw_device_cancel_halt_clear(cdev);
if (ret == -EBUSY) {
ccw_device_set_timeout(cdev, 3*HZ);
/* OK, i/o is dead now. Call interrupt handler. */
if (cdev->handler)
cdev->handler(cdev, cdev->private->intparm,
- ERR_PTR(-EIO));
+ ERR_PTR(cdev->private->async_kill_io_rc));
}
static void
ccw_device_online_verify(cdev, 0);
if (cdev->handler)
cdev->handler(cdev, cdev->private->intparm,
- ERR_PTR(-EIO));
+ ERR_PTR(cdev->private->async_kill_io_rc));
}
void ccw_device_kill_io(struct ccw_device *cdev)
{
int ret;
+ ccw_device_set_timeout(cdev, 0);
cdev->private->iretry = 255;
+ cdev->private->async_kill_io_rc = -EIO;
ret = ccw_device_cancel_halt_clear(cdev);
if (ret == -EBUSY) {
ccw_device_set_timeout(cdev, 3*HZ);
}
/**
- * ccw_device_start_key() - start a s390 channel program with key
+ * ccw_device_start_timeout_key() - start a s390 channel program with timeout and key
* @cdev: target ccw device
* @cpa: logical start address of channel program
* @intparm: user specific interruption parameter; will be presented back to
* @key: storage key to be used for the I/O
* @flags: additional flags; defines the action to be performed for I/O
* processing.
+ * @expires: timeout value in jiffies
*
* Start a S/390 channel program. When the interrupt arrives, the
* IRQ handler is called, either immediately, delayed (dev-end missing,
* or sense required) or never (no IRQ handler registered).
+ * This function notifies the device driver if the channel program has not
+ * completed during the time specified by @expires. If a timeout occurs, the
+ * channel program is terminated via xsch, hsch or csch, and the device's
+ * interrupt handler will be called with an irb containing ERR_PTR(-%ETIMEDOUT).
* Returns:
* %0, if the operation was successful;
* -%EBUSY, if the device is busy, or status pending;
* Context:
* Interrupts disabled, ccw device lock held
*/
-int ccw_device_start_key(struct ccw_device *cdev, struct ccw1 *cpa,
- unsigned long intparm, __u8 lpm, __u8 key,
- unsigned long flags)
+int ccw_device_start_timeout_key(struct ccw_device *cdev, struct ccw1 *cpa,
+ unsigned long intparm, __u8 lpm, __u8 key,
+ unsigned long flags, int expires)
{
struct subchannel *sch;
int ret;
switch (ret) {
case 0:
cdev->private->intparm = intparm;
+ if (expires)
+ ccw_device_set_timeout(cdev, expires);
break;
case -EACCES:
case -ENODEV:
}
/**
- * ccw_device_start_timeout_key() - start a s390 channel program with timeout and key
+ * ccw_device_start_key() - start a s390 channel program with key
* @cdev: target ccw device
* @cpa: logical start address of channel program
* @intparm: user specific interruption parameter; will be presented back to
* @key: storage key to be used for the I/O
* @flags: additional flags; defines the action to be performed for I/O
* processing.
- * @expires: timeout value in jiffies
*
* Start a S/390 channel program. When the interrupt arrives, the
* IRQ handler is called, either immediately, delayed (dev-end missing,
* or sense required) or never (no IRQ handler registered).
- * This function notifies the device driver if the channel program has not
- * completed during the time specified by @expires. If a timeout occurs, the
- * channel program is terminated via xsch, hsch or csch, and the device's
- * interrupt handler will be called with an irb containing ERR_PTR(-%ETIMEDOUT).
* Returns:
* %0, if the operation was successful;
* -%EBUSY, if the device is busy, or status pending;
* Context:
* Interrupts disabled, ccw device lock held
*/
-int ccw_device_start_timeout_key(struct ccw_device *cdev, struct ccw1 *cpa,
- unsigned long intparm, __u8 lpm, __u8 key,
- unsigned long flags, int expires)
+int ccw_device_start_key(struct ccw_device *cdev, struct ccw1 *cpa,
+ unsigned long intparm, __u8 lpm, __u8 key,
+ unsigned long flags)
{
- int ret;
-
- if (!cdev)
- return -ENODEV;
- ccw_device_set_timeout(cdev, expires);
- ret = ccw_device_start_key(cdev, cpa, intparm, lpm, key, flags);
- if (ret != 0)
- ccw_device_set_timeout(cdev, 0);
- return ret;
+ return ccw_device_start_timeout_key(cdev, cpa, intparm, lpm, key,
+ flags, 0);
}
/**
EXPORT_SYMBOL(ccw_device_get_id);
/**
- * ccw_device_tm_start_key() - perform start function
+ * ccw_device_tm_start_timeout_key() - perform start function
* @cdev: ccw device on which to perform the start function
* @tcw: transport-command word to be started
* @intparm: user defined parameter to be passed to the interrupt handler
* @lpm: mask of paths to use
* @key: storage key to use for storage access
+ * @expires: time span in jiffies after which to abort request
*
* Start the tcw on the given ccw device. Return zero on success, non-zero
* otherwise.
*/
-int ccw_device_tm_start_key(struct ccw_device *cdev, struct tcw *tcw,
- unsigned long intparm, u8 lpm, u8 key)
+int ccw_device_tm_start_timeout_key(struct ccw_device *cdev, struct tcw *tcw,
+ unsigned long intparm, u8 lpm, u8 key,
+ int expires)
{
struct subchannel *sch;
int rc;
return -EACCES;
}
rc = cio_tm_start_key(sch, tcw, lpm, key);
- if (rc == 0)
+ if (rc == 0) {
cdev->private->intparm = intparm;
+ if (expires)
+ ccw_device_set_timeout(cdev, expires);
+ }
return rc;
}
-EXPORT_SYMBOL(ccw_device_tm_start_key);
+EXPORT_SYMBOL(ccw_device_tm_start_timeout_key);
/**
- * ccw_device_tm_start_timeout_key() - perform start function
+ * ccw_device_tm_start_key() - perform start function
* @cdev: ccw device on which to perform the start function
* @tcw: transport-command word to be started
* @intparm: user defined parameter to be passed to the interrupt handler
* @lpm: mask of paths to use
* @key: storage key to use for storage access
- * @expires: time span in jiffies after which to abort request
*
* Start the tcw on the given ccw device. Return zero on success, non-zero
* otherwise.
*/
-int ccw_device_tm_start_timeout_key(struct ccw_device *cdev, struct tcw *tcw,
- unsigned long intparm, u8 lpm, u8 key,
- int expires)
+int ccw_device_tm_start_key(struct ccw_device *cdev, struct tcw *tcw,
+ unsigned long intparm, u8 lpm, u8 key)
{
- int ret;
-
- ccw_device_set_timeout(cdev, expires);
- ret = ccw_device_tm_start_key(cdev, tcw, intparm, lpm, key);
- if (ret != 0)
- ccw_device_set_timeout(cdev, 0);
- return ret;
+ return ccw_device_tm_start_timeout_key(cdev, tcw, intparm, lpm, key, 0);
}
-EXPORT_SYMBOL(ccw_device_tm_start_timeout_key);
+EXPORT_SYMBOL(ccw_device_tm_start_key);
/**
* ccw_device_tm_start() - perform start function
unsigned long intparm; /* user interruption parameter */
struct qdio_irq *qdio_data;
struct irb irb; /* device status */
+ int async_kill_io_rc;
struct senseid senseid; /* SenseID info */
struct pgid pgid[8]; /* path group IDs per chpid*/
struct ccw1 iccws[2]; /* ccws for SNID/SID/SPGID commands */
queue == card->qdio.no_in_queues - 1;
}
-
-static int qeth_issue_next_read(struct qeth_card *card)
+static int __qeth_issue_next_read(struct qeth_card *card)
{
int rc;
struct qeth_cmd_buffer *iob;
return rc;
}
+static int qeth_issue_next_read(struct qeth_card *card)
+{
+ int ret;
+
+ spin_lock_irq(get_ccwdev_lock(CARD_RDEV(card)));
+ ret = __qeth_issue_next_read(card);
+ spin_unlock_irq(get_ccwdev_lock(CARD_RDEV(card)));
+
+ return ret;
+}
+
static struct qeth_reply *qeth_alloc_reply(struct qeth_card *card)
{
struct qeth_reply *reply;
spin_lock_irqsave(&card->thread_mask_lock, flags);
card->thread_running_mask &= ~thread;
spin_unlock_irqrestore(&card->thread_mask_lock, flags);
- wake_up(&card->wait_q);
+ wake_up_all(&card->wait_q);
}
EXPORT_SYMBOL_GPL(qeth_clear_thread_running_bit);
}
rc = qeth_get_problem(cdev, irb);
if (rc) {
+ card->read_or_write_problem = 1;
qeth_clear_ipacmd_list(card);
qeth_schedule_recovery(card);
goto out;
return;
if (channel == &card->read &&
channel->state == CH_STATE_UP)
- qeth_issue_next_read(card);
+ __qeth_issue_next_read(card);
iob = channel->iob;
index = channel->buf_no;
QETH_DBF_HEX(SETUP, 2, &card, sizeof(void *));
qeth_clean_channel(&card->read);
qeth_clean_channel(&card->write);
- if (card->dev)
- free_netdev(card->dev);
qeth_free_qdio_buffers(card);
unregister_service_level(&card->qeth_service_level);
kfree(card);
qeth_l2_set_offline(cgdev);
if (card->dev) {
- netif_napi_del(&card->napi);
unregister_netdev(card->dev);
+ free_netdev(card->dev);
card->dev = NULL;
}
return;
qeth_l3_set_offline(cgdev);
if (card->dev) {
- netif_napi_del(&card->napi);
unregister_netdev(card->dev);
+ free_netdev(card->dev);
card->dev = NULL;
}
if (shost->work_q)
destroy_workqueue(shost->work_q);
- destroy_rcu_head(&shost->rcu);
-
if (shost->shost_state == SHOST_CREATED) {
/*
* Free the shost_dev device name here if scsi_host_alloc()
INIT_LIST_HEAD(&shost->starved_list);
init_waitqueue_head(&shost->host_wait);
mutex_init(&shost->scan_mutex);
- init_rcu_head(&shost->rcu);
index = ida_simple_get(&host_index_ida, 0, 0, GFP_KERNEL);
if (index < 0)
static void sas_eh_finish_cmd(struct scsi_cmnd *cmd)
{
struct sas_ha_struct *sas_ha = SHOST_TO_SAS_HA(cmd->device->host);
+ struct domain_device *dev = cmd_to_domain_dev(cmd);
struct sas_task *task = TO_SAS_TASK(cmd);
/* At this point, we only get called following an actual abort
*/
sas_end_task(cmd, task);
+ if (dev_is_sata(dev)) {
+ /* defer commands to libata so that libata EH can
+ * handle ata qcs correctly
+ */
+ list_move_tail(&cmd->eh_entry, &sas_ha->eh_ata_q);
+ return;
+ }
+
/* now finish the command and move it on to the error
* handler done list, this also takes it off the
* error handler pending list.
scsi_eh_finish_cmd(cmd, &sas_ha->eh_done_q);
}
-static void sas_eh_defer_cmd(struct scsi_cmnd *cmd)
-{
- struct domain_device *dev = cmd_to_domain_dev(cmd);
- struct sas_ha_struct *ha = dev->port->ha;
- struct sas_task *task = TO_SAS_TASK(cmd);
-
- if (!dev_is_sata(dev)) {
- sas_eh_finish_cmd(cmd);
- return;
- }
-
- /* report the timeout to libata */
- sas_end_task(cmd, task);
- list_move_tail(&cmd->eh_entry, &ha->eh_ata_q);
-}
-
static void sas_scsi_clear_queue_lu(struct list_head *error_q, struct scsi_cmnd *my_cmd)
{
struct scsi_cmnd *cmd, *n;
list_for_each_entry_safe(cmd, n, error_q, eh_entry) {
if (cmd->device->sdev_target == my_cmd->device->sdev_target &&
cmd->device->lun == my_cmd->device->lun)
- sas_eh_defer_cmd(cmd);
+ sas_eh_finish_cmd(cmd);
}
}
case TASK_IS_DONE:
SAS_DPRINTK("%s: task 0x%p is done\n", __func__,
task);
- sas_eh_defer_cmd(cmd);
+ sas_eh_finish_cmd(cmd);
continue;
case TASK_IS_ABORTED:
SAS_DPRINTK("%s: task 0x%p is aborted\n",
__func__, task);
- sas_eh_defer_cmd(cmd);
+ sas_eh_finish_cmd(cmd);
continue;
case TASK_IS_AT_LU:
SAS_DPRINTK("task 0x%p is at LU: lu recover\n", task);
"recovered\n",
SAS_ADDR(task->dev),
cmd->device->lun);
- sas_eh_defer_cmd(cmd);
+ sas_eh_finish_cmd(cmd);
sas_scsi_clear_queue_lu(work_q, cmd);
goto Again;
}
/**
* megasas_fire_cmd_fusion - Sends command to the FW
* @instance: Adapter soft state
- * @req_desc: 32bit or 64bit Request descriptor
+ * @req_desc: 64bit Request descriptor
*
- * Perform PCI Write. Ventura supports 32 bit Descriptor.
- * Prior to Ventura (12G) MR controller supports 64 bit Descriptor.
+ * Perform PCI Write.
*/
static void
megasas_fire_cmd_fusion(struct megasas_instance *instance,
union MEGASAS_REQUEST_DESCRIPTOR_UNION *req_desc)
{
- if (instance->adapter_type == VENTURA_SERIES)
- writel(le32_to_cpu(req_desc->u.low),
- &instance->reg_set->inbound_single_queue_port);
- else {
#if defined(writeq) && defined(CONFIG_64BIT)
- u64 req_data = (((u64)le32_to_cpu(req_desc->u.high) << 32) |
- le32_to_cpu(req_desc->u.low));
+ u64 req_data = (((u64)le32_to_cpu(req_desc->u.high) << 32) |
+ le32_to_cpu(req_desc->u.low));
- writeq(req_data, &instance->reg_set->inbound_low_queue_port);
+ writeq(req_data, &instance->reg_set->inbound_low_queue_port);
#else
- unsigned long flags;
- spin_lock_irqsave(&instance->hba_lock, flags);
- writel(le32_to_cpu(req_desc->u.low),
- &instance->reg_set->inbound_low_queue_port);
- writel(le32_to_cpu(req_desc->u.high),
- &instance->reg_set->inbound_high_queue_port);
- mmiowb();
- spin_unlock_irqrestore(&instance->hba_lock, flags);
+ unsigned long flags;
+ spin_lock_irqsave(&instance->hba_lock, flags);
+ writel(le32_to_cpu(req_desc->u.low),
+ &instance->reg_set->inbound_low_queue_port);
+ writel(le32_to_cpu(req_desc->u.high),
+ &instance->reg_set->inbound_high_queue_port);
+ mmiowb();
+ spin_unlock_irqrestore(&instance->hba_lock, flags);
#endif
- }
}
/**
const char *sys_info;
MFI_CAPABILITIES *drv_ops;
u32 scratch_pad_2;
- unsigned long flags;
ktime_t time;
bool cur_fw_64bit_dma_capable;
break;
}
- /* For Ventura also IOC INIT required 64 bit Descriptor write. */
- spin_lock_irqsave(&instance->hba_lock, flags);
- writel(le32_to_cpu(req_desc.u.low),
- &instance->reg_set->inbound_low_queue_port);
- writel(le32_to_cpu(req_desc.u.high),
- &instance->reg_set->inbound_high_queue_port);
- mmiowb();
- spin_unlock_irqrestore(&instance->hba_lock, flags);
+ megasas_fire_cmd_fusion(instance, &req_desc);
wait_and_poll(instance, cmd, MFI_POLL_TIMEOUT_SECS);
}
/**
- * _wait_for_commands_to_complete - reset controller
+ * mpt3sas_wait_for_commands_to_complete - reset controller
* @ioc: Pointer to MPT_ADAPTER structure
*
* This function is waiting 10s for all pending commands to complete
* prior to putting controller in reset.
*/
-static void
-_wait_for_commands_to_complete(struct MPT3SAS_ADAPTER *ioc)
+void
+mpt3sas_wait_for_commands_to_complete(struct MPT3SAS_ADAPTER *ioc)
{
u32 ioc_state;
is_fault = 1;
}
_base_reset_handler(ioc, MPT3_IOC_PRE_RESET);
- _wait_for_commands_to_complete(ioc);
+ mpt3sas_wait_for_commands_to_complete(ioc);
_base_mask_interrupts(ioc);
r = _base_make_ioc_ready(ioc, type);
if (r)
int mpt3sas_port_enable(struct MPT3SAS_ADAPTER *ioc);
+void
+mpt3sas_wait_for_commands_to_complete(struct MPT3SAS_ADAPTER *ioc);
+
/* scsih shared API */
struct scsi_cmnd *mpt3sas_scsih_scsi_lookup_get(struct MPT3SAS_ADAPTER *ioc,
_scsih_tm_display_info(ioc, scmd);
sas_device_priv_data = scmd->device->hostdata;
- if (!sas_device_priv_data || !sas_device_priv_data->sas_target) {
+ if (!sas_device_priv_data || !sas_device_priv_data->sas_target ||
+ ioc->remove_host) {
sdev_printk(KERN_INFO, scmd->device,
"device been deleted! scmd(%p)\n", scmd);
scmd->result = DID_NO_CONNECT << 16;
_scsih_tm_display_info(ioc, scmd);
sas_device_priv_data = scmd->device->hostdata;
- if (!sas_device_priv_data || !sas_device_priv_data->sas_target) {
+ if (!sas_device_priv_data || !sas_device_priv_data->sas_target ||
+ ioc->remove_host) {
sdev_printk(KERN_INFO, scmd->device,
"device been deleted! scmd(%p)\n", scmd);
scmd->result = DID_NO_CONNECT << 16;
_scsih_tm_display_info(ioc, scmd);
sas_device_priv_data = scmd->device->hostdata;
- if (!sas_device_priv_data || !sas_device_priv_data->sas_target) {
+ if (!sas_device_priv_data || !sas_device_priv_data->sas_target ||
+ ioc->remove_host) {
starget_printk(KERN_INFO, starget, "target been deleted! scmd(%p)\n",
scmd);
scmd->result = DID_NO_CONNECT << 16;
ioc->name, scmd);
scsi_print_command(scmd);
- if (ioc->is_driver_loading) {
+ if (ioc->is_driver_loading || ioc->remove_host) {
pr_info(MPT3SAS_FMT "Blocking the host reset\n",
ioc->name);
r = FAILED;
st = scsi_cmd_priv(scmd);
mpt3sas_base_clear_st(ioc, st);
scsi_dma_unmap(scmd);
- if (ioc->pci_error_recovery)
+ if (ioc->pci_error_recovery || ioc->remove_host)
scmd->result = DID_NO_CONNECT << 16;
else
scmd->result = DID_RESET << 16;
unsigned long flags;
ioc->remove_host = 1;
+
+ mpt3sas_wait_for_commands_to_complete(ioc);
+ _scsih_flush_running_cmds(ioc);
+
_scsih_fw_event_cleanup_queue(ioc);
spin_lock_irqsave(&ioc->fw_event_lock, flags);
unsigned long flags;
ioc->remove_host = 1;
+
+ mpt3sas_wait_for_commands_to_complete(ioc);
+ _scsih_flush_running_cmds(ioc);
+
_scsih_fw_event_cleanup_queue(ioc);
spin_lock_irqsave(&ioc->fw_event_lock, flags);
snprintf(ioc->firmware_event_name, sizeof(ioc->firmware_event_name),
"fw_event_%s%d", ioc->driver_name, ioc->id);
ioc->firmware_event_thread = alloc_ordered_workqueue(
- ioc->firmware_event_name, WQ_MEM_RECLAIM);
+ ioc->firmware_event_name, 0);
if (!ioc->firmware_event_thread) {
pr_err(MPT3SAS_FMT "failure at %s:%d/%s()!\n",
ioc->name, __FILE__, __LINE__, __func__);
iscsi_cid = cqe->conn_id;
qedi_conn = qedi->cid_que.conn_cid_tbl[iscsi_cid];
+ if (!qedi_conn) {
+ QEDI_INFO(&qedi->dbg_ctx, QEDI_LOG_INFO,
+ "icid not found 0x%x\n", cqe->conn_id);
+ return;
+ }
/* Based on this itt get the corresponding qedi_cmd */
spin_lock_bh(&qedi_conn->tmf_work_lock);
struct name_list_extended {
struct get_name_list_extended *l;
dma_addr_t ldma;
- struct list_head fcports; /* protect by sess_list */
+ struct list_head fcports;
+ spinlock_t fcports_lock;
u32 size;
- u8 sent;
};
/*
* Timeout timer counts in seconds
/* FCP-4 types */
#define FC4_TYPE_FCP_SCSI 0x08
+#define FC4_TYPE_NVME 0x28
#define FC4_TYPE_OTHER 0x0
#define FC4_TYPE_UNKNOWN 0xff
sp->free(sp);
fcport->flags &= ~FCF_ASYNC_SENT;
done:
+ fcport->flags &= ~FCF_ASYNC_ACTIVE;
return rval;
}
sp->free(sp);
fcport->flags &= ~FCF_ASYNC_SENT;
done:
+ fcport->flags &= ~FCF_ASYNC_ACTIVE;
return rval;
}
spin_lock_irqsave(&vha->work_lock, flags);
vha->scan.scan_flags &= ~SF_SCANNING;
spin_unlock_irqrestore(&vha->work_lock, flags);
+
+ if ((fc4type == FC4_TYPE_FCP_SCSI) && vha->flags.nvme_enabled)
+ qla24xx_async_gpnft(vha, FC4_TYPE_NVME);
}
static void qla2x00_async_gpnft_gnnft_sp_done(void *s, int res)
sp->free(sp);
fcport->flags &= ~FCF_ASYNC_SENT;
done:
+ fcport->flags &= ~FCF_ASYNC_ACTIVE;
return rval;
}
done_free_sp:
sp->free(sp);
done:
- fcport->flags &= ~FCF_ASYNC_SENT;
+ fcport->flags &= ~(FCF_ASYNC_SENT | FCF_ASYNC_ACTIVE);
return rval;
}
qla2x00_async_prlo_done(struct scsi_qla_host *vha, fc_port_t *fcport,
uint16_t *data)
{
+ fcport->flags &= ~FCF_ASYNC_ACTIVE;
/* Don't re-login in target mode */
if (!fcport->tgt_session)
qla2x00_mark_device_lost(vha, fcport, 1, 0);
struct srb_iocb *lio = &sp->u.iocb_cmd;
struct scsi_qla_host *vha = sp->vha;
+ sp->fcport->flags &= ~FCF_ASYNC_ACTIVE;
if (!test_bit(UNLOADING, &vha->dpc_flags))
qla2x00_post_async_prlo_done_work(sp->fcport->vha, sp->fcport,
lio->u.logio.data);
done_free_sp:
sp->free(sp);
done:
+ fcport->flags &= ~FCF_ASYNC_ACTIVE;
return rval;
}
"Async done-%s res %x %8phC\n",
sp->name, res, sp->fcport->port_name);
+ sp->fcport->flags &= ~FCF_ASYNC_SENT;
+
memset(&ea, 0, sizeof(ea));
ea.event = FCME_ADISC_DONE;
ea.rc = res;
done_free_sp:
sp->free(sp);
done:
- fcport->flags &= ~FCF_ASYNC_SENT;
+ fcport->flags &= ~(FCF_ASYNC_SENT | FCF_ASYNC_ACTIVE);
qla2x00_post_async_adisc_work(vha, fcport, data);
return rval;
}
(loop_id & 0x7fff));
}
- spin_lock_irqsave(&vha->hw->tgt.sess_lock, flags);
- vha->gnl.sent = 0;
+ spin_lock_irqsave(&vha->gnl.fcports_lock, flags);
INIT_LIST_HEAD(&h);
fcport = tf = NULL;
list_for_each_entry_safe(fcport, tf, &h, gnl_entry) {
list_del_init(&fcport->gnl_entry);
+ spin_lock(&vha->hw->tgt.sess_lock);
fcport->flags &= ~(FCF_ASYNC_SENT | FCF_ASYNC_ACTIVE);
+ spin_unlock(&vha->hw->tgt.sess_lock);
ea.fcport = fcport;
qla2x00_fcport_event_handler(vha, &ea);
}
+ spin_unlock_irqrestore(&vha->gnl.fcports_lock, flags);
+ spin_lock_irqsave(&vha->hw->tgt.sess_lock, flags);
/* create new fcport if fw has knowledge of new sessions */
for (i = 0; i < n; i++) {
port_id_t id;
ql_dbg(ql_dbg_disc, vha, 0x20d9,
"Async-gnlist WWPN %8phC \n", fcport->port_name);
- spin_lock_irqsave(&vha->hw->tgt.sess_lock, flags);
+ spin_lock_irqsave(&vha->gnl.fcports_lock, flags);
+ if (!list_empty(&fcport->gnl_entry)) {
+ spin_unlock_irqrestore(&vha->gnl.fcports_lock, flags);
+ rval = QLA_SUCCESS;
+ goto done;
+ }
+
+ spin_lock(&vha->hw->tgt.sess_lock);
fcport->disc_state = DSC_GNL;
fcport->last_rscn_gen = fcport->rscn_gen;
fcport->last_login_gen = fcport->login_gen;
+ spin_unlock(&vha->hw->tgt.sess_lock);
list_add_tail(&fcport->gnl_entry, &vha->gnl.fcports);
- if (vha->gnl.sent) {
- spin_unlock_irqrestore(&vha->hw->tgt.sess_lock, flags);
- return QLA_SUCCESS;
- }
- vha->gnl.sent = 1;
- spin_unlock_irqrestore(&vha->hw->tgt.sess_lock, flags);
+ spin_unlock_irqrestore(&vha->gnl.fcports_lock, flags);
sp = qla2x00_get_sp(vha, fcport, GFP_KERNEL);
if (!sp)
fc_port_t *fcport = ea->fcport;
struct port_database_24xx *pd;
struct srb *sp = ea->sp;
+ uint8_t ls;
pd = (struct port_database_24xx *)sp->u.iocb_cmd.u.mbx.in;
if (fcport->disc_state == DSC_DELETE_PEND)
return;
- switch (pd->current_login_state) {
+ if (fcport->fc4f_nvme)
+ ls = pd->current_login_state >> 4;
+ else
+ ls = pd->current_login_state & 0xf;
+
+ switch (ls) {
case PDS_PRLI_COMPLETE:
__qla24xx_parse_gpdb(vha, fcport, pd);
break;
if (fcport->scan_state != QLA_FCPORT_FOUND)
return 0;
- if ((fcport->fw_login_state == DSC_LS_PLOGI_PEND) ||
- (fcport->fw_login_state == DSC_LS_PRLI_PEND))
+ if ((fcport->loop_id != FC_NO_LOOP_ID) &&
+ ((fcport->fw_login_state == DSC_LS_PLOGI_PEND) ||
+ (fcport->fw_login_state == DSC_LS_PRLI_PEND)))
return 0;
if (fcport->fw_login_state == DSC_LS_PLOGI_COMP) {
srb_t *sp = ptr;
struct srb_iocb *abt = &sp->u.iocb_cmd;
+ del_timer(&sp->u.iocb_cmd.timer);
complete(&abt->u.abt.comp);
}
set_bit(ea->fcport->loop_id, vha->hw->loop_id_map);
spin_lock_irqsave(&vha->hw->tgt.sess_lock, flags);
- ea->fcport->loop_id = FC_NO_LOOP_ID;
ea->fcport->chip_reset = vha->hw->base_qpair->chip_reset;
ea->fcport->logout_on_delete = 1;
ea->fcport->send_els_logo = 0;
qla2x00_mark_device_lost(vha, fcport, 1, 0);
qlt_logo_completion_handler(fcport, data[0]);
fcport->login_gen++;
+ fcport->flags &= ~FCF_ASYNC_ACTIVE;
return;
}
qla2x00_async_adisc_done(struct scsi_qla_host *vha, fc_port_t *fcport,
uint16_t *data)
{
+ fcport->flags &= ~(FCF_ASYNC_SENT | FCF_ASYNC_ACTIVE);
if (data[0] == MBS_COMMAND_COMPLETE) {
qla2x00_update_fcport(vha, fcport);
}
/* Retry login. */
- fcport->flags &= ~FCF_ASYNC_SENT;
if (data[1] & QLA_LOGIO_LOGIN_RETRIED)
set_bit(RELOGIN_NEEDED, &vha->dpc_flags);
else
ha->req_q_map[0] = req;
set_bit(0, ha->rsp_qid_map);
set_bit(0, ha->req_qid_map);
- return 1;
+ return 0;
fail_qpair_map:
kfree(ha->base_qpair);
static void qla2x00_free_req_que(struct qla_hw_data *ha, struct req_que *req)
{
+ if (!ha->req_q_map)
+ return;
+
if (IS_QLAFX00(ha)) {
if (req && req->ring_fx00)
dma_free_coherent(&ha->pdev->dev,
(req->length + 1) * sizeof(request_t),
req->ring, req->dma);
- if (req)
+ if (req) {
kfree(req->outstanding_cmds);
-
- kfree(req);
+ kfree(req);
+ }
}
static void qla2x00_free_rsp_que(struct qla_hw_data *ha, struct rsp_que *rsp)
{
+ if (!ha->rsp_q_map)
+ return;
+
if (IS_QLAFX00(ha)) {
if (rsp && rsp->ring)
dma_free_coherent(&ha->pdev->dev,
(rsp->length + 1) * sizeof(response_t),
rsp->ring, rsp->dma);
}
- kfree(rsp);
+ if (rsp)
+ kfree(rsp);
}
static void qla2x00_free_queues(struct qla_hw_data *ha)
struct qla_tgt_cmd *cmd;
uint8_t trace = 0;
+ if (!ha->req_q_map)
+ return;
spin_lock_irqsave(qp->qp_lock_ptr, flags);
req = qp->req;
for (cnt = 1; cnt < req->num_outstanding_cmds; cnt++) {
/* Set up the irqs */
ret = qla2x00_request_irqs(ha, rsp);
if (ret)
- goto probe_hw_failed;
+ goto probe_failed;
/* Alloc arrays of request and response ring ptrs */
- if (!qla2x00_alloc_queues(ha, req, rsp)) {
+ if (qla2x00_alloc_queues(ha, req, rsp)) {
ql_log(ql_log_fatal, base_vha, 0x003d,
"Failed to allocate memory for queue pointers..."
"aborting.\n");
- goto probe_init_failed;
+ goto probe_failed;
}
if (ha->mqenable && shost_use_blk_mq(host)) {
return 0;
-probe_init_failed:
- qla2x00_free_req_que(ha, req);
- ha->req_q_map[0] = NULL;
- clear_bit(0, ha->req_qid_map);
- qla2x00_free_rsp_que(ha, rsp);
- ha->rsp_q_map[0] = NULL;
- clear_bit(0, ha->rsp_qid_map);
- ha->max_req_queues = ha->max_rsp_queues = 0;
-
probe_failed:
if (base_vha->timer_active)
qla2x00_stop_timer(base_vha);
if (ha->init_cb)
dma_free_coherent(&ha->pdev->dev, ha->init_cb_size,
ha->init_cb, ha->init_cb_dma);
- vfree(ha->optrom_buffer);
- kfree(ha->nvram);
- kfree(ha->npiv_info);
- kfree(ha->swl);
- kfree(ha->loop_id_map);
+
+ if (ha->optrom_buffer)
+ vfree(ha->optrom_buffer);
+ if (ha->nvram)
+ kfree(ha->nvram);
+ if (ha->npiv_info)
+ kfree(ha->npiv_info);
+ if (ha->swl)
+ kfree(ha->swl);
+ if (ha->loop_id_map)
+ kfree(ha->loop_id_map);
ha->srb_mempool = NULL;
ha->ctx_mempool = NULL;
ha->ex_init_cb_dma = 0;
ha->async_pd = NULL;
ha->async_pd_dma = 0;
+ ha->loop_id_map = NULL;
+ ha->npiv_info = NULL;
+ ha->optrom_buffer = NULL;
+ ha->swl = NULL;
+ ha->nvram = NULL;
+ ha->mctp_dump = NULL;
+ ha->dcbx_tlv = NULL;
+ ha->xgmac_data = NULL;
+ ha->sfp_data = NULL;
ha->s_dma_pool = NULL;
ha->dl_dma_pool = NULL;
spin_lock_init(&vha->work_lock);
spin_lock_init(&vha->cmd_list_lock);
+ spin_lock_init(&vha->gnl.fcports_lock);
init_waitqueue_head(&vha->fcport_waitQ);
init_waitqueue_head(&vha->vref_waitq);
fcport->d_id = e->u.new_sess.id;
fcport->flags |= FCF_FABRIC_DEVICE;
fcport->fw_login_state = DSC_LS_PLOGI_PEND;
- if (e->u.new_sess.fc4_type == FC4_TYPE_FCP_SCSI)
+ if (e->u.new_sess.fc4_type == FC4_TYPE_FCP_SCSI) {
fcport->fc4_type = FC4_TYPE_FCP_SCSI;
-
+ } else if (e->u.new_sess.fc4_type == FC4_TYPE_NVME) {
+ fcport->fc4_type = FC4_TYPE_OTHER;
+ fcport->fc4f_nvme = FC4_TYPE_NVME;
+ }
memcpy(fcport->port_name, e->u.new_sess.port_name,
WWN_SIZE);
} else {
}
qlt_plogi_ack_unref(vha, pla);
} else {
+ fc_port_t *dfcp = NULL;
+
spin_lock_irqsave(&vha->hw->tgt.sess_lock, flags);
tfcp = qla2x00_find_fcport_by_nportid(vha,
&e->u.new_sess.id, 1);
default:
fcport->login_pause = 1;
tfcp->conflict = fcport;
- qlt_schedule_sess_for_deletion(tfcp);
+ dfcp = tfcp;
break;
}
}
spin_unlock_irqrestore(&vha->hw->tgt.sess_lock, flags);
+ if (dfcp)
+ qlt_schedule_sess_for_deletion(tfcp);
wwn = wwn_to_u64(fcport->node_name);
}
}
-/* ha->tgt.sess_lock supposed to be held on entry */
void qlt_schedule_sess_for_deletion(struct fc_port *sess)
{
struct qla_tgt *tgt = sess->tgt;
+ struct qla_hw_data *ha = sess->vha->hw;
unsigned long flags;
if (sess->disc_state == DSC_DELETE_PEND)
return;
}
+ spin_lock_irqsave(&ha->tgt.sess_lock, flags);
if (sess->deleted == QLA_SESS_DELETED)
sess->logout_on_delete = 0;
- spin_lock_irqsave(&sess->vha->work_lock, flags);
if (sess->deleted == QLA_SESS_DELETION_IN_PROGRESS) {
- spin_unlock_irqrestore(&sess->vha->work_lock, flags);
+ spin_unlock_irqrestore(&ha->tgt.sess_lock, flags);
return;
}
sess->deleted = QLA_SESS_DELETION_IN_PROGRESS;
- spin_unlock_irqrestore(&sess->vha->work_lock, flags);
+ spin_unlock_irqrestore(&ha->tgt.sess_lock, flags);
sess->disc_state = DSC_DELETE_PEND;
ql_dbg(ql_dbg_tgt, sess->vha, 0xe001,
"Scheduling sess %p for deletion\n", sess);
- /* use cancel to push work element through before re-queue */
- cancel_work_sync(&sess->del_work);
INIT_WORK(&sess->del_work, qla24xx_delete_sess_fn);
- queue_work(sess->vha->hw->wq, &sess->del_work);
+ WARN_ON(!queue_work(sess->vha->hw->wq, &sess->del_work));
}
-/* ha->tgt.sess_lock supposed to be held on entry */
static void qlt_clear_tgt_db(struct qla_tgt *tgt)
{
struct fc_port *sess;
ql_dbg(ql_dbg_tgt_mgt, vha, 0xf008, "qla_tgt_fc_port_deleted %p", sess);
sess->local = 1;
- qlt_schedule_sess_for_deletion(sess);
spin_unlock_irqrestore(&vha->hw->tgt.sess_lock, flags);
+ qlt_schedule_sess_for_deletion(sess);
}
static inline int test_tgt_sess_count(struct qla_tgt *tgt)
* Lock is needed, because we still can get an incoming packet.
*/
mutex_lock(&vha->vha_tgt.tgt_mutex);
- spin_lock_irqsave(&ha->tgt.sess_lock, flags);
tgt->tgt_stop = 1;
qlt_clear_tgt_db(tgt);
- spin_unlock_irqrestore(&ha->tgt.sess_lock, flags);
mutex_unlock(&vha->vha_tgt.tgt_mutex);
mutex_unlock(&qla_tgt_mutex);
static void scsi_eh_inc_host_failed(struct rcu_head *head)
{
- struct Scsi_Host *shost = container_of(head, typeof(*shost), rcu);
+ struct scsi_cmnd *scmd = container_of(head, typeof(*scmd), rcu);
+ struct Scsi_Host *shost = scmd->device->host;
unsigned long flags;
spin_lock_irqsave(shost->host_lock, flags);
* Ensure that all tasks observe the host state change before the
* host_failed change.
*/
- call_rcu(&shost->rcu, scsi_eh_inc_host_failed);
+ call_rcu(&scmd->rcu, scsi_eh_inc_host_failed);
}
/**
if (!blk_rq_is_scsi(req)) {
WARN_ON_ONCE(!(cmd->flags & SCMD_INITIALIZED));
cmd->flags &= ~SCMD_INITIALIZED;
+ destroy_rcu_head(&cmd->rcu);
}
if (req->mq_ctx) {
int result)
{
switch (host_byte(result)) {
+ case DID_OK:
+ return BLK_STS_OK;
case DID_TRANSPORT_FAILFAST:
return BLK_STS_TRANSPORT;
case DID_TARGET_FAILURE:
struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(rq);
scsi_req_init(&cmd->req);
+ init_rcu_head(&cmd->rcu);
cmd->jiffies_at_alloc = jiffies;
cmd->retries = 0;
}
int res;
struct scsi_device *sdp = sdkp->device;
struct scsi_mode_data data;
+ int disk_ro = get_disk_ro(sdkp->disk);
int old_wp = sdkp->write_prot;
set_disk_ro(sdkp->disk, 0);
"Test WP failed, assume Write Enabled\n");
} else {
sdkp->write_prot = ((data.device_specific & 0x80) != 0);
- set_disk_ro(sdkp->disk, sdkp->write_prot);
+ set_disk_ro(sdkp->disk, sdkp->write_prot || disk_ro);
if (sdkp->first_scan || old_wp != sdkp->write_prot) {
sd_printk(KERN_NOTICE, sdkp, "Write Protect is %s\n",
sdkp->write_prot ? "on" : "off");
*/
static int sd_zbc_check_zone_size(struct scsi_disk *sdkp)
{
- u64 zone_blocks;
+ u64 zone_blocks = 0;
sector_t block = 0;
unsigned char *buf;
unsigned char *rec;
/* Do a report zone to get the same field */
ret = sd_zbc_report_zones(sdkp, buf, SD_ZBC_BUF_SIZE, 0);
- if (ret) {
- zone_blocks = 0;
- goto out;
- }
+ if (ret)
+ goto out_free;
same = buf[4] & 0x0f;
if (same > 0) {
ret = sd_zbc_report_zones(sdkp, buf,
SD_ZBC_BUF_SIZE, block);
if (ret)
- return ret;
+ goto out_free;
}
} while (block < sdkp->capacity);
zone_blocks = sdkp->zone_blocks;
out:
- kfree(buf);
-
if (!zone_blocks) {
if (sdkp->first_scan)
sd_printk(KERN_NOTICE, sdkp,
"Devices with non constant zone "
"size are not supported\n");
- return -ENODEV;
- }
-
- if (!is_power_of_2(zone_blocks)) {
+ ret = -ENODEV;
+ } else if (!is_power_of_2(zone_blocks)) {
if (sdkp->first_scan)
sd_printk(KERN_NOTICE, sdkp,
"Devices with non power of 2 zone "
"size are not supported\n");
- return -ENODEV;
- }
-
- if (logical_to_sectors(sdkp->device, zone_blocks) > UINT_MAX) {
+ ret = -ENODEV;
+ } else if (logical_to_sectors(sdkp->device, zone_blocks) > UINT_MAX) {
if (sdkp->first_scan)
sd_printk(KERN_NOTICE, sdkp,
"Zone size too large\n");
- return -ENODEV;
+ ret = -ENODEV;
+ } else {
+ sdkp->zone_blocks = zone_blocks;
+ sdkp->zone_shift = ilog2(zone_blocks);
}
- sdkp->zone_blocks = zone_blocks;
- sdkp->zone_shift = ilog2(zone_blocks);
+out_free:
+ kfree(buf);
- return 0;
+ return ret;
}
/**
*/
cpumask_and(&alloced_mask, &stor_device->alloced_cpus,
cpumask_of_node(cpu_to_node(q_num)));
- for_each_cpu(tgt_cpu, &alloced_mask) {
+ for_each_cpu_wrap(tgt_cpu, &alloced_mask,
+ outgoing_channel->target_cpu + 1) {
if (tgt_cpu != outgoing_channel->target_cpu) {
outgoing_channel =
stor_device->stor_chns[tgt_cpu];
struct completion completion;
};
-static int qman_delete_cgr_thread(void *p)
+static void qman_delete_cgr_smp_call(void *p)
{
- struct cgr_comp *cgr_comp = (struct cgr_comp *)p;
- int ret;
-
- ret = qman_delete_cgr(cgr_comp->cgr);
- complete(&cgr_comp->completion);
-
- return ret;
+ qman_delete_cgr((struct qman_cgr *)p);
}
void qman_delete_cgr_safe(struct qman_cgr *cgr)
{
- struct task_struct *thread;
- struct cgr_comp cgr_comp;
-
preempt_disable();
if (qman_cgr_cpus[cgr->cgrid] != smp_processor_id()) {
- init_completion(&cgr_comp.completion);
- cgr_comp.cgr = cgr;
- thread = kthread_create(qman_delete_cgr_thread, &cgr_comp,
- "cgr_del");
-
- if (IS_ERR(thread))
- goto out;
-
- kthread_bind(thread, qman_cgr_cpus[cgr->cgrid]);
- wake_up_process(thread);
- wait_for_completion(&cgr_comp.completion);
+ smp_call_function_single(qman_cgr_cpus[cgr->cgrid],
+ qman_delete_cgr_smp_call, cgr, true);
preempt_enable();
return;
}
-out:
+
qman_delete_cgr(cgr);
preempt_enable();
}
config SSB_DRIVER_PCICORE_POSSIBLE
bool
- depends on SSB_PCIHOST
+ depends on SSB_PCIHOST && SSB = y
default y
config SSB_DRIVER_PCICORE
/* Set dev to NULL to not unregister
* dev on error unwinding. */
sdev->dev = NULL;
- kfree(devwrap);
+ put_device(dev);
goto error;
}
dev_idx++;
chip_id == 43231 || chip_id == 43222);
}
- return 0;
+ return false;
}
u32 ssb_dma_translation(struct ssb_device *dev)
mutex_lock(&ashmem_mutex);
if (asma->size == 0) {
- ret = -EINVAL;
- goto out;
+ mutex_unlock(&ashmem_mutex);
+ return -EINVAL;
}
if (!asma->file) {
- ret = -EBADF;
- goto out;
+ mutex_unlock(&ashmem_mutex);
+ return -EBADF;
}
+ mutex_unlock(&ashmem_mutex);
+
ret = vfs_llseek(asma->file, offset, origin);
if (ret < 0)
- goto out;
+ return ret;
/** Copy f_pos from backing file, since f_ops->llseek() sets it */
file->f_pos = asma->file->f_pos;
-
-out:
- mutex_unlock(&ashmem_mutex);
return ret;
}
size_t pgstart, pgend;
int ret = -EINVAL;
+ if (unlikely(copy_from_user(&pin, p, sizeof(pin))))
+ return -EFAULT;
+
mutex_lock(&ashmem_mutex);
if (unlikely(!asma->file))
goto out_unlock;
- if (unlikely(copy_from_user(&pin, p, sizeof(pin)))) {
- ret = -EFAULT;
- goto out_unlock;
- }
-
/* per custom, you can pass zero for len to mean "everything onward" */
if (!pin.len)
pin.len = PAGE_ALIGN(asma->size) - pin.offset;
struct comedi_cmd *cmd = &async->cmd;
if (cmd->stop_src == TRIG_COUNT) {
- unsigned int nscans = nsamples / cmd->scan_end_arg;
- unsigned int scans_left = __comedi_nscans_left(s, nscans);
+ unsigned int scans_left = __comedi_nscans_left(s, cmd->stop_arg);
unsigned int scan_pos =
comedi_bytes_to_samples(s, async->scan_progress);
unsigned long long samples_left = 0;
}
if (tty_hung_up_p(file))
break;
+ /*
+ * Abort readers for ttys which never actually
+ * get hung up. See __tty_hangup().
+ */
+ if (test_bit(TTY_HUPPING, &tty->flags))
+ break;
if (!timeout)
break;
if (file->f_flags & O_NONBLOCK) {
/*
* If it is not a communications device or the programming
* interface is greater than 6, give up.
- *
- * (Should we try to make guesses for multiport serial devices
- * later?)
*/
if ((((dev->class >> 8) != PCI_CLASS_COMMUNICATION_SERIAL) &&
+ ((dev->class >> 8) != PCI_CLASS_COMMUNICATION_MULTISERIAL) &&
((dev->class >> 8) != PCI_CLASS_COMMUNICATION_MODEM)) ||
(dev->class & 0xff) > 6)
return -ENODEV;
{
int num_iomem, num_port, first_port = -1, i;
+ /*
+ * Should we try to make guesses for multiport serial devices later?
+ */
+ if ((dev->class >> 8) == PCI_CLASS_COMMUNICATION_MULTISERIAL)
+ return -ENODEV;
+
num_iomem = num_port = 0;
for (i = 0; i < PCI_NUM_BAR_RESOURCES; i++) {
if (pci_resource_flags(dev, i) & IORESOURCE_IO) {
{ PCI_VENDOR_ID_INTASHIELD, PCI_DEVICE_ID_INTASHIELD_IS400,
PCI_ANY_ID, PCI_ANY_ID, 0, 0, /* 135a.0dc0 */
pbn_b2_4_115200 },
+ /*
+ * BrainBoxes UC-260
+ */
+ { PCI_VENDOR_ID_INTASHIELD, 0x0D21,
+ PCI_ANY_ID, PCI_ANY_ID,
+ PCI_CLASS_COMMUNICATION_MULTISERIAL << 8, 0xffff00,
+ pbn_b2_4_115200 },
+ { PCI_VENDOR_ID_INTASHIELD, 0x0E34,
+ PCI_ANY_ID, PCI_ANY_ID,
+ PCI_CLASS_COMMUNICATION_MULTISERIAL << 8, 0xffff00,
+ pbn_b2_4_115200 },
/*
* Perle PCI-RAS cards
*/
switch (version) {
case 0x302:
case 0x10213:
+ case 0x10302:
dev_dbg(port->dev, "This version is usart\n");
atmel_port->has_frac_baudrate = true;
atmel_port->has_hw_timer = true;
}
port->mapbase = addr;
port->uartclk = BASE_BAUD * 16;
- port->membase = earlycon_map(port->mapbase, SZ_4K);
val = of_get_flat_dt_prop(node, "reg-offset", NULL);
if (val)
port->mapbase += be32_to_cpu(*val);
+ port->membase = earlycon_map(port->mapbase, SZ_4K);
+
val = of_get_flat_dt_prop(node, "reg-shift", NULL);
if (val)
port->regshift = be32_to_cpu(*val);
uart_get_rs485_mode(&pdev->dev, &sport->port.rs485);
if (sport->port.rs485.flags & SER_RS485_ENABLED &&
- (!sport->have_rtscts || !sport->have_rtsgpio))
+ (!sport->have_rtscts && !sport->have_rtsgpio))
dev_err(&pdev->dev, "no RTS control, disabling rs485\n");
imx_rs485_config(&sport->port, &sport->port.rs485);
uport->ops->config_port(uport, flags);
ret = uart_startup(tty, state, 1);
+ if (ret == 0)
+ tty_port_set_initialized(port, true);
if (ret > 0)
ret = 0;
}
/* Tell the rest of the system the news. New characters! */
tty_flip_buffer_push(tport);
} else {
+ /* TTY buffers full; read from RX reg to prevent lockup */
+ serial_port_in(port, SCxRDR);
serial_port_in(port, SCxSR); /* dummy read */
sci_clear_SCxSR(port, SCxSR_RDxF_CLEAR(port));
}
return;
}
+ /*
+ * Some console devices aren't actually hung up for technical and
+ * historical reasons, which can lead to indefinite interruptible
+ * sleep in n_tty_read(). The following explicitly tells
+ * n_tty_read() to abort readers.
+ */
+ set_bit(TTY_HUPPING, &tty->flags);
+
/* inuse_filps is protected by the single tty lock,
this really needs to change if we want to flush the
workqueue with the lock held */
* from the ldisc side, which is now guaranteed.
*/
set_bit(TTY_HUPPED, &tty->flags);
+ clear_bit(TTY_HUPPING, &tty->flags);
tty_unlock(tty);
if (f)
ret = usb_internal_control_msg(dev, pipe, dr, data, size, timeout);
+ /* Linger a bit, prior to the next control message. */
+ if (dev->quirks & USB_QUIRK_DELAY_CTRL_MSG)
+ msleep(200);
+
kfree(dr);
return ret;
{ USB_DEVICE(0x1b1c, 0x1b13), .driver_info = USB_QUIRK_DELAY_INIT },
/* Corsair Strafe RGB */
- { USB_DEVICE(0x1b1c, 0x1b20), .driver_info = USB_QUIRK_DELAY_INIT },
+ { USB_DEVICE(0x1b1c, 0x1b20), .driver_info = USB_QUIRK_DELAY_INIT |
+ USB_QUIRK_DELAY_CTRL_MSG },
/* Corsair K70 LUX */
{ USB_DEVICE(0x1b1c, 0x1b36), .driver_info = USB_QUIRK_DELAY_INIT },
p->activate_stm_fs_transceiver = true;
}
-static void dwc2_set_stm32f7xx_hsotg_params(struct dwc2_hsotg *hsotg)
+static void dwc2_set_stm32f7_hsotg_params(struct dwc2_hsotg *hsotg)
{
struct dwc2_core_params *p = &hsotg->params;
{ .compatible = "st,stm32f4x9-fsotg",
.data = dwc2_set_stm32f4x9_fsotg_params },
{ .compatible = "st,stm32f4x9-hsotg" },
- { .compatible = "st,stm32f7xx-hsotg",
- .data = dwc2_set_stm32f7xx_hsotg_params },
+ { .compatible = "st,stm32f7-hsotg",
+ .data = dwc2_set_stm32f7_hsotg_params },
{},
};
MODULE_DEVICE_TABLE(of, dwc2_of_match_table);
dwc->desired_dr_role = mode;
spin_unlock_irqrestore(&dwc->lock, flags);
- queue_work(system_power_efficient_wq, &dwc->drd_work);
+ queue_work(system_freezable_wq, &dwc->drd_work);
}
u32 dwc3_core_fifo_space(struct dwc3_ep *dep, u8 type)
if (sb->s_fs_info) {
ffs_release_dev(sb->s_fs_info);
ffs_data_closed(sb->s_fs_info);
- ffs_data_put(sb->s_fs_info);
}
}
struct usb_hcd *hcd = ohci_to_hcd(ohci);
/* Accept arbitrarily long scatter-gather lists */
- hcd->self.sg_tablesize = ~0;
+ if (!(hcd->driver->flags & HCD_LOCAL_MEM))
+ hcd->self.sg_tablesize = ~0;
if (distrust_firmware)
ohci->flags |= OHCI_QUIRK_HUB_POWER;
int dbc_ep_queue(struct dbc_ep *dep, struct dbc_request *req,
gfp_t gfp_flags)
{
+ unsigned long flags;
struct xhci_dbc *dbc = dep->dbc;
int ret = -ESHUTDOWN;
- spin_lock(&dbc->lock);
+ spin_lock_irqsave(&dbc->lock, flags);
if (dbc->state == DS_CONFIGURED)
ret = dbc_ep_do_queue(dep, req);
- spin_unlock(&dbc->lock);
+ spin_unlock_irqrestore(&dbc->lock, flags);
mod_delayed_work(system_wq, &dbc->event_work, 0);
static int xhci_dbc_start(struct xhci_hcd *xhci)
{
int ret;
+ unsigned long flags;
struct xhci_dbc *dbc = xhci->dbc;
WARN_ON(!dbc);
pm_runtime_get_sync(xhci_to_hcd(xhci)->self.controller);
- spin_lock(&dbc->lock);
+ spin_lock_irqsave(&dbc->lock, flags);
ret = xhci_do_dbc_start(xhci);
- spin_unlock(&dbc->lock);
+ spin_unlock_irqrestore(&dbc->lock, flags);
if (ret) {
pm_runtime_put(xhci_to_hcd(xhci)->self.controller);
static void xhci_dbc_stop(struct xhci_hcd *xhci)
{
+ unsigned long flags;
struct xhci_dbc *dbc = xhci->dbc;
struct dbc_port *port = &dbc->port;
if (port->registered)
xhci_dbc_tty_unregister_device(xhci);
- spin_lock(&dbc->lock);
+ spin_lock_irqsave(&dbc->lock, flags);
xhci_do_dbc_stop(xhci);
- spin_unlock(&dbc->lock);
+ spin_unlock_irqrestore(&dbc->lock, flags);
pm_runtime_put_sync(xhci_to_hcd(xhci)->self.controller);
}
int ret;
enum evtreturn evtr;
struct xhci_dbc *dbc;
+ unsigned long flags;
struct xhci_hcd *xhci;
dbc = container_of(to_delayed_work(work), struct xhci_dbc, event_work);
xhci = dbc->xhci;
- spin_lock(&dbc->lock);
+ spin_lock_irqsave(&dbc->lock, flags);
evtr = xhci_dbc_do_handle_events(dbc);
- spin_unlock(&dbc->lock);
+ spin_unlock_irqrestore(&dbc->lock, flags);
switch (evtr) {
case EVT_GSER:
static void
dbc_read_complete(struct xhci_hcd *xhci, struct dbc_request *req)
{
+ unsigned long flags;
struct xhci_dbc *dbc = xhci->dbc;
struct dbc_port *port = &dbc->port;
- spin_lock(&port->port_lock);
+ spin_lock_irqsave(&port->port_lock, flags);
list_add_tail(&req->list_pool, &port->read_queue);
tasklet_schedule(&port->push);
- spin_unlock(&port->port_lock);
+ spin_unlock_irqrestore(&port->port_lock, flags);
}
static void dbc_write_complete(struct xhci_hcd *xhci, struct dbc_request *req)
{
+ unsigned long flags;
struct xhci_dbc *dbc = xhci->dbc;
struct dbc_port *port = &dbc->port;
- spin_lock(&port->port_lock);
+ spin_lock_irqsave(&port->port_lock, flags);
list_add(&req->list_pool, &port->write_pool);
switch (req->status) {
case 0:
req->status);
break;
}
- spin_unlock(&port->port_lock);
+ spin_unlock_irqrestore(&port->port_lock, flags);
}
static void xhci_dbc_free_req(struct dbc_ep *dep, struct dbc_request *req)
{
struct dbc_request *req;
struct tty_struct *tty;
+ unsigned long flags;
bool do_push = false;
bool disconnect = false;
struct dbc_port *port = (void *)_port;
struct list_head *queue = &port->read_queue;
- spin_lock_irq(&port->port_lock);
+ spin_lock_irqsave(&port->port_lock, flags);
tty = port->port.tty;
while (!list_empty(queue)) {
req = list_first_entry(queue, struct dbc_request, list_pool);
if (!disconnect)
dbc_start_rx(port);
- spin_unlock_irq(&port->port_lock);
+ spin_unlock_irqrestore(&port->port_lock, flags);
}
static int dbc_port_activate(struct tty_port *_port, struct tty_struct *tty)
{
+ unsigned long flags;
struct dbc_port *port = container_of(_port, struct dbc_port, port);
- spin_lock_irq(&port->port_lock);
+ spin_lock_irqsave(&port->port_lock, flags);
dbc_start_rx(port);
- spin_unlock_irq(&port->port_lock);
+ spin_unlock_irqrestore(&port->port_lock, flags);
return 0;
}
if (pdev->vendor == PCI_VENDOR_ID_AMD && usb_amd_find_chipset_info())
xhci->quirks |= XHCI_AMD_PLL_FIX;
+ if (pdev->vendor == PCI_VENDOR_ID_AMD && pdev->device == 0x43bb)
+ xhci->quirks |= XHCI_SUSPEND_DELAY;
+
if (pdev->vendor == PCI_VENDOR_ID_AMD)
xhci->quirks |= XHCI_TRUST_TX_LENGTH;
{
struct usb_hcd *hcd = dev_get_drvdata(dev);
struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- int ret;
/*
* xhci_suspend() needs `do_wakeup` to know whether host is allowed
* reconsider this when xhci_plat_suspend enlarges its scope, e.g.,
* also applies to runtime suspend.
*/
- ret = xhci_suspend(xhci, device_may_wakeup(dev));
-
- if (!device_may_wakeup(dev) && !IS_ERR(xhci->clk))
- clk_disable_unprepare(xhci->clk);
-
- return ret;
+ return xhci_suspend(xhci, device_may_wakeup(dev));
}
static int __maybe_unused xhci_plat_resume(struct device *dev)
struct xhci_hcd *xhci = hcd_to_xhci(hcd);
int ret;
- if (!device_may_wakeup(dev) && !IS_ERR(xhci->clk))
- clk_prepare_enable(xhci->clk);
-
ret = xhci_priv_resume_quirk(hcd);
if (ret)
return ret;
.soc_id = "r8a7796",
.data = (void *)RCAR_XHCI_FIRMWARE_V3,
},
+ {
+ .soc_id = "r8a77965",
+ .data = (void *)RCAR_XHCI_FIRMWARE_V3,
+ },
{ /* sentinel */ },
};
clear_bit(HCD_FLAG_POLL_RH, &xhci->shared_hcd->flags);
del_timer_sync(&xhci->shared_hcd->rh_timer);
+ if (xhci->quirks & XHCI_SUSPEND_DELAY)
+ usleep_range(1000, 1500);
+
spin_lock_irq(&xhci->lock);
clear_bit(HCD_FLAG_HW_ACCESSIBLE, &hcd->flags);
clear_bit(HCD_FLAG_HW_ACCESSIBLE, &xhci->shared_hcd->flags);
/* bits 10:14 are Max Primary Streams */
/* bit 15 is Linear Stream Array */
/* Interval - period between requests to an endpoint - 125u increments. */
-#define EP_INTERVAL(p) (((p) & 0xff) << 16)
-#define EP_INTERVAL_TO_UFRAMES(p) (1 << (((p) >> 16) & 0xff))
-#define CTX_TO_EP_INTERVAL(p) (((p) >> 16) & 0xff)
-#define EP_MAXPSTREAMS_MASK (0x1f << 10)
-#define EP_MAXPSTREAMS(p) (((p) << 10) & EP_MAXPSTREAMS_MASK)
+#define EP_INTERVAL(p) (((p) & 0xff) << 16)
+#define EP_INTERVAL_TO_UFRAMES(p) (1 << (((p) >> 16) & 0xff))
+#define CTX_TO_EP_INTERVAL(p) (((p) >> 16) & 0xff)
+#define EP_MAXPSTREAMS_MASK (0x1f << 10)
+#define EP_MAXPSTREAMS(p) (((p) << 10) & EP_MAXPSTREAMS_MASK)
+#define CTX_TO_EP_MAXPSTREAMS(p) (((p) & EP_MAXPSTREAMS_MASK) >> 10)
/* Endpoint is set up with a Linear Stream Array (vs. Secondary Stream Array) */
#define EP_HAS_LSA (1 << 15)
/* hosts with LEC=1 use bits 31:24 as ESIT high bits. */
#define XHCI_U2_DISABLE_WAKE (1 << 27)
#define XHCI_ASMEDIA_MODIFY_FLOWCONTROL (1 << 28)
#define XHCI_HW_LPM_DISABLE (1 << 29)
+#define XHCI_SUSPEND_DELAY (1 << 30)
unsigned int num_active_eps;
unsigned int limit_active_eps;
u8 burst;
u8 cerr;
u8 mult;
- u8 lsa;
- u8 hid;
+
+ bool lsa;
+ bool hid;
esit = CTX_TO_MAX_ESIT_PAYLOAD_HI(info) << 16 |
CTX_TO_MAX_ESIT_PAYLOAD(tx_info);
ep_state = info & EP_STATE_MASK;
- max_pstr = info & EP_MAXPSTREAMS_MASK;
+ max_pstr = CTX_TO_EP_MAXPSTREAMS(info);
interval = CTX_TO_EP_INTERVAL(info);
mult = CTX_TO_EP_MULT(info) + 1;
- lsa = info & EP_HAS_LSA;
+ lsa = !!(info & EP_HAS_LSA);
cerr = (info2 & (3 << 1)) >> 1;
ep_type = CTX_TO_EP_TYPE(info2);
- hid = info2 & (1 << 7);
+ hid = !!(info2 & (1 << 7));
burst = CTX_TO_MAX_BURST(info2);
maxp = MAX_PACKET_DECODED(info2);
wait_queue_head_t wait;
int printf_size;
+ size_t printf_offset;
+ size_t printf_togo;
char *printf_buf;
struct mutex printf_lock;
return rc;
}
-/*
- * For simplicity, we read one record in one system call and throw out
- * what does not fit. This means that the following does not work:
- * dd if=/dbg/usbmon/0t bs=10
- * Also, we do not allow seeks and do not bother advancing the offset.
- */
+static ssize_t mon_text_copy_to_user(struct mon_reader_text *rp,
+ char __user * const buf, const size_t nbytes)
+{
+ const size_t togo = min(nbytes, rp->printf_togo);
+
+ if (copy_to_user(buf, &rp->printf_buf[rp->printf_offset], togo))
+ return -EFAULT;
+ rp->printf_togo -= togo;
+ rp->printf_offset += togo;
+ return togo;
+}
+
+/* ppos is not advanced since the llseek operation is not permitted. */
static ssize_t mon_text_read_t(struct file *file, char __user *buf,
- size_t nbytes, loff_t *ppos)
+ size_t nbytes, loff_t *ppos)
{
struct mon_reader_text *rp = file->private_data;
struct mon_event_text *ep;
struct mon_text_ptr ptr;
+ ssize_t ret;
- ep = mon_text_read_wait(rp, file);
- if (IS_ERR(ep))
- return PTR_ERR(ep);
mutex_lock(&rp->printf_lock);
- ptr.cnt = 0;
- ptr.pbuf = rp->printf_buf;
- ptr.limit = rp->printf_size;
-
- mon_text_read_head_t(rp, &ptr, ep);
- mon_text_read_statset(rp, &ptr, ep);
- ptr.cnt += snprintf(ptr.pbuf + ptr.cnt, ptr.limit - ptr.cnt,
- " %d", ep->length);
- mon_text_read_data(rp, &ptr, ep);
-
- if (copy_to_user(buf, rp->printf_buf, ptr.cnt))
- ptr.cnt = -EFAULT;
+
+ if (rp->printf_togo == 0) {
+
+ ep = mon_text_read_wait(rp, file);
+ if (IS_ERR(ep)) {
+ mutex_unlock(&rp->printf_lock);
+ return PTR_ERR(ep);
+ }
+ ptr.cnt = 0;
+ ptr.pbuf = rp->printf_buf;
+ ptr.limit = rp->printf_size;
+
+ mon_text_read_head_t(rp, &ptr, ep);
+ mon_text_read_statset(rp, &ptr, ep);
+ ptr.cnt += snprintf(ptr.pbuf + ptr.cnt, ptr.limit - ptr.cnt,
+ " %d", ep->length);
+ mon_text_read_data(rp, &ptr, ep);
+
+ rp->printf_togo = ptr.cnt;
+ rp->printf_offset = 0;
+
+ kmem_cache_free(rp->e_slab, ep);
+ }
+
+ ret = mon_text_copy_to_user(rp, buf, nbytes);
mutex_unlock(&rp->printf_lock);
- kmem_cache_free(rp->e_slab, ep);
- return ptr.cnt;
+ return ret;
}
+/* ppos is not advanced since the llseek operation is not permitted. */
static ssize_t mon_text_read_u(struct file *file, char __user *buf,
- size_t nbytes, loff_t *ppos)
+ size_t nbytes, loff_t *ppos)
{
struct mon_reader_text *rp = file->private_data;
struct mon_event_text *ep;
struct mon_text_ptr ptr;
+ ssize_t ret;
- ep = mon_text_read_wait(rp, file);
- if (IS_ERR(ep))
- return PTR_ERR(ep);
mutex_lock(&rp->printf_lock);
- ptr.cnt = 0;
- ptr.pbuf = rp->printf_buf;
- ptr.limit = rp->printf_size;
- mon_text_read_head_u(rp, &ptr, ep);
- if (ep->type == 'E') {
- mon_text_read_statset(rp, &ptr, ep);
- } else if (ep->xfertype == USB_ENDPOINT_XFER_ISOC) {
- mon_text_read_isostat(rp, &ptr, ep);
- mon_text_read_isodesc(rp, &ptr, ep);
- } else if (ep->xfertype == USB_ENDPOINT_XFER_INT) {
- mon_text_read_intstat(rp, &ptr, ep);
- } else {
- mon_text_read_statset(rp, &ptr, ep);
+ if (rp->printf_togo == 0) {
+
+ ep = mon_text_read_wait(rp, file);
+ if (IS_ERR(ep)) {
+ mutex_unlock(&rp->printf_lock);
+ return PTR_ERR(ep);
+ }
+ ptr.cnt = 0;
+ ptr.pbuf = rp->printf_buf;
+ ptr.limit = rp->printf_size;
+
+ mon_text_read_head_u(rp, &ptr, ep);
+ if (ep->type == 'E') {
+ mon_text_read_statset(rp, &ptr, ep);
+ } else if (ep->xfertype == USB_ENDPOINT_XFER_ISOC) {
+ mon_text_read_isostat(rp, &ptr, ep);
+ mon_text_read_isodesc(rp, &ptr, ep);
+ } else if (ep->xfertype == USB_ENDPOINT_XFER_INT) {
+ mon_text_read_intstat(rp, &ptr, ep);
+ } else {
+ mon_text_read_statset(rp, &ptr, ep);
+ }
+ ptr.cnt += snprintf(ptr.pbuf + ptr.cnt, ptr.limit - ptr.cnt,
+ " %d", ep->length);
+ mon_text_read_data(rp, &ptr, ep);
+
+ rp->printf_togo = ptr.cnt;
+ rp->printf_offset = 0;
+
+ kmem_cache_free(rp->e_slab, ep);
}
- ptr.cnt += snprintf(ptr.pbuf + ptr.cnt, ptr.limit - ptr.cnt,
- " %d", ep->length);
- mon_text_read_data(rp, &ptr, ep);
- if (copy_to_user(buf, rp->printf_buf, ptr.cnt))
- ptr.cnt = -EFAULT;
+ ret = mon_text_copy_to_user(rp, buf, nbytes);
mutex_unlock(&rp->printf_lock);
- kmem_cache_free(rp->e_slab, ep);
- return ptr.cnt;
+ return ret;
}
static struct mon_event_text *mon_text_read_wait(struct mon_reader_text *rp,
int vbus;
u8 devctl;
+ pm_runtime_get_sync(dev);
spin_lock_irqsave(&musb->lock, flags);
val = musb->a_wait_bcon;
vbus = musb_platform_get_vbus_status(musb);
vbus = 0;
}
spin_unlock_irqrestore(&musb->lock, flags);
+ pm_runtime_put_sync(dev);
return sprintf(buf, "Vbus %s, timeout %lu msec\n",
vbus ? "on" : "off", val);
musb_disable_interrupts(musb);
musb_writeb(musb->mregs, MUSB_DEVCTL, 0);
spin_unlock_irqrestore(&musb->lock, flags);
+ musb_platform_exit(musb);
pm_runtime_dont_use_autosuspend(musb->controller);
pm_runtime_put_sync(musb->controller);
pm_runtime_disable(musb->controller);
- musb_platform_exit(musb);
musb_phy_callback = NULL;
if (musb->dma_controller)
musb_dma_controller_destroy(musb->dma_controller);
return 0;
err = uas_configure_endpoints(devinfo);
- if (err && err != ENODEV)
+ if (err && err != -ENODEV)
shost_printk(KERN_ERR, shost,
"%s: alloc streams error %d after reset",
__func__, err);
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_BROKEN_FUA ),
+/* Reported by Teijo Kinnunen <teijo.kinnunen@code-q.fi> */
+UNUSUAL_DEV( 0x152d, 0x2567, 0x0117, 0x0117,
+ "JMicron",
+ "USB to ATA/ATAPI Bridge",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_BROKEN_FUA ),
+
/* Reported-by George Cherian <george.cherian@cavium.com> */
UNUSUAL_DEV(0x152d, 0x9561, 0x0000, 0x9999,
"JMicron",
chip->tcpm_port = tcpm_register_port(&client->dev, &chip->tcpc_dev);
if (IS_ERR(chip->tcpm_port)) {
ret = PTR_ERR(chip->tcpm_port);
- dev_err(dev, "cannot register tcpm port, ret=%d", ret);
+ if (ret != -EPROBE_DEFER)
+ dev_err(dev, "cannot register tcpm port, ret=%d", ret);
goto destroy_workqueue;
}
unsigned int nr_src_pdo;
u32 snk_pdo[PDO_MAX_OBJECTS];
unsigned int nr_snk_pdo;
- unsigned int nr_fixed; /* number of fixed sink PDOs */
- unsigned int nr_var; /* number of variable sink PDOs */
- unsigned int nr_batt; /* number of battery sink PDOs */
u32 snk_vdo[VDO_MAX_OBJECTS];
unsigned int nr_snk_vdo;
return 0;
}
-#define min_power(x, y) min(pdo_max_power(x), pdo_max_power(y))
-#define min_current(x, y) min(pdo_max_current(x), pdo_max_current(y))
-
-static int tcpm_pd_select_pdo(struct tcpm_port *port, int *sink_pdo,
- int *src_pdo)
+static int tcpm_pd_select_pdo(struct tcpm_port *port)
{
- unsigned int i, j, max_mw = 0, max_mv = 0, mw = 0, mv = 0, ma = 0;
+ unsigned int i, max_mw = 0, max_mv = 0;
int ret = -EINVAL;
/*
- * Select the source PDO providing the most power which has a
- * matchig sink cap.
+ * Select the source PDO providing the most power while staying within
+ * the board's voltage limits. Prefer PDO providing exp
*/
for (i = 0; i < port->nr_source_caps; i++) {
u32 pdo = port->source_caps[i];
enum pd_pdo_type type = pdo_type(pdo);
+ unsigned int mv, ma, mw;
- if (type == PDO_TYPE_FIXED) {
- for (j = 0; j < port->nr_fixed; j++) {
- if (pdo_fixed_voltage(pdo) ==
- pdo_fixed_voltage(port->snk_pdo[j])) {
- ma = min_current(pdo, port->snk_pdo[j]);
- mv = pdo_fixed_voltage(pdo);
- mw = ma * mv / 1000;
- if (mw > max_mw ||
- (mw == max_mw && mv > max_mv)) {
- ret = 0;
- *src_pdo = i;
- *sink_pdo = j;
- max_mw = mw;
- max_mv = mv;
- }
- /* There could only be one fixed pdo
- * at a specific voltage level.
- * So breaking here.
- */
- break;
- }
- }
- } else if (type == PDO_TYPE_BATT) {
- for (j = port->nr_fixed;
- j < port->nr_fixed +
- port->nr_batt;
- j++) {
- if (pdo_min_voltage(pdo) >=
- pdo_min_voltage(port->snk_pdo[j]) &&
- pdo_max_voltage(pdo) <=
- pdo_max_voltage(port->snk_pdo[j])) {
- mw = min_power(pdo, port->snk_pdo[j]);
- mv = pdo_min_voltage(pdo);
- if (mw > max_mw ||
- (mw == max_mw && mv > max_mv)) {
- ret = 0;
- *src_pdo = i;
- *sink_pdo = j;
- max_mw = mw;
- max_mv = mv;
- }
- }
- }
- } else if (type == PDO_TYPE_VAR) {
- for (j = port->nr_fixed +
- port->nr_batt;
- j < port->nr_fixed +
- port->nr_batt +
- port->nr_var;
- j++) {
- if (pdo_min_voltage(pdo) >=
- pdo_min_voltage(port->snk_pdo[j]) &&
- pdo_max_voltage(pdo) <=
- pdo_max_voltage(port->snk_pdo[j])) {
- ma = min_current(pdo, port->snk_pdo[j]);
- mv = pdo_min_voltage(pdo);
- mw = ma * mv / 1000;
- if (mw > max_mw ||
- (mw == max_mw && mv > max_mv)) {
- ret = 0;
- *src_pdo = i;
- *sink_pdo = j;
- max_mw = mw;
- max_mv = mv;
- }
- }
- }
+ if (type == PDO_TYPE_FIXED)
+ mv = pdo_fixed_voltage(pdo);
+ else
+ mv = pdo_min_voltage(pdo);
+
+ if (type == PDO_TYPE_BATT) {
+ mw = pdo_max_power(pdo);
+ } else {
+ ma = min(pdo_max_current(pdo),
+ port->max_snk_ma);
+ mw = ma * mv / 1000;
+ }
+
+ /* Perfer higher voltages if available */
+ if ((mw > max_mw || (mw == max_mw && mv > max_mv)) &&
+ mv <= port->max_snk_mv) {
+ ret = i;
+ max_mw = mw;
+ max_mv = mv;
}
}
unsigned int mv, ma, mw, flags;
unsigned int max_ma, max_mw;
enum pd_pdo_type type;
- int src_pdo_index, snk_pdo_index;
- u32 pdo, matching_snk_pdo;
+ int index;
+ u32 pdo;
- if (tcpm_pd_select_pdo(port, &snk_pdo_index, &src_pdo_index) < 0)
+ index = tcpm_pd_select_pdo(port);
+ if (index < 0)
return -EINVAL;
-
- pdo = port->source_caps[src_pdo_index];
- matching_snk_pdo = port->snk_pdo[snk_pdo_index];
+ pdo = port->source_caps[index];
type = pdo_type(pdo);
if (type == PDO_TYPE_FIXED)
else
mv = pdo_min_voltage(pdo);
- /* Select maximum available current within the sink pdo's limit */
+ /* Select maximum available current within the board's power limit */
if (type == PDO_TYPE_BATT) {
- mw = min_power(pdo, matching_snk_pdo);
- ma = 1000 * mw / mv;
+ mw = pdo_max_power(pdo);
+ ma = 1000 * min(mw, port->max_snk_mw) / mv;
} else {
- ma = min_current(pdo, matching_snk_pdo);
- mw = ma * mv / 1000;
+ ma = min(pdo_max_current(pdo),
+ 1000 * port->max_snk_mw / mv);
}
+ ma = min(ma, port->max_snk_ma);
flags = RDO_USB_COMM | RDO_NO_SUSPEND;
/* Set mismatch bit if offered power is less than operating power */
+ mw = ma * mv / 1000;
max_ma = ma;
max_mw = mw;
if (mw < port->operating_snk_mw) {
flags |= RDO_CAP_MISMATCH;
- if (type == PDO_TYPE_BATT &&
- (pdo_max_power(matching_snk_pdo) > pdo_max_power(pdo)))
- max_mw = pdo_max_power(matching_snk_pdo);
- else if (pdo_max_current(matching_snk_pdo) >
- pdo_max_current(pdo))
- max_ma = pdo_max_current(matching_snk_pdo);
+ max_mw = port->operating_snk_mw;
+ max_ma = max_mw * 1000 / mv;
}
tcpm_log(port, "cc=%d cc1=%d cc2=%d vbus=%d vconn=%s polarity=%d",
port->polarity);
if (type == PDO_TYPE_BATT) {
- *rdo = RDO_BATT(src_pdo_index + 1, mw, max_mw, flags);
+ *rdo = RDO_BATT(index + 1, mw, max_mw, flags);
tcpm_log(port, "Requesting PDO %d: %u mV, %u mW%s",
- src_pdo_index, mv, mw,
+ index, mv, mw,
flags & RDO_CAP_MISMATCH ? " [mismatch]" : "");
} else {
- *rdo = RDO_FIXED(src_pdo_index + 1, ma, max_ma, flags);
+ *rdo = RDO_FIXED(index + 1, ma, max_ma, flags);
tcpm_log(port, "Requesting PDO %d: %u mV, %u mA%s",
- src_pdo_index, mv, ma,
+ index, mv, ma,
flags & RDO_CAP_MISMATCH ? " [mismatch]" : "");
}
}
EXPORT_SYMBOL_GPL(tcpm_update_sink_capabilities);
-static int nr_type_pdos(const u32 *pdo, unsigned int nr_pdo,
- enum pd_pdo_type type)
-{
- int count = 0;
- int i;
-
- for (i = 0; i < nr_pdo; i++) {
- if (pdo_type(pdo[i]) == type)
- count++;
- }
- return count;
-}
-
struct tcpm_port *tcpm_register_port(struct device *dev, struct tcpc_dev *tcpc)
{
struct tcpm_port *port;
tcpc->config->nr_src_pdo);
port->nr_snk_pdo = tcpm_copy_pdos(port->snk_pdo, tcpc->config->snk_pdo,
tcpc->config->nr_snk_pdo);
- port->nr_fixed = nr_type_pdos(port->snk_pdo,
- port->nr_snk_pdo,
- PDO_TYPE_FIXED);
- port->nr_var = nr_type_pdos(port->snk_pdo,
- port->nr_snk_pdo,
- PDO_TYPE_VAR);
- port->nr_batt = nr_type_pdos(port->snk_pdo,
- port->nr_snk_pdo,
- PDO_TYPE_BATT);
port->nr_snk_vdo = tcpm_copy_vdos(port->snk_vdo, tcpc->config->snk_vdo,
tcpc->config->nr_snk_vdo);
if (rv != 0)
return -EINVAL;
+ if (!udc) {
+ dev_err(dev, "no device");
+ return -ENODEV;
+ }
spin_lock_irqsave(&udc->lock, flags);
/* Don't export what we don't have */
- if (!udc || !udc->driver || !udc->pullup) {
- dev_err(dev, "no device or gadget not bound");
+ if (!udc->driver || !udc->pullup) {
+ dev_err(dev, "gadget not bound");
ret = -ENODEV;
goto unlock;
}
if (nvq->rx_ring && !vhost_net_buf_is_empty(rxq)) {
ptr_ring_unconsume(nvq->rx_ring, rxq->queue + rxq->head,
vhost_net_buf_get_size(rxq),
- __skb_array_destroy_skb);
+ tun_ptr_free);
rxq->head = rxq->tail = 0;
}
}
n->vqs[i].done_idx = 0;
n->vqs[i].vhost_hlen = 0;
n->vqs[i].sock_hlen = 0;
+ n->vqs[i].rx_ring = NULL;
vhost_net_buf_init(&n->vqs[i].rxq);
}
vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX);
vhost_net_disable_vq(n, vq);
vq->private_data = NULL;
vhost_net_buf_unproduce(nvq);
+ nvq->rx_ring = NULL;
mutex_unlock(&vq->mutex);
return sock;
}
vhost_net_disable_vq(n, vq);
vq->private_data = sock;
vhost_net_buf_unproduce(nvq);
- if (index == VHOST_NET_VQ_RX)
- nvq->rx_ring = get_tap_ptr_ring(fd);
r = vhost_vq_init_access(vq);
if (r)
goto err_used;
r = vhost_net_enable_vq(n, vq);
if (r)
goto err_used;
+ if (index == VHOST_NET_VQ_RX)
+ nvq->rx_ring = get_tap_ptr_ring(fd);
oldubufs = nvq->ubufs;
nvq->ubufs = ubufs;
unsigned char __user *ured;
unsigned char __user *ugreen;
unsigned char __user *ublue;
- int index, count, i;
+ unsigned int index, count, i;
if (get_user(index, &c->index) ||
__get_user(count, &c->count) ||
unsigned char __user *ugreen;
unsigned char __user *ublue;
struct fb_cmap *cmap = &info->cmap;
- int index, count, i;
+ unsigned int index, count, i;
u8 red, green, blue;
if (get_user(index, &c->index) ||
i = virtio16_to_cpu(_vq->vdev, vq->vring.desc[i].next);
}
- vq->vq.num_free += total_sg;
-
if (indirect)
kfree(desc);
char c;
if (get_user(c, buf + i))
return -EFAULT;
- expect_close = (c == 'V');
+ if (c == 'V')
+ expect_close = true;
}
/* Properly order writes across fork()ed processes */
#include <linux/types.h>
#include <linux/uaccess.h>
#include <linux/watchdog.h>
-#ifdef CONFIG_HPWDT_NMI_DECODING
-#include <linux/dmi.h>
-#include <linux/spinlock.h>
-#include <linux/nmi.h>
-#include <linux/kdebug.h>
-#include <linux/notifier.h>
-#include <asm/set_memory.h>
-#endif /* CONFIG_HPWDT_NMI_DECODING */
#include <asm/nmi.h>
-#include <asm/frame.h>
#define HPWDT_VERSION "1.4.0"
#define SECS_TO_TICKS(secs) ((secs) * 1000 / 128)
static unsigned int soft_margin = DEFAULT_MARGIN; /* in seconds */
static unsigned int reload; /* the computed soft_margin */
static bool nowayout = WATCHDOG_NOWAYOUT;
+#ifdef CONFIG_HPWDT_NMI_DECODING
+static unsigned int allow_kdump = 1;
+#endif
static char expect_release;
static unsigned long hpwdt_is_open;
};
MODULE_DEVICE_TABLE(pci, hpwdt_devices);
-#ifdef CONFIG_HPWDT_NMI_DECODING
-#define PCI_BIOS32_SD_VALUE 0x5F32335F /* "_32_" */
-#define CRU_BIOS_SIGNATURE_VALUE 0x55524324
-#define PCI_BIOS32_PARAGRAPH_LEN 16
-#define PCI_ROM_BASE1 0x000F0000
-#define ROM_SIZE 0x10000
-
-struct bios32_service_dir {
- u32 signature;
- u32 entry_point;
- u8 revision;
- u8 length;
- u8 checksum;
- u8 reserved[5];
-};
-
-/* type 212 */
-struct smbios_cru64_info {
- u8 type;
- u8 byte_length;
- u16 handle;
- u32 signature;
- u64 physical_address;
- u32 double_length;
- u32 double_offset;
-};
-#define SMBIOS_CRU64_INFORMATION 212
-
-/* type 219 */
-struct smbios_proliant_info {
- u8 type;
- u8 byte_length;
- u16 handle;
- u32 power_features;
- u32 omega_features;
- u32 reserved;
- u32 misc_features;
-};
-#define SMBIOS_ICRU_INFORMATION 219
-
-
-struct cmn_registers {
- union {
- struct {
- u8 ral;
- u8 rah;
- u16 rea2;
- };
- u32 reax;
- } u1;
- union {
- struct {
- u8 rbl;
- u8 rbh;
- u8 reb2l;
- u8 reb2h;
- };
- u32 rebx;
- } u2;
- union {
- struct {
- u8 rcl;
- u8 rch;
- u16 rec2;
- };
- u32 recx;
- } u3;
- union {
- struct {
- u8 rdl;
- u8 rdh;
- u16 red2;
- };
- u32 redx;
- } u4;
-
- u32 resi;
- u32 redi;
- u16 rds;
- u16 res;
- u32 reflags;
-} __attribute__((packed));
-
-static unsigned int hpwdt_nmi_decoding;
-static unsigned int allow_kdump = 1;
-static unsigned int is_icru;
-static unsigned int is_uefi;
-static DEFINE_SPINLOCK(rom_lock);
-static void *cru_rom_addr;
-static struct cmn_registers cmn_regs;
-
-extern asmlinkage void asminline_call(struct cmn_registers *pi86Regs,
- unsigned long *pRomEntry);
-
-#ifdef CONFIG_X86_32
-/* --32 Bit Bios------------------------------------------------------------ */
-
-#define HPWDT_ARCH 32
-
-asm(".text \n\t"
- ".align 4 \n\t"
- ".globl asminline_call \n"
- "asminline_call: \n\t"
- "pushl %ebp \n\t"
- "movl %esp, %ebp \n\t"
- "pusha \n\t"
- "pushf \n\t"
- "push %es \n\t"
- "push %ds \n\t"
- "pop %es \n\t"
- "movl 8(%ebp),%eax \n\t"
- "movl 4(%eax),%ebx \n\t"
- "movl 8(%eax),%ecx \n\t"
- "movl 12(%eax),%edx \n\t"
- "movl 16(%eax),%esi \n\t"
- "movl 20(%eax),%edi \n\t"
- "movl (%eax),%eax \n\t"
- "push %cs \n\t"
- "call *12(%ebp) \n\t"
- "pushf \n\t"
- "pushl %eax \n\t"
- "movl 8(%ebp),%eax \n\t"
- "movl %ebx,4(%eax) \n\t"
- "movl %ecx,8(%eax) \n\t"
- "movl %edx,12(%eax) \n\t"
- "movl %esi,16(%eax) \n\t"
- "movl %edi,20(%eax) \n\t"
- "movw %ds,24(%eax) \n\t"
- "movw %es,26(%eax) \n\t"
- "popl %ebx \n\t"
- "movl %ebx,(%eax) \n\t"
- "popl %ebx \n\t"
- "movl %ebx,28(%eax) \n\t"
- "pop %es \n\t"
- "popf \n\t"
- "popa \n\t"
- "leave \n\t"
- "ret \n\t"
- ".previous");
-
-
-/*
- * cru_detect
- *
- * Routine Description:
- * This function uses the 32-bit BIOS Service Directory record to
- * search for a $CRU record.
- *
- * Return Value:
- * 0 : SUCCESS
- * <0 : FAILURE
- */
-static int cru_detect(unsigned long map_entry,
- unsigned long map_offset)
-{
- void *bios32_map;
- unsigned long *bios32_entrypoint;
- unsigned long cru_physical_address;
- unsigned long cru_length;
- unsigned long physical_bios_base = 0;
- unsigned long physical_bios_offset = 0;
- int retval = -ENODEV;
-
- bios32_map = ioremap(map_entry, (2 * PAGE_SIZE));
-
- if (bios32_map == NULL)
- return -ENODEV;
-
- bios32_entrypoint = bios32_map + map_offset;
-
- cmn_regs.u1.reax = CRU_BIOS_SIGNATURE_VALUE;
-
- set_memory_x((unsigned long)bios32_map, 2);
- asminline_call(&cmn_regs, bios32_entrypoint);
-
- if (cmn_regs.u1.ral != 0) {
- pr_warn("Call succeeded but with an error: 0x%x\n",
- cmn_regs.u1.ral);
- } else {
- physical_bios_base = cmn_regs.u2.rebx;
- physical_bios_offset = cmn_regs.u4.redx;
- cru_length = cmn_regs.u3.recx;
- cru_physical_address =
- physical_bios_base + physical_bios_offset;
-
- /* If the values look OK, then map it in. */
- if ((physical_bios_base + physical_bios_offset)) {
- cru_rom_addr =
- ioremap(cru_physical_address, cru_length);
- if (cru_rom_addr) {
- set_memory_x((unsigned long)cru_rom_addr & PAGE_MASK,
- (cru_length + PAGE_SIZE - 1) >> PAGE_SHIFT);
- retval = 0;
- }
- }
-
- pr_debug("CRU Base Address: 0x%lx\n", physical_bios_base);
- pr_debug("CRU Offset Address: 0x%lx\n", physical_bios_offset);
- pr_debug("CRU Length: 0x%lx\n", cru_length);
- pr_debug("CRU Mapped Address: %p\n", &cru_rom_addr);
- }
- iounmap(bios32_map);
- return retval;
-}
-
-/*
- * bios_checksum
- */
-static int bios_checksum(const char __iomem *ptr, int len)
-{
- char sum = 0;
- int i;
-
- /*
- * calculate checksum of size bytes. This should add up
- * to zero if we have a valid header.
- */
- for (i = 0; i < len; i++)
- sum += ptr[i];
-
- return ((sum == 0) && (len > 0));
-}
-
-/*
- * bios32_present
- *
- * Routine Description:
- * This function finds the 32-bit BIOS Service Directory
- *
- * Return Value:
- * 0 : SUCCESS
- * <0 : FAILURE
- */
-static int bios32_present(const char __iomem *p)
-{
- struct bios32_service_dir *bios_32_ptr;
- int length;
- unsigned long map_entry, map_offset;
-
- bios_32_ptr = (struct bios32_service_dir *) p;
-
- /*
- * Search for signature by checking equal to the swizzled value
- * instead of calling another routine to perform a strcmp.
- */
- if (bios_32_ptr->signature == PCI_BIOS32_SD_VALUE) {
- length = bios_32_ptr->length * PCI_BIOS32_PARAGRAPH_LEN;
- if (bios_checksum(p, length)) {
- /*
- * According to the spec, we're looking for the
- * first 4KB-aligned address below the entrypoint
- * listed in the header. The Service Directory code
- * is guaranteed to occupy no more than 2 4KB pages.
- */
- map_entry = bios_32_ptr->entry_point & ~(PAGE_SIZE - 1);
- map_offset = bios_32_ptr->entry_point - map_entry;
-
- return cru_detect(map_entry, map_offset);
- }
- }
- return -ENODEV;
-}
-
-static int detect_cru_service(void)
-{
- char __iomem *p, *q;
- int rc = -1;
-
- /*
- * Search from 0x0f0000 through 0x0fffff, inclusive.
- */
- p = ioremap(PCI_ROM_BASE1, ROM_SIZE);
- if (p == NULL)
- return -ENOMEM;
-
- for (q = p; q < p + ROM_SIZE; q += 16) {
- rc = bios32_present(q);
- if (!rc)
- break;
- }
- iounmap(p);
- return rc;
-}
-/* ------------------------------------------------------------------------- */
-#endif /* CONFIG_X86_32 */
-#ifdef CONFIG_X86_64
-/* --64 Bit Bios------------------------------------------------------------ */
-
-#define HPWDT_ARCH 64
-
-asm(".text \n\t"
- ".align 4 \n\t"
- ".globl asminline_call \n\t"
- ".type asminline_call, @function \n\t"
- "asminline_call: \n\t"
- FRAME_BEGIN
- "pushq %rax \n\t"
- "pushq %rbx \n\t"
- "pushq %rdx \n\t"
- "pushq %r12 \n\t"
- "pushq %r9 \n\t"
- "movq %rsi, %r12 \n\t"
- "movq %rdi, %r9 \n\t"
- "movl 4(%r9),%ebx \n\t"
- "movl 8(%r9),%ecx \n\t"
- "movl 12(%r9),%edx \n\t"
- "movl 16(%r9),%esi \n\t"
- "movl 20(%r9),%edi \n\t"
- "movl (%r9),%eax \n\t"
- "call *%r12 \n\t"
- "pushfq \n\t"
- "popq %r12 \n\t"
- "movl %eax, (%r9) \n\t"
- "movl %ebx, 4(%r9) \n\t"
- "movl %ecx, 8(%r9) \n\t"
- "movl %edx, 12(%r9) \n\t"
- "movl %esi, 16(%r9) \n\t"
- "movl %edi, 20(%r9) \n\t"
- "movq %r12, %rax \n\t"
- "movl %eax, 28(%r9) \n\t"
- "popq %r9 \n\t"
- "popq %r12 \n\t"
- "popq %rdx \n\t"
- "popq %rbx \n\t"
- "popq %rax \n\t"
- FRAME_END
- "ret \n\t"
- ".previous");
-
-/*
- * dmi_find_cru
- *
- * Routine Description:
- * This function checks whether or not a SMBIOS/DMI record is
- * the 64bit CRU info or not
- */
-static void dmi_find_cru(const struct dmi_header *dm, void *dummy)
-{
- struct smbios_cru64_info *smbios_cru64_ptr;
- unsigned long cru_physical_address;
-
- if (dm->type == SMBIOS_CRU64_INFORMATION) {
- smbios_cru64_ptr = (struct smbios_cru64_info *) dm;
- if (smbios_cru64_ptr->signature == CRU_BIOS_SIGNATURE_VALUE) {
- cru_physical_address =
- smbios_cru64_ptr->physical_address +
- smbios_cru64_ptr->double_offset;
- cru_rom_addr = ioremap(cru_physical_address,
- smbios_cru64_ptr->double_length);
- set_memory_x((unsigned long)cru_rom_addr & PAGE_MASK,
- smbios_cru64_ptr->double_length >> PAGE_SHIFT);
- }
- }
-}
-
-static int detect_cru_service(void)
-{
- cru_rom_addr = NULL;
-
- dmi_walk(dmi_find_cru, NULL);
-
- /* if cru_rom_addr has been set then we found a CRU service */
- return ((cru_rom_addr != NULL) ? 0 : -ENODEV);
-}
-/* ------------------------------------------------------------------------- */
-#endif /* CONFIG_X86_64 */
-#endif /* CONFIG_HPWDT_NMI_DECODING */
/*
* Watchdog operations
*/
static int hpwdt_pretimeout(unsigned int ulReason, struct pt_regs *regs)
{
- unsigned long rom_pl;
- static int die_nmi_called;
-
- if (!hpwdt_nmi_decoding)
- return NMI_DONE;
-
if ((ulReason == NMI_UNKNOWN) && !hpwdt_my_nmi())
return NMI_DONE;
- spin_lock_irqsave(&rom_lock, rom_pl);
- if (!die_nmi_called && !is_icru && !is_uefi)
- asminline_call(&cmn_regs, cru_rom_addr);
- die_nmi_called = 1;
- spin_unlock_irqrestore(&rom_lock, rom_pl);
-
if (allow_kdump)
hpwdt_stop();
- if (!is_icru && !is_uefi) {
- if (cmn_regs.u1.ral == 0) {
- nmi_panic(regs, "An NMI occurred, but unable to determine source.\n");
- return NMI_HANDLED;
- }
- }
nmi_panic(regs, "An NMI occurred. Depending on your system the reason "
"for the NMI is logged in any one of the following "
"resources:\n"
* Init & Exit
*/
-#ifdef CONFIG_HPWDT_NMI_DECODING
-#ifdef CONFIG_X86_LOCAL_APIC
-static void hpwdt_check_nmi_decoding(struct pci_dev *dev)
-{
- /*
- * If nmi_watchdog is turned off then we can turn on
- * our nmi decoding capability.
- */
- hpwdt_nmi_decoding = 1;
-}
-#else
-static void hpwdt_check_nmi_decoding(struct pci_dev *dev)
-{
- dev_warn(&dev->dev, "NMI decoding is disabled. "
- "Your kernel does not support a NMI Watchdog.\n");
-}
-#endif /* CONFIG_X86_LOCAL_APIC */
-
-/*
- * dmi_find_icru
- *
- * Routine Description:
- * This function checks whether or not we are on an iCRU-based server.
- * This check is independent of architecture and needs to be made for
- * any ProLiant system.
- */
-static void dmi_find_icru(const struct dmi_header *dm, void *dummy)
-{
- struct smbios_proliant_info *smbios_proliant_ptr;
-
- if (dm->type == SMBIOS_ICRU_INFORMATION) {
- smbios_proliant_ptr = (struct smbios_proliant_info *) dm;
- if (smbios_proliant_ptr->misc_features & 0x01)
- is_icru = 1;
- if (smbios_proliant_ptr->misc_features & 0x1400)
- is_uefi = 1;
- }
-}
static int hpwdt_init_nmi_decoding(struct pci_dev *dev)
{
+#ifdef CONFIG_HPWDT_NMI_DECODING
int retval;
-
- /*
- * On typical CRU-based systems we need to map that service in
- * the BIOS. For 32 bit Operating Systems we need to go through
- * the 32 Bit BIOS Service Directory. For 64 bit Operating
- * Systems we get that service through SMBIOS.
- *
- * On systems that support the new iCRU service all we need to
- * do is call dmi_walk to get the supported flag value and skip
- * the old cru detect code.
- */
- dmi_walk(dmi_find_icru, NULL);
- if (!is_icru && !is_uefi) {
-
- /*
- * We need to map the ROM to get the CRU service.
- * For 32 bit Operating Systems we need to go through the 32 Bit
- * BIOS Service Directory
- * For 64 bit Operating Systems we get that service through SMBIOS.
- */
- retval = detect_cru_service();
- if (retval < 0) {
- dev_warn(&dev->dev,
- "Unable to detect the %d Bit CRU Service.\n",
- HPWDT_ARCH);
- return retval;
- }
-
- /*
- * We know this is the only CRU call we need to make so lets keep as
- * few instructions as possible once the NMI comes in.
- */
- cmn_regs.u1.rah = 0x0D;
- cmn_regs.u1.ral = 0x02;
- }
-
/*
* Only one function can register for NMI_UNKNOWN
*/
dev_warn(&dev->dev,
"Unable to register a die notifier (err=%d).\n",
retval);
- if (cru_rom_addr)
- iounmap(cru_rom_addr);
return retval;
+#endif /* CONFIG_HPWDT_NMI_DECODING */
+ return 0;
}
static void hpwdt_exit_nmi_decoding(void)
{
+#ifdef CONFIG_HPWDT_NMI_DECODING
unregister_nmi_handler(NMI_UNKNOWN, "hpwdt");
unregister_nmi_handler(NMI_SERR, "hpwdt");
unregister_nmi_handler(NMI_IO_CHECK, "hpwdt");
- if (cru_rom_addr)
- iounmap(cru_rom_addr);
-}
-#else /* !CONFIG_HPWDT_NMI_DECODING */
-static void hpwdt_check_nmi_decoding(struct pci_dev *dev)
-{
-}
-
-static int hpwdt_init_nmi_decoding(struct pci_dev *dev)
-{
- return 0;
+#endif
}
-static void hpwdt_exit_nmi_decoding(void)
-{
-}
-#endif /* CONFIG_HPWDT_NMI_DECODING */
-
static int hpwdt_init_one(struct pci_dev *dev,
const struct pci_device_id *ent)
{
int retval;
- /*
- * Check if we can do NMI decoding or not
- */
- hpwdt_check_nmi_decoding(dev);
-
/*
* First let's find out if we are on an iLO2+ server. We will
* not run on a legacy ASM box.
#ifdef CONFIG_HPWDT_NMI_DECODING
module_param(allow_kdump, int, 0);
MODULE_PARM_DESC(allow_kdump, "Start a kernel dump after NMI occurs");
-#endif /* !CONFIG_HPWDT_NMI_DECODING */
+#endif /* CONFIG_HPWDT_NMI_DECODING */
module_pci_driver(hpwdt_driver);
*/
#include <linux/io.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
!(readl(gwdt->control_base + SBSA_GWDT_WCS) & SBSA_GWDT_WCS_WS0))
timeleft += readl(gwdt->control_base + SBSA_GWDT_WOR);
- timeleft += readq(gwdt->control_base + SBSA_GWDT_WCV) -
+ timeleft += lo_hi_readq(gwdt->control_base + SBSA_GWDT_WCV) -
arch_counter_get_cntvct();
do_div(timeleft, gwdt->clk);
memset(&r, 0, sizeof(r));
r.start = gas->address;
- r.end = r.start + gas->access_width;
+ r.end = r.start + gas->access_width - 1;
if (gas->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY) {
r.flags = IORESOURCE_MEM;
} else if (gas->space_id == ACPI_ADR_SPACE_SYSTEM_IO) {
/* Register with generic device framework. */
err = device_register(&xendev->dev);
- if (err)
+ if (err) {
+ put_device(&xendev->dev);
+ xendev = NULL;
goto fail;
+ }
return 0;
fail:
bool ret_reply0; /* T if should return reply[0] on success */
bool upgrade; /* T to request service upgrade */
u16 service_id; /* Actual service ID (after upgrade) */
+ unsigned int debug_id; /* Trace ID */
u32 operation_ID; /* operation ID for an incoming call */
u32 count; /* count for use in unmarshalling */
__be32 tmp; /* place to extract temporary data */
call->type = type;
call->net = net;
+ call->debug_id = atomic_inc_return(&rxrpc_debug_id);
atomic_set(&call->usage, 1);
INIT_WORK(&call->async_work, afs_process_async_call);
init_waitqueue_head(&call->waitq);
afs_put_server(call->net, call->cm_server);
afs_put_cb_interest(call->net, call->cbi);
kfree(call->request);
- kfree(call);
- o = atomic_dec_return(&net->nr_outstanding_calls);
trace_afs_call(call, afs_call_trace_free, 0, o,
__builtin_return_address(0));
+ kfree(call);
+
+ o = atomic_dec_return(&net->nr_outstanding_calls);
if (o == 0)
wake_up_atomic_t(&net->nr_outstanding_calls);
}
(async ?
afs_wake_up_async_call :
afs_wake_up_call_waiter),
- call->upgrade);
+ call->upgrade,
+ call->debug_id);
if (IS_ERR(rxcall)) {
ret = PTR_ERR(rxcall);
goto error_kill_call;
afs_wake_up_async_call,
afs_rx_attach,
(unsigned long)call,
- GFP_KERNEL) < 0)
+ GFP_KERNEL,
+ call->debug_id) < 0)
break;
call = NULL;
}
#define AIO_RING_PAGES 8
struct kioctx_table {
- struct rcu_head rcu;
- unsigned nr;
- struct kioctx *table[];
+ struct rcu_head rcu;
+ unsigned nr;
+ struct kioctx __rcu *table[];
};
struct kioctx_cpu {
struct page **ring_pages;
long nr_pages;
- struct work_struct free_work;
+ struct rcu_head free_rcu;
+ struct work_struct free_work; /* see free_ioctx() */
/*
* signals when all in-flight requests are done
for (i = 0; i < table->nr; i++) {
struct kioctx *ctx;
- ctx = table->table[i];
+ ctx = rcu_dereference(table->table[i]);
if (ctx && ctx->aio_ring_file == file) {
if (!atomic_read(&ctx->dead)) {
ctx->user_id = ctx->mmap_base = vma->vm_start;
return cancel(&kiocb->common);
}
+/*
+ * free_ioctx() should be RCU delayed to synchronize against the RCU
+ * protected lookup_ioctx() and also needs process context to call
+ * aio_free_ring(), so the double bouncing through kioctx->free_rcu and
+ * ->free_work.
+ */
static void free_ioctx(struct work_struct *work)
{
struct kioctx *ctx = container_of(work, struct kioctx, free_work);
kmem_cache_free(kioctx_cachep, ctx);
}
+static void free_ioctx_rcufn(struct rcu_head *head)
+{
+ struct kioctx *ctx = container_of(head, struct kioctx, free_rcu);
+
+ INIT_WORK(&ctx->free_work, free_ioctx);
+ schedule_work(&ctx->free_work);
+}
+
static void free_ioctx_reqs(struct percpu_ref *ref)
{
struct kioctx *ctx = container_of(ref, struct kioctx, reqs);
if (ctx->rq_wait && atomic_dec_and_test(&ctx->rq_wait->count))
complete(&ctx->rq_wait->comp);
- INIT_WORK(&ctx->free_work, free_ioctx);
- schedule_work(&ctx->free_work);
+ /* Synchronize against RCU protected table->table[] dereferences */
+ call_rcu(&ctx->free_rcu, free_ioctx_rcufn);
}
/*
while (1) {
if (table)
for (i = 0; i < table->nr; i++)
- if (!table->table[i]) {
+ if (!rcu_access_pointer(table->table[i])) {
ctx->id = i;
- table->table[i] = ctx;
+ rcu_assign_pointer(table->table[i], ctx);
spin_unlock(&mm->ioctx_lock);
/* While kioctx setup is in progress,
}
table = rcu_dereference_raw(mm->ioctx_table);
- WARN_ON(ctx != table->table[ctx->id]);
- table->table[ctx->id] = NULL;
+ WARN_ON(ctx != rcu_access_pointer(table->table[ctx->id]));
+ RCU_INIT_POINTER(table->table[ctx->id], NULL);
spin_unlock(&mm->ioctx_lock);
- /* percpu_ref_kill() will do the necessary call_rcu() */
+ /* free_ioctx_reqs() will do the necessary RCU synchronization */
wake_up_all(&ctx->wait);
/*
skipped = 0;
for (i = 0; i < table->nr; ++i) {
- struct kioctx *ctx = table->table[i];
+ struct kioctx *ctx =
+ rcu_dereference_protected(table->table[i], true);
if (!ctx) {
skipped++;
if (!table || id >= table->nr)
goto out;
- ctx = table->table[id];
+ ctx = rcu_dereference(table->table[id]);
if (ctx && ctx->user_id == ctx_id) {
percpu_ref_get(&ctx->users);
ret = ctx;
if (!node)
break;
bytenr = node->val;
+ shared.share_count = 0;
cond_resched();
}
stripe_start = stripe->physical;
if (physical >= stripe_start &&
physical < stripe_start + rbio->stripe_len &&
+ stripe->dev->bdev &&
bio->bi_disk == stripe->dev->bdev->bd_disk &&
bio->bi_partno == stripe->dev->bdev->bd_partno) {
return i;
{
struct btrfs_fs_info *fs_info = to_fs_info(kobj);
- return snprintf(buf, PAGE_SIZE, "%u\n", fs_info->nodesize);
+ return snprintf(buf, PAGE_SIZE, "%u\n", fs_info->super_copy->nodesize);
}
BTRFS_ATTR(, nodesize, btrfs_nodesize_show);
{
struct btrfs_fs_info *fs_info = to_fs_info(kobj);
- return snprintf(buf, PAGE_SIZE, "%u\n", fs_info->sectorsize);
+ return snprintf(buf, PAGE_SIZE, "%u\n",
+ fs_info->super_copy->sectorsize);
}
BTRFS_ATTR(, sectorsize, btrfs_sectorsize_show);
{
struct btrfs_fs_info *fs_info = to_fs_info(kobj);
- return snprintf(buf, PAGE_SIZE, "%u\n", fs_info->sectorsize);
+ return snprintf(buf, PAGE_SIZE, "%u\n",
+ fs_info->super_copy->sectorsize);
}
BTRFS_ATTR(, clone_alignment, btrfs_clone_alignment_show);
super = fs_info->super_copy;
- /* update latest btrfs_super_block::chunk_root refs */
root_item = &fs_info->chunk_root->root_item;
- btrfs_set_super_chunk_root(super, root_item->bytenr);
- btrfs_set_super_chunk_root_generation(super, root_item->generation);
- btrfs_set_super_chunk_root_level(super, root_item->level);
+ super->chunk_root = root_item->bytenr;
+ super->chunk_root_generation = root_item->generation;
+ super->chunk_root_level = root_item->level;
- /* update latest btrfs_super_block::root refs */
root_item = &fs_info->tree_root->root_item;
- btrfs_set_super_root(super, root_item->bytenr);
- btrfs_set_super_generation(super, root_item->generation);
- btrfs_set_super_root_level(super, root_item->level);
-
+ super->root = root_item->bytenr;
+ super->generation = root_item->generation;
+ super->root_level = root_item->level;
if (btrfs_test_opt(fs_info, SPACE_CACHE))
- btrfs_set_super_cache_generation(super, root_item->generation);
+ super->cache_generation = root_item->generation;
if (test_bit(BTRFS_FS_UPDATE_UUID_TREE_GEN, &fs_info->flags))
- btrfs_set_super_uuid_tree_generation(super,
- root_item->generation);
+ super->uuid_tree_generation = root_item->generation;
}
int btrfs_transaction_in_commit(struct btrfs_fs_info *info)
spin_unlock(&parent->d_lock);
goto again;
}
- rcu_read_unlock();
- if (parent != dentry)
+ if (parent != dentry) {
spin_lock_nested(&dentry->d_lock, DENTRY_D_LOCK_NESTED);
- else
+ if (unlikely(dentry->d_lockref.count < 0)) {
+ spin_unlock(&parent->d_lock);
+ parent = NULL;
+ }
+ } else {
parent = NULL;
+ }
+ rcu_read_unlock();
return parent;
}
retry:
rcu_read_lock();
- seq = smp_load_acquire(&parent->d_inode->i_dir_seq) & ~1;
+ seq = smp_load_acquire(&parent->d_inode->i_dir_seq);
r_seq = read_seqbegin(&rename_lock);
dentry = __d_lookup_rcu(parent, name, &d_seq);
if (unlikely(dentry)) {
rcu_read_unlock();
goto retry;
}
+
+ if (unlikely(seq & 1)) {
+ rcu_read_unlock();
+ goto retry;
+ }
+
hlist_bl_lock(b);
- if (unlikely(parent->d_inode->i_dir_seq != seq)) {
+ if (unlikely(READ_ONCE(parent->d_inode->i_dir_seq) != seq)) {
hlist_bl_unlock(b);
rcu_read_unlock();
goto retry;
iomap->length = hole_size(inode, lblock, &mp);
else
iomap->length = size - pos;
- } else {
- if (height <= ip->i_height)
- iomap->length = hole_size(inode, lblock, &mp);
}
goto out_release;
}
pagevec_reinit(pvec);
}
+/*
+ * Mask used when checking the page offset value passed in via system
+ * calls. This value will be converted to a loff_t which is signed.
+ * Therefore, we want to check the upper PAGE_SHIFT + 1 bits of the
+ * value. The extra bit (- 1 in the shift value) is to take the sign
+ * bit into account.
+ */
+#define PGOFF_LOFFT_MAX \
+ (((1UL << (PAGE_SHIFT + 1)) - 1) << (BITS_PER_LONG - (PAGE_SHIFT + 1)))
+
static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)
{
struct inode *inode = file_inode(file);
vma->vm_ops = &hugetlb_vm_ops;
/*
- * Offset passed to mmap (before page shift) could have been
- * negative when represented as a (l)off_t.
+ * page based offset in vm_pgoff could be sufficiently large to
+ * overflow a (l)off_t when converted to byte offset.
*/
- if (((loff_t)vma->vm_pgoff << PAGE_SHIFT) < 0)
+ if (vma->vm_pgoff & PGOFF_LOFFT_MAX)
return -EINVAL;
+ /* must be huge page aligned */
if (vma->vm_pgoff & (~huge_page_mask(h) >> PAGE_SHIFT))
return -EINVAL;
.exit = lockd_exit_net,
.id = &lockd_net_id,
.size = sizeof(struct lockd_net),
- .async = true,
};
static bool path_connected(const struct path *path)
{
struct vfsmount *mnt = path->mnt;
+ struct super_block *sb = mnt->mnt_sb;
- /* Only bind mounts can have disconnected paths */
- if (mnt->mnt_root == mnt->mnt_sb->s_root)
+ /* Bind mounts and multi-root filesystems can have disconnected paths */
+ if (!(sb->s_iflags & SB_I_MULTIROOT) && (mnt->mnt_root == sb->s_root))
return true;
return is_subdir(path->dentry, mnt->mnt_root);
struct nfs_direct_mirror mirrors[NFS_PAGEIO_DESCRIPTOR_MIRROR_MAX];
int mirror_count;
+ loff_t io_start; /* Start offset for I/O */
ssize_t count, /* bytes actually processed */
max_count, /* max expected count */
bytes_left, /* bytes left to be sent */
- io_start, /* start of IO */
error; /* any reported error */
struct completion completion; /* wait for i/o completion */
.exit = nfs_net_exit,
.id = &nfs_net_id,
.size = sizeof(struct nfs_net),
- .async = true,
};
/*
void
pnfs_put_layout_hdr(struct pnfs_layout_hdr *lo)
{
- struct inode *inode = lo->plh_inode;
+ struct inode *inode;
+ if (!lo)
+ return;
+ inode = lo->plh_inode;
pnfs_layoutreturn_before_put_layout_hdr(lo);
if (refcount_dec_and_lock(&lo->plh_refcount, &inode->i_lock)) {
spin_lock(&ino->i_lock);
lo = nfsi->layout;
if (!lo || !pnfs_layout_is_valid(lo) ||
- test_bit(NFS_LAYOUT_BULK_RECALL, &lo->plh_flags))
+ test_bit(NFS_LAYOUT_BULK_RECALL, &lo->plh_flags)) {
+ lo = NULL;
goto out_noroc;
+ }
+ pnfs_get_layout_hdr(lo);
if (test_bit(NFS_LAYOUT_RETURN_LOCK, &lo->plh_flags)) {
- pnfs_get_layout_hdr(lo);
spin_unlock(&ino->i_lock);
wait_on_bit(&lo->plh_flags, NFS_LAYOUT_RETURN,
TASK_UNINTERRUPTIBLE);
struct pnfs_layoutdriver_type *ld = NFS_SERVER(ino)->pnfs_curr_ld;
if (ld->prepare_layoutreturn)
ld->prepare_layoutreturn(args);
+ pnfs_put_layout_hdr(lo);
return true;
}
if (layoutreturn)
pnfs_send_layoutreturn(lo, &stateid, iomode, true);
+ pnfs_put_layout_hdr(lo);
return false;
}
/* initial superblock/root creation */
mount_info->fill_super(s, mount_info);
nfs_get_cache_cookie(s, mount_info->parsed, mount_info->cloned);
+ if (!(server->flags & NFS_MOUNT_UNSHARED))
+ s->s_iflags |= SB_I_MULTIROOT;
}
mntroot = nfs_get_root(s, mount_info->mntfh, dev_name);
return status;
}
-int nfs_commit_inode(struct inode *inode, int how)
+static int __nfs_commit_inode(struct inode *inode, int how,
+ struct writeback_control *wbc)
{
LIST_HEAD(head);
struct nfs_commit_info cinfo;
int may_wait = how & FLUSH_SYNC;
- int error = 0;
- int res;
+ int ret, nscan;
nfs_init_cinfo_from_inode(&cinfo, inode);
nfs_commit_begin(cinfo.mds);
- res = nfs_scan_commit(inode, &head, &cinfo);
- if (res)
- error = nfs_generic_commit_list(inode, &head, how, &cinfo);
+ for (;;) {
+ ret = nscan = nfs_scan_commit(inode, &head, &cinfo);
+ if (ret <= 0)
+ break;
+ ret = nfs_generic_commit_list(inode, &head, how, &cinfo);
+ if (ret < 0)
+ break;
+ ret = 0;
+ if (wbc && wbc->sync_mode == WB_SYNC_NONE) {
+ if (nscan < wbc->nr_to_write)
+ wbc->nr_to_write -= nscan;
+ else
+ wbc->nr_to_write = 0;
+ }
+ if (nscan < INT_MAX)
+ break;
+ cond_resched();
+ }
nfs_commit_end(cinfo.mds);
- if (res == 0)
- return res;
- if (error < 0)
- goto out_error;
- if (!may_wait)
- goto out_mark_dirty;
- error = wait_on_commit(cinfo.mds);
- if (error < 0)
- return error;
- return res;
-out_error:
- res = error;
- /* Note: If we exit without ensuring that the commit is complete,
- * we must mark the inode as dirty. Otherwise, future calls to
- * sync_inode() with the WB_SYNC_ALL flag set will fail to ensure
- * that the data is on the disk.
- */
-out_mark_dirty:
- __mark_inode_dirty(inode, I_DIRTY_DATASYNC);
- return res;
+ if (ret || !may_wait)
+ return ret;
+ return wait_on_commit(cinfo.mds);
+}
+
+int nfs_commit_inode(struct inode *inode, int how)
+{
+ return __nfs_commit_inode(inode, how, NULL);
}
EXPORT_SYMBOL_GPL(nfs_commit_inode);
int flags = FLUSH_SYNC;
int ret = 0;
- /* no commits means nothing needs to be done */
- if (!atomic_long_read(&nfsi->commit_info.ncommit))
- return ret;
-
if (wbc->sync_mode == WB_SYNC_NONE) {
+ /* no commits means nothing needs to be done */
+ if (!atomic_long_read(&nfsi->commit_info.ncommit))
+ goto check_requests_outstanding;
+
/* Don't commit yet if this is a non-blocking flush and there
* are a lot of outstanding writes for this mapping.
*/
flags = 0;
}
- ret = nfs_commit_inode(inode, flags);
- if (ret >= 0) {
- if (wbc->sync_mode == WB_SYNC_NONE) {
- if (ret < wbc->nr_to_write)
- wbc->nr_to_write -= ret;
- else
- wbc->nr_to_write = 0;
- }
- return 0;
- }
+ ret = __nfs_commit_inode(inode, flags, wbc);
+ if (!ret) {
+ if (flags & FLUSH_SYNC)
+ return 0;
+ } else if (atomic_long_read(&nfsi->commit_info.ncommit))
+ goto out_mark_dirty;
+
+check_requests_outstanding:
+ if (!atomic_read(&nfsi->commit_info.rpcs_out))
+ return ret;
out_mark_dirty:
__mark_inode_dirty(inode, I_DIRTY_DATASYNC);
return ret;
.exit = grace_exit_net,
.id = &grace_net_id,
.size = sizeof(struct list_head),
- .async = true,
};
static int __init
kfree(nbl);
}
+static void
+remove_blocked_locks(struct nfs4_lockowner *lo)
+{
+ struct nfs4_client *clp = lo->lo_owner.so_client;
+ struct nfsd_net *nn = net_generic(clp->net, nfsd_net_id);
+ struct nfsd4_blocked_lock *nbl;
+ LIST_HEAD(reaplist);
+
+ /* Dequeue all blocked locks */
+ spin_lock(&nn->blocked_locks_lock);
+ while (!list_empty(&lo->lo_blocked)) {
+ nbl = list_first_entry(&lo->lo_blocked,
+ struct nfsd4_blocked_lock,
+ nbl_list);
+ list_del_init(&nbl->nbl_list);
+ list_move(&nbl->nbl_lru, &reaplist);
+ }
+ spin_unlock(&nn->blocked_locks_lock);
+
+ /* Now free them */
+ while (!list_empty(&reaplist)) {
+ nbl = list_first_entry(&reaplist, struct nfsd4_blocked_lock,
+ nbl_lru);
+ list_del_init(&nbl->nbl_lru);
+ posix_unblock_lock(&nbl->nbl_lock);
+ free_blocked_lock(nbl);
+ }
+}
+
static int
nfsd4_cb_notify_lock_done(struct nfsd4_callback *cb, struct rpc_task *task)
{
static void
__destroy_client(struct nfs4_client *clp)
{
+ int i;
struct nfs4_openowner *oo;
struct nfs4_delegation *dp;
struct list_head reaplist;
nfs4_get_stateowner(&oo->oo_owner);
release_openowner(oo);
}
+ for (i = 0; i < OWNER_HASH_SIZE; i++) {
+ struct nfs4_stateowner *so, *tmp;
+
+ list_for_each_entry_safe(so, tmp, &clp->cl_ownerstr_hashtbl[i],
+ so_strhash) {
+ /* Should be no openowners at this point */
+ WARN_ON_ONCE(so->so_is_open_owner);
+ remove_blocked_locks(lockowner(so));
+ }
+ }
nfsd4_return_all_client_layouts(clp);
nfsd4_shutdown_callback(clp);
if (clp->cl_cb_conn.cb_xprt)
}
spin_unlock(&clp->cl_lock);
free_ol_stateid_reaplist(&reaplist);
+ remove_blocked_locks(lo);
nfs4_put_stateowner(&lo->lo_owner);
return status;
}
}
+ WARN_ON(!list_empty(&nn->blocked_locks_lru));
+
for (i = 0; i < CLIENT_HASH_SIZE; i++) {
while (!list_empty(&nn->unconf_id_hashtbl[i])) {
clp = list_entry(nn->unconf_id_hashtbl[i].next, struct nfs4_client, cl_idhash);
struct nfs4_delegation *dp = NULL;
struct list_head *pos, *next, reaplist;
struct nfsd_net *nn = net_generic(net, nfsd_net_id);
- struct nfsd4_blocked_lock *nbl;
cancel_delayed_work_sync(&nn->laundromat_work);
locks_end_grace(&nn->nfsd4_manager);
nfs4_put_stid(&dp->dl_stid);
}
- BUG_ON(!list_empty(&reaplist));
- spin_lock(&nn->blocked_locks_lock);
- while (!list_empty(&nn->blocked_locks_lru)) {
- nbl = list_first_entry(&nn->blocked_locks_lru,
- struct nfsd4_blocked_lock, nbl_lru);
- list_move(&nbl->nbl_lru, &reaplist);
- list_del_init(&nbl->nbl_list);
- }
- spin_unlock(&nn->blocked_locks_lock);
-
- while (!list_empty(&reaplist)) {
- nbl = list_first_entry(&reaplist,
- struct nfsd4_blocked_lock, nbl_lru);
- list_del_init(&nbl->nbl_lru);
- posix_unblock_lock(&nbl->nbl_lock);
- free_blocked_lock(nbl);
- }
-
nfsd4_client_tracking_exit(net);
nfs4_state_destroy_net(net);
}
an overlay which has redirects on a kernel that doesn't support this
feature will have unexpected results.
+ If unsure, say N.
+
config OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW
bool "Overlayfs: follow redirects even if redirects are turned off"
default y
Disable this to get a possibly more secure configuration, but that
might not be backward compatible with previous kernels.
+ If backward compatibility is not an issue, then it is safe and
+ recommended to say N here.
+
For more information, see Documentation/filesystems/overlayfs.txt
+ If unsure, say Y.
+
config OVERLAY_FS_INDEX
bool "Overlayfs: turn on inodes index feature by default"
depends on OVERLAY_FS
That is, mounting an overlay which has an inodes index on a kernel
that doesn't support this feature will have unexpected results.
+ If unsure, say N.
+
config OVERLAY_FS_NFS_EXPORT
bool "Overlayfs: turn on NFS export feature by default"
depends on OVERLAY_FS
Note, that the NFS export feature is not backward compatible.
That is, mounting an overlay which has a full index on a kernel
that doesn't support this feature will have unexpected results.
+
+ Most users should say N here and enable this feature on a case-by-
+ case basis with the "nfs_export=on" mount option.
+
+ Say N unless you fully understand the consequences.
#include <linux/ratelimit.h>
#include "overlayfs.h"
+static int ovl_encode_maybe_copy_up(struct dentry *dentry)
+{
+ int err;
+
+ if (ovl_dentry_upper(dentry))
+ return 0;
+
+ err = ovl_want_write(dentry);
+ if (!err) {
+ err = ovl_copy_up(dentry);
+ ovl_drop_write(dentry);
+ }
+
+ if (err) {
+ pr_warn_ratelimited("overlayfs: failed to copy up on encode (%pd2, err=%i)\n",
+ dentry, err);
+ }
+
+ return err;
+}
+
+/*
+ * Before encoding a non-upper directory file handle from real layer N, we need
+ * to check if it will be possible to reconnect an overlay dentry from the real
+ * lower decoded dentry. This is done by following the overlay ancestry up to a
+ * "layer N connected" ancestor and verifying that all parents along the way are
+ * "layer N connectable". If an ancestor that is NOT "layer N connectable" is
+ * found, we need to copy up an ancestor, which is "layer N connectable", thus
+ * making that ancestor "layer N connected". For example:
+ *
+ * layer 1: /a
+ * layer 2: /a/b/c
+ *
+ * The overlay dentry /a is NOT "layer 2 connectable", because if dir /a is
+ * copied up and renamed, upper dir /a will be indexed by lower dir /a from
+ * layer 1. The dir /a from layer 2 will never be indexed, so the algorithm (*)
+ * in ovl_lookup_real_ancestor() will not be able to lookup a connected overlay
+ * dentry from the connected lower dentry /a/b/c.
+ *
+ * To avoid this problem on decode time, we need to copy up an ancestor of
+ * /a/b/c, which is "layer 2 connectable", on encode time. That ancestor is
+ * /a/b. After copy up (and index) of /a/b, it will become "layer 2 connected"
+ * and when the time comes to decode the file handle from lower dentry /a/b/c,
+ * ovl_lookup_real_ancestor() will find the indexed ancestor /a/b and decoding
+ * a connected overlay dentry will be accomplished.
+ *
+ * (*) the algorithm in ovl_lookup_real_ancestor() can be improved to lookup an
+ * entry /a in the lower layers above layer N and find the indexed dir /a from
+ * layer 1. If that improvement is made, then the check for "layer N connected"
+ * will need to verify there are no redirects in lower layers above N. In the
+ * example above, /a will be "layer 2 connectable". However, if layer 2 dir /a
+ * is a target of a layer 1 redirect, then /a will NOT be "layer 2 connectable":
+ *
+ * layer 1: /A (redirect = /a)
+ * layer 2: /a/b/c
+ */
+
+/* Return the lowest layer for encoding a connectable file handle */
+static int ovl_connectable_layer(struct dentry *dentry)
+{
+ struct ovl_entry *oe = OVL_E(dentry);
+
+ /* We can get overlay root from root of any layer */
+ if (dentry == dentry->d_sb->s_root)
+ return oe->numlower;
+
+ /*
+ * If it's an unindexed merge dir, then it's not connectable with any
+ * lower layer
+ */
+ if (ovl_dentry_upper(dentry) &&
+ !ovl_test_flag(OVL_INDEX, d_inode(dentry)))
+ return 0;
+
+ /* We can get upper/overlay path from indexed/lower dentry */
+ return oe->lowerstack[0].layer->idx;
+}
+
+/*
+ * @dentry is "connected" if all ancestors up to root or a "connected" ancestor
+ * have the same uppermost lower layer as the origin's layer. We may need to
+ * copy up a "connectable" ancestor to make it "connected". A "connected" dentry
+ * cannot become non "connected", so cache positive result in dentry flags.
+ *
+ * Return the connected origin layer or < 0 on error.
+ */
+static int ovl_connect_layer(struct dentry *dentry)
+{
+ struct dentry *next, *parent = NULL;
+ int origin_layer;
+ int err = 0;
+
+ if (WARN_ON(dentry == dentry->d_sb->s_root) ||
+ WARN_ON(!ovl_dentry_lower(dentry)))
+ return -EIO;
+
+ origin_layer = OVL_E(dentry)->lowerstack[0].layer->idx;
+ if (ovl_dentry_test_flag(OVL_E_CONNECTED, dentry))
+ return origin_layer;
+
+ /* Find the topmost origin layer connectable ancestor of @dentry */
+ next = dget(dentry);
+ for (;;) {
+ parent = dget_parent(next);
+ if (WARN_ON(parent == next)) {
+ err = -EIO;
+ break;
+ }
+
+ /*
+ * If @parent is not origin layer connectable, then copy up
+ * @next which is origin layer connectable and we are done.
+ */
+ if (ovl_connectable_layer(parent) < origin_layer) {
+ err = ovl_encode_maybe_copy_up(next);
+ break;
+ }
+
+ /* If @parent is connected or indexed we are done */
+ if (ovl_dentry_test_flag(OVL_E_CONNECTED, parent) ||
+ ovl_test_flag(OVL_INDEX, d_inode(parent)))
+ break;
+
+ dput(next);
+ next = parent;
+ }
+
+ dput(parent);
+ dput(next);
+
+ if (!err)
+ ovl_dentry_set_flag(OVL_E_CONNECTED, dentry);
+
+ return err ?: origin_layer;
+}
+
/*
* We only need to encode origin if there is a chance that the same object was
* encoded pre copy up and then we need to stay consistent with the same
* L = lower file handle
*
* (*) Connecting an overlay dir from real lower dentry is not always
- * possible when there are redirects in lower layers. To mitigate this case,
- * we copy up the lower dir first and then encode an upper dir file handle.
+ * possible when there are redirects in lower layers and non-indexed merge dirs.
+ * To mitigate those case, we may copy up the lower dir ancestor before encode
+ * a lower dir file handle.
+ *
+ * Return 0 for upper file handle, > 0 for lower file handle or < 0 on error.
*/
-static bool ovl_should_encode_origin(struct dentry *dentry)
+static int ovl_check_encode_origin(struct dentry *dentry)
{
struct ovl_fs *ofs = dentry->d_sb->s_fs_info;
+ /* Upper file handle for pure upper */
if (!ovl_dentry_lower(dentry))
- return false;
+ return 0;
/*
- * Decoding a merge dir, whose origin's parent is under a redirected
- * lower dir is not always possible. As a simple aproximation, we do
- * not encode lower dir file handles when overlay has multiple lower
- * layers and origin is below the topmost lower layer.
+ * Upper file handle for non-indexed upper.
*
- * TODO: copy up only the parent that is under redirected lower.
+ * Root is never indexed, so if there's an upper layer, encode upper for
+ * root.
*/
- if (d_is_dir(dentry) && ofs->upper_mnt &&
- OVL_E(dentry)->lowerstack[0].layer->idx > 1)
- return false;
-
- /* Decoding a non-indexed upper from origin is not implemented */
if (ovl_dentry_upper(dentry) &&
!ovl_test_flag(OVL_INDEX, d_inode(dentry)))
- return false;
-
- return true;
-}
-
-static int ovl_encode_maybe_copy_up(struct dentry *dentry)
-{
- int err;
-
- if (ovl_dentry_upper(dentry))
return 0;
- err = ovl_want_write(dentry);
- if (err)
- return err;
-
- err = ovl_copy_up(dentry);
+ /*
+ * Decoding a merge dir, whose origin's ancestor is under a redirected
+ * lower dir or under a non-indexed upper is not always possible.
+ * ovl_connect_layer() will try to make origin's layer "connected" by
+ * copying up a "connectable" ancestor.
+ */
+ if (d_is_dir(dentry) && ofs->upper_mnt)
+ return ovl_connect_layer(dentry);
- ovl_drop_write(dentry);
- return err;
+ /* Lower file handle for indexed and non-upper dir/non-dir */
+ return 1;
}
static int ovl_d_to_fh(struct dentry *dentry, char *buf, int buflen)
{
- struct dentry *origin = ovl_dentry_lower(dentry);
struct ovl_fh *fh = NULL;
- int err;
+ int err, enc_lower;
/*
- * If we should not encode a lower dir file handle, copy up and encode
- * an upper dir file handle.
+ * Check if we should encode a lower or upper file handle and maybe
+ * copy up an ancestor to make lower file handle connectable.
*/
- if (!ovl_should_encode_origin(dentry)) {
- err = ovl_encode_maybe_copy_up(dentry);
- if (err)
- goto fail;
-
- origin = NULL;
- }
+ err = enc_lower = ovl_check_encode_origin(dentry);
+ if (enc_lower < 0)
+ goto fail;
- /* Encode an upper or origin file handle */
- fh = ovl_encode_fh(origin ?: ovl_dentry_upper(dentry), !origin);
+ /* Encode an upper or lower file handle */
+ fh = ovl_encode_fh(enc_lower ? ovl_dentry_lower(dentry) :
+ ovl_dentry_upper(dentry), !enc_lower);
err = PTR_ERR(fh);
if (IS_ERR(fh))
goto fail;
dput(upper);
}
- if (!this)
- return NULL;
+ if (IS_ERR_OR_NULL(this))
+ return this;
if (WARN_ON(ovl_dentry_real_at(this, layer->idx) != real)) {
dput(this);
if (err == -ECHILD) {
this = ovl_lookup_real_ancestor(sb, real,
layer);
- err = IS_ERR(this) ? PTR_ERR(this) : 0;
+ err = PTR_ERR_OR_ZERO(this);
}
if (!err) {
dput(connected);
return inode;
}
+/*
+ * Does overlay inode need to be hashed by lower inode?
+ */
+static bool ovl_hash_bylower(struct super_block *sb, struct dentry *upper,
+ struct dentry *lower, struct dentry *index)
+{
+ struct ovl_fs *ofs = sb->s_fs_info;
+
+ /* No, if pure upper */
+ if (!lower)
+ return false;
+
+ /* Yes, if already indexed */
+ if (index)
+ return true;
+
+ /* Yes, if won't be copied up */
+ if (!ofs->upper_mnt)
+ return true;
+
+ /* No, if lower hardlink is or will be broken on copy up */
+ if ((upper || !ovl_indexdir(sb)) &&
+ !d_is_dir(lower) && d_inode(lower)->i_nlink > 1)
+ return false;
+
+ /* No, if non-indexed upper with NFS export */
+ if (sb->s_export_op && upper)
+ return false;
+
+ /* Otherwise, hash by lower inode for fsnotify */
+ return true;
+}
+
struct inode *ovl_get_inode(struct super_block *sb, struct dentry *upperdentry,
struct dentry *lowerdentry, struct dentry *index,
unsigned int numlower)
{
- struct ovl_fs *ofs = sb->s_fs_info;
struct inode *realinode = upperdentry ? d_inode(upperdentry) : NULL;
struct inode *inode;
- /* Already indexed or could be indexed on copy up? */
- bool indexed = (index || (ovl_indexdir(sb) && !upperdentry));
- struct dentry *origin = indexed ? lowerdentry : NULL;
+ bool bylower = ovl_hash_bylower(sb, upperdentry, lowerdentry, index);
bool is_dir;
- if (WARN_ON(upperdentry && indexed && !lowerdentry))
- return ERR_PTR(-EIO);
-
if (!realinode)
realinode = d_inode(lowerdentry);
/*
- * Copy up origin (lower) may exist for non-indexed non-dir upper, but
- * we must not use lower as hash key in that case.
- * Hash non-dir that is or could be indexed by origin inode.
- * Hash dir that is or could be merged by origin inode.
- * Hash pure upper and non-indexed non-dir by upper inode.
- * Hash non-indexed dir by upper inode for NFS export.
+ * Copy up origin (lower) may exist for non-indexed upper, but we must
+ * not use lower as hash key if this is a broken hardlink.
*/
is_dir = S_ISDIR(realinode->i_mode);
- if (is_dir && (indexed || !sb->s_export_op || !ofs->upper_mnt))
- origin = lowerdentry;
-
- if (upperdentry || origin) {
- struct inode *key = d_inode(origin ?: upperdentry);
+ if (upperdentry || bylower) {
+ struct inode *key = d_inode(bylower ? lowerdentry :
+ upperdentry);
unsigned int nlink = is_dir ? 1 : realinode->i_nlink;
inode = iget5_locked(sb, (unsigned long) key,
nlink = ovl_get_nlink(lowerdentry, upperdentry, nlink);
set_nlink(inode, nlink);
} else {
+ /* Lower hardlink that will be broken on copy up */
inode = new_inode(sb);
if (!inode)
goto out_nomem;
stack[ctr].layer = lower.layer;
ctr++;
- if (d.stop)
- break;
-
/*
* Following redirects can have security consequences: it's like
* a symlink into the lower layer without the permission checks.
goto out_put;
}
+ if (d.stop)
+ break;
+
if (d.redirect && d.redirect[0] == '/' && poe != roe) {
poe = roe;
/* Find the current layer on the root dentry */
enum ovl_entry_flag {
OVL_E_UPPER_ALIAS,
OVL_E_OPAQUE,
+ OVL_E_CONNECTED,
};
/*
/* Root is always merge -> can have whiteouts */
ovl_set_flag(OVL_WHITEOUTS, d_inode(root_dentry));
+ ovl_dentry_set_flag(OVL_E_CONNECTED, root_dentry);
ovl_inode_init(d_inode(root_dentry), upperpath.dentry,
ovl_dentry_lower(root_dentry));
static struct pernet_operations __net_initdata proc_net_ns_ops = {
.init = proc_net_ns_init,
.exit = proc_net_ns_exit,
- .async = true,
};
int __init proc_net_init(void)
{
return sysfs_do_create_link(kobj, target, name, 0);
}
+EXPORT_SYMBOL_GPL(sysfs_create_link_nowarn);
/**
* sysfs_delete_link - remove symlink in object's directory.
(IS_DAX(inode) && imap->br_state == XFS_EXT_UNWRITTEN);
}
+static inline bool needs_cow_for_zeroing(struct xfs_bmbt_irec *imap, int nimaps)
+{
+ return nimaps &&
+ imap->br_startblock != HOLESTARTBLOCK &&
+ imap->br_state != XFS_EXT_UNWRITTEN;
+}
+
static inline bool need_excl_ilock(struct xfs_inode *ip, unsigned flags)
{
/*
- * COW writes will allocate delalloc space, so we need to make sure
- * to take the lock exclusively here.
+ * COW writes may allocate delalloc space or convert unwritten COW
+ * extents, so we need to make sure to take the lock exclusively here.
*/
if (xfs_is_reflink_inode(ip) && (flags & (IOMAP_WRITE | IOMAP_ZERO)))
return true;
- if ((flags & IOMAP_DIRECT) && (flags & IOMAP_WRITE))
+
+ /*
+ * Extents not yet cached requires exclusive access, don't block.
+ * This is an opencoded xfs_ilock_data_map_shared() to cater for the
+ * non-blocking behaviour.
+ */
+ if (ip->i_d.di_format == XFS_DINODE_FMT_BTREE &&
+ !(ip->i_df.if_flags & XFS_IFEXTENTS))
return true;
return false;
}
return xfs_file_iomap_begin_delay(inode, offset, length, iomap);
}
- if (need_excl_ilock(ip, flags)) {
+ if (need_excl_ilock(ip, flags))
lockmode = XFS_ILOCK_EXCL;
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- } else {
- lockmode = xfs_ilock_data_map_shared(ip);
- }
+ else
+ lockmode = XFS_ILOCK_SHARED;
- if ((flags & IOMAP_NOWAIT) && !(ip->i_df.if_flags & XFS_IFEXTENTS)) {
- error = -EAGAIN;
- goto out_unlock;
+ if (flags & IOMAP_NOWAIT) {
+ if (!(ip->i_df.if_flags & XFS_IFEXTENTS))
+ return -EAGAIN;
+ if (!xfs_ilock_nowait(ip, lockmode))
+ return -EAGAIN;
+ } else {
+ xfs_ilock(ip, lockmode);
}
ASSERT(offset <= mp->m_super->s_maxbytes);
goto out_unlock;
}
- if ((flags & (IOMAP_WRITE | IOMAP_ZERO)) && xfs_is_reflink_inode(ip)) {
+ if (xfs_is_reflink_inode(ip) &&
+ ((flags & IOMAP_WRITE) ||
+ ((flags & IOMAP_ZERO) && needs_cow_for_zeroing(&imap, nimaps)))) {
if (flags & IOMAP_DIRECT) {
/*
* A reflinked inode will result in CoW alloc.
int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot);
int pud_clear_huge(pud_t *pud);
int pmd_clear_huge(pmd_t *pmd);
+int pud_free_pmd_page(pud_t *pud);
+int pmd_free_pte_page(pmd_t *pmd);
#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
static inline int p4d_set_huge(p4d_t *p4d, phys_addr_t addr, pgprot_t prot)
{
{
return 0;
}
+static inline int pud_free_pmd_page(pud_t *pud)
+{
+ return 0;
+}
+static inline int pmd_free_pte_page(pmd_t *pmd)
+{
+ return 0;
+}
#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
#ifndef __HAVE_ARCH_FLUSH_PMD_TLB_RANGE
bool kvm_vcpu_has_pending_irqs(struct kvm_vcpu *vcpu);
void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu);
void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu);
+void kvm_vgic_reset_mapped_irq(struct kvm_vcpu *vcpu, u32 vintid);
void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
struct perf_event;
struct bpf_prog;
struct bpf_map;
+struct sock;
/* map is generic key/value storage optionally accesible by eBPF programs */
struct bpf_map_ops {
BPF_PROG_TYPE(BPF_PROG_TYPE_LWT_XMIT, lwt_xmit)
BPF_PROG_TYPE(BPF_PROG_TYPE_SOCK_OPS, sock_ops)
BPF_PROG_TYPE(BPF_PROG_TYPE_SK_SKB, sk_skb)
+BPF_PROG_TYPE(BPF_PROG_TYPE_SK_MSG, sk_msg)
#endif
#ifdef CONFIG_BPF_EVENTS
BPF_PROG_TYPE(BPF_PROG_TYPE_KPROBE, kprobe)
* updaters and return part of the previous pointer as the prioidx or
* classid. Such races are short-lived and the result isn't critical.
*/
-static inline u16 sock_cgroup_prioidx(struct sock_cgroup_data *skcd)
+static inline u16 sock_cgroup_prioidx(const struct sock_cgroup_data *skcd)
{
/* fallback to 1 which is always the ID of the root cgroup */
return (skcd->is_data & 1) ? skcd->prioidx : 1;
}
-static inline u32 sock_cgroup_classid(struct sock_cgroup_data *skcd)
+static inline u32 sock_cgroup_classid(const struct sock_cgroup_data *skcd)
{
/* fallback to 0 which is the unconfigured default classid */
return (skcd->is_data & 1) ? skcd->classid : 0;
#include <linux/if.h>
#include <linux/fs.h>
#include <linux/aio_abi.h> /* for aio_context_t */
+#include <linux/uaccess.h>
#include <linux/unistd.h>
#include <asm/compat.h>
short int _addr_lsb; /* Valid LSB of the reported address. */
/* used when si_code=SEGV_BNDERR */
struct {
- short _dummy_bnd;
+ compat_uptr_t _dummy_bnd;
compat_uptr_t _lower;
compat_uptr_t _upper;
} _addr_bnd;
/* used when si_code=SEGV_PKUERR */
struct {
- short _dummy_pkey;
+ compat_uptr_t _dummy_pkey;
u32 _pkey;
} _addr_pkey;
};
asmlinkage long compat_sys_adjtimex(struct compat_timex __user *utp);
extern int get_compat_sigset(sigset_t *set, const compat_sigset_t __user *compat);
-extern int put_compat_sigset(compat_sigset_t __user *compat,
- const sigset_t *set, unsigned int size);
+
+/*
+ * Defined inline such that size can be compile time constant, which avoids
+ * CONFIG_HARDENED_USERCOPY complaining about copies from task_struct
+ */
+static inline int
+put_compat_sigset(compat_sigset_t __user *compat, const sigset_t *set,
+ unsigned int size)
+{
+ /* size <= sizeof(compat_sigset_t) <= sizeof(sigset_t) */
+#ifdef __BIG_ENDIAN
+ compat_sigset_t v;
+ switch (_NSIG_WORDS) {
+ case 4: v.sig[7] = (set->sig[3] >> 32); v.sig[6] = set->sig[3];
+ case 3: v.sig[5] = (set->sig[2] >> 32); v.sig[4] = set->sig[2];
+ case 2: v.sig[3] = (set->sig[1] >> 32); v.sig[2] = set->sig[1];
+ case 1: v.sig[1] = (set->sig[0] >> 32); v.sig[0] = set->sig[0];
+ }
+ return copy_to_user(compat, &v, size) ? -EFAULT : 0;
+#else
+ return copy_to_user(compat, set, size) ? -EFAULT : 0;
+#endif
+}
asmlinkage long compat_sys_migrate_pages(compat_pid_t pid,
compat_ulong_t maxnode, const compat_ulong_t __user *old_nodes,
struct xdp_rxq_info *rxq;
};
+struct sk_msg_buff {
+ void *data;
+ void *data_end;
+ __u32 apply_bytes;
+ __u32 cork_bytes;
+ int sg_copybreak;
+ int sg_start;
+ int sg_curr;
+ int sg_end;
+ struct scatterlist sg_data[MAX_SKB_FRAGS];
+ bool sg_copy[MAX_SKB_FRAGS];
+ __u32 key;
+ __u32 flags;
+ struct bpf_map *map;
+};
+
/* Compute the linear packet data range [data, data_end) which
* will be accessed by various program types (cls_bpf, act_bpf,
* lwt, ...). Subsystems allowing direct data access must (!)
void bpf_warn_invalid_xdp_action(u32 act);
struct sock *do_sk_redirect_map(struct sk_buff *skb);
+struct sock *do_msg_redirect_map(struct sk_msg_buff *md);
#ifdef CONFIG_BPF_JIT
extern int bpf_jit_enable;
#define SB_I_CGROUPWB 0x00000001 /* cgroup-aware writeback enabled */
#define SB_I_NOEXEC 0x00000002 /* Ignore executables on this fs */
#define SB_I_NODEV 0x00000004 /* Ignore devices on this fs */
+#define SB_I_MULTIROOT 0x00000008 /* Multiple roots to the dentry tree */
/* sb->s_iflags to limit user namespace mounts */
#define SB_I_USERNS_VISIBLE 0x00000010 /* fstype already mounted */
bool tun_is_xdp_buff(void *ptr);
void *tun_xdp_to_ptr(void *ptr);
void *tun_ptr_to_xdp(void *ptr);
+void tun_ptr_free(void *ptr);
#else
#include <linux/err.h>
#include <linux/errno.h>
{
return NULL;
}
+static inline void tun_ptr_free(void *ptr)
+{
+}
#endif /* CONFIG_TUN */
#endif /* __IF_TUN_H */
}
/**
- * __vlan_insert_tag - regular VLAN tag inserting
+ * __vlan_insert_inner_tag - inner VLAN tag inserting
* @skb: skbuff to tag
* @vlan_proto: VLAN encapsulation protocol
* @vlan_tci: VLAN TCI to insert
+ * @mac_len: MAC header length including outer vlan headers
*
- * Inserts the VLAN tag into @skb as part of the payload
+ * Inserts the VLAN tag into @skb as part of the payload at offset mac_len
* Returns error if skb_cow_head failes.
*
* Does not change skb->protocol so this function can be used during receive.
*/
-static inline int __vlan_insert_tag(struct sk_buff *skb,
- __be16 vlan_proto, u16 vlan_tci)
+static inline int __vlan_insert_inner_tag(struct sk_buff *skb,
+ __be16 vlan_proto, u16 vlan_tci,
+ unsigned int mac_len)
{
struct vlan_ethhdr *veth;
if (skb_cow_head(skb, VLAN_HLEN) < 0)
return -ENOMEM;
- veth = skb_push(skb, VLAN_HLEN);
+ skb_push(skb, VLAN_HLEN);
- /* Move the mac addresses to the beginning of the new header. */
- memmove(skb->data, skb->data + VLAN_HLEN, 2 * ETH_ALEN);
+ /* Move the mac header sans proto to the beginning of the new header. */
+ memmove(skb->data, skb->data + VLAN_HLEN, mac_len - ETH_TLEN);
skb->mac_header -= VLAN_HLEN;
+ veth = (struct vlan_ethhdr *)(skb->data + mac_len - ETH_HLEN);
+
/* first, the ethernet type */
veth->h_vlan_proto = vlan_proto;
}
/**
- * vlan_insert_tag - regular VLAN tag inserting
+ * __vlan_insert_tag - regular VLAN tag inserting
* @skb: skbuff to tag
* @vlan_proto: VLAN encapsulation protocol
* @vlan_tci: VLAN TCI to insert
*
* Inserts the VLAN tag into @skb as part of the payload
+ * Returns error if skb_cow_head failes.
+ *
+ * Does not change skb->protocol so this function can be used during receive.
+ */
+static inline int __vlan_insert_tag(struct sk_buff *skb,
+ __be16 vlan_proto, u16 vlan_tci)
+{
+ return __vlan_insert_inner_tag(skb, vlan_proto, vlan_tci, ETH_HLEN);
+}
+
+/**
+ * vlan_insert_inner_tag - inner VLAN tag inserting
+ * @skb: skbuff to tag
+ * @vlan_proto: VLAN encapsulation protocol
+ * @vlan_tci: VLAN TCI to insert
+ * @mac_len: MAC header length including outer vlan headers
+ *
+ * Inserts the VLAN tag into @skb as part of the payload at offset mac_len
* Returns a VLAN tagged skb. If a new skb is created, @skb is freed.
*
* Following the skb_unshare() example, in case of error, the calling function
*
* Does not change skb->protocol so this function can be used during receive.
*/
-static inline struct sk_buff *vlan_insert_tag(struct sk_buff *skb,
- __be16 vlan_proto, u16 vlan_tci)
+static inline struct sk_buff *vlan_insert_inner_tag(struct sk_buff *skb,
+ __be16 vlan_proto,
+ u16 vlan_tci,
+ unsigned int mac_len)
{
int err;
- err = __vlan_insert_tag(skb, vlan_proto, vlan_tci);
+ err = __vlan_insert_inner_tag(skb, vlan_proto, vlan_tci, mac_len);
if (err) {
dev_kfree_skb_any(skb);
return NULL;
return skb;
}
+/**
+ * vlan_insert_tag - regular VLAN tag inserting
+ * @skb: skbuff to tag
+ * @vlan_proto: VLAN encapsulation protocol
+ * @vlan_tci: VLAN TCI to insert
+ *
+ * Inserts the VLAN tag into @skb as part of the payload
+ * Returns a VLAN tagged skb. If a new skb is created, @skb is freed.
+ *
+ * Following the skb_unshare() example, in case of error, the calling function
+ * doesn't have to worry about freeing the original skb.
+ *
+ * Does not change skb->protocol so this function can be used during receive.
+ */
+static inline struct sk_buff *vlan_insert_tag(struct sk_buff *skb,
+ __be16 vlan_proto, u16 vlan_tci)
+{
+ return vlan_insert_inner_tag(skb, vlan_proto, vlan_tci, ETH_HLEN);
+}
+
/**
* vlan_insert_tag_set_proto - regular VLAN tag inserting
* @skb: skbuff to tag
#define ICH_HCR_EN (1 << 0)
#define ICH_HCR_UIE (1 << 1)
+#define ICH_HCR_NPIE (1 << 3)
#define ICH_HCR_TC (1 << 10)
#define ICH_HCR_TALL0 (1 << 11)
#define ICH_HCR_TALL1 (1 << 12)
#define GICH_HCR_EN (1 << 0)
#define GICH_HCR_UIE (1 << 1)
+#define GICH_HCR_NPIE (1 << 3)
#define GICH_LR_VIRTUALID (0x3ff << 0)
#define GICH_LR_PHYSID_CPUID_SHIFT (10)
unsigned long *end_pfn);
void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
unsigned long *out_end_pfn, int *out_nid);
-unsigned long memblock_next_valid_pfn(unsigned long pfn, unsigned long max_pfn);
/**
* for_each_mem_pfn_range - early memory pfn range iterator
int mlx5_core_modify_cq_moderation(struct mlx5_core_dev *dev,
struct mlx5_core_cq *cq, u16 cq_period,
u16 cq_max_count);
+static inline void mlx5_dump_err_cqe(struct mlx5_core_dev *dev,
+ struct mlx5_err_cqe *err_cqe)
+{
+ print_hex_dump(KERN_WARNING, "", DUMP_PREFIX_OFFSET, 16, 1, err_cqe,
+ sizeof(*err_cqe), false);
+}
int mlx5_debug_cq_add(struct mlx5_core_dev *dev, struct mlx5_core_cq *cq);
void mlx5_debug_cq_remove(struct mlx5_core_dev *dev, struct mlx5_core_cq *cq);
MLX5_CAP_RESERVED,
MLX5_CAP_VECTOR_CALC,
MLX5_CAP_QOS,
+ MLX5_CAP_DEBUG,
/* NUM OF CAP Types */
MLX5_CAP_NUM
};
#define MLX5_CAP_QOS(mdev, cap)\
MLX5_GET(qos_cap, mdev->caps.hca_cur[MLX5_CAP_QOS], cap)
+#define MLX5_CAP_DEBUG(mdev, cap)\
+ MLX5_GET(debug_cap, mdev->caps.hca_cur[MLX5_CAP_DEBUG], cap)
+
#define MLX5_CAP_PCAM_FEATURE(mdev, fld) \
MLX5_GET(pcam_reg, (mdev)->caps.pcam, feature_cap_mask.enhanced_features.fld)
struct mlx5_core_rsc_common common; /* must be first */
u32 srqn;
int max;
- int max_gs;
- int max_avail_gather;
+ size_t max_gs;
+ size_t max_avail_gather;
int wqe_shift;
void (*event) (struct mlx5_core_srq *, enum mlx5_event);
mlx5_create_flow_group(struct mlx5_flow_table *ft, u32 *in);
void mlx5_destroy_flow_group(struct mlx5_flow_group *fg);
+struct mlx5_fs_vlan {
+ u16 ethtype;
+ u16 vid;
+ u8 prio;
+};
+
struct mlx5_flow_act {
u32 action;
bool has_flow_tag;
u32 encap_id;
u32 modify_id;
uintptr_t esp_id;
+ struct mlx5_fs_vlan vlan;
};
#define MLX5_DECLARE_FLOW_ACT(name) \
MLX5_CMD_OP_MODIFY_HCA_VPORT_CONTEXT = 0x763,
MLX5_CMD_OP_QUERY_HCA_VPORT_GID = 0x764,
MLX5_CMD_OP_QUERY_HCA_VPORT_PKEY = 0x765,
+ MLX5_CMD_OP_QUERY_VNIC_ENV = 0x76f,
MLX5_CMD_OP_QUERY_VPORT_COUNTER = 0x770,
MLX5_CMD_OP_ALLOC_Q_COUNTER = 0x771,
MLX5_CMD_OP_DEALLOC_Q_COUNTER = 0x772,
u8 flow_table_modify[0x1];
u8 encap[0x1];
u8 decap[0x1];
- u8 reserved_at_9[0x17];
+ u8 reserved_at_9[0x1];
+ u8 pop_vlan[0x1];
+ u8 push_vlan[0x1];
+ u8 reserved_at_c[0x14];
u8 reserved_at_20[0x2];
u8 log_max_ft_size[0x6];
u8 reserved_at_100[0x700];
};
+struct mlx5_ifc_debug_cap_bits {
+ u8 reserved_at_0[0x20];
+
+ u8 reserved_at_20[0x2];
+ u8 stall_detect[0x1];
+ u8 reserved_at_23[0x1d];
+
+ u8 reserved_at_40[0x7c0];
+};
+
struct mlx5_ifc_per_protocol_networking_offload_caps_bits {
u8 csum_cap[0x1];
u8 vlan_cap[0x1];
u8 out_of_seq_cnt[0x1];
u8 vport_counters[0x1];
u8 retransmission_q_counters[0x1];
- u8 reserved_at_183[0x1];
+ u8 debug[0x1];
u8 modify_rq_counter_set_id[0x1];
u8 rq_delay_drop[0x1];
u8 max_qp_cnt[0xa];
u8 vhca_group_manager[0x1];
u8 ib_virt[0x1];
u8 eth_virt[0x1];
- u8 reserved_at_1a4[0x1];
+ u8 vnic_env_queue_counters[0x1];
u8 ets[0x1];
u8 nic_flow_table[0x1];
u8 eswitch_flow_table[0x1];
u8 reserved_at_330[0xb];
u8 log_max_xrcd[0x5];
- u8 reserved_at_340[0x8];
+ u8 nic_receive_steering_discard[0x1];
+ u8 receive_discard_vport_down[0x1];
+ u8 transmit_discard_vport_down[0x1];
+ u8 reserved_at_343[0x5];
u8 log_max_flow_counter_bulk[0x8];
u8 max_flow_counter_15_0[0x10];
u8 rx_pause_transition_low[0x20];
- u8 reserved_at_3c0[0x400];
+ u8 reserved_at_3c0[0x40];
+
+ u8 device_stall_minor_watermark_cnt_high[0x20];
+
+ u8 device_stall_minor_watermark_cnt_low[0x20];
+
+ u8 device_stall_critical_watermark_cnt_high[0x20];
+
+ u8 device_stall_critical_watermark_cnt_low[0x20];
+
+ u8 reserved_at_480[0x340];
};
struct mlx5_ifc_eth_extended_cntrs_grp_data_layout_bits {
MLX5_FLOW_CONTEXT_ACTION_ENCAP = 0x10,
MLX5_FLOW_CONTEXT_ACTION_DECAP = 0x20,
MLX5_FLOW_CONTEXT_ACTION_MOD_HDR = 0x40,
+ MLX5_FLOW_CONTEXT_ACTION_VLAN_POP = 0x80,
+ MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH = 0x100,
+};
+
+struct mlx5_ifc_vlan_bits {
+ u8 ethtype[0x10];
+ u8 prio[0x3];
+ u8 cfi[0x1];
+ u8 vid[0xc];
};
struct mlx5_ifc_flow_context_bits {
- u8 reserved_at_0[0x20];
+ struct mlx5_ifc_vlan_bits push_vlan;
u8 group_id[0x20];
u8 reserved_at_180[0x80];
};
+struct mlx5_ifc_vnic_diagnostic_statistics_bits {
+ u8 counter_error_queues[0x20];
+
+ u8 total_error_queues[0x20];
+
+ u8 send_queue_priority_update_flow[0x20];
+
+ u8 reserved_at_60[0x20];
+
+ u8 nic_receive_steering_discard[0x40];
+
+ u8 receive_discard_vport_down[0x40];
+
+ u8 transmit_discard_vport_down[0x40];
+
+ u8 reserved_at_140[0xec0];
+};
+
struct mlx5_ifc_traffic_counter_bits {
u8 packets[0x40];
u8 reserved_at_60[0x20];
};
+struct mlx5_ifc_query_vnic_env_out_bits {
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+
+ u8 syndrome[0x20];
+
+ u8 reserved_at_40[0x40];
+
+ struct mlx5_ifc_vnic_diagnostic_statistics_bits vport_env;
+};
+
+enum {
+ MLX5_QUERY_VNIC_ENV_IN_OP_MOD_VPORT_DIAG_STATISTICS = 0x0,
+};
+
+struct mlx5_ifc_query_vnic_env_in_bits {
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+
+ u8 other_vport[0x1];
+ u8 reserved_at_41[0xf];
+ u8 vport_number[0x10];
+
+ u8 reserved_at_60[0x20];
+};
+
struct mlx5_ifc_query_vport_counter_out_bits {
u8 status[0x8];
u8 reserved_at_8[0x18];
struct mlx5_ifc_pfcc_reg_bits {
u8 reserved_at_0[0x8];
u8 local_port[0x8];
- u8 reserved_at_10[0x10];
+ u8 reserved_at_10[0xb];
+ u8 ppan_mask_n[0x1];
+ u8 minor_stall_mask[0x1];
+ u8 critical_stall_mask[0x1];
+ u8 reserved_at_1e[0x2];
u8 ppan[0x4];
u8 reserved_at_24[0x4];
u8 pptx[0x1];
u8 aptx[0x1];
- u8 reserved_at_42[0x6];
+ u8 pptx_mask_n[0x1];
+ u8 reserved_at_43[0x5];
u8 pfctx[0x8];
u8 reserved_at_50[0x10];
u8 pprx[0x1];
u8 aprx[0x1];
- u8 reserved_at_62[0x6];
+ u8 pprx_mask_n[0x1];
+ u8 reserved_at_63[0x5];
u8 pfcrx[0x8];
u8 reserved_at_70[0x10];
- u8 reserved_at_80[0x80];
+ u8 device_stall_minor_watermark[0x10];
+ u8 device_stall_critical_watermark[0x10];
+
+ u8 reserved_at_a0[0x60];
};
struct mlx5_ifc_pelc_reg_bits {
};
struct mlx5_ifc_pcam_enhanced_features_bits {
- u8 reserved_at_0[0x7b];
+ u8 reserved_at_0[0x76];
+ u8 pfcc_mask[0x1];
+ u8 reserved_at_77[0x4];
u8 rx_buffer_fullness_counters[0x1];
u8 ptys_connector_type[0x1];
u8 reserved_at_7d[0x1];
int mlx5_query_port_pfc(struct mlx5_core_dev *dev, u8 *pfc_en_tx,
u8 *pfc_en_rx);
+int mlx5_set_port_stall_watermark(struct mlx5_core_dev *dev,
+ u16 stall_critical_watermark,
+ u16 stall_minor_watermark);
+int mlx5_query_port_stall_watermark(struct mlx5_core_dev *dev,
+ u16 *stall_critical_watermark, u16 *stall_minor_watermark);
+
int mlx5_max_tc(struct mlx5_core_dev *mdev);
int mlx5_set_port_prio_tc(struct mlx5_core_dev *mdev, u8 *prio_tc);
int mlx5_core_modify_sq(struct mlx5_core_dev *dev, u32 sqn, u32 *in, int inlen);
void mlx5_core_destroy_sq(struct mlx5_core_dev *dev, u32 sqn);
int mlx5_core_query_sq(struct mlx5_core_dev *dev, u32 sqn, u32 *out);
+int mlx5_core_query_sq_state(struct mlx5_core_dev *dev, u32 sqn, u8 *state);
int mlx5_core_create_tir(struct mlx5_core_dev *dev, u32 *in, int inlen,
u32 *tirn);
int mlx5_core_modify_tir(struct mlx5_core_dev *dev, u32 tirn, u32 *in,
int mlx5_nic_vport_enable_roce(struct mlx5_core_dev *mdev);
int mlx5_nic_vport_disable_roce(struct mlx5_core_dev *mdev);
+int mlx5_query_vport_down_stats(struct mlx5_core_dev *mdev, u16 vport,
+ u64 *rx_discard_vport_down,
+ u64 *tx_discard_vport_down);
int mlx5_core_query_vport_counter(struct mlx5_core_dev *dev, u8 other_vport,
int vf, u8 port_num, void *out,
size_t out_sz);
}
#endif
-struct vif_entry_notifier_info {
- struct fib_notifier_info info;
- struct net_device *dev;
- vifi_t vif_index;
- unsigned short vif_flags;
- u32 tb_id;
-};
-
#define VIFF_STATIC 0x8000
struct mfc_cache_cmp_arg {
};
};
-struct mfc_entry_notifier_info {
- struct fib_notifier_info info;
- struct mfc_cache *mfc;
- u32 tb_id;
-};
-
struct rtmsg;
int ipmr_get_route(struct net *net, struct sk_buff *skb,
__be32 saddr, __be32 daddr,
struct rtmsg *rtm, u32 portid);
-
-#ifdef CONFIG_IP_MROUTE
-void ipmr_cache_free(struct mfc_cache *mfc_cache);
-#else
-static inline void ipmr_cache_free(struct mfc_cache *mfc_cache)
-{
-}
-#endif
-
-static inline void ipmr_cache_put(struct mfc_cache *c)
-{
- if (refcount_dec_and_test(&c->_c.mfc_un.res.refcount))
- ipmr_cache_free(c);
-}
-static inline void ipmr_cache_hold(struct mfc_cache *c)
-{
- refcount_inc(&c->_c.mfc_un.res.refcount);
-}
-
#endif
#include <net/net_namespace.h>
#include <uapi/linux/mroute6.h>
#include <linux/mroute_base.h>
+#include <net/fib_rules.h>
#ifdef CONFIG_IPV6_MROUTE
static inline int ip6_mroute_opt(int opt)
}
#endif
+#ifdef CONFIG_IPV6_MROUTE_MULTIPLE_TABLES
+bool ip6mr_rule_default(const struct fib_rule *rule);
+#else
+static inline bool ip6mr_rule_default(const struct fib_rule *rule)
+{
+ return true;
+}
+#endif
+
#define VIFF_STATIC 0x8000
struct mfc6_cache_cmp_arg {
#include <linux/spinlock.h>
#include <net/net_namespace.h>
#include <net/sock.h>
+#include <net/fib_notifier.h>
/**
* struct vif_device - interface representor for multicast routing
__be32 local, remote;
};
+struct vif_entry_notifier_info {
+ struct fib_notifier_info info;
+ struct net_device *dev;
+ unsigned short vif_index;
+ unsigned short vif_flags;
+ u32 tb_id;
+};
+
+static inline int mr_call_vif_notifier(struct notifier_block *nb,
+ struct net *net,
+ unsigned short family,
+ enum fib_event_type event_type,
+ struct vif_device *vif,
+ unsigned short vif_index, u32 tb_id)
+{
+ struct vif_entry_notifier_info info = {
+ .info = {
+ .family = family,
+ .net = net,
+ },
+ .dev = vif->dev,
+ .vif_index = vif_index,
+ .vif_flags = vif->flags,
+ .tb_id = tb_id,
+ };
+
+ return call_fib_notifier(nb, net, event_type, &info.info);
+}
+
+static inline int mr_call_vif_notifiers(struct net *net,
+ unsigned short family,
+ enum fib_event_type event_type,
+ struct vif_device *vif,
+ unsigned short vif_index, u32 tb_id,
+ unsigned int *ipmr_seq)
+{
+ struct vif_entry_notifier_info info = {
+ .info = {
+ .family = family,
+ .net = net,
+ },
+ .dev = vif->dev,
+ .vif_index = vif_index,
+ .vif_flags = vif->flags,
+ .tb_id = tb_id,
+ };
+
+ ASSERT_RTNL();
+ (*ipmr_seq)++;
+ return call_fib_notifiers(net, event_type, &info.info);
+}
+
#ifndef MAXVIFS
/* This one is nasty; value is defined in uapi using different symbols for
* mroute and morute6 but both map into same 32.
* @refcount: reference count for this entry
* @list: global entry list
* @rcu: used for entry destruction
+ * @free: Operation used for freeing an entry under RCU
*/
struct mr_mfc {
struct rhlist_head mnode;
} mfc_un;
struct list_head list;
struct rcu_head rcu;
+ void (*free)(struct rcu_head *head);
+};
+
+static inline void mr_cache_put(struct mr_mfc *c)
+{
+ if (refcount_dec_and_test(&c->mfc_un.res.refcount))
+ call_rcu(&c->rcu, c->free);
+}
+
+static inline void mr_cache_hold(struct mr_mfc *c)
+{
+ refcount_inc(&c->mfc_un.res.refcount);
+}
+
+struct mfc_entry_notifier_info {
+ struct fib_notifier_info info;
+ struct mr_mfc *mfc;
+ u32 tb_id;
};
+static inline int mr_call_mfc_notifier(struct notifier_block *nb,
+ struct net *net,
+ unsigned short family,
+ enum fib_event_type event_type,
+ struct mr_mfc *mfc, u32 tb_id)
+{
+ struct mfc_entry_notifier_info info = {
+ .info = {
+ .family = family,
+ .net = net,
+ },
+ .mfc = mfc,
+ .tb_id = tb_id
+ };
+
+ return call_fib_notifier(nb, net, event_type, &info.info);
+}
+
+static inline int mr_call_mfc_notifiers(struct net *net,
+ unsigned short family,
+ enum fib_event_type event_type,
+ struct mr_mfc *mfc, u32 tb_id,
+ unsigned int *ipmr_seq)
+{
+ struct mfc_entry_notifier_info info = {
+ .info = {
+ .family = family,
+ .net = net,
+ },
+ .mfc = mfc,
+ .tb_id = tb_id
+ };
+
+ ASSERT_RTNL();
+ (*ipmr_seq)++;
+ return call_fib_notifiers(net, event_type, &info.info);
+}
+
struct mr_table;
/**
u32 portid, u32 seq, struct mr_mfc *c,
int cmd, int flags),
spinlock_t *lock);
+
+int mr_dump(struct net *net, struct notifier_block *nb, unsigned short family,
+ int (*rules_dump)(struct net *net,
+ struct notifier_block *nb),
+ struct mr_table *(*mr_iter)(struct net *net,
+ struct mr_table *mrt),
+ rwlock_t *mrt_lock);
#else
static inline void vif_device_init(struct vif_device *v,
struct net_device *dev,
{
return -EINVAL;
}
+
+static inline int mr_dump(struct net *net, struct notifier_block *nb,
+ unsigned short family,
+ int (*rules_dump)(struct net *net,
+ struct notifier_block *nb),
+ struct mr_table *(*mr_iter)(struct net *net,
+ struct mr_table *mrt),
+ rwlock_t *mrt_lock)
+{
+ return -EINVAL;
+}
#endif
static inline void *mr_mfc_find(struct mr_table *mrt, void *hasharg)
int sock_wake_async(struct socket_wq *sk_wq, int how, int band);
int sock_register(const struct net_proto_family *fam);
void sock_unregister(int family);
+bool sock_is_registered(int family);
int __sock_create(struct net *net, int family, int type, int proto,
struct socket **res, int kern);
int sock_create(int family, int type, int proto, struct socket **res);
#include <linux/notifier.h>
-/* netdevice notifier chain. Please remember to update the rtnetlink
- * notification exclusion list in rtnetlink_event() when adding new
- * types.
+/* netdevice notifier chain. Please remember to update netdev_cmd_to_name()
+ * and the rtnetlink notification exclusion list in rtnetlink_event() when
+ * adding new types.
*/
-#define NETDEV_UP 0x0001 /* For now you can't veto a device up/down */
-#define NETDEV_DOWN 0x0002
-#define NETDEV_REBOOT 0x0003 /* Tell a protocol stack a network interface
+enum netdev_cmd {
+ NETDEV_UP = 1, /* For now you can't veto a device up/down */
+ NETDEV_DOWN,
+ NETDEV_REBOOT, /* Tell a protocol stack a network interface
detected a hardware crash and restarted
- we can use this eg to kick tcp sessions
once done */
-#define NETDEV_CHANGE 0x0004 /* Notify device state change */
-#define NETDEV_REGISTER 0x0005
-#define NETDEV_UNREGISTER 0x0006
-#define NETDEV_CHANGEMTU 0x0007 /* notify after mtu change happened */
-#define NETDEV_CHANGEADDR 0x0008
-#define NETDEV_GOING_DOWN 0x0009
-#define NETDEV_CHANGENAME 0x000A
-#define NETDEV_FEAT_CHANGE 0x000B
-#define NETDEV_BONDING_FAILOVER 0x000C
-#define NETDEV_PRE_UP 0x000D
-#define NETDEV_PRE_TYPE_CHANGE 0x000E
-#define NETDEV_POST_TYPE_CHANGE 0x000F
-#define NETDEV_POST_INIT 0x0010
-#define NETDEV_UNREGISTER_FINAL 0x0011
-#define NETDEV_RELEASE 0x0012
-#define NETDEV_NOTIFY_PEERS 0x0013
-#define NETDEV_JOIN 0x0014
-#define NETDEV_CHANGEUPPER 0x0015
-#define NETDEV_RESEND_IGMP 0x0016
-#define NETDEV_PRECHANGEMTU 0x0017 /* notify before mtu change happened */
-#define NETDEV_CHANGEINFODATA 0x0018
-#define NETDEV_BONDING_INFO 0x0019
-#define NETDEV_PRECHANGEUPPER 0x001A
-#define NETDEV_CHANGELOWERSTATE 0x001B
-#define NETDEV_UDP_TUNNEL_PUSH_INFO 0x001C
-#define NETDEV_UDP_TUNNEL_DROP_INFO 0x001D
-#define NETDEV_CHANGE_TX_QUEUE_LEN 0x001E
+ NETDEV_CHANGE, /* Notify device state change */
+ NETDEV_REGISTER,
+ NETDEV_UNREGISTER,
+ NETDEV_CHANGEMTU, /* notify after mtu change happened */
+ NETDEV_CHANGEADDR,
+ NETDEV_GOING_DOWN,
+ NETDEV_CHANGENAME,
+ NETDEV_FEAT_CHANGE,
+ NETDEV_BONDING_FAILOVER,
+ NETDEV_PRE_UP,
+ NETDEV_PRE_TYPE_CHANGE,
+ NETDEV_POST_TYPE_CHANGE,
+ NETDEV_POST_INIT,
+ NETDEV_RELEASE,
+ NETDEV_NOTIFY_PEERS,
+ NETDEV_JOIN,
+ NETDEV_CHANGEUPPER,
+ NETDEV_RESEND_IGMP,
+ NETDEV_PRECHANGEMTU, /* notify before mtu change happened */
+ NETDEV_CHANGEINFODATA,
+ NETDEV_BONDING_INFO,
+ NETDEV_PRECHANGEUPPER,
+ NETDEV_CHANGELOWERSTATE,
+ NETDEV_UDP_TUNNEL_PUSH_INFO,
+ NETDEV_UDP_TUNNEL_DROP_INFO,
+ NETDEV_CHANGE_TX_QUEUE_LEN,
+};
+const char *netdev_cmd_to_name(enum netdev_cmd cmd);
int register_netdevice_notifier(struct notifier_block *nb);
int unregister_netdevice_notifier(struct notifier_block *nb);
bool xt_find_jump_offset(const unsigned int *offsets,
unsigned int target, unsigned int size);
+int xt_check_proc_name(const char *name, unsigned int size);
+
int xt_check_match(struct xt_mtchk_param *, unsigned int size, u_int8_t proto,
bool inv_proto);
int xt_check_target(struct xt_tgchk_param *, unsigned int size, u_int8_t proto,
struct device_node *of_pci_find_child_device(struct device_node *parent,
unsigned int devfn);
int of_pci_get_devfn(struct device_node *np);
-int of_irq_parse_and_map_pci(const struct pci_dev *dev, u8 slot, u8 pin);
int of_pci_parse_bus_range(struct device_node *node, struct resource *res);
int of_get_pci_domain_nr(struct device_node *node);
int of_pci_get_max_link_speed(struct device_node *node);
return -EINVAL;
}
-static inline int
-of_irq_parse_and_map_pci(const struct pci_dev *dev, u8 slot, u8 pin)
-{
- return 0;
-}
-
static inline int
of_pci_parse_bus_range(struct device_node *node, struct resource *res)
{
static inline void of_pci_check_probe_only(void) { }
#endif
+#if IS_ENABLED(CONFIG_OF_IRQ)
+int of_irq_parse_and_map_pci(const struct pci_dev *dev, u8 slot, u8 pin);
+#else
+static inline int
+of_irq_parse_and_map_pci(const struct pci_dev *dev, u8 slot, u8 pin)
+{
+ return 0;
+}
+#endif
+
#if defined(CONFIG_OF_ADDRESS)
int of_pci_get_host_bridge_resources(struct device_node *dev,
unsigned char busno, unsigned char bus_max,
* calls io_destroy() or the process exits.
*
* In the aio code, kill_ioctx() is called when we wish to destroy a kioctx; it
- * calls percpu_ref_kill(), then hlist_del_rcu() and synchronize_rcu() to remove
- * the kioctx from the proccess's list of kioctxs - after that, there can't be
- * any new users of the kioctx (from lookup_ioctx()) and it's then safe to drop
- * the initial ref with percpu_ref_put().
+ * removes the kioctx from the proccess's table of kioctxs and kills percpu_ref.
+ * After that, there can't be any new users of the kioctx (from lookup_ioctx())
+ * and it's then safe to drop the initial ref with percpu_ref_put().
+ *
+ * Note that the free path, free_ioctx(), needs to go through explicit call_rcu()
+ * to synchronize with RCU protected lookup_ioctx(). percpu_ref operations don't
+ * imply RCU grace periods of any kind and if a user wants to combine percpu_ref
+ * with RCU protection, it must be done explicitly.
*
* Code that does a two stage shutdown like this often needs some kind of
* explicit synchronization to ensure the initial refcount can only be dropped
* Must be used to drop the initial ref on a percpu refcount; must be called
* precisely once before shutdown.
*
- * Puts @ref in non percpu mode, then does a call_rcu() before gathering up the
- * percpu counters and dropping the initial ref.
+ * Switches @ref into atomic mode before gathering up the percpu counters
+ * and dropping the initial ref.
+ *
+ * There are no implied RCU grace periods between kill and release.
*/
static inline void percpu_ref_kill(struct percpu_ref *ref)
{
{
return 0;
}
+int genphy_read_mmd_unsupported(struct phy_device *phdev, int devad,
+ u16 regnum);
+int genphy_write_mmd_unsupported(struct phy_device *phdev, int devnum,
+ u16 regnum, u16 val);
/* Clause 45 PHY */
int genphy_c45_restart_aneg(struct phy_device *phydev);
int phy_drivers_register(struct phy_driver *new_driver, int n,
struct module *owner);
void phy_state_machine(struct work_struct *work);
-void phy_change(struct phy_device *phydev);
void phy_change_work(struct work_struct *work);
void phy_mac_interrupt(struct phy_device *phydev);
void phy_start_machine(struct phy_device *phydev);
#define FW_MAJOR_VERSION 8
#define FW_MINOR_VERSION 33
-#define FW_REVISION_VERSION 1
+#define FW_REVISION_VERSION 11
#define FW_ENGINEERING_VERSION 0
/***********************/
#define ETH_CTL_FRAME_ETH_TYPE_NUM 4
/* GFS constants */
-#define ETH_GFT_TRASH_CAN_VPORT 0x1FF
+#define ETH_GFT_TRASHCAN_VPORT 0x1FF /* GFT drop flow vport number */
/* Destination port mode */
enum dest_port_mode {
#define E4_YSTORM_ISCSI_TASK_AG_CTX_BIT1_SHIFT 5
#define E4_YSTORM_ISCSI_TASK_AG_CTX_VALID_MASK 0x1
#define E4_YSTORM_ISCSI_TASK_AG_CTX_VALID_SHIFT 6
-#define E4_YSTORM_ISCSI_TASK_AG_CTX_BIT3_MASK 0x1
-#define E4_YSTORM_ISCSI_TASK_AG_CTX_BIT3_SHIFT 7
+#define E4_YSTORM_ISCSI_TASK_AG_CTX_TTT_VALID_MASK 0x1 /* bit3 */
+#define E4_YSTORM_ISCSI_TASK_AG_CTX_TTT_VALID_SHIFT 7
u8 flags1;
#define E4_YSTORM_ISCSI_TASK_AG_CTX_CF0_MASK 0x3
#define E4_YSTORM_ISCSI_TASK_AG_CTX_CF0_SHIFT 0
u8 used_cnt;
};
+#define QED_NVM_SIGNATURE 0x12435687
+
+enum qed_nvm_flash_cmd {
+ QED_NVM_FLASH_CMD_FILE_DATA = 0x2,
+ QED_NVM_FLASH_CMD_FILE_START = 0x3,
+ QED_NVM_FLASH_CMD_NVM_CHANGE = 0x4,
+ QED_NVM_FLASH_CMD_NVM_MAX,
+};
+
struct qed_common_cb_ops {
void (*arfs_filter_op)(void *dev, void *fltr, u8 fw_rc);
void (*link_update)(void *dev,
void (*chain_free)(struct qed_dev *cdev,
struct qed_chain *p_chain);
+/**
+ * @brief nvm_flash - Flash nvm data.
+ *
+ * @param cdev
+ * @param name - file containing the data
+ *
+ * @return 0 on success, error otherwise.
+ */
+ int (*nvm_flash)(struct qed_dev *cdev, const char *name);
+
/**
* @brief nvm_get_image - reads an entire image from nvram
*
#define RDMA_MAX_CQS (64 * 1024)
#define RDMA_MAX_TIDS (128 * 1024 - 1)
#define RDMA_MAX_PDS (64 * 1024)
+#define RDMA_MAX_XRC_SRQS (1024)
+#define RDMA_MAX_SRQS (32 * 1024)
#define RDMA_NUM_STATISTIC_COUNTERS MAX_NUM_VPORTS
#define RDMA_NUM_STATISTIC_COUNTERS_K2 MAX_NUM_VPORTS_K2
ROCE_ASYNC_EVENT_CQ_OVERFLOW_ERR,
ROCE_ASYNC_EVENT_SRQ_EMPTY,
ROCE_ASYNC_EVENT_DESTROY_QP_DONE,
+ ROCE_ASYNC_EVENT_XRC_DOMAIN_ERR,
+ ROCE_ASYNC_EVENT_INVALID_XRCETH_ERR,
+ ROCE_ASYNC_EVENT_XRC_SRQ_CATASTROPHIC_ERR,
MAX_ROCE_ASYNC_EVENTS_TYPE
};
if (!key ||
(params.obj_cmpfn ?
params.obj_cmpfn(&arg, rht_obj(ht, head)) :
- rhashtable_compare(&arg, rht_obj(ht, head))))
+ rhashtable_compare(&arg, rht_obj(ht, head)))) {
+ pprev = &head->next;
continue;
+ }
data = rht_obj(ht, head);
extern int rtnl_lock_killable(void);
extern wait_queue_head_t netdev_unregistering_wq;
-extern struct rw_semaphore net_sem;
+extern struct rw_semaphore pernet_ops_rwsem;
+extern struct rw_semaphore net_rwsem;
#ifdef CONFIG_PROVE_LOCKING
extern bool lockdep_rtnl_is_held(void);
return skb_shinfo(skb)->gso_type & SKB_GSO_TCPV6;
}
+/* Note: Should be called only if skb_is_gso(skb) is true */
+static inline bool skb_is_gso_sctp(const struct sk_buff *skb)
+{
+ return skb_shinfo(skb)->gso_type & SKB_GSO_SCTP;
+}
+
static inline void skb_gso_reset(struct sk_buff *skb)
{
skb_shinfo(skb)->gso_size = 0;
skb_shinfo(skb)->gso_type = 0;
}
+static inline void skb_increase_gso_size(struct skb_shared_info *shinfo,
+ u16 increment)
+{
+ if (WARN_ON_ONCE(shinfo->gso_size == GSO_BY_FRAGS))
+ return;
+ shinfo->gso_size += increment;
+}
+
+static inline void skb_decrease_gso_size(struct skb_shared_info *shinfo,
+ u16 decrement)
+{
+ if (WARN_ON_ONCE(shinfo->gso_size == GSO_BY_FRAGS))
+ return;
+ shinfo->gso_size -= decrement;
+}
+
void __skb_warn_lro_forwarding(const struct sk_buff *skb);
static inline bool skb_warn_if_lro(const struct sk_buff *skb)
#define MSG_SENDPAGE_NOTLAST 0x20000 /* sendpage() internal : not the last page */
#define MSG_BATCH 0x40000 /* sendmmsg(): more messages coming */
#define MSG_EOF MSG_FIN
+#define MSG_NO_SHARED_FRAGS 0x80000 /* sendpage() internal : page frags are not shared */
#define MSG_ZEROCOPY 0x4000000 /* Use user data in kernel path */
#define MSG_FASTOPEN 0x20000000 /* Send data in TCP SYN */
#define TTY_PTY_LOCK 16 /* pty private */
#define TTY_NO_WRITE_SPLIT 17 /* Preserve write boundaries to driver */
#define TTY_HUPPED 18 /* Post driver->hangup() */
+#define TTY_HUPPING 19 /* Hangup in progress */
#define TTY_LDISC_HALTED 22 /* Line discipline is halted */
/* Values for tty->flow_change */
#endif
}
+static inline unsigned long
+u64_stats_update_begin_irqsave(struct u64_stats_sync *syncp)
+{
+ unsigned long flags = 0;
+
+#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+ local_irq_save(flags);
+ write_seqcount_begin(&syncp->seq);
+#endif
+ return flags;
+}
+
+static inline void
+u64_stats_update_end_irqrestore(struct u64_stats_sync *syncp,
+ unsigned long flags)
+{
+#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
+ write_seqcount_end(&syncp->seq);
+ local_irq_restore(flags);
+#endif
+}
+
static inline void u64_stats_update_begin_raw(struct u64_stats_sync *syncp)
{
#if BITS_PER_LONG==32 && defined(CONFIG_SMP)
*/
#define USB_QUIRK_DISCONNECT_SUSPEND BIT(12)
+/* Device needs a pause after every control message. */
+#define USB_QUIRK_DELAY_CTRL_MSG BIT(13)
+
#endif /* __LINUX_USB_QUIRKS_H */
int execute_in_process_context(work_func_t fn, struct execute_work *);
extern bool flush_work(struct work_struct *work);
-extern bool cancel_work(struct work_struct *work);
extern bool cancel_work_sync(struct work_struct *work);
extern bool flush_delayed_work(struct delayed_work *dwork);
int tcf_idr_create(struct tc_action_net *tn, u32 index, struct nlattr *est,
struct tc_action **a, const struct tc_action_ops *ops,
int bind, bool cpustats);
-void tcf_idr_cleanup(struct tc_action *a, struct nlattr *est);
void tcf_idr_insert(struct tc_action_net *tn, struct tc_action *a);
int __tcf_idr_release(struct tc_action *a, bool bind, bool strict);
NR__RXRPC_CALL_COMPLETIONS
};
+/*
+ * Debug ID counter for tracing.
+ */
+extern atomic_t rxrpc_debug_id;
+
typedef void (*rxrpc_notify_rx_t)(struct sock *, struct rxrpc_call *,
unsigned long);
typedef void (*rxrpc_notify_end_tx_t)(struct sock *, struct rxrpc_call *,
s64,
gfp_t,
rxrpc_notify_rx_t,
- bool);
+ bool,
+ unsigned int);
int rxrpc_kernel_send_data(struct socket *, struct rxrpc_call *,
struct msghdr *, size_t,
rxrpc_notify_end_tx_t);
struct sockaddr_rxrpc *);
u64 rxrpc_kernel_get_rtt(struct socket *, struct rxrpc_call *);
int rxrpc_kernel_charge_accept(struct socket *, rxrpc_notify_rx_t,
- rxrpc_user_attach_call_t, unsigned long, gfp_t);
+ rxrpc_user_attach_call_t, unsigned long, gfp_t,
+ unsigned int);
void rxrpc_kernel_set_tx_length(struct socket *, struct rxrpc_call *, s64);
int rxrpc_kernel_retry_call(struct socket *, struct rxrpc_call *,
struct sockaddr_rxrpc *, struct key *);
int devlink_resource_register(struct devlink *devlink,
const char *resource_name,
- bool top_hierarchy,
u64 resource_size,
u64 resource_id,
u64 parent_resource_id,
static inline int
devlink_resource_register(struct devlink *devlink,
const char *resource_name,
- bool top_hierarchy,
u64 resource_size,
u64 resource_id,
u64 parent_resource_id,
return 0;
}
+/* Special input handler for packets caught by router alert option.
+ They are selected only by protocol field, and then processed likely
+ local ones; but only if someone wants them! Otherwise, router
+ not running rsvpd will kill RSVP.
+
+ It is user level problem, what it will make with them.
+ I have no idea, how it will masquearde or NAT them (it is joke, joke :-)),
+ but receiver should be enough clever f.e. to forward mtrace requests,
+ sent to multicast group to reach destination designated router.
+ */
+
struct ip_ra_chain {
struct ip_ra_chain __rcu *next;
struct sock *sk;
struct rcu_head rcu;
};
-extern struct ip_ra_chain __rcu *ip_ra_chain;
-
/* IP flags. */
#define IP_CE 0x8000 /* Flag: "Congestion" */
#define IP_DF 0x4000 /* Flag: "Don't Fragment" */
return --iph->ttl;
}
+static inline int ip_mtu_locked(const struct dst_entry *dst)
+{
+ const struct rtable *rt = (const struct rtable *)dst;
+
+ return rt->rt_mtu_locked || dst_metric_locked(dst, RTAX_MTU);
+}
+
static inline
int ip_dont_fragment(const struct sock *sk, const struct dst_entry *dst)
{
return pmtudisc == IP_PMTUDISC_DO ||
(pmtudisc == IP_PMTUDISC_WANT &&
- !(dst_metric_locked(dst, RTAX_MTU)));
+ !ip_mtu_locked(dst));
}
static inline bool ip_sk_accept_pmtu(const struct sock *sk)
struct net *net = dev_net(dst->dev);
if (net->ipv4.sysctl_ip_fwd_use_pmtu ||
- dst_metric_locked(dst, RTAX_MTU) ||
+ ip_mtu_locked(dst) ||
!forwarding)
return dst_mtu(dst);
void rt6_sync_down_dev(struct net_device *dev, unsigned long event);
void rt6_multipath_rebalance(struct rt6_info *rt);
+void rt6_uncached_list_add(struct rt6_info *rt);
+void rt6_uncached_list_del(struct rt6_info *rt);
+
static inline const struct rt6_info *skb_rt6_info(const struct sk_buff *skb)
{
const struct dst_entry *dst = skb_dst(skb);
int fnhe_genid;
__be32 fnhe_daddr;
u32 fnhe_pmtu;
+ bool fnhe_mtu_locked;
__be32 fnhe_gw;
unsigned long fnhe_expires;
struct rtable __rcu *fnhe_rth_input;
* virtual interface might not be given air time for the transmission of
* the frame, as it is not synced with the AP/P2P GO yet, and thus the
* deauthentication frame might not be transmitted.
+ >
+ * @IEEE80211_HW_DOESNT_SUPPORT_QOS_NDP: The driver (or firmware) doesn't
+ * support QoS NDP for AP probing - that's most likely a driver bug.
*
* @NUM_IEEE80211_HW_FLAGS: number of hardware flags, used for sizing arrays
*/
IEEE80211_HW_SUPPORTS_TX_FRAG,
IEEE80211_HW_SUPPORTS_TDLS_BUFFER_STA,
IEEE80211_HW_DEAUTH_NEED_MGD_TX_PREP,
+ IEEE80211_HW_DOESNT_SUPPORT_QOS_NDP,
/* keep last, obviously */
NUM_IEEE80211_HW_FLAGS
struct sock;
struct ctl_table_header;
struct net_generic;
-struct sock;
+struct uevent_sock;
struct netns_ipvs;
struct list_head list; /* list of network namespaces */
struct list_head exit_list; /* To linked to call pernet exit
- * methods on dead net (net_sem
- * read locked), or to unregister
- * pernet ops (net_sem wr locked).
+ * methods on dead net (
+ * pernet_ops_rwsem read locked),
+ * or to unregister pernet ops
+ * (pernet_ops_rwsem write locked).
*/
struct llist_node cleanup_list; /* namespaces on death row */
struct sock *rtnl; /* rtnetlink socket */
struct sock *genl_sock;
+ struct uevent_sock *uevent_sock; /* uevent socket */
+
struct list_head dev_base_head;
struct hlist_head *dev_name_head;
struct hlist_head *dev_index_head;
/* core fib_rules */
struct list_head rules_ops;
- struct list_head fib_notifier_ops; /* protected by net_sem */
-
+ struct list_head fib_notifier_ops; /* Populated by
+ * register_pernet_subsys()
+ */
struct net_device *loopback_dev; /* The loopback */
struct netns_core core;
struct netns_mib mib;
#endif
}
+/* Protected by net_rwsem */
#define for_each_net(VAR) \
list_for_each_entry(VAR, &net_namespace_list, list)
* have to keep in mind all other pernet_operations and
* to introduce a locking, if they share common resources.
*
+ * The only time they are called with exclusive lock is
+ * from register_pernet_subsys(), unregister_pernet_subsys()
+ * register_pernet_device() and unregister_pernet_device().
+ *
* Exit methods using blocking RCU primitives, such as
* synchronize_rcu(), should be implemented via exit_batch.
* Then, destruction of a group of net requires single
void (*exit_batch)(struct list_head *net_exit_list);
unsigned int *id;
size_t size;
- /*
- * Indicates above methods are allowed to be executed in parallel
- * with methods of any other pernet_operations, i.e. they are not
- * need write locked net_sem.
- */
- bool async;
};
/*
#endif
struct ipv4_devconf *devconf_all;
struct ipv4_devconf *devconf_dflt;
+ struct ip_ra_chain __rcu *ra_chain;
+ struct mutex ra_mutex;
#ifdef CONFIG_IP_MULTIPLE_TABLES
struct fib_rules_ops *rules_ops;
bool fib_has_custom_rules;
atomic_t fib6_sernum;
struct seg6_pernet_data *seg6_data;
struct fib_notifier_ops *notifier_ops;
+ struct fib_notifier_ops *ip6mr_notifier_ops;
+ unsigned int ipmr_seq; /* protected by rtnl_mutex */
struct {
struct hlist_head head;
spinlock_t lock;
__be32 rt_gateway;
/* Miscellaneous cached information */
- u32 rt_pmtu;
+ u32 rt_mtu_locked:1,
+ rt_pmtu:31;
struct list_head rt_uncached;
struct uncached_list *rt_uncached_list;
void fib_add_ifaddr(struct in_ifaddr *);
void fib_del_ifaddr(struct in_ifaddr *, struct in_ifaddr *);
+void rt_add_uncached_list(struct rtable *rt);
+void rt_del_uncached_list(struct rtable *rt);
+
static inline void ip_rt_put(struct rtable *rt)
{
/* dst_release() accepts a NULL parameter.
--- /dev/null
+/**
+ * Copyright (c) 2017 Redpine Signals Inc.
+ *
+ * Permission to use, copy, modify, and/or distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#ifndef __RSI_HEADER_H__
+#define __RSI_HEADER_H__
+
+#include <linux/skbuff.h>
+
+/* HAL queue information */
+#define RSI_COEX_Q 0x0
+#define RSI_BT_Q 0x2
+#define RSI_WLAN_Q 0x3
+#define RSI_WIFI_MGMT_Q 0x4
+#define RSI_WIFI_DATA_Q 0x5
+#define RSI_BT_MGMT_Q 0x6
+#define RSI_BT_DATA_Q 0x7
+
+enum rsi_coex_queues {
+ RSI_COEX_Q_INVALID = -1,
+ RSI_COEX_Q_COMMON = 0,
+ RSI_COEX_Q_BT,
+ RSI_COEX_Q_WLAN
+};
+
+enum rsi_host_intf {
+ RSI_HOST_INTF_SDIO = 0,
+ RSI_HOST_INTF_USB
+};
+
+struct rsi_proto_ops {
+ int (*coex_send_pkt)(void *priv, struct sk_buff *skb, u8 hal_queue);
+ enum rsi_host_intf (*get_host_intf)(void *priv);
+ void (*set_bt_context)(void *priv, void *context);
+};
+
+struct rsi_mod_ops {
+ int (*attach)(void *priv, struct rsi_proto_ops *ops);
+ void (*detach)(void *priv);
+ int (*recv_pkt)(void *priv, const u8 *msg);
+};
+
+extern const struct rsi_mod_ops rsi_bt_ops;
+#endif
*to_free = skb;
}
+static inline void __qdisc_drop_all(struct sk_buff *skb,
+ struct sk_buff **to_free)
+{
+ if (skb->prev)
+ skb->prev->next = *to_free;
+ else
+ skb->next = *to_free;
+ *to_free = skb;
+}
+
static inline unsigned int __qdisc_queue_drop_head(struct Qdisc *sch,
struct qdisc_skb_head *qh,
struct sk_buff **to_free)
return NET_XMIT_DROP;
}
+static inline int qdisc_drop_all(struct sk_buff *skb, struct Qdisc *sch,
+ struct sk_buff **to_free)
+{
+ __qdisc_drop_all(skb, to_free);
+ qdisc_qstats_drop(sch);
+
+ return NET_XMIT_DROP;
+}
+
/* Length to Time (L2T) lookup in a qdisc_rate_table, to determine how
long it will take to send a packet given its size.
*/
const struct sctp_endpoint *ep,
const union sctp_addr *paddr,
struct sctp_transport **);
-int sctp_endpoint_is_peeled_off(struct sctp_endpoint *,
- const union sctp_addr *);
+bool sctp_endpoint_is_peeled_off(struct sctp_endpoint *ep,
+ const union sctp_addr *paddr);
struct sctp_endpoint *sctp_endpoint_is_match(struct sctp_endpoint *,
struct net *, const union sctp_addr *);
-int sctp_has_association(struct net *net, const union sctp_addr *laddr,
- const union sctp_addr *paddr);
+bool sctp_has_association(struct net *net, const union sctp_addr *laddr,
+ const union sctp_addr *paddr);
int sctp_verify_init(struct net *net, const struct sctp_endpoint *ep,
const struct sctp_association *asoc,
int proto_register(struct proto *prot, int alloc_slab);
void proto_unregister(struct proto *prot);
+int sock_load_diag_module(int family, int protocol);
#ifdef SOCK_REFCNT_DEBUG
static inline void sk_refcnt_debug_inc(struct sock *sk)
bool sk_page_frag_refill(struct sock *sk, struct page_frag *pfrag);
+int sk_alloc_sg(struct sock *sk, int len, struct scatterlist *sg,
+ int sg_start, int *sg_curr, unsigned int *sg_size,
+ int first_coalesce);
+
/*
* Default write policy as shown to user space via poll/select/SIGIO
*/
#include <linux/socket.h>
#include <linux/tcp.h>
#include <net/tcp.h>
+#include <net/strparser.h>
#include <uapi/linux/tls.h>
struct tls_sw_context {
struct crypto_aead *aead_send;
+ struct crypto_aead *aead_recv;
struct crypto_wait async_wait;
+ /* Receive context */
+ struct strparser strp;
+ void (*saved_data_ready)(struct sock *sk);
+ unsigned int (*sk_poll)(struct file *file, struct socket *sock,
+ struct poll_table_struct *wait);
+ struct sk_buff *recv_pkt;
+ u8 control;
+ bool decrypted;
+
/* Sending context */
char aad_space[TLS_AAD_SPACE_SIZE];
TLS_PENDING_CLOSED_RECORD
};
+struct cipher_context {
+ u16 prepend_size;
+ u16 tag_size;
+ u16 overhead_size;
+ u16 iv_size;
+ char *iv;
+ u16 rec_seq_size;
+ char *rec_seq;
+};
+
struct tls_context {
union {
struct tls_crypto_info crypto_send;
struct tls12_crypto_info_aes_gcm_128 crypto_send_aes_gcm_128;
};
+ union {
+ struct tls_crypto_info crypto_recv;
+ struct tls12_crypto_info_aes_gcm_128 crypto_recv_aes_gcm_128;
+ };
void *priv_ctx;
- u8 tx_conf:2;
+ u8 conf:2;
- u16 prepend_size;
- u16 tag_size;
- u16 overhead_size;
- u16 iv_size;
- char *iv;
- u16 rec_seq_size;
- char *rec_seq;
+ struct cipher_context tx;
+ struct cipher_context rx;
struct scatterlist *partially_sent_record;
u16 partially_sent_offset;
unsigned int optlen);
-int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx);
+int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx);
int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
int tls_sw_sendpage(struct sock *sk, struct page *page,
int offset, size_t size, int flags);
void tls_sw_close(struct sock *sk, long timeout);
-void tls_sw_free_tx_resources(struct sock *sk);
+void tls_sw_free_resources(struct sock *sk);
+int tls_sw_recvmsg(struct sock *sk, struct msghdr *msg, size_t len,
+ int nonblock, int flags, int *addr_len);
+unsigned int tls_sw_poll(struct file *file, struct socket *sock,
+ struct poll_table_struct *wait);
+ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos,
+ struct pipe_inode_info *pipe,
+ size_t len, unsigned int flags);
void tls_sk_destruct(struct sock *sk, struct tls_context *ctx);
void tls_icsk_clean_acked(struct sock *sk);
return tls_ctx->pending_open_record_frags;
}
-static inline void tls_err_abort(struct sock *sk)
+static inline void tls_err_abort(struct sock *sk, int err)
{
- sk->sk_err = EBADMSG;
+ sk->sk_err = err;
sk->sk_error_report(sk);
}
}
static inline void tls_advance_record_sn(struct sock *sk,
- struct tls_context *ctx)
+ struct cipher_context *ctx)
{
if (tls_bigint_increment(ctx->rec_seq, ctx->rec_seq_size))
- tls_err_abort(sk);
+ tls_err_abort(sk, EBADMSG);
tls_bigint_increment(ctx->iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE,
ctx->iv_size);
}
size_t plaintext_len,
unsigned char record_type)
{
- size_t pkt_len, iv_size = ctx->iv_size;
+ size_t pkt_len, iv_size = ctx->tx.iv_size;
- pkt_len = plaintext_len + iv_size + ctx->tag_size;
+ pkt_len = plaintext_len + iv_size + ctx->tx.tag_size;
/* we cover nonce explicit here as well, so buf should be of
* size KTLS_DTLS_HEADER_SIZE + KTLS_DTLS_NONCE_EXPLICIT_SIZE
buf[3] = pkt_len >> 8;
buf[4] = pkt_len & 0xFF;
memcpy(buf + TLS_NONCE_OFFSET,
- ctx->iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE, iv_size);
+ ctx->tx.iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE, iv_size);
}
static inline void tls_make_aad(char *buf,
struct mutex tgt_qp_mutex;
struct list_head tgt_qp_list;
- /*
- * Implementation details of the RDMA core, don't use in drivers:
- */
- struct rdma_restrack_entry res;
};
struct ib_ah {
* net device of device @device at port @port_num or NULL if such
* a net device doesn't exist. The vendor driver should call dev_hold
* on this net device. The HW vendor's device driver must guarantee
- * that this function returns NULL before the net device reaches
- * NETDEV_UNREGISTER_FINAL state.
+ * that this function returns NULL before the net device has finished
+ * NETDEV_UNREGISTER state.
*/
struct net_device *(*get_netdev)(struct ib_device *device,
u8 port_num);
struct list_head list; /* scsi_cmnd participates in queue lists */
struct list_head eh_entry; /* entry for the host eh_cmd_q */
struct delayed_work abort_work;
+
+ struct rcu_head rcu;
+
int eh_eflags; /* Used by error handlr */
/*
struct blk_mq_tag_set tag_set;
};
- struct rcu_head rcu;
-
atomic_t host_busy; /* commands actually active on low-level */
atomic_t host_blocked;
TP_ARGS(call, count, offset, want_more, ret),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, rxcall )
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(enum afs_call_state, state )
__field(unsigned int, count )
__field(unsigned int, offset )
),
TP_fast_assign(
- __entry->rxcall = call->rxcall;
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->state = call->state;
__entry->unmarshall = call->unmarshall;
__entry->count = count;
__entry->ret = ret;
),
- TP_printk("c=%p ac=%p s=%u u=%u %u/%u wm=%u ret=%d",
- __entry->rxcall,
+ TP_printk("c=%08x s=%u u=%u %u/%u wm=%u ret=%d",
__entry->call,
__entry->state, __entry->unmarshall,
__entry->offset, __entry->count,
TP_ARGS(rxcall, call),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, rxcall )
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(enum afs_call_state, state )
__field(unsigned short, unmarshall )
),
TP_fast_assign(
- __entry->rxcall = rxcall;
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->state = call->state;
__entry->unmarshall = call->unmarshall;
),
- TP_printk("c=%p ac=%p s=%u u=%u",
- __entry->rxcall,
+ TP_printk("c=%08x s=%u u=%u",
__entry->call,
__entry->state, __entry->unmarshall)
);
TP_ARGS(call),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, rxcall )
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(const char *, name )
__field(u32, op )
),
TP_fast_assign(
- __entry->rxcall = call->rxcall;
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->name = call->type->name;
__entry->op = call->operation_ID;
),
- TP_printk("c=%p ac=%p %s o=%u",
- __entry->rxcall,
+ TP_printk("c=%08x %s o=%u",
__entry->call,
__entry->name,
__entry->op)
TP_ARGS(call, op, usage, outstanding, where),
TP_STRUCT__entry(
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(int, op )
__field(int, usage )
__field(int, outstanding )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->op = op;
__entry->usage = usage;
__entry->outstanding = outstanding;
__entry->where = where;
),
- TP_printk("c=%p %s u=%d o=%d sp=%pSR",
+ TP_printk("c=%08x %s u=%d o=%d sp=%pSR",
__entry->call,
__print_symbolic(__entry->op, afs_call_traces),
__entry->usage,
TP_ARGS(call, fid),
TP_STRUCT__entry(
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(enum afs_fs_operation, op )
__field_struct(struct afs_fid, fid )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->op = call->operation_ID;
if (fid) {
__entry->fid = *fid;
}
),
- TP_printk("c=%p %06x:%06x:%06x %s",
+ TP_printk("c=%08x %06x:%06x:%06x %s",
__entry->call,
__entry->fid.vid,
__entry->fid.vnode,
TP_ARGS(call),
TP_STRUCT__entry(
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(enum afs_vl_operation, op )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->op = call->operation_ID;
),
- TP_printk("c=%p %s",
+ TP_printk("c=%08x %s",
__entry->call,
__print_symbolic(__entry->op, afs_vl_operations))
);
TP_ARGS(call),
TP_STRUCT__entry(
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(struct rxrpc_call *, rx_call )
__field(int, ret )
__field(u32, abort_code )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->rx_call = call->rxcall;
__entry->ret = call->error;
__entry->abort_code = call->abort_code;
),
- TP_printk(" c=%p ret=%d ab=%d [%p]",
+ TP_printk(" c=%08x ret=%d ab=%d [%p]",
__entry->call,
__entry->ret,
__entry->abort_code,
TP_ARGS(call, msg, first, last, offset),
TP_STRUCT__entry(
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(pgoff_t, first )
__field(pgoff_t, last )
__field(unsigned int, nr )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->first = first;
__entry->last = last;
__entry->nr = msg->msg_iter.nr_segs;
__entry->flags = msg->msg_flags;
),
- TP_printk(" c=%p %lx-%lx-%lx b=%x o=%x f=%x",
+ TP_printk(" c=%08x %lx-%lx-%lx b=%x o=%x f=%x",
__entry->call,
__entry->first, __entry->first + __entry->nr - 1, __entry->last,
__entry->bytes, __entry->offset,
TP_ARGS(call, first, last, cursor, ret),
TP_STRUCT__entry(
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(pgoff_t, first )
__field(pgoff_t, last )
__field(pgoff_t, cursor )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->first = first;
__entry->last = last;
__entry->cursor = cursor;
__entry->ret = ret;
),
- TP_printk(" c=%p %lx-%lx c=%lx r=%d",
+ TP_printk(" c=%08x %lx-%lx c=%lx r=%d",
__entry->call,
__entry->first, __entry->last,
__entry->cursor, __entry->ret)
TP_ARGS(call, from, to, ret, remote_abort),
TP_STRUCT__entry(
- __field(struct afs_call *, call )
+ __field(unsigned int, call )
__field(enum afs_call_state, from )
__field(enum afs_call_state, to )
__field(int, ret )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->from = from;
__entry->to = to;
__entry->ret = ret;
__entry->abort = remote_abort;
),
- TP_printk("c=%p %u->%u r=%d ab=%d",
+ TP_printk("c=%08x %u->%u r=%d ab=%d",
__entry->call,
__entry->from, __entry->to,
__entry->ret, __entry->abort)
__entry->stop_flags, __entry->stop_retries,
__entry->sbc_opcode, __entry->sbc_arg,
__entry->sbc_flags, __entry->sbc_retries,
- __entry->blocks, __entry->blk_addr,
- __entry->blksz, __entry->data_flags, __entry->tag,
+ __entry->blocks, __entry->blksz,
+ __entry->blk_addr, __entry->data_flags, __entry->tag,
__entry->can_retune, __entry->doing_retune,
__entry->retune_now, __entry->need_retune,
__entry->hold_retune, __entry->retune_period)
EM(RXRPC_ACK_IDLE, "IDL") \
E_(RXRPC_ACK__INVALID, "-?-")
+#define rxrpc_completions \
+ EM(RXRPC_CALL_SUCCEEDED, "Succeeded") \
+ EM(RXRPC_CALL_REMOTELY_ABORTED, "RemoteAbort") \
+ EM(RXRPC_CALL_LOCALLY_ABORTED, "LocalAbort") \
+ EM(RXRPC_CALL_LOCAL_ERROR, "LocalError") \
+ E_(RXRPC_CALL_NETWORK_ERROR, "NetError")
+
/*
* Export enum symbols via userspace.
*/
rxrpc_timer_traces;
rxrpc_propose_ack_traces;
rxrpc_propose_ack_outcomes;
+rxrpc_congest_modes;
rxrpc_congest_changes;
/*
TP_ARGS(conn, op, usage, where),
TP_STRUCT__entry(
- __field(struct rxrpc_connection *, conn )
- __field(int, op )
- __field(int, usage )
- __field(const void *, where )
+ __field(unsigned int, conn )
+ __field(int, op )
+ __field(int, usage )
+ __field(const void *, where )
),
TP_fast_assign(
- __entry->conn = conn;
+ __entry->conn = conn->debug_id;
__entry->op = op;
__entry->usage = usage;
__entry->where = where;
),
- TP_printk("C=%p %s u=%d sp=%pSR",
+ TP_printk("C=%08x %s u=%d sp=%pSR",
__entry->conn,
__print_symbolic(__entry->op, rxrpc_conn_traces),
__entry->usage,
TP_ARGS(conn, channel, op),
TP_STRUCT__entry(
- __field(struct rxrpc_connection *, conn )
+ __field(unsigned int, conn )
__field(u32, cid )
__field(int, channel )
__field(int, usage )
),
TP_fast_assign(
- __entry->conn = conn;
+ __entry->conn = conn->debug_id;
__entry->channel = channel;
__entry->usage = atomic_read(&conn->usage);
__entry->op = op;
__entry->cs = conn->cache_state;
),
- TP_printk("C=%p h=%2d %s %s i=%08x u=%d",
+ TP_printk("C=%08x h=%2d %s %s i=%08x u=%d",
__entry->conn,
__entry->channel,
__print_symbolic(__entry->op, rxrpc_client_traces),
TP_ARGS(call, op, usage, where, aux),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(int, op )
__field(int, usage )
__field(const void *, where )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->op = op;
__entry->usage = usage;
__entry->where = where;
__entry->aux = aux;
),
- TP_printk("c=%p %s u=%d sp=%pSR a=%p",
+ TP_printk("c=%08x %s u=%d sp=%pSR a=%p",
__entry->call,
__print_symbolic(__entry->op, rxrpc_call_traces),
__entry->usage,
);
TRACE_EVENT(rxrpc_abort,
- TP_PROTO(const char *why, u32 cid, u32 call_id, rxrpc_seq_t seq,
- int abort_code, int error),
+ TP_PROTO(unsigned int call_nr, const char *why, u32 cid, u32 call_id,
+ rxrpc_seq_t seq, int abort_code, int error),
- TP_ARGS(why, cid, call_id, seq, abort_code, error),
+ TP_ARGS(call_nr, why, cid, call_id, seq, abort_code, error),
TP_STRUCT__entry(
+ __field(unsigned int, call_nr )
__array(char, why, 4 )
__field(u32, cid )
__field(u32, call_id )
TP_fast_assign(
memcpy(__entry->why, why, 4);
+ __entry->call_nr = call_nr;
__entry->cid = cid;
__entry->call_id = call_id;
__entry->abort_code = abort_code;
__entry->seq = seq;
),
- TP_printk("%08x:%08x s=%u a=%d e=%d %s",
+ TP_printk("c=%08x %08x:%08x s=%u a=%d e=%d %s",
+ __entry->call_nr,
__entry->cid, __entry->call_id, __entry->seq,
__entry->abort_code, __entry->error, __entry->why)
);
+TRACE_EVENT(rxrpc_call_complete,
+ TP_PROTO(struct rxrpc_call *call),
+
+ TP_ARGS(call),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, call )
+ __field(enum rxrpc_call_completion, compl )
+ __field(int, error )
+ __field(u32, abort_code )
+ ),
+
+ TP_fast_assign(
+ __entry->call = call->debug_id;
+ __entry->compl = call->completion;
+ __entry->error = call->error;
+ __entry->abort_code = call->abort_code;
+ ),
+
+ TP_printk("c=%08x %s r=%d ac=%d",
+ __entry->call,
+ __print_symbolic(__entry->compl, rxrpc_completions),
+ __entry->error,
+ __entry->abort_code)
+ );
+
TRACE_EVENT(rxrpc_transmit,
TP_PROTO(struct rxrpc_call *call, enum rxrpc_transmit_trace why),
TP_ARGS(call, why),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_transmit_trace, why )
__field(rxrpc_seq_t, tx_hard_ack )
__field(rxrpc_seq_t, tx_top )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->why = why;
__entry->tx_hard_ack = call->tx_hard_ack;
__entry->tx_top = call->tx_top;
__entry->tx_winsize = call->tx_winsize;
),
- TP_printk("c=%p %s f=%08x n=%u/%u",
+ TP_printk("c=%08x %s f=%08x n=%u/%u",
__entry->call,
__print_symbolic(__entry->why, rxrpc_transmit_traces),
__entry->tx_hard_ack + 1,
TP_ARGS(call, seq, serial, flags, anno),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_seq_t, seq )
__field(rxrpc_serial_t, serial )
__field(u8, flags )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->seq = seq;
__entry->serial = serial;
__entry->flags = flags;
__entry->anno = anno;
),
- TP_printk("c=%p DATA %08x q=%08x fl=%02x a=%02x",
+ TP_printk("c=%08x DATA %08x q=%08x fl=%02x a=%02x",
__entry->call,
__entry->serial,
__entry->seq,
TP_ARGS(call, serial, ack_serial, first, prev, reason, n_acks),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_serial_t, serial )
__field(rxrpc_serial_t, ack_serial )
__field(rxrpc_seq_t, first )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->serial = serial;
__entry->ack_serial = ack_serial;
__entry->first = first;
__entry->n_acks = n_acks;
),
- TP_printk("c=%p %08x %s r=%08x f=%08x p=%08x n=%u",
+ TP_printk("c=%08x %08x %s r=%08x f=%08x p=%08x n=%u",
__entry->call,
__entry->serial,
__print_symbolic(__entry->reason, rxrpc_ack_names),
TP_ARGS(call, serial, abort_code),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_serial_t, serial )
__field(u32, abort_code )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->serial = serial;
__entry->abort_code = abort_code;
),
- TP_printk("c=%p ABORT %08x ac=%d",
+ TP_printk("c=%08x ABORT %08x ac=%d",
__entry->call,
__entry->serial,
__entry->abort_code)
TP_ARGS(call, serial, rwind, wake),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_serial_t, serial )
__field(u32, rwind )
__field(bool, wake )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->serial = serial;
__entry->rwind = rwind;
__entry->wake = wake;
),
- TP_printk("c=%p %08x rw=%u%s",
+ TP_printk("c=%08x %08x rw=%u%s",
__entry->call,
__entry->serial,
__entry->rwind,
TP_ARGS(call, seq, serial, flags, retrans, lose),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_seq_t, seq )
__field(rxrpc_serial_t, serial )
__field(u8, flags )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->seq = seq;
__entry->serial = serial;
__entry->flags = flags;
__entry->lose = lose;
),
- TP_printk("c=%p DATA %08x q=%08x fl=%02x%s%s",
+ TP_printk("c=%08x DATA %08x q=%08x fl=%02x%s%s",
__entry->call,
__entry->serial,
__entry->seq,
TP_ARGS(call, serial, ack_first, ack_serial, reason, n_acks),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_serial_t, serial )
__field(rxrpc_seq_t, ack_first )
__field(rxrpc_serial_t, ack_serial )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call ? call->debug_id : 0;
__entry->serial = serial;
__entry->ack_first = ack_first;
__entry->ack_serial = ack_serial;
__entry->n_acks = n_acks;
),
- TP_printk(" c=%p ACK %08x %s f=%08x r=%08x n=%u",
+ TP_printk(" c=%08x ACK %08x %s f=%08x r=%08x n=%u",
__entry->call,
__entry->serial,
__print_symbolic(__entry->reason, rxrpc_ack_names),
TP_ARGS(call, why, serial, seq),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_receive_trace, why )
__field(rxrpc_serial_t, serial )
__field(rxrpc_seq_t, seq )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->why = why;
__entry->serial = serial;
__entry->seq = seq;
__entry->top = call->rx_top;
),
- TP_printk("c=%p %s r=%08x q=%08x w=%08x-%08x",
+ TP_printk("c=%08x %s r=%08x q=%08x w=%08x-%08x",
__entry->call,
__print_symbolic(__entry->why, rxrpc_receive_traces),
__entry->serial,
TP_ARGS(call, why, seq, offset, len, ret),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_recvmsg_trace, why )
__field(rxrpc_seq_t, seq )
__field(unsigned int, offset )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->why = why;
__entry->seq = seq;
__entry->offset = offset;
__entry->ret = ret;
),
- TP_printk("c=%p %s q=%08x o=%u l=%u ret=%d",
+ TP_printk("c=%08x %s q=%08x o=%u l=%u ret=%d",
__entry->call,
__print_symbolic(__entry->why, rxrpc_recvmsg_traces),
__entry->seq,
TP_ARGS(call, why, send_serial),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_rtt_tx_trace, why )
__field(rxrpc_serial_t, send_serial )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->why = why;
__entry->send_serial = send_serial;
),
- TP_printk("c=%p %s sr=%08x",
+ TP_printk("c=%08x %s sr=%08x",
__entry->call,
__print_symbolic(__entry->why, rxrpc_rtt_tx_traces),
__entry->send_serial)
TP_ARGS(call, why, send_serial, resp_serial, rtt, nr, avg),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_rtt_rx_trace, why )
__field(u8, nr )
__field(rxrpc_serial_t, send_serial )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->why = why;
__entry->send_serial = send_serial;
__entry->resp_serial = resp_serial;
__entry->avg = avg;
),
- TP_printk("c=%p %s sr=%08x rr=%08x rtt=%lld nr=%u avg=%lld",
+ TP_printk("c=%08x %s sr=%08x rr=%08x rtt=%lld nr=%u avg=%lld",
__entry->call,
__print_symbolic(__entry->why, rxrpc_rtt_rx_traces),
__entry->send_serial,
TP_ARGS(call, why, now),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_timer_trace, why )
__field(long, now )
__field(long, ack_at )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->why = why;
__entry->now = now;
__entry->ack_at = call->ack_at;
__entry->timer = call->timer.expires;
),
- TP_printk("c=%p %s a=%ld la=%ld r=%ld xr=%ld xq=%ld xt=%ld t=%ld",
+ TP_printk("c=%08x %s a=%ld la=%ld r=%ld xr=%ld xq=%ld xt=%ld t=%ld",
__entry->call,
__print_symbolic(__entry->why, rxrpc_timer_traces),
__entry->ack_at - __entry->now,
outcome),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_propose_ack_trace, why )
__field(rxrpc_serial_t, serial )
__field(u8, ack_reason )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->why = why;
__entry->serial = serial;
__entry->ack_reason = ack_reason;
__entry->outcome = outcome;
),
- TP_printk("c=%p %s %s r=%08x i=%u b=%u%s",
+ TP_printk("c=%08x %s %s r=%08x i=%u b=%u%s",
__entry->call,
__print_symbolic(__entry->why, rxrpc_propose_ack_traces),
__print_symbolic(__entry->ack_reason, rxrpc_ack_names),
TP_ARGS(call, seq, annotation, expiry),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_seq_t, seq )
__field(u8, annotation )
__field(s64, expiry )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->seq = seq;
__entry->annotation = annotation;
__entry->expiry = expiry;
),
- TP_printk("c=%p q=%x a=%02x xp=%lld",
+ TP_printk("c=%08x q=%x a=%02x xp=%lld",
__entry->call,
__entry->seq,
__entry->annotation,
TP_ARGS(call, summary, ack_serial, change),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(enum rxrpc_congest_change, change )
__field(rxrpc_seq_t, hard_ack )
__field(rxrpc_seq_t, top )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->change = change;
__entry->hard_ack = call->tx_hard_ack;
__entry->top = call->tx_top;
memcpy(&__entry->sum, summary, sizeof(__entry->sum));
),
- TP_printk("c=%p r=%08x %s q=%08x %s cw=%u ss=%u nr=%u,%u nw=%u,%u r=%u b=%u u=%u d=%u l=%x%s%s%s",
+ TP_printk("c=%08x r=%08x %s q=%08x %s cw=%u ss=%u nr=%u,%u nw=%u,%u r=%u b=%u u=%u d=%u l=%x%s%s%s",
__entry->call,
__entry->ack_serial,
__print_symbolic(__entry->sum.ack_reason, rxrpc_ack_names),
TP_ARGS(call),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(u32, abort_code )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->abort_code = call->abort_code;
),
- TP_printk("c=%p ab=%08x",
+ TP_printk("c=%08x ab=%08x",
__entry->call,
__entry->abort_code)
);
TP_ARGS(call),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(u32, abort_code )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->abort_code = call->abort_code;
),
- TP_printk("c=%p ab=%08x",
+ TP_printk("c=%08x ab=%08x",
__entry->call,
__entry->abort_code)
);
TP_ARGS(call, serial, why),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(rxrpc_serial_t, serial )
__field(const char *, why )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->serial = serial;
__entry->why = why;
),
- TP_printk("c=%p EPROTO %08x %s",
+ TP_printk("c=%08x EPROTO %08x %s",
__entry->call,
__entry->serial,
__entry->why)
TP_ARGS(call),
TP_STRUCT__entry(
- __field(struct rxrpc_call *, call )
+ __field(unsigned int, call )
__field(unsigned long, user_call_ID )
__field(u32, cid )
__field(u32, call_id )
),
TP_fast_assign(
- __entry->call = call;
+ __entry->call = call->debug_id;
__entry->user_call_ID = call->user_call_ID;
__entry->cid = call->cid;
__entry->call_id = call->call_id;
),
- TP_printk("c=%p u=%p %08x:%08x",
+ TP_printk("c=%08x u=%p %08x:%08x",
__entry->call,
(void *)__entry->user_call_ID,
__entry->cid,
__entry->call_id)
);
+TRACE_EVENT(rxrpc_resend,
+ TP_PROTO(struct rxrpc_call *call, int ix),
+
+ TP_ARGS(call, ix),
+
+ TP_STRUCT__entry(
+ __field(unsigned int, call )
+ __field(int, ix )
+ __array(u8, anno, 64 )
+ ),
+
+ TP_fast_assign(
+ __entry->call = call->debug_id;
+ __entry->ix = ix;
+ memcpy(__entry->anno, call->rxtx_annotations, 64);
+ ),
+
+ TP_printk("c=%08x ix=%u a=%64phN",
+ __entry->call,
+ __entry->ix,
+ __entry->anno)
+ );
+
#endif /* _TRACE_RXRPC_H */
/* This part must be outside protection */
short _addr_lsb; /* LSB of the reported address */
/* used when si_code=SEGV_BNDERR */
struct {
- short _dummy_bnd;
+ void *_dummy_bnd;
void __user *_lower;
void __user *_upper;
} _addr_bnd;
/* used when si_code=SEGV_PKUERR */
struct {
- short _dummy_pkey;
+ void *_dummy_pkey;
__u32 _pkey;
} _addr_pkey;
};
BATADV_TT_CLIENT_TEMP = (1 << 11),
};
+/**
+ * enum batadv_mcast_flags_priv - Private, own multicast flags
+ *
+ * These are internal, multicast related flags. Currently they describe certain
+ * multicast related attributes of the segment this originator bridges into the
+ * mesh.
+ *
+ * Those attributes are used to determine the public multicast flags this
+ * originator is going to announce via TT.
+ *
+ * For netlink, if BATADV_MCAST_FLAGS_BRIDGED is unset then all querier
+ * related flags are undefined.
+ */
+enum batadv_mcast_flags_priv {
+ /**
+ * @BATADV_MCAST_FLAGS_BRIDGED: There is a bridge on top of the mesh
+ * interface.
+ */
+ BATADV_MCAST_FLAGS_BRIDGED = (1 << 0),
+
+ /**
+ * @BATADV_MCAST_FLAGS_QUERIER_IPV4_EXISTS: Whether an IGMP querier
+ * exists in the mesh
+ */
+ BATADV_MCAST_FLAGS_QUERIER_IPV4_EXISTS = (1 << 1),
+
+ /**
+ * @BATADV_MCAST_FLAGS_QUERIER_IPV6_EXISTS: Whether an MLD querier
+ * exists in the mesh
+ */
+ BATADV_MCAST_FLAGS_QUERIER_IPV6_EXISTS = (1 << 2),
+
+ /**
+ * @BATADV_MCAST_FLAGS_QUERIER_IPV4_SHADOWING: If an IGMP querier
+ * exists, whether it is potentially shadowing multicast listeners
+ * (i.e. querier is behind our own bridge segment)
+ */
+ BATADV_MCAST_FLAGS_QUERIER_IPV4_SHADOWING = (1 << 3),
+
+ /**
+ * @BATADV_MCAST_FLAGS_QUERIER_IPV6_SHADOWING: If an MLD querier
+ * exists, whether it is potentially shadowing multicast listeners
+ * (i.e. querier is behind our own bridge segment)
+ */
+ BATADV_MCAST_FLAGS_QUERIER_IPV6_SHADOWING = (1 << 4),
+};
+
/**
* enum batadv_nl_attrs - batman-adv netlink attributes
*/
*/
BATADV_ATTR_BLA_CRC,
+ /**
+ * @BATADV_ATTR_DAT_CACHE_IP4ADDRESS: Client IPv4 address
+ */
+ BATADV_ATTR_DAT_CACHE_IP4ADDRESS,
+
+ /**
+ * @BATADV_ATTR_DAT_CACHE_HWADDRESS: Client MAC address
+ */
+ BATADV_ATTR_DAT_CACHE_HWADDRESS,
+
+ /**
+ * @BATADV_ATTR_DAT_CACHE_VID: VLAN ID
+ */
+ BATADV_ATTR_DAT_CACHE_VID,
+
+ /**
+ * @BATADV_ATTR_MCAST_FLAGS: Per originator multicast flags
+ */
+ BATADV_ATTR_MCAST_FLAGS,
+
+ /**
+ * @BATADV_ATTR_MCAST_FLAGS_PRIV: Private, own multicast flags
+ */
+ BATADV_ATTR_MCAST_FLAGS_PRIV,
+
/* add attributes above here, update the policy in netlink.c */
/**
*/
BATADV_CMD_GET_BLA_BACKBONE,
+ /**
+ * @BATADV_CMD_GET_DAT_CACHE: Query list of DAT cache entries
+ */
+ BATADV_CMD_GET_DAT_CACHE,
+
+ /**
+ * @BATADV_CMD_GET_MCAST_FLAGS: Query list of multicast flags
+ */
+ BATADV_CMD_GET_MCAST_FLAGS,
+
/* add new commands above here */
/**
BPF_PROG_TYPE_SOCK_OPS,
BPF_PROG_TYPE_SK_SKB,
BPF_PROG_TYPE_CGROUP_DEVICE,
+ BPF_PROG_TYPE_SK_MSG,
};
enum bpf_attach_type {
BPF_SK_SKB_STREAM_PARSER,
BPF_SK_SKB_STREAM_VERDICT,
BPF_CGROUP_DEVICE,
+ BPF_SK_MSG_VERDICT,
__MAX_BPF_ATTACH_TYPE
};
#define BPF_F_RDONLY (1U << 3)
#define BPF_F_WRONLY (1U << 4)
+/* Flag for stack_map, store build_id+offset instead of pointer */
+#define BPF_F_STACK_BUILD_ID (1U << 5)
+
+enum bpf_stack_build_id_status {
+ /* user space need an empty entry to identify end of a trace */
+ BPF_STACK_BUILD_ID_EMPTY = 0,
+ /* with valid build_id and offset */
+ BPF_STACK_BUILD_ID_VALID = 1,
+ /* couldn't get build_id, fallback to ip */
+ BPF_STACK_BUILD_ID_IP = 2,
+};
+
+#define BPF_BUILD_ID_SIZE 20
+struct bpf_stack_build_id {
+ __s32 status;
+ unsigned char build_id[BPF_BUILD_ID_SIZE];
+ union {
+ __u64 offset;
+ __u64 ip;
+ };
+};
+
union bpf_attr {
struct { /* anonymous struct used by BPF_MAP_CREATE command */
__u32 map_type; /* one of enum bpf_map_type */
* int bpf_override_return(pt_regs, rc)
* @pt_regs: pointer to struct pt_regs
* @rc: the return value to set
+ *
+ * int bpf_msg_redirect_map(map, key, flags)
+ * Redirect msg to a sock in map using key as a lookup key for the
+ * sock in map.
+ * @map: pointer to sockmap
+ * @key: key to lookup sock in map
+ * @flags: reserved for future use
+ * Return: SK_PASS
+ *
*/
#define __BPF_FUNC_MAPPER(FN) \
FN(unspec), \
FN(perf_prog_read_value), \
FN(getsockopt), \
FN(override_return), \
- FN(sock_ops_cb_flags_set),
+ FN(sock_ops_cb_flags_set), \
+ FN(msg_redirect_map), \
+ FN(msg_apply_bytes), \
+ FN(msg_cork_bytes), \
+ FN(msg_pull_data),
/* integer value in 'imm' field of BPF_CALL instruction selects which helper
* function eBPF program intends to call
SK_PASS,
};
+/* user accessible metadata for SK_MSG packet hook, new fields must
+ * be added to the end of this structure
+ */
+struct sk_msg_md {
+ void *data;
+ void *data_end;
+};
+
#define BPF_TAG_SIZE 8
struct bpf_prog_info {
struct bpf_perf_event_data {
bpf_user_pt_regs_t regs;
__u64 sample_period;
+ __u64 addr;
};
#endif /* _UAPI__LINUX_BPF_PERF_EVENT_H__ */
__u32 data;
};
+#define PFC_STORM_PREVENTION_AUTO 0xffff
+#define PFC_STORM_PREVENTION_DISABLE 0
+
enum tunable_id {
ETHTOOL_ID_UNSPEC,
ETHTOOL_RX_COPYBREAK,
ETHTOOL_TX_COPYBREAK,
+ ETHTOOL_PFC_PREVENTION_TOUT, /* timeout in msecs */
/*
* Add your fresh new tubale attribute above and remember to update
* tunable_strings[] in net/core/ethtool.c
*/
#define ETH_ALEN 6 /* Octets in one ethernet addr */
+#define ETH_TLEN 2 /* Octets in ethernet type field */
#define ETH_HLEN 14 /* Total octets in header. */
#define ETH_ZLEN 60 /* Min. octets in frame sans FCS */
#define ETH_DATA_LEN 1500 /* Max. octets in payload */
#define IFLA_TUN_MAX (__IFLA_TUN_MAX - 1)
+/* rmnet section */
+
+#define RMNET_FLAGS_INGRESS_DEAGGREGATION (1U << 0)
+#define RMNET_FLAGS_INGRESS_MAP_COMMANDS (1U << 1)
+#define RMNET_FLAGS_INGRESS_MAP_CKSUMV4 (1U << 2)
+#define RMNET_FLAGS_EGRESS_MAP_CKSUMV4 (1U << 3)
+
+enum {
+ IFLA_RMNET_UNSPEC,
+ IFLA_RMNET_MUX_ID,
+ IFLA_RMNET_FLAGS,
+ __IFLA_RMNET_MAX,
+};
+
+#define IFLA_RMNET_MAX (__IFLA_RMNET_MAX - 1)
+
+struct ifla_rmnet_flags {
+ __u32 flags;
+ __u32 mask;
+};
+
#endif /* _UAPI_LINUX_IF_LINK_H */
TIPC_NLA_SOCK_REF, /* u32 */
TIPC_NLA_SOCK_CON, /* nest */
TIPC_NLA_SOCK_HAS_PUBL, /* flag */
+ TIPC_NLA_SOCK_STAT, /* nest */
+ TIPC_NLA_SOCK_TYPE, /* u32 */
+ TIPC_NLA_SOCK_INO, /* u32 */
+ TIPC_NLA_SOCK_UID, /* u32 */
+ TIPC_NLA_SOCK_TIPC_STATE, /* u32 */
+ TIPC_NLA_SOCK_COOKIE, /* u64 */
+ TIPC_NLA_SOCK_PAD, /* flag */
__TIPC_NLA_SOCK_MAX,
TIPC_NLA_SOCK_MAX = __TIPC_NLA_SOCK_MAX - 1
TIPC_NLA_NET_UNSPEC,
TIPC_NLA_NET_ID, /* u32 */
TIPC_NLA_NET_ADDR, /* u32 */
+ TIPC_NLA_NET_NODEID, /* u64 */
+ TIPC_NLA_NET_NODEID_W1, /* u64 */
__TIPC_NLA_NET_MAX,
TIPC_NLA_NET_MAX = __TIPC_NLA_NET_MAX - 1
TIPC_NLA_CON_MAX = __TIPC_NLA_CON_MAX - 1
};
+/* Nest, socket statistics info */
+enum {
+ TIPC_NLA_SOCK_STAT_RCVQ, /* u32 */
+ TIPC_NLA_SOCK_STAT_SENDQ, /* u32 */
+ TIPC_NLA_SOCK_STAT_LINK_CONG, /* flag */
+ TIPC_NLA_SOCK_STAT_CONN_CONG, /* flag */
+ TIPC_NLA_SOCK_STAT_DROP, /* u32 */
+
+ __TIPC_NLA_SOCK_STAT_MAX,
+ TIPC_NLA_SOCK_STAT_MAX = __TIPC_NLA_SOCK_STAT_MAX - 1
+};
+
/* Nest, link propreties. Valid for link, media and bearer */
enum {
TIPC_NLA_PROP_UNSPEC,
--- /dev/null
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* AF_TIPC sock_diag interface for querying open sockets */
+
+#ifndef _UAPI__TIPC_SOCKETS_DIAG_H__
+#define _UAPI__TIPC_SOCKETS_DIAG_H__
+
+#include <linux/types.h>
+#include <linux/sock_diag.h>
+
+/* Request */
+struct tipc_sock_diag_req {
+ __u8 sdiag_family; /* must be AF_TIPC */
+ __u8 sdiag_protocol; /* must be 0 */
+ __u16 pad; /* must be 0 */
+ __u32 tidiag_states; /* query*/
+};
+#endif /* _UAPI__TIPC_SOCKETS_DIAG_H__ */
/* TLS socket options */
#define TLS_TX 1 /* Set transmit parameters */
+#define TLS_RX 2 /* Set receive parameters */
/* Supported versions */
#define TLS_VERSION_MINOR(ver) ((ver) & 0xFF)
#define TLS_CIPHER_AES_GCM_128_REC_SEQ_SIZE 8
#define TLS_SET_RECORD_TYPE 1
+#define TLS_GET_RECORD_TYPE 2
struct tls_crypto_info {
__u16 version;
__u64 reserved3;
};
+struct ocxl_ioctl_metadata {
+ __u16 version; // struct version, always backwards compatible
+
+ // Version 0 fields
+ __u8 afu_version_major;
+ __u8 afu_version_minor;
+ __u32 pasid; // PASID assigned to the current context
+
+ __u64 pp_mmio_size; // Per PASID MMIO size
+ __u64 global_mmio_size;
+
+ // End version 0 fields
+
+ __u64 reserved[13]; // Total of 16*u64
+};
+
struct ocxl_ioctl_irq_fd {
__u64 irq_offset;
__s32 eventfd;
#define OCXL_IOCTL_IRQ_ALLOC _IOR(OCXL_MAGIC, 0x11, __u64)
#define OCXL_IOCTL_IRQ_FREE _IOW(OCXL_MAGIC, 0x12, __u64)
#define OCXL_IOCTL_IRQ_SET_FD _IOW(OCXL_MAGIC, 0x13, struct ocxl_ioctl_irq_fd)
+#define OCXL_IOCTL_GET_METADATA _IOR(OCXL_MAGIC, 0x14, struct ocxl_ioctl_metadata)
#endif /* _UAPI_MISC_OCXL_H */
.exit = audit_net_exit,
.id = &audit_net_id,
.size = sizeof(struct audit_net),
- .async = true,
};
/* Initialize audit support at boot time. */
static struct dentry *
bpf_lookup(struct inode *dir, struct dentry *dentry, unsigned flags)
{
+ /* Dots in names (e.g. "/sys/fs/bpf/foo.bar") are reserved for future
+ * extensions.
+ */
if (strchr(dentry->d_name.name, '.'))
return ERR_PTR(-EPERM);
#include <linux/skbuff.h>
#include <linux/workqueue.h>
#include <linux/list.h>
+#include <linux/mm.h>
#include <net/strparser.h>
#include <net/tcp.h>
struct bpf_stab {
struct bpf_map map;
struct sock **sock_map;
+ struct bpf_prog *bpf_tx_msg;
struct bpf_prog *bpf_parse;
struct bpf_prog *bpf_verdict;
};
struct smap_psock {
struct rcu_head rcu;
- /* refcnt is used inside sk_callback_lock */
- u32 refcnt;
+ refcount_t refcnt;
/* datapath variables */
struct sk_buff_head rxqueue;
int save_off;
struct sk_buff *save_skb;
+ /* datapath variables for tx_msg ULP */
+ struct sock *sk_redir;
+ int apply_bytes;
+ int cork_bytes;
+ int sg_size;
+ int eval;
+ struct sk_msg_buff *cork;
+
struct strparser strp;
+ struct bpf_prog *bpf_tx_msg;
struct bpf_prog *bpf_parse;
struct bpf_prog *bpf_verdict;
struct list_head maps;
void (*save_write_space)(struct sock *sk);
};
+static void smap_release_sock(struct smap_psock *psock, struct sock *sock);
+static int bpf_tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size);
+static int bpf_tcp_sendpage(struct sock *sk, struct page *page,
+ int offset, size_t size, int flags);
+
static inline struct smap_psock *smap_psock_sk(const struct sock *sk)
{
return rcu_dereference_sk_user_data(sk);
psock->save_close = sk->sk_prot->close;
psock->sk_proto = sk->sk_prot;
+
+ if (psock->bpf_tx_msg) {
+ tcp_bpf_proto.sendmsg = bpf_tcp_sendmsg;
+ tcp_bpf_proto.sendpage = bpf_tcp_sendpage;
+ }
+
sk->sk_prot = &tcp_bpf_proto;
rcu_read_unlock();
return 0;
}
+static void smap_release_sock(struct smap_psock *psock, struct sock *sock);
+static int free_start_sg(struct sock *sk, struct sk_msg_buff *md);
+
static void bpf_tcp_release(struct sock *sk)
{
struct smap_psock *psock;
rcu_read_lock();
psock = smap_psock_sk(sk);
+ if (unlikely(!psock))
+ goto out;
- if (likely(psock)) {
- sk->sk_prot = psock->sk_proto;
- psock->sk_proto = NULL;
+ if (psock->cork) {
+ free_start_sg(psock->sock, psock->cork);
+ kfree(psock->cork);
+ psock->cork = NULL;
}
+
+ sk->sk_prot = psock->sk_proto;
+ psock->sk_proto = NULL;
+out:
rcu_read_unlock();
}
-static void smap_release_sock(struct smap_psock *psock, struct sock *sock);
-
static void bpf_tcp_close(struct sock *sk, long timeout)
{
void (*close_fun)(struct sock *sk, long timeout);
__SK_DROP = 0,
__SK_PASS,
__SK_REDIRECT,
+ __SK_NONE,
};
static struct tcp_ulp_ops bpf_tcp_ulp_ops __read_mostly = {
.release = bpf_tcp_release,
};
+static int memcopy_from_iter(struct sock *sk,
+ struct sk_msg_buff *md,
+ struct iov_iter *from, int bytes)
+{
+ struct scatterlist *sg = md->sg_data;
+ int i = md->sg_curr, rc = -ENOSPC;
+
+ do {
+ int copy;
+ char *to;
+
+ if (md->sg_copybreak >= sg[i].length) {
+ md->sg_copybreak = 0;
+
+ if (++i == MAX_SKB_FRAGS)
+ i = 0;
+
+ if (i == md->sg_end)
+ break;
+ }
+
+ copy = sg[i].length - md->sg_copybreak;
+ to = sg_virt(&sg[i]) + md->sg_copybreak;
+ md->sg_copybreak += copy;
+
+ if (sk->sk_route_caps & NETIF_F_NOCACHE_COPY)
+ rc = copy_from_iter_nocache(to, copy, from);
+ else
+ rc = copy_from_iter(to, copy, from);
+
+ if (rc != copy) {
+ rc = -EFAULT;
+ goto out;
+ }
+
+ bytes -= copy;
+ if (!bytes)
+ break;
+
+ md->sg_copybreak = 0;
+ if (++i == MAX_SKB_FRAGS)
+ i = 0;
+ } while (i != md->sg_end);
+out:
+ md->sg_curr = i;
+ return rc;
+}
+
+static int bpf_tcp_push(struct sock *sk, int apply_bytes,
+ struct sk_msg_buff *md,
+ int flags, bool uncharge)
+{
+ bool apply = apply_bytes;
+ struct scatterlist *sg;
+ int offset, ret = 0;
+ struct page *p;
+ size_t size;
+
+ while (1) {
+ sg = md->sg_data + md->sg_start;
+ size = (apply && apply_bytes < sg->length) ?
+ apply_bytes : sg->length;
+ offset = sg->offset;
+
+ tcp_rate_check_app_limited(sk);
+ p = sg_page(sg);
+retry:
+ ret = do_tcp_sendpages(sk, p, offset, size, flags);
+ if (ret != size) {
+ if (ret > 0) {
+ if (apply)
+ apply_bytes -= ret;
+ size -= ret;
+ offset += ret;
+ if (uncharge)
+ sk_mem_uncharge(sk, ret);
+ goto retry;
+ }
+
+ sg->length = size;
+ sg->offset = offset;
+ return ret;
+ }
+
+ if (apply)
+ apply_bytes -= ret;
+ sg->offset += ret;
+ sg->length -= ret;
+ if (uncharge)
+ sk_mem_uncharge(sk, ret);
+
+ if (!sg->length) {
+ put_page(p);
+ md->sg_start++;
+ if (md->sg_start == MAX_SKB_FRAGS)
+ md->sg_start = 0;
+ memset(sg, 0, sizeof(*sg));
+
+ if (md->sg_start == md->sg_end)
+ break;
+ }
+
+ if (apply && !apply_bytes)
+ break;
+ }
+ return 0;
+}
+
+static inline void bpf_compute_data_pointers_sg(struct sk_msg_buff *md)
+{
+ struct scatterlist *sg = md->sg_data + md->sg_start;
+
+ if (md->sg_copy[md->sg_start]) {
+ md->data = md->data_end = 0;
+ } else {
+ md->data = sg_virt(sg);
+ md->data_end = md->data + sg->length;
+ }
+}
+
+static void return_mem_sg(struct sock *sk, int bytes, struct sk_msg_buff *md)
+{
+ struct scatterlist *sg = md->sg_data;
+ int i = md->sg_start;
+
+ do {
+ int uncharge = (bytes < sg[i].length) ? bytes : sg[i].length;
+
+ sk_mem_uncharge(sk, uncharge);
+ bytes -= uncharge;
+ if (!bytes)
+ break;
+ i++;
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ } while (i != md->sg_end);
+}
+
+static void free_bytes_sg(struct sock *sk, int bytes, struct sk_msg_buff *md)
+{
+ struct scatterlist *sg = md->sg_data;
+ int i = md->sg_start, free;
+
+ while (bytes && sg[i].length) {
+ free = sg[i].length;
+ if (bytes < free) {
+ sg[i].length -= bytes;
+ sg[i].offset += bytes;
+ sk_mem_uncharge(sk, bytes);
+ break;
+ }
+
+ sk_mem_uncharge(sk, sg[i].length);
+ put_page(sg_page(&sg[i]));
+ bytes -= sg[i].length;
+ sg[i].length = 0;
+ sg[i].page_link = 0;
+ sg[i].offset = 0;
+ i++;
+
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ }
+}
+
+static int free_sg(struct sock *sk, int start, struct sk_msg_buff *md)
+{
+ struct scatterlist *sg = md->sg_data;
+ int i = start, free = 0;
+
+ while (sg[i].length) {
+ free += sg[i].length;
+ sk_mem_uncharge(sk, sg[i].length);
+ put_page(sg_page(&sg[i]));
+ sg[i].length = 0;
+ sg[i].page_link = 0;
+ sg[i].offset = 0;
+ i++;
+
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ }
+
+ return free;
+}
+
+static int free_start_sg(struct sock *sk, struct sk_msg_buff *md)
+{
+ int free = free_sg(sk, md->sg_start, md);
+
+ md->sg_start = md->sg_end;
+ return free;
+}
+
+static int free_curr_sg(struct sock *sk, struct sk_msg_buff *md)
+{
+ return free_sg(sk, md->sg_curr, md);
+}
+
+static int bpf_map_msg_verdict(int _rc, struct sk_msg_buff *md)
+{
+ return ((_rc == SK_PASS) ?
+ (md->map ? __SK_REDIRECT : __SK_PASS) :
+ __SK_DROP);
+}
+
+static unsigned int smap_do_tx_msg(struct sock *sk,
+ struct smap_psock *psock,
+ struct sk_msg_buff *md)
+{
+ struct bpf_prog *prog;
+ unsigned int rc, _rc;
+
+ preempt_disable();
+ rcu_read_lock();
+
+ /* If the policy was removed mid-send then default to 'accept' */
+ prog = READ_ONCE(psock->bpf_tx_msg);
+ if (unlikely(!prog)) {
+ _rc = SK_PASS;
+ goto verdict;
+ }
+
+ bpf_compute_data_pointers_sg(md);
+ rc = (*prog->bpf_func)(md, prog->insnsi);
+ psock->apply_bytes = md->apply_bytes;
+
+ /* Moving return codes from UAPI namespace into internal namespace */
+ _rc = bpf_map_msg_verdict(rc, md);
+
+ /* The psock has a refcount on the sock but not on the map and because
+ * we need to drop rcu read lock here its possible the map could be
+ * removed between here and when we need it to execute the sock
+ * redirect. So do the map lookup now for future use.
+ */
+ if (_rc == __SK_REDIRECT) {
+ if (psock->sk_redir)
+ sock_put(psock->sk_redir);
+ psock->sk_redir = do_msg_redirect_map(md);
+ if (!psock->sk_redir) {
+ _rc = __SK_DROP;
+ goto verdict;
+ }
+ sock_hold(psock->sk_redir);
+ }
+verdict:
+ rcu_read_unlock();
+ preempt_enable();
+
+ return _rc;
+}
+
+static int bpf_tcp_sendmsg_do_redirect(struct sock *sk, int send,
+ struct sk_msg_buff *md,
+ int flags)
+{
+ struct smap_psock *psock;
+ struct scatterlist *sg;
+ int i, err, free = 0;
+
+ sg = md->sg_data;
+
+ rcu_read_lock();
+ psock = smap_psock_sk(sk);
+ if (unlikely(!psock))
+ goto out_rcu;
+
+ if (!refcount_inc_not_zero(&psock->refcnt))
+ goto out_rcu;
+
+ rcu_read_unlock();
+ lock_sock(sk);
+ err = bpf_tcp_push(sk, send, md, flags, false);
+ release_sock(sk);
+ smap_release_sock(psock, sk);
+ if (unlikely(err))
+ goto out;
+ return 0;
+out_rcu:
+ rcu_read_unlock();
+out:
+ i = md->sg_start;
+ while (sg[i].length) {
+ free += sg[i].length;
+ put_page(sg_page(&sg[i]));
+ sg[i].length = 0;
+ i++;
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ }
+ return free;
+}
+
+static inline void bpf_md_init(struct smap_psock *psock)
+{
+ if (!psock->apply_bytes) {
+ psock->eval = __SK_NONE;
+ if (psock->sk_redir) {
+ sock_put(psock->sk_redir);
+ psock->sk_redir = NULL;
+ }
+ }
+}
+
+static void apply_bytes_dec(struct smap_psock *psock, int i)
+{
+ if (psock->apply_bytes) {
+ if (psock->apply_bytes < i)
+ psock->apply_bytes = 0;
+ else
+ psock->apply_bytes -= i;
+ }
+}
+
+static int bpf_exec_tx_verdict(struct smap_psock *psock,
+ struct sk_msg_buff *m,
+ struct sock *sk,
+ int *copied, int flags)
+{
+ bool cork = false, enospc = (m->sg_start == m->sg_end);
+ struct sock *redir;
+ int err = 0;
+ int send;
+
+more_data:
+ if (psock->eval == __SK_NONE)
+ psock->eval = smap_do_tx_msg(sk, psock, m);
+
+ if (m->cork_bytes &&
+ m->cork_bytes > psock->sg_size && !enospc) {
+ psock->cork_bytes = m->cork_bytes - psock->sg_size;
+ if (!psock->cork) {
+ psock->cork = kcalloc(1,
+ sizeof(struct sk_msg_buff),
+ GFP_ATOMIC | __GFP_NOWARN);
+
+ if (!psock->cork) {
+ err = -ENOMEM;
+ goto out_err;
+ }
+ }
+ memcpy(psock->cork, m, sizeof(*m));
+ goto out_err;
+ }
+
+ send = psock->sg_size;
+ if (psock->apply_bytes && psock->apply_bytes < send)
+ send = psock->apply_bytes;
+
+ switch (psock->eval) {
+ case __SK_PASS:
+ err = bpf_tcp_push(sk, send, m, flags, true);
+ if (unlikely(err)) {
+ *copied -= free_start_sg(sk, m);
+ break;
+ }
+
+ apply_bytes_dec(psock, send);
+ psock->sg_size -= send;
+ break;
+ case __SK_REDIRECT:
+ redir = psock->sk_redir;
+ apply_bytes_dec(psock, send);
+
+ if (psock->cork) {
+ cork = true;
+ psock->cork = NULL;
+ }
+
+ return_mem_sg(sk, send, m);
+ release_sock(sk);
+
+ err = bpf_tcp_sendmsg_do_redirect(redir, send, m, flags);
+ lock_sock(sk);
+
+ if (cork) {
+ free_start_sg(sk, m);
+ kfree(m);
+ m = NULL;
+ }
+ if (unlikely(err))
+ *copied -= err;
+ else
+ psock->sg_size -= send;
+ break;
+ case __SK_DROP:
+ default:
+ free_bytes_sg(sk, send, m);
+ apply_bytes_dec(psock, send);
+ *copied -= send;
+ psock->sg_size -= send;
+ err = -EACCES;
+ break;
+ }
+
+ if (likely(!err)) {
+ bpf_md_init(psock);
+ if (m &&
+ m->sg_data[m->sg_start].page_link &&
+ m->sg_data[m->sg_start].length)
+ goto more_data;
+ }
+
+out_err:
+ return err;
+}
+
+static int bpf_tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size)
+{
+ int flags = msg->msg_flags | MSG_NO_SHARED_FRAGS;
+ struct sk_msg_buff md = {0};
+ unsigned int sg_copy = 0;
+ struct smap_psock *psock;
+ int copied = 0, err = 0;
+ struct scatterlist *sg;
+ long timeo;
+
+ /* Its possible a sock event or user removed the psock _but_ the ops
+ * have not been reprogrammed yet so we get here. In this case fallback
+ * to tcp_sendmsg. Note this only works because we _only_ ever allow
+ * a single ULP there is no hierarchy here.
+ */
+ rcu_read_lock();
+ psock = smap_psock_sk(sk);
+ if (unlikely(!psock)) {
+ rcu_read_unlock();
+ return tcp_sendmsg(sk, msg, size);
+ }
+
+ /* Increment the psock refcnt to ensure its not released while sending a
+ * message. Required because sk lookup and bpf programs are used in
+ * separate rcu critical sections. Its OK if we lose the map entry
+ * but we can't lose the sock reference.
+ */
+ if (!refcount_inc_not_zero(&psock->refcnt)) {
+ rcu_read_unlock();
+ return tcp_sendmsg(sk, msg, size);
+ }
+
+ sg = md.sg_data;
+ sg_init_table(sg, MAX_SKB_FRAGS);
+ rcu_read_unlock();
+
+ lock_sock(sk);
+ timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT);
+
+ while (msg_data_left(msg)) {
+ struct sk_msg_buff *m;
+ bool enospc = false;
+ int copy;
+
+ if (sk->sk_err) {
+ err = sk->sk_err;
+ goto out_err;
+ }
+
+ copy = msg_data_left(msg);
+ if (!sk_stream_memory_free(sk))
+ goto wait_for_sndbuf;
+
+ m = psock->cork_bytes ? psock->cork : &md;
+ m->sg_curr = m->sg_copybreak ? m->sg_curr : m->sg_end;
+ err = sk_alloc_sg(sk, copy, m->sg_data,
+ m->sg_start, &m->sg_end, &sg_copy,
+ m->sg_end - 1);
+ if (err) {
+ if (err != -ENOSPC)
+ goto wait_for_memory;
+ enospc = true;
+ copy = sg_copy;
+ }
+
+ err = memcopy_from_iter(sk, m, &msg->msg_iter, copy);
+ if (err < 0) {
+ free_curr_sg(sk, m);
+ goto out_err;
+ }
+
+ psock->sg_size += copy;
+ copied += copy;
+ sg_copy = 0;
+
+ /* When bytes are being corked skip running BPF program and
+ * applying verdict unless there is no more buffer space. In
+ * the ENOSPC case simply run BPF prorgram with currently
+ * accumulated data. We don't have much choice at this point
+ * we could try extending the page frags or chaining complex
+ * frags but even in these cases _eventually_ we will hit an
+ * OOM scenario. More complex recovery schemes may be
+ * implemented in the future, but BPF programs must handle
+ * the case where apply_cork requests are not honored. The
+ * canonical method to verify this is to check data length.
+ */
+ if (psock->cork_bytes) {
+ if (copy > psock->cork_bytes)
+ psock->cork_bytes = 0;
+ else
+ psock->cork_bytes -= copy;
+
+ if (psock->cork_bytes && !enospc)
+ goto out_cork;
+
+ /* All cork bytes accounted for re-run filter */
+ psock->eval = __SK_NONE;
+ psock->cork_bytes = 0;
+ }
+
+ err = bpf_exec_tx_verdict(psock, m, sk, &copied, flags);
+ if (unlikely(err < 0))
+ goto out_err;
+ continue;
+wait_for_sndbuf:
+ set_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
+wait_for_memory:
+ err = sk_stream_wait_memory(sk, &timeo);
+ if (err)
+ goto out_err;
+ }
+out_err:
+ if (err < 0)
+ err = sk_stream_error(sk, msg->msg_flags, err);
+out_cork:
+ release_sock(sk);
+ smap_release_sock(psock, sk);
+ return copied ? copied : err;
+}
+
+static int bpf_tcp_sendpage(struct sock *sk, struct page *page,
+ int offset, size_t size, int flags)
+{
+ struct sk_msg_buff md = {0}, *m = NULL;
+ int err = 0, copied = 0;
+ struct smap_psock *psock;
+ struct scatterlist *sg;
+ bool enospc = false;
+
+ rcu_read_lock();
+ psock = smap_psock_sk(sk);
+ if (unlikely(!psock))
+ goto accept;
+
+ if (!refcount_inc_not_zero(&psock->refcnt))
+ goto accept;
+ rcu_read_unlock();
+
+ lock_sock(sk);
+
+ if (psock->cork_bytes)
+ m = psock->cork;
+ else
+ m = &md;
+
+ /* Catch case where ring is full and sendpage is stalled. */
+ if (unlikely(m->sg_end == m->sg_start &&
+ m->sg_data[m->sg_end].length))
+ goto out_err;
+
+ psock->sg_size += size;
+ sg = &m->sg_data[m->sg_end];
+ sg_set_page(sg, page, size, offset);
+ get_page(page);
+ m->sg_copy[m->sg_end] = true;
+ sk_mem_charge(sk, size);
+ m->sg_end++;
+ copied = size;
+
+ if (m->sg_end == MAX_SKB_FRAGS)
+ m->sg_end = 0;
+
+ if (m->sg_end == m->sg_start)
+ enospc = true;
+
+ if (psock->cork_bytes) {
+ if (size > psock->cork_bytes)
+ psock->cork_bytes = 0;
+ else
+ psock->cork_bytes -= size;
+
+ if (psock->cork_bytes && !enospc)
+ goto out_err;
+
+ /* All cork bytes accounted for re-run filter */
+ psock->eval = __SK_NONE;
+ psock->cork_bytes = 0;
+ }
+
+ err = bpf_exec_tx_verdict(psock, m, sk, &copied, flags);
+out_err:
+ release_sock(sk);
+ smap_release_sock(psock, sk);
+ return copied ? copied : err;
+accept:
+ rcu_read_unlock();
+ return tcp_sendpage(sk, page, offset, size, flags);
+}
+
+static void bpf_tcp_msg_add(struct smap_psock *psock,
+ struct sock *sk,
+ struct bpf_prog *tx_msg)
+{
+ struct bpf_prog *orig_tx_msg;
+
+ orig_tx_msg = xchg(&psock->bpf_tx_msg, tx_msg);
+ if (orig_tx_msg)
+ bpf_prog_put(orig_tx_msg);
+}
+
static int bpf_tcp_ulp_register(void)
{
tcp_bpf_proto = tcp_prot;
tcp_bpf_proto.close = bpf_tcp_close;
+ /* Once BPF TX ULP is registered it is never unregistered. It
+ * will be in the ULP list for the lifetime of the system. Doing
+ * duplicate registers is not a problem.
+ */
return tcp_register_ulp(&bpf_tcp_ulp_ops);
}
static void smap_release_sock(struct smap_psock *psock, struct sock *sock)
{
- psock->refcnt--;
- if (psock->refcnt)
- return;
-
- tcp_cleanup_ulp(sock);
- smap_stop_sock(psock, sock);
- clear_bit(SMAP_TX_RUNNING, &psock->state);
- rcu_assign_sk_user_data(sock, NULL);
- call_rcu_sched(&psock->rcu, smap_destroy_psock);
+ if (refcount_dec_and_test(&psock->refcnt)) {
+ tcp_cleanup_ulp(sock);
+ smap_stop_sock(psock, sock);
+ clear_bit(SMAP_TX_RUNNING, &psock->state);
+ rcu_assign_sk_user_data(sock, NULL);
+ call_rcu_sched(&psock->rcu, smap_destroy_psock);
+ }
}
static int smap_parse_func_strparser(struct strparser *strp,
return rc;
}
-
static int smap_read_sock_done(struct strparser *strp, int err)
{
return err;
bpf_prog_put(psock->bpf_parse);
if (psock->bpf_verdict)
bpf_prog_put(psock->bpf_verdict);
+ if (psock->bpf_tx_msg)
+ bpf_prog_put(psock->bpf_tx_msg);
+
+ if (psock->cork) {
+ free_start_sg(psock->sock, psock->cork);
+ kfree(psock->cork);
+ }
list_for_each_entry_safe(e, tmp, &psock->maps, list) {
list_del(&e->list);
kfree(e);
}
+ if (psock->sk_redir)
+ sock_put(psock->sk_redir);
+
sock_put(psock->sock);
kfree(psock);
}
if (!psock)
return ERR_PTR(-ENOMEM);
+ psock->eval = __SK_NONE;
psock->sock = sock;
skb_queue_head_init(&psock->rxqueue);
INIT_WORK(&psock->tx_work, smap_tx_work);
INIT_WORK(&psock->gc_work, smap_gc_work);
INIT_LIST_HEAD(&psock->maps);
- psock->refcnt = 1;
+ refcount_set(&psock->refcnt, 1);
rcu_assign_sk_user_data(sock, psock);
sock_hold(sock);
{
struct bpf_stab *stab = container_of(map, struct bpf_stab, map);
struct smap_psock_map_entry *e = NULL;
- struct bpf_prog *verdict, *parse;
+ struct bpf_prog *verdict, *parse, *tx_msg;
struct sock *osock, *sock;
struct smap_psock *psock;
u32 i = *(u32 *)key;
+ bool new = false;
int err;
if (unlikely(flags > BPF_EXIST))
*/
verdict = READ_ONCE(stab->bpf_verdict);
parse = READ_ONCE(stab->bpf_parse);
+ tx_msg = READ_ONCE(stab->bpf_tx_msg);
if (parse && verdict) {
/* bpf prog refcnt may be zero if a concurrent attach operation
}
}
+ if (tx_msg) {
+ tx_msg = bpf_prog_inc_not_zero(stab->bpf_tx_msg);
+ if (IS_ERR(tx_msg)) {
+ if (verdict)
+ bpf_prog_put(verdict);
+ if (parse)
+ bpf_prog_put(parse);
+ return PTR_ERR(tx_msg);
+ }
+ }
+
write_lock_bh(&sock->sk_callback_lock);
psock = smap_psock_sk(sock);
err = -EBUSY;
goto out_progs;
}
- psock->refcnt++;
+ if (READ_ONCE(psock->bpf_tx_msg) && tx_msg) {
+ err = -EBUSY;
+ goto out_progs;
+ }
+ if (!refcount_inc_not_zero(&psock->refcnt)) {
+ err = -EAGAIN;
+ goto out_progs;
+ }
} else {
psock = smap_init_psock(sock, stab);
if (IS_ERR(psock)) {
goto out_progs;
}
- err = tcp_set_ulp_id(sock, TCP_ULP_BPF);
- if (err)
- goto out_progs;
-
set_bit(SMAP_TX_RUNNING, &psock->state);
+ new = true;
}
e = kzalloc(sizeof(*e), GFP_ATOMIC | __GFP_NOWARN);
/* 3. At this point we have a reference to a valid psock that is
* running. Attach any BPF programs needed.
*/
+ if (tx_msg)
+ bpf_tcp_msg_add(psock, sock, tx_msg);
+ if (new) {
+ err = tcp_set_ulp_id(sock, TCP_ULP_BPF);
+ if (err)
+ goto out_free;
+ }
+
if (parse && verdict && !psock->strp_enabled) {
err = smap_init_sock(psock, sock);
if (err)
struct smap_psock *opsock = smap_psock_sk(osock);
write_lock_bh(&osock->sk_callback_lock);
- if (osock != sock && parse)
- smap_stop_sock(opsock, osock);
smap_list_remove(opsock, &stab->sock_map[i]);
smap_release_sock(opsock, osock);
write_unlock_bh(&osock->sk_callback_lock);
bpf_prog_put(verdict);
if (parse)
bpf_prog_put(parse);
+ if (tx_msg)
+ bpf_prog_put(tx_msg);
write_unlock_bh(&sock->sk_callback_lock);
kfree(e);
return err;
return -EINVAL;
switch (type) {
+ case BPF_SK_MSG_VERDICT:
+ orig = xchg(&stab->bpf_tx_msg, prog);
+ break;
case BPF_SK_SKB_STREAM_PARSER:
orig = xchg(&stab->bpf_parse, prog);
break;
orig = xchg(&stab->bpf_verdict, NULL);
if (orig)
bpf_prog_put(orig);
+
+ orig = xchg(&stab->bpf_tx_msg, NULL);
+ if (orig)
+ bpf_prog_put(orig);
}
const struct bpf_map_ops sock_map_ops = {
#include <linux/filter.h>
#include <linux/stacktrace.h>
#include <linux/perf_event.h>
+#include <linux/elf.h>
+#include <linux/pagemap.h>
#include "percpu_freelist.h"
-#define STACK_CREATE_FLAG_MASK \
- (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY)
+#define STACK_CREATE_FLAG_MASK \
+ (BPF_F_NUMA_NODE | BPF_F_RDONLY | BPF_F_WRONLY | \
+ BPF_F_STACK_BUILD_ID)
struct stack_map_bucket {
struct pcpu_freelist_node fnode;
u32 hash;
u32 nr;
- u64 ip[];
+ u64 data[];
};
struct bpf_stack_map {
struct stack_map_bucket *buckets[];
};
+static inline bool stack_map_use_build_id(struct bpf_map *map)
+{
+ return (map->map_flags & BPF_F_STACK_BUILD_ID);
+}
+
+static inline int stack_map_data_size(struct bpf_map *map)
+{
+ return stack_map_use_build_id(map) ?
+ sizeof(struct bpf_stack_build_id) : sizeof(u64);
+}
+
static int prealloc_elems_and_freelist(struct bpf_stack_map *smap)
{
u32 elem_size = sizeof(struct stack_map_bucket) + smap->map.value_size;
/* check sanity of attributes */
if (attr->max_entries == 0 || attr->key_size != 4 ||
- value_size < 8 || value_size % 8 ||
- value_size / 8 > sysctl_perf_event_max_stack)
+ value_size < 8 || value_size % 8)
+ return ERR_PTR(-EINVAL);
+
+ BUILD_BUG_ON(sizeof(struct bpf_stack_build_id) % sizeof(u64));
+ if (attr->map_flags & BPF_F_STACK_BUILD_ID) {
+ if (value_size % sizeof(struct bpf_stack_build_id) ||
+ value_size / sizeof(struct bpf_stack_build_id)
+ > sysctl_perf_event_max_stack)
+ return ERR_PTR(-EINVAL);
+ } else if (value_size / 8 > sysctl_perf_event_max_stack)
return ERR_PTR(-EINVAL);
/* hash table size must be power of 2 */
return ERR_PTR(err);
}
+#define BPF_BUILD_ID 3
+/*
+ * Parse build id from the note segment. This logic can be shared between
+ * 32-bit and 64-bit system, because Elf32_Nhdr and Elf64_Nhdr are
+ * identical.
+ */
+static inline int stack_map_parse_build_id(void *page_addr,
+ unsigned char *build_id,
+ void *note_start,
+ Elf32_Word note_size)
+{
+ Elf32_Word note_offs = 0, new_offs;
+
+ /* check for overflow */
+ if (note_start < page_addr || note_start + note_size < note_start)
+ return -EINVAL;
+
+ /* only supports note that fits in the first page */
+ if (note_start + note_size > page_addr + PAGE_SIZE)
+ return -EINVAL;
+
+ while (note_offs + sizeof(Elf32_Nhdr) < note_size) {
+ Elf32_Nhdr *nhdr = (Elf32_Nhdr *)(note_start + note_offs);
+
+ if (nhdr->n_type == BPF_BUILD_ID &&
+ nhdr->n_namesz == sizeof("GNU") &&
+ nhdr->n_descsz == BPF_BUILD_ID_SIZE) {
+ memcpy(build_id,
+ note_start + note_offs +
+ ALIGN(sizeof("GNU"), 4) + sizeof(Elf32_Nhdr),
+ BPF_BUILD_ID_SIZE);
+ return 0;
+ }
+ new_offs = note_offs + sizeof(Elf32_Nhdr) +
+ ALIGN(nhdr->n_namesz, 4) + ALIGN(nhdr->n_descsz, 4);
+ if (new_offs <= note_offs) /* overflow */
+ break;
+ note_offs = new_offs;
+ }
+ return -EINVAL;
+}
+
+/* Parse build ID from 32-bit ELF */
+static int stack_map_get_build_id_32(void *page_addr,
+ unsigned char *build_id)
+{
+ Elf32_Ehdr *ehdr = (Elf32_Ehdr *)page_addr;
+ Elf32_Phdr *phdr;
+ int i;
+
+ /* only supports phdr that fits in one page */
+ if (ehdr->e_phnum >
+ (PAGE_SIZE - sizeof(Elf32_Ehdr)) / sizeof(Elf32_Phdr))
+ return -EINVAL;
+
+ phdr = (Elf32_Phdr *)(page_addr + sizeof(Elf32_Ehdr));
+
+ for (i = 0; i < ehdr->e_phnum; ++i)
+ if (phdr[i].p_type == PT_NOTE)
+ return stack_map_parse_build_id(page_addr, build_id,
+ page_addr + phdr[i].p_offset,
+ phdr[i].p_filesz);
+ return -EINVAL;
+}
+
+/* Parse build ID from 64-bit ELF */
+static int stack_map_get_build_id_64(void *page_addr,
+ unsigned char *build_id)
+{
+ Elf64_Ehdr *ehdr = (Elf64_Ehdr *)page_addr;
+ Elf64_Phdr *phdr;
+ int i;
+
+ /* only supports phdr that fits in one page */
+ if (ehdr->e_phnum >
+ (PAGE_SIZE - sizeof(Elf64_Ehdr)) / sizeof(Elf64_Phdr))
+ return -EINVAL;
+
+ phdr = (Elf64_Phdr *)(page_addr + sizeof(Elf64_Ehdr));
+
+ for (i = 0; i < ehdr->e_phnum; ++i)
+ if (phdr[i].p_type == PT_NOTE)
+ return stack_map_parse_build_id(page_addr, build_id,
+ page_addr + phdr[i].p_offset,
+ phdr[i].p_filesz);
+ return -EINVAL;
+}
+
+/* Parse build ID of ELF file mapped to vma */
+static int stack_map_get_build_id(struct vm_area_struct *vma,
+ unsigned char *build_id)
+{
+ Elf32_Ehdr *ehdr;
+ struct page *page;
+ void *page_addr;
+ int ret;
+
+ /* only works for page backed storage */
+ if (!vma->vm_file)
+ return -EINVAL;
+
+ page = find_get_page(vma->vm_file->f_mapping, 0);
+ if (!page)
+ return -EFAULT; /* page not mapped */
+
+ ret = -EINVAL;
+ page_addr = page_address(page);
+ ehdr = (Elf32_Ehdr *)page_addr;
+
+ /* compare magic x7f "ELF" */
+ if (memcmp(ehdr->e_ident, ELFMAG, SELFMAG) != 0)
+ goto out;
+
+ /* only support executable file and shared object file */
+ if (ehdr->e_type != ET_EXEC && ehdr->e_type != ET_DYN)
+ goto out;
+
+ if (ehdr->e_ident[EI_CLASS] == ELFCLASS32)
+ ret = stack_map_get_build_id_32(page_addr, build_id);
+ else if (ehdr->e_ident[EI_CLASS] == ELFCLASS64)
+ ret = stack_map_get_build_id_64(page_addr, build_id);
+out:
+ put_page(page);
+ return ret;
+}
+
+static void stack_map_get_build_id_offset(struct bpf_map *map,
+ struct stack_map_bucket *bucket,
+ u64 *ips, u32 trace_nr, bool user)
+{
+ int i;
+ struct vm_area_struct *vma;
+ struct bpf_stack_build_id *id_offs;
+
+ bucket->nr = trace_nr;
+ id_offs = (struct bpf_stack_build_id *)bucket->data;
+
+ /*
+ * We cannot do up_read() in nmi context, so build_id lookup is
+ * only supported for non-nmi events. If at some point, it is
+ * possible to run find_vma() without taking the semaphore, we
+ * would like to allow build_id lookup in nmi context.
+ *
+ * Same fallback is used for kernel stack (!user) on a stackmap
+ * with build_id.
+ */
+ if (!user || !current || !current->mm || in_nmi() ||
+ down_read_trylock(¤t->mm->mmap_sem) == 0) {
+ /* cannot access current->mm, fall back to ips */
+ for (i = 0; i < trace_nr; i++) {
+ id_offs[i].status = BPF_STACK_BUILD_ID_IP;
+ id_offs[i].ip = ips[i];
+ }
+ return;
+ }
+
+ for (i = 0; i < trace_nr; i++) {
+ vma = find_vma(current->mm, ips[i]);
+ if (!vma || stack_map_get_build_id(vma, id_offs[i].build_id)) {
+ /* per entry fall back to ips */
+ id_offs[i].status = BPF_STACK_BUILD_ID_IP;
+ id_offs[i].ip = ips[i];
+ continue;
+ }
+ id_offs[i].offset = (vma->vm_pgoff << PAGE_SHIFT) + ips[i]
+ - vma->vm_start;
+ id_offs[i].status = BPF_STACK_BUILD_ID_VALID;
+ }
+ up_read(¤t->mm->mmap_sem);
+}
+
BPF_CALL_3(bpf_get_stackid, struct pt_regs *, regs, struct bpf_map *, map,
u64, flags)
{
struct bpf_stack_map *smap = container_of(map, struct bpf_stack_map, map);
struct perf_callchain_entry *trace;
struct stack_map_bucket *bucket, *new_bucket, *old_bucket;
- u32 max_depth = map->value_size / 8;
+ u32 max_depth = map->value_size / stack_map_data_size(map);
/* stack_map_alloc() checks that max_depth <= sysctl_perf_event_max_stack */
u32 init_nr = sysctl_perf_event_max_stack - max_depth;
u32 skip = flags & BPF_F_SKIP_FIELD_MASK;
bool user = flags & BPF_F_USER_STACK;
bool kernel = !user;
u64 *ips;
+ bool hash_matches;
if (unlikely(flags & ~(BPF_F_SKIP_FIELD_MASK | BPF_F_USER_STACK |
BPF_F_FAST_STACK_CMP | BPF_F_REUSE_STACKID)))
id = hash & (smap->n_buckets - 1);
bucket = READ_ONCE(smap->buckets[id]);
- if (bucket && bucket->hash == hash) {
- if (flags & BPF_F_FAST_STACK_CMP)
+ hash_matches = bucket && bucket->hash == hash;
+ /* fast cmp */
+ if (hash_matches && flags & BPF_F_FAST_STACK_CMP)
+ return id;
+
+ if (stack_map_use_build_id(map)) {
+ /* for build_id+offset, pop a bucket before slow cmp */
+ new_bucket = (struct stack_map_bucket *)
+ pcpu_freelist_pop(&smap->freelist);
+ if (unlikely(!new_bucket))
+ return -ENOMEM;
+ stack_map_get_build_id_offset(map, new_bucket, ips,
+ trace_nr, user);
+ trace_len = trace_nr * sizeof(struct bpf_stack_build_id);
+ if (hash_matches && bucket->nr == trace_nr &&
+ memcmp(bucket->data, new_bucket->data, trace_len) == 0) {
+ pcpu_freelist_push(&smap->freelist, &new_bucket->fnode);
return id;
- if (bucket->nr == trace_nr &&
- memcmp(bucket->ip, ips, trace_len) == 0)
+ }
+ if (bucket && !(flags & BPF_F_REUSE_STACKID)) {
+ pcpu_freelist_push(&smap->freelist, &new_bucket->fnode);
+ return -EEXIST;
+ }
+ } else {
+ if (hash_matches && bucket->nr == trace_nr &&
+ memcmp(bucket->data, ips, trace_len) == 0)
return id;
+ if (bucket && !(flags & BPF_F_REUSE_STACKID))
+ return -EEXIST;
+
+ new_bucket = (struct stack_map_bucket *)
+ pcpu_freelist_pop(&smap->freelist);
+ if (unlikely(!new_bucket))
+ return -ENOMEM;
+ memcpy(new_bucket->data, ips, trace_len);
}
- /* this call stack is not in the map, try to add it */
- if (bucket && !(flags & BPF_F_REUSE_STACKID))
- return -EEXIST;
-
- new_bucket = (struct stack_map_bucket *)
- pcpu_freelist_pop(&smap->freelist);
- if (unlikely(!new_bucket))
- return -ENOMEM;
-
- memcpy(new_bucket->ip, ips, trace_len);
new_bucket->hash = hash;
new_bucket->nr = trace_nr;
if (!bucket)
return -ENOENT;
- trace_len = bucket->nr * sizeof(u64);
- memcpy(value, bucket->ip, trace_len);
+ trace_len = bucket->nr * stack_map_data_size(map);
+ memcpy(value, bucket->data, trace_len);
memset(value + trace_len, 0, map->value_size - trace_len);
old_bucket = xchg(&smap->buckets[id], bucket);
#define BPF_PROG_ATTACH_LAST_FIELD attach_flags
-static int sockmap_get_from_fd(const union bpf_attr *attr, bool attach)
+static int sockmap_get_from_fd(const union bpf_attr *attr,
+ int type, bool attach)
{
struct bpf_prog *prog = NULL;
int ufd = attr->target_fd;
return PTR_ERR(map);
if (attach) {
- prog = bpf_prog_get_type(attr->attach_bpf_fd,
- BPF_PROG_TYPE_SK_SKB);
+ prog = bpf_prog_get_type(attr->attach_bpf_fd, type);
if (IS_ERR(prog)) {
fdput(f);
return PTR_ERR(prog);
case BPF_CGROUP_DEVICE:
ptype = BPF_PROG_TYPE_CGROUP_DEVICE;
break;
+ case BPF_SK_MSG_VERDICT:
+ return sockmap_get_from_fd(attr, BPF_PROG_TYPE_SK_MSG, true);
case BPF_SK_SKB_STREAM_PARSER:
case BPF_SK_SKB_STREAM_VERDICT:
- return sockmap_get_from_fd(attr, true);
+ return sockmap_get_from_fd(attr, BPF_PROG_TYPE_SK_SKB, true);
default:
return -EINVAL;
}
case BPF_CGROUP_DEVICE:
ptype = BPF_PROG_TYPE_CGROUP_DEVICE;
break;
+ case BPF_SK_MSG_VERDICT:
+ return sockmap_get_from_fd(attr, BPF_PROG_TYPE_SK_MSG, false);
case BPF_SK_SKB_STREAM_PARSER:
case BPF_SK_SKB_STREAM_VERDICT:
- return sockmap_get_from_fd(attr, false);
+ return sockmap_get_from_fd(attr, BPF_PROG_TYPE_SK_SKB, false);
default:
return -EINVAL;
}
union bpf_attr attr = {};
int err;
- if (!capable(CAP_SYS_ADMIN) && sysctl_unprivileged_bpf_disabled)
+ if (sysctl_unprivileged_bpf_disabled && !capable(CAP_SYS_ADMIN))
return -EPERM;
err = check_uarg_tail_zero(uattr, sizeof(attr), size);
case BPF_PROG_TYPE_XDP:
case BPF_PROG_TYPE_LWT_XMIT:
case BPF_PROG_TYPE_SK_SKB:
+ case BPF_PROG_TYPE_SK_MSG:
if (meta)
return meta->pkt_access;
case BPF_MAP_TYPE_SOCKMAP:
if (func_id != BPF_FUNC_sk_redirect_map &&
func_id != BPF_FUNC_sock_map_update &&
- func_id != BPF_FUNC_map_delete_elem)
+ func_id != BPF_FUNC_map_delete_elem &&
+ func_id != BPF_FUNC_msg_redirect_map)
goto error;
break;
default:
goto error;
break;
case BPF_FUNC_sk_redirect_map:
+ case BPF_FUNC_msg_redirect_map:
if (map->map_type != BPF_MAP_TYPE_SOCKMAP)
goto error;
break;
if (cgroup_is_threaded(cgrp))
return 0;
+ /*
+ * If @cgroup is populated or has domain controllers enabled, it
+ * can't be switched. While the below cgroup_can_be_thread_root()
+ * test can catch the same conditions, that's only when @parent is
+ * not mixable, so let's check it explicitly.
+ */
+ if (cgroup_is_populated(cgrp) ||
+ cgrp->subtree_control & ~cgrp_dfl_threaded_ss_mask)
+ return -EOPNOTSUPP;
+
/* we're joining the parent's domain, ensure its validity */
if (!cgroup_is_valid_domain(dom_cgrp) ||
!cgroup_can_be_thread_root(dom_cgrp))
}
EXPORT_SYMBOL_GPL(get_compat_sigset);
-int
-put_compat_sigset(compat_sigset_t __user *compat, const sigset_t *set,
- unsigned int size)
-{
- /* size <= sizeof(compat_sigset_t) <= sizeof(sigset_t) */
-#ifdef __BIG_ENDIAN
- compat_sigset_t v;
- switch (_NSIG_WORDS) {
- case 4: v.sig[7] = (set->sig[3] >> 32); v.sig[6] = set->sig[3];
- case 3: v.sig[5] = (set->sig[2] >> 32); v.sig[4] = set->sig[2];
- case 2: v.sig[3] = (set->sig[1] >> 32); v.sig[2] = set->sig[1];
- case 1: v.sig[1] = (set->sig[0] >> 32); v.sig[0] = set->sig[0];
- }
- return copy_to_user(compat, &v, size) ? -EFAULT : 0;
-#else
- return copy_to_user(compat, set, size) ? -EFAULT : 0;
-#endif
-}
-
#ifdef CONFIG_NUMA
COMPAT_SYSCALL_DEFINE6(move_pages, pid_t, pid, compat_ulong_t, nr_pages,
compat_uptr_t __user *, pages32,
struct perf_event_context *task_ctx,
enum event_type_t event_type)
{
- enum event_type_t ctx_event_type = event_type & EVENT_ALL;
+ enum event_type_t ctx_event_type;
bool cpu_event = !!(event_type & EVENT_CPU);
/*
if (event_type & EVENT_PINNED)
event_type |= EVENT_FLEXIBLE;
+ ctx_event_type = event_type & EVENT_ALL;
+
perf_pmu_disable(cpuctx->ctx.pmu);
if (task_ctx)
task_ctx_sched_out(cpuctx, task_ctx, event_type);
static int fei_kprobe_handler(struct kprobe *kp, struct pt_regs *regs);
+static void fei_post_handler(struct kprobe *kp, struct pt_regs *regs,
+ unsigned long flags)
+{
+ /*
+ * A dummy post handler is required to prohibit optimizing, because
+ * jump optimization does not support execution path overriding.
+ */
+}
+
struct fei_attr {
struct list_head list;
struct kprobe kp;
return NULL;
}
attr->kp.pre_handler = fei_kprobe_handler;
+ attr->kp.post_handler = fei_post_handler;
attr->retval = adjust_error_retval(addr, 0);
INIT_LIST_HEAD(&attr->list);
}
if (kernel_text_address(entry->code))
arch_jump_label_transform(entry, jump_label_type(entry));
else
- WARN_ONCE(1, "can't patch jump_label at %pS", (void *)entry->code);
+ WARN_ONCE(1, "can't patch jump_label at %pS",
+ (void *)(unsigned long)entry->code);
}
}
}
void __sched rt_mutex_futex_unlock(struct rt_mutex *lock)
{
DEFINE_WAKE_Q(wake_q);
+ unsigned long flags;
bool postunlock;
- raw_spin_lock_irq(&lock->wait_lock);
+ raw_spin_lock_irqsave(&lock->wait_lock, flags);
postunlock = __rt_mutex_futex_unlock(lock, &wake_q);
- raw_spin_unlock_irq(&lock->wait_lock);
+ raw_spin_unlock_irqrestore(&lock->wait_lock, flags);
if (postunlock)
rt_mutex_postunlock(&wake_q);
err_pfn_remap:
err_radix:
pgmap_radix_release(res, pgoff);
- devres_free(pgmap);
return ERR_PTR(error);
}
EXPORT_SYMBOL(devm_memremap_pages);
m->private = kallsyms_show_value() ? NULL : (void *)8ul;
}
- return 0;
+ return err;
}
static const struct file_operations proc_modules_operations = {
*/
__visible void __stack_chk_fail(void)
{
- panic("stack-protector: Kernel stack is corrupted in: %p\n",
+ panic("stack-protector: Kernel stack is corrupted in: %pB\n",
__builtin_return_address(0));
}
EXPORT_SYMBOL(__stack_chk_fail);
parent_quota = parent_b->hierarchical_quota;
/*
- * Ensure max(child_quota) <= parent_quota, inherit when no
+ * Ensure max(child_quota) <= parent_quota. On cgroup2,
+ * always take the min. On cgroup1, only inherit when no
* limit is set:
*/
- if (quota == RUNTIME_INF)
- quota = parent_quota;
- else if (parent_quota != RUNTIME_INF && quota > parent_quota)
- return -EINVAL;
+ if (cgroup_subsys_on_dfl(cpu_cgrp_subsys)) {
+ quota = min(quota, parent_quota);
+ } else {
+ if (quota == RUNTIME_INF)
+ quota = parent_quota;
+ else if (parent_quota != RUNTIME_INF && quota > parent_quota)
+ return -EINVAL;
+ }
}
cfs_b->hierarchical_quota = quota;
.arg3_type = ARG_ANYTHING,
};
-BPF_CALL_3(bpf_perf_prog_read_value_tp, struct bpf_perf_event_data_kern *, ctx,
+static const struct bpf_func_proto *tp_prog_func_proto(enum bpf_func_id func_id)
+{
+ switch (func_id) {
+ case BPF_FUNC_perf_event_output:
+ return &bpf_perf_event_output_proto_tp;
+ case BPF_FUNC_get_stackid:
+ return &bpf_get_stackid_proto_tp;
+ default:
+ return tracing_func_proto(func_id);
+ }
+}
+
+static bool tp_prog_is_valid_access(int off, int size, enum bpf_access_type type,
+ struct bpf_insn_access_aux *info)
+{
+ if (off < sizeof(void *) || off >= PERF_MAX_TRACE_SIZE)
+ return false;
+ if (type != BPF_READ)
+ return false;
+ if (off % size != 0)
+ return false;
+
+ BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(__u64));
+ return true;
+}
+
+const struct bpf_verifier_ops tracepoint_verifier_ops = {
+ .get_func_proto = tp_prog_func_proto,
+ .is_valid_access = tp_prog_is_valid_access,
+};
+
+const struct bpf_prog_ops tracepoint_prog_ops = {
+};
+
+BPF_CALL_3(bpf_perf_prog_read_value, struct bpf_perf_event_data_kern *, ctx,
struct bpf_perf_event_value *, buf, u32, size)
{
int err = -EINVAL;
return err;
}
-static const struct bpf_func_proto bpf_perf_prog_read_value_proto_tp = {
- .func = bpf_perf_prog_read_value_tp,
+static const struct bpf_func_proto bpf_perf_prog_read_value_proto = {
+ .func = bpf_perf_prog_read_value,
.gpl_only = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg3_type = ARG_CONST_SIZE,
};
-static const struct bpf_func_proto *tp_prog_func_proto(enum bpf_func_id func_id)
+static const struct bpf_func_proto *pe_prog_func_proto(enum bpf_func_id func_id)
{
switch (func_id) {
case BPF_FUNC_perf_event_output:
case BPF_FUNC_get_stackid:
return &bpf_get_stackid_proto_tp;
case BPF_FUNC_perf_prog_read_value:
- return &bpf_perf_prog_read_value_proto_tp;
+ return &bpf_perf_prog_read_value_proto;
default:
return tracing_func_proto(func_id);
}
}
-static bool tp_prog_is_valid_access(int off, int size, enum bpf_access_type type,
- struct bpf_insn_access_aux *info)
-{
- if (off < sizeof(void *) || off >= PERF_MAX_TRACE_SIZE)
- return false;
- if (type != BPF_READ)
- return false;
- if (off % size != 0)
- return false;
-
- BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(__u64));
- return true;
-}
-
-const struct bpf_verifier_ops tracepoint_verifier_ops = {
- .get_func_proto = tp_prog_func_proto,
- .is_valid_access = tp_prog_is_valid_access,
-};
-
-const struct bpf_prog_ops tracepoint_prog_ops = {
-};
-
static bool pe_prog_is_valid_access(int off, int size, enum bpf_access_type type,
struct bpf_insn_access_aux *info)
{
- const int size_sp = FIELD_SIZEOF(struct bpf_perf_event_data,
- sample_period);
+ const int size_u64 = sizeof(u64);
if (off < 0 || off >= sizeof(struct bpf_perf_event_data))
return false;
switch (off) {
case bpf_ctx_range(struct bpf_perf_event_data, sample_period):
- bpf_ctx_record_field_size(info, size_sp);
- if (!bpf_ctx_narrow_access_ok(off, size, size_sp))
+ bpf_ctx_record_field_size(info, size_u64);
+ if (!bpf_ctx_narrow_access_ok(off, size, size_u64))
+ return false;
+ break;
+ case bpf_ctx_range(struct bpf_perf_event_data, addr):
+ bpf_ctx_record_field_size(info, size_u64);
+ if (!bpf_ctx_narrow_access_ok(off, size, size_u64))
return false;
break;
default:
bpf_target_off(struct perf_sample_data, period, 8,
target_size));
break;
+ case offsetof(struct bpf_perf_event_data, addr):
+ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_perf_event_data_kern,
+ data), si->dst_reg, si->src_reg,
+ offsetof(struct bpf_perf_event_data_kern, data));
+ *insn++ = BPF_LDX_MEM(BPF_DW, si->dst_reg, si->dst_reg,
+ bpf_target_off(struct perf_sample_data, addr, 8,
+ target_size));
+ break;
default:
*insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct bpf_perf_event_data_kern,
regs), si->dst_reg, si->src_reg,
}
const struct bpf_verifier_ops perf_event_verifier_ops = {
- .get_func_proto = tp_prog_func_proto,
+ .get_func_proto = pe_prog_func_proto,
.is_valid_access = pe_prog_is_valid_access,
.convert_ctx_access = pe_prog_convert_ctx_access,
};
return ret;
}
-/*
- * See cancel_delayed_work()
- */
-bool cancel_work(struct work_struct *work)
-{
- return __cancel_work(work, false);
-}
-
/**
* cancel_delayed_work - cancel a delayed work
* @dwork: delayed_work to cancel
ret = device_register(&wq_dev->dev);
if (ret) {
- kfree(wq_dev);
+ put_device(&wq_dev->dev);
wq->wq_dev = NULL;
return ret;
}
*
* As should be obvious for Linux kernel code, license is GPLv2
*
- * Copyright (c) 2007-2008 Joern Engel <joern@logfs.org>
+ * Copyright (c) 2007-2008 Joern Engel <joern@purestorage.com>
* Bits and pieces stolen from Peter Zijlstra's code, which is
* Copyright 2007, Red Hat Inc. Peter Zijlstra
* GPLv2
};
EXPORT_SYMBOL_GPL(btree_geo128);
+#define MAX_KEYLEN (2 * LONG_PER_U64)
+
static struct kmem_cache *btree_cachep;
void *btree_alloc(gfp_t gfp_mask, void *pool_data)
{
int i, height;
unsigned long *node, *oldnode;
- unsigned long *retry_key = NULL, key[geo->keylen];
+ unsigned long *retry_key = NULL, key[MAX_KEYLEN];
if (keyzero(geo, __key))
return NULL;
int btree_merge(struct btree_head *target, struct btree_head *victim,
struct btree_geo *geo, gfp_t gfp)
{
- unsigned long key[geo->keylen];
- unsigned long dup[geo->keylen];
+ unsigned long key[MAX_KEYLEN];
+ unsigned long dup[MAX_KEYLEN];
void *val;
int err;
return BUG_TRAP_TYPE_NONE;
bug = find_bug(bugaddr);
+ if (!bug)
+ return BUG_TRAP_TYPE_NONE;
file = NULL;
line = 0;
if (file)
pr_crit("kernel BUG at %s:%u!\n", file, line);
else
- pr_crit("Kernel BUG at %p [verbose debug info unavailable]\n",
+ pr_crit("Kernel BUG at %pB [verbose debug info unavailable]\n",
(void *)bugaddr);
return BUG_TRAP_TYPE_BUG;
if (ioremap_pmd_enabled() &&
((next - addr) == PMD_SIZE) &&
- IS_ALIGNED(phys_addr + addr, PMD_SIZE)) {
+ IS_ALIGNED(phys_addr + addr, PMD_SIZE) &&
+ pmd_free_pte_page(pmd)) {
if (pmd_set_huge(pmd, phys_addr + addr, prot))
continue;
}
if (ioremap_pud_enabled() &&
((next - addr) == PUD_SIZE) &&
- IS_ALIGNED(phys_addr + addr, PUD_SIZE)) {
+ IS_ALIGNED(phys_addr + addr, PUD_SIZE) &&
+ pud_free_pmd_page(pud)) {
if (pud_set_huge(pud, phys_addr + addr, prot))
continue;
}
#include <linux/uuid.h>
#include <linux/ctype.h>
#include <net/sock.h>
+#include <net/netlink.h>
#include <net/net_namespace.h>
#ifdef CONFIG_UEVENT_HELPER
char uevent_helper[UEVENT_HELPER_PATH_LEN] = CONFIG_UEVENT_HELPER_PATH;
#endif
-#ifdef CONFIG_NET
+
struct uevent_sock {
struct list_head list;
struct sock *sk;
};
+
+#ifdef CONFIG_NET
static LIST_HEAD(uevent_sock_list);
#endif
EXPORT_SYMBOL_GPL(add_uevent_var);
#if defined(CONFIG_NET)
+static int uevent_net_broadcast(struct sock *usk, struct sk_buff *skb,
+ struct netlink_ext_ack *extack)
+{
+ /* u64 to chars: 2^64 - 1 = 21 chars */
+ char buf[sizeof("SEQNUM=") + 21];
+ struct sk_buff *skbc;
+ int ret;
+
+ /* bump and prepare sequence number */
+ ret = snprintf(buf, sizeof(buf), "SEQNUM=%llu", ++uevent_seqnum);
+ if (ret < 0 || (size_t)ret >= sizeof(buf))
+ return -ENOMEM;
+ ret++;
+
+ /* verify message does not overflow */
+ if ((skb->len + ret) > UEVENT_BUFFER_SIZE) {
+ NL_SET_ERR_MSG(extack, "uevent message too big");
+ return -EINVAL;
+ }
+
+ /* copy skb and extend to accommodate sequence number */
+ skbc = skb_copy_expand(skb, 0, ret, GFP_KERNEL);
+ if (!skbc)
+ return -ENOMEM;
+
+ /* append sequence number */
+ skb_put_data(skbc, buf, ret);
+
+ /* remove msg header */
+ skb_pull(skbc, NLMSG_HDRLEN);
+
+ /* set portid 0 to inform userspace message comes from kernel */
+ NETLINK_CB(skbc).portid = 0;
+ NETLINK_CB(skbc).dst_group = 1;
+
+ ret = netlink_broadcast(usk, skbc, 0, 1, GFP_KERNEL);
+ /* ENOBUFS should be handled in userspace */
+ if (ret == -ENOBUFS || ret == -ESRCH)
+ ret = 0;
+
+ return ret;
+}
+
+static int uevent_net_rcv_skb(struct sk_buff *skb, struct nlmsghdr *nlh,
+ struct netlink_ext_ack *extack)
+{
+ struct net *net;
+ int ret;
+
+ if (!nlmsg_data(nlh))
+ return -EINVAL;
+
+ /*
+ * Verify that we are allowed to send messages to the target
+ * network namespace. The caller must have CAP_SYS_ADMIN in the
+ * owning user namespace of the target network namespace.
+ */
+ net = sock_net(NETLINK_CB(skb).sk);
+ if (!netlink_ns_capable(skb, net->user_ns, CAP_SYS_ADMIN)) {
+ NL_SET_ERR_MSG(extack, "missing CAP_SYS_ADMIN capability");
+ return -EPERM;
+ }
+
+ mutex_lock(&uevent_sock_mutex);
+ ret = uevent_net_broadcast(net->uevent_sock->sk, skb, extack);
+ mutex_unlock(&uevent_sock_mutex);
+
+ return ret;
+}
+
+static void uevent_net_rcv(struct sk_buff *skb)
+{
+ netlink_rcv_skb(skb, &uevent_net_rcv_skb);
+}
+
static int uevent_net_init(struct net *net)
{
struct uevent_sock *ue_sk;
struct netlink_kernel_cfg cfg = {
.groups = 1,
- .flags = NL_CFG_F_NONROOT_RECV,
+ .input = uevent_net_rcv,
+ .flags = NL_CFG_F_NONROOT_RECV
};
ue_sk = kzalloc(sizeof(*ue_sk), GFP_KERNEL);
kfree(ue_sk);
return -ENODEV;
}
+
+ net->uevent_sock = ue_sk;
+
mutex_lock(&uevent_sock_mutex);
list_add_tail(&ue_sk->list, &uevent_sock_list);
mutex_unlock(&uevent_sock_mutex);
static void uevent_net_exit(struct net *net)
{
- struct uevent_sock *ue_sk;
+ struct uevent_sock *ue_sk = net->uevent_sock;
mutex_lock(&uevent_sock_mutex);
- list_for_each_entry(ue_sk, &uevent_sock_list, list) {
- if (sock_net(ue_sk->sk) == net)
- goto found;
- }
- mutex_unlock(&uevent_sock_mutex);
- return;
-
-found:
list_del(&ue_sk->list);
mutex_unlock(&uevent_sock_mutex);
static struct pernet_operations uevent_net_ops = {
.init = uevent_net_init,
.exit = uevent_net_exit,
- .async = true,
};
static int __init kobject_uevent_init(void)
* This function normally doesn't block and can be called from any context
* but it may block if @confirm_kill is specified and @ref is in the
* process of switching to atomic mode by percpu_ref_switch_to_atomic().
+ *
+ * There are no implied RCU grace periods between kill and release.
*/
void percpu_ref_kill_and_confirm(struct percpu_ref *ref,
percpu_ref_func_t *confirm_kill)
if (!key ||
(ht->p.obj_cmpfn ?
ht->p.obj_cmpfn(&arg, rht_obj(ht, head)) :
- rhashtable_compare(&arg, rht_obj(ht, head))))
+ rhashtable_compare(&arg, rht_obj(ht, head)))) {
+ pprev = &head->next;
continue;
+ }
if (!ht->rhlist)
return rht_obj(ht, head);
{
"BPF_MAXINSNS: Jump, gap, jump, ...",
{ },
-#ifdef CONFIG_BPF_JIT_ALWAYS_ON
+#if defined(CONFIG_BPF_JIT_ALWAYS_ON) && defined(CONFIG_X86)
CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL,
#else
CLASSIC | FLAG_NO_DATA,
return test_id < test_range[0] || test_id > test_range[1];
}
+static __init struct sk_buff *build_test_skb(void)
+{
+ u32 headroom = NET_SKB_PAD + NET_IP_ALIGN + ETH_HLEN;
+ struct sk_buff *skb[2];
+ struct page *page[2];
+ int i, data_size = 8;
+
+ for (i = 0; i < 2; i++) {
+ page[i] = alloc_page(GFP_KERNEL);
+ if (!page[i]) {
+ if (i == 0)
+ goto err_page0;
+ else
+ goto err_page1;
+ }
+
+ /* this will set skb[i]->head_frag */
+ skb[i] = dev_alloc_skb(headroom + data_size);
+ if (!skb[i]) {
+ if (i == 0)
+ goto err_skb0;
+ else
+ goto err_skb1;
+ }
+
+ skb_reserve(skb[i], headroom);
+ skb_put(skb[i], data_size);
+ skb[i]->protocol = htons(ETH_P_IP);
+ skb_reset_network_header(skb[i]);
+ skb_set_mac_header(skb[i], -ETH_HLEN);
+
+ skb_add_rx_frag(skb[i], 0, page[i], 0, 64, 64);
+ // skb_headlen(skb[i]): 8, skb[i]->head_frag = 1
+ }
+
+ /* setup shinfo */
+ skb_shinfo(skb[0])->gso_size = 1448;
+ skb_shinfo(skb[0])->gso_type = SKB_GSO_TCPV4;
+ skb_shinfo(skb[0])->gso_type |= SKB_GSO_DODGY;
+ skb_shinfo(skb[0])->gso_segs = 0;
+ skb_shinfo(skb[0])->frag_list = skb[1];
+
+ /* adjust skb[0]'s len */
+ skb[0]->len += skb[1]->len;
+ skb[0]->data_len += skb[1]->data_len;
+ skb[0]->truesize += skb[1]->truesize;
+
+ return skb[0];
+
+err_skb1:
+ __free_page(page[1]);
+err_page1:
+ kfree_skb(skb[0]);
+err_skb0:
+ __free_page(page[0]);
+err_page0:
+ return NULL;
+}
+
+static __init int test_skb_segment(void)
+{
+ netdev_features_t features;
+ struct sk_buff *skb, *segs;
+ int ret = -1;
+
+ features = NETIF_F_SG | NETIF_F_GSO_PARTIAL | NETIF_F_IP_CSUM |
+ NETIF_F_IPV6_CSUM;
+ features |= NETIF_F_RXCSUM;
+ skb = build_test_skb();
+ if (!skb) {
+ pr_info("%s: failed to build_test_skb", __func__);
+ goto done;
+ }
+
+ segs = skb_segment(skb, features);
+ if (!IS_ERR(segs)) {
+ kfree_skb_list(segs);
+ ret = 0;
+ pr_info("%s: success in skb_segment!", __func__);
+ } else {
+ pr_info("%s: failed in skb_segment!", __func__);
+ }
+ kfree_skb(skb);
+done:
+ return ret;
+}
+
static __init int test_bpf(void)
{
int i, err_cnt = 0, pass_cnt = 0;
return ret;
ret = test_bpf();
-
destroy_bpf_tests();
- return ret;
+ if (ret)
+ return ret;
+
+ return test_skb_segment();
}
static void __exit test_bpf_exit(void)
mutex_lock(®_dev_mutex);
/* int should suffice for number of devices, test for wrap */
- if (unlikely(num_test_devs + 1) < 0) {
+ if (num_test_devs + 1 == INT_MAX) {
pr_err("reached limit of number of test devices\n");
goto out;
}
struct test_obj *objs;
};
+static u32 my_hashfn(const void *data, u32 len, u32 seed)
+{
+ const struct test_obj_rhl *obj = data;
+
+ return (obj->value.id % 10) << RHT_HASH_RESERVED_SPACE;
+}
+
+static int my_cmpfn(struct rhashtable_compare_arg *arg, const void *obj)
+{
+ const struct test_obj_rhl *test_obj = obj;
+ const struct test_obj_val *val = arg->key;
+
+ return test_obj->value.id - val->id;
+}
+
static struct rhashtable_params test_rht_params = {
.head_offset = offsetof(struct test_obj, node),
.key_offset = offsetof(struct test_obj, value),
.nulls_base = (3U << RHT_BASE_SHIFT),
};
+static struct rhashtable_params test_rht_params_dup = {
+ .head_offset = offsetof(struct test_obj_rhl, list_node),
+ .key_offset = offsetof(struct test_obj_rhl, value),
+ .key_len = sizeof(struct test_obj_val),
+ .hashfn = jhash,
+ .obj_hashfn = my_hashfn,
+ .obj_cmpfn = my_cmpfn,
+ .nelem_hint = 128,
+ .automatic_shrinking = false,
+};
+
static struct semaphore prestart_sem;
static struct semaphore startup_sem = __SEMAPHORE_INITIALIZER(startup_sem, 0);
return err;
}
+static unsigned int __init print_ht(struct rhltable *rhlt)
+{
+ struct rhashtable *ht;
+ const struct bucket_table *tbl;
+ char buff[512] = "";
+ unsigned int i, cnt = 0;
+
+ ht = &rhlt->ht;
+ tbl = rht_dereference(ht->tbl, ht);
+ for (i = 0; i < tbl->size; i++) {
+ struct rhash_head *pos, *next;
+ struct test_obj_rhl *p;
+
+ pos = rht_dereference(tbl->buckets[i], ht);
+ next = !rht_is_a_nulls(pos) ? rht_dereference(pos->next, ht) : NULL;
+
+ if (!rht_is_a_nulls(pos)) {
+ sprintf(buff, "%s\nbucket[%d] -> ", buff, i);
+ }
+
+ while (!rht_is_a_nulls(pos)) {
+ struct rhlist_head *list = container_of(pos, struct rhlist_head, rhead);
+ sprintf(buff, "%s[[", buff);
+ do {
+ pos = &list->rhead;
+ list = rht_dereference(list->next, ht);
+ p = rht_obj(ht, pos);
+
+ sprintf(buff, "%s val %d (tid=%d)%s", buff, p->value.id, p->value.tid,
+ list? ", " : " ");
+ cnt++;
+ } while (list);
+
+ pos = next,
+ next = !rht_is_a_nulls(pos) ?
+ rht_dereference(pos->next, ht) : NULL;
+
+ sprintf(buff, "%s]]%s", buff, !rht_is_a_nulls(pos) ? " -> " : "");
+ }
+ }
+ printk(KERN_ERR "\n---- ht: ----%s\n-------------\n", buff);
+
+ return cnt;
+}
+
+static int __init test_insert_dup(struct test_obj_rhl *rhl_test_objects,
+ int cnt, bool slow)
+{
+ struct rhltable rhlt;
+ unsigned int i, ret;
+ const char *key;
+ int err = 0;
+
+ err = rhltable_init(&rhlt, &test_rht_params_dup);
+ if (WARN_ON(err))
+ return err;
+
+ for (i = 0; i < cnt; i++) {
+ rhl_test_objects[i].value.tid = i;
+ key = rht_obj(&rhlt.ht, &rhl_test_objects[i].list_node.rhead);
+ key += test_rht_params_dup.key_offset;
+
+ if (slow) {
+ err = PTR_ERR(rhashtable_insert_slow(&rhlt.ht, key,
+ &rhl_test_objects[i].list_node.rhead));
+ if (err == -EAGAIN)
+ err = 0;
+ } else
+ err = rhltable_insert(&rhlt,
+ &rhl_test_objects[i].list_node,
+ test_rht_params_dup);
+ if (WARN(err, "error %d on element %d/%d (%s)\n", err, i, cnt, slow? "slow" : "fast"))
+ goto skip_print;
+ }
+
+ ret = print_ht(&rhlt);
+ WARN(ret != cnt, "missing rhltable elements (%d != %d, %s)\n", ret, cnt, slow? "slow" : "fast");
+
+skip_print:
+ rhltable_destroy(&rhlt);
+
+ return 0;
+}
+
+static int __init test_insert_duplicates_run(void)
+{
+ struct test_obj_rhl rhl_test_objects[3] = {};
+
+ pr_info("test inserting duplicates\n");
+
+ /* two different values that map to same bucket */
+ rhl_test_objects[0].value.id = 1;
+ rhl_test_objects[1].value.id = 21;
+
+ /* and another duplicate with same as [0] value
+ * which will be second on the bucket list */
+ rhl_test_objects[2].value.id = rhl_test_objects[0].value.id;
+
+ test_insert_dup(rhl_test_objects, 2, false);
+ test_insert_dup(rhl_test_objects, 3, false);
+ test_insert_dup(rhl_test_objects, 2, true);
+ test_insert_dup(rhl_test_objects, 3, true);
+
+ return 0;
+}
+
static int thread_lookup_test(struct thread_data *tdata)
{
unsigned int entries = tdata->entries;
do_div(total_time, runs);
pr_info("Average test time: %llu\n", total_time);
+ test_insert_duplicates_run();
+
if (!tcount)
return 0;
}
if (ret & VM_FAULT_RETRY) {
- if (nonblocking)
+ if (nonblocking && !(fault_flags & FAULT_FLAG_RETRY_NOWAIT))
*nonblocking = 0;
return -EBUSY;
}
break;
}
if (*locked) {
- /* VM_FAULT_RETRY didn't trigger */
+ /*
+ * VM_FAULT_RETRY didn't trigger or it was a
+ * FOLL_NOWAIT.
+ */
if (!pages_done)
pages_done = ret;
break;
VM_BUG_ON_PAGE(!PageCompound(page), page);
- if (mem_cgroup_try_charge(page, vma->vm_mm, gfp, &memcg, true)) {
+ if (mem_cgroup_try_charge(page, vma->vm_mm, gfp | __GFP_NORETRY, &memcg,
+ true)) {
put_page(page);
count_vm_event(THP_FAULT_FALLBACK);
return VM_FAULT_FALLBACK;
}
if (unlikely(mem_cgroup_try_charge(new_page, vma->vm_mm,
- huge_gfp, &memcg, true))) {
+ huge_gfp | __GFP_NORETRY, &memcg, true))) {
put_page(new_page);
split_huge_pmd(vma, vmf->pmd, vmf->address);
if (page)
list_for_each_safe(pos, next, &list) {
page = list_entry((void *)pos, struct page, mapping);
- lock_page(page);
+ if (!trylock_page(page))
+ goto next;
/* split_huge_page() removes page from list on success */
if (!split_huge_page(page))
split++;
unlock_page(page);
+next:
put_page(page);
}
#include <linux/bootmem.h>
#include <linux/sysfs.h>
#include <linux/slab.h>
+#include <linux/mmdebug.h>
#include <linux/sched/signal.h>
#include <linux/rmap.h>
#include <linux/string_helpers.h>
page = NULL;
} else {
h->surplus_huge_pages++;
- h->nr_huge_pages_node[page_to_nid(page)]++;
+ h->surplus_huge_pages_node[page_to_nid(page)]++;
}
out_unlock:
struct resv_map *resv_map;
long gbl_reserve;
+ /* This should never happen */
+ if (from > to) {
+ VM_WARN(1, "%s called with a negative range\n", __func__);
+ return -EINVAL;
+ }
+
/*
* Only apply hugepage reservation if asked. At fault time, an
* attempt will be made for VM_NORESERVE to allocate a page
goto out;
}
- VM_BUG_ON_PAGE(PageCompound(page), page);
+ /* TODO: teach khugepaged to collapse THP mapped with pte */
+ if (PageCompound(page)) {
+ result = SCAN_PAGE_COMPOUND;
+ goto out;
+ }
+
VM_BUG_ON_PAGE(!PageAnon(page), page);
/*
goto out_nolock;
}
- if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp, &memcg, true))) {
+ /* Do not oom kill for khugepaged charges */
+ if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp | __GFP_NORETRY,
+ &memcg, true))) {
result = SCAN_CGROUP_CHARGE_FAIL;
goto out_nolock;
}
goto out;
}
- if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp, &memcg, true))) {
+ /* Do not oom kill for khugepaged charges */
+ if (unlikely(mem_cgroup_try_charge(new_page, mm, gfp | __GFP_NORETRY,
+ &memcg, true))) {
result = SCAN_CGROUP_CHARGE_FAIL;
goto out;
}
*out_nid = r->nid;
}
-unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn,
- unsigned long max_pfn)
-{
- struct memblock_type *type = &memblock.memory;
- unsigned int right = type->cnt;
- unsigned int mid, left = 0;
- phys_addr_t addr = PFN_PHYS(pfn + 1);
-
- do {
- mid = (right + left) / 2;
-
- if (addr < type->regions[mid].base)
- right = mid;
- else if (addr >= (type->regions[mid].base +
- type->regions[mid].size))
- left = mid + 1;
- else {
- /* addr is within the region, so pfn + 1 is valid */
- return min(pfn + 1, max_pfn);
- }
- } while (left < right);
-
- if (right == type->cnt)
- return max_pfn;
- else
- return min(PHYS_PFN(type->regions[right].base), max_pfn);
-}
-
/**
* memblock_set_node - set node ID on memblock regions
* @base: base of area to set node ID for
case MPOL_INTERLEAVE:
return !!nodes_equal(a->v.nodes, b->v.nodes);
case MPOL_PREFERRED:
+ /* a's ->flags is the same as b's */
+ if (a->flags & MPOL_F_LOCAL)
+ return true;
return a->v.preferred_node == b->v.preferred_node;
default:
BUG();
* Remove at a later date when no bug reports exist related to
* grouping pages by mobility
*/
- VM_BUG_ON(page_zone(start_page) != page_zone(end_page));
+ VM_BUG_ON(pfn_valid(page_to_pfn(start_page)) &&
+ pfn_valid(page_to_pfn(end_page)) &&
+ page_zone(start_page) != page_zone(end_page));
#endif
if (num_movable)
return false;
/* this guy won't enter reclaim */
- if ((current->flags & PF_MEMALLOC) && !(gfp_mask & __GFP_NOMEMALLOC))
+ if (current->flags & PF_MEMALLOC)
return false;
/* We're only interested __GFP_FS allocations for now */
if (context != MEMMAP_EARLY)
goto not_early;
- if (!early_pfn_valid(pfn)) {
-#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
- /*
- * Skip to the pfn preceding the next valid one (or
- * end_pfn), such that we hit a valid pfn (or end_pfn)
- * on our next iteration of the loop.
- */
- pfn = memblock_next_valid_pfn(pfn, end_pfn) - 1;
-#endif
+ if (!early_pfn_valid(pfn))
continue;
- }
if (!early_pfn_in_nid(pfn, nid))
continue;
if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
#include <linux/log2.h>
static int pcpu_populate_chunk(struct pcpu_chunk *chunk,
- int page_start, int page_end)
+ int page_start, int page_end, gfp_t gfp)
{
return 0;
}
/* nada */
}
-static struct pcpu_chunk *pcpu_create_chunk(void)
+static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp)
{
const int nr_pages = pcpu_group_sizes[0] >> PAGE_SHIFT;
struct pcpu_chunk *chunk;
struct page *pages;
int i;
- chunk = pcpu_alloc_chunk();
+ chunk = pcpu_alloc_chunk(gfp);
if (!chunk)
return NULL;
- pages = alloc_pages(GFP_KERNEL, order_base_2(nr_pages));
+ pages = alloc_pages(gfp, order_base_2(nr_pages));
if (!pages) {
pcpu_free_chunk(chunk);
return NULL;
lockdep_assert_held(&pcpu_alloc_mutex);
if (!pages)
- pages = pcpu_mem_zalloc(pages_size);
+ pages = pcpu_mem_zalloc(pages_size, GFP_KERNEL);
return pages;
}
* @pages: array to put the allocated pages into, indexed by pcpu_page_idx()
* @page_start: page index of the first page to be allocated
* @page_end: page index of the last page to be allocated + 1
+ * @gfp: allocation flags passed to the underlying allocator
*
* Allocate pages [@page_start,@page_end) into @pages for all units.
* The allocation is for @chunk. Percpu core doesn't care about the
* content of @pages and will pass it verbatim to pcpu_map_pages().
*/
static int pcpu_alloc_pages(struct pcpu_chunk *chunk,
- struct page **pages, int page_start, int page_end)
+ struct page **pages, int page_start, int page_end,
+ gfp_t gfp)
{
- const gfp_t gfp = GFP_KERNEL | __GFP_HIGHMEM;
unsigned int cpu, tcpu;
int i;
+ gfp |= __GFP_HIGHMEM;
+
for_each_possible_cpu(cpu) {
for (i = page_start; i < page_end; i++) {
struct page **pagep = &pages[pcpu_page_idx(cpu, i)];
* @chunk: chunk of interest
* @page_start: the start page
* @page_end: the end page
+ * @gfp: allocation flags passed to the underlying memory allocator
*
* For each cpu, populate and map pages [@page_start,@page_end) into
* @chunk.
* pcpu_alloc_mutex, does GFP_KERNEL allocation.
*/
static int pcpu_populate_chunk(struct pcpu_chunk *chunk,
- int page_start, int page_end)
+ int page_start, int page_end, gfp_t gfp)
{
struct page **pages;
if (!pages)
return -ENOMEM;
- if (pcpu_alloc_pages(chunk, pages, page_start, page_end))
+ if (pcpu_alloc_pages(chunk, pages, page_start, page_end, gfp))
return -ENOMEM;
if (pcpu_map_pages(chunk, pages, page_start, page_end)) {
pcpu_free_pages(chunk, pages, page_start, page_end);
}
-static struct pcpu_chunk *pcpu_create_chunk(void)
+static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp)
{
struct pcpu_chunk *chunk;
struct vm_struct **vms;
- chunk = pcpu_alloc_chunk();
+ chunk = pcpu_alloc_chunk(gfp);
if (!chunk)
return NULL;
#include <linux/vmalloc.h>
#include <linux/workqueue.h>
#include <linux/kmemleak.h>
+#include <linux/sched.h>
#include <asm/cacheflush.h>
#include <asm/sections.h>
/**
* pcpu_mem_zalloc - allocate memory
* @size: bytes to allocate
+ * @gfp: allocation flags
*
* Allocate @size bytes. If @size is smaller than PAGE_SIZE,
- * kzalloc() is used; otherwise, vzalloc() is used. The returned
- * memory is always zeroed.
- *
- * CONTEXT:
- * Does GFP_KERNEL allocation.
+ * kzalloc() is used; otherwise, the equivalent of vzalloc() is used.
+ * This is to facilitate passing through whitelisted flags. The
+ * returned memory is always zeroed.
*
* RETURNS:
* Pointer to the allocated area on success, NULL on failure.
*/
-static void *pcpu_mem_zalloc(size_t size)
+static void *pcpu_mem_zalloc(size_t size, gfp_t gfp)
{
if (WARN_ON_ONCE(!slab_is_available()))
return NULL;
if (size <= PAGE_SIZE)
- return kzalloc(size, GFP_KERNEL);
+ return kzalloc(size, gfp);
else
- return vzalloc(size);
+ return __vmalloc(size, gfp | __GFP_ZERO, PAGE_KERNEL);
}
/**
return chunk;
}
-static struct pcpu_chunk *pcpu_alloc_chunk(void)
+static struct pcpu_chunk *pcpu_alloc_chunk(gfp_t gfp)
{
struct pcpu_chunk *chunk;
int region_bits;
- chunk = pcpu_mem_zalloc(pcpu_chunk_struct_size);
+ chunk = pcpu_mem_zalloc(pcpu_chunk_struct_size, gfp);
if (!chunk)
return NULL;
region_bits = pcpu_chunk_map_bits(chunk);
chunk->alloc_map = pcpu_mem_zalloc(BITS_TO_LONGS(region_bits) *
- sizeof(chunk->alloc_map[0]));
+ sizeof(chunk->alloc_map[0]), gfp);
if (!chunk->alloc_map)
goto alloc_map_fail;
chunk->bound_map = pcpu_mem_zalloc(BITS_TO_LONGS(region_bits + 1) *
- sizeof(chunk->bound_map[0]));
+ sizeof(chunk->bound_map[0]), gfp);
if (!chunk->bound_map)
goto bound_map_fail;
chunk->md_blocks = pcpu_mem_zalloc(pcpu_chunk_nr_blocks(chunk) *
- sizeof(chunk->md_blocks[0]));
+ sizeof(chunk->md_blocks[0]), gfp);
if (!chunk->md_blocks)
goto md_blocks_fail;
* pcpu_addr_to_page - translate address to physical address
* pcpu_verify_alloc_info - check alloc_info is acceptable during init
*/
-static int pcpu_populate_chunk(struct pcpu_chunk *chunk, int off, int size);
-static void pcpu_depopulate_chunk(struct pcpu_chunk *chunk, int off, int size);
-static struct pcpu_chunk *pcpu_create_chunk(void);
+static int pcpu_populate_chunk(struct pcpu_chunk *chunk,
+ int page_start, int page_end, gfp_t gfp);
+static void pcpu_depopulate_chunk(struct pcpu_chunk *chunk,
+ int page_start, int page_end);
+static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp);
static void pcpu_destroy_chunk(struct pcpu_chunk *chunk);
static struct page *pcpu_addr_to_page(void *addr);
static int __init pcpu_verify_alloc_info(const struct pcpu_alloc_info *ai);
static void __percpu *pcpu_alloc(size_t size, size_t align, bool reserved,
gfp_t gfp)
{
+ /* whitelisted flags that can be passed to the backing allocators */
+ gfp_t pcpu_gfp = gfp & (GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN);
bool is_atomic = (gfp & GFP_KERNEL) != GFP_KERNEL;
bool do_warn = !(gfp & __GFP_NOWARN);
static int warn_limit = 10;
return NULL;
}
- if (!is_atomic)
- mutex_lock(&pcpu_alloc_mutex);
+ if (!is_atomic) {
+ /*
+ * pcpu_balance_workfn() allocates memory under this mutex,
+ * and it may wait for memory reclaim. Allow current task
+ * to become OOM victim, in case of memory pressure.
+ */
+ if (gfp & __GFP_NOFAIL)
+ mutex_lock(&pcpu_alloc_mutex);
+ else if (mutex_lock_killable(&pcpu_alloc_mutex))
+ return NULL;
+ }
spin_lock_irqsave(&pcpu_lock, flags);
}
if (list_empty(&pcpu_slot[pcpu_nr_slots - 1])) {
- chunk = pcpu_create_chunk();
+ chunk = pcpu_create_chunk(pcpu_gfp);
if (!chunk) {
err = "failed to allocate new chunk";
goto fail;
page_start, page_end) {
WARN_ON(chunk->immutable);
- ret = pcpu_populate_chunk(chunk, rs, re);
+ ret = pcpu_populate_chunk(chunk, rs, re, pcpu_gfp);
spin_lock_irqsave(&pcpu_lock, flags);
if (ret) {
* pcpu_balance_workfn - manage the amount of free chunks and populated pages
* @work: unused
*
- * Reclaim all fully free chunks except for the first one.
+ * Reclaim all fully free chunks except for the first one. This is also
+ * responsible for maintaining the pool of empty populated pages. However,
+ * it is possible that this is called when physical memory is scarce causing
+ * OOM killer to be triggered. We should avoid doing so until an actual
+ * allocation causes the failure as it is possible that requests can be
+ * serviced from already backed regions.
*/
static void pcpu_balance_workfn(struct work_struct *work)
{
+ /* gfp flags passed to underlying allocators */
+ const gfp_t gfp = GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN;
LIST_HEAD(to_free);
struct list_head *free_head = &pcpu_slot[pcpu_nr_slots - 1];
struct pcpu_chunk *chunk, *next;
spin_unlock_irq(&pcpu_lock);
}
pcpu_destroy_chunk(chunk);
+ cond_resched();
}
/*
chunk->nr_pages) {
int nr = min(re - rs, nr_to_pop);
- ret = pcpu_populate_chunk(chunk, rs, rs + nr);
+ ret = pcpu_populate_chunk(chunk, rs, rs + nr, gfp);
if (!ret) {
nr_to_pop -= nr;
spin_lock_irq(&pcpu_lock);
if (nr_to_pop) {
/* ran out of chunks to populate, create a new one and retry */
- chunk = pcpu_create_chunk();
+ chunk = pcpu_create_chunk(gfp);
if (chunk) {
spin_lock_irq(&pcpu_lock);
pcpu_chunk_relocate(chunk, -1);
info = list_entry(pos, struct shmem_inode_info, shrinklist);
inode = &info->vfs_inode;
- if (nr_to_split && split >= nr_to_split) {
- iput(inode);
- continue;
- }
+ if (nr_to_split && split >= nr_to_split)
+ goto leave;
- page = find_lock_page(inode->i_mapping,
+ page = find_get_page(inode->i_mapping,
(inode->i_size & HPAGE_PMD_MASK) >> PAGE_SHIFT);
if (!page)
goto drop;
+ /* No huge page at the end of the file: nothing to split */
if (!PageTransHuge(page)) {
- unlock_page(page);
put_page(page);
goto drop;
}
+ /*
+ * Leave the inode on the list if we failed to lock
+ * the page at this time.
+ *
+ * Waiting for the lock may lead to deadlock in the
+ * reclaim path.
+ */
+ if (!trylock_page(page)) {
+ put_page(page);
+ goto leave;
+ }
+
ret = split_huge_page(page);
unlock_page(page);
put_page(page);
- if (ret) {
- /* split failed: leave it on the list */
- iput(inode);
- continue;
- }
+ /* If split failed leave the inode on the list */
+ if (ret)
+ goto leave;
split++;
drop:
list_del_init(&info->shrinklist);
removed++;
+leave:
iput(inode);
}
if (stat.nr_writeback && stat.nr_writeback == nr_taken)
set_bit(PGDAT_WRITEBACK, &pgdat->flags);
+ /*
+ * If dirty pages are scanned that are not queued for IO, it
+ * implies that flushers are not doing their job. This can
+ * happen when memory pressure pushes dirty pages to the end of
+ * the LRU before the dirty limits are breached and the dirty
+ * data has expired. It can also happen when the proportion of
+ * dirty pages grows not through writes but through memory
+ * pressure reclaiming all the clean cache. And in some cases,
+ * the flushers simply cannot keep up with the allocation
+ * rate. Nudge the flusher threads in case they are asleep.
+ */
+ if (stat.nr_unqueued_dirty == nr_taken)
+ wakeup_flusher_threads(WB_REASON_VMSCAN);
+
/*
* Legacy memcg will stall in page writeback so avoid forcibly
* stalling here.
if (stat.nr_dirty && stat.nr_dirty == stat.nr_congested)
set_bit(PGDAT_CONGESTED, &pgdat->flags);
- /*
- * If dirty pages are scanned that are not queued for IO, it
- * implies that flushers are not doing their job. This can
- * happen when memory pressure pushes dirty pages to the end of
- * the LRU before the dirty limits are breached and the dirty
- * data has expired. It can also happen when the proportion of
- * dirty pages grows not through writes but through memory
- * pressure reclaiming all the clean cache. And in some cases,
- * the flushers simply cannot keep up with the allocation
- * rate. Nudge the flusher threads in case they are asleep, but
- * also allow kswapd to start writing pages during reclaim.
- */
- if (stat.nr_unqueued_dirty == nr_taken) {
- wakeup_flusher_threads(WB_REASON_VMSCAN);
+ /* Allow kswapd to start writing pages during reclaim. */
+ if (stat.nr_unqueued_dirty == nr_taken)
set_bit(PGDAT_DIRTY, &pgdat->flags);
- }
/*
* If kswapd scans pages marked marked for immediate
.exit = vlan_exit_net,
.id = &vlan_net_id,
.size = sizeof(struct vlan_net),
- .async = true,
};
static int __init vlan_proto_init(void)
* original position later
*/
skb_push(skb, offset);
- skb = *skbp = vlan_insert_tag(skb, skb->vlan_proto,
- skb->vlan_tci);
+ skb = *skbp = vlan_insert_inner_tag(skb, skb->vlan_proto,
+ skb->vlan_tci, skb->mac_len);
if (!skb)
return false;
skb_pull(skb, offset + VLAN_HLEN);
if (!vn->proc_vlan_dir)
goto err;
- vn->proc_vlan_conf = proc_create(name_conf, S_IFREG|S_IRUSR|S_IWUSR,
- vn->proc_vlan_dir, &vlan_fops);
+ vn->proc_vlan_conf = proc_create(name_conf, S_IFREG | 0600,
+ vn->proc_vlan_dir, &vlan_fops);
if (!vn->proc_vlan_conf)
goto err;
return 0;
if (!strcmp(vlandev->name, name_conf))
return -EINVAL;
vlan->dent =
- proc_create_data(vlandev->name, S_IFREG|S_IRUSR|S_IWUSR,
+ proc_create_data(vlandev->name, S_IFREG | 0600,
vn->proc_vlan_dir, &vlandev_fops, vlandev);
if (!vlan->dent)
return -ENOBUFS;
if (!atalk_proc_dir)
goto out;
- p = proc_create("interface", S_IRUGO, atalk_proc_dir,
+ p = proc_create("interface", 0444, atalk_proc_dir,
&atalk_seq_interface_fops);
if (!p)
goto out_interface;
- p = proc_create("route", S_IRUGO, atalk_proc_dir,
+ p = proc_create("route", 0444, atalk_proc_dir,
&atalk_seq_route_fops);
if (!p)
goto out_route;
- p = proc_create("socket", S_IRUGO, atalk_proc_dir,
+ p = proc_create("socket", 0444, atalk_proc_dir,
&atalk_seq_socket_fops);
if (!p)
goto out_socket;
- p = proc_create("arp", S_IRUGO, atalk_proc_dir, &atalk_seq_arp_fops);
+ p = proc_create("arp", 0444, atalk_proc_dir, &atalk_seq_arp_fops);
if (!p)
goto out_arp;
return scnprintf(buf, PAGE_SIZE, "%d\n", link_rate);
}
-static DEVICE_ATTR(address, S_IRUGO, show_address, NULL);
-static DEVICE_ATTR(atmaddress, S_IRUGO, show_atmaddress, NULL);
-static DEVICE_ATTR(atmindex, S_IRUGO, show_atmindex, NULL);
-static DEVICE_ATTR(carrier, S_IRUGO, show_carrier, NULL);
-static DEVICE_ATTR(type, S_IRUGO, show_type, NULL);
-static DEVICE_ATTR(link_rate, S_IRUGO, show_link_rate, NULL);
+static DEVICE_ATTR(address, 0444, show_address, NULL);
+static DEVICE_ATTR(atmaddress, 0444, show_atmaddress, NULL);
+static DEVICE_ATTR(atmindex, 0444, show_atmindex, NULL);
+static DEVICE_ATTR(carrier, 0444, show_carrier, NULL);
+static DEVICE_ATTR(type, 0444, show_type, NULL);
+static DEVICE_ATTR(link_rate, 0444, show_link_rate, NULL);
static struct device_attribute *atm_attrs[] = {
&dev_attr_atmaddress,
{
struct proc_dir_entry *p;
- p = proc_create("arp", S_IRUGO, atm_proc_root, &arp_seq_fops);
+ p = proc_create("arp", 0444, atm_proc_root, &arp_seq_fops);
if (!p) {
pr_err("Unable to initialize /proc/net/atm/arp\n");
atm_clip_exit_noproc();
#ifdef CONFIG_PROC_FS
struct proc_dir_entry *p;
- p = proc_create("lec", S_IRUGO, atm_proc_root, &lec_seq_fops);
+ p = proc_create("lec", 0444, atm_proc_root, &lec_seq_fops);
if (!p) {
pr_err("Unable to initialize /proc/net/atm/lec\n");
return -ENOMEM;
for (e = atm_proc_ents; e->name; e++) {
struct proc_dir_entry *dirent;
- dirent = proc_create(e->name, S_IRUGO,
+ dirent = proc_create(e->name, 0444,
atm_proc_root, e->proc_fops);
if (!dirent)
goto err_out_remove;
dev_add_pack(&ax25_packet_type);
register_netdevice_notifier(&ax25_dev_notifier);
- proc_create("ax25_route", S_IRUGO, init_net.proc_net,
+ proc_create("ax25_route", 0444, init_net.proc_net,
&ax25_route_fops);
- proc_create("ax25", S_IRUGO, init_net.proc_net, &ax25_info_fops);
- proc_create("ax25_calls", S_IRUGO, init_net.proc_net, &ax25_uid_fops);
+ proc_create("ax25", 0444, init_net.proc_net, &ax25_info_fops);
+ proc_create("ax25_calls", 0444, init_net.proc_net, &ax25_uid_fops);
out:
return rc;
}
#include <linux/kernel.h>
#include <linux/kref.h>
#include <linux/list.h>
+#include <linux/netlink.h>
#include <linux/rculist.h>
#include <linux/rcupdate.h>
#include <linux/seq_file.h>
#include <linux/string.h>
#include <linux/workqueue.h>
#include <net/arp.h>
+#include <net/genetlink.h>
+#include <net/netlink.h>
+#include <net/sock.h>
+#include <uapi/linux/batman_adv.h>
#include "bridge_loop_avoidance.h"
#include "hard-interface.h"
#include "hash.h"
#include "log.h"
+#include "netlink.h"
#include "originator.h"
#include "send.h"
+#include "soft-interface.h"
#include "translation-table.h"
#include "tvlv.h"
batadv_arp_hw_src(skb, hdr_size), &ip_src,
batadv_arp_hw_dst(skb, hdr_size), &ip_dst);
- if (hdr_size == 0)
+ if (hdr_size < sizeof(struct batadv_unicast_packet))
return;
unicast_4addr_packet = (struct batadv_unicast_4addr_packet *)skb->data;
}
#endif
+/**
+ * batadv_dat_cache_dump_entry() - dump one entry of the DAT cache table to a
+ * netlink socket
+ * @msg: buffer for the message
+ * @portid: netlink port
+ * @seq: Sequence number of netlink message
+ * @dat_entry: entry to dump
+ *
+ * Return: 0 or error code.
+ */
+static int
+batadv_dat_cache_dump_entry(struct sk_buff *msg, u32 portid, u32 seq,
+ struct batadv_dat_entry *dat_entry)
+{
+ int msecs;
+ void *hdr;
+
+ hdr = genlmsg_put(msg, portid, seq, &batadv_netlink_family,
+ NLM_F_MULTI, BATADV_CMD_GET_DAT_CACHE);
+ if (!hdr)
+ return -ENOBUFS;
+
+ msecs = jiffies_to_msecs(jiffies - dat_entry->last_update);
+
+ if (nla_put_in_addr(msg, BATADV_ATTR_DAT_CACHE_IP4ADDRESS,
+ dat_entry->ip) ||
+ nla_put(msg, BATADV_ATTR_DAT_CACHE_HWADDRESS, ETH_ALEN,
+ dat_entry->mac_addr) ||
+ nla_put_u16(msg, BATADV_ATTR_DAT_CACHE_VID, dat_entry->vid) ||
+ nla_put_u32(msg, BATADV_ATTR_LAST_SEEN_MSECS, msecs)) {
+ genlmsg_cancel(msg, hdr);
+ return -EMSGSIZE;
+ }
+
+ genlmsg_end(msg, hdr);
+ return 0;
+}
+
+/**
+ * batadv_dat_cache_dump_bucket() - dump one bucket of the DAT cache table to
+ * a netlink socket
+ * @msg: buffer for the message
+ * @portid: netlink port
+ * @seq: Sequence number of netlink message
+ * @head: bucket to dump
+ * @idx_skip: How many entries to skip
+ *
+ * Return: 0 or error code.
+ */
+static int
+batadv_dat_cache_dump_bucket(struct sk_buff *msg, u32 portid, u32 seq,
+ struct hlist_head *head, int *idx_skip)
+{
+ struct batadv_dat_entry *dat_entry;
+ int idx = 0;
+
+ rcu_read_lock();
+ hlist_for_each_entry_rcu(dat_entry, head, hash_entry) {
+ if (idx < *idx_skip)
+ goto skip;
+
+ if (batadv_dat_cache_dump_entry(msg, portid, seq,
+ dat_entry)) {
+ rcu_read_unlock();
+ *idx_skip = idx;
+
+ return -EMSGSIZE;
+ }
+
+skip:
+ idx++;
+ }
+ rcu_read_unlock();
+
+ return 0;
+}
+
+/**
+ * batadv_dat_cache_dump() - dump DAT cache table to a netlink socket
+ * @msg: buffer for the message
+ * @cb: callback structure containing arguments
+ *
+ * Return: message length.
+ */
+int batadv_dat_cache_dump(struct sk_buff *msg, struct netlink_callback *cb)
+{
+ struct batadv_hard_iface *primary_if = NULL;
+ int portid = NETLINK_CB(cb->skb).portid;
+ struct net *net = sock_net(cb->skb->sk);
+ struct net_device *soft_iface;
+ struct batadv_hashtable *hash;
+ struct batadv_priv *bat_priv;
+ int bucket = cb->args[0];
+ struct hlist_head *head;
+ int idx = cb->args[1];
+ int ifindex;
+ int ret = 0;
+
+ ifindex = batadv_netlink_get_ifindex(cb->nlh,
+ BATADV_ATTR_MESH_IFINDEX);
+ if (!ifindex)
+ return -EINVAL;
+
+ soft_iface = dev_get_by_index(net, ifindex);
+ if (!soft_iface || !batadv_softif_is_valid(soft_iface)) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ bat_priv = netdev_priv(soft_iface);
+ hash = bat_priv->dat.hash;
+
+ primary_if = batadv_primary_if_get_selected(bat_priv);
+ if (!primary_if || primary_if->if_status != BATADV_IF_ACTIVE) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+ while (bucket < hash->size) {
+ head = &hash->table[bucket];
+
+ if (batadv_dat_cache_dump_bucket(msg, portid,
+ cb->nlh->nlmsg_seq, head,
+ &idx))
+ break;
+
+ bucket++;
+ idx = 0;
+ }
+
+ cb->args[0] = bucket;
+ cb->args[1] = idx;
+
+ ret = msg->len;
+
+out:
+ if (primary_if)
+ batadv_hardif_put(primary_if);
+
+ if (soft_iface)
+ dev_put(soft_iface);
+
+ return ret;
+}
+
/**
* batadv_arp_get_type() - parse an ARP packet and gets the type
* @bat_priv: the bat priv with all the soft interface information
#include "originator.h"
+struct netlink_callback;
struct seq_file;
struct sk_buff;
int batadv_dat_init(struct batadv_priv *bat_priv);
void batadv_dat_free(struct batadv_priv *bat_priv);
int batadv_dat_cache_seq_print_text(struct seq_file *seq, void *offset);
+int batadv_dat_cache_dump(struct sk_buff *msg, struct netlink_callback *cb);
/**
* batadv_dat_inc_counter() - increment the correct DAT packet counter
{
}
+static inline int
+batadv_dat_cache_dump(struct sk_buff *msg, struct netlink_callback *cb)
+{
+ return -EOPNOTSUPP;
+}
+
static inline void batadv_dat_inc_counter(struct batadv_priv *bat_priv,
u8 subtype)
{
#include <linux/debugfs.h>
#include <linux/errno.h>
#include <linux/etherdevice.h>
+#include <linux/eventpoll.h>
#include <linux/export.h>
#include <linux/fcntl.h>
#include <linux/fs.h>
#include <linux/compiler.h>
#include <linux/debugfs.h>
#include <linux/errno.h>
+#include <linux/eventpoll.h>
#include <linux/export.h>
#include <linux/fcntl.h>
#include <linux/fs.h>
#include <linux/list.h>
#include <linux/lockdep.h>
#include <linux/netdevice.h>
+#include <linux/netlink.h>
#include <linux/printk.h>
#include <linux/rculist.h>
#include <linux/rcupdate.h>
#include <linux/types.h>
#include <linux/workqueue.h>
#include <net/addrconf.h>
+#include <net/genetlink.h>
#include <net/if_inet6.h>
#include <net/ip.h>
#include <net/ipv6.h>
+#include <net/netlink.h>
+#include <net/sock.h>
#include <uapi/linux/batadv_packet.h>
+#include <uapi/linux/batman_adv.h>
#include "hard-interface.h"
#include "hash.h"
#include "log.h"
+#include "netlink.h"
+#include "soft-interface.h"
#include "translation-table.h"
#include "tvlv.h"
return upper;
}
+/**
+ * batadv_mcast_addr_is_ipv4() - check if multicast MAC is IPv4
+ * @addr: the MAC address to check
+ *
+ * Return: True, if MAC address is one reserved for IPv4 multicast, false
+ * otherwise.
+ */
+static bool batadv_mcast_addr_is_ipv4(const u8 *addr)
+{
+ static const u8 prefix[] = {0x01, 0x00, 0x5E};
+
+ return memcmp(prefix, addr, sizeof(prefix)) == 0;
+}
+
+/**
+ * batadv_mcast_addr_is_ipv6() - check if multicast MAC is IPv6
+ * @addr: the MAC address to check
+ *
+ * Return: True, if MAC address is one reserved for IPv6 multicast, false
+ * otherwise.
+ */
+static bool batadv_mcast_addr_is_ipv6(const u8 *addr)
+{
+ static const u8 prefix[] = {0x33, 0x33};
+
+ return memcmp(prefix, addr, sizeof(prefix)) == 0;
+}
+
/**
* batadv_mcast_mla_softif_get() - get softif multicast listeners
+ * @bat_priv: the bat priv with all the soft interface information
* @dev: the device to collect multicast addresses from
* @mcast_list: a list to put found addresses into
*
* Return: -ENOMEM on memory allocation error or the number of
* items added to the mcast_list otherwise.
*/
-static int batadv_mcast_mla_softif_get(struct net_device *dev,
+static int batadv_mcast_mla_softif_get(struct batadv_priv *bat_priv,
+ struct net_device *dev,
struct hlist_head *mcast_list)
{
+ bool all_ipv4 = bat_priv->mcast.flags & BATADV_MCAST_WANT_ALL_IPV4;
+ bool all_ipv6 = bat_priv->mcast.flags & BATADV_MCAST_WANT_ALL_IPV6;
struct net_device *bridge = batadv_mcast_get_bridge(dev);
struct netdev_hw_addr *mc_list_entry;
struct batadv_hw_addr *new;
netif_addr_lock_bh(bridge ? bridge : dev);
netdev_for_each_mc_addr(mc_list_entry, bridge ? bridge : dev) {
+ if (all_ipv4 && batadv_mcast_addr_is_ipv4(mc_list_entry->addr))
+ continue;
+
+ if (all_ipv6 && batadv_mcast_addr_is_ipv6(mc_list_entry->addr))
+ continue;
+
new = kmalloc(sizeof(*new), GFP_ATOMIC);
if (!new) {
ret = -ENOMEM;
/**
* batadv_mcast_mla_bridge_get() - get bridged-in multicast listeners
+ * @bat_priv: the bat priv with all the soft interface information
* @dev: a bridge slave whose bridge to collect multicast addresses from
* @mcast_list: a list to put found addresses into
*
* Return: -ENOMEM on memory allocation error or the number of
* items added to the mcast_list otherwise.
*/
-static int batadv_mcast_mla_bridge_get(struct net_device *dev,
+static int batadv_mcast_mla_bridge_get(struct batadv_priv *bat_priv,
+ struct net_device *dev,
struct hlist_head *mcast_list)
{
struct list_head bridge_mcast_list = LIST_HEAD_INIT(bridge_mcast_list);
+ bool all_ipv4 = bat_priv->mcast.flags & BATADV_MCAST_WANT_ALL_IPV4;
+ bool all_ipv6 = bat_priv->mcast.flags & BATADV_MCAST_WANT_ALL_IPV6;
struct br_ip_list *br_ip_entry, *tmp;
struct batadv_hw_addr *new;
u8 mcast_addr[ETH_ALEN];
goto out;
list_for_each_entry(br_ip_entry, &bridge_mcast_list, list) {
+ if (all_ipv4 && br_ip_entry->addr.proto == htons(ETH_P_IP))
+ continue;
+
+ if (all_ipv6 && br_ip_entry->addr.proto == htons(ETH_P_IPV6))
+ continue;
+
batadv_mcast_mla_br_addr_cpy(mcast_addr, &br_ip_entry->addr);
if (batadv_mcast_mla_is_duplicate(mcast_addr, mcast_list))
continue;
bat_priv->mcast.enabled = true;
}
- return !(mcast_data.flags &
- (BATADV_MCAST_WANT_ALL_IPV4 | BATADV_MCAST_WANT_ALL_IPV6));
+ return !(mcast_data.flags & BATADV_MCAST_WANT_ALL_IPV4 &&
+ mcast_data.flags & BATADV_MCAST_WANT_ALL_IPV6);
}
/**
if (!batadv_mcast_mla_tvlv_update(bat_priv))
goto update;
- ret = batadv_mcast_mla_softif_get(soft_iface, &mcast_list);
+ ret = batadv_mcast_mla_softif_get(bat_priv, soft_iface, &mcast_list);
if (ret < 0)
goto out;
- ret = batadv_mcast_mla_bridge_get(soft_iface, &mcast_list);
+ ret = batadv_mcast_mla_bridge_get(bat_priv, soft_iface, &mcast_list);
if (ret < 0)
goto out;
}
#endif
+/**
+ * batadv_mcast_mesh_info_put() - put multicast info into a netlink message
+ * @msg: buffer for the message
+ * @bat_priv: the bat priv with all the soft interface information
+ *
+ * Return: 0 or error code.
+ */
+int batadv_mcast_mesh_info_put(struct sk_buff *msg,
+ struct batadv_priv *bat_priv)
+{
+ u32 flags = bat_priv->mcast.flags;
+ u32 flags_priv = BATADV_NO_FLAGS;
+
+ if (bat_priv->mcast.bridged) {
+ flags_priv |= BATADV_MCAST_FLAGS_BRIDGED;
+
+ if (bat_priv->mcast.querier_ipv4.exists)
+ flags_priv |= BATADV_MCAST_FLAGS_QUERIER_IPV4_EXISTS;
+ if (bat_priv->mcast.querier_ipv6.exists)
+ flags_priv |= BATADV_MCAST_FLAGS_QUERIER_IPV6_EXISTS;
+ if (bat_priv->mcast.querier_ipv4.shadowing)
+ flags_priv |= BATADV_MCAST_FLAGS_QUERIER_IPV4_SHADOWING;
+ if (bat_priv->mcast.querier_ipv6.shadowing)
+ flags_priv |= BATADV_MCAST_FLAGS_QUERIER_IPV6_SHADOWING;
+ }
+
+ if (nla_put_u32(msg, BATADV_ATTR_MCAST_FLAGS, flags) ||
+ nla_put_u32(msg, BATADV_ATTR_MCAST_FLAGS_PRIV, flags_priv))
+ return -EMSGSIZE;
+
+ return 0;
+}
+
+/**
+ * batadv_mcast_flags_dump_entry() - dump one entry of the multicast flags table
+ * to a netlink socket
+ * @msg: buffer for the message
+ * @portid: netlink port
+ * @seq: Sequence number of netlink message
+ * @orig_node: originator to dump the multicast flags of
+ *
+ * Return: 0 or error code.
+ */
+static int
+batadv_mcast_flags_dump_entry(struct sk_buff *msg, u32 portid, u32 seq,
+ struct batadv_orig_node *orig_node)
+{
+ void *hdr;
+
+ hdr = genlmsg_put(msg, portid, seq, &batadv_netlink_family,
+ NLM_F_MULTI, BATADV_CMD_GET_MCAST_FLAGS);
+ if (!hdr)
+ return -ENOBUFS;
+
+ if (nla_put(msg, BATADV_ATTR_ORIG_ADDRESS, ETH_ALEN,
+ orig_node->orig)) {
+ genlmsg_cancel(msg, hdr);
+ return -EMSGSIZE;
+ }
+
+ if (test_bit(BATADV_ORIG_CAPA_HAS_MCAST,
+ &orig_node->capabilities)) {
+ if (nla_put_u32(msg, BATADV_ATTR_MCAST_FLAGS,
+ orig_node->mcast_flags)) {
+ genlmsg_cancel(msg, hdr);
+ return -EMSGSIZE;
+ }
+ }
+
+ genlmsg_end(msg, hdr);
+ return 0;
+}
+
+/**
+ * batadv_mcast_flags_dump_bucket() - dump one bucket of the multicast flags
+ * table to a netlink socket
+ * @msg: buffer for the message
+ * @portid: netlink port
+ * @seq: Sequence number of netlink message
+ * @head: bucket to dump
+ * @idx_skip: How many entries to skip
+ *
+ * Return: 0 or error code.
+ */
+static int
+batadv_mcast_flags_dump_bucket(struct sk_buff *msg, u32 portid, u32 seq,
+ struct hlist_head *head, long *idx_skip)
+{
+ struct batadv_orig_node *orig_node;
+ long idx = 0;
+
+ rcu_read_lock();
+ hlist_for_each_entry_rcu(orig_node, head, hash_entry) {
+ if (!test_bit(BATADV_ORIG_CAPA_HAS_MCAST,
+ &orig_node->capa_initialized))
+ continue;
+
+ if (idx < *idx_skip)
+ goto skip;
+
+ if (batadv_mcast_flags_dump_entry(msg, portid, seq,
+ orig_node)) {
+ rcu_read_unlock();
+ *idx_skip = idx;
+
+ return -EMSGSIZE;
+ }
+
+skip:
+ idx++;
+ }
+ rcu_read_unlock();
+
+ return 0;
+}
+
+/**
+ * __batadv_mcast_flags_dump() - dump multicast flags table to a netlink socket
+ * @msg: buffer for the message
+ * @portid: netlink port
+ * @seq: Sequence number of netlink message
+ * @bat_priv: the bat priv with all the soft interface information
+ * @bucket: current bucket to dump
+ * @idx: index in current bucket to the next entry to dump
+ *
+ * Return: 0 or error code.
+ */
+static int
+__batadv_mcast_flags_dump(struct sk_buff *msg, u32 portid, u32 seq,
+ struct batadv_priv *bat_priv, long *bucket, long *idx)
+{
+ struct batadv_hashtable *hash = bat_priv->orig_hash;
+ long bucket_tmp = *bucket;
+ struct hlist_head *head;
+ long idx_tmp = *idx;
+
+ while (bucket_tmp < hash->size) {
+ head = &hash->table[bucket_tmp];
+
+ if (batadv_mcast_flags_dump_bucket(msg, portid, seq, head,
+ &idx_tmp))
+ break;
+
+ bucket_tmp++;
+ idx_tmp = 0;
+ }
+
+ *bucket = bucket_tmp;
+ *idx = idx_tmp;
+
+ return msg->len;
+}
+
+/**
+ * batadv_mcast_netlink_get_primary() - get primary interface from netlink
+ * callback
+ * @cb: netlink callback structure
+ * @primary_if: the primary interface pointer to return the result in
+ *
+ * Return: 0 or error code.
+ */
+static int
+batadv_mcast_netlink_get_primary(struct netlink_callback *cb,
+ struct batadv_hard_iface **primary_if)
+{
+ struct batadv_hard_iface *hard_iface = NULL;
+ struct net *net = sock_net(cb->skb->sk);
+ struct net_device *soft_iface;
+ struct batadv_priv *bat_priv;
+ int ifindex;
+ int ret = 0;
+
+ ifindex = batadv_netlink_get_ifindex(cb->nlh, BATADV_ATTR_MESH_IFINDEX);
+ if (!ifindex)
+ return -EINVAL;
+
+ soft_iface = dev_get_by_index(net, ifindex);
+ if (!soft_iface || !batadv_softif_is_valid(soft_iface)) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ bat_priv = netdev_priv(soft_iface);
+
+ hard_iface = batadv_primary_if_get_selected(bat_priv);
+ if (!hard_iface || hard_iface->if_status != BATADV_IF_ACTIVE) {
+ ret = -ENOENT;
+ goto out;
+ }
+
+out:
+ if (soft_iface)
+ dev_put(soft_iface);
+
+ if (!ret && primary_if)
+ *primary_if = hard_iface;
+ else
+ batadv_hardif_put(hard_iface);
+
+ return ret;
+}
+
+/**
+ * batadv_mcast_flags_dump() - dump multicast flags table to a netlink socket
+ * @msg: buffer for the message
+ * @cb: callback structure containing arguments
+ *
+ * Return: message length.
+ */
+int batadv_mcast_flags_dump(struct sk_buff *msg, struct netlink_callback *cb)
+{
+ struct batadv_hard_iface *primary_if = NULL;
+ int portid = NETLINK_CB(cb->skb).portid;
+ struct batadv_priv *bat_priv;
+ long *bucket = &cb->args[0];
+ long *idx = &cb->args[1];
+ int ret;
+
+ ret = batadv_mcast_netlink_get_primary(cb, &primary_if);
+ if (ret)
+ return ret;
+
+ bat_priv = netdev_priv(primary_if->soft_iface);
+ ret = __batadv_mcast_flags_dump(msg, portid, cb->nlh->nlmsg_seq,
+ bat_priv, bucket, idx);
+
+ batadv_hardif_put(primary_if);
+ return ret;
+}
+
/**
* batadv_mcast_free() - free the multicast optimizations structures
* @bat_priv: the bat priv with all the soft interface information
#include "main.h"
+struct netlink_callback;
struct seq_file;
struct sk_buff;
int batadv_mcast_flags_seq_print_text(struct seq_file *seq, void *offset);
+int batadv_mcast_mesh_info_put(struct sk_buff *msg,
+ struct batadv_priv *bat_priv);
+
+int batadv_mcast_flags_dump(struct sk_buff *msg, struct netlink_callback *cb);
+
void batadv_mcast_free(struct batadv_priv *bat_priv);
void batadv_mcast_purge_orig(struct batadv_orig_node *orig_node);
return 0;
}
+static inline int
+batadv_mcast_mesh_info_put(struct sk_buff *msg, struct batadv_priv *bat_priv)
+{
+ return 0;
+}
+
+static inline int batadv_mcast_flags_dump(struct sk_buff *msg,
+ struct netlink_callback *cb)
+{
+ return -EOPNOTSUPP;
+}
+
static inline void batadv_mcast_free(struct batadv_priv *bat_priv)
{
}
#include "bat_algo.h"
#include "bridge_loop_avoidance.h"
+#include "distributed-arp-table.h"
#include "gateway_client.h"
#include "hard-interface.h"
+#include "multicast.h"
#include "originator.h"
#include "soft-interface.h"
#include "tp_meter.h"
};
static const struct nla_policy batadv_netlink_policy[NUM_BATADV_ATTR] = {
- [BATADV_ATTR_VERSION] = { .type = NLA_STRING },
- [BATADV_ATTR_ALGO_NAME] = { .type = NLA_STRING },
- [BATADV_ATTR_MESH_IFINDEX] = { .type = NLA_U32 },
- [BATADV_ATTR_MESH_IFNAME] = { .type = NLA_STRING },
- [BATADV_ATTR_MESH_ADDRESS] = { .len = ETH_ALEN },
- [BATADV_ATTR_HARD_IFINDEX] = { .type = NLA_U32 },
- [BATADV_ATTR_HARD_IFNAME] = { .type = NLA_STRING },
- [BATADV_ATTR_HARD_ADDRESS] = { .len = ETH_ALEN },
- [BATADV_ATTR_ORIG_ADDRESS] = { .len = ETH_ALEN },
- [BATADV_ATTR_TPMETER_RESULT] = { .type = NLA_U8 },
- [BATADV_ATTR_TPMETER_TEST_TIME] = { .type = NLA_U32 },
- [BATADV_ATTR_TPMETER_BYTES] = { .type = NLA_U64 },
- [BATADV_ATTR_TPMETER_COOKIE] = { .type = NLA_U32 },
- [BATADV_ATTR_ACTIVE] = { .type = NLA_FLAG },
- [BATADV_ATTR_TT_ADDRESS] = { .len = ETH_ALEN },
- [BATADV_ATTR_TT_TTVN] = { .type = NLA_U8 },
- [BATADV_ATTR_TT_LAST_TTVN] = { .type = NLA_U8 },
- [BATADV_ATTR_TT_CRC32] = { .type = NLA_U32 },
- [BATADV_ATTR_TT_VID] = { .type = NLA_U16 },
- [BATADV_ATTR_TT_FLAGS] = { .type = NLA_U32 },
- [BATADV_ATTR_FLAG_BEST] = { .type = NLA_FLAG },
- [BATADV_ATTR_LAST_SEEN_MSECS] = { .type = NLA_U32 },
- [BATADV_ATTR_NEIGH_ADDRESS] = { .len = ETH_ALEN },
- [BATADV_ATTR_TQ] = { .type = NLA_U8 },
- [BATADV_ATTR_THROUGHPUT] = { .type = NLA_U32 },
- [BATADV_ATTR_BANDWIDTH_UP] = { .type = NLA_U32 },
- [BATADV_ATTR_BANDWIDTH_DOWN] = { .type = NLA_U32 },
- [BATADV_ATTR_ROUTER] = { .len = ETH_ALEN },
- [BATADV_ATTR_BLA_OWN] = { .type = NLA_FLAG },
- [BATADV_ATTR_BLA_ADDRESS] = { .len = ETH_ALEN },
- [BATADV_ATTR_BLA_VID] = { .type = NLA_U16 },
- [BATADV_ATTR_BLA_BACKBONE] = { .len = ETH_ALEN },
- [BATADV_ATTR_BLA_CRC] = { .type = NLA_U16 },
+ [BATADV_ATTR_VERSION] = { .type = NLA_STRING },
+ [BATADV_ATTR_ALGO_NAME] = { .type = NLA_STRING },
+ [BATADV_ATTR_MESH_IFINDEX] = { .type = NLA_U32 },
+ [BATADV_ATTR_MESH_IFNAME] = { .type = NLA_STRING },
+ [BATADV_ATTR_MESH_ADDRESS] = { .len = ETH_ALEN },
+ [BATADV_ATTR_HARD_IFINDEX] = { .type = NLA_U32 },
+ [BATADV_ATTR_HARD_IFNAME] = { .type = NLA_STRING },
+ [BATADV_ATTR_HARD_ADDRESS] = { .len = ETH_ALEN },
+ [BATADV_ATTR_ORIG_ADDRESS] = { .len = ETH_ALEN },
+ [BATADV_ATTR_TPMETER_RESULT] = { .type = NLA_U8 },
+ [BATADV_ATTR_TPMETER_TEST_TIME] = { .type = NLA_U32 },
+ [BATADV_ATTR_TPMETER_BYTES] = { .type = NLA_U64 },
+ [BATADV_ATTR_TPMETER_COOKIE] = { .type = NLA_U32 },
+ [BATADV_ATTR_ACTIVE] = { .type = NLA_FLAG },
+ [BATADV_ATTR_TT_ADDRESS] = { .len = ETH_ALEN },
+ [BATADV_ATTR_TT_TTVN] = { .type = NLA_U8 },
+ [BATADV_ATTR_TT_LAST_TTVN] = { .type = NLA_U8 },
+ [BATADV_ATTR_TT_CRC32] = { .type = NLA_U32 },
+ [BATADV_ATTR_TT_VID] = { .type = NLA_U16 },
+ [BATADV_ATTR_TT_FLAGS] = { .type = NLA_U32 },
+ [BATADV_ATTR_FLAG_BEST] = { .type = NLA_FLAG },
+ [BATADV_ATTR_LAST_SEEN_MSECS] = { .type = NLA_U32 },
+ [BATADV_ATTR_NEIGH_ADDRESS] = { .len = ETH_ALEN },
+ [BATADV_ATTR_TQ] = { .type = NLA_U8 },
+ [BATADV_ATTR_THROUGHPUT] = { .type = NLA_U32 },
+ [BATADV_ATTR_BANDWIDTH_UP] = { .type = NLA_U32 },
+ [BATADV_ATTR_BANDWIDTH_DOWN] = { .type = NLA_U32 },
+ [BATADV_ATTR_ROUTER] = { .len = ETH_ALEN },
+ [BATADV_ATTR_BLA_OWN] = { .type = NLA_FLAG },
+ [BATADV_ATTR_BLA_ADDRESS] = { .len = ETH_ALEN },
+ [BATADV_ATTR_BLA_VID] = { .type = NLA_U16 },
+ [BATADV_ATTR_BLA_BACKBONE] = { .len = ETH_ALEN },
+ [BATADV_ATTR_BLA_CRC] = { .type = NLA_U16 },
+ [BATADV_ATTR_DAT_CACHE_IP4ADDRESS] = { .type = NLA_U32 },
+ [BATADV_ATTR_DAT_CACHE_HWADDRESS] = { .len = ETH_ALEN },
+ [BATADV_ATTR_DAT_CACHE_VID] = { .type = NLA_U16 },
+ [BATADV_ATTR_MCAST_FLAGS] = { .type = NLA_U32 },
+ [BATADV_ATTR_MCAST_FLAGS_PRIV] = { .type = NLA_U32 },
};
/**
goto out;
#endif
+ if (batadv_mcast_mesh_info_put(msg, bat_priv))
+ goto out;
+
primary_if = batadv_primary_if_get_selected(bat_priv);
if (primary_if && primary_if->if_status == BATADV_IF_ACTIVE) {
hard_iface = primary_if->net_dev;
.policy = batadv_netlink_policy,
.dumpit = batadv_bla_backbone_dump,
},
+ {
+ .cmd = BATADV_CMD_GET_DAT_CACHE,
+ .flags = GENL_ADMIN_PERM,
+ .policy = batadv_netlink_policy,
+ .dumpit = batadv_dat_cache_dump,
+ },
+ {
+ .cmd = BATADV_CMD_GET_MCAST_FLAGS,
+ .flags = GENL_ADMIN_PERM,
+ .policy = batadv_netlink_policy,
+ .dumpit = batadv_mcast_flags_dump,
+ },
};
/**
* batadv_reroute_unicast_packet() - update the unicast header for re-routing
* @bat_priv: the bat priv with all the soft interface information
+ * @skb: unicast packet to process
* @unicast_packet: the unicast header to be updated
* @dst_addr: the payload destination
* @vid: VLAN identifier
* Return: true if the packet header has been updated, false otherwise
*/
static bool
-batadv_reroute_unicast_packet(struct batadv_priv *bat_priv,
+batadv_reroute_unicast_packet(struct batadv_priv *bat_priv, struct sk_buff *skb,
struct batadv_unicast_packet *unicast_packet,
u8 *dst_addr, unsigned short vid)
{
}
/* update the packet header */
+ skb_postpull_rcsum(skb, unicast_packet, sizeof(*unicast_packet));
ether_addr_copy(unicast_packet->dest, orig_addr);
unicast_packet->ttvn = orig_ttvn;
+ skb_postpush_rcsum(skb, unicast_packet, sizeof(*unicast_packet));
ret = true;
out:
* the packet to
*/
if (batadv_tt_local_client_is_roaming(bat_priv, ethhdr->h_dest, vid)) {
- if (batadv_reroute_unicast_packet(bat_priv, unicast_packet,
+ if (batadv_reroute_unicast_packet(bat_priv, skb, unicast_packet,
ethhdr->h_dest, vid))
batadv_dbg_ratelimited(BATADV_DBG_TT,
bat_priv,
* destination can possibly be updated and forwarded towards the new
* target host
*/
- if (batadv_reroute_unicast_packet(bat_priv, unicast_packet,
+ if (batadv_reroute_unicast_packet(bat_priv, skb, unicast_packet,
ethhdr->h_dest, vid)) {
batadv_dbg_ratelimited(BATADV_DBG_TT, bat_priv,
"Rerouting unicast packet to %pM (dst=%pM): TTVN mismatch old_ttvn=%u new_ttvn=%u\n",
if (!primary_if)
return false;
+ /* update the packet header */
+ skb_postpull_rcsum(skb, unicast_packet, sizeof(*unicast_packet));
ether_addr_copy(unicast_packet->dest, primary_if->net_dev->dev_addr);
+ unicast_packet->ttvn = curr_ttvn;
+ skb_postpush_rcsum(skb, unicast_packet, sizeof(*unicast_packet));
batadv_hardif_put(primary_if);
- unicast_packet->ttvn = curr_ttvn;
-
return true;
}
struct batadv_orig_node *orig_node = NULL, *orig_node_gw = NULL;
int check, hdr_size = sizeof(*unicast_packet);
enum batadv_subtype subtype;
- struct ethhdr *ethhdr;
int ret = NET_RX_DROP;
bool is4addr, is_gw;
unicast_packet = (struct batadv_unicast_packet *)skb->data;
- unicast_4addr_packet = (struct batadv_unicast_4addr_packet *)skb->data;
- ethhdr = eth_hdr(skb);
-
is4addr = unicast_packet->packet_type == BATADV_UNICAST_4ADDR;
/* the caller function should have already pulled 2 bytes */
if (is4addr)
if (!batadv_check_unicast_ttvn(bat_priv, skb, hdr_size))
goto free_skb;
+ unicast_packet = (struct batadv_unicast_packet *)skb->data;
+
/* packet for me */
if (batadv_is_my_mac(bat_priv, unicast_packet->dest)) {
/* If this is a unicast packet from another backgone gw,
* drop it.
*/
- orig_addr_gw = ethhdr->h_source;
+ orig_addr_gw = eth_hdr(skb)->h_source;
orig_node_gw = batadv_orig_hash_find(bat_priv, orig_addr_gw);
if (orig_node_gw) {
is_gw = batadv_bla_is_backbone_gw(skb, orig_node_gw,
}
if (is4addr) {
+ unicast_4addr_packet =
+ (struct batadv_unicast_4addr_packet *)skb->data;
subtype = unicast_4addr_packet->subtype;
batadv_dat_inc_counter(bat_priv, subtype);
return sprintf(buf, "%d\n", dev->channel);
}
-static DEVICE_ATTR(address, S_IRUGO, show_address, NULL);
-static DEVICE_ATTR(channel, S_IRUGO, show_channel, NULL);
+static DEVICE_ATTR(address, 0444, show_address, NULL);
+static DEVICE_ATTR(channel, 0444, show_channel, NULL);
static struct rfcomm_dev *__rfcomm_dev_add(struct rfcomm_dev_req *req,
struct rfcomm_dlc *dlc)
else
sec_level = authreq_to_seclevel(auth);
- if (smp_sufficient_security(hcon, sec_level, SMP_USE_LTK))
+ if (smp_sufficient_security(hcon, sec_level, SMP_USE_LTK)) {
+ /* If link is already encrypted with sufficient security we
+ * still need refresh encryption as per Core Spec 5.0 Vol 3,
+ * Part H 2.4.6
+ */
+ smp_ltk_encrypt(conn, hcon->sec_level);
return 0;
+ }
if (sec_level > hcon->pending_sec_level)
hcon->pending_sec_level = sec_level;
switch (event) {
case NETDEV_CHANGEMTU:
- dev_set_mtu(br->dev, br_min_mtu(br));
+ dev_set_mtu(br->dev, br_mtu(br));
break;
case NETDEV_CHANGEADDR:
static struct pernet_operations br_net_ops = {
.exit = br_net_exit,
- .async = true,
};
static const struct stp_proto br_stp_proto = {
static int br_change_mtu(struct net_device *dev, int new_mtu)
{
struct net_bridge *br = netdev_priv(dev);
- if (new_mtu > br_min_mtu(br))
+ if (new_mtu > br_mtu(br))
return -EINVAL;
dev->mtu = new_mtu;
return ret;
}
+static bool min_mtu(int a, int b)
+{
+ return a < b ? 1 : 0;
+}
+
+static bool max_mtu(int a, int b)
+{
+ return a > b ? 1 : 0;
+}
+
/* MTU of the bridge pseudo-device: ETH_DATA_LEN or the minimum of the ports */
-int br_min_mtu(const struct net_bridge *br)
+static int __br_mtu(const struct net_bridge *br, bool (compare_fn)(int, int))
{
const struct net_bridge_port *p;
int mtu = 0;
mtu = ETH_DATA_LEN;
else {
list_for_each_entry(p, &br->port_list, list) {
- if (!mtu || p->dev->mtu < mtu)
+ if (!mtu || compare_fn(p->dev->mtu, mtu))
mtu = p->dev->mtu;
}
}
return mtu;
}
+int br_mtu(const struct net_bridge *br)
+{
+ if (br_vlan_enabled(br->dev))
+ return __br_mtu(br, max_mtu);
+ else
+ return __br_mtu(br, min_mtu);
+}
+
static void br_set_gso_limits(struct net_bridge *br)
{
unsigned int gso_max_size = GSO_MAX_SIZE;
if (changed_addr)
call_netdevice_notifiers(NETDEV_CHANGEADDR, br->dev);
- dev_set_mtu(br->dev, br_min_mtu(br));
+ dev_set_mtu(br->dev, br_mtu(br));
br_set_gso_limits(br);
kobject_uevent(&p->kobj, KOBJ_ADD);
*/
del_nbp(p);
- dev_set_mtu(br->dev, br_min_mtu(br));
+ dev_set_mtu(br->dev, br_mtu(br));
br_set_gso_limits(br);
spin_lock_bh(&br->lock);
.exit = brnf_exit_net,
.id = &brnf_net_id,
.size = sizeof(struct brnf_net),
- .async = true,
};
static struct notifier_block brnf_notifier __read_mostly = {
int br_add_if(struct net_bridge *br, struct net_device *dev,
struct netlink_ext_ack *extack);
int br_del_if(struct net_bridge *br, struct net_device *dev);
-int br_min_mtu(const struct net_bridge *br);
+int br_mtu(const struct net_bridge *br);
netdev_features_t br_features_recompute(struct net_bridge *br,
netdev_features_t features);
void br_port_flags_change(struct net_bridge_port *port, unsigned long mask);
static struct bin_attribute bridge_forward = {
.attr = { .name = SYSFS_BRIDGE_FDB,
- .mode = S_IRUGO, },
+ .mode = 0444, },
.read = brforward_read,
};
{ \
return store_flag(p, v, _mask); \
} \
-static BRPORT_ATTR(_name, S_IRUGO | S_IWUSR, \
+static BRPORT_ATTR(_name, 0644, \
show_##_name, store_##_name)
static int store_flag(struct net_bridge_port *p, unsigned long v,
return sprintf(buf, "%d\n", p->path_cost);
}
-static BRPORT_ATTR(path_cost, S_IRUGO | S_IWUSR,
+static BRPORT_ATTR(path_cost, 0644,
show_path_cost, br_stp_set_path_cost);
static ssize_t show_priority(struct net_bridge_port *p, char *buf)
return sprintf(buf, "%d\n", p->priority);
}
-static BRPORT_ATTR(priority, S_IRUGO | S_IWUSR,
+static BRPORT_ATTR(priority, 0644,
show_priority, br_stp_set_port_priority);
static ssize_t show_designated_root(struct net_bridge_port *p, char *buf)
{
return br_show_bridge_id(buf, &p->designated_root);
}
-static BRPORT_ATTR(designated_root, S_IRUGO, show_designated_root, NULL);
+static BRPORT_ATTR(designated_root, 0444, show_designated_root, NULL);
static ssize_t show_designated_bridge(struct net_bridge_port *p, char *buf)
{
return br_show_bridge_id(buf, &p->designated_bridge);
}
-static BRPORT_ATTR(designated_bridge, S_IRUGO, show_designated_bridge, NULL);
+static BRPORT_ATTR(designated_bridge, 0444, show_designated_bridge, NULL);
static ssize_t show_designated_port(struct net_bridge_port *p, char *buf)
{
return sprintf(buf, "%d\n", p->designated_port);
}
-static BRPORT_ATTR(designated_port, S_IRUGO, show_designated_port, NULL);
+static BRPORT_ATTR(designated_port, 0444, show_designated_port, NULL);
static ssize_t show_designated_cost(struct net_bridge_port *p, char *buf)
{
return sprintf(buf, "%d\n", p->designated_cost);
}
-static BRPORT_ATTR(designated_cost, S_IRUGO, show_designated_cost, NULL);
+static BRPORT_ATTR(designated_cost, 0444, show_designated_cost, NULL);
static ssize_t show_port_id(struct net_bridge_port *p, char *buf)
{
return sprintf(buf, "0x%x\n", p->port_id);
}
-static BRPORT_ATTR(port_id, S_IRUGO, show_port_id, NULL);
+static BRPORT_ATTR(port_id, 0444, show_port_id, NULL);
static ssize_t show_port_no(struct net_bridge_port *p, char *buf)
{
return sprintf(buf, "0x%x\n", p->port_no);
}
-static BRPORT_ATTR(port_no, S_IRUGO, show_port_no, NULL);
+static BRPORT_ATTR(port_no, 0444, show_port_no, NULL);
static ssize_t show_change_ack(struct net_bridge_port *p, char *buf)
{
return sprintf(buf, "%d\n", p->topology_change_ack);
}
-static BRPORT_ATTR(change_ack, S_IRUGO, show_change_ack, NULL);
+static BRPORT_ATTR(change_ack, 0444, show_change_ack, NULL);
static ssize_t show_config_pending(struct net_bridge_port *p, char *buf)
{
return sprintf(buf, "%d\n", p->config_pending);
}
-static BRPORT_ATTR(config_pending, S_IRUGO, show_config_pending, NULL);
+static BRPORT_ATTR(config_pending, 0444, show_config_pending, NULL);
static ssize_t show_port_state(struct net_bridge_port *p, char *buf)
{
return sprintf(buf, "%d\n", p->state);
}
-static BRPORT_ATTR(state, S_IRUGO, show_port_state, NULL);
+static BRPORT_ATTR(state, 0444, show_port_state, NULL);
static ssize_t show_message_age_timer(struct net_bridge_port *p,
char *buf)
{
return sprintf(buf, "%ld\n", br_timer_value(&p->message_age_timer));
}
-static BRPORT_ATTR(message_age_timer, S_IRUGO, show_message_age_timer, NULL);
+static BRPORT_ATTR(message_age_timer, 0444, show_message_age_timer, NULL);
static ssize_t show_forward_delay_timer(struct net_bridge_port *p,
char *buf)
{
return sprintf(buf, "%ld\n", br_timer_value(&p->forward_delay_timer));
}
-static BRPORT_ATTR(forward_delay_timer, S_IRUGO, show_forward_delay_timer, NULL);
+static BRPORT_ATTR(forward_delay_timer, 0444, show_forward_delay_timer, NULL);
static ssize_t show_hold_timer(struct net_bridge_port *p,
char *buf)
{
return sprintf(buf, "%ld\n", br_timer_value(&p->hold_timer));
}
-static BRPORT_ATTR(hold_timer, S_IRUGO, show_hold_timer, NULL);
+static BRPORT_ATTR(hold_timer, 0444, show_hold_timer, NULL);
static int store_flush(struct net_bridge_port *p, unsigned long v)
{
br_fdb_delete_by_port(p->br, p, 0, 0); // Don't delete local entry
return 0;
}
-static BRPORT_ATTR(flush, S_IWUSR, NULL, store_flush);
+static BRPORT_ATTR(flush, 0200, NULL, store_flush);
static ssize_t show_group_fwd_mask(struct net_bridge_port *p, char *buf)
{
return 0;
}
-static BRPORT_ATTR(group_fwd_mask, S_IRUGO | S_IWUSR, show_group_fwd_mask,
+static BRPORT_ATTR(group_fwd_mask, 0644, show_group_fwd_mask,
store_group_fwd_mask);
BRPORT_ATTR_FLAG(hairpin_mode, BR_HAIRPIN_MODE);
{
return br_multicast_set_port_router(p, v);
}
-static BRPORT_ATTR(multicast_router, S_IRUGO | S_IWUSR, show_multicast_router,
+static BRPORT_ATTR(multicast_router, 0644, show_multicast_router,
store_multicast_router);
BRPORT_ATTR_FLAG(multicast_fast_leave, BR_MULTICAST_FAST_LEAVE);
return w && w->poolsize >= (INT_MAX / sizeof(struct ebt_mac_wormhash_tuple));
}
+static bool wormhash_offset_invalid(int off, unsigned int len)
+{
+ if (off == 0) /* not present */
+ return false;
+
+ if (off < (int)sizeof(struct ebt_among_info) ||
+ off % __alignof__(struct ebt_mac_wormhash))
+ return true;
+
+ off += sizeof(struct ebt_mac_wormhash);
+
+ return off > len;
+}
+
+static bool wormhash_sizes_valid(const struct ebt_mac_wormhash *wh, int a, int b)
+{
+ if (a == 0)
+ a = sizeof(struct ebt_among_info);
+
+ return ebt_mac_wormhash_size(wh) + a == b;
+}
+
static int ebt_among_mt_check(const struct xt_mtchk_param *par)
{
const struct ebt_among_info *info = par->matchinfo;
if (expected_length > em->match_size)
return -EINVAL;
+ if (wormhash_offset_invalid(info->wh_dst_ofs, em->match_size) ||
+ wormhash_offset_invalid(info->wh_src_ofs, em->match_size))
+ return -EINVAL;
+
wh_dst = ebt_among_wh_dst(info);
if (poolsize_invalid(wh_dst))
return -EINVAL;
if (poolsize_invalid(wh_src))
return -EINVAL;
+ if (info->wh_src_ofs < info->wh_dst_ofs) {
+ if (!wormhash_sizes_valid(wh_src, info->wh_src_ofs, info->wh_dst_ofs))
+ return -EINVAL;
+ } else {
+ if (!wormhash_sizes_valid(wh_dst, info->wh_dst_ofs, info->wh_src_ofs))
+ return -EINVAL;
+ }
+
expected_length += ebt_mac_wormhash_size(wh_src);
if (em->match_size != EBT_ALIGN(expected_length)) {
static struct pernet_operations broute_net_ops = {
.init = broute_net_init,
.exit = broute_net_exit,
- .async = true,
};
static int __init ebtable_broute_init(void)
static struct pernet_operations frame_filter_net_ops = {
.init = frame_filter_net_init,
.exit = frame_filter_net_exit,
- .async = true,
};
static int __init ebtable_filter_init(void)
static struct pernet_operations frame_nat_net_ops = {
.init = frame_nat_net_init,
.exit = frame_nat_net_exit,
- .async = true,
};
static int __init ebtable_nat_init(void)
* offsets are relative to beginning of struct ebt_entry (i.e., 0).
*/
for (i = 0; i < 4 ; ++i) {
- if (offsets[i] >= *total)
+ if (offsets[i] > *total)
return -EINVAL;
+
+ if (i < 3 && offsets[i] == *total)
+ return -EINVAL;
+
if (i == 0)
continue;
if (offsets[i-1] > offsets[i])
static struct pernet_operations nf_log_bridge_net_ops = {
.init = nf_log_bridge_net_init,
.exit = nf_log_bridge_net_exit,
- .async = true,
};
static int __init nf_log_bridge_init(void)
.exit = caif_exit_net,
.id = &caif_net_id,
.size = sizeof(struct caif_net),
- .async = true,
};
/* Initialize Caif devices list */
MODULE_ALIAS_NETPROTO(PF_CAN);
static int stats_timer __read_mostly = 1;
-module_param(stats_timer, int, S_IRUGO);
+module_param(stats_timer, int, 0444);
MODULE_PARM_DESC(stats_timer, "enable timer for statistics (default:on)");
static struct kmem_cache *rcv_cache __read_mostly;
static struct pernet_operations canbcm_pernet_ops __read_mostly = {
.init = canbcm_pernet_init,
.exit = canbcm_pernet_exit,
- .async = true,
};
static int __init bcm_module_init(void)
#define CGW_DEFAULT_HOPS 1
static unsigned int max_hops __read_mostly = CGW_DEFAULT_HOPS;
-module_param(max_hops, uint, S_IRUGO);
+module_param(max_hops, uint, 0444);
MODULE_PARM_DESC(max_hops,
"maximum " CAN_GW_NAME " routing hops for CAN frames "
"(valid values: " __stringify(CGW_MIN_HOPS) "-"
static struct pernet_operations cangw_pernet_ops = {
.init = cangw_pernet_init,
.exit = cangw_pernet_exit,
- .async = true,
};
static __init int cgw_module_init(void)
.get = param_get_supported_features,
};
module_param_cb(supported_features, ¶m_ops_supported_features, NULL,
- S_IRUGO);
+ 0444);
const char *ceph_msg_type_name(int type)
{
netdev_WARN(dev, "failed to disable GRO_HW!\n");
}
+const char *netdev_cmd_to_name(enum netdev_cmd cmd)
+{
+#define N(val) \
+ case NETDEV_##val: \
+ return "NETDEV_" __stringify(val);
+ switch (cmd) {
+ N(UP) N(DOWN) N(REBOOT) N(CHANGE) N(REGISTER) N(UNREGISTER)
+ N(CHANGEMTU) N(CHANGEADDR) N(GOING_DOWN) N(CHANGENAME) N(FEAT_CHANGE)
+ N(BONDING_FAILOVER) N(PRE_UP) N(PRE_TYPE_CHANGE) N(POST_TYPE_CHANGE)
+ N(POST_INIT) N(RELEASE) N(NOTIFY_PEERS) N(JOIN) N(CHANGEUPPER)
+ N(RESEND_IGMP) N(PRECHANGEMTU) N(CHANGEINFODATA) N(BONDING_INFO)
+ N(PRECHANGEUPPER) N(CHANGELOWERSTATE) N(UDP_TUNNEL_PUSH_INFO)
+ N(UDP_TUNNEL_DROP_INFO) N(CHANGE_TX_QUEUE_LEN)
+ };
+#undef N
+ return "UNKNOWN_NETDEV_EVENT";
+}
+EXPORT_SYMBOL_GPL(netdev_cmd_to_name);
+
static int call_netdevice_notifier(struct notifier_block *nb, unsigned long val,
struct net_device *dev)
{
goto unlock;
if (dev_boot_phase)
goto unlock;
+ down_read(&net_rwsem);
for_each_net(net) {
for_each_netdev(net, dev) {
err = call_netdevice_notifier(nb, NETDEV_REGISTER, dev);
call_netdevice_notifier(nb, NETDEV_UP, dev);
}
}
+ up_read(&net_rwsem);
unlock:
rtnl_unlock();
}
outroll:
+ up_read(&net_rwsem);
raw_notifier_chain_unregister(&netdev_chain, nb);
goto unlock;
}
if (err)
goto unlock;
+ down_read(&net_rwsem);
for_each_net(net) {
for_each_netdev(net, dev) {
if (dev->flags & IFF_UP) {
call_netdevice_notifier(nb, NETDEV_UNREGISTER, dev);
}
}
+ up_read(&net_rwsem);
unlock:
rtnl_unlock();
return err;
#if IS_ENABLED(CONFIG_CGROUP_NET_PRIO)
static void skb_update_prio(struct sk_buff *skb)
{
- struct netprio_map *map = rcu_dereference_bh(skb->dev->priomap);
+ const struct netprio_map *map;
+ const struct sock *sk;
+ unsigned int prioidx;
+
+ if (skb->priority)
+ return;
+ map = rcu_dereference_bh(skb->dev->priomap);
+ if (!map)
+ return;
+ sk = skb_to_full_sk(skb);
+ if (!sk)
+ return;
- if (!skb->priority && skb->sk && map) {
- unsigned int prioidx =
- sock_cgroup_prioidx(&skb->sk->sk_cgrp_data);
+ prioidx = sock_cgroup_prioidx(&sk->sk_cgrp_data);
- if (prioidx < map->priomap_len)
- skb->priority = map->priomap[prioidx];
- }
+ if (prioidx < map->priomap_len)
+ skb->priority = map->priomap[prioidx];
}
#else
#define skb_update_prio(skb)
rcu_barrier();
rtnl_lock();
- call_netdevice_notifiers(NETDEV_UNREGISTER_FINAL, dev);
if (test_bit(__LINK_STATE_LINKWATCH_PENDING,
&dev->state)) {
/* We must not have linkwatch events
= list_first_entry(&list, struct net_device, todo_list);
list_del(&dev->todo_list);
- rtnl_lock();
- call_netdevice_notifiers(NETDEV_UNREGISTER_FINAL, dev);
- __rtnl_unlock();
-
if (unlikely(dev->reg_state != NETREG_UNREGISTERING)) {
pr_err("network todo '%s' but state %d\n",
dev->name, dev->reg_state);
*/
call_netdevice_notifiers(NETDEV_UNREGISTER, dev);
rcu_barrier();
- call_netdevice_notifiers(NETDEV_UNREGISTER_FINAL, dev);
new_nsid = peernet2id_alloc(dev_net(dev), net);
/* If there is an ifindex conflict assign a new one */
static struct pernet_operations __net_initdata netdev_net_ops = {
.init = netdev_init,
.exit = netdev_exit,
- .async = true,
};
static void __net_exit default_device_exit(struct net *net)
static struct pernet_operations __net_initdata default_device_ops = {
.exit = default_device_exit,
.exit_batch = default_device_exit_batch,
- .async = true,
};
/*
if (colon)
*colon = 0;
- dev_load(net, ifr->ifr_name);
-
/*
* See which interface the caller is talking about.
*/
case SIOCGIFMAP:
case SIOCGIFINDEX:
case SIOCGIFTXQLEN:
+ dev_load(net, ifr->ifr_name);
rcu_read_lock();
ret = dev_ifsioc_locked(net, ifr, cmd);
rcu_read_unlock();
return ret;
case SIOCETHTOOL:
+ dev_load(net, ifr->ifr_name);
rtnl_lock();
ret = dev_ethtool(net, ifr);
rtnl_unlock();
case SIOCGMIIPHY:
case SIOCGMIIREG:
case SIOCSIFNAME:
+ dev_load(net, ifr->ifr_name);
if (!ns_capable(net->user_ns, CAP_NET_ADMIN))
return -EPERM;
rtnl_lock();
/* fall through */
case SIOCBONDSLAVEINFOQUERY:
case SIOCBONDINFOQUERY:
+ dev_load(net, ifr->ifr_name);
rtnl_lock();
ret = dev_ifsioc(net, ifr, cmd);
rtnl_unlock();
cmd == SIOCGHWTSTAMP ||
(cmd >= SIOCDEVPRIVATE &&
cmd <= SIOCDEVPRIVATE + 15)) {
+ dev_load(net, ifr->ifr_name);
rtnl_lock();
ret = dev_ifsioc(net, ifr, cmd);
rtnl_unlock();
if (!nlh) {
err = devlink_dpipe_send_and_alloc_skb(&skb, info);
if (err)
- goto err_skb_send_alloc;
+ return err;
goto send_done;
}
nla_put_failure:
err = -EMSGSIZE;
err_table_put:
-err_skb_send_alloc:
genlmsg_cancel(skb, hdr);
nlmsg_free(skb);
return err;
table->counters_enabled,
&dump_ctx);
if (err)
- goto err_entries_dump;
+ return err;
send_done:
nlh = nlmsg_put(dump_ctx.skb, info->snd_portid, info->snd_seq,
if (!nlh) {
err = devlink_dpipe_send_and_alloc_skb(&dump_ctx.skb, info);
if (err)
- goto err_skb_send_alloc;
+ return err;
goto send_done;
}
return genlmsg_reply(dump_ctx.skb, info);
-
-err_entries_dump:
-err_skb_send_alloc:
- genlmsg_cancel(dump_ctx.skb, dump_ctx.hdr);
- nlmsg_free(dump_ctx.skb);
- return err;
}
static int devlink_nl_cmd_dpipe_entries_get(struct sk_buff *skb,
if (!nlh) {
err = devlink_dpipe_send_and_alloc_skb(&skb, info);
if (err)
- goto err_skb_send_alloc;
+ return err;
goto send_done;
}
return genlmsg_reply(skb, info);
nla_put_failure:
err = -EMSGSIZE;
err_table_put:
-err_skb_send_alloc:
genlmsg_cancel(skb, hdr);
nlmsg_free(skb);
return err;
*/
int devlink_resource_register(struct devlink *devlink,
const char *resource_name,
- bool top_hierarchy,
u64 resource_size,
u64 resource_id,
u64 parent_resource_id,
{
struct devlink_resource *resource;
struct list_head *resource_list;
+ bool top_hierarchy;
int err = 0;
+ top_hierarchy = parent_resource_id == DEVLINK_RESOURCE_ID_PARENT_TOP;
+
mutex_lock(&devlink->lock);
resource = devlink_resource_find(devlink, NULL, resource_id);
if (resource) {
[ETHTOOL_ID_UNSPEC] = "Unspec",
[ETHTOOL_RX_COPYBREAK] = "rx-copybreak",
[ETHTOOL_TX_COPYBREAK] = "tx-copybreak",
+ [ETHTOOL_PFC_PREVENTION_TOUT] = "pfc-prevention-tout",
};
static const char
tuna->type_id != ETHTOOL_TUNABLE_U32)
return -EINVAL;
break;
+ case ETHTOOL_PFC_PREVENTION_TOUT:
+ if (tuna->len != sizeof(u16) ||
+ tuna->type_id != ETHTOOL_TUNABLE_U16)
+ return -EINVAL;
+ break;
default:
return -EINVAL;
}
enum fib_event_type event_type,
struct fib_notifier_info *info)
{
+ int err;
+
info->net = net;
- return nb->notifier_call(nb, event_type, info);
+ err = nb->notifier_call(nb, event_type, info);
+ return notifier_to_errno(err);
}
EXPORT_SYMBOL(call_fib_notifier);
int call_fib_notifiers(struct net *net, enum fib_event_type event_type,
struct fib_notifier_info *info)
{
+ int err;
+
info->net = net;
- return atomic_notifier_call_chain(&fib_chain, event_type, info);
+ err = atomic_notifier_call_chain(&fib_chain, event_type, info);
+ return notifier_to_errno(err);
}
EXPORT_SYMBOL(call_fib_notifiers);
struct net *net;
rtnl_lock();
+ down_read(&net_rwsem);
for_each_net(net) {
rcu_read_lock();
list_for_each_entry_rcu(ops, &net->fib_notifier_ops, list) {
}
rcu_read_unlock();
}
+ up_read(&net_rwsem);
rtnl_unlock();
return fib_seq;
static struct pernet_operations fib_notifier_net_ops = {
.init = fib_notifier_net_init,
.exit = fib_notifier_net_exit,
- .async = true,
};
static int __init fib_notifier_init(void)
if (err < 0)
goto errout_free;
+ err = call_fib_rule_notifiers(net, FIB_EVENT_RULE_ADD, rule, ops,
+ extack);
+ if (err < 0)
+ goto errout_free;
+
list_for_each_entry(r, &ops->rules_list, list) {
if (r->pref > rule->pref)
break;
if (rule->tun_id)
ip_tunnel_need_metadata();
- call_fib_rule_notifiers(net, FIB_EVENT_RULE_ADD, rule, ops, extack);
notify_rule_change(RTM_NEWRULE, rule, ops, nlh, NETLINK_CB(skb).portid);
flush_route_cache(ops);
rules_ops_put(ops);
static struct pernet_operations fib_rules_net_ops = {
.init = fib_rules_net_init,
.exit = fib_rules_net_exit,
- .async = true,
};
static int __init fib_rules_init(void)
.arg4_type = ARG_ANYTHING,
};
+BPF_CALL_4(bpf_msg_redirect_map, struct sk_msg_buff *, msg,
+ struct bpf_map *, map, u32, key, u64, flags)
+{
+ /* If user passes invalid input drop the packet. */
+ if (unlikely(flags))
+ return SK_DROP;
+
+ msg->key = key;
+ msg->flags = flags;
+ msg->map = map;
+
+ return SK_PASS;
+}
+
+struct sock *do_msg_redirect_map(struct sk_msg_buff *msg)
+{
+ struct sock *sk = NULL;
+
+ if (msg->map) {
+ sk = __sock_map_lookup_elem(msg->map, msg->key);
+
+ msg->key = 0;
+ msg->map = NULL;
+ }
+
+ return sk;
+}
+
+static const struct bpf_func_proto bpf_msg_redirect_map_proto = {
+ .func = bpf_msg_redirect_map,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_CONST_MAP_PTR,
+ .arg3_type = ARG_ANYTHING,
+ .arg4_type = ARG_ANYTHING,
+};
+
+BPF_CALL_2(bpf_msg_apply_bytes, struct sk_msg_buff *, msg, u32, bytes)
+{
+ msg->apply_bytes = bytes;
+ return 0;
+}
+
+static const struct bpf_func_proto bpf_msg_apply_bytes_proto = {
+ .func = bpf_msg_apply_bytes,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_ANYTHING,
+};
+
+BPF_CALL_2(bpf_msg_cork_bytes, struct sk_msg_buff *, msg, u32, bytes)
+{
+ msg->cork_bytes = bytes;
+ return 0;
+}
+
+static const struct bpf_func_proto bpf_msg_cork_bytes_proto = {
+ .func = bpf_msg_cork_bytes,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_ANYTHING,
+};
+
+BPF_CALL_4(bpf_msg_pull_data,
+ struct sk_msg_buff *, msg, u32, start, u32, end, u64, flags)
+{
+ unsigned int len = 0, offset = 0, copy = 0;
+ struct scatterlist *sg = msg->sg_data;
+ int first_sg, last_sg, i, shift;
+ unsigned char *p, *to, *from;
+ int bytes = end - start;
+ struct page *page;
+
+ if (unlikely(flags || end <= start))
+ return -EINVAL;
+
+ /* First find the starting scatterlist element */
+ i = msg->sg_start;
+ do {
+ len = sg[i].length;
+ offset += len;
+ if (start < offset + len)
+ break;
+ i++;
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ } while (i != msg->sg_end);
+
+ if (unlikely(start >= offset + len))
+ return -EINVAL;
+
+ if (!msg->sg_copy[i] && bytes <= len)
+ goto out;
+
+ first_sg = i;
+
+ /* At this point we need to linearize multiple scatterlist
+ * elements or a single shared page. Either way we need to
+ * copy into a linear buffer exclusively owned by BPF. Then
+ * place the buffer in the scatterlist and fixup the original
+ * entries by removing the entries now in the linear buffer
+ * and shifting the remaining entries. For now we do not try
+ * to copy partial entries to avoid complexity of running out
+ * of sg_entry slots. The downside is reading a single byte
+ * will copy the entire sg entry.
+ */
+ do {
+ copy += sg[i].length;
+ i++;
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ if (bytes < copy)
+ break;
+ } while (i != msg->sg_end);
+ last_sg = i;
+
+ if (unlikely(copy < end - start))
+ return -EINVAL;
+
+ page = alloc_pages(__GFP_NOWARN | GFP_ATOMIC, get_order(copy));
+ if (unlikely(!page))
+ return -ENOMEM;
+ p = page_address(page);
+ offset = 0;
+
+ i = first_sg;
+ do {
+ from = sg_virt(&sg[i]);
+ len = sg[i].length;
+ to = p + offset;
+
+ memcpy(to, from, len);
+ offset += len;
+ sg[i].length = 0;
+ put_page(sg_page(&sg[i]));
+
+ i++;
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ } while (i != last_sg);
+
+ sg[first_sg].length = copy;
+ sg_set_page(&sg[first_sg], page, copy, 0);
+
+ /* To repair sg ring we need to shift entries. If we only
+ * had a single entry though we can just replace it and
+ * be done. Otherwise walk the ring and shift the entries.
+ */
+ shift = last_sg - first_sg - 1;
+ if (!shift)
+ goto out;
+
+ i = first_sg + 1;
+ do {
+ int move_from;
+
+ if (i + shift >= MAX_SKB_FRAGS)
+ move_from = i + shift - MAX_SKB_FRAGS;
+ else
+ move_from = i + shift;
+
+ if (move_from == msg->sg_end)
+ break;
+
+ sg[i] = sg[move_from];
+ sg[move_from].length = 0;
+ sg[move_from].page_link = 0;
+ sg[move_from].offset = 0;
+
+ i++;
+ if (i == MAX_SKB_FRAGS)
+ i = 0;
+ } while (1);
+ msg->sg_end -= shift;
+ if (msg->sg_end < 0)
+ msg->sg_end += MAX_SKB_FRAGS;
+out:
+ msg->data = sg_virt(&sg[i]) + start - offset;
+ msg->data_end = msg->data + bytes;
+
+ return 0;
+}
+
+static const struct bpf_func_proto bpf_msg_pull_data_proto = {
+ .func = bpf_msg_pull_data,
+ .gpl_only = false,
+ .ret_type = RET_INTEGER,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_ANYTHING,
+ .arg3_type = ARG_ANYTHING,
+ .arg4_type = ARG_ANYTHING,
+};
+
BPF_CALL_1(bpf_get_cgroup_classid, const struct sk_buff *, skb)
{
return task_get_classid(skb);
u32 off = skb_mac_header_len(skb);
int ret;
+ /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */
+ if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb)))
+ return -ENOTSUPP;
+
ret = skb_cow(skb, len_diff);
if (unlikely(ret < 0))
return ret;
return ret;
if (skb_is_gso(skb)) {
+ struct skb_shared_info *shinfo = skb_shinfo(skb);
+
/* SKB_GSO_TCPV4 needs to be changed into
* SKB_GSO_TCPV6.
*/
- if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4) {
- skb_shinfo(skb)->gso_type &= ~SKB_GSO_TCPV4;
- skb_shinfo(skb)->gso_type |= SKB_GSO_TCPV6;
+ if (shinfo->gso_type & SKB_GSO_TCPV4) {
+ shinfo->gso_type &= ~SKB_GSO_TCPV4;
+ shinfo->gso_type |= SKB_GSO_TCPV6;
}
/* Due to IPv6 header, MSS needs to be downgraded. */
- skb_shinfo(skb)->gso_size -= len_diff;
+ skb_decrease_gso_size(shinfo, len_diff);
/* Header must be checked, and gso_segs recomputed. */
- skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY;
- skb_shinfo(skb)->gso_segs = 0;
+ shinfo->gso_type |= SKB_GSO_DODGY;
+ shinfo->gso_segs = 0;
}
skb->protocol = htons(ETH_P_IPV6);
u32 off = skb_mac_header_len(skb);
int ret;
+ /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */
+ if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb)))
+ return -ENOTSUPP;
+
ret = skb_unclone(skb, GFP_ATOMIC);
if (unlikely(ret < 0))
return ret;
return ret;
if (skb_is_gso(skb)) {
+ struct skb_shared_info *shinfo = skb_shinfo(skb);
+
/* SKB_GSO_TCPV6 needs to be changed into
* SKB_GSO_TCPV4.
*/
- if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV6) {
- skb_shinfo(skb)->gso_type &= ~SKB_GSO_TCPV6;
- skb_shinfo(skb)->gso_type |= SKB_GSO_TCPV4;
+ if (shinfo->gso_type & SKB_GSO_TCPV6) {
+ shinfo->gso_type &= ~SKB_GSO_TCPV6;
+ shinfo->gso_type |= SKB_GSO_TCPV4;
}
/* Due to IPv4 header, MSS can be upgraded. */
- skb_shinfo(skb)->gso_size += len_diff;
+ skb_increase_gso_size(shinfo, len_diff);
/* Header must be checked, and gso_segs recomputed. */
- skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY;
- skb_shinfo(skb)->gso_segs = 0;
+ shinfo->gso_type |= SKB_GSO_DODGY;
+ shinfo->gso_segs = 0;
}
skb->protocol = htons(ETH_P_IP);
u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb);
int ret;
+ /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */
+ if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb)))
+ return -ENOTSUPP;
+
ret = skb_cow(skb, len_diff);
if (unlikely(ret < 0))
return ret;
return ret;
if (skb_is_gso(skb)) {
+ struct skb_shared_info *shinfo = skb_shinfo(skb);
+
/* Due to header grow, MSS needs to be downgraded. */
- skb_shinfo(skb)->gso_size -= len_diff;
+ skb_decrease_gso_size(shinfo, len_diff);
/* Header must be checked, and gso_segs recomputed. */
- skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY;
- skb_shinfo(skb)->gso_segs = 0;
+ shinfo->gso_type |= SKB_GSO_DODGY;
+ shinfo->gso_segs = 0;
}
return 0;
u32 off = skb_mac_header_len(skb) + bpf_skb_net_base_len(skb);
int ret;
+ /* SCTP uses GSO_BY_FRAGS, thus cannot adjust it. */
+ if (skb_is_gso(skb) && unlikely(skb_is_gso_sctp(skb)))
+ return -ENOTSUPP;
+
ret = skb_unclone(skb, GFP_ATOMIC);
if (unlikely(ret < 0))
return ret;
return ret;
if (skb_is_gso(skb)) {
+ struct skb_shared_info *shinfo = skb_shinfo(skb);
+
/* Due to header shrink, MSS can be upgraded. */
- skb_shinfo(skb)->gso_size += len_diff;
+ skb_increase_gso_size(shinfo, len_diff);
/* Header must be checked, and gso_segs recomputed. */
- skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY;
- skb_shinfo(skb)->gso_segs = 0;
+ shinfo->gso_type |= SKB_GSO_DODGY;
+ shinfo->gso_segs = 0;
}
return 0;
func == bpf_l3_csum_replace ||
func == bpf_l4_csum_replace ||
func == bpf_xdp_adjust_head ||
- func == bpf_xdp_adjust_meta)
+ func == bpf_xdp_adjust_meta ||
+ func == bpf_msg_pull_data)
return true;
return false;
}
}
+static const struct bpf_func_proto *sk_msg_func_proto(enum bpf_func_id func_id)
+{
+ switch (func_id) {
+ case BPF_FUNC_msg_redirect_map:
+ return &bpf_msg_redirect_map_proto;
+ case BPF_FUNC_msg_apply_bytes:
+ return &bpf_msg_apply_bytes_proto;
+ case BPF_FUNC_msg_cork_bytes:
+ return &bpf_msg_cork_bytes_proto;
+ case BPF_FUNC_msg_pull_data:
+ return &bpf_msg_pull_data_proto;
+ default:
+ return bpf_base_func_proto(func_id);
+ }
+}
+
static const struct bpf_func_proto *sk_skb_func_proto(enum bpf_func_id func_id)
{
switch (func_id) {
return bpf_skb_is_valid_access(off, size, type, info);
}
+static bool sk_msg_is_valid_access(int off, int size,
+ enum bpf_access_type type,
+ struct bpf_insn_access_aux *info)
+{
+ if (type == BPF_WRITE)
+ return false;
+
+ switch (off) {
+ case offsetof(struct sk_msg_md, data):
+ info->reg_type = PTR_TO_PACKET;
+ break;
+ case offsetof(struct sk_msg_md, data_end):
+ info->reg_type = PTR_TO_PACKET_END;
+ break;
+ }
+
+ if (off < 0 || off >= sizeof(struct sk_msg_md))
+ return false;
+ if (off % size != 0)
+ return false;
+ if (size != sizeof(__u64))
+ return false;
+
+ return true;
+}
+
static u32 bpf_convert_ctx_access(enum bpf_access_type type,
const struct bpf_insn *si,
struct bpf_insn *insn_buf,
return insn - insn_buf;
}
+static u32 sk_msg_convert_ctx_access(enum bpf_access_type type,
+ const struct bpf_insn *si,
+ struct bpf_insn *insn_buf,
+ struct bpf_prog *prog, u32 *target_size)
+{
+ struct bpf_insn *insn = insn_buf;
+
+ switch (si->off) {
+ case offsetof(struct sk_msg_md, data):
+ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct sk_msg_buff, data),
+ si->dst_reg, si->src_reg,
+ offsetof(struct sk_msg_buff, data));
+ break;
+ case offsetof(struct sk_msg_md, data_end):
+ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF(struct sk_msg_buff, data_end),
+ si->dst_reg, si->src_reg,
+ offsetof(struct sk_msg_buff, data_end));
+ break;
+ }
+
+ return insn - insn_buf;
+}
+
const struct bpf_verifier_ops sk_filter_verifier_ops = {
.get_func_proto = sk_filter_func_proto,
.is_valid_access = sk_filter_is_valid_access,
const struct bpf_prog_ops sk_skb_prog_ops = {
};
+const struct bpf_verifier_ops sk_msg_verifier_ops = {
+ .get_func_proto = sk_msg_func_proto,
+ .is_valid_access = sk_msg_is_valid_access,
+ .convert_ctx_access = sk_msg_convert_ctx_access,
+};
+
+const struct bpf_prog_ops sk_msg_prog_ops = {
+};
+
int sk_detach_filter(struct sock *sk)
{
int ret = -ENOENT;
{
int rc = -ENOMEM;
- if (!proc_create("dev", S_IRUGO, net->proc_net, &dev_seq_fops))
+ if (!proc_create("dev", 0444, net->proc_net, &dev_seq_fops))
goto out;
- if (!proc_create("softnet_stat", S_IRUGO, net->proc_net,
+ if (!proc_create("softnet_stat", 0444, net->proc_net,
&softnet_seq_fops))
goto out_dev;
- if (!proc_create("ptype", S_IRUGO, net->proc_net, &ptype_seq_fops))
+ if (!proc_create("ptype", 0444, net->proc_net, &ptype_seq_fops))
goto out_softnet;
if (wext_proc_init(net))
static struct pernet_operations __net_initdata dev_proc_ops = {
.init = dev_proc_net_init,
.exit = dev_proc_net_exit,
- .async = true,
};
static int dev_mc_seq_show(struct seq_file *seq, void *v)
static struct pernet_operations __net_initdata dev_mc_net_ops = {
.init = dev_mc_net_init,
.exit = dev_mc_net_exit,
- .async = true,
};
int __init dev_proc_init(void)
return netdev_store(dev, attr, buf, len, change_group);
}
NETDEVICE_SHOW(group, fmt_dec);
-static DEVICE_ATTR(netdev_group, S_IRUGO | S_IWUSR, group_show, group_store);
+static DEVICE_ATTR(netdev_group, 0644, group_show, group_store);
static int change_proto_down(struct net_device *dev, unsigned long proto_down)
{
}
static struct rx_queue_attribute rps_cpus_attribute __ro_after_init
- = __ATTR(rps_cpus, S_IRUGO | S_IWUSR, show_rps_map, store_rps_map);
+ = __ATTR(rps_cpus, 0644, show_rps_map, store_rps_map);
static struct rx_queue_attribute rps_dev_flow_table_cnt_attribute __ro_after_init
- = __ATTR(rps_flow_cnt, S_IRUGO | S_IWUSR,
+ = __ATTR(rps_flow_cnt, 0644,
show_rps_dev_flow_table_cnt, store_rps_dev_flow_table_cnt);
#endif /* CONFIG_RPS */
}
static struct netdev_queue_attribute bql_hold_time_attribute __ro_after_init
- = __ATTR(hold_time, S_IRUGO | S_IWUSR,
+ = __ATTR(hold_time, 0644,
bql_show_hold_time, bql_set_hold_time);
static ssize_t bql_show_inflight(struct netdev_queue *queue,
}
static struct netdev_queue_attribute bql_inflight_attribute __ro_after_init =
- __ATTR(inflight, S_IRUGO, bql_show_inflight, NULL);
+ __ATTR(inflight, 0444, bql_show_inflight, NULL);
#define BQL_ATTR(NAME, FIELD) \
static ssize_t bql_show_ ## NAME(struct netdev_queue *queue, \
} \
\
static struct netdev_queue_attribute bql_ ## NAME ## _attribute __ro_after_init \
- = __ATTR(NAME, S_IRUGO | S_IWUSR, \
+ = __ATTR(NAME, 0644, \
bql_show_ ## NAME, bql_set_ ## NAME)
BQL_ATTR(limit, limit);
LIST_HEAD(net_namespace_list);
EXPORT_SYMBOL_GPL(net_namespace_list);
+/* Protects net_namespace_list. Nests iside rtnl_lock() */
+DECLARE_RWSEM(net_rwsem);
+EXPORT_SYMBOL_GPL(net_rwsem);
+
struct net init_net = {
.count = REFCOUNT_INIT(1),
.dev_base_head = LIST_HEAD_INIT(init_net.dev_base_head),
EXPORT_SYMBOL(init_net);
static bool init_net_initialized;
-static unsigned nr_sync_pernet_ops;
/*
- * net_sem: protects: pernet_list, net_generic_ids, nr_sync_pernet_ops,
+ * pernet_ops_rwsem: protects: pernet_list, net_generic_ids,
* init_net_initialized and first_device pointer.
+ * This is internal net namespace object. Please, don't use it
+ * outside.
*/
-DECLARE_RWSEM(net_sem);
+DECLARE_RWSEM(pernet_ops_rwsem);
#define MIN_PERNET_OPS_ID \
((sizeof(struct net_generic) + sizeof(void *) - 1) / sizeof(void *))
BUG_ON(id < MIN_PERNET_OPS_ID);
old_ng = rcu_dereference_protected(net->gen,
- lockdep_is_held(&net_sem));
+ lockdep_is_held(&pernet_ops_rwsem));
if (old_ng->s.len > id) {
old_ng->ptr[id] = data;
return 0;
*/
static __net_init int setup_net(struct net *net, struct user_namespace *user_ns)
{
- /* Must be called with net_sem held */
+ /* Must be called with pernet_ops_rwsem held */
const struct pernet_operations *ops, *saved_ops;
int error = 0;
LIST_HEAD(net_exit_list);
net->user_ns = user_ns;
idr_init(&net->netns_ids);
spin_lock_init(&net->nsid_lock);
+ mutex_init(&net->ipv4.ra_mutex);
list_for_each_entry(ops, &pernet_list, list) {
error = ops_init(ops, net);
if (error < 0)
goto out_undo;
}
- rtnl_lock();
+ down_write(&net_rwsem);
list_add_tail_rcu(&net->list, &net_namespace_list);
- rtnl_unlock();
+ up_write(&net_rwsem);
out:
return error;
static struct pernet_operations net_defaults_ops = {
.init = net_defaults_init_net,
- .async = true,
};
static __init int net_defaults_init(void)
{
struct ucounts *ucounts;
struct net *net;
- unsigned write;
int rv;
if (!(flags & CLONE_NEWNET))
refcount_set(&net->passive, 1);
net->ucounts = ucounts;
get_user_ns(user_ns);
-again:
- write = READ_ONCE(nr_sync_pernet_ops);
- if (write)
- rv = down_write_killable(&net_sem);
- else
- rv = down_read_killable(&net_sem);
+
+ rv = down_read_killable(&pernet_ops_rwsem);
if (rv < 0)
goto put_userns;
- if (!write && unlikely(READ_ONCE(nr_sync_pernet_ops))) {
- up_read(&net_sem);
- goto again;
- }
rv = setup_net(net, user_ns);
- if (write)
- up_write(&net_sem);
- else
- up_read(&net_sem);
+ up_read(&pernet_ops_rwsem);
if (rv < 0) {
put_userns:
* and this work is the only process, that may delete
* a net from net_namespace_list. So, when the below
* is executing, the list may only grow. Thus, we do not
- * use for_each_net_rcu() or rtnl_lock().
+ * use for_each_net_rcu() or net_rwsem.
*/
for_each_net(tmp) {
int id;
struct net *net, *tmp, *last;
struct llist_node *net_kill_list;
LIST_HEAD(net_exit_list);
- unsigned write;
/* Atomically snapshot the list of namespaces to cleanup */
net_kill_list = llist_del_all(&cleanup_list);
-again:
- write = READ_ONCE(nr_sync_pernet_ops);
- if (write)
- down_write(&net_sem);
- else
- down_read(&net_sem);
- if (!write && unlikely(READ_ONCE(nr_sync_pernet_ops))) {
- up_read(&net_sem);
- goto again;
- }
+ down_read(&pernet_ops_rwsem);
/* Don't let anyone else find us. */
- rtnl_lock();
+ down_write(&net_rwsem);
llist_for_each_entry(net, net_kill_list, cleanup_list)
list_del_rcu(&net->list);
/* Cache last net. After we unlock rtnl, no one new net
* useless anyway, as netns_ids are destroyed there.
*/
last = list_last_entry(&net_namespace_list, struct net, list);
- rtnl_unlock();
+ up_write(&net_rwsem);
llist_for_each_entry(net, net_kill_list, cleanup_list) {
unhash_nsid(net, last);
list_for_each_entry_reverse(ops, &pernet_list, list)
ops_free_list(ops, &net_exit_list);
- if (write)
- up_write(&net_sem);
- else
- up_read(&net_sem);
+ up_read(&pernet_ops_rwsem);
/* Ensure there are no outstanding rcu callbacks using this
* network namespace.
*/
void net_ns_barrier(void)
{
- down_write(&net_sem);
- up_write(&net_sem);
+ down_write(&pernet_ops_rwsem);
+ up_write(&pernet_ops_rwsem);
}
EXPORT_SYMBOL(net_ns_barrier);
static struct pernet_operations __net_initdata net_ns_ops = {
.init = net_ns_net_init,
.exit = net_ns_net_exit,
- .async = true,
};
static const struct nla_policy rtnl_net_policy[NETNSA_MAX + 1] = {
rcu_assign_pointer(init_net.gen, ng);
- down_write(&net_sem);
+ down_write(&pernet_ops_rwsem);
if (setup_net(&init_net, &init_user_ns))
panic("Could not setup the initial network namespace");
init_net_initialized = true;
- up_write(&net_sem);
+ up_write(&pernet_ops_rwsem);
register_pernet_subsys(&net_ns_ops);
list_add_tail(&ops->list, list);
if (ops->init || (ops->id && ops->size)) {
+ /* We held write locked pernet_ops_rwsem, and parallel
+ * setup_net() and cleanup_net() are not possible.
+ */
for_each_net(net) {
error = ops_init(ops, net);
if (error)
LIST_HEAD(net_exit_list);
list_del(&ops->list);
+ /* See comment in __register_pernet_operations() */
for_each_net(net)
list_add_tail(&net->exit_list, &net_exit_list);
ops_exit_list(ops, &net_exit_list);
rcu_barrier();
if (ops->id)
ida_remove(&net_generic_ids, *ops->id);
- } else if (!ops->async) {
- pr_info_once("Pernet operations %ps are sync.\n", ops);
- nr_sync_pernet_ops++;
}
return error;
static void unregister_pernet_operations(struct pernet_operations *ops)
{
- if (!ops->async)
- BUG_ON(nr_sync_pernet_ops-- == 0);
__unregister_pernet_operations(ops);
rcu_barrier();
if (ops->id)
int register_pernet_subsys(struct pernet_operations *ops)
{
int error;
- down_write(&net_sem);
+ down_write(&pernet_ops_rwsem);
error = register_pernet_operations(first_device, ops);
- up_write(&net_sem);
+ up_write(&pernet_ops_rwsem);
return error;
}
EXPORT_SYMBOL_GPL(register_pernet_subsys);
*/
void unregister_pernet_subsys(struct pernet_operations *ops)
{
- down_write(&net_sem);
+ down_write(&pernet_ops_rwsem);
unregister_pernet_operations(ops);
- up_write(&net_sem);
+ up_write(&pernet_ops_rwsem);
}
EXPORT_SYMBOL_GPL(unregister_pernet_subsys);
int register_pernet_device(struct pernet_operations *ops)
{
int error;
- down_write(&net_sem);
+ down_write(&pernet_ops_rwsem);
error = register_pernet_operations(&pernet_list, ops);
if (!error && (first_device == &pernet_list))
first_device = &ops->list;
- up_write(&net_sem);
+ up_write(&pernet_ops_rwsem);
return error;
}
EXPORT_SYMBOL_GPL(register_pernet_device);
*/
void unregister_pernet_device(struct pernet_operations *ops)
{
- down_write(&net_sem);
+ down_write(&pernet_ops_rwsem);
if (&ops->list == first_device)
first_device = first_device->next;
unregister_pernet_operations(ops);
- up_write(&net_sem);
+ up_write(&pernet_ops_rwsem);
}
EXPORT_SYMBOL_GPL(unregister_pernet_device);
.exit = pg_net_exit,
.id = &pg_net_id,
.size = sizeof(struct pktgen_net),
- .async = true,
};
static int __init pg_init(void)
{
struct net *net;
+ down_read(&net_rwsem);
for_each_net(net) {
__rtnl_kill_links(net, ops);
}
+ up_read(&net_rwsem);
list_del(&ops->list);
}
EXPORT_SYMBOL_GPL(__rtnl_link_unregister);
for (;;) {
unregistering = false;
rtnl_lock();
+ /* We held write locked pernet_ops_rwsem, and parallel
+ * setup_net() and cleanup_net() are not possible.
+ */
for_each_net(net) {
if (net->dev_unreg_count > 0) {
unregistering = true;
*/
void rtnl_link_unregister(struct rtnl_link_ops *ops)
{
- /* Close the race with cleanup_net() */
- down_write(&net_sem);
+ /* Close the race with setup_net() and cleanup_net() */
+ down_write(&pernet_ops_rwsem);
rtnl_lock_unregistering_all();
__rtnl_link_unregister(ops);
rtnl_unlock();
- up_write(&net_sem);
+ up_write(&pernet_ops_rwsem);
}
EXPORT_SYMBOL_GPL(rtnl_link_unregister);
static struct pernet_operations rtnetlink_net_ops = {
.init = rtnetlink_net_init,
.exit = rtnetlink_net_exit,
- .async = true,
};
void __init rtnetlink_init(void)
}
EXPORT_SYMBOL_GPL(skb_pull_rcsum);
+static inline skb_frag_t skb_head_frag_to_page_desc(struct sk_buff *frag_skb)
+{
+ skb_frag_t head_frag;
+ struct page *page;
+
+ page = virt_to_head_page(frag_skb->head);
+ head_frag.page.p = page;
+ head_frag.page_offset = frag_skb->data -
+ (unsigned char *)page_address(page);
+ head_frag.size = skb_headlen(frag_skb);
+ return head_frag;
+}
+
/**
* skb_segment - Perform protocol segmentation on skb.
* @head_skb: buffer to segment
while (pos < offset + len) {
if (i >= nfrags) {
- BUG_ON(skb_headlen(list_skb));
-
i = 0;
nfrags = skb_shinfo(list_skb)->nr_frags;
frag = skb_shinfo(list_skb)->frags;
frag_skb = list_skb;
+ if (!skb_headlen(list_skb)) {
+ BUG_ON(!nfrags);
+ } else {
+ BUG_ON(!list_skb->head_frag);
- BUG_ON(!nfrags);
-
+ /* to make room for head_frag. */
+ i--;
+ frag--;
+ }
if (skb_orphan_frags(frag_skb, GFP_ATOMIC) ||
skb_zerocopy_clone(nskb, frag_skb,
GFP_ATOMIC))
goto err;
}
- *nskb_frag = *frag;
+ *nskb_frag = (i < 0) ? skb_head_frag_to_page_desc(frag_skb) : *frag;
__skb_frag_ref(nskb_frag);
size = skb_frag_size(nskb_frag);
skb_queue_tail(&sk->sk_error_queue, skb);
if (!sock_flag(sk, SOCK_DEAD))
- sk->sk_data_ready(sk);
+ sk->sk_error_report(sk);
return 0;
}
EXPORT_SYMBOL(sock_queue_err_skb);
thlen += inner_tcp_hdrlen(skb);
} else if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) {
thlen = tcp_hdrlen(skb);
- } else if (unlikely(shinfo->gso_type & SKB_GSO_SCTP)) {
+ } else if (unlikely(skb_is_gso_sctp(skb))) {
thlen = sizeof(struct sctphdr);
}
/* UFO sets gso_size to the size of the fragmentation
static struct sk_buff *skb_reorder_vlan_header(struct sk_buff *skb)
{
+ int mac_len;
+
if (skb_cow(skb, skb_headroom(skb)) < 0) {
kfree_skb(skb);
return NULL;
}
- memmove(skb->data - ETH_HLEN, skb->data - skb->mac_len - VLAN_HLEN,
- 2 * ETH_ALEN);
+ mac_len = skb->data - skb_mac_header(skb);
+ memmove(skb_mac_header(skb) + VLAN_HLEN, skb_mac_header(skb),
+ mac_len - VLAN_HLEN - ETH_TLEN);
skb->mac_header += VLAN_HLEN;
return skb;
}
}
EXPORT_SYMBOL(sk_page_frag_refill);
+int sk_alloc_sg(struct sock *sk, int len, struct scatterlist *sg,
+ int sg_start, int *sg_curr_index, unsigned int *sg_curr_size,
+ int first_coalesce)
+{
+ int sg_curr = *sg_curr_index, use = 0, rc = 0;
+ unsigned int size = *sg_curr_size;
+ struct page_frag *pfrag;
+ struct scatterlist *sge;
+
+ len -= size;
+ pfrag = sk_page_frag(sk);
+
+ while (len > 0) {
+ unsigned int orig_offset;
+
+ if (!sk_page_frag_refill(sk, pfrag)) {
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ use = min_t(int, len, pfrag->size - pfrag->offset);
+
+ if (!sk_wmem_schedule(sk, use)) {
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ sk_mem_charge(sk, use);
+ size += use;
+ orig_offset = pfrag->offset;
+ pfrag->offset += use;
+
+ sge = sg + sg_curr - 1;
+ if (sg_curr > first_coalesce && sg_page(sg) == pfrag->page &&
+ sg->offset + sg->length == orig_offset) {
+ sg->length += use;
+ } else {
+ sge = sg + sg_curr;
+ sg_unmark_end(sge);
+ sg_set_page(sge, pfrag->page, use, orig_offset);
+ get_page(pfrag->page);
+ sg_curr++;
+
+ if (sg_curr == MAX_SKB_FRAGS)
+ sg_curr = 0;
+
+ if (sg_curr == sg_start) {
+ rc = -ENOSPC;
+ break;
+ }
+ }
+
+ len -= use;
+ }
+out:
+ *sg_curr_size = size;
+ *sg_curr_index = sg_curr;
+ return rc;
+}
+EXPORT_SYMBOL(sk_alloc_sg);
+
static void __lock_sock(struct sock *sk)
__releases(&sk->sk_lock.slock)
__acquires(&sk->sk_lock.slock)
static struct pernet_operations net_inuse_ops = {
.init = sock_inuse_init_net,
.exit = sock_inuse_exit_net,
- .async = true,
};
static __init int net_inuse_init(void)
}
EXPORT_SYMBOL(proto_unregister);
+int sock_load_diag_module(int family, int protocol)
+{
+ if (!protocol) {
+ if (!sock_is_registered(family))
+ return -ENOENT;
+
+ return request_module("net-pf-%d-proto-%d-type-%d", PF_NETLINK,
+ NETLINK_SOCK_DIAG, family);
+ }
+
+#ifdef CONFIG_INET
+ if (family == AF_INET &&
+ !rcu_access_pointer(inet_protos[protocol]))
+ return -ENOENT;
+#endif
+
+ return request_module("net-pf-%d-proto-%d-type-%d-%d", PF_NETLINK,
+ NETLINK_SOCK_DIAG, family, protocol);
+}
+EXPORT_SYMBOL(sock_load_diag_module);
+
#ifdef CONFIG_PROC_FS
static void *proto_seq_start(struct seq_file *seq, loff_t *pos)
__acquires(proto_list_mutex)
static __net_init int proto_init_net(struct net *net)
{
- if (!proc_create("protocols", S_IRUGO, net->proc_net, &proto_seq_fops))
+ if (!proc_create("protocols", 0444, net->proc_net, &proto_seq_fops))
return -ENOMEM;
return 0;
static __net_initdata struct pernet_operations proto_net_ops = {
.init = proto_init_net,
.exit = proto_exit_net,
- .async = true,
};
static int __init proto_init(void)
return -EINVAL;
if (sock_diag_handlers[req->sdiag_family] == NULL)
- request_module("net-pf-%d-proto-%d-type-%d", PF_NETLINK,
- NETLINK_SOCK_DIAG, req->sdiag_family);
+ sock_load_diag_module(req->sdiag_family, 0);
mutex_lock(&sock_diag_table_mutex);
hndl = sock_diag_handlers[req->sdiag_family];
case TCPDIAG_GETSOCK:
case DCCPDIAG_GETSOCK:
if (inet_rcv_compat == NULL)
- request_module("net-pf-%d-proto-%d-type-%d", PF_NETLINK,
- NETLINK_SOCK_DIAG, AF_INET);
+ sock_load_diag_module(AF_INET, 0);
mutex_lock(&sock_diag_table_mutex);
if (inet_rcv_compat != NULL)
case SKNLGRP_INET_TCP_DESTROY:
case SKNLGRP_INET_UDP_DESTROY:
if (!sock_diag_handlers[AF_INET])
- request_module("net-pf-%d-proto-%d-type-%d", PF_NETLINK,
- NETLINK_SOCK_DIAG, AF_INET);
+ sock_load_diag_module(AF_INET, 0);
break;
case SKNLGRP_INET6_TCP_DESTROY:
case SKNLGRP_INET6_UDP_DESTROY:
if (!sock_diag_handlers[AF_INET6])
- request_module("net-pf-%d-proto-%d-type-%d", PF_NETLINK,
- NETLINK_SOCK_DIAG, AF_INET6);
+ sock_load_diag_module(AF_INET6, 0);
break;
}
return 0;
static struct pernet_operations diag_net_ops = {
.init = diag_net_init,
.exit = diag_net_exit,
- .async = true,
};
static int __init sock_diag_init(void)
static __net_initdata struct pernet_operations sysctl_core_ops = {
.init = sysctl_core_net_init,
.exit = sysctl_core_net_exit,
- .async = true,
};
static __init int sysctl_core_init(void)
.init = dccp_v4_init_net,
.exit = dccp_v4_exit_net,
.exit_batch = dccp_v4_exit_batch,
- .async = true,
};
static int __init dccp_v4_init(void)
.init = dccp_v6_init_net,
.exit = dccp_v6_exit_net,
.exit_batch = dccp_v6_exit_batch,
- .async = true,
};
static int __init dccp_v6_init(void)
if (skb == NULL)
goto out_release;
+ if (sk->sk_state == DCCP_CLOSED) {
+ rc = -ENOTCONN;
+ goto out_discard;
+ }
+
skb_reserve(skb, sk->sk_prot->max_header);
rc = memcpy_from_msg(skb_put(skb, len), msg, len);
if (rc != 0)
dev_add_pack(&dn_dix_packet_type);
register_netdevice_notifier(&dn_dev_notifier);
- proc_create("decnet", S_IRUGO, init_net.proc_net, &dn_socket_seq_fops);
+ proc_create("decnet", 0444, init_net.proc_net, &dn_socket_seq_fops);
dn_register_sysctl();
out:
return rc;
rtnl_register_module(THIS_MODULE, PF_DECnet, RTM_GETADDR,
NULL, dn_nl_dump_ifaddr, 0);
- proc_create("decnet_dev", S_IRUGO, init_net.proc_net, &dn_dev_seq_fops);
+ proc_create("decnet_dev", 0444, init_net.proc_net, &dn_dev_seq_fops);
#ifdef CONFIG_SYSCTL
{
void __init dn_neigh_init(void)
{
neigh_table_init(NEIGH_DN_TABLE, &dn_neigh_table);
- proc_create("decnet_neigh", S_IRUGO, init_net.proc_net,
+ proc_create("decnet_neigh", 0444, init_net.proc_net,
&dn_neigh_seq_fops);
}
dn_dst_ops.gc_thresh = (dn_rt_hash_mask + 1);
- proc_create("decnet_cache", S_IRUGO, init_net.proc_net,
+ proc_create("decnet_cache", 0444, init_net.proc_net,
&dn_rt_cache_seq_fops);
#ifdef CONFIG_DECNET_ROUTER
MODULE_LICENSE("GPL");
unsigned int dns_resolver_debug;
-module_param_named(debug, dns_resolver_debug, uint, S_IWUSR | S_IRUGO);
+module_param_named(debug, dns_resolver_debug, uint, 0644);
MODULE_PARM_DESC(debug, "DNS Resolver debugging mask");
const struct cred *dns_resolver_cache;
ds->ports[i].dn = cd->port_dn[i];
ds->ports[i].cpu_dp = dst->cpu_dp;
- if (dsa_is_user_port(ds, i))
+ if (!dsa_is_user_port(ds, i))
continue;
ret = dsa_slave_create(&ds->ports[i]);
static int lowpan_device_event(struct notifier_block *unused,
unsigned long event, void *ptr)
{
- struct net_device *wdev = netdev_notifier_info_to_dev(ptr);
+ struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
+ struct wpan_dev *wpan_dev;
- if (wdev->type != ARPHRD_IEEE802154)
+ if (ndev->type != ARPHRD_IEEE802154)
+ return NOTIFY_DONE;
+ wpan_dev = ndev->ieee802154_ptr;
+ if (!wpan_dev)
return NOTIFY_DONE;
switch (event) {
* also delete possible lowpan interfaces which belongs
* to the wpan interface.
*/
- if (wdev->ieee802154_ptr->lowpan_dev)
- lowpan_dellink(wdev->ieee802154_ptr->lowpan_dev, NULL);
+ if (wpan_dev->lowpan_dev)
+ lowpan_dellink(wpan_dev->lowpan_dev, NULL);
break;
default:
return NOTIFY_DONE;
static struct pernet_operations cfg802154_pernet_ops = {
.exit = cfg802154_pernet_exit,
- .async = true,
};
static int __init wpan_phy_class_init(void)
static __net_initdata struct pernet_operations ipv4_mib_ops = {
.init = ipv4_mib_init_net,
.exit = ipv4_mib_exit_net,
- .async = true,
};
static int __init init_ipv4_mibs(void)
static __net_initdata struct pernet_operations af_inet_ops = {
.init = inet_init_net,
.exit = inet_exit_net,
- .async = true,
};
static int __init init_inet_pernet_ops(void)
static int __net_init arp_net_init(struct net *net)
{
- if (!proc_create("arp", S_IRUGO, net->proc_net, &arp_seq_fops))
+ if (!proc_create("arp", 0444, net->proc_net, &arp_seq_fops))
return -ENOMEM;
return 0;
}
static struct pernet_operations arp_net_ops = {
.init = arp_net_init,
.exit = arp_net_exit,
- .async = true,
};
static int __init arp_proc_init(void)
static __net_initdata struct pernet_operations devinet_ops = {
.init = devinet_init_net,
.exit = devinet_exit_net,
- .async = true,
};
static struct rtnl_af_ops inet_af_ops __read_mostly = {
static void esp_input_set_header(struct sk_buff *skb, __be32 *seqhi)
{
struct xfrm_state *x = xfrm_input_state(skb);
- struct ip_esp_hdr *esph = (struct ip_esp_hdr *)skb->data;
+ struct ip_esp_hdr *esph;
/* For ESN we move the header forward by 4 bytes to
* accomodate the high bits. We will move it back after
if (!(features & NETIF_F_HW_ESP) || !x->xso.offload_handle ||
(x->xso.dev != skb->dev))
esp_features = features & ~(NETIF_F_SG | NETIF_F_CSUM_MASK);
+ else if (!(features & NETIF_F_HW_ESP_TX_CSUM))
+ esp_features = features & ~NETIF_F_CSUM_MASK;
xo->flags |= XFRM_GSO_SEGMENT;
static struct pernet_operations fib_net_ops = {
.init = fib_net_init,
.exit = fib_net_exit,
- .async = true,
};
void __init ip_fib_init(void)
return -ENOMEM;
}
+/* fib notifier for ADD is sent before calling fib_insert_alias with
+ * the expectation that the only possible failure ENOMEM
+ */
static int fib_insert_alias(struct trie *t, struct key_vector *tp,
struct key_vector *l, struct fib_alias *new,
struct fib_alias *fa, t_key key)
new_fa->tb_id = tb->tb_id;
new_fa->fa_default = -1;
- call_fib_entry_notifiers(net, FIB_EVENT_ENTRY_REPLACE,
- key, plen, new_fa, extack);
+ err = call_fib_entry_notifiers(net,
+ FIB_EVENT_ENTRY_REPLACE,
+ key, plen, new_fa,
+ extack);
+ if (err)
+ goto out_free_new_fa;
+
rtmsg_fib(RTM_NEWROUTE, htonl(key), new_fa, plen,
tb->tb_id, &cfg->fc_nlinfo, nlflags);
new_fa->tb_id = tb->tb_id;
new_fa->fa_default = -1;
+ err = call_fib_entry_notifiers(net, event, key, plen, new_fa, extack);
+ if (err)
+ goto out_free_new_fa;
+
/* Insert new entry to the list. */
err = fib_insert_alias(t, tp, l, new_fa, fa, key);
if (err)
- goto out_free_new_fa;
+ goto out_fib_notif;
if (!plen)
tb->tb_num_default++;
rt_cache_flush(cfg->fc_nlinfo.nl_net);
- call_fib_entry_notifiers(net, event, key, plen, new_fa, extack);
rtmsg_fib(RTM_NEWROUTE, htonl(key), new_fa, plen, new_fa->tb_id,
&cfg->fc_nlinfo, nlflags);
succeeded:
return 0;
+out_fib_notif:
+ /* notifier was sent that entry would be added to trie, but
+ * the add failed and need to recover. Only failure for
+ * fib_insert_alias is ENOMEM.
+ */
+ NL_SET_ERR_MSG(extack, "Failed to insert route into trie");
+ call_fib_entry_notifiers(net, FIB_EVENT_ENTRY_DEL, key,
+ plen, new_fa, NULL);
out_free_new_fa:
kmem_cache_free(fn_alias_kmem, new_fa);
out:
int __net_init fib_proc_init(struct net *net)
{
- if (!proc_create("fib_trie", S_IRUGO, net->proc_net, &fib_trie_fops))
+ if (!proc_create("fib_trie", 0444, net->proc_net, &fib_trie_fops))
goto out1;
- if (!proc_create("fib_triestat", S_IRUGO, net->proc_net,
+ if (!proc_create("fib_triestat", 0444, net->proc_net,
&fib_triestat_fops))
goto out2;
- if (!proc_create("route", S_IRUGO, net->proc_net, &fib_route_fops))
+ if (!proc_create("route", 0444, net->proc_net, &fib_route_fops))
goto out3;
return 0;
.exit = fou_exit_net,
.id = &fou_net_id,
.size = sizeof(struct fou_net),
- .async = true,
};
static int __init fou_init(void)
static struct pernet_operations __net_initdata icmp_sk_ops = {
.init = icmp_sk_init,
.exit = icmp_sk_exit,
- .async = true,
};
int __init icmp_init(void)
struct proc_dir_entry *pde;
int err;
- pde = proc_create("igmp", S_IRUGO, net->proc_net, &igmp_mc_seq_fops);
+ pde = proc_create("igmp", 0444, net->proc_net, &igmp_mc_seq_fops);
if (!pde)
goto out_igmp;
- pde = proc_create("mcfilter", S_IRUGO, net->proc_net,
+ pde = proc_create("mcfilter", 0444, net->proc_net,
&igmp_mcf_seq_fops);
if (!pde)
goto out_mcfilter;
static struct pernet_operations igmp_net_ops = {
.init = igmp_net_init,
.exit = igmp_net_exit,
- .async = true,
};
#endif
static const struct inet_diag_handler *inet_diag_lock_handler(int proto)
{
if (!inet_diag_table[proto])
- request_module("net-pf-%d-proto-%d-type-%d-%d", PF_NETLINK,
- NETLINK_SOCK_DIAG, AF_INET, proto);
+ sock_load_diag_module(AF_INET, proto);
mutex_lock(&inet_diag_table_mutex);
if (!inet_diag_table[proto])
static bool inet_fragq_should_evict(const struct inet_frag_queue *q)
{
+ if (!hlist_unhashed(&q->list_evictor))
+ return false;
+
return q->net->low_thresh == 0 ||
frag_mem_limit(q->net) >= q->net->low_thresh;
}
static struct pernet_operations ip4_frags_ops = {
.init = ipv4_frags_init_net,
.exit = ipv4_frags_exit_net,
- .async = true,
};
void __init ipfrag_init(void)
(TUNNEL_CSUM | TUNNEL_KEY | TUNNEL_SEQ);
gre_build_header(skb, tunnel_hlen, flags, proto,
tunnel_id_to_key32(tun_info->key.tun_id),
- (flags | TUNNEL_SEQ) ? htonl(tunnel->o_seqno++) : 0);
+ (flags & TUNNEL_SEQ) ? htonl(tunnel->o_seqno++) : 0);
df = key->tun_flags & TUNNEL_DONT_FRAGMENT ? htons(IP_DF) : 0;
.exit_batch = ipgre_exit_batch_net,
.id = &ipgre_net_id,
.size = sizeof(struct ip_tunnel_net),
- .async = true,
};
static int ipgre_tunnel_validate(struct nlattr *tb[], struct nlattr *data[],
.exit_batch = ipgre_tap_exit_batch_net,
.id = &gre_tap_net_id,
.size = sizeof(struct ip_tunnel_net),
- .async = true,
};
static int __net_init erspan_init_net(struct net *net)
.exit_batch = erspan_exit_batch_net,
.id = &erspan_net_id,
.size = sizeof(struct ip_tunnel_net),
- .async = true,
};
static int __init ipgre_init(void)
struct net_device *dev = skb->dev;
struct net *net = dev_net(dev);
- for (ra = rcu_dereference(ip_ra_chain); ra; ra = rcu_dereference(ra->next)) {
+ for (ra = rcu_dereference(net->ipv4.ra_chain); ra; ra = rcu_dereference(ra->next)) {
struct sock *sk = ra->sk;
/* If socket is bound to an interface, only report
*/
if (sk && inet_sk(sk)->inet_num == protocol &&
(!sk->sk_bound_dev_if ||
- sk->sk_bound_dev_if == dev->ifindex) &&
- net_eq(sock_net(sk), net)) {
+ sk->sk_bound_dev_if == dev->ifindex)) {
if (ip_is_fragment(ip_hdr(skb))) {
if (ip_defrag(net, skb, IP_DEFRAG_CALL_RA_CHAIN))
return true;
return 0;
}
-
-/* Special input handler for packets caught by router alert option.
- They are selected only by protocol field, and then processed likely
- local ones; but only if someone wants them! Otherwise, router
- not running rsvpd will kill RSVP.
-
- It is user level problem, what it will make with them.
- I have no idea, how it will masquearde or NAT them (it is joke, joke :-)),
- but receiver should be enough clever f.e. to forward mtrace requests,
- sent to multicast group to reach destination designated router.
- */
-struct ip_ra_chain __rcu *ip_ra_chain;
-
-
static void ip_ra_destroy_rcu(struct rcu_head *head)
{
struct ip_ra_chain *ra = container_of(head, struct ip_ra_chain, rcu);
{
struct ip_ra_chain *ra, *new_ra;
struct ip_ra_chain __rcu **rap;
+ struct net *net = sock_net(sk);
if (sk->sk_type != SOCK_RAW || inet_sk(sk)->inet_num == IPPROTO_RAW)
return -EINVAL;
new_ra = on ? kmalloc(sizeof(*new_ra), GFP_KERNEL) : NULL;
- for (rap = &ip_ra_chain;
- (ra = rtnl_dereference(*rap)) != NULL;
+ mutex_lock(&net->ipv4.ra_mutex);
+ for (rap = &net->ipv4.ra_chain;
+ (ra = rcu_dereference_protected(*rap,
+ lockdep_is_held(&net->ipv4.ra_mutex))) != NULL;
rap = &ra->next) {
if (ra->sk == sk) {
if (on) {
+ mutex_unlock(&net->ipv4.ra_mutex);
kfree(new_ra);
return -EADDRINUSE;
}
/* dont let ip_call_ra_chain() use sk again */
ra->sk = NULL;
RCU_INIT_POINTER(*rap, ra->next);
+ mutex_unlock(&net->ipv4.ra_mutex);
if (ra->destructor)
ra->destructor(sk);
return 0;
}
}
- if (!new_ra)
+ if (!new_ra) {
+ mutex_unlock(&net->ipv4.ra_mutex);
return -ENOBUFS;
+ }
new_ra->sk = sk;
new_ra->destructor = destructor;
RCU_INIT_POINTER(new_ra->next, ra);
rcu_assign_pointer(*rap, new_ra);
sock_hold(sk);
+ mutex_unlock(&net->ipv4.ra_mutex);
return 0;
}
case MCAST_LEAVE_GROUP:
case MCAST_LEAVE_SOURCE_GROUP:
case MCAST_UNBLOCK_SOURCE:
- case IP_ROUTER_ALERT:
return true;
}
return false;
/* If optlen==0, it is equivalent to val == 0 */
+ if (optname == IP_ROUTER_ALERT)
+ return ip_ra_control(sk, val ? 1 : 0, NULL);
if (ip_mroute_opt(optname))
return ip_mroute_setsockopt(sk, optname, optval, optlen);
goto e_inval;
inet->mc_all = val;
break;
- case IP_ROUTER_ALERT:
- err = ip_ra_control(sk, val ? 1 : 0, NULL);
- break;
case IP_FREEBIND:
if (optlen < 1)
.exit_batch = vti_exit_batch_net,
.id = &vti_net_id,
.size = sizeof(struct ip_tunnel_net),
- .async = true,
};
static int vti_tunnel_validate(struct nlattr *tb[], struct nlattr *data[],
unsigned int i;
#ifdef CONFIG_PROC_FS
- proc_create("pnp", S_IRUGO, init_net.proc_net, &pnp_seq_fops);
+ proc_create("pnp", 0444, init_net.proc_net, &pnp_seq_fops);
#endif /* CONFIG_PROC_FS */
if (!ic_enable)
.exit_batch = ipip_exit_batch_net,
.id = &ipip_net_id,
.size = sizeof(struct ip_tunnel_net),
- .async = true,
};
static int __init ipip_init(void)
}
#endif
-static int call_ipmr_vif_entry_notifier(struct notifier_block *nb,
- struct net *net,
- enum fib_event_type event_type,
- struct vif_device *vif,
- vifi_t vif_index, u32 tb_id)
-{
- struct vif_entry_notifier_info info = {
- .info = {
- .family = RTNL_FAMILY_IPMR,
- .net = net,
- },
- .dev = vif->dev,
- .vif_index = vif_index,
- .vif_flags = vif->flags,
- .tb_id = tb_id,
- };
-
- return call_fib_notifier(nb, net, event_type, &info.info);
-}
-
static int call_ipmr_vif_entry_notifiers(struct net *net,
enum fib_event_type event_type,
struct vif_device *vif,
vifi_t vif_index, u32 tb_id)
{
- struct vif_entry_notifier_info info = {
- .info = {
- .family = RTNL_FAMILY_IPMR,
- .net = net,
- },
- .dev = vif->dev,
- .vif_index = vif_index,
- .vif_flags = vif->flags,
- .tb_id = tb_id,
- };
-
- ASSERT_RTNL();
- net->ipv4.ipmr_seq++;
- return call_fib_notifiers(net, event_type, &info.info);
-}
-
-static int call_ipmr_mfc_entry_notifier(struct notifier_block *nb,
- struct net *net,
- enum fib_event_type event_type,
- struct mfc_cache *mfc, u32 tb_id)
-{
- struct mfc_entry_notifier_info info = {
- .info = {
- .family = RTNL_FAMILY_IPMR,
- .net = net,
- },
- .mfc = mfc,
- .tb_id = tb_id
- };
-
- return call_fib_notifier(nb, net, event_type, &info.info);
+ return mr_call_vif_notifiers(net, RTNL_FAMILY_IPMR, event_type,
+ vif, vif_index, tb_id,
+ &net->ipv4.ipmr_seq);
}
static int call_ipmr_mfc_entry_notifiers(struct net *net,
enum fib_event_type event_type,
struct mfc_cache *mfc, u32 tb_id)
{
- struct mfc_entry_notifier_info info = {
- .info = {
- .family = RTNL_FAMILY_IPMR,
- .net = net,
- },
- .mfc = mfc,
- .tb_id = tb_id
- };
-
- ASSERT_RTNL();
- net->ipv4.ipmr_seq++;
- return call_fib_notifiers(net, event_type, &info.info);
+ return mr_call_mfc_notifiers(net, RTNL_FAMILY_IPMR, event_type,
+ &mfc->_c, tb_id, &net->ipv4.ipmr_seq);
}
/**
kmem_cache_free(mrt_cachep, (struct mfc_cache *)c);
}
-void ipmr_cache_free(struct mfc_cache *c)
+static void ipmr_cache_free(struct mfc_cache *c)
{
call_rcu(&c->_c.rcu, ipmr_cache_free_rcu);
}
-EXPORT_SYMBOL(ipmr_cache_free);
/* Destroy an unresolved cache entry, killing queued skbs
* and reporting error to netlink readers.
if (c) {
c->_c.mfc_un.res.last_assert = jiffies - MFC_ASSERT_THRESH - 1;
c->_c.mfc_un.res.minvif = MAXVIFS;
+ c->_c.free = ipmr_cache_free_rcu;
refcount_set(&c->_c.mfc_un.res.refcount, 1);
}
return c;
list_del_rcu(&c->_c.list);
call_ipmr_mfc_entry_notifiers(net, FIB_EVENT_ENTRY_DEL, c, mrt->id);
mroute_netlink_event(mrt, c, RTM_DELROUTE);
- ipmr_cache_put(c);
+ mr_cache_put(&c->_c);
return 0;
}
call_ipmr_mfc_entry_notifiers(net, FIB_EVENT_ENTRY_DEL, cache,
mrt->id);
mroute_netlink_event(mrt, cache, RTM_DELROUTE);
- ipmr_cache_put(cache);
+ mr_cache_put(c);
}
if (atomic_read(&mrt->cache_resolve_queue_len) != 0) {
struct net *net = sock_net(sk);
struct mr_table *mrt;
- ASSERT_RTNL();
+ rtnl_lock();
ipmr_for_each_table(mrt, net) {
if (sk == rtnl_dereference(mrt->mroute_sk)) {
IPV4_DEVCONF_ALL(net, MC_FORWARDING)--;
mroute_clean_tables(mrt, false);
}
}
+ rtnl_unlock();
}
/* Socket options and virtual interface manipulation. The whole
if (sk != rcu_access_pointer(mrt->mroute_sk)) {
ret = -EACCES;
} else {
+ /* We need to unlock here because mrtsock_destruct takes
+ * care of rtnl itself and we can't change that due to
+ * the IP_ROUTER_ALERT setsockopt which runs without it.
+ */
+ rtnl_unlock();
ret = ip_ra_control(sk, 0, NULL);
- goto out_unlock;
+ goto out;
}
break;
case MRT_ADD_VIF:
}
out_unlock:
rtnl_unlock();
+out:
return ret;
}
static int ipmr_dump(struct net *net, struct notifier_block *nb)
{
- struct mr_table *mrt;
- int err;
-
- err = ipmr_rules_dump(net, nb);
- if (err)
- return err;
-
- ipmr_for_each_table(mrt, net) {
- struct vif_device *v = &mrt->vif_table[0];
- struct mr_mfc *mfc;
- int vifi;
-
- /* Notifiy on table VIF entries */
- read_lock(&mrt_lock);
- for (vifi = 0; vifi < mrt->maxvif; vifi++, v++) {
- if (!v->dev)
- continue;
-
- call_ipmr_vif_entry_notifier(nb, net, FIB_EVENT_VIF_ADD,
- v, vifi, mrt->id);
- }
- read_unlock(&mrt_lock);
-
- /* Notify on table MFC entries */
- list_for_each_entry_rcu(mfc, &mrt->mfc_cache_list, list)
- call_ipmr_mfc_entry_notifier(nb, net,
- FIB_EVENT_ENTRY_ADD,
- (struct mfc_cache *)mfc,
- mrt->id);
- }
-
- return 0;
+ return mr_dump(net, nb, RTNL_FAMILY_IPMR, ipmr_rules_dump,
+ ipmr_mr_table_iter, &mrt_lock);
}
static const struct fib_notifier_ops ipmr_notifier_ops_template = {
static struct pernet_operations ipmr_net_ops = {
.init = ipmr_net_init,
.exit = ipmr_net_exit,
- .async = true,
};
int __init ip_mr_init(void)
return skb->len;
}
EXPORT_SYMBOL(mr_rtm_dumproute);
+
+int mr_dump(struct net *net, struct notifier_block *nb, unsigned short family,
+ int (*rules_dump)(struct net *net,
+ struct notifier_block *nb),
+ struct mr_table *(*mr_iter)(struct net *net,
+ struct mr_table *mrt),
+ rwlock_t *mrt_lock)
+{
+ struct mr_table *mrt;
+ int err;
+
+ err = rules_dump(net, nb);
+ if (err)
+ return err;
+
+ for (mrt = mr_iter(net, NULL); mrt; mrt = mr_iter(net, mrt)) {
+ struct vif_device *v = &mrt->vif_table[0];
+ struct mr_mfc *mfc;
+ int vifi;
+
+ /* Notifiy on table VIF entries */
+ read_lock(mrt_lock);
+ for (vifi = 0; vifi < mrt->maxvif; vifi++, v++) {
+ if (!v->dev)
+ continue;
+
+ mr_call_vif_notifier(nb, net, family,
+ FIB_EVENT_VIF_ADD,
+ v, vifi, mrt->id);
+ }
+ read_unlock(mrt_lock);
+
+ /* Notify on table MFC entries */
+ list_for_each_entry_rcu(mfc, &mrt->mfc_cache_list, list)
+ mr_call_mfc_notifier(nb, net, family,
+ FIB_EVENT_ENTRY_ADD,
+ mfc, mrt->id);
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(mr_dump);
static struct pernet_operations arp_tables_net_ops = {
.init = arp_tables_net_init,
.exit = arp_tables_net_exit,
- .async = true,
};
static int __init arp_tables_init(void)
static struct pernet_operations arptable_filter_net_ops = {
.exit = arptable_filter_net_exit,
- .async = true,
};
static int __init arptable_filter_init(void)
static struct pernet_operations ip_tables_net_ops = {
.init = ip_tables_net_init,
.exit = ip_tables_net_exit,
- .async = true,
};
static int __init ip_tables_init(void)
/* create proc dir entry */
sprintf(buffer, "%pI4", &ip);
- c->pde = proc_create_data(buffer, S_IWUSR|S_IRUSR,
+ c->pde = proc_create_data(buffer, 0600,
cn->procdir,
&clusterip_proc_fops, c);
if (!c->pde) {
.exit = clusterip_net_exit,
.id = &clusterip_net_id,
.size = sizeof(struct clusterip_net),
- .async = true,
};
static int __init clusterip_tg_init(void)
static struct pernet_operations iptable_filter_net_ops = {
.init = iptable_filter_net_init,
.exit = iptable_filter_net_exit,
- .async = true,
};
static int __init iptable_filter_init(void)
static struct pernet_operations iptable_mangle_net_ops = {
.exit = iptable_mangle_net_exit,
- .async = true,
};
static int __init iptable_mangle_init(void)
static struct pernet_operations iptable_nat_net_ops = {
.exit = iptable_nat_net_exit,
- .async = true,
};
static int __init iptable_nat_init(void)
static struct pernet_operations iptable_raw_net_ops = {
.exit = iptable_raw_net_exit,
- .async = true,
};
static int __init iptable_raw_init(void)
static struct pernet_operations iptable_security_net_ops = {
.exit = iptable_security_net_exit,
- .async = true,
};
static int __init iptable_security_init(void)
.exit = ipv4_net_exit,
.id = &conntrack4_net_id,
.size = sizeof(struct conntrack4_net),
- .async = true,
};
static int __init nf_conntrack_l3proto_ipv4_init(void)
static struct pernet_operations defrag4_net_ops = {
.exit = defrag4_net_exit,
- .async = true,
};
static int __init nf_defrag_init(void)
static struct pernet_operations nf_log_arp_net_ops = {
.init = nf_log_arp_net_init,
.exit = nf_log_arp_net_exit,
- .async = true,
};
static int __init nf_log_arp_init(void)
static struct pernet_operations nf_log_ipv4_net_ops = {
.init = nf_log_ipv4_net_init,
.exit = nf_log_ipv4_net_exit,
- .async = true,
};
static int __init nf_log_ipv4_init(void)
int ping_proc_register(struct net *net, struct ping_seq_afinfo *afinfo)
{
struct proc_dir_entry *p;
- p = proc_create_data(afinfo->name, S_IRUGO, net->proc_net,
+ p = proc_create_data(afinfo->name, 0444, net->proc_net,
afinfo->seq_fops, afinfo);
if (!p)
return -ENOMEM;
static struct pernet_operations ping_v4_net_ops = {
.init = ping_v4_proc_init_net,
.exit = ping_v4_proc_exit_net,
- .async = true,
};
int __init ping_proc_init(void)
static __net_init int ip_proc_init_net(struct net *net)
{
- if (!proc_create("sockstat", S_IRUGO, net->proc_net,
+ if (!proc_create("sockstat", 0444, net->proc_net,
&sockstat_seq_fops))
goto out_sockstat;
- if (!proc_create("netstat", S_IRUGO, net->proc_net, &netstat_seq_fops))
+ if (!proc_create("netstat", 0444, net->proc_net, &netstat_seq_fops))
goto out_netstat;
- if (!proc_create("snmp", S_IRUGO, net->proc_net, &snmp_seq_fops))
+ if (!proc_create("snmp", 0444, net->proc_net, &snmp_seq_fops))
goto out_snmp;
return 0;
static __net_initdata struct pernet_operations ip_proc_ops = {
.init = ip_proc_init_net,
.exit = ip_proc_exit_net,
- .async = true,
};
int __init ip_misc_proc_init(void)
/*
* Raw sockets may have direct kernel references. Kill them.
*/
- rtnl_lock();
ip_ra_control(sk, 0, NULL);
- rtnl_unlock();
sk_common_release(sk);
}
static __net_init int raw_init_net(struct net *net)
{
- if (!proc_create("raw", S_IRUGO, net->proc_net, &raw_seq_fops))
+ if (!proc_create("raw", 0444, net->proc_net, &raw_seq_fops))
return -ENOMEM;
return 0;
static __net_initdata struct pernet_operations raw_net_ops = {
.init = raw_init_net,
.exit = raw_exit_net,
- .async = true,
};
int __init raw_proc_init(void)
{
struct proc_dir_entry *pde;
- pde = proc_create("rt_cache", S_IRUGO, net->proc_net,
+ pde = proc_create("rt_cache", 0444, net->proc_net,
&rt_cache_seq_fops);
if (!pde)
goto err1;
- pde = proc_create("rt_cache", S_IRUGO,
+ pde = proc_create("rt_cache", 0444,
net->proc_net_stat, &rt_cpu_seq_fops);
if (!pde)
goto err2;
static struct pernet_operations ip_rt_proc_ops __net_initdata = {
.init = ip_rt_do_proc_init,
.exit = ip_rt_do_proc_exit,
- .async = true,
};
static int __init ip_rt_proc_init(void)
static void fill_route_from_fnhe(struct rtable *rt, struct fib_nh_exception *fnhe)
{
rt->rt_pmtu = fnhe->fnhe_pmtu;
+ rt->rt_mtu_locked = fnhe->fnhe_mtu_locked;
rt->dst.expires = fnhe->fnhe_expires;
if (fnhe->fnhe_gw) {
}
static void update_or_create_fnhe(struct fib_nh *nh, __be32 daddr, __be32 gw,
- u32 pmtu, unsigned long expires)
+ u32 pmtu, bool lock, unsigned long expires)
{
struct fnhe_hash_bucket *hash;
struct fib_nh_exception *fnhe;
fnhe->fnhe_genid = genid;
if (gw)
fnhe->fnhe_gw = gw;
- if (pmtu)
+ if (pmtu) {
fnhe->fnhe_pmtu = pmtu;
+ fnhe->fnhe_mtu_locked = lock;
+ }
fnhe->fnhe_expires = max(1UL, expires);
/* Update all cached dsts too */
rt = rcu_dereference(fnhe->fnhe_rth_input);
fnhe->fnhe_daddr = daddr;
fnhe->fnhe_gw = gw;
fnhe->fnhe_pmtu = pmtu;
+ fnhe->fnhe_mtu_locked = lock;
fnhe->fnhe_expires = expires;
/* Exception created; mark the cached routes for the nexthop
struct fib_nh *nh = &FIB_RES_NH(res);
update_or_create_fnhe(nh, fl4->daddr, new_gw,
- 0, jiffies + ip_rt_gc_timeout);
+ 0, false,
+ jiffies + ip_rt_gc_timeout);
}
if (kill_route)
rt->dst.obsolete = DST_OBSOLETE_KILL;
{
struct dst_entry *dst = &rt->dst;
struct fib_result res;
+ bool lock = false;
- if (dst_metric_locked(dst, RTAX_MTU))
+ if (ip_mtu_locked(dst))
return;
if (ipv4_mtu(dst) < mtu)
return;
- if (mtu < ip_rt_min_pmtu)
+ if (mtu < ip_rt_min_pmtu) {
+ lock = true;
mtu = ip_rt_min_pmtu;
+ }
if (rt->rt_pmtu == mtu &&
time_before(jiffies, dst->expires - ip_rt_mtu_expires / 2))
if (fib_lookup(dev_net(dst->dev), fl4, &res, 0) == 0) {
struct fib_nh *nh = &FIB_RES_NH(res);
- update_or_create_fnhe(nh, fl4->daddr, 0, mtu,
+ update_or_create_fnhe(nh, fl4->daddr, 0, mtu, lock,
jiffies + ip_rt_mtu_expires);
}
rcu_read_unlock();
mtu = READ_ONCE(dst->dev->mtu);
- if (unlikely(dst_metric_locked(dst, RTAX_MTU))) {
+ if (unlikely(ip_mtu_locked(dst))) {
if (rt->rt_uses_gateway && mtu > 576)
mtu = 576;
}
static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt_uncached_list);
-static void rt_add_uncached_list(struct rtable *rt)
+void rt_add_uncached_list(struct rtable *rt)
{
struct uncached_list *ul = raw_cpu_ptr(&rt_uncached_list);
spin_unlock_bh(&ul->lock);
}
-static void ipv4_dst_destroy(struct dst_entry *dst)
+void rt_del_uncached_list(struct rtable *rt)
{
- struct dst_metrics *p = (struct dst_metrics *)DST_METRICS_PTR(dst);
- struct rtable *rt = (struct rtable *) dst;
-
- if (p != &dst_default_metrics && refcount_dec_and_test(&p->refcnt))
- kfree(p);
-
if (!list_empty(&rt->rt_uncached)) {
struct uncached_list *ul = rt->rt_uncached_list;
}
}
+static void ipv4_dst_destroy(struct dst_entry *dst)
+{
+ struct dst_metrics *p = (struct dst_metrics *)DST_METRICS_PTR(dst);
+ struct rtable *rt = (struct rtable *)dst;
+
+ if (p != &dst_default_metrics && refcount_dec_and_test(&p->refcnt))
+ kfree(p);
+
+ rt_del_uncached_list(rt);
+}
+
void rt_flush_dev(struct net_device *dev)
{
struct net *net = dev_net(dev);
rt->rt_is_input = 0;
rt->rt_iif = 0;
rt->rt_pmtu = 0;
+ rt->rt_mtu_locked = 0;
rt->rt_gateway = 0;
rt->rt_uses_gateway = 0;
INIT_LIST_HEAD(&rt->rt_uncached);
rt->rt_is_input = ort->rt_is_input;
rt->rt_iif = ort->rt_iif;
rt->rt_pmtu = ort->rt_pmtu;
+ rt->rt_mtu_locked = ort->rt_mtu_locked;
rt->rt_genid = rt_genid_ipv4(net);
rt->rt_flags = ort->rt_flags;
memcpy(metrics, dst_metrics_ptr(&rt->dst), sizeof(metrics));
if (rt->rt_pmtu && expires)
metrics[RTAX_MTU - 1] = rt->rt_pmtu;
+ if (rt->rt_mtu_locked && expires)
+ metrics[RTAX_LOCK - 1] |= BIT(RTAX_MTU);
if (rtnetlink_put_metrics(skb, metrics) < 0)
goto nla_put_failure;
static __net_initdata struct pernet_operations sysctl_route_ops = {
.init = sysctl_route_net_init,
.exit = sysctl_route_net_exit,
- .async = true,
};
#endif
static __net_initdata struct pernet_operations rt_genid_ops = {
.init = rt_genid_init,
- .async = true,
};
static int __net_init ipv4_inetpeer_init(struct net *net)
static __net_initdata struct pernet_operations ipv4_inetpeer_ops = {
.init = ipv4_inetpeer_init,
.exit = ipv4_inetpeer_exit,
- .async = true,
};
#ifdef CONFIG_IP_ROUTE_CLASSID
static __net_initdata struct pernet_operations ipv4_sysctl_ops = {
.init = ipv4_sysctl_init_net,
.exit = ipv4_sysctl_exit_net,
- .async = true,
};
static __init int sysctl_ipv4_init(void)
get_page(page);
skb_fill_page_desc(skb, i, page, offset, copy);
}
- skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG;
+
+ if (!(flags & MSG_NO_SHARED_FRAGS))
+ skb_shinfo(skb)->tx_flags |= SKBTX_SHARED_FRAG;
skb->len += copy;
skb->data_len += copy;
bh_unlock_sock(sk);
local_bh_enable();
+ tcp_write_queue_purge(sk);
release_sock(sk);
return 0;
}
afinfo->seq_ops.next = tcp_seq_next;
afinfo->seq_ops.stop = tcp_seq_stop;
- p = proc_create_data(afinfo->name, S_IRUGO, net->proc_net,
+ p = proc_create_data(afinfo->name, 0444, net->proc_net,
afinfo->seq_fops, afinfo);
if (!p)
rc = -ENOMEM;
static struct pernet_operations tcp4_net_ops = {
.init = tcp4_proc_init_net,
.exit = tcp4_proc_exit_net,
- .async = true,
};
int __init tcp4_proc_init(void)
.init = tcp_sk_init,
.exit = tcp_sk_exit,
.exit_batch = tcp_sk_exit_batch,
- .async = true,
};
void __init tcp_v4_init(void)
static __net_initdata struct pernet_operations tcp_net_metrics_ops = {
.init = tcp_net_metrics_init,
.exit_batch = tcp_net_metrics_exit_batch,
- .async = true,
};
void __init tcp_metrics_init(void)
sk->sk_err = sk->sk_err_soft ? : ETIMEDOUT;
sk->sk_error_report(sk);
+ tcp_write_queue_purge(sk);
tcp_done(sk);
__NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONTIMEOUT);
}
afinfo->seq_ops.next = udp_seq_next;
afinfo->seq_ops.stop = udp_seq_stop;
- p = proc_create_data(afinfo->name, S_IRUGO, net->proc_net,
+ p = proc_create_data(afinfo->name, 0444, net->proc_net,
afinfo->seq_fops, afinfo);
if (!p)
rc = -ENOMEM;
static struct pernet_operations udp4_net_ops = {
.init = udp4_proc_init_net,
.exit = udp4_proc_exit_net,
- .async = true,
};
int __init udp4_proc_init(void)
}
static struct pernet_operations __net_initdata udp_sysctl_ops = {
- .init = udp_sysctl_init,
+ .init = udp_sysctl_init,
};
void __init udp_init(void)
static struct pernet_operations udplite4_net_ops = {
.init = udplite4_proc_init_net,
.exit = udplite4_proc_exit_net,
- .async = true,
};
static __init int udplite4_proc_init(void)
skb_reset_network_header(skb);
skb_mac_header_rebuild(skb);
- eth_hdr(skb)->h_proto = skb->protocol;
+ if (skb->mac_len)
+ eth_hdr(skb)->h_proto = skb->protocol;
err = 0;
xdst->u.rt.rt_gateway = rt->rt_gateway;
xdst->u.rt.rt_uses_gateway = rt->rt_uses_gateway;
xdst->u.rt.rt_pmtu = rt->rt_pmtu;
+ xdst->u.rt.rt_mtu_locked = rt->rt_mtu_locked;
INIT_LIST_HEAD(&xdst->u.rt.rt_uncached);
+ rt_add_uncached_list(&xdst->u.rt);
return 0;
}
struct xfrm_dst *xdst = (struct xfrm_dst *)dst;
dst_destroy_metrics_generic(dst);
-
+ if (xdst->u.rt.rt_uncached_list)
+ rt_del_uncached_list(&xdst->u.rt);
xfrm_dst_destroy(xdst);
}
static struct pernet_operations __net_initdata xfrm4_net_ops = {
.init = xfrm4_net_init,
.exit = xfrm4_net_exit,
- .async = true,
};
static void __init xfrm4_policy_init(void)
#include <linux/seq_file.h>
#include <linux/export.h>
-/* Set to 3 to get tracing... */
-#define ACONF_DEBUG 2
-
-#if ACONF_DEBUG >= 3
-#define ADBG(fmt, ...) printk(fmt, ##__VA_ARGS__)
-#else
-#define ADBG(fmt, ...) do { if (0) printk(fmt, ##__VA_ARGS__); } while (0)
-#endif
-
#define INFINITY_LIFE_TIME 0xFFFFFFFF
#define IPV6_MAX_STRLEN \
dev_hold(dev);
if (snmp6_alloc_dev(ndev) < 0) {
- ADBG(KERN_WARNING
- "%s: cannot allocate memory for statistics; dev=%s.\n",
- __func__, dev->name);
+ netdev_dbg(dev, "%s: cannot allocate memory for statistics\n",
+ __func__);
neigh_parms_release(&nd_tbl, ndev->nd_parms);
dev_put(dev);
kfree(ndev);
}
if (snmp6_register_dev(ndev) < 0) {
- ADBG(KERN_WARNING
- "%s: cannot create /proc/net/dev_snmp6/%s\n",
- __func__, dev->name);
+ netdev_dbg(dev, "%s: cannot create /proc/net/dev_snmp6/%s\n",
+ __func__, dev->name);
goto err_release;
}
/* Ignore adding duplicate addresses on an interface */
if (ipv6_chk_same_addr(dev_net(dev), &ifa->addr, dev, hash)) {
- ADBG("ipv6_add_addr: already assigned\n");
+ netdev_dbg(dev, "ipv6_add_addr: already assigned\n");
err = -EEXIST;
} else {
hlist_add_head_rcu(&ifa->addr_lst, &inet6_addr_lst[hash]);
ifa = kzalloc(sizeof(*ifa), gfp_flags);
if (!ifa) {
- ADBG("ipv6_add_addr: malloc failed\n");
err = -ENOBUFS;
goto out;
}
pinfo = (struct prefix_info *) opt;
if (len < sizeof(struct prefix_info)) {
- ADBG("addrconf: prefix option too short\n");
+ netdev_dbg(dev, "addrconf: prefix option too short\n");
return;
}
static int __net_init if6_proc_net_init(struct net *net)
{
- if (!proc_create("if_inet6", S_IRUGO, net->proc_net, &if6_fops))
+ if (!proc_create("if_inet6", 0444, net->proc_net, &if6_fops))
return -ENOMEM;
return 0;
}
static struct pernet_operations if6_proc_net_ops = {
.init = if6_proc_net_init,
.exit = if6_proc_net_exit,
- .async = true,
};
int __init if6_proc_init(void)
if (time_before(next_sched, jiffies + ADDRCONF_TIMER_FUZZ_MAX))
next_sched = jiffies + ADDRCONF_TIMER_FUZZ_MAX;
- ADBG(KERN_DEBUG "now = %lu, schedule = %lu, rounded schedule = %lu => %lu\n",
- now, next, next_sec, next_sched);
+ pr_debug("now = %lu, schedule = %lu, rounded schedule = %lu => %lu\n",
+ now, next, next_sec, next_sched);
mod_delayed_work(addrconf_wq, &addr_chk_work, next_sched - now);
rcu_read_unlock_bh();
}
static struct pernet_operations addrconf_ops = {
.init = addrconf_init_net,
.exit = addrconf_exit_net,
- .async = true,
};
static struct rtnl_af_ops inet6_ops __read_mostly = {
static struct pernet_operations ipv6_addr_label_ops = {
.init = ip6addrlbl_net_init,
.exit = ip6addrlbl_net_exit,
- .async = true,
};
int __init ipv6_addr_label_init(void)
static struct pernet_operations inet6_net_ops = {
.init = inet6_net_init,
.exit = inet6_net_exit,
- .async = true,
};
static const struct ipv6_stub ipv6_stub_impl = {
int __net_init ac6_proc_init(struct net *net)
{
- if (!proc_create("anycast6", S_IRUGO, net->proc_net, &ac6_seq_fops))
+ if (!proc_create("anycast6", 0444, net->proc_net, &ac6_seq_fops))
return -ENOMEM;
return 0;
struct sockaddr_in6 *usin = (struct sockaddr_in6 *) uaddr;
struct inet_sock *inet = inet_sk(sk);
struct ipv6_pinfo *np = inet6_sk(sk);
- struct in6_addr *daddr;
+ struct in6_addr *daddr, old_daddr;
+ __be32 fl6_flowlabel = 0;
+ __be32 old_fl6_flowlabel;
+ __be16 old_dport;
int addr_type;
int err;
- __be32 fl6_flowlabel = 0;
if (usin->sin6_family == AF_INET) {
if (__ipv6_only_sock(sk))
}
}
+ /* save the current peer information before updating it */
+ old_daddr = sk->sk_v6_daddr;
+ old_fl6_flowlabel = np->flow_label;
+ old_dport = inet->inet_dport;
+
sk->sk_v6_daddr = *daddr;
np->flow_label = fl6_flowlabel;
-
inet->inet_dport = usin->sin6_port;
/*
err = ip6_datagram_dst_update(sk, true);
if (err) {
- /* Reset daddr and dport so that udp_v6_early_demux()
- * fails to find this socket
+ /* Restore the socket peer info, to keep it consistent with
+ * the old socket state
*/
- memset(&sk->sk_v6_daddr, 0, sizeof(sk->sk_v6_daddr));
- inet->inet_dport = 0;
+ sk->sk_v6_daddr = old_daddr;
+ np->flow_label = old_fl6_flowlabel;
+ inet->inet_dport = old_dport;
goto out;
}
if (!(features & NETIF_F_HW_ESP) || !x->xso.offload_handle ||
(x->xso.dev != skb->dev))
esp_features = features & ~(NETIF_F_SG | NETIF_F_CSUM_MASK);
+ else if (!(features & NETIF_F_HW_ESP_TX_CSUM))
+ esp_features = features & ~NETIF_F_CSUM_MASK;
xo->flags |= XFRM_GSO_SEGMENT;
static struct pernet_operations fib6_rules_net_ops = {
.init = fib6_rules_net_init,
.exit = fib6_rules_net_exit,
- .async = true,
};
int __init fib6_rules_init(void)
static struct pernet_operations icmpv6_sk_ops = {
.init = icmpv6_sk_init,
.exit = icmpv6_sk_exit,
- .async = true,
};
int __init icmpv6_init(void)
.exit = ila_exit_net,
.id = &ila_net_id,
.size = sizeof(struct ila_net),
- .async = true,
};
static int ila_xlat_addr(struct sk_buff *skb, bool sir2ila)
if (err)
return err;
+ err = call_fib6_entry_notifiers(info->nl_net,
+ FIB_EVENT_ENTRY_ADD,
+ rt, extack);
+ if (err)
+ return err;
+
rcu_assign_pointer(rt->rt6_next, iter);
atomic_inc(&rt->rt6i_ref);
rcu_assign_pointer(rt->rt6i_node, fn);
rcu_assign_pointer(*ins, rt);
- call_fib6_entry_notifiers(info->nl_net, FIB_EVENT_ENTRY_ADD,
- rt, extack);
if (!info->skip_notify)
inet6_rt_notify(RTM_NEWROUTE, rt, info, nlflags);
info->nl_net->ipv6.rt6_stats->fib_rt_entries++;
if (err)
return err;
+ err = call_fib6_entry_notifiers(info->nl_net,
+ FIB_EVENT_ENTRY_REPLACE,
+ rt, extack);
+ if (err)
+ return err;
+
atomic_inc(&rt->rt6i_ref);
rcu_assign_pointer(rt->rt6i_node, fn);
rt->rt6_next = iter->rt6_next;
rcu_assign_pointer(*ins, rt);
- call_fib6_entry_notifiers(info->nl_net, FIB_EVENT_ENTRY_REPLACE,
- rt, extack);
if (!info->skip_notify)
inet6_rt_notify(RTM_NEWROUTE, rt, info, NLM_F_REPLACE);
if (!(fn->fn_flags & RTN_RTINFO)) {
static struct pernet_operations fib6_net_ops = {
.init = fib6_net_init,
.exit = fib6_net_exit,
- .async = true,
};
int __init fib6_init(void)
static int __net_init ip6_flowlabel_proc_init(struct net *net)
{
- if (!proc_create("ip6_flowlabel", S_IRUGO, net->proc_net,
+ if (!proc_create("ip6_flowlabel", 0444, net->proc_net,
&ip6fl_seq_fops))
return -ENOMEM;
return 0;
static struct pernet_operations ip6_flowlabel_net_ops = {
.init = ip6_flowlabel_proc_init,
.exit = ip6_flowlabel_net_exit,
- .async = true,
};
int ip6_flowlabel_init(void)
struct ip6_tnl *t, *cand = NULL;
struct ip6gre_net *ign = net_generic(net, ip6gre_net_id);
int dev_type = (gre_proto == htons(ETH_P_TEB) ||
- gre_proto == htons(ETH_P_ERSPAN)) ?
+ gre_proto == htons(ETH_P_ERSPAN) ||
+ gre_proto == htons(ETH_P_ERSPAN2)) ?
ARPHRD_ETHER : ARPHRD_IP6GRE;
int score, cand_score = 4;
gre_build_header(skb, tunnel->tun_hlen,
flags, protocol,
tunnel_id_to_key32(tun_info->key.tun_id),
- (flags | TUNNEL_SEQ) ? htonl(tunnel->o_seqno++)
+ (flags & TUNNEL_SEQ) ? htonl(tunnel->o_seqno++)
: 0);
} else {
truncate = true;
}
+ if (skb_cow_head(skb, dev->needed_headroom))
+ goto tx_err;
+
t->parms.o_flags &= ~TUNNEL_KEY;
IPCB(skb)->flags = 0;
md->u.md2.dir,
get_hwid(&md->u.md2),
truncate, false);
+ } else {
+ goto tx_err;
}
} else {
switch (skb->protocol) {
.exit_batch = ip6gre_exit_batch_net,
.id = &ip6gre_net_id,
.size = sizeof(struct ip6gre_net),
- .async = true,
};
static int ip6gre_tunnel_validate(struct nlattr *tb[], struct nlattr *data[],
.exit_batch = ip6_tnl_exit_batch_net,
.id = &ip6_tnl_net_id,
.size = sizeof(struct ip6_tnl_net),
- .async = true,
};
/**
.exit_batch = vti6_exit_batch_net,
.id = &vti6_net_id,
.size = sizeof(struct vti6_net),
- .async = true,
};
static struct xfrm6_protocol vti_esp6_protocol __read_mostly = {
fib_rules_unregister(net->ipv6.mr6_rules_ops);
rtnl_unlock();
}
+
+static int ip6mr_rules_dump(struct net *net, struct notifier_block *nb)
+{
+ return fib_rules_dump(net, nb, RTNL_FAMILY_IP6MR);
+}
+
+static unsigned int ip6mr_rules_seq_read(struct net *net)
+{
+ return fib_rules_seq_read(net, RTNL_FAMILY_IP6MR);
+}
+
+bool ip6mr_rule_default(const struct fib_rule *rule)
+{
+ return fib_rule_matchall(rule) && rule->action == FR_ACT_TO_TBL &&
+ rule->table == RT6_TABLE_DFLT && !rule->l3mdev;
+}
+EXPORT_SYMBOL(ip6mr_rule_default);
#else
#define ip6mr_for_each_table(mrt, net) \
for (mrt = net->ipv6.mrt6; mrt; mrt = NULL)
net->ipv6.mrt6 = NULL;
rtnl_unlock();
}
+
+static int ip6mr_rules_dump(struct net *net, struct notifier_block *nb)
+{
+ return 0;
+}
+
+static unsigned int ip6mr_rules_seq_read(struct net *net)
+{
+ return 0;
+}
#endif
static int ip6mr_hash_cmp(struct rhashtable_compare_arg *arg,
}
#endif
-/*
- * Delete a VIF entry
- */
+static int call_ip6mr_vif_entry_notifiers(struct net *net,
+ enum fib_event_type event_type,
+ struct vif_device *vif,
+ mifi_t vif_index, u32 tb_id)
+{
+ return mr_call_vif_notifiers(net, RTNL_FAMILY_IP6MR, event_type,
+ vif, vif_index, tb_id,
+ &net->ipv6.ipmr_seq);
+}
+static int call_ip6mr_mfc_entry_notifiers(struct net *net,
+ enum fib_event_type event_type,
+ struct mfc6_cache *mfc, u32 tb_id)
+{
+ return mr_call_mfc_notifiers(net, RTNL_FAMILY_IP6MR, event_type,
+ &mfc->_c, tb_id, &net->ipv6.ipmr_seq);
+}
+
+/* Delete a VIF entry */
static int mif6_delete(struct mr_table *mrt, int vifi, int notify,
struct list_head *head)
{
v = &mrt->vif_table[vifi];
+ if (VIF_EXISTS(mrt, vifi))
+ call_ip6mr_vif_entry_notifiers(read_pnet(&mrt->net),
+ FIB_EVENT_VIF_DEL, v, vifi,
+ mrt->id);
+
write_lock_bh(&mrt_lock);
dev = v->dev;
v->dev = NULL;
if (vifi + 1 > mrt->maxvif)
mrt->maxvif = vifi + 1;
write_unlock_bh(&mrt_lock);
+ call_ip6mr_vif_entry_notifiers(net, FIB_EVENT_VIF_ADD,
+ v, vifi, mrt->id);
return 0;
}
return NULL;
c->_c.mfc_un.res.last_assert = jiffies - MFC_ASSERT_THRESH - 1;
c->_c.mfc_un.res.minvif = MAXMIFS;
+ c->_c.free = ip6mr_cache_free_rcu;
+ refcount_set(&c->_c.mfc_un.res.refcount, 1);
return c;
}
rhltable_remove(&mrt->mfc_hash, &c->_c.mnode, ip6mr_rht_params);
list_del_rcu(&c->_c.list);
+ call_ip6mr_mfc_entry_notifiers(read_pnet(&mrt->net),
+ FIB_EVENT_ENTRY_DEL, c, mrt->id);
mr6_netlink_event(mrt, c, RTM_DELROUTE);
- ip6mr_cache_free(c);
+ mr_cache_put(&c->_c);
return 0;
}
return NOTIFY_DONE;
}
+static unsigned int ip6mr_seq_read(struct net *net)
+{
+ ASSERT_RTNL();
+
+ return net->ipv6.ipmr_seq + ip6mr_rules_seq_read(net);
+}
+
+static int ip6mr_dump(struct net *net, struct notifier_block *nb)
+{
+ return mr_dump(net, nb, RTNL_FAMILY_IP6MR, ip6mr_rules_dump,
+ ip6mr_mr_table_iter, &mrt_lock);
+}
+
static struct notifier_block ip6_mr_notifier = {
.notifier_call = ip6mr_device_event
};
-/*
- * Setup for IP multicast routing
- */
+static const struct fib_notifier_ops ip6mr_notifier_ops_template = {
+ .family = RTNL_FAMILY_IP6MR,
+ .fib_seq_read = ip6mr_seq_read,
+ .fib_dump = ip6mr_dump,
+ .owner = THIS_MODULE,
+};
+
+static int __net_init ip6mr_notifier_init(struct net *net)
+{
+ struct fib_notifier_ops *ops;
+
+ net->ipv6.ipmr_seq = 0;
+ ops = fib_notifier_ops_register(&ip6mr_notifier_ops_template, net);
+ if (IS_ERR(ops))
+ return PTR_ERR(ops);
+
+ net->ipv6.ip6mr_notifier_ops = ops;
+
+ return 0;
+}
+
+static void __net_exit ip6mr_notifier_exit(struct net *net)
+{
+ fib_notifier_ops_unregister(net->ipv6.ip6mr_notifier_ops);
+ net->ipv6.ip6mr_notifier_ops = NULL;
+}
+
+/* Setup for IP multicast routing */
static int __net_init ip6mr_net_init(struct net *net)
{
int err;
+ err = ip6mr_notifier_init(net);
+ if (err)
+ return err;
+
err = ip6mr_rules_init(net);
if (err < 0)
- goto fail;
+ goto ip6mr_rules_fail;
#ifdef CONFIG_PROC_FS
err = -ENOMEM;
proc_vif_fail:
ip6mr_rules_exit(net);
#endif
-fail:
+ip6mr_rules_fail:
+ ip6mr_notifier_exit(net);
return err;
}
remove_proc_entry("ip6_mr_vif", net->proc_net);
#endif
ip6mr_rules_exit(net);
+ ip6mr_notifier_exit(net);
}
static struct pernet_operations ip6mr_net_ops = {
.init = ip6mr_net_init,
.exit = ip6mr_net_exit,
- .async = true,
};
int __init ip6_mr_init(void)
if (!mrtsock)
c->_c.mfc_flags |= MFC_STATIC;
write_unlock_bh(&mrt_lock);
+ call_ip6mr_mfc_entry_notifiers(net, FIB_EVENT_ENTRY_REPLACE,
+ c, mrt->id);
mr6_netlink_event(mrt, c, RTM_NEWROUTE);
return 0;
}
ip6mr_cache_resolve(net, mrt, uc, c);
ip6mr_cache_free(uc);
}
+ call_ip6mr_mfc_entry_notifiers(net, FIB_EVENT_ENTRY_ADD,
+ c, mrt->id);
mr6_netlink_event(mrt, c, RTM_NEWROUTE);
return 0;
}
rhltable_remove(&mrt->mfc_hash, &c->mnode, ip6mr_rht_params);
list_del_rcu(&c->list);
mr6_netlink_event(mrt, (struct mfc6_cache *)c, RTM_DELROUTE);
- ip6mr_cache_free((struct mfc6_cache *)c);
+ mr_cache_put(c);
}
if (atomic_read(&mrt->cache_resolve_queue_len) != 0) {
spin_lock_bh(&mfc_unres_lock);
list_for_each_entry_safe(c, tmp, &mrt->mfc_unres_queue, list) {
list_del(&c->list);
+ call_ip6mr_mfc_entry_notifiers(read_pnet(&mrt->net),
+ FIB_EVENT_ENTRY_DEL,
+ (struct mfc6_cache *)c,
+ mrt->id);
mr6_netlink_event(mrt, (struct mfc6_cache *)c,
RTM_DELROUTE);
ip6mr_destroy_unres(mrt, (struct mfc6_cache *)c);
int err;
err = -ENOMEM;
- if (!proc_create("igmp6", S_IRUGO, net->proc_net, &igmp6_mc_seq_fops))
+ if (!proc_create("igmp6", 0444, net->proc_net, &igmp6_mc_seq_fops))
goto out;
- if (!proc_create("mcfilter6", S_IRUGO, net->proc_net,
+ if (!proc_create("mcfilter6", 0444, net->proc_net,
&igmp6_mcf_seq_fops))
goto out_proc_net_igmp6;
static struct pernet_operations igmp6_net_ops = {
.init = igmp6_net_init,
.exit = igmp6_net_exit,
- .async = true,
};
int __init igmp6_init(void)
*(opt++) = (rd_len >> 3);
opt += 6;
- memcpy(opt, ipv6_hdr(orig_skb), rd_len - 8);
+ skb_copy_bits(orig_skb, skb_network_offset(orig_skb), opt,
+ rd_len - 8);
}
void ndisc_send_redirect(struct sk_buff *skb, const struct in6_addr *target)
static struct pernet_operations ndisc_net_ops = {
.init = ndisc_net_init,
.exit = ndisc_net_exit,
- .async = true,
};
int __init ndisc_init(void)
static struct pernet_operations ip6_tables_net_ops = {
.init = ip6_tables_net_init,
.exit = ip6_tables_net_exit,
- .async = true,
};
static int __init ip6_tables_init(void)
static struct pernet_operations ip6table_filter_net_ops = {
.init = ip6table_filter_net_init,
.exit = ip6table_filter_net_exit,
- .async = true,
};
static int __init ip6table_filter_init(void)
static struct pernet_operations ip6table_mangle_net_ops = {
.exit = ip6table_mangle_net_exit,
- .async = true,
};
static int __init ip6table_mangle_init(void)
static struct pernet_operations ip6table_nat_net_ops = {
.exit = ip6table_nat_net_exit,
- .async = true,
};
static int __init ip6table_nat_init(void)
static struct pernet_operations ip6table_raw_net_ops = {
.exit = ip6table_raw_net_exit,
- .async = true,
};
static int __init ip6table_raw_init(void)
static struct pernet_operations ip6table_security_net_ops = {
.exit = ip6table_security_net_exit,
- .async = true,
};
static int __init ip6table_security_init(void)
.exit = ipv6_net_exit,
.id = &conntrack6_net_id,
.size = sizeof(struct conntrack6_net),
- .async = true,
};
static int __init nf_conntrack_l3proto_ipv6_init(void)
static struct pernet_operations defrag6_net_ops = {
.exit = defrag6_net_exit,
- .async = true,
};
static int __init nf_defrag_init(void)
static struct pernet_operations nf_log_ipv6_net_ops = {
.init = nf_log_ipv6_net_init,
.exit = nf_log_ipv6_net_exit,
- .async = true,
};
static int __init nf_log_ipv6_init(void)
static struct pernet_operations ping_v6_net_ops = {
.init = ping_v6_proc_init_net,
.exit = ping_v6_proc_exit_net,
- .async = true,
};
#endif
if (!net->mib.proc_net_devsnmp6)
return -ENOENT;
- p = proc_create_data(idev->dev->name, S_IRUGO,
+ p = proc_create_data(idev->dev->name, 0444,
net->mib.proc_net_devsnmp6,
&snmp6_dev_seq_fops, idev);
if (!p)
static int __net_init ipv6_proc_init_net(struct net *net)
{
- if (!proc_create("sockstat6", S_IRUGO, net->proc_net,
+ if (!proc_create("sockstat6", 0444, net->proc_net,
&sockstat6_seq_fops))
return -ENOMEM;
- if (!proc_create("snmp6", S_IRUGO, net->proc_net, &snmp6_seq_fops))
+ if (!proc_create("snmp6", 0444, net->proc_net, &snmp6_seq_fops))
goto proc_snmp6_fail;
net->mib.proc_net_devsnmp6 = proc_mkdir("dev_snmp6", net->proc_net);
static struct pernet_operations ipv6_proc_ops = {
.init = ipv6_proc_init_net,
.exit = ipv6_proc_exit_net,
- .async = true,
};
int __init ipv6_misc_proc_init(void)
static int __net_init raw6_init_net(struct net *net)
{
- if (!proc_create("raw6", S_IRUGO, net->proc_net, &raw6_seq_fops))
+ if (!proc_create("raw6", 0444, net->proc_net, &raw6_seq_fops))
return -ENOMEM;
return 0;
static struct pernet_operations raw6_net_ops = {
.init = raw6_init_net,
.exit = raw6_exit_net,
- .async = true,
};
int __init raw6_proc_init(void)
table[1].data = &net->ipv6.frags.low_thresh;
table[1].extra2 = &net->ipv6.frags.high_thresh;
table[2].data = &net->ipv6.frags.timeout;
-
- /* Don't export sysctls to unprivileged users */
- if (net->user_ns != &init_user_ns)
- table[0].procname = NULL;
}
hdr = register_net_sysctl(net, "net/ipv6", table);
static struct pernet_operations ip6_frags_ops = {
.init = ipv6_frags_init_net,
.exit = ipv6_frags_exit_net,
- .async = true,
};
int __init ipv6_frag_init(void)
static DEFINE_PER_CPU_ALIGNED(struct uncached_list, rt6_uncached_list);
-static void rt6_uncached_list_add(struct rt6_info *rt)
+void rt6_uncached_list_add(struct rt6_info *rt)
{
struct uncached_list *ul = raw_cpu_ptr(&rt6_uncached_list);
spin_unlock_bh(&ul->lock);
}
-static void rt6_uncached_list_del(struct rt6_info *rt)
+void rt6_uncached_list_del(struct rt6_info *rt)
{
if (!list_empty(&rt->rt6i_uncached)) {
struct uncached_list *ul = rt->rt6i_uncached_list;
}
}
-static void rt6_exceptions_update_pmtu(struct rt6_info *rt, int mtu)
+static bool rt6_mtu_change_route_allowed(struct inet6_dev *idev,
+ struct rt6_info *rt, int mtu)
+{
+ /* If the new MTU is lower than the route PMTU, this new MTU will be the
+ * lowest MTU in the path: always allow updating the route PMTU to
+ * reflect PMTU decreases.
+ *
+ * If the new MTU is higher, and the route PMTU is equal to the local
+ * MTU, this means the old MTU is the lowest in the path, so allow
+ * updating it: if other nodes now have lower MTUs, PMTU discovery will
+ * handle this.
+ */
+
+ if (dst_mtu(&rt->dst) >= mtu)
+ return true;
+
+ if (dst_mtu(&rt->dst) == idev->cnf.mtu6)
+ return true;
+
+ return false;
+}
+
+static void rt6_exceptions_update_pmtu(struct inet6_dev *idev,
+ struct rt6_info *rt, int mtu)
{
struct rt6_exception_bucket *bucket;
struct rt6_exception *rt6_ex;
bucket = rcu_dereference_protected(rt->rt6i_exception_bucket,
lockdep_is_held(&rt6_exception_lock));
- if (bucket) {
- for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
- hlist_for_each_entry(rt6_ex, &bucket->chain, hlist) {
- struct rt6_info *entry = rt6_ex->rt6i;
- /* For RTF_CACHE with rt6i_pmtu == 0
- * (i.e. a redirected route),
- * the metrics of its rt->dst.from has already
- * been updated.
- */
- if (entry->rt6i_pmtu && entry->rt6i_pmtu > mtu)
- entry->rt6i_pmtu = mtu;
- }
- bucket++;
+ if (!bucket)
+ return;
+
+ for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
+ hlist_for_each_entry(rt6_ex, &bucket->chain, hlist) {
+ struct rt6_info *entry = rt6_ex->rt6i;
+
+ /* For RTF_CACHE with rt6i_pmtu == 0 (i.e. a redirected
+ * route), the metrics of its rt->dst.from have already
+ * been updated.
+ */
+ if (entry->rt6i_pmtu &&
+ rt6_mtu_change_route_allowed(idev, entry, mtu))
+ entry->rt6i_pmtu = mtu;
}
+ bucket++;
}
}
Since RFC 1981 doesn't include administrative MTU increase
update PMTU increase is a MUST. (i.e. jumbo frame)
*/
- /*
- If new MTU is less than route PMTU, this new MTU will be the
- lowest MTU in the path, update the route PMTU to reflect PMTU
- decreases; if new MTU is greater than route PMTU, and the
- old MTU is the lowest MTU in the path, update the route PMTU
- to reflect the increase. In this case if the other nodes' MTU
- also have the lowest MTU, TOO BIG MESSAGE will be lead to
- PMTU discovery.
- */
if (rt->dst.dev == arg->dev &&
- dst_metric_raw(&rt->dst, RTAX_MTU) &&
!dst_metric_locked(&rt->dst, RTAX_MTU)) {
spin_lock_bh(&rt6_exception_lock);
- if (dst_mtu(&rt->dst) >= arg->mtu ||
- (dst_mtu(&rt->dst) < arg->mtu &&
- dst_mtu(&rt->dst) == idev->cnf.mtu6)) {
+ if (dst_metric_raw(&rt->dst, RTAX_MTU) &&
+ rt6_mtu_change_route_allowed(idev, rt, arg->mtu))
dst_metric_set(&rt->dst, RTAX_MTU, arg->mtu);
- }
- rt6_exceptions_update_pmtu(rt, arg->mtu);
+ rt6_exceptions_update_pmtu(idev, rt, arg->mtu);
spin_unlock_bh(&rt6_exception_lock);
}
return 0;
r_cfg.fc_encap_type = nla_get_u16(nla);
}
+ r_cfg.fc_flags |= (rtnh->rtnh_flags & RTNH_F_ONLINK);
rt = ip6_route_info_create(&r_cfg, extack);
if (IS_ERR(rt)) {
err = PTR_ERR(rt);
{
#ifdef CONFIG_PROC_FS
proc_create("ipv6_route", 0, net->proc_net, &ipv6_route_proc_fops);
- proc_create("rt6_stats", S_IRUGO, net->proc_net, &rt6_stats_seq_fops);
+ proc_create("rt6_stats", 0444, net->proc_net, &rt6_stats_seq_fops);
#endif
return 0;
}
static struct pernet_operations ip6_route_net_ops = {
.init = ip6_route_net_init,
.exit = ip6_route_net_exit,
- .async = true,
};
static int __net_init ipv6_inetpeer_init(struct net *net)
static struct pernet_operations ipv6_inetpeer_ops = {
.init = ipv6_inetpeer_init,
.exit = ipv6_inetpeer_exit,
- .async = true,
};
static struct pernet_operations ip6_route_net_late_ops = {
.init = ip6_route_net_init_late,
.exit = ip6_route_net_exit_late,
- .async = true,
};
static struct notifier_block ip6_route_dev_notifier = {
static struct pernet_operations ip6_segments_ops = {
.init = seg6_net_init,
.exit = seg6_net_exit,
- .async = true,
};
static const struct genl_ops seg6_genl_ops[] = {
/* encapsulate an IPv6 packet within an outer IPv6 header with a given SRH */
int seg6_do_srh_encap(struct sk_buff *skb, struct ipv6_sr_hdr *osrh, int proto)
{
- struct net *net = dev_net(skb_dst(skb)->dev);
+ struct dst_entry *dst = skb_dst(skb);
+ struct net *net = dev_net(dst->dev);
struct ipv6hdr *hdr, *inner_hdr;
struct ipv6_sr_hdr *isrh;
int hdrlen, tot_len, err;
isrh->nexthdr = proto;
hdr->daddr = isrh->segments[isrh->first_segment];
- set_tun_src(net, skb->dev, &hdr->daddr, &hdr->saddr);
+ set_tun_src(net, ip6_dst_idev(dst)->dev, &hdr->daddr, &hdr->saddr);
#ifdef CONFIG_IPV6_SEG6_HMAC
if (sr_has_hmac(isrh)) {
slwt = seg6_lwt_lwtunnel(newts);
- err = dst_cache_init(&slwt->cache, GFP_KERNEL);
+ err = dst_cache_init(&slwt->cache, GFP_ATOMIC);
if (err) {
kfree(newts);
return err;
.exit_batch = sit_exit_batch_net,
.id = &sit_net_id,
.size = sizeof(struct sit_net),
- .async = true,
};
static void __exit sit_cleanup(void)
static struct pernet_operations ipv6_sysctl_net_ops = {
.init = ipv6_sysctl_net_init,
.exit = ipv6_sysctl_net_exit,
- .async = true,
};
static struct ctl_table_header *ip6_header;
.init = tcpv6_net_init,
.exit = tcpv6_net_exit,
.exit_batch = tcpv6_net_exit_batch,
- .async = true,
};
int __init tcpv6_init(void)
static struct pernet_operations udplite6_net_ops = {
.init = udplite6_proc_init_net,
.exit = udplite6_proc_exit_net,
- .async = true,
};
int __init udplite6_proc_init(void)
skb_reset_network_header(skb);
skb_mac_header_rebuild(skb);
- eth_hdr(skb)->h_proto = skb->protocol;
+ if (skb->mac_len)
+ eth_hdr(skb)->h_proto = skb->protocol;
err = 0;
xdst->u.rt6.rt6i_gateway = rt->rt6i_gateway;
xdst->u.rt6.rt6i_dst = rt->rt6i_dst;
xdst->u.rt6.rt6i_src = rt->rt6i_src;
+ INIT_LIST_HEAD(&xdst->u.rt6.rt6i_uncached);
+ rt6_uncached_list_add(&xdst->u.rt6);
+ atomic_inc(&dev_net(dev)->ipv6.rt6_stats->fib_rt_uncache);
return 0;
}
if (likely(xdst->u.rt6.rt6i_idev))
in6_dev_put(xdst->u.rt6.rt6i_idev);
dst_destroy_metrics_generic(dst);
+ if (xdst->u.rt6.rt6i_uncached_list)
+ rt6_uncached_list_del(&xdst->u.rt6);
xfrm_dst_destroy(xdst);
}
static struct pernet_operations xfrm6_net_ops = {
.init = xfrm6_net_init,
.exit = xfrm6_net_exit,
- .async = true,
};
int __init xfrm6_init(void)
.exit = xfrm6_tunnel_net_exit,
.id = &xfrm6_tunnel_net_id,
.size = sizeof(struct xfrm6_tunnel_net),
- .async = true,
};
static int __init xfrm6_tunnel_init(void)
af_iucv_dev->driver = &af_iucv_driver;
err = device_register(af_iucv_dev);
if (err)
- goto out_driver;
+ goto out_iucv_dev;
return 0;
+out_iucv_dev:
+ put_device(af_iucv_dev);
out_driver:
driver_unregister(&af_iucv_driver);
out_iucv:
struct proc_dir_entry *p;
int rc = 0;
- p = proc_create_data(muxinfo->name, S_IRUGO, net->proc_net,
+ p = proc_create_data(muxinfo->name, 0444, net->proc_net,
muxinfo->seq_fops, muxinfo);
if (!p)
rc = -ENOMEM;
{
int err;
- if (!proc_create("kcm_stats", S_IRUGO, net->proc_net,
+ if (!proc_create("kcm_stats", 0444, net->proc_net,
&kcm_stats_seq_fops)) {
err = -ENOMEM;
goto out_kcm_stats;
static struct pernet_operations kcm_net_ops = {
.init = kcm_proc_init_net,
.exit = kcm_proc_exit_net,
- .async = true,
};
int __init kcm_proc_init(void)
.parse_msg = kcm_parse_func_strparser,
.read_sock_done = kcm_read_sock_done,
};
- int err;
+ int err = 0;
csk = csock->sk;
if (!csk)
return -EINVAL;
+ lock_sock(csk);
+
/* Only allow TCP sockets to be attached for now */
if ((csk->sk_family != AF_INET && csk->sk_family != AF_INET6) ||
- csk->sk_protocol != IPPROTO_TCP)
- return -EOPNOTSUPP;
+ csk->sk_protocol != IPPROTO_TCP) {
+ err = -EOPNOTSUPP;
+ goto out;
+ }
/* Don't allow listeners or closed sockets */
- if (csk->sk_state == TCP_LISTEN || csk->sk_state == TCP_CLOSE)
- return -EOPNOTSUPP;
+ if (csk->sk_state == TCP_LISTEN || csk->sk_state == TCP_CLOSE) {
+ err = -EOPNOTSUPP;
+ goto out;
+ }
psock = kmem_cache_zalloc(kcm_psockp, GFP_KERNEL);
- if (!psock)
- return -ENOMEM;
+ if (!psock) {
+ err = -ENOMEM;
+ goto out;
+ }
psock->mux = mux;
psock->sk = csk;
err = strp_init(&psock->strp, csk, &cb);
if (err) {
kmem_cache_free(kcm_psockp, psock);
- return err;
+ goto out;
}
write_lock_bh(&csk->sk_callback_lock);
strp_stop(&psock->strp);
strp_done(&psock->strp);
kmem_cache_free(kcm_psockp, psock);
- return -EALREADY;
+ err = -EALREADY;
+ goto out;
}
psock->save_data_ready = csk->sk_data_ready;
/* Schedule RX work in case there are already bytes queued */
strp_check_rcv(&psock->strp);
- return 0;
+out:
+ release_sock(csk);
+
+ return err;
}
static int kcm_attach_ioctl(struct socket *sock, struct kcm_attach *info)
if (WARN_ON(psock->rx_kcm)) {
write_unlock_bh(&csk->sk_callback_lock);
+ release_sock(csk);
return;
}
.exit = kcm_exit_net,
.id = &kcm_net_id,
.size = sizeof(struct kcm_net),
- .async = true,
};
static int __init kcm_init(void)
.exit = pfkey_net_exit,
.id = &pfkey_net_id,
.size = sizeof(struct netns_pfkey),
- .async = true,
};
static void __exit ipsec_pfkey_exit(void)
spinlock_t l2tp_session_hlist_lock;
};
+#if IS_ENABLED(CONFIG_IPV6)
+static bool l2tp_sk_is_v6(struct sock *sk)
+{
+ return sk->sk_family == PF_INET6 &&
+ !ipv6_addr_v4mapped(&sk->sk_v6_daddr);
+}
+#endif
static inline struct l2tp_tunnel *l2tp_tunnel(struct sock *sk)
{
/* Queue the packet to IP for output */
skb->ignore_df = 1;
#if IS_ENABLED(CONFIG_IPV6)
- if (tunnel->sock->sk_family == PF_INET6 && !tunnel->v4mapped)
+ if (l2tp_sk_is_v6(tunnel->sock))
error = inet6_csk_xmit(tunnel->sock, skb, NULL);
else
#endif
goto out_unlock;
}
+ /* The user-space may change the connection status for the user-space
+ * provided socket at run time: we must check it under the socket lock
+ */
+ if (tunnel->fd >= 0 && sk->sk_state != TCP_ESTABLISHED) {
+ kfree_skb(skb);
+ ret = NET_XMIT_DROP;
+ goto out_unlock;
+ }
+
/* Get routing info from the tunnel socket */
skb_dst_drop(skb);
skb_dst_set(skb, dst_clone(__sk_dst_check(sk, 0)));
/* Calculate UDP checksum if configured to do so */
#if IS_ENABLED(CONFIG_IPV6)
- if (sk->sk_family == PF_INET6 && !tunnel->v4mapped)
+ if (l2tp_sk_is_v6(sk))
udp6_set_csum(udp_get_no_check6_tx(sk),
skb, &inet6_sk(sk)->saddr,
&sk->sk_v6_daddr, udp_len);
encap = cfg->encap;
/* Quick sanity checks */
+ err = -EPROTONOSUPPORT;
+ if (sk->sk_type != SOCK_DGRAM) {
+ pr_debug("tunl %hu: fd %d wrong socket type\n",
+ tunnel_id, fd);
+ goto err;
+ }
switch (encap) {
case L2TP_ENCAPTYPE_UDP:
- err = -EPROTONOSUPPORT;
if (sk->sk_protocol != IPPROTO_UDP) {
pr_err("tunl %hu: fd %d wrong protocol, got %d, expected %d\n",
tunnel_id, fd, sk->sk_protocol, IPPROTO_UDP);
}
break;
case L2TP_ENCAPTYPE_IP:
- err = -EPROTONOSUPPORT;
if (sk->sk_protocol != IPPROTO_L2TP) {
pr_err("tunl %hu: fd %d wrong protocol, got %d, expected %d\n",
tunnel_id, fd, sk->sk_protocol, IPPROTO_L2TP);
if (cfg != NULL)
tunnel->debug = cfg->debug;
-#if IS_ENABLED(CONFIG_IPV6)
- if (sk->sk_family == PF_INET6) {
- struct ipv6_pinfo *np = inet6_sk(sk);
-
- if (ipv6_addr_v4mapped(&np->saddr) &&
- ipv6_addr_v4mapped(&sk->sk_v6_daddr)) {
- struct inet_sock *inet = inet_sk(sk);
-
- tunnel->v4mapped = true;
- inet->inet_saddr = np->saddr.s6_addr32[3];
- inet->inet_rcv_saddr = sk->sk_v6_rcv_saddr.s6_addr32[3];
- inet->inet_daddr = sk->sk_v6_daddr.s6_addr32[3];
- } else {
- tunnel->v4mapped = false;
- }
- }
-#endif
-
/* Mark socket as an encapsulation socket. See net/ipv4/udp.c */
tunnel->encap = encap;
if (encap == L2TP_ENCAPTYPE_UDP) {
.exit = l2tp_exit_net,
.id = &l2tp_net_id,
.size = sizeof(struct l2tp_net),
- .async = true,
};
static int __init l2tp_init(void)
struct sock *sock; /* Parent socket */
int fd; /* Parent fd, if tunnel socket
* was created by userspace */
-#if IS_ENABLED(CONFIG_IPV6)
- bool v4mapped;
-#endif
struct work_struct del_work;
struct proc_dir_entry *pde;
int err = 0;
- pde = proc_create("pppol2tp", S_IRUGO, net->proc_net,
+ pde = proc_create("pppol2tp", 0444, net->proc_net,
&pppol2tp_proc_fops);
if (!pde) {
err = -ENOMEM;
.init = pppol2tp_init_net,
.exit = pppol2tp_exit_net,
.id = &pppol2tp_net_id,
- .async = true,
};
/*****************************************************************************
if (!llc_proc_dir)
goto out;
- p = proc_create("socket", S_IRUGO, llc_proc_dir, &llc_seq_socket_fops);
+ p = proc_create("socket", 0444, llc_proc_dir, &llc_seq_socket_fops);
if (!p)
goto out_socket;
- p = proc_create("core", S_IRUGO, llc_proc_dir, &llc_seq_core_fops);
+ p = proc_create("core", 0444, llc_proc_dir, &llc_seq_core_fops);
if (!p)
goto out_core;
FLAG(SUPPORTS_TX_FRAG),
FLAG(SUPPORTS_TDLS_BUFFER_STA),
FLAG(DEAUTH_NEED_MGD_TX_PREP),
+ FLAG(DOESNT_SUPPORT_QOS_NDP),
#undef FLAG
};
struct ieee80211_hdr_3addr *nullfunc;
struct ieee80211_if_managed *ifmgd = &sdata->u.mgd;
- skb = ieee80211_nullfunc_get(&local->hw, &sdata->vif, true);
+ skb = ieee80211_nullfunc_get(&local->hw, &sdata->vif,
+ !ieee80211_hw_check(&local->hw, DOESNT_SUPPORT_QOS_NDP));
if (!skb)
return;
#ifdef CONFIG_MAC80211_DEBUGFS
mp->fixed_rate_idx = (u32) -1;
mp->dbg_fixed_rate = debugfs_create_u32("fixed_rate_idx",
- S_IRUGO | S_IWUGO, debugfsdir, &mp->fixed_rate_idx);
+ 0666, debugfsdir, &mp->fixed_rate_idx);
#endif
minstrel_init_cck_rates(mp);
{
struct minstrel_sta_info *mi = priv_sta;
- mi->dbg_stats = debugfs_create_file("rc_stats", S_IRUGO, dir, mi,
- &minstrel_stat_fops);
+ mi->dbg_stats = debugfs_create_file("rc_stats", 0444, dir, mi,
+ &minstrel_stat_fops);
- mi->dbg_stats_csv = debugfs_create_file("rc_stats_csv", S_IRUGO, dir,
- mi, &minstrel_stat_csv_fops);
+ mi->dbg_stats_csv = debugfs_create_file("rc_stats_csv", 0444, dir, mi,
+ &minstrel_stat_csv_fops);
}
void
{
struct minstrel_ht_sta_priv *msp = priv_sta;
- msp->dbg_stats = debugfs_create_file("rc_stats", S_IRUGO, dir, msp,
- &minstrel_ht_stat_fops);
- msp->dbg_stats_csv = debugfs_create_file("rc_stats_csv", S_IRUGO,
- dir, msp, &minstrel_ht_stat_csv_fops);
+ msp->dbg_stats = debugfs_create_file("rc_stats", 0444, dir, msp,
+ &minstrel_ht_stat_fops);
+ msp->dbg_stats_csv = debugfs_create_file("rc_stats_csv", 0444, dir, msp,
+ &minstrel_ht_stat_csv_fops);
}
void
static struct pernet_operations mpls_net_ops = {
.init = mpls_net_init,
.exit = mpls_net_exit,
- .async = true,
};
static struct rtnl_af_ops mpls_af_ops __read_mostly = {
package_id = nla_get_u32(info->attrs[NCSI_ATTR_PACKAGE_ID]);
attr = nla_nest_start(skb, NCSI_ATTR_PACKAGE_LIST);
+ if (!attr) {
+ kfree_skb(skb);
+ return -EMSGSIZE;
+ }
rc = ncsi_write_package_info(skb, ndp, package_id);
if (rc) {
static struct pernet_operations netfilter_net_ops = {
.init = netfilter_net_init,
.exit = netfilter_net_exit,
- .async = true,
};
int __init netfilter_init(void)
.exit = ip_set_net_exit,
.id = &ip_set_net_id,
.size = sizeof(struct ip_set_net),
- .async = true,
};
static int __init
.exit = __ip_vs_cleanup,
.id = &ip_vs_net_id,
.size = sizeof(struct netns_ipvs),
- .async = true,
};
static struct pernet_operations ipvs_core_dev_ops = {
.exit = __ip_vs_dev_cleanup,
- .async = true,
};
/*
static struct pernet_operations ip_vs_ftp_ops = {
.init = __ip_vs_ftp_init,
.exit = __ip_vs_ftp_exit,
- .async = true,
};
static int __init ip_vs_ftp_init(void)
static struct pernet_operations ip_vs_lblc_ops = {
.init = __ip_vs_lblc_init,
.exit = __ip_vs_lblc_exit,
- .async = true,
};
static int __init ip_vs_lblc_init(void)
static struct pernet_operations ip_vs_lblcr_ops = {
.init = __ip_vs_lblcr_init,
.exit = __ip_vs_lblcr_exit,
- .async = true,
};
static int __init ip_vs_lblcr_init(void)
{
struct net *net;
- rtnl_lock();
+ down_read(&net_rwsem);
for_each_net(net) {
if (atomic_read(&net->ct.count) == 0)
continue;
__nf_ct_unconfirmed_destroy(net);
nf_queue_nf_hook_drop(net);
}
- rtnl_unlock();
+ up_read(&net_rwsem);
/* Need to wait for netns cleanup worker to finish, if its
* running -- it might have deleted a net namespace from
MODULE_ALIAS_NFCT_HELPER("netbios_ns");
static unsigned int timeout __read_mostly = 3;
-module_param(timeout, uint, S_IRUSR);
+module_param(timeout, uint, 0400);
MODULE_PARM_DESC(timeout, "timeout for master connection/replies in seconds");
static struct nf_conntrack_expect_policy exp_policy = {
static struct pernet_operations ctnetlink_net_ops = {
.init = ctnetlink_net_init,
.exit_batch = ctnetlink_net_exit_batch,
- .async = true,
};
static int __init ctnetlink_init(void)
.exit = proto_gre_net_exit,
.id = &proto_gre_net_id,
.size = sizeof(struct netns_proto_gre),
- .async = true,
};
static int __init nf_ct_proto_gre_init(void)
MODULE_ALIAS_NFCT_HELPER("snmp");
static unsigned int timeout __read_mostly = 30;
-module_param(timeout, uint, S_IRUSR);
+module_param(timeout, uint, 0400);
MODULE_PARM_DESC(timeout, "timeout for master connection/replies in seconds");
int (*nf_nat_snmp_hook)(struct sk_buff *skb,
if (uid_valid(root_uid) && gid_valid(root_gid))
proc_set_user(pde, root_uid, root_gid);
- pde = proc_create("nf_conntrack", S_IRUGO, net->proc_net_stat,
+ pde = proc_create("nf_conntrack", 0444, net->proc_net_stat,
&ct_cpu_seq_fops);
if (!pde)
goto out_stat_nf_conntrack;
static struct pernet_operations nf_conntrack_net_ops = {
.init = nf_conntrack_pernet_init,
.exit_batch = nf_conntrack_pernet_exit,
- .async = true,
};
static int __init nf_conntrack_standalone_init(void)
int ret = -ENOMEM;
#ifdef CONFIG_PROC_FS
- if (!proc_create("nf_log", S_IRUGO,
+ if (!proc_create("nf_log", 0444,
net->nf.proc_netfilter, &nflog_file_ops))
return ret;
#endif
static struct pernet_operations nf_log_net_ops = {
.init = nf_log_net_init,
.exit = nf_log_net_exit,
- .async = true,
};
int __init netfilter_log_init(void)
static struct pernet_operations nf_log_netdev_net_ops = {
.init = nf_log_netdev_net_init,
.exit = nf_log_netdev_net_exit,
- .async = true,
};
static int __init nf_log_netdev_init(void)
static int __net_init synproxy_proc_init(struct net *net)
{
- if (!proc_create("synproxy", S_IRUGO, net->proc_net_stat,
+ if (!proc_create("synproxy", 0444, net->proc_net_stat,
&synproxy_cpu_seq_fops))
return -ENOMEM;
return 0;
.exit = synproxy_net_exit,
.id = &synproxy_net_id,
.size = sizeof(struct synproxy_net),
- .async = true,
};
static int __init synproxy_core_init(void)
static void nf_tables_flowtable_destroy(struct nft_flowtable *flowtable)
{
cancel_delayed_work_sync(&flowtable->data.gc_work);
+ kfree(flowtable->ops);
kfree(flowtable->name);
flowtable->data.type->free(&flowtable->data);
rhashtable_destroy(&flowtable->data.rhashtable);
static struct pernet_operations nf_tables_net_ops = {
.init = nf_tables_init_net,
.exit = nf_tables_exit_net,
- .async = true,
};
static int __init nf_tables_module_init(void)
static struct pernet_operations nfnetlink_net_ops = {
.init = nfnetlink_net_init,
.exit_batch = nfnetlink_net_exit_batch,
- .async = true,
};
static int __init nfnetlink_init(void)
static struct pernet_operations nfnl_acct_ops = {
.init = nfnl_acct_net_init,
.exit = nfnl_acct_net_exit,
- .async = true,
};
static int __init nfnl_acct_init(void)
static struct pernet_operations cttimeout_ops = {
.init = cttimeout_net_init,
.exit = cttimeout_net_exit,
- .async = true,
};
static int __init cttimeout_init(void)
.exit = nfnl_log_net_exit,
.id = &nfnl_log_net_id,
.size = sizeof(struct nfnl_log_net),
- .async = true,
};
static int __init nfnetlink_log_init(void)
.exit_batch = nfnl_queue_net_exit_batch,
.id = &nfnl_queue_net_id,
.size = sizeof(struct nfnl_queue_net),
- .async = true,
};
static int __init nfnetlink_queue_init(void)
nft_hash_select_ops(const struct nft_ctx *ctx, const struct nft_set_desc *desc,
u32 flags)
{
- if (desc->size) {
+ if (desc->size && !(flags & NFT_SET_TIMEOUT)) {
switch (desc->klen) {
case 4:
return &nft_hash_fast_ops;
return buf;
}
+/**
+ * xt_check_proc_name - check that name is suitable for /proc file creation
+ *
+ * @name: file name candidate
+ * @size: length of buffer
+ *
+ * some x_tables modules wish to create a file in /proc.
+ * This function makes sure that the name is suitable for this
+ * purpose, it checks that name is NUL terminated and isn't a 'special'
+ * name, like "..".
+ *
+ * returns negative number on error or 0 if name is useable.
+ */
+int xt_check_proc_name(const char *name, unsigned int size)
+{
+ if (name[0] == '\0')
+ return -EINVAL;
+
+ if (strnlen(name, size) == size)
+ return -ENAMETOOLONG;
+
+ if (strcmp(name, ".") == 0 ||
+ strcmp(name, "..") == 0 ||
+ strchr(name, '/'))
+ return -EINVAL;
+
+ return 0;
+}
+EXPORT_SYMBOL(xt_check_proc_name);
+
int xt_check_match(struct xt_mtchk_param *par,
unsigned int size, u_int8_t proto, bool inv_proto)
{
static struct pernet_operations xt_net_ops = {
.init = xt_net_init,
.exit = xt_net_exit,
- .async = true,
};
static int __init xt_init(void)
ret = -ENOMEM;
goto out_free_timer;
}
- info->timer->attr.attr.mode = S_IRUGO;
+ info->timer->attr.attr.mode = 0444;
info->timer->attr.show = idletimer_tg_show;
ret = sysfs_create_file(idletimer_tg_kobj, &info->timer->attr.attr);
struct hashlimit_cfg3 cfg = {};
int ret;
- if (info->name[sizeof(info->name) - 1] != '\0')
- return -EINVAL;
+ ret = xt_check_proc_name(info->name, sizeof(info->name));
+ if (ret)
+ return ret;
ret = cfg_copy(&cfg, (void *)&info->cfg, 1);
struct hashlimit_cfg3 cfg = {};
int ret;
- if (info->name[sizeof(info->name) - 1] != '\0')
- return -EINVAL;
+ ret = xt_check_proc_name(info->name, sizeof(info->name));
+ if (ret)
+ return ret;
ret = cfg_copy(&cfg, (void *)&info->cfg, 2);
static int hashlimit_mt_check(const struct xt_mtchk_param *par)
{
struct xt_hashlimit_mtinfo3 *info = par->matchinfo;
+ int ret;
- if (info->name[sizeof(info->name) - 1] != '\0')
- return -EINVAL;
+ ret = xt_check_proc_name(info->name, sizeof(info->name));
+ if (ret)
+ return ret;
return hashlimit_mt_check_common(par, &info->hinfo, &info->cfg,
info->name, 3);
.exit = hashlimit_net_exit,
.id = &hashlimit_net_id,
.size = sizeof(struct hashlimit_net),
- .async = true,
};
static int __init hashlimit_mt_init(void)
module_param(ip_list_tot, uint, 0400);
module_param(ip_list_hash_size, uint, 0400);
module_param(ip_list_perms, uint, 0400);
-module_param(ip_list_uid, uint, S_IRUGO | S_IWUSR);
-module_param(ip_list_gid, uint, S_IRUGO | S_IWUSR);
+module_param(ip_list_uid, uint, 0644);
+module_param(ip_list_gid, uint, 0644);
MODULE_PARM_DESC(ip_list_tot, "number of IPs to remember per list");
MODULE_PARM_DESC(ip_list_hash_size, "size of hash table used to look up IPs");
MODULE_PARM_DESC(ip_list_perms, "permissions on /proc/net/xt_recent/* files");
info->hit_count, XT_RECENT_MAX_NSTAMPS - 1);
return -EINVAL;
}
- if (info->name[0] == '\0' ||
- strnlen(info->name, XT_RECENT_NAME_LEN) == XT_RECENT_NAME_LEN)
- return -EINVAL;
+ ret = xt_check_proc_name(info->name, sizeof(info->name));
+ if (ret)
+ return ret;
if (ip_pkt_list_tot && info->hit_count < ip_pkt_list_tot)
nstamp_mask = roundup_pow_of_two(ip_pkt_list_tot) - 1;
.exit = recent_net_exit,
.id = &recent_net_id,
.size = sizeof(struct recent_net),
- .async = true,
};
static struct xt_match recent_mt_reg[] __read_mostly = {
.exit = netlink_tap_exit_net,
.id = &netlink_tap_net_id,
.size = sizeof(struct netlink_tap_net),
- .async = true,
};
static bool netlink_filter_tap(const struct sk_buff *skb)
static struct pernet_operations __net_initdata netlink_net_ops = {
.init = netlink_net_init,
.exit = netlink_net_exit,
- .async = true,
};
static inline u32 netlink_hash(const void *data, u32 len, u32 seed)
static struct pernet_operations genl_pernet_ops = {
.init = genl_pernet_init,
.exit = genl_pernet_exit,
- .async = true,
};
static int __init genl_init(void)
if (!err)
delivered = true;
else if (err != -ESRCH)
- goto error;
+ return err;
return delivered ? 0 : -ESRCH;
error:
kfree_skb(skb);
nr_loopback_init();
- proc_create("nr", S_IRUGO, init_net.proc_net, &nr_info_fops);
- proc_create("nr_neigh", S_IRUGO, init_net.proc_net, &nr_neigh_fops);
- proc_create("nr_nodes", S_IRUGO, init_net.proc_net, &nr_nodes_fops);
+ proc_create("nr", 0444, init_net.proc_net, &nr_info_fops);
+ proc_create("nr_neigh", 0444, init_net.proc_net, &nr_neigh_fops);
+ proc_create("nr_nodes", 0444, init_net.proc_net, &nr_nodes_fops);
out:
return rc;
fail:
list_for_each_entry_safe(dp, dp_next, &ovs_net->dps, list_node)
__dp_destroy(dp);
- rtnl_lock();
+ down_read(&net_rwsem);
for_each_net(net)
list_vports_from_net(net, dnet, &head);
- rtnl_unlock();
+ up_read(&net_rwsem);
/* Detach all vports from given namespace. */
list_for_each_entry_safe(vport, vport_next, &head, detach_list) {
.exit = ovs_exit_net,
.id = &ovs_net_id,
.size = sizeof(struct ovs_net),
- .async = true,
};
static int __init dp_init(void)
band->type = nla_get_u32(attr[OVS_BAND_ATTR_TYPE]);
band->rate = nla_get_u32(attr[OVS_BAND_ATTR_RATE]);
+ if (band->rate == 0) {
+ err = -EINVAL;
+ goto exit_free_meter;
+ }
+
band->burst_size = nla_get_u32(attr[OVS_BAND_ATTR_BURST]);
/* Figure out max delta_t that is enough to fill any bucket.
* Keep max_delta_t size to the bucket units:
* pkts => 1/1000 packets, kilobits => bits.
+ *
+ * Start with a full bucket.
*/
- band_max_delta_t = (band->burst_size + band->rate) * 1000;
- /* Start with a full bucket. */
- band->bucket = band_max_delta_t;
+ band->bucket = (band->burst_size + band->rate) * 1000;
+ band_max_delta_t = band->bucket / band->rate;
if (band_max_delta_t > meter->max_delta_t)
meter->max_delta_t = band_max_delta_t;
band++;
static struct pernet_operations packet_net_ops = {
.init = packet_net_init,
.exit = packet_net_exit,
- .async = true,
};
.exit = phonet_exit_net,
.id = &phonet_net_id,
.size = sizeof(struct phonet_net),
- .async = true,
};
/* Initialize Phonet devices list */
return err;
}
-static void __net_exit rds_tcp_exit_net(struct net *net)
-{
- struct rds_tcp_net *rtn = net_generic(net, rds_tcp_netid);
-
- if (rtn->rds_tcp_sysctl)
- unregister_net_sysctl_table(rtn->rds_tcp_sysctl);
-
- if (net != &init_net && rtn->ctl_table)
- kfree(rtn->ctl_table);
-
- /* If rds_tcp_exit_net() is called as a result of netns deletion,
- * the rds_tcp_kill_sock() device notifier would already have cleaned
- * up the listen socket, thus there is no work to do in this function.
- *
- * If rds_tcp_exit_net() is called as a result of module unload,
- * i.e., due to rds_tcp_exit() -> unregister_pernet_subsys(), then
- * we do need to clean up the listen socket here.
- */
- if (rtn->rds_tcp_listen_sock) {
- struct socket *lsock = rtn->rds_tcp_listen_sock;
-
- rtn->rds_tcp_listen_sock = NULL;
- rds_tcp_listen_stop(lsock, &rtn->rds_tcp_accept_w);
- }
-}
-
-static struct pernet_operations rds_tcp_net_ops = {
- .init = rds_tcp_init_net,
- .exit = rds_tcp_exit_net,
- .id = &rds_tcp_netid,
- .size = sizeof(struct rds_tcp_net),
- .async = true,
-};
-
static void rds_tcp_kill_sock(struct net *net)
{
struct rds_tcp_connection *tc, *_tc;
rds_conn_destroy(tc->t_cpath->cp_conn);
}
-void *rds_tcp_listen_sock_def_readable(struct net *net)
+static void __net_exit rds_tcp_exit_net(struct net *net)
{
struct rds_tcp_net *rtn = net_generic(net, rds_tcp_netid);
- struct socket *lsock = rtn->rds_tcp_listen_sock;
- if (!lsock)
- return NULL;
+ rds_tcp_kill_sock(net);
- return lsock->sk->sk_user_data;
+ if (rtn->rds_tcp_sysctl)
+ unregister_net_sysctl_table(rtn->rds_tcp_sysctl);
+
+ if (net != &init_net && rtn->ctl_table)
+ kfree(rtn->ctl_table);
}
-static int rds_tcp_dev_event(struct notifier_block *this,
- unsigned long event, void *ptr)
+static struct pernet_operations rds_tcp_net_ops = {
+ .init = rds_tcp_init_net,
+ .exit = rds_tcp_exit_net,
+ .id = &rds_tcp_netid,
+ .size = sizeof(struct rds_tcp_net),
+};
+
+void *rds_tcp_listen_sock_def_readable(struct net *net)
{
- struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+ struct rds_tcp_net *rtn = net_generic(net, rds_tcp_netid);
+ struct socket *lsock = rtn->rds_tcp_listen_sock;
- /* rds-tcp registers as a pernet subys, so the ->exit will only
- * get invoked after network acitivity has quiesced. We need to
- * clean up all sockets to quiesce network activity, and use
- * the unregistration of the per-net loopback device as a trigger
- * to start that cleanup.
- */
- if (event == NETDEV_UNREGISTER_FINAL &&
- dev->ifindex == LOOPBACK_IFINDEX)
- rds_tcp_kill_sock(dev_net(dev));
+ if (!lsock)
+ return NULL;
- return NOTIFY_DONE;
+ return lsock->sk->sk_user_data;
}
-static struct notifier_block rds_tcp_dev_notifier = {
- .notifier_call = rds_tcp_dev_event,
- .priority = -10, /* must be called after other network notifiers */
-};
-
/* when sysctl is used to modify some kernel socket parameters,this
* function resets the RDS connections in that netns so that we can
* restart with new parameters. The assumption is that such reset
rds_tcp_set_unloading();
synchronize_rcu();
rds_info_deregister_func(RDS_INFO_TCP_SOCKETS, rds_tcp_tc_info);
- unregister_pernet_subsys(&rds_tcp_net_ops);
- if (unregister_netdevice_notifier(&rds_tcp_dev_notifier))
- pr_warn("could not unregister rds_tcp_dev_notifier\n");
+ unregister_pernet_device(&rds_tcp_net_ops);
rds_tcp_destroy_conns();
rds_trans_unregister(&rds_tcp_transport);
rds_tcp_recv_exit();
if (ret)
goto out_slab;
- ret = register_pernet_subsys(&rds_tcp_net_ops);
+ ret = register_pernet_device(&rds_tcp_net_ops);
if (ret)
goto out_recv;
- ret = register_netdevice_notifier(&rds_tcp_dev_notifier);
- if (ret) {
- pr_warn("could not register rds_tcp_dev_notifier\n");
- goto out_pernet;
- }
-
rds_trans_register(&rds_tcp_transport);
rds_info_register_func(RDS_INFO_TCP_SOCKETS, rds_tcp_tc_info);
goto out;
-
-out_pernet:
- unregister_pernet_subsys(&rds_tcp_net_ops);
out_recv:
rds_tcp_recv_exit();
out_slab:
rose_add_loopback_neigh();
- proc_create("rose", S_IRUGO, init_net.proc_net, &rose_info_fops);
- proc_create("rose_neigh", S_IRUGO, init_net.proc_net,
+ proc_create("rose", 0444, init_net.proc_net, &rose_info_fops);
+ proc_create("rose_neigh", 0444, init_net.proc_net,
&rose_neigh_fops);
- proc_create("rose_nodes", S_IRUGO, init_net.proc_net,
+ proc_create("rose_nodes", 0444, init_net.proc_net,
&rose_nodes_fops);
- proc_create("rose_routes", S_IRUGO, init_net.proc_net,
+ proc_create("rose_routes", 0444, init_net.proc_net,
&rose_routes_fops);
out:
return rc;
MODULE_ALIAS_NETPROTO(PF_RXRPC);
unsigned int rxrpc_debug; // = RXRPC_DEBUG_KPROTO;
-module_param_named(debug, rxrpc_debug, uint, S_IWUSR | S_IRUGO);
+module_param_named(debug, rxrpc_debug, uint, 0644);
MODULE_PARM_DESC(debug, "RxRPC debugging mask");
static struct proto rxrpc_proto;
/* current debugging ID */
atomic_t rxrpc_debug_id;
+EXPORT_SYMBOL(rxrpc_debug_id);
/* count of skbs currently in use */
atomic_t rxrpc_n_tx_skbs, rxrpc_n_rx_skbs;
* @gfp: The allocation constraints
* @notify_rx: Where to send notifications instead of socket queue
* @upgrade: Request service upgrade for call
+ * @debug_id: The debug ID for tracing to be assigned to the call
*
* Allow a kernel service to begin a call on the nominated socket. This just
* sets up all the internal tracking structures and allocates connection and
s64 tx_total_len,
gfp_t gfp,
rxrpc_notify_rx_t notify_rx,
- bool upgrade)
+ bool upgrade,
+ unsigned int debug_id)
{
struct rxrpc_conn_parameters cp;
struct rxrpc_call_params p;
cp.exclusive = false;
cp.upgrade = upgrade;
cp.service_id = srx->srx_service;
- call = rxrpc_new_client_call(rx, &cp, srx, &p, gfp);
+ call = rxrpc_new_client_call(rx, &cp, srx, &p, gfp, debug_id);
/* The socket has been unlocked. */
if (!IS_ERR(call)) {
call->notify_rx = notify_rx;
* af_rxrpc.c
*/
extern atomic_t rxrpc_n_tx_skbs, rxrpc_n_rx_skbs;
-extern atomic_t rxrpc_debug_id;
extern struct workqueue_struct *rxrpc_workqueue;
/*
extern struct kmem_cache *rxrpc_call_jar;
struct rxrpc_call *rxrpc_find_call_by_user_ID(struct rxrpc_sock *, unsigned long);
-struct rxrpc_call *rxrpc_alloc_call(struct rxrpc_sock *, gfp_t);
+struct rxrpc_call *rxrpc_alloc_call(struct rxrpc_sock *, gfp_t, unsigned int);
struct rxrpc_call *rxrpc_new_client_call(struct rxrpc_sock *,
struct rxrpc_conn_parameters *,
struct sockaddr_rxrpc *,
- struct rxrpc_call_params *, gfp_t);
+ struct rxrpc_call_params *, gfp_t,
+ unsigned int);
int rxrpc_retry_client_call(struct rxrpc_sock *,
struct rxrpc_call *,
struct rxrpc_conn_parameters *,
call->error = error;
call->completion = compl,
call->state = RXRPC_CALL_COMPLETE;
+ trace_rxrpc_call_complete(call);
wake_up(&call->waitq);
return true;
}
rxrpc_seq_t seq,
u32 abort_code, int error)
{
- trace_rxrpc_abort(why, call->cid, call->call_id, seq,
+ trace_rxrpc_abort(call->debug_id, why, call->cid, call->call_id, seq,
abort_code, error);
return __rxrpc_set_call_completion(call, RXRPC_CALL_LOCALLY_ABORTED,
abort_code, error);
struct rxrpc_backlog *b,
rxrpc_notify_rx_t notify_rx,
rxrpc_user_attach_call_t user_attach_call,
- unsigned long user_call_ID, gfp_t gfp)
+ unsigned long user_call_ID, gfp_t gfp,
+ unsigned int debug_id)
{
const void *here = __builtin_return_address(0);
struct rxrpc_call *call;
/* Now it gets complicated, because calls get registered with the
* socket here, particularly if a user ID is preassigned by the user.
*/
- call = rxrpc_alloc_call(rx, gfp);
+ call = rxrpc_alloc_call(rx, gfp, debug_id);
if (!call)
return -ENOMEM;
call->flags |= (1 << RXRPC_CALL_IS_SERVICE);
if (rx->discard_new_call)
return 0;
- while (rxrpc_service_prealloc_one(rx, b, NULL, NULL, 0, gfp) == 0)
+ while (rxrpc_service_prealloc_one(rx, b, NULL, NULL, 0, gfp,
+ atomic_inc_return(&rxrpc_debug_id)) == 0)
;
return 0;
service_id == rx->second_service))
goto found_service;
- trace_rxrpc_abort("INV", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
+ trace_rxrpc_abort(0, "INV", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RX_INVALID_OPERATION, EOPNOTSUPP);
skb->mark = RXRPC_SKB_MARK_LOCAL_ABORT;
skb->priority = RX_INVALID_OPERATION;
spin_lock(&rx->incoming_lock);
if (rx->sk.sk_state == RXRPC_SERVER_LISTEN_DISABLED ||
rx->sk.sk_state == RXRPC_CLOSE) {
- trace_rxrpc_abort("CLS", sp->hdr.cid, sp->hdr.callNumber,
+ trace_rxrpc_abort(0, "CLS", sp->hdr.cid, sp->hdr.callNumber,
sp->hdr.seq, RX_INVALID_OPERATION, ESHUTDOWN);
skb->mark = RXRPC_SKB_MARK_LOCAL_ABORT;
skb->priority = RX_INVALID_OPERATION;
* @user_attach_call: Func to attach call to user_call_ID
* @user_call_ID: The tag to attach to the preallocated call
* @gfp: The allocation conditions.
+ * @debug_id: The tracing debug ID.
*
* Charge up the socket with preallocated calls, each with a user ID. A
* function should be provided to effect the attachment from the user's side.
int rxrpc_kernel_charge_accept(struct socket *sock,
rxrpc_notify_rx_t notify_rx,
rxrpc_user_attach_call_t user_attach_call,
- unsigned long user_call_ID, gfp_t gfp)
+ unsigned long user_call_ID, gfp_t gfp,
+ unsigned int debug_id)
{
struct rxrpc_sock *rx = rxrpc_sk(sock->sk);
struct rxrpc_backlog *b = rx->backlog;
return rxrpc_service_prealloc_one(rx, b, notify_rx,
user_attach_call, user_call_ID,
- gfp);
+ gfp, debug_id);
}
EXPORT_SYMBOL(rxrpc_kernel_charge_accept);
* the packets in the Tx buffer we're going to resend and what the new
* resend timeout will be.
*/
+ trace_rxrpc_resend(call, (cursor + 1) & RXRPC_RXTX_BUFF_MASK);
oldest = now;
for (seq = cursor + 1; before_eq(seq, top); seq++) {
ix = seq & RXRPC_RXTX_BUFF_MASK;
/*
* allocate a new call
*/
-struct rxrpc_call *rxrpc_alloc_call(struct rxrpc_sock *rx, gfp_t gfp)
+struct rxrpc_call *rxrpc_alloc_call(struct rxrpc_sock *rx, gfp_t gfp,
+ unsigned int debug_id)
{
struct rxrpc_call *call;
spin_lock_init(&call->notify_lock);
rwlock_init(&call->state_lock);
atomic_set(&call->usage, 1);
- call->debug_id = atomic_inc_return(&rxrpc_debug_id);
+ call->debug_id = debug_id;
call->tx_total_len = -1;
call->next_rx_timo = 20 * HZ;
call->next_req_timo = 1 * HZ;
*/
static struct rxrpc_call *rxrpc_alloc_client_call(struct rxrpc_sock *rx,
struct sockaddr_rxrpc *srx,
- gfp_t gfp)
+ gfp_t gfp,
+ unsigned int debug_id)
{
struct rxrpc_call *call;
ktime_t now;
_enter("");
- call = rxrpc_alloc_call(rx, gfp);
+ call = rxrpc_alloc_call(rx, gfp, debug_id);
if (!call)
return ERR_PTR(-ENOMEM);
call->state = RXRPC_CALL_CLIENT_AWAIT_CONN;
struct rxrpc_conn_parameters *cp,
struct sockaddr_rxrpc *srx,
struct rxrpc_call_params *p,
- gfp_t gfp)
+ gfp_t gfp,
+ unsigned int debug_id)
__releases(&rx->sk.sk_lock.slock)
{
struct rxrpc_call *call, *xcall;
_enter("%p,%lx", rx, p->user_call_ID);
- call = rxrpc_alloc_client_call(rx, srx, gfp);
+ call = rxrpc_alloc_client_call(rx, srx, gfp, debug_id);
if (IS_ERR(call)) {
release_sock(&rx->sk);
_leave(" = %ld", PTR_ERR(call));
lockdep_is_held(&conn->channel_lock));
if (call) {
if (compl == RXRPC_CALL_LOCALLY_ABORTED)
- trace_rxrpc_abort("CON", call->cid,
+ trace_rxrpc_abort(call->debug_id,
+ "CON", call->cid,
call->call_id, 0,
abort_code, error);
if (rxrpc_set_call_completion(call, compl,
wrong_security:
rcu_read_unlock();
- trace_rxrpc_abort("SEC", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
+ trace_rxrpc_abort(0, "SEC", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RXKADINCONSISTENCY, EBADMSG);
skb->priority = RXKADINCONSISTENCY;
goto post_abort;
reupgrade:
rcu_read_unlock();
- trace_rxrpc_abort("UPG", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
+ trace_rxrpc_abort(0, "UPG", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RX_PROTOCOL_ERROR, EBADMSG);
goto protocol_error;
bad_message_unlock:
rcu_read_unlock();
bad_message:
- trace_rxrpc_abort("BAD", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
+ trace_rxrpc_abort(0, "BAD", sp->hdr.cid, sp->hdr.callNumber, sp->hdr.seq,
RX_PROTOCOL_ERROR, EBADMSG);
protocol_error:
skb->priority = RX_PROTOCOL_ERROR;
cp.exclusive = rx->exclusive | p->exclusive;
cp.upgrade = p->upgrade;
cp.service_id = srx->srx_service;
- call = rxrpc_new_client_call(rx, &cp, srx, &p->call, GFP_KERNEL);
+ call = rxrpc_new_client_call(rx, &cp, srx, &p->call, GFP_KERNEL,
+ atomic_inc_return(&rxrpc_debug_id));
/* The socket is now unlocked */
_leave(" = %p\n", call);
}
EXPORT_SYMBOL(tcf_idr_check);
-void tcf_idr_cleanup(struct tc_action *a, struct nlattr *est)
-{
- if (est)
- gen_kill_estimator(&a->tcfa_rate_est);
- free_tcf(a);
-}
-EXPORT_SYMBOL(tcf_idr_cleanup);
-
int tcf_idr_create(struct tc_action_net *tn, u32 index, struct nlattr *est,
struct tc_action **a, const struct tc_action_ops *ops,
int bind, bool cpustats)
.exit = tcf_action_net_exit,
.id = &tcf_action_net_id,
.size = sizeof(struct tcf_action_net),
- .async = true,
};
static int __init tc_action_init(void)
return res;
out:
if (res == ACT_P_CREATED)
- tcf_idr_cleanup(*act, est);
+ tcf_idr_release(*act, bind);
return ret;
}
.exit_batch = bpf_exit_net,
.id = &bpf_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int __init bpf_init_module(void)
.exit_batch = connmark_exit_net,
.id = &connmark_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int __init connmark_init_module(void)
{
struct sctphdr *sctph;
- if (skb_is_gso(skb) && skb_shinfo(skb)->gso_type & SKB_GSO_SCTP)
+ if (skb_is_gso(skb) && skb_is_gso_sctp(skb))
return 1;
sctph = tcf_csum_skb_nextlayer(skb, ihl, ipl, sizeof(*sctph));
struct tcf_csum_params *params;
params = rcu_dereference_protected(p->params, 1);
- kfree_rcu(params, rcu);
+ if (params)
+ kfree_rcu(params, rcu);
}
static int tcf_csum_walker(struct net *net, struct sk_buff *skb,
.exit_batch = csum_exit_net,
.id = &csum_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_DESCRIPTION("Checksum updating actions");
.exit_batch = gact_exit_net,
.id = &gact_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_AUTHOR("Jamal Hadi Salim(2002-4)");
.exit_batch = ife_exit_net,
.id = &ife_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int __init ife_init_module(void)
static void tcf_ipt_release(struct tc_action *a)
{
struct tcf_ipt *ipt = to_ipt(a);
- ipt_destroy_target(ipt->tcfi_t);
+
+ if (ipt->tcfi_t) {
+ ipt_destroy_target(ipt->tcfi_t);
+ kfree(ipt->tcfi_t);
+ }
kfree(ipt->tcfi_tname);
- kfree(ipt->tcfi_t);
}
static const struct nla_policy ipt_policy[TCA_IPT_MAX + 1] = {
kfree(tname);
err1:
if (ret == ACT_P_CREATED)
- tcf_idr_cleanup(*a, est);
+ tcf_idr_release(*a, bind);
return err;
}
.exit_batch = ipt_exit_net,
.id = &ipt_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int tcf_xt_walker(struct net *net, struct sk_buff *skb,
.exit_batch = xt_exit_net,
.id = &xt_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_AUTHOR("Jamal Hadi Salim(2002-13)");
.exit_batch = mirred_exit_net,
.id = &mirred_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_AUTHOR("Jamal Hadi Salim(2002)");
.exit_batch = nat_exit_net,
.id = &nat_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_DESCRIPTION("Stateless NAT actions");
p = to_pedit(*a);
keys = kmalloc(ksize, GFP_KERNEL);
if (keys == NULL) {
- tcf_idr_cleanup(*a, est);
+ tcf_idr_release(*a, bind);
kfree(keys_ex);
return -ENOMEM;
}
.exit_batch = pedit_exit_net,
.id = &pedit_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_AUTHOR("Jamal Hadi Salim(2002-4)");
qdisc_put_rtab(P_tab);
qdisc_put_rtab(R_tab);
if (ret == ACT_P_CREATED)
- tcf_idr_cleanup(*a, est);
+ tcf_idr_release(*a, bind);
return err;
}
.exit_batch = police_exit_net,
.id = &police_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int __init police_init_module(void)
psample_group = rtnl_dereference(s->psample_group);
RCU_INIT_POINTER(s->psample_group, NULL);
- psample_group_put(psample_group);
+ if (psample_group)
+ psample_group_put(psample_group);
}
static bool tcf_sample_dev_ok_push(struct net_device *dev)
.exit_batch = sample_exit_net,
.id = &sample_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int __init sample_init_module(void)
d = to_defact(*a);
ret = alloc_defdata(d, defdata);
if (ret < 0) {
- tcf_idr_cleanup(*a, est);
+ tcf_idr_release(*a, bind);
return ret;
}
d->tcf_action = parm->action;
.exit_batch = simp_exit_net,
.id = &simp_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_AUTHOR("Jamal Hadi Salim(2005)");
.exit_batch = skbedit_exit_net,
.id = &skbedit_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_AUTHOR("Alexander Duyck, <alexander.h.duyck@intel.com>");
ASSERT_RTNL();
p = kzalloc(sizeof(struct tcf_skbmod_params), GFP_KERNEL);
if (unlikely(!p)) {
- if (ovr)
+ if (ret == ACT_P_CREATED)
tcf_idr_release(*a, bind);
return -ENOMEM;
}
struct tcf_skbmod_params *p;
p = rcu_dereference_protected(d->skbmod_p, 1);
- kfree_rcu(p, rcu);
+ if (p)
+ kfree_rcu(p, rcu);
}
static int tcf_skbmod_dump(struct sk_buff *skb, struct tc_action *a,
.exit_batch = skbmod_exit_net,
.id = &skbmod_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
MODULE_AUTHOR("Jamal Hadi Salim, <jhs@mojatatu.com>");
metadata->u.tun_info.mode |= IP_TUNNEL_INFO_TX;
break;
default:
+ ret = -EINVAL;
goto err_out;
}
struct tcf_tunnel_key_params *params;
params = rcu_dereference_protected(t->params, 1);
+ if (params) {
+ if (params->tcft_action == TCA_TUNNEL_KEY_ACT_SET)
+ dst_release(¶ms->tcft_enc_metadata->dst);
- if (params->tcft_action == TCA_TUNNEL_KEY_ACT_SET)
- dst_release(¶ms->tcft_enc_metadata->dst);
-
- kfree_rcu(params, rcu);
+ kfree_rcu(params, rcu);
+ }
}
static int tunnel_key_dump_addresses(struct sk_buff *skb,
.exit_batch = tunnel_key_exit_net,
.id = &tunnel_key_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int __init tunnel_key_init_module(void)
struct tc_vlan *parm;
struct tcf_vlan *v;
int action;
- __be16 push_vid = 0;
+ u16 push_vid = 0;
__be16 push_proto = 0;
u8 push_prio = 0;
bool exists = false;
ASSERT_RTNL();
p = kzalloc(sizeof(*p), GFP_KERNEL);
if (!p) {
- if (ovr)
+ if (ret == ACT_P_CREATED)
tcf_idr_release(*a, bind);
return -ENOMEM;
}
struct tcf_vlan_params *p;
p = rcu_dereference_protected(v->vlan_p, 1);
- kfree_rcu(p, rcu);
+ if (p)
+ kfree_rcu(p, rcu);
}
static int tcf_vlan_dump(struct sk_buff *skb, struct tc_action *a,
.exit_batch = vlan_exit_net,
.id = &vlan_net_id,
.size = sizeof(struct tc_action_net),
- .async = true,
};
static int __init vlan_init_module(void)
.exit = tcf_net_exit,
.id = &tcf_net_id,
.size = sizeof(struct tcf_net),
- .async = true,
};
static int __init tc_filter_init(void)
static struct pernet_operations psched_net_ops = {
.init = psched_net_init,
.exit = psched_net_exit,
- .async = true,
};
static int __init pktsched_init(void)
__skb_queue_tail(&q->skb_bad_txq, skb);
+ if (qdisc_is_percpu_stats(q)) {
+ qdisc_qstats_cpu_backlog_inc(q, skb);
+ qdisc_qstats_cpu_qlen_inc(q);
+ } else {
+ qdisc_qstats_backlog_inc(q, skb);
+ q->q.qlen++;
+ }
+
if (lock)
spin_unlock(lock);
}
break;
if (unlikely(skb_get_queue_mapping(nskb) != mapping)) {
qdisc_enqueue_skb_bad_txq(q, nskb);
-
- if (qdisc_is_percpu_stats(q)) {
- qdisc_qstats_cpu_backlog_inc(q, nskb);
- qdisc_qstats_cpu_qlen_inc(q);
- } else {
- qdisc_qstats_backlog_inc(q, nskb);
- q->q.qlen++;
- }
break;
}
skb->next = nskb;
int band = prio2band[skb->priority & TC_PRIO_MAX];
struct pfifo_fast_priv *priv = qdisc_priv(qdisc);
struct skb_array *q = band2list(priv, band);
+ unsigned int pkt_len = qdisc_pkt_len(skb);
int err;
err = skb_array_produce(q, skb);
return qdisc_drop_cpu(skb, qdisc, to_free);
qdisc_qstats_cpu_qlen_inc(qdisc);
- qdisc_qstats_cpu_backlog_inc(qdisc, skb);
+ /* Note: skb can not be used after skb_array_produce(),
+ * so we better not use qdisc_qstats_cpu_backlog_inc()
+ */
+ this_cpu_add(qdisc->cpu_qstats->backlog, pkt_len);
return NET_XMIT_SUCCESS;
}
}
if (unlikely(sch->q.qlen >= sch->limit))
- return qdisc_drop(skb, sch, to_free);
+ return qdisc_drop_all(skb, sch, to_free);
qdisc_qstats_backlog_inc(sch, skb);
/* Look for any peeled off association from the endpoint that matches the
* given peer address.
*/
-int sctp_endpoint_is_peeled_off(struct sctp_endpoint *ep,
- const union sctp_addr *paddr)
+bool sctp_endpoint_is_peeled_off(struct sctp_endpoint *ep,
+ const union sctp_addr *paddr)
{
struct sctp_sockaddr_entry *addr;
struct sctp_bind_addr *bp;
*/
list_for_each_entry(addr, &bp->address_list, list) {
if (sctp_has_association(net, &addr->a, paddr))
- return 1;
+ return true;
}
- return 0;
+ return false;
}
/* Do delayed input processing. This is scheduled by sctp_rcv().
int family;
struct sctp_af *af;
struct net *net = dev_net(skb->dev);
+ bool is_gso = skb_is_gso(skb) && skb_is_gso_sctp(skb);
if (skb->pkt_type != PACKET_HOST)
goto discard_it;
* it's better to just linearize it otherwise crc computing
* takes longer.
*/
- if ((!(skb_shinfo(skb)->gso_type & SKB_GSO_SCTP) &&
- skb_linearize(skb)) ||
+ if ((!is_gso && skb_linearize(skb)) ||
!pskb_may_pull(skb, sizeof(struct sctphdr)))
goto discard_it;
if (skb_csum_unnecessary(skb))
__skb_decr_checksum_unnecessary(skb);
else if (!sctp_checksum_disable &&
- !(skb_shinfo(skb)->gso_type & SKB_GSO_SCTP) &&
+ !is_gso &&
sctp_rcv_checksum(net, skb) < 0)
goto discard_it;
skb->csum_valid = 1;
}
/* Is there an association matching the given local and peer addresses? */
-int sctp_has_association(struct net *net,
- const union sctp_addr *laddr,
- const union sctp_addr *paddr)
+bool sctp_has_association(struct net *net,
+ const union sctp_addr *laddr,
+ const union sctp_addr *paddr)
{
- struct sctp_association *asoc;
struct sctp_transport *transport;
- if ((asoc = sctp_lookup_association(net, laddr, paddr, &transport))) {
+ if (sctp_lookup_association(net, laddr, paddr, &transport)) {
sctp_transport_put(transport);
- return 1;
+ return true;
}
- return 0;
+ return false;
}
/*
* issue as packets hitting this are mostly INIT or INIT-ACK and
* those cannot be on GSO-style anyway.
*/
- if ((skb_shinfo(skb)->gso_type & SKB_GSO_SCTP) == SKB_GSO_SCTP)
+ if (skb_is_gso(skb) && skb_is_gso_sctp(skb))
return NULL;
ch = (struct sctp_chunkhdr *)skb->data;
chunk = list_entry(entry, struct sctp_chunk, list);
- if ((skb_shinfo(chunk->skb)->gso_type & SKB_GSO_SCTP) == SKB_GSO_SCTP) {
+ if (skb_is_gso(chunk->skb) && skb_is_gso_sctp(chunk->skb)) {
/* GSO-marked skbs but without frags, handle
* them normally
*/
struct sk_buff *segs = ERR_PTR(-EINVAL);
struct sctphdr *sh;
- if (!(skb_shinfo(skb)->gso_type & SKB_GSO_SCTP))
+ if (!skb_is_gso_sctp(skb))
goto out;
sh = sctp_hdr(skb);
net->sctp.proc_net_sctp = proc_net_mkdir(net, "sctp", net->proc_net);
if (!net->sctp.proc_net_sctp)
return -ENOMEM;
- if (!proc_create("snmp", S_IRUGO, net->sctp.proc_net_sctp,
- &sctp_snmp_seq_fops))
+ if (!proc_create("snmp", 0444, net->sctp.proc_net_sctp,
+ &sctp_snmp_seq_fops))
goto cleanup;
- if (!proc_create("eps", S_IRUGO, net->sctp.proc_net_sctp,
- &sctp_eps_seq_fops))
+ if (!proc_create("eps", 0444, net->sctp.proc_net_sctp,
+ &sctp_eps_seq_fops))
goto cleanup;
- if (!proc_create("assocs", S_IRUGO, net->sctp.proc_net_sctp,
- &sctp_assocs_seq_fops))
+ if (!proc_create("assocs", 0444, net->sctp.proc_net_sctp,
+ &sctp_assocs_seq_fops))
goto cleanup;
- if (!proc_create("remaddr", S_IRUGO, net->sctp.proc_net_sctp,
- &sctp_remaddr_seq_fops))
+ if (!proc_create("remaddr", 0444, net->sctp.proc_net_sctp,
+ &sctp_remaddr_seq_fops))
goto cleanup;
return 0;
return 0;
+#ifdef CONFIG_PROC_FS
err_init_proc:
cleanup_sctp_mibs(net);
+#endif
err_init_mibs:
sctp_sysctl_net_unregister(net);
err_sysctl_register:
static struct pernet_operations sctp_defaults_ops = {
.init = sctp_defaults_init,
.exit = sctp_defaults_exit,
- .async = true,
};
static int __net_init sctp_ctrlsock_init(struct net *net)
static struct pernet_operations sctp_ctrlsock_ops = {
.init = sctp_ctrlsock_init,
.exit = sctp_ctrlsock_exit,
- .async = true,
};
/* Initialize the universe into something sensible. */
lsmc->clcsock = NULL;
}
release_sock(lsk);
- /* no more listening, wake up smc_close_wait_listen_clcsock and
- * accept
- */
- lsk->sk_state_change(lsk);
sock_put(&lsmc->sk); /* sock_hold in smc_listen */
}
smc_close_non_accepted(sk);
}
-static void smc_close_wait_listen_clcsock(struct smc_sock *smc)
-{
- DEFINE_WAIT_FUNC(wait, woken_wake_function);
- struct sock *sk = &smc->sk;
- signed long timeout;
-
- timeout = SMC_CLOSE_WAIT_LISTEN_CLCSOCK_TIME;
- add_wait_queue(sk_sleep(sk), &wait);
- do {
- release_sock(sk);
- if (smc->clcsock)
- timeout = wait_woken(&wait, TASK_UNINTERRUPTIBLE,
- timeout);
- sched_annotate_sleep();
- lock_sock(sk);
- if (!smc->clcsock)
- break;
- } while (timeout);
- remove_wait_queue(sk_sleep(sk), &wait);
-}
-
/* wait for sndbuf data being transmitted */
static void smc_close_stream_wait(struct smc_sock *smc, long timeout)
{
rc = kernel_sock_shutdown(smc->clcsock, SHUT_RDWR);
/* wake up kernel_accept of smc_tcp_listen_worker */
smc->clcsock->sk->sk_data_ready(smc->clcsock->sk);
- smc_close_wait_listen_clcsock(smc);
}
smc_close_cleanup_listen(sk);
+ release_sock(sk);
+ flush_work(&smc->tcp_listen_work);
+ lock_sock(sk);
break;
case SMC_ACTIVE:
smc_close_stream_wait(smc, timeout);
}
EXPORT_SYMBOL(sock_unregister);
+bool sock_is_registered(int family)
+{
+ return family < NPROTO && rcu_access_pointer(net_families[family]);
+}
+
static int __init sock_init(void)
{
int err;
struct proc_dir_entry **p = &sn->use_gssp_proc;
sn->use_gss_proxy = -1;
- *p = proc_create_data("use-gss-proxy", S_IFREG|S_IRUSR|S_IWUSR,
+ *p = proc_create_data("use-gss-proxy", S_IFREG | 0600,
sn->proc_net_rpc,
&use_gss_proxy_ops, net);
if (!*p)
if (cd->procfs == NULL)
goto out_nomem;
- p = proc_create_data("flush", S_IFREG|S_IRUSR|S_IWUSR,
+ p = proc_create_data("flush", S_IFREG | 0600,
cd->procfs, &cache_flush_operations_procfs, cd);
if (p == NULL)
goto out_nomem;
if (cd->cache_request || cd->cache_parse) {
- p = proc_create_data("channel", S_IFREG|S_IRUSR|S_IWUSR,
- cd->procfs, &cache_file_operations_procfs, cd);
+ p = proc_create_data("channel", S_IFREG | 0600, cd->procfs,
+ &cache_file_operations_procfs, cd);
if (p == NULL)
goto out_nomem;
}
if (cd->cache_show) {
- p = proc_create_data("content", S_IFREG|S_IRUSR,
- cd->procfs, &content_file_operations_procfs, cd);
+ p = proc_create_data("content", S_IFREG | 0400, cd->procfs,
+ &content_file_operations_procfs, cd);
if (p == NULL)
goto out_nomem;
}
return;
/* make tasks file */
- if (!debugfs_create_file("tasks", S_IFREG | S_IRUSR, clnt->cl_debugfs,
+ if (!debugfs_create_file("tasks", S_IFREG | 0400, clnt->cl_debugfs,
clnt, &tasks_fops))
goto out_err;
return;
/* make tasks file */
- if (!debugfs_create_file("info", S_IFREG | S_IRUSR, xprt->debugfs,
+ if (!debugfs_create_file("info", S_IFREG | 0400, xprt->debugfs,
xprt, &xprt_info_fops)) {
debugfs_remove_recursive(xprt->debugfs);
xprt->debugfs = NULL;
if (!faultdir)
return NULL;
- if (!debugfs_create_file("disconnect", S_IFREG | S_IRUSR, faultdir,
+ if (!debugfs_create_file("disconnect", S_IFREG | 0400, faultdir,
NULL, &fault_disconnect_fops))
return NULL;
{
struct dentry *dentry;
struct inode *dir = d_inode(parent);
- umode_t umode = S_IFIFO | S_IRUSR | S_IWUSR;
+ umode_t umode = S_IFIFO | 0600;
int err;
if (pipe->ops->upcall == NULL)
- umode &= ~S_IRUGO;
+ umode &= ~0444;
if (pipe->ops->downcall == NULL)
- umode &= ~S_IWUGO;
+ umode &= ~0222;
inode_lock_nested(dir, I_MUTEX_PARENT);
dentry = __rpc_lookup_create_exclusive(parent, name);
[RPCAUTH_info] = {
.name = "info",
.i_fop = &rpc_info_operations,
- .mode = S_IFREG | S_IRUSR,
+ .mode = S_IFREG | 0400,
},
};
{
struct dentry *ret;
- ret = rpc_mkdir_populate(dentry, name, S_IRUGO | S_IXUGO, NULL,
- rpc_clntdir_populate, rpc_client);
+ ret = rpc_mkdir_populate(dentry, name, 0555, NULL,
+ rpc_clntdir_populate, rpc_client);
if (!IS_ERR(ret)) {
rpc_client->cl_pipedir_objects.pdh_dentry = ret;
rpc_create_pipe_dir_objects(&rpc_client->cl_pipedir_objects);
[0] = {
.name = "channel",
.i_fop = &cache_file_operations_pipefs,
- .mode = S_IFREG|S_IRUSR|S_IWUSR,
+ .mode = S_IFREG | 0600,
},
[1] = {
.name = "content",
.i_fop = &content_file_operations_pipefs,
- .mode = S_IFREG|S_IRUSR,
+ .mode = S_IFREG | 0400,
},
[2] = {
.name = "flush",
.i_fop = &cache_flush_operations_pipefs,
- .mode = S_IFREG|S_IRUSR|S_IWUSR,
+ .mode = S_IFREG | 0600,
},
};
static const struct rpc_filelist files[] = {
[RPCAUTH_lockd] = {
.name = "lockd",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_mount] = {
.name = "mount",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_nfs] = {
.name = "nfs",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_portmap] = {
.name = "portmap",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_statd] = {
.name = "statd",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_nfsd4_cb] = {
.name = "nfsd4_cb",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_cache] = {
.name = "cache",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_nfsd] = {
.name = "nfsd",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
[RPCAUTH_gssd] = {
.name = "gssd",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
};
static const struct rpc_filelist gssd_dummy_clnt_dir[] = {
[0] = {
.name = "clntXX",
- .mode = S_IFDIR | S_IRUGO | S_IXUGO,
+ .mode = S_IFDIR | 0555,
},
};
[0] = {
.name = "info",
.i_fop = &rpc_dummy_info_operations,
- .mode = S_IFREG | S_IRUSR,
+ .mode = S_IFREG | 0400,
},
};
sb->s_d_op = &simple_dentry_operations;
sb->s_time_gran = 1;
- inode = rpc_get_inode(sb, S_IFDIR | S_IRUGO | S_IXUGO);
+ inode = rpc_get_inode(sb, S_IFDIR | 0555);
sb->s_root = root = d_make_root(inode);
if (!root)
return -ENOMEM;
static struct pernet_operations sysctl_pernet_ops = {
.init = sysctl_net_init,
.exit = sysctl_net_exit,
- .async = true,
};
static struct ctl_table_header *net_header;
Saying Y here will enable support for running TIPC over IP/UDP
bool
default y
+
+config TIPC_DIAG
+ tristate "TIPC: socket monitoring interface"
+ depends on TIPC
+ default y
+ ---help---
+ Support for TIPC socket monitoring interface used by ss tool.
+ If unsure, say Y.
tipc-$(CONFIG_TIPC_MEDIA_UDP) += udp_media.o
tipc-$(CONFIG_TIPC_MEDIA_IB) += ib_media.o
tipc-$(CONFIG_SYSCTL) += sysctl.o
+
+
+obj-$(CONFIG_TIPC_DIAG) += diag.o
+
+tipc_diag-y := diag.o
/*
* net/tipc/addr.c: TIPC address utility routines
*
- * Copyright (c) 2000-2006, Ericsson AB
+ * Copyright (c) 2000-2006, 2018, Ericsson AB
* Copyright (c) 2004-2005, 2010-2011, Wind River Systems
* All rights reserved.
*
* POSSIBILITY OF SUCH DAMAGE.
*/
-#include <linux/kernel.h>
#include "addr.h"
#include "core.h"
-/**
- * in_own_cluster - test for cluster inclusion; <0.0.0> always matches
- */
-int in_own_cluster(struct net *net, u32 addr)
+bool tipc_in_scope(bool legacy_format, u32 domain, u32 addr)
{
- return in_own_cluster_exact(net, addr) || !addr;
+ if (!domain || (domain == addr))
+ return true;
+ if (!legacy_format)
+ return false;
+ if (domain == tipc_cluster_mask(addr)) /* domain <Z.C.0> */
+ return true;
+ if (domain == (addr & TIPC_ZONE_CLUSTER_MASK)) /* domain <Z.C.0> */
+ return true;
+ if (domain == (addr & TIPC_ZONE_MASK)) /* domain <Z.0.0> */
+ return true;
+ return false;
}
-int in_own_cluster_exact(struct net *net, u32 addr)
+void tipc_set_node_id(struct net *net, u8 *id)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
+ struct tipc_net *tn = tipc_net(net);
+ u32 *tmp = (u32 *)id;
- return !((addr ^ tn->own_addr) >> 12);
+ memcpy(tn->node_id, id, NODE_ID_LEN);
+ tipc_nodeid2string(tn->node_id_string, id);
+ tn->trial_addr = tmp[0] ^ tmp[1] ^ tmp[2] ^ tmp[3];
+ pr_info("Own node identity %s, cluster identity %u\n",
+ tipc_own_id_string(net), tn->net_id);
}
-/**
- * in_own_node - test for node inclusion; <0.0.0> always matches
- */
-int in_own_node(struct net *net, u32 addr)
+void tipc_set_node_addr(struct net *net, u32 addr)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
+ struct tipc_net *tn = tipc_net(net);
+ u8 node_id[NODE_ID_LEN] = {0,};
- return (addr == tn->own_addr) || !addr;
+ tn->node_addr = addr;
+ if (!tipc_own_id(net)) {
+ sprintf(node_id, "%x", addr);
+ tipc_set_node_id(net, node_id);
+ }
+ tn->trial_addr = addr;
+ pr_info("32-bit node address hash set to %x\n", addr);
}
-/**
- * tipc_addr_domain_valid - validates a network domain address
- *
- * Accepts <Z.C.N>, <Z.C.0>, <Z.0.0>, and <0.0.0>,
- * where Z, C, and N are non-zero.
- *
- * Returns 1 if domain address is valid, otherwise 0
- */
-int tipc_addr_domain_valid(u32 addr)
+char *tipc_nodeid2string(char *str, u8 *id)
{
- u32 n = tipc_node(addr);
- u32 c = tipc_cluster(addr);
- u32 z = tipc_zone(addr);
+ int i;
+ u8 c;
- if (n && (!z || !c))
- return 0;
- if (c && !z)
- return 0;
- return 1;
-}
+ /* Already a string ? */
+ for (i = 0; i < NODE_ID_LEN; i++) {
+ c = id[i];
+ if (c >= '0' && c <= '9')
+ continue;
+ if (c >= 'A' && c <= 'Z')
+ continue;
+ if (c >= 'a' && c <= 'z')
+ continue;
+ if (c == '.')
+ continue;
+ if (c == ':')
+ continue;
+ if (c == '_')
+ continue;
+ if (c == '-')
+ continue;
+ if (c == '@')
+ continue;
+ if (c != 0)
+ break;
+ }
+ if (i == NODE_ID_LEN) {
+ memcpy(str, id, NODE_ID_LEN);
+ str[NODE_ID_LEN] = 0;
+ return str;
+ }
-/**
- * tipc_addr_node_valid - validates a proposed network address for this node
- *
- * Accepts <Z.C.N>, where Z, C, and N are non-zero.
- *
- * Returns 1 if address can be used, otherwise 0
- */
-int tipc_addr_node_valid(u32 addr)
-{
- return tipc_addr_domain_valid(addr) && tipc_node(addr);
-}
+ /* Translate to hex string */
+ for (i = 0; i < NODE_ID_LEN; i++)
+ sprintf(&str[2 * i], "%02x", id[i]);
-int tipc_in_scope(u32 domain, u32 addr)
-{
- if (!domain || (domain == addr))
- return 1;
- if (domain == tipc_cluster_mask(addr)) /* domain <Z.C.0> */
- return 1;
- if (domain == tipc_zone_mask(addr)) /* domain <Z.0.0> */
- return 1;
- return 0;
-}
+ /* Strip off trailing zeroes */
+ for (i = NODE_ID_STR_LEN - 2; str[i] == '0'; i--)
+ str[i] = 0;
-char *tipc_addr_string_fill(char *string, u32 addr)
-{
- snprintf(string, 16, "<%u.%u.%u>",
- tipc_zone(addr), tipc_cluster(addr), tipc_node(addr));
- return string;
+ return str;
}
/*
* net/tipc/addr.h: Include file for TIPC address utility routines
*
- * Copyright (c) 2000-2006, Ericsson AB
+ * Copyright (c) 2000-2006, 2018, Ericsson AB
* Copyright (c) 2004-2005, Wind River Systems
* All rights reserved.
*
static inline u32 tipc_own_addr(struct net *net)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
+ return tipc_net(net)->node_addr;
+}
+
+static inline u8 *tipc_own_id(struct net *net)
+{
+ struct tipc_net *tn = tipc_net(net);
- return tn->own_addr;
+ if (!strlen(tn->node_id_string))
+ return NULL;
+ return tn->node_id;
}
-static inline u32 tipc_zone_mask(u32 addr)
+static inline char *tipc_own_id_string(struct net *net)
{
- return addr & TIPC_ZONE_MASK;
+ return tipc_net(net)->node_id_string;
}
static inline u32 tipc_cluster_mask(u32 addr)
return sc != TIPC_NODE_SCOPE ? 0 : tipc_own_addr(net);
}
-u32 tipc_own_addr(struct net *net);
-int in_own_cluster(struct net *net, u32 addr);
-int in_own_cluster_exact(struct net *net, u32 addr);
-int in_own_node(struct net *net, u32 addr);
-u32 addr_domain(struct net *net, u32 sc);
-int tipc_addr_domain_valid(u32);
-int tipc_addr_node_valid(u32 addr);
-int tipc_in_scope(u32 domain, u32 addr);
-int tipc_addr_scope(u32 domain);
-char *tipc_addr_string_fill(char *string, u32 addr);
+static inline int in_own_node(struct net *net, u32 addr)
+{
+ return addr == tipc_own_addr(net) || !addr;
+}
+
+bool tipc_in_scope(bool legacy_format, u32 domain, u32 addr);
+void tipc_set_node_id(struct net *net, u8 *id);
+void tipc_set_node_addr(struct net *net, u32 addr);
+char *tipc_nodeid2string(char *str, u8 *id);
+u32 tipc_node_id2hash(u8 *id128);
#endif
rcu_read_lock();
b = rcu_dereference_rtnl(tn->bearer_list[bearer_id]);
if (b)
- tipc_disc_add_dest(b->link_req);
+ tipc_disc_add_dest(b->disc);
rcu_read_unlock();
}
rcu_read_lock();
b = rcu_dereference_rtnl(tn->bearer_list[bearer_id]);
if (b)
- tipc_disc_remove_dest(b->link_req);
+ tipc_disc_remove_dest(b->disc);
rcu_read_unlock();
}
* tipc_enable_bearer - enable bearer with the given name
*/
static int tipc_enable_bearer(struct net *net, const char *name,
- u32 disc_domain, u32 priority,
+ u32 disc_domain, u32 prio,
struct nlattr *attr[])
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
+ struct tipc_net *tn = tipc_net(net);
+ struct tipc_bearer_names b_names;
+ int with_this_prio = 1;
struct tipc_bearer *b;
struct tipc_media *m;
- struct tipc_bearer_names b_names;
struct sk_buff *skb;
- char addr_string[16];
- u32 bearer_id;
- u32 with_this_prio;
- u32 i;
+ int bearer_id = 0;
int res = -EINVAL;
+ char *errstr = "";
- if (!tn->own_addr) {
- pr_warn("Bearer <%s> rejected, not supported in standalone mode\n",
- name);
- return -ENOPROTOOPT;
- }
if (!bearer_name_validate(name, &b_names)) {
- pr_warn("Bearer <%s> rejected, illegal name\n", name);
- return -EINVAL;
- }
- if (tipc_addr_domain_valid(disc_domain) &&
- (disc_domain != tn->own_addr)) {
- if (tipc_in_scope(disc_domain, tn->own_addr)) {
- disc_domain = tn->own_addr & TIPC_ZONE_CLUSTER_MASK;
- res = 0; /* accept any node in own cluster */
- } else if (in_own_cluster_exact(net, disc_domain))
- res = 0; /* accept specified node in own cluster */
+ errstr = "illegal name";
+ goto rejected;
}
- if (res) {
- pr_warn("Bearer <%s> rejected, illegal discovery domain\n",
- name);
- return -EINVAL;
- }
- if ((priority > TIPC_MAX_LINK_PRI) &&
- (priority != TIPC_MEDIA_LINK_PRI)) {
- pr_warn("Bearer <%s> rejected, illegal priority\n", name);
- return -EINVAL;
+
+ if (prio > TIPC_MAX_LINK_PRI && prio != TIPC_MEDIA_LINK_PRI) {
+ errstr = "illegal priority";
+ goto rejected;
}
m = tipc_media_find(b_names.media_name);
if (!m) {
- pr_warn("Bearer <%s> rejected, media <%s> not registered\n",
- name, b_names.media_name);
- return -EINVAL;
+ errstr = "media not registered";
+ goto rejected;
}
- if (priority == TIPC_MEDIA_LINK_PRI)
- priority = m->priority;
+ if (prio == TIPC_MEDIA_LINK_PRI)
+ prio = m->priority;
-restart:
- bearer_id = MAX_BEARERS;
- with_this_prio = 1;
- for (i = MAX_BEARERS; i-- != 0; ) {
- b = rtnl_dereference(tn->bearer_list[i]);
- if (!b) {
- bearer_id = i;
- continue;
- }
+ /* Check new bearer vs existing ones and find free bearer id if any */
+ while (bearer_id < MAX_BEARERS) {
+ b = rtnl_dereference(tn->bearer_list[bearer_id]);
+ if (!b)
+ break;
if (!strcmp(name, b->name)) {
- pr_warn("Bearer <%s> rejected, already enabled\n",
- name);
- return -EINVAL;
+ errstr = "already enabled";
+ goto rejected;
}
- if ((b->priority == priority) &&
- (++with_this_prio > 2)) {
- if (priority-- == 0) {
- pr_warn("Bearer <%s> rejected, duplicate priority\n",
- name);
- return -EINVAL;
- }
- pr_warn("Bearer <%s> priority adjustment required %u->%u\n",
- name, priority + 1, priority);
- goto restart;
+ bearer_id++;
+ if (b->priority != prio)
+ continue;
+ if (++with_this_prio <= 2)
+ continue;
+ pr_warn("Bearer <%s>: already 2 bearers with priority %u\n",
+ name, prio);
+ if (prio == TIPC_MIN_LINK_PRI) {
+ errstr = "cannot adjust to lower";
+ goto rejected;
}
+ pr_warn("Bearer <%s>: trying with adjusted priority\n", name);
+ prio--;
+ bearer_id = 0;
+ with_this_prio = 1;
}
+
if (bearer_id >= MAX_BEARERS) {
- pr_warn("Bearer <%s> rejected, bearer limit reached (%u)\n",
- name, MAX_BEARERS);
- return -EINVAL;
+ errstr = "max 3 bearers permitted";
+ goto rejected;
}
b = kzalloc(sizeof(*b), GFP_ATOMIC);
b->media = m;
res = m->enable_media(net, b, attr);
if (res) {
- pr_warn("Bearer <%s> rejected, enable failure (%d)\n",
- name, -res);
kfree(b);
- return -EINVAL;
+ errstr = "failed to enable media";
+ goto rejected;
}
b->identity = bearer_id;
b->window = m->window;
b->domain = disc_domain;
b->net_plane = bearer_id + 'A';
- b->priority = priority;
+ b->priority = prio;
test_and_set_bit_lock(0, &b->up);
res = tipc_disc_create(net, b, &b->bcast_addr, &skb);
if (res) {
bearer_disable(net, b);
- pr_warn("Bearer <%s> rejected, discovery object creation failed\n",
- name);
- return -EINVAL;
+ kfree(b);
+ errstr = "failed to create discoverer";
+ goto rejected;
}
rcu_assign_pointer(tn->bearer_list[bearer_id], b);
return -ENOMEM;
}
- pr_info("Enabled bearer <%s>, discovery domain %s, priority %u\n",
- name,
- tipc_addr_string_fill(addr_string, disc_domain), priority);
+ pr_info("Enabled bearer <%s>, priority %u\n", name, prio);
+
+ return res;
+rejected:
+ pr_warn("Enabling of bearer <%s> rejected, %s\n", name, errstr);
return res;
}
tipc_node_delete_links(net, bearer_id);
b->media->disable_media(b);
RCU_INIT_POINTER(b->media_ptr, NULL);
- if (b->link_req)
- tipc_disc_delete(b->link_req);
+ if (b->disc)
+ tipc_disc_delete(b->disc);
RCU_INIT_POINTER(tn->bearer_list[bearer_id], NULL);
kfree_rcu(b, rcu);
tipc_mon_delete(net, bearer_id);
int tipc_enable_l2_media(struct net *net, struct tipc_bearer *b,
struct nlattr *attr[])
{
+ char *dev_name = strchr((const char *)b->name, ':') + 1;
+ int hwaddr_len = b->media->hwaddr_len;
+ u8 node_id[NODE_ID_LEN] = {0,};
struct net_device *dev;
- char *driver_name = strchr((const char *)b->name, ':') + 1;
/* Find device with specified name */
- dev = dev_get_by_name(net, driver_name);
+ dev = dev_get_by_name(net, dev_name);
if (!dev)
return -ENODEV;
if (tipc_mtu_bad(dev, 0)) {
return -EINVAL;
}
+ /* Autoconfigure own node identity if needed */
+ if (!tipc_own_id(net) && hwaddr_len <= NODE_ID_LEN) {
+ memcpy(node_id, dev->dev_addr, hwaddr_len);
+ tipc_net_init(net, node_id, 0);
+ }
+ if (!tipc_own_id(net)) {
+ pr_warn("Failed to obtain node identity\n");
+ return -EINVAL;
+ }
+
/* Associate TIPC bearer with L2 bearer */
rcu_assign_pointer(b->media_ptr, dev);
b->pt.dev = dev;
b->pt.func = tipc_l2_rcv_msg;
dev_add_pack(&b->pt);
memset(&b->bcast_addr, 0, sizeof(b->bcast_addr));
- memcpy(b->bcast_addr.value, dev->broadcast, b->media->hwaddr_len);
+ memcpy(b->bcast_addr.value, dev->broadcast, hwaddr_len);
b->bcast_addr.media_id = b->media->type_id;
b->bcast_addr.broadcast = TIPC_BROADCAST_SUPPORT;
b->mtu = dev->mtu;
char *bearer;
struct nlattr *attrs[TIPC_NLA_BEARER_MAX + 1];
struct net *net = sock_net(skb->sk);
- struct tipc_net *tn = net_generic(net, tipc_net_id);
- u32 domain;
+ u32 domain = 0;
u32 prio;
prio = TIPC_MEDIA_LINK_PRI;
- domain = tn->own_addr & TIPC_ZONE_CLUSTER_MASK;
if (!info->attrs[TIPC_NLA_BEARER])
return -EINVAL;
u32 tolerance;
u32 domain;
u32 identity;
- struct tipc_link_req *link_req;
+ struct tipc_discoverer *disc;
char net_plane;
unsigned long up;
};
int err;
tn->net_id = 4711;
- tn->own_addr = 0;
+ tn->node_addr = 0;
+ tn->trial_addr = 0;
+ tn->addr_trial_end = 0;
+ memset(tn->node_id, 0, sizeof(tn->node_id));
+ memset(tn->node_id_string, 0, sizeof(tn->node_id_string));
tn->mon_threshold = TIPC_DEF_MON_THRESHOLD;
get_random_bytes(&tn->random, sizeof(int));
INIT_LIST_HEAD(&tn->node_list);
.exit = tipc_exit_net,
.id = &tipc_net_id,
.size = sizeof(struct tipc_net),
- .async = true,
};
static int __init tipc_init(void)
/*
* net/tipc/core.h: Include file for TIPC global declarations
*
- * Copyright (c) 2005-2006, 2013 Ericsson AB
+ * Copyright (c) 2005-2006, 2013-2018 Ericsson AB
* Copyright (c) 2005-2007, 2010-2013, Wind River Systems
* All rights reserved.
*
#define NODE_HTABLE_SIZE 512
#define MAX_BEARERS 3
#define TIPC_DEF_MON_THRESHOLD 32
+#define NODE_ID_LEN 16
+#define NODE_ID_STR_LEN (NODE_ID_LEN * 2 + 1)
extern unsigned int tipc_net_id __read_mostly;
extern int sysctl_tipc_rmem[3] __read_mostly;
extern int sysctl_tipc_named_timeout __read_mostly;
struct tipc_net {
- u32 own_addr;
+ u8 node_id[NODE_ID_LEN];
+ u32 node_addr;
+ u32 trial_addr;
+ unsigned long addr_trial_end;
+ char node_id_string[NODE_ID_STR_LEN];
int net_id;
int random;
+ bool legacy_addr_format;
/* Node table and node list */
spinlock_t node_list_lock;
--- /dev/null
+/*
+ * net/tipc/diag.c: TIPC socket diag
+ *
+ * Copyright (c) 2018, Ericsson AB
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * 3. Neither the names of the copyright holders nor the names of its
+ * contributors may be used to endorse or promote products derived from
+ * this software without specific prior written permission.
+ *
+ * Alternatively, this software may be distributed under the terms of the
+ * GNU General Public License ("GPL") version 2 as published by the Free
+ * Software Foundation.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "ASIS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "core.h"
+#include "socket.h"
+#include <linux/sock_diag.h>
+#include <linux/tipc_sockets_diag.h>
+
+static u64 __tipc_diag_gen_cookie(struct sock *sk)
+{
+ u32 res[2];
+
+ sock_diag_save_cookie(sk, res);
+ return *((u64 *)res);
+}
+
+static int __tipc_add_sock_diag(struct sk_buff *skb,
+ struct netlink_callback *cb,
+ struct tipc_sock *tsk)
+{
+ struct tipc_sock_diag_req *req = nlmsg_data(cb->nlh);
+ struct nlmsghdr *nlh;
+ int err;
+
+ nlh = nlmsg_put_answer(skb, cb, SOCK_DIAG_BY_FAMILY, 0,
+ NLM_F_MULTI);
+ if (!nlh)
+ return -EMSGSIZE;
+
+ err = tipc_sk_fill_sock_diag(skb, tsk, req->tidiag_states,
+ __tipc_diag_gen_cookie);
+ if (err)
+ return err;
+
+ nlmsg_end(skb, nlh);
+ return 0;
+}
+
+static int tipc_diag_dump(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ return tipc_nl_sk_walk(skb, cb, __tipc_add_sock_diag);
+}
+
+static int tipc_sock_diag_handler_dump(struct sk_buff *skb,
+ struct nlmsghdr *h)
+{
+ int hdrlen = sizeof(struct tipc_sock_diag_req);
+ struct net *net = sock_net(skb->sk);
+
+ if (nlmsg_len(h) < hdrlen)
+ return -EINVAL;
+
+ if (h->nlmsg_flags & NLM_F_DUMP) {
+ struct netlink_dump_control c = {
+ .dump = tipc_diag_dump,
+ };
+ netlink_dump_start(net->diag_nlsk, skb, h, &c);
+ return 0;
+ }
+ return -EOPNOTSUPP;
+}
+
+static const struct sock_diag_handler tipc_sock_diag_handler = {
+ .family = AF_TIPC,
+ .dump = tipc_sock_diag_handler_dump,
+};
+
+static int __init tipc_diag_init(void)
+{
+ return sock_diag_register(&tipc_sock_diag_handler);
+}
+
+static void __exit tipc_diag_exit(void)
+{
+ sock_diag_unregister(&tipc_sock_diag_handler);
+}
+
+module_init(tipc_diag_init);
+module_exit(tipc_diag_exit);
+
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_ALIAS_NET_PF_PROTO_TYPE(PF_NETLINK, NETLINK_SOCK_DIAG, AF_TIPC);
/*
* net/tipc/discover.c
*
- * Copyright (c) 2003-2006, 2014-2015, Ericsson AB
+ * Copyright (c) 2003-2006, 2014-2018, Ericsson AB
* Copyright (c) 2005-2006, 2010-2011, Wind River Systems
* All rights reserved.
*
#include "discover.h"
/* min delay during bearer start up */
-#define TIPC_LINK_REQ_INIT msecs_to_jiffies(125)
+#define TIPC_DISC_INIT msecs_to_jiffies(125)
/* max delay if bearer has no links */
-#define TIPC_LINK_REQ_FAST msecs_to_jiffies(1000)
+#define TIPC_DISC_FAST msecs_to_jiffies(1000)
/* max delay if bearer has links */
-#define TIPC_LINK_REQ_SLOW msecs_to_jiffies(60000)
+#define TIPC_DISC_SLOW msecs_to_jiffies(60000)
/* indicates no timer in use */
-#define TIPC_LINK_REQ_INACTIVE 0xffffffff
+#define TIPC_DISC_INACTIVE 0xffffffff
/**
- * struct tipc_link_req - information about an ongoing link setup request
+ * struct tipc_discoverer - information about an ongoing link setup request
* @bearer_id: identity of bearer issuing requests
* @net: network namespace instance
* @dest: destination address for request messages
* @domain: network domain to which links can be established
* @num_nodes: number of nodes currently discovered (i.e. with an active link)
* @lock: spinlock for controlling access to requests
- * @buf: request message to be (repeatedly) sent
+ * @skb: request message to be (repeatedly) sent
* @timer: timer governing period between requests
* @timer_intv: current interval between requests (in ms)
*/
-struct tipc_link_req {
+struct tipc_discoverer {
u32 bearer_id;
struct tipc_media_addr dest;
struct net *net;
u32 domain;
int num_nodes;
spinlock_t lock;
- struct sk_buff *buf;
+ struct sk_buff *skb;
struct timer_list timer;
unsigned long timer_intv;
};
* @type: message type (request or response)
* @b: ptr to bearer issuing message
*/
-static void tipc_disc_init_msg(struct net *net, struct sk_buff *buf, u32 type,
- struct tipc_bearer *b)
+static void tipc_disc_init_msg(struct net *net, struct sk_buff *skb,
+ u32 mtyp, struct tipc_bearer *b)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
- struct tipc_msg *msg;
+ struct tipc_net *tn = tipc_net(net);
u32 dest_domain = b->domain;
+ struct tipc_msg *hdr;
- msg = buf_msg(buf);
- tipc_msg_init(tn->own_addr, msg, LINK_CONFIG, type,
+ hdr = buf_msg(skb);
+ tipc_msg_init(tn->trial_addr, hdr, LINK_CONFIG, mtyp,
MAX_H_SIZE, dest_domain);
- msg_set_non_seq(msg, 1);
- msg_set_node_sig(msg, tn->random);
- msg_set_node_capabilities(msg, TIPC_NODE_CAPABILITIES);
- msg_set_dest_domain(msg, dest_domain);
- msg_set_bc_netid(msg, tn->net_id);
- b->media->addr2msg(msg_media_addr(msg), &b->addr);
+ msg_set_size(hdr, MAX_H_SIZE + NODE_ID_LEN);
+ msg_set_non_seq(hdr, 1);
+ msg_set_node_sig(hdr, tn->random);
+ msg_set_node_capabilities(hdr, TIPC_NODE_CAPABILITIES);
+ msg_set_dest_domain(hdr, dest_domain);
+ msg_set_bc_netid(hdr, tn->net_id);
+ b->media->addr2msg(msg_media_addr(hdr), &b->addr);
+ msg_set_node_id(hdr, tipc_own_id(net));
+}
+
+static void tipc_disc_msg_xmit(struct net *net, u32 mtyp, u32 dst,
+ u32 src, u32 sugg_addr,
+ struct tipc_media_addr *maddr,
+ struct tipc_bearer *b)
+{
+ struct tipc_msg *hdr;
+ struct sk_buff *skb;
+
+ skb = tipc_buf_acquire(MAX_H_SIZE + NODE_ID_LEN, GFP_ATOMIC);
+ if (!skb)
+ return;
+ hdr = buf_msg(skb);
+ tipc_disc_init_msg(net, skb, mtyp, b);
+ msg_set_sugg_node_addr(hdr, sugg_addr);
+ msg_set_dest_domain(hdr, dst);
+ tipc_bearer_xmit_skb(net, b->identity, skb, maddr);
}
/**
static void disc_dupl_alert(struct tipc_bearer *b, u32 node_addr,
struct tipc_media_addr *media_addr)
{
- char node_addr_str[16];
char media_addr_str[64];
- tipc_addr_string_fill(node_addr_str, node_addr);
tipc_media_addr_printf(media_addr_str, sizeof(media_addr_str),
media_addr);
- pr_warn("Duplicate %s using %s seen on <%s>\n", node_addr_str,
+ pr_warn("Duplicate %x using %s seen on <%s>\n", node_addr,
media_addr_str, b->name);
}
+/* tipc_disc_addr_trial(): - handle an address uniqueness trial from peer
+ */
+static bool tipc_disc_addr_trial_msg(struct tipc_discoverer *d,
+ struct tipc_media_addr *maddr,
+ struct tipc_bearer *b,
+ u32 dst, u32 src,
+ u32 sugg_addr,
+ u8 *peer_id,
+ int mtyp)
+{
+ struct net *net = d->net;
+ struct tipc_net *tn = tipc_net(net);
+ bool trial = time_before(jiffies, tn->addr_trial_end);
+ u32 self = tipc_own_addr(net);
+
+ if (mtyp == DSC_TRIAL_FAIL_MSG) {
+ if (!trial)
+ return true;
+
+ /* Ignore if somebody else already gave new suggestion */
+ if (dst != tn->trial_addr)
+ return true;
+
+ /* Otherwise update trial address and restart trial period */
+ tn->trial_addr = sugg_addr;
+ msg_set_prevnode(buf_msg(d->skb), sugg_addr);
+ tn->addr_trial_end = jiffies + msecs_to_jiffies(1000);
+ return true;
+ }
+
+ /* Apply trial address if we just left trial period */
+ if (!trial && !self) {
+ tipc_net_finalize(net, tn->trial_addr);
+ msg_set_type(buf_msg(d->skb), DSC_REQ_MSG);
+ }
+
+ if (mtyp != DSC_TRIAL_MSG)
+ return false;
+
+ sugg_addr = tipc_node_try_addr(net, peer_id, src);
+ if (sugg_addr)
+ tipc_disc_msg_xmit(net, DSC_TRIAL_FAIL_MSG, src,
+ self, sugg_addr, maddr, b);
+ return true;
+}
+
/**
* tipc_disc_rcv - handle incoming discovery message (request or response)
- * @net: the applicable net namespace
- * @buf: buffer containing message
- * @bearer: bearer that message arrived on
+ * @net: applicable net namespace
+ * @skb: buffer containing message
+ * @b: bearer that message arrived on
*/
void tipc_disc_rcv(struct net *net, struct sk_buff *skb,
- struct tipc_bearer *bearer)
+ struct tipc_bearer *b)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
- struct tipc_media_addr maddr;
- struct sk_buff *rskb;
+ struct tipc_net *tn = tipc_net(net);
struct tipc_msg *hdr = buf_msg(skb);
- u32 ddom = msg_dest_domain(hdr);
- u32 onode = msg_prevnode(hdr);
+ u16 caps = msg_node_capabilities(hdr);
+ bool legacy = tn->legacy_addr_format;
+ u32 sugg = msg_sugg_node_addr(hdr);
+ u32 signature = msg_node_sig(hdr);
+ u8 peer_id[NODE_ID_LEN] = {0,};
+ u32 dst = msg_dest_domain(hdr);
u32 net_id = msg_bc_netid(hdr);
+ struct tipc_media_addr maddr;
+ u32 src = msg_prevnode(hdr);
u32 mtyp = msg_type(hdr);
- u32 signature = msg_node_sig(hdr);
- u16 caps = msg_node_capabilities(hdr);
- bool respond = false;
bool dupl_addr = false;
+ bool respond = false;
+ u32 self;
int err;
- err = bearer->media->msg2addr(bearer, &maddr, msg_media_addr(hdr));
- kfree_skb(skb);
- if (err)
- return;
+ skb_linearize(skb);
+ hdr = buf_msg(skb);
- /* Ensure message from node is valid and communication is permitted */
- if (net_id != tn->net_id)
+ if (caps & TIPC_NODE_ID128)
+ memcpy(peer_id, msg_node_id(hdr), NODE_ID_LEN);
+ else
+ sprintf(peer_id, "%x", src);
+
+ err = b->media->msg2addr(b, &maddr, msg_media_addr(hdr));
+ kfree_skb(skb);
+ if (err || maddr.broadcast) {
+ pr_warn_ratelimited("Rcv corrupt discovery message\n");
return;
- if (maddr.broadcast)
+ }
+ /* Ignore discovery messages from own node */
+ if (!memcmp(&maddr, &b->addr, sizeof(maddr)))
return;
- if (!tipc_addr_domain_valid(ddom))
+ if (net_id != tn->net_id)
return;
- if (!tipc_addr_node_valid(onode))
+ if (tipc_disc_addr_trial_msg(b->disc, &maddr, b, dst,
+ src, sugg, peer_id, mtyp))
return;
+ self = tipc_own_addr(net);
- if (in_own_node(net, onode)) {
- if (memcmp(&maddr, &bearer->addr, sizeof(maddr)))
- disc_dupl_alert(bearer, tn->own_addr, &maddr);
+ /* Message from somebody using this node's address */
+ if (in_own_node(net, src)) {
+ disc_dupl_alert(b, self, &maddr);
return;
}
- if (!tipc_in_scope(ddom, tn->own_addr))
+ if (!tipc_in_scope(legacy, dst, self))
return;
- if (!tipc_in_scope(bearer->domain, onode))
+ if (!tipc_in_scope(legacy, b->domain, src))
return;
-
- tipc_node_check_dest(net, onode, bearer, caps, signature,
+ tipc_node_check_dest(net, src, peer_id, b, caps, signature,
&maddr, &respond, &dupl_addr);
if (dupl_addr)
- disc_dupl_alert(bearer, onode, &maddr);
-
- /* Send response, if necessary */
- if (respond && (mtyp == DSC_REQ_MSG)) {
- rskb = tipc_buf_acquire(MAX_H_SIZE, GFP_ATOMIC);
- if (!rskb)
- return;
- tipc_disc_init_msg(net, rskb, DSC_RESP_MSG, bearer);
- tipc_bearer_xmit_skb(net, bearer->identity, rskb, &maddr);
- }
+ disc_dupl_alert(b, src, &maddr);
+ if (!respond)
+ return;
+ if (mtyp != DSC_REQ_MSG)
+ return;
+ tipc_disc_msg_xmit(net, DSC_RESP_MSG, src, self, 0, &maddr, b);
}
-/**
- * disc_update - update frequency of periodic link setup requests
- * @req: ptr to link request structure
- *
- * Reinitiates discovery process if discovery object has no associated nodes
- * and is either not currently searching or is searching at a slow rate
+/* tipc_disc_add_dest - increment set of discovered nodes
*/
-static void disc_update(struct tipc_link_req *req)
+void tipc_disc_add_dest(struct tipc_discoverer *d)
{
- if (!req->num_nodes) {
- if ((req->timer_intv == TIPC_LINK_REQ_INACTIVE) ||
- (req->timer_intv > TIPC_LINK_REQ_FAST)) {
- req->timer_intv = TIPC_LINK_REQ_INIT;
- mod_timer(&req->timer, jiffies + req->timer_intv);
- }
- }
+ spin_lock_bh(&d->lock);
+ d->num_nodes++;
+ spin_unlock_bh(&d->lock);
}
-/**
- * tipc_disc_add_dest - increment set of discovered nodes
- * @req: ptr to link request structure
+/* tipc_disc_remove_dest - decrement set of discovered nodes
*/
-void tipc_disc_add_dest(struct tipc_link_req *req)
+void tipc_disc_remove_dest(struct tipc_discoverer *d)
{
- spin_lock_bh(&req->lock);
- req->num_nodes++;
- spin_unlock_bh(&req->lock);
-}
+ int intv, num;
-/**
- * tipc_disc_remove_dest - decrement set of discovered nodes
- * @req: ptr to link request structure
- */
-void tipc_disc_remove_dest(struct tipc_link_req *req)
-{
- spin_lock_bh(&req->lock);
- req->num_nodes--;
- disc_update(req);
- spin_unlock_bh(&req->lock);
+ spin_lock_bh(&d->lock);
+ d->num_nodes--;
+ num = d->num_nodes;
+ intv = d->timer_intv;
+ if (!num && (intv == TIPC_DISC_INACTIVE || intv > TIPC_DISC_FAST)) {
+ d->timer_intv = TIPC_DISC_INIT;
+ mod_timer(&d->timer, jiffies + d->timer_intv);
+ }
+ spin_unlock_bh(&d->lock);
}
-/**
- * disc_timeout - send a periodic link setup request
- * @data: ptr to link request structure
- *
+/* tipc_disc_timeout - send a periodic link setup request
* Called whenever a link setup request timer associated with a bearer expires.
+ * - Keep doubling time between sent request until limit is reached;
+ * - Hold at fast polling rate if we don't have any associated nodes
+ * - Otherwise hold at slow polling rate
*/
-static void disc_timeout(struct timer_list *t)
+static void tipc_disc_timeout(struct timer_list *t)
{
- struct tipc_link_req *req = from_timer(req, t, timer);
- struct sk_buff *skb;
- int max_delay;
+ struct tipc_discoverer *d = from_timer(d, t, timer);
+ struct tipc_net *tn = tipc_net(d->net);
+ u32 self = tipc_own_addr(d->net);
+ struct tipc_media_addr maddr;
+ struct sk_buff *skb = NULL;
+ struct net *net = d->net;
+ u32 bearer_id;
- spin_lock_bh(&req->lock);
+ spin_lock_bh(&d->lock);
/* Stop searching if only desired node has been found */
- if (tipc_node(req->domain) && req->num_nodes) {
- req->timer_intv = TIPC_LINK_REQ_INACTIVE;
+ if (tipc_node(d->domain) && d->num_nodes) {
+ d->timer_intv = TIPC_DISC_INACTIVE;
goto exit;
}
- /*
- * Send discovery message, then update discovery timer
- *
- * Keep doubling time between requests until limit is reached;
- * hold at fast polling rate if don't have any associated nodes,
- * otherwise hold at slow polling rate
- */
- skb = skb_clone(req->buf, GFP_ATOMIC);
- if (skb)
- tipc_bearer_xmit_skb(req->net, req->bearer_id, skb, &req->dest);
- req->timer_intv *= 2;
- if (req->num_nodes)
- max_delay = TIPC_LINK_REQ_SLOW;
- else
- max_delay = TIPC_LINK_REQ_FAST;
- if (req->timer_intv > max_delay)
- req->timer_intv = max_delay;
+ /* Did we just leave the address trial period ? */
+ if (!self && !time_before(jiffies, tn->addr_trial_end)) {
+ self = tn->trial_addr;
+ tipc_net_finalize(net, self);
+ msg_set_prevnode(buf_msg(d->skb), self);
+ msg_set_type(buf_msg(d->skb), DSC_REQ_MSG);
+ }
+
+ /* Adjust timeout interval according to discovery phase */
+ if (time_before(jiffies, tn->addr_trial_end)) {
+ d->timer_intv = TIPC_DISC_INIT;
+ } else {
+ d->timer_intv *= 2;
+ if (d->num_nodes && d->timer_intv > TIPC_DISC_SLOW)
+ d->timer_intv = TIPC_DISC_SLOW;
+ else if (!d->num_nodes && d->timer_intv > TIPC_DISC_FAST)
+ d->timer_intv = TIPC_DISC_FAST;
+ }
- mod_timer(&req->timer, jiffies + req->timer_intv);
+ mod_timer(&d->timer, jiffies + d->timer_intv);
+ memcpy(&maddr, &d->dest, sizeof(maddr));
+ skb = skb_clone(d->skb, GFP_ATOMIC);
+ bearer_id = d->bearer_id;
exit:
- spin_unlock_bh(&req->lock);
+ spin_unlock_bh(&d->lock);
+ if (skb)
+ tipc_bearer_xmit_skb(net, bearer_id, skb, &maddr);
}
/**
int tipc_disc_create(struct net *net, struct tipc_bearer *b,
struct tipc_media_addr *dest, struct sk_buff **skb)
{
- struct tipc_link_req *req;
+ struct tipc_net *tn = tipc_net(net);
+ struct tipc_discoverer *d;
- req = kmalloc(sizeof(*req), GFP_ATOMIC);
- if (!req)
+ d = kmalloc(sizeof(*d), GFP_ATOMIC);
+ if (!d)
return -ENOMEM;
- req->buf = tipc_buf_acquire(MAX_H_SIZE, GFP_ATOMIC);
- if (!req->buf) {
- kfree(req);
+ d->skb = tipc_buf_acquire(MAX_H_SIZE + NODE_ID_LEN, GFP_ATOMIC);
+ if (!d->skb) {
+ kfree(d);
return -ENOMEM;
}
+ tipc_disc_init_msg(net, d->skb, DSC_REQ_MSG, b);
- tipc_disc_init_msg(net, req->buf, DSC_REQ_MSG, b);
- memcpy(&req->dest, dest, sizeof(*dest));
- req->net = net;
- req->bearer_id = b->identity;
- req->domain = b->domain;
- req->num_nodes = 0;
- req->timer_intv = TIPC_LINK_REQ_INIT;
- spin_lock_init(&req->lock);
- timer_setup(&req->timer, disc_timeout, 0);
- mod_timer(&req->timer, jiffies + req->timer_intv);
- b->link_req = req;
- *skb = skb_clone(req->buf, GFP_ATOMIC);
+ /* Do we need an address trial period first ? */
+ if (!tipc_own_addr(net)) {
+ tn->addr_trial_end = jiffies + msecs_to_jiffies(1000);
+ msg_set_type(buf_msg(d->skb), DSC_TRIAL_MSG);
+ }
+ memcpy(&d->dest, dest, sizeof(*dest));
+ d->net = net;
+ d->bearer_id = b->identity;
+ d->domain = b->domain;
+ d->num_nodes = 0;
+ d->timer_intv = TIPC_DISC_INIT;
+ spin_lock_init(&d->lock);
+ timer_setup(&d->timer, tipc_disc_timeout, 0);
+ mod_timer(&d->timer, jiffies + d->timer_intv);
+ b->disc = d;
+ *skb = skb_clone(d->skb, GFP_ATOMIC);
return 0;
}
/**
* tipc_disc_delete - destroy object sending periodic link setup requests
- * @req: ptr to link request structure
+ * @d: ptr to link duest structure
*/
-void tipc_disc_delete(struct tipc_link_req *req)
+void tipc_disc_delete(struct tipc_discoverer *d)
{
- del_timer_sync(&req->timer);
- kfree_skb(req->buf);
- kfree(req);
+ del_timer_sync(&d->timer);
+ kfree_skb(d->skb);
+ kfree(d);
}
/**
*/
void tipc_disc_reset(struct net *net, struct tipc_bearer *b)
{
- struct tipc_link_req *req = b->link_req;
+ struct tipc_discoverer *d = b->disc;
+ struct tipc_media_addr maddr;
struct sk_buff *skb;
- spin_lock_bh(&req->lock);
- tipc_disc_init_msg(net, req->buf, DSC_REQ_MSG, b);
- req->net = net;
- req->bearer_id = b->identity;
- req->domain = b->domain;
- req->num_nodes = 0;
- req->timer_intv = TIPC_LINK_REQ_INIT;
- mod_timer(&req->timer, jiffies + req->timer_intv);
- skb = skb_clone(req->buf, GFP_ATOMIC);
+ spin_lock_bh(&d->lock);
+ tipc_disc_init_msg(net, d->skb, DSC_REQ_MSG, b);
+ d->net = net;
+ d->bearer_id = b->identity;
+ d->domain = b->domain;
+ d->num_nodes = 0;
+ d->timer_intv = TIPC_DISC_INIT;
+ memcpy(&maddr, &d->dest, sizeof(maddr));
+ mod_timer(&d->timer, jiffies + d->timer_intv);
+ skb = skb_clone(d->skb, GFP_ATOMIC);
+ spin_unlock_bh(&d->lock);
if (skb)
- tipc_bearer_xmit_skb(net, req->bearer_id, skb, &req->dest);
- spin_unlock_bh(&req->lock);
+ tipc_bearer_xmit_skb(net, b->identity, skb, &maddr);
}
#ifndef _TIPC_DISCOVER_H
#define _TIPC_DISCOVER_H
-struct tipc_link_req;
+struct tipc_discoverer;
int tipc_disc_create(struct net *net, struct tipc_bearer *b_ptr,
struct tipc_media_addr *dest, struct sk_buff **skb);
-void tipc_disc_delete(struct tipc_link_req *req);
+void tipc_disc_delete(struct tipc_discoverer *req);
void tipc_disc_reset(struct net *net, struct tipc_bearer *b_ptr);
-void tipc_disc_add_dest(struct tipc_link_req *req);
-void tipc_disc_remove_dest(struct tipc_link_req *req);
+void tipc_disc_add_dest(struct tipc_discoverer *req);
+void tipc_disc_remove_dest(struct tipc_discoverer *req);
void tipc_disc_rcv(struct net *net, struct sk_buff *buf,
struct tipc_bearer *b_ptr);
*/
bool tipc_link_create(struct net *net, char *if_name, int bearer_id,
int tolerance, char net_plane, u32 mtu, int priority,
- int window, u32 session, u32 ownnode, u32 peer,
- u16 peer_caps,
+ int window, u32 session, u32 self,
+ u32 peer, u8 *peer_id, u16 peer_caps,
struct tipc_link *bc_sndlink,
struct tipc_link *bc_rcvlink,
struct sk_buff_head *inputq,
struct sk_buff_head *namedq,
struct tipc_link **link)
{
+ char peer_str[NODE_ID_STR_LEN] = {0,};
+ char self_str[NODE_ID_STR_LEN] = {0,};
struct tipc_link *l;
l = kzalloc(sizeof(*l), GFP_ATOMIC);
*link = l;
l->session = session;
- /* Note: peer i/f name is completed by reset/activate message */
- sprintf(l->name, "%u.%u.%u:%s-%u.%u.%u:unknown",
- tipc_zone(ownnode), tipc_cluster(ownnode), tipc_node(ownnode),
- if_name, tipc_zone(peer), tipc_cluster(peer), tipc_node(peer));
+ /* Set link name for unicast links only */
+ if (peer_id) {
+ tipc_nodeid2string(self_str, tipc_own_id(net));
+ if (strlen(self_str) > 16)
+ sprintf(self_str, "%x", self);
+ tipc_nodeid2string(peer_str, peer_id);
+ if (strlen(peer_str) > 16)
+ sprintf(peer_str, "%x", peer);
+ }
+ /* Peer i/f name will be completed by reset/activate message */
+ sprintf(l->name, "%s:%s-%s:unknown", self_str, if_name, peer_str);
+
strcpy(l->if_name, if_name);
l->addr = peer;
l->peer_caps = peer_caps;
struct tipc_link *l;
if (!tipc_link_create(net, "", MAX_BEARERS, 0, 'Z', mtu, 0, window,
- 0, ownnode, peer, peer_caps, bc_sndlink,
+ 0, ownnode, peer, NULL, peer_caps, bc_sndlink,
NULL, inputq, namedq, link))
return false;
int __tipc_nl_add_link(struct net *net, struct tipc_nl_msg *msg,
struct tipc_link *link, int nlflags)
{
- int err;
- void *hdr;
+ u32 self = tipc_own_addr(net);
struct nlattr *attrs;
struct nlattr *prop;
- struct tipc_net *tn = net_generic(net, tipc_net_id);
+ void *hdr;
+ int err;
hdr = genlmsg_put(msg->skb, msg->portid, msg->seq, &tipc_genl_family,
nlflags, TIPC_NL_LINK_GET);
if (nla_put_string(msg->skb, TIPC_NLA_LINK_NAME, link->name))
goto attr_msg_full;
- if (nla_put_u32(msg->skb, TIPC_NLA_LINK_DEST,
- tipc_cluster_mask(tn->own_addr)))
+ if (nla_put_u32(msg->skb, TIPC_NLA_LINK_DEST, tipc_cluster_mask(self)))
goto attr_msg_full;
if (nla_put_u32(msg->skb, TIPC_NLA_LINK_MTU, link->mtu))
goto attr_msg_full;
bool tipc_link_create(struct net *net, char *if_name, int bearer_id,
int tolerance, char net_plane, u32 mtu, int priority,
- int window, u32 session, u32 ownnode, u32 peer,
- u16 peer_caps,
+ int window, u32 session, u32 ownnode,
+ u32 peer, u8 *peer_id, u16 peer_caps,
struct tipc_link *bc_sndlink,
struct tipc_link *bc_rcvlink,
struct sk_buff_head *inputq,
*/
#define DSC_REQ_MSG 0
#define DSC_RESP_MSG 1
+#define DSC_TRIAL_MSG 2
+#define DSC_TRIAL_FAIL_MSG 3
/*
* Group protocol message types
msg_set_bits(m, 2, 0, 0xffff, n);
}
-
/*
* Word 4
*/
return (msg_user(hdr) == LINK_PROTOCOL) && (msg_type(hdr) == RESET_MSG);
}
+static inline u32 msg_sugg_node_addr(struct tipc_msg *m)
+{
+ return msg_word(m, 14);
+}
+
+static inline void msg_set_sugg_node_addr(struct tipc_msg *m, u32 n)
+{
+ msg_set_word(m, 14, n);
+}
+
+static inline void msg_set_node_id(struct tipc_msg *hdr, u8 *id)
+{
+ memcpy(msg_data(hdr), id, 16);
+}
+
+static inline u8 *msg_node_id(struct tipc_msg *hdr)
+{
+ return (u8 *)msg_data(hdr);
+}
+
struct sk_buff *tipc_buf_acquire(u32 size, gfp_t gfp);
bool tipc_msg_validate(struct sk_buff **_skb);
bool tipc_msg_reverse(u32 own_addr, struct sk_buff **skb, int err);
static struct sk_buff *named_prepare_buf(struct net *net, u32 type, u32 size,
u32 dest)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
struct sk_buff *buf = tipc_buf_acquire(INT_H_SIZE + size, GFP_ATOMIC);
+ u32 self = tipc_own_addr(net);
struct tipc_msg *msg;
if (buf != NULL) {
msg = buf_msg(buf);
- tipc_msg_init(tn->own_addr, msg, NAME_DISTRIBUTOR, type,
- INT_H_SIZE, dest);
+ tipc_msg_init(self, msg, NAME_DISTRIBUTOR,
+ type, INT_H_SIZE, dest);
msg_set_size(msg, INT_H_SIZE + size);
}
return buf;
{
struct distr_queue_item *e, *tmp;
struct tipc_net *tn = net_generic(net, tipc_net_id);
- char addr[16];
unsigned long now = get_jiffies_64();
list_for_each_entry_safe(e, tmp, &tn->dist_queue, next) {
if (!tipc_update_nametbl(net, &e->i, e->node, e->dtype))
continue;
} else {
- tipc_addr_string_fill(addr, e->node);
- pr_warn_ratelimited("Dropping name table update (%d) of {%u, %u, %u} from %s key=%u\n",
+ pr_warn_ratelimited("Dropping name table update (%d) of {%u, %u, %u} from %x key=%u\n",
e->dtype, ntohl(e->i.type),
ntohl(e->i.lower),
ntohl(e->i.upper),
- addr, ntohl(e->i.key));
+ e->node, ntohl(e->i.key));
}
list_del(&e->next);
kfree(e);
struct name_table *nt = tipc_name_table(net);
struct tipc_net *tn = tipc_net(net);
struct publication *publ;
+ u32 self = tipc_own_addr(net);
spin_lock_bh(&tn->nametbl_lock);
list_for_each_entry_rcu(publ, &nt->node_scope, binding_node)
- publ->node = tn->own_addr;
+ publ->node = self;
list_for_each_entry_rcu(publ, &nt->cluster_scope, binding_node)
- publ->node = tn->own_addr;
+ publ->node = self;
spin_unlock_bh(&tn->nametbl_lock);
}
u32 tipc_nametbl_translate(struct net *net, u32 type, u32 instance,
u32 *destnode)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
+ struct tipc_net *tn = tipc_net(net);
+ bool legacy = tn->legacy_addr_format;
+ u32 self = tipc_own_addr(net);
struct sub_seq *sseq;
struct name_info *info;
struct publication *publ;
u32 port = 0;
u32 node = 0;
- if (!tipc_in_scope(*destnode, tn->own_addr))
+ if (!tipc_in_scope(legacy, *destnode, self))
return 0;
rcu_read_lock();
info = sseq->info;
/* Closest-First Algorithm */
- if (likely(!*destnode)) {
+ if (legacy && !*destnode) {
if (!list_empty(&info->local_publ)) {
publ = list_first_entry(&info->local_publ,
struct publication,
}
/* Round-Robin Algorithm */
- else if (*destnode == tn->own_addr) {
+ else if (*destnode == tipc_own_addr(net)) {
if (list_empty(&info->local_publ))
goto no_match;
publ = list_first_entry(&info->local_publ, struct publication,
}
publ = tipc_nametbl_insert_publ(net, type, lower, upper, scope,
- tn->own_addr, port_ref, key);
+ tipc_own_addr(net), port_ref, key);
if (likely(publ)) {
tn->nametbl->local_publ_count++;
buf = tipc_named_publish(net, publ);
struct tipc_net *tn = net_generic(net, tipc_net_id);
spin_lock_bh(&tn->nametbl_lock);
- publ = tipc_nametbl_remove_publ(net, type, lower, tn->own_addr,
+ publ = tipc_nametbl_remove_publ(net, type, lower, tipc_own_addr(net),
port, key);
if (likely(publ)) {
tn->nametbl->local_publ_count--;
* - A local spin_lock protecting the queue of subscriber events.
*/
-int tipc_net_start(struct net *net, u32 addr)
+int tipc_net_init(struct net *net, u8 *node_id, u32 addr)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
- char addr_string[16];
+ if (tipc_own_id(net)) {
+ pr_info("Cannot configure node identity twice\n");
+ return -1;
+ }
+ pr_info("Started in network mode\n");
- tn->own_addr = addr;
+ if (node_id)
+ tipc_set_node_id(net, node_id);
+ if (addr)
+ tipc_net_finalize(net, addr);
+ return 0;
+}
- /* Ensure that the new address is visible before we reinit. */
+void tipc_net_finalize(struct net *net, u32 addr)
+{
+ tipc_set_node_addr(net, addr);
smp_mb();
-
tipc_named_reinit(net);
tipc_sk_reinit(net);
-
- tipc_nametbl_publish(net, TIPC_CFG_SRV, tn->own_addr, tn->own_addr,
- TIPC_CLUSTER_SCOPE, 0, tn->own_addr);
-
- pr_info("Started in network mode\n");
- pr_info("Own node address %s, network identity %u\n",
- tipc_addr_string_fill(addr_string, tn->own_addr),
- tn->net_id);
- return 0;
+ tipc_nametbl_publish(net, TIPC_CFG_SRV, addr, addr,
+ TIPC_CLUSTER_SCOPE, 0, addr);
}
void tipc_net_stop(struct net *net)
{
- struct tipc_net *tn = net_generic(net, tipc_net_id);
+ u32 self = tipc_own_addr(net);
- if (!tn->own_addr)
+ if (!self)
return;
- tipc_nametbl_withdraw(net, TIPC_CFG_SRV, tn->own_addr, 0,
- tn->own_addr);
+ tipc_nametbl_withdraw(net, TIPC_CFG_SRV, self, 0, self);
rtnl_lock();
tipc_bearer_stop(net);
tipc_node_stop(net);
static int __tipc_nl_add_net(struct net *net, struct tipc_nl_msg *msg)
{
struct tipc_net *tn = net_generic(net, tipc_net_id);
- void *hdr;
+ u64 *w0 = (u64 *)&tn->node_id[0];
+ u64 *w1 = (u64 *)&tn->node_id[8];
struct nlattr *attrs;
+ void *hdr;
hdr = genlmsg_put(msg->skb, msg->portid, msg->seq, &tipc_genl_family,
NLM_F_MULTI, TIPC_NL_NET_GET);
if (nla_put_u32(msg->skb, TIPC_NLA_NET_ID, tn->net_id))
goto attr_msg_full;
-
+ if (nla_put_u64_64bit(msg->skb, TIPC_NLA_NET_NODEID, *w0, 0))
+ goto attr_msg_full;
+ if (nla_put_u64_64bit(msg->skb, TIPC_NLA_NET_NODEID_W1, *w1, 0))
+ goto attr_msg_full;
nla_nest_end(msg->skb, attrs);
genlmsg_end(msg->skb, hdr);
int __tipc_nl_net_set(struct sk_buff *skb, struct genl_info *info)
{
- struct net *net = sock_net(skb->sk);
- struct tipc_net *tn = net_generic(net, tipc_net_id);
struct nlattr *attrs[TIPC_NLA_NET_MAX + 1];
+ struct net *net = sock_net(skb->sk);
+ struct tipc_net *tn = tipc_net(net);
int err;
if (!info->attrs[TIPC_NLA_NET])
err = nla_parse_nested(attrs, TIPC_NLA_NET_MAX,
info->attrs[TIPC_NLA_NET], tipc_nl_net_policy,
info->extack);
+
if (err)
return err;
+ /* Can't change net id once TIPC has joined a network */
+ if (tipc_own_addr(net))
+ return -EPERM;
+
if (attrs[TIPC_NLA_NET_ID]) {
u32 val;
- /* Can't change net id once TIPC has joined a network */
- if (tn->own_addr)
- return -EPERM;
-
val = nla_get_u32(attrs[TIPC_NLA_NET_ID]);
if (val < 1 || val > 9999)
return -EINVAL;
if (attrs[TIPC_NLA_NET_ADDR]) {
u32 addr;
- /* Can't change net addr once TIPC has joined a network */
- if (tn->own_addr)
- return -EPERM;
-
addr = nla_get_u32(attrs[TIPC_NLA_NET_ADDR]);
- if (!tipc_addr_node_valid(addr))
+ if (!addr)
return -EINVAL;
-
- tipc_net_start(net, addr);
+ tn->legacy_addr_format = true;
+ tipc_net_init(net, NULL, addr);
}
+ if (attrs[TIPC_NLA_NET_NODEID]) {
+ u8 node_id[NODE_ID_LEN];
+ u64 *w0 = (u64 *)&node_id[0];
+ u64 *w1 = (u64 *)&node_id[8];
+
+ *w0 = nla_get_u64(attrs[TIPC_NLA_NET_NODEID]);
+ *w1 = nla_get_u64(attrs[TIPC_NLA_NET_NODEID_W1]);
+ tipc_net_init(net, node_id, 0);
+ }
return 0;
}
extern const struct nla_policy tipc_nl_net_policy[];
-int tipc_net_start(struct net *net, u32 addr);
-
+int tipc_net_init(struct net *net, u8 *node_id, u32 addr);
+void tipc_net_finalize(struct net *net, u32 addr);
void tipc_net_stop(struct net *net);
-
int tipc_nl_net_dump(struct sk_buff *skb, struct netlink_callback *cb);
int tipc_nl_net_set(struct sk_buff *skb, struct genl_info *info);
int __tipc_nl_net_set(struct sk_buff *skb, struct genl_info *info);
u16 capabilities;
u32 signature;
u32 link_id;
+ u8 peer_id[16];
struct list_head publ_list;
struct list_head conn_sks;
unsigned long keepalive_intv;
static void tipc_node_timeout(struct timer_list *t);
static void tipc_node_fsm_evt(struct tipc_node *n, int evt);
static struct tipc_node *tipc_node_find(struct net *net, u32 addr);
+static struct tipc_node *tipc_node_find_by_id(struct net *net, u8 *id);
static void tipc_node_put(struct tipc_node *node);
static bool node_is_up(struct tipc_node *n);
struct tipc_node *node;
unsigned int thash = tipc_hashfn(addr);
- if (unlikely(!in_own_cluster_exact(net, addr)))
- return NULL;
-
rcu_read_lock();
hlist_for_each_entry_rcu(node, &tn->node_htable[thash], hash) {
if (node->addr != addr)
return node;
}
+/* tipc_node_find_by_id - locate specified node object by its 128-bit id
+ * Note: this function is called only when a discovery request failed
+ * to find the node by its 32-bit id, and is not time critical
+ */
+static struct tipc_node *tipc_node_find_by_id(struct net *net, u8 *id)
+{
+ struct tipc_net *tn = tipc_net(net);
+ struct tipc_node *n;
+ bool found = false;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(n, &tn->node_list, list) {
+ read_lock_bh(&n->lock);
+ if (!memcmp(id, n->peer_id, 16) &&
+ kref_get_unless_zero(&n->kref))
+ found = true;
+ read_unlock_bh(&n->lock);
+ if (found)
+ break;
+ }
+ rcu_read_unlock();
+ return found ? n : NULL;
+}
+
static void tipc_node_read_lock(struct tipc_node *n)
{
read_lock_bh(&n->lock);
}
}
-struct tipc_node *tipc_node_create(struct net *net, u32 addr, u16 capabilities)
+static struct tipc_node *tipc_node_create(struct net *net, u32 addr,
+ u8 *peer_id, u16 capabilities)
{
struct tipc_net *tn = net_generic(net, tipc_net_id);
struct tipc_node *n, *temp_node;
goto exit;
}
n->addr = addr;
+ memcpy(&n->peer_id, peer_id, 16);
n->net = net;
n->capabilities = capabilities;
kref_init(&n->kref);
n->signature = INVALID_NODE_SIG;
n->active_links[0] = INVALID_BEARER_ID;
n->active_links[1] = INVALID_BEARER_ID;
- if (!tipc_link_bc_create(net, tipc_own_addr(net), n->addr,
- U16_MAX,
+ if (!tipc_link_bc_create(net, tipc_own_addr(net),
+ addr, U16_MAX,
tipc_link_window(tipc_bc_sndlink(net)),
n->capabilities,
&n->bc_entry.inputq1,
return retval;
}
-void tipc_node_check_dest(struct net *net, u32 onode,
- struct tipc_bearer *b,
+static u32 tipc_node_suggest_addr(struct net *net, u32 addr)
+{
+ struct tipc_node *n;
+
+ addr ^= tipc_net(net)->random;
+ while ((n = tipc_node_find(net, addr))) {
+ tipc_node_put(n);
+ addr++;
+ }
+ return addr;
+}
+
+/* tipc_node_try_addr(): Check if addr can be used by peer, suggest other if not
+ */
+u32 tipc_node_try_addr(struct net *net, u8 *id, u32 addr)
+{
+ struct tipc_net *tn = tipc_net(net);
+ struct tipc_node *n;
+
+ /* Suggest new address if some other peer is using this one */
+ n = tipc_node_find(net, addr);
+ if (n) {
+ if (!memcmp(n->peer_id, id, NODE_ID_LEN))
+ addr = 0;
+ tipc_node_put(n);
+ if (!addr)
+ return 0;
+ return tipc_node_suggest_addr(net, addr);
+ }
+
+ /* Suggest previously used address if peer is known */
+ n = tipc_node_find_by_id(net, id);
+ if (n) {
+ addr = n->addr;
+ tipc_node_put(n);
+ }
+ /* Even this node may be in trial phase */
+ if (tn->trial_addr == addr)
+ return tipc_node_suggest_addr(net, addr);
+
+ return addr;
+}
+
+void tipc_node_check_dest(struct net *net, u32 addr,
+ u8 *peer_id, struct tipc_bearer *b,
u16 capabilities, u32 signature,
struct tipc_media_addr *maddr,
bool *respond, bool *dupl_addr)
*dupl_addr = false;
*respond = false;
- n = tipc_node_create(net, onode, capabilities);
+ n = tipc_node_create(net, addr, peer_id, capabilities);
if (!n)
return;
/* Now create new link if not already existing */
if (!l) {
- if (n->link_cnt == 2) {
- pr_warn("Cannot establish 3rd link to %x\n", n->addr);
+ if (n->link_cnt == 2)
goto exit;
- }
+
if_name = strchr(b->name, ':') + 1;
if (!tipc_link_create(net, if_name, b->identity, b->tolerance,
b->net_plane, b->mtu, b->priority,
b->window, mod(tipc_net(net)->random),
- tipc_own_addr(net), onode,
+ tipc_own_addr(net), addr, peer_id,
n->capabilities,
tipc_bc_sndlink(n->net), n->bc_entry.link,
&le->inputq,
static void tipc_node_reset_links(struct tipc_node *n)
{
- char addr_string[16];
int i;
- pr_warn("Resetting all links to %s\n",
- tipc_addr_string_fill(addr_string, n->addr));
+ pr_warn("Resetting all links to %x\n", n->addr);
for (i = 0; i < MAX_BEARERS; i++) {
tipc_node_link_down(n, i, false);
static void node_lost_contact(struct tipc_node *n,
struct sk_buff_head *inputq)
{
- char addr_string[16];
struct tipc_sock_conn *conn, *safe;
struct tipc_link *l;
struct list_head *conns = &n->conn_sks;
struct sk_buff *skb;
uint i;
- pr_debug("Lost contact with %s\n",
- tipc_addr_string_fill(addr_string, n->addr));
+ pr_debug("Lost contact with %x\n", n->addr);
/* Clean up broadcast state */
tipc_bcast_remove_peer(n->net, n->bc_entry.link);
TIPC_BCAST_STATE_NACK = (1 << 2),
TIPC_BLOCK_FLOWCTL = (1 << 3),
TIPC_BCAST_RCAST = (1 << 4),
- TIPC_MCAST_GROUPS = (1 << 5)
+ TIPC_NODE_ID128 = (1 << 5)
};
#define TIPC_NODE_CAPABILITIES (TIPC_BCAST_SYNCH | \
TIPC_BCAST_STATE_NACK | \
TIPC_BCAST_RCAST | \
- TIPC_BLOCK_FLOWCTL)
+ TIPC_BLOCK_FLOWCTL | \
+ TIPC_NODE_ID128)
#define INVALID_BEARER_ID -1
void tipc_node_stop(struct net *net);
-void tipc_node_check_dest(struct net *net, u32 onode,
+u32 tipc_node_try_addr(struct net *net, u8 *id, u32 addr);
+void tipc_node_check_dest(struct net *net, u32 onode, u8 *peer_id128,
struct tipc_bearer *bearer,
u16 capabilities, u32 signature,
struct tipc_media_addr *maddr,
static bool tsk_peer_msg(struct tipc_sock *tsk, struct tipc_msg *msg)
{
struct sock *sk = &tsk->sk;
- struct tipc_net *tn = net_generic(sock_net(sk), tipc_net_id);
+ u32 self = tipc_own_addr(sock_net(sk));
u32 peer_port = tsk_peer_port(tsk);
- u32 orig_node;
- u32 peer_node;
+ u32 orig_node, peer_node;
if (unlikely(!tipc_sk_connected(sk)))
return false;
if (likely(orig_node == peer_node))
return true;
- if (!orig_node && (peer_node == tn->own_addr))
+ if (!orig_node && peer_node == self)
return true;
- if (!peer_node && (orig_node == tn->own_addr))
+ if (!peer_node && orig_node == self)
return true;
return false;
/* Ensure tsk is visible before we read own_addr. */
smp_mb();
- tipc_msg_init(tn->own_addr, msg, TIPC_LOW_IMPORTANCE, TIPC_NAMED_MSG,
- NAMED_H_SIZE, 0);
+ tipc_msg_init(tipc_own_addr(net), msg, TIPC_LOW_IMPORTANCE,
+ TIPC_NAMED_MSG, NAMED_H_SIZE, 0);
msg_set_origport(msg, tsk->portid);
timer_setup(&sk->sk_timer, tipc_sk_timeout, 0);
struct sockaddr_tipc *addr = (struct sockaddr_tipc *)uaddr;
struct sock *sk = sock->sk;
struct tipc_sock *tsk = tipc_sk(sk);
- struct tipc_net *tn = net_generic(sock_net(sock->sk), tipc_net_id);
memset(addr, 0, sizeof(*addr));
if (peer) {
addr->addr.id.node = tsk_peer_node(tsk);
} else {
addr->addr.id.ref = tsk->portid;
- addr->addr.id.node = tn->own_addr;
+ addr->addr.id.node = tipc_own_addr(sock_net(sk));
}
addr->addrtype = TIPC_ADDR_ID;
(!sk_conn && msg_connected(hdr)) ||
(!grp && msg_in_group(hdr)))
err = TIPC_ERR_NO_PORT;
- else if (sk_rmem_alloc_get(sk) + skb->truesize >= limit)
+ else if (sk_rmem_alloc_get(sk) + skb->truesize >= limit) {
+ atomic_inc(&sk->sk_drops);
err = TIPC_ERR_OVERLOAD;
+ }
if (unlikely(err)) {
tipc_skb_reject(net, err, skb, xmitq);
/* Overload => reject message back to sender */
onode = tipc_own_addr(sock_net(sk));
+ atomic_inc(&sk->sk_drops);
if (tipc_msg_reverse(onode, &skb, TIPC_ERR_OVERLOAD))
__skb_queue_tail(xmitq, skb);
break;
while ((tsk = rhashtable_walk_next(&iter)) && !IS_ERR(tsk)) {
spin_lock_bh(&tsk->sk.sk_lock.slock);
msg = &tsk->phdr;
- msg_set_prevnode(msg, tn->own_addr);
- msg_set_orignode(msg, tn->own_addr);
+ msg_set_prevnode(msg, tipc_own_addr(net));
+ msg_set_orignode(msg, tipc_own_addr(net));
spin_unlock_bh(&tsk->sk.sk_lock.slock);
}
return -EMSGSIZE;
}
+static int __tipc_nl_add_sk_info(struct sk_buff *skb, struct tipc_sock
+ *tsk)
+{
+ struct net *net = sock_net(skb->sk);
+ struct sock *sk = &tsk->sk;
+
+ if (nla_put_u32(skb, TIPC_NLA_SOCK_REF, tsk->portid) ||
+ nla_put_u32(skb, TIPC_NLA_SOCK_ADDR, tipc_own_addr(net)))
+ return -EMSGSIZE;
+
+ if (tipc_sk_connected(sk)) {
+ if (__tipc_nl_add_sk_con(skb, tsk))
+ return -EMSGSIZE;
+ } else if (!list_empty(&tsk->publications)) {
+ if (nla_put_flag(skb, TIPC_NLA_SOCK_HAS_PUBL))
+ return -EMSGSIZE;
+ }
+ return 0;
+}
+
/* Caller should hold socket lock for the passed tipc socket. */
static int __tipc_nl_add_sk(struct sk_buff *skb, struct netlink_callback *cb,
struct tipc_sock *tsk)
{
- int err;
- void *hdr;
struct nlattr *attrs;
- struct net *net = sock_net(skb->sk);
- struct tipc_net *tn = net_generic(net, tipc_net_id);
- struct sock *sk = &tsk->sk;
+ void *hdr;
hdr = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq,
&tipc_genl_family, NLM_F_MULTI, TIPC_NL_SOCK_GET);
attrs = nla_nest_start(skb, TIPC_NLA_SOCK);
if (!attrs)
goto genlmsg_cancel;
- if (nla_put_u32(skb, TIPC_NLA_SOCK_REF, tsk->portid))
- goto attr_msg_cancel;
- if (nla_put_u32(skb, TIPC_NLA_SOCK_ADDR, tn->own_addr))
+
+ if (__tipc_nl_add_sk_info(skb, tsk))
goto attr_msg_cancel;
- if (tipc_sk_connected(sk)) {
- err = __tipc_nl_add_sk_con(skb, tsk);
- if (err)
- goto attr_msg_cancel;
- } else if (!list_empty(&tsk->publications)) {
- if (nla_put_flag(skb, TIPC_NLA_SOCK_HAS_PUBL))
- goto attr_msg_cancel;
- }
nla_nest_end(skb, attrs);
genlmsg_end(skb, hdr);
return -EMSGSIZE;
}
-int tipc_nl_sk_dump(struct sk_buff *skb, struct netlink_callback *cb)
+int tipc_nl_sk_walk(struct sk_buff *skb, struct netlink_callback *cb,
+ int (*skb_handler)(struct sk_buff *skb,
+ struct netlink_callback *cb,
+ struct tipc_sock *tsk))
{
- int err;
- struct tipc_sock *tsk;
- const struct bucket_table *tbl;
- struct rhash_head *pos;
struct net *net = sock_net(skb->sk);
- struct tipc_net *tn = net_generic(net, tipc_net_id);
- u32 tbl_id = cb->args[0];
+ struct tipc_net *tn = tipc_net(net);
+ const struct bucket_table *tbl;
u32 prev_portid = cb->args[1];
+ u32 tbl_id = cb->args[0];
+ struct rhash_head *pos;
+ struct tipc_sock *tsk;
+ int err;
rcu_read_lock();
tbl = rht_dereference_rcu((&tn->sk_rht)->tbl, &tn->sk_rht);
continue;
}
- err = __tipc_nl_add_sk(skb, cb, tsk);
+ err = skb_handler(skb, cb, tsk);
if (err) {
prev_portid = tsk->portid;
spin_unlock_bh(&tsk->sk.sk_lock.slock);
goto out;
}
+
prev_portid = 0;
spin_unlock_bh(&tsk->sk.sk_lock.slock);
}
return skb->len;
}
+EXPORT_SYMBOL(tipc_nl_sk_walk);
+
+int tipc_sk_fill_sock_diag(struct sk_buff *skb, struct tipc_sock *tsk,
+ u32 sk_filter_state,
+ u64 (*tipc_diag_gen_cookie)(struct sock *sk))
+{
+ struct sock *sk = &tsk->sk;
+ struct nlattr *attrs;
+ struct nlattr *stat;
+
+ /*filter response w.r.t sk_state*/
+ if (!(sk_filter_state & (1 << sk->sk_state)))
+ return 0;
+
+ attrs = nla_nest_start(skb, TIPC_NLA_SOCK);
+ if (!attrs)
+ goto msg_cancel;
+
+ if (__tipc_nl_add_sk_info(skb, tsk))
+ goto attr_msg_cancel;
+
+ if (nla_put_u32(skb, TIPC_NLA_SOCK_TYPE, (u32)sk->sk_type) ||
+ nla_put_u32(skb, TIPC_NLA_SOCK_TIPC_STATE, (u32)sk->sk_state) ||
+ nla_put_u32(skb, TIPC_NLA_SOCK_INO, sock_i_ino(sk)) ||
+ nla_put_u32(skb, TIPC_NLA_SOCK_UID,
+ from_kuid_munged(sk_user_ns(sk), sock_i_uid(sk))) ||
+ nla_put_u64_64bit(skb, TIPC_NLA_SOCK_COOKIE,
+ tipc_diag_gen_cookie(sk),
+ TIPC_NLA_SOCK_PAD))
+ goto attr_msg_cancel;
+
+ stat = nla_nest_start(skb, TIPC_NLA_SOCK_STAT);
+ if (!stat)
+ goto attr_msg_cancel;
+
+ if (nla_put_u32(skb, TIPC_NLA_SOCK_STAT_RCVQ,
+ skb_queue_len(&sk->sk_receive_queue)) ||
+ nla_put_u32(skb, TIPC_NLA_SOCK_STAT_SENDQ,
+ skb_queue_len(&sk->sk_write_queue)) ||
+ nla_put_u32(skb, TIPC_NLA_SOCK_STAT_DROP,
+ atomic_read(&sk->sk_drops)))
+ goto stat_msg_cancel;
+
+ if (tsk->cong_link_cnt &&
+ nla_put_flag(skb, TIPC_NLA_SOCK_STAT_LINK_CONG))
+ goto stat_msg_cancel;
+
+ if (tsk_conn_cong(tsk) &&
+ nla_put_flag(skb, TIPC_NLA_SOCK_STAT_CONN_CONG))
+ goto stat_msg_cancel;
+
+ nla_nest_end(skb, stat);
+ nla_nest_end(skb, attrs);
+
+ return 0;
+
+stat_msg_cancel:
+ nla_nest_cancel(skb, stat);
+attr_msg_cancel:
+ nla_nest_cancel(skb, attrs);
+msg_cancel:
+ return -EMSGSIZE;
+}
+EXPORT_SYMBOL(tipc_sk_fill_sock_diag);
+
+int tipc_nl_sk_dump(struct sk_buff *skb, struct netlink_callback *cb)
+{
+ return tipc_nl_sk_walk(skb, cb, __tipc_nl_add_sk);
+}
/* Caller should hold socket lock for the passed tipc socket. */
static int __tipc_nl_add_sk_publ(struct sk_buff *skb,
#define RCVBUF_DEF (FLOWCTL_BLK_SZ * 1024 * 2)
#define RCVBUF_MAX (FLOWCTL_BLK_SZ * 1024 * 16)
+struct tipc_sock;
+
int tipc_socket_init(void);
void tipc_socket_stop(void);
void tipc_sk_rcv(struct net *net, struct sk_buff_head *inputq);
void tipc_sk_rht_destroy(struct net *net);
int tipc_nl_sk_dump(struct sk_buff *skb, struct netlink_callback *cb);
int tipc_nl_publ_dump(struct sk_buff *skb, struct netlink_callback *cb);
-
+int tipc_sk_fill_sock_diag(struct sk_buff *skb, struct tipc_sock *tsk,
+ u32 sk_filter_state,
+ u64 (*tipc_diag_gen_cookie)(struct sock *sk));
+int tipc_nl_sk_walk(struct sk_buff *skb, struct netlink_callback *cb,
+ int (*skb_handler)(struct sk_buff *skb,
+ struct netlink_callback *cb,
+ struct tipc_sock *tsk));
#endif
#include <net/addrconf.h>
#include <linux/tipc_netlink.h>
#include "core.h"
+#include "addr.h"
+#include "net.h"
#include "bearer.h"
#include "netlink.h"
#include "msg.h"
struct udp_port_cfg udp_conf = {0};
struct udp_tunnel_sock_cfg tuncfg = {NULL};
struct nlattr *opts[TIPC_NLA_UDP_MAX + 1];
+ u8 node_id[NODE_ID_LEN] = {0,};
ub = kzalloc(sizeof(*ub), GFP_ATOMIC);
if (!ub)
if (err)
goto err;
+ /* Autoconfigure own node identity if needed */
+ if (!tipc_own_id(net)) {
+ memcpy(node_id, local.ipv6.in6_u.u6_addr8, 16);
+ tipc_net_init(net, node_id, 0);
+ }
+ if (!tipc_own_id(net)) {
+ pr_warn("Failed to set node id, please configure manually\n");
+ err = -EINVAL;
+ goto err;
+ }
+
b->bcast_addr.media_id = TIPC_MEDIA_TYPE_UDP;
b->bcast_addr.broadcast = TIPC_BROADCAST_SUPPORT;
rcu_assign_pointer(b->media_ptr, ub);
select CRYPTO
select CRYPTO_AES
select CRYPTO_GCM
+ select STREAM_PARSER
default n
---help---
Enable kernel support for TLS protocol. This allows symmetric
};
enum {
- TLS_BASE_TX,
+ TLS_BASE,
TLS_SW_TX,
+ TLS_SW_RX,
+ TLS_SW_RXTX,
TLS_NUM_CONFIG,
};
static struct proto *saved_tcpv6_prot;
static DEFINE_MUTEX(tcpv6_prot_mutex);
static struct proto tls_prots[TLS_NUM_PROTS][TLS_NUM_CONFIG];
+static struct proto_ops tls_sw_proto_ops;
static inline void update_sk_prot(struct sock *sk, struct tls_context *ctx)
{
int ip_ver = sk->sk_family == AF_INET6 ? TLSV6 : TLSV4;
- sk->sk_prot = &tls_prots[ip_ver][ctx->tx_conf];
+ sk->sk_prot = &tls_prots[ip_ver][ctx->conf];
}
int wait_on_pending_writer(struct sock *sk, long *timeo)
lock_sock(sk);
sk_proto_close = ctx->sk_proto_close;
- if (ctx->tx_conf == TLS_BASE_TX) {
+ if (ctx->conf == TLS_BASE) {
kfree(ctx);
goto skip_tx_cleanup;
}
}
}
- kfree(ctx->rec_seq);
- kfree(ctx->iv);
+ kfree(ctx->tx.rec_seq);
+ kfree(ctx->tx.iv);
+ kfree(ctx->rx.rec_seq);
+ kfree(ctx->rx.iv);
- if (ctx->tx_conf == TLS_SW_TX)
- tls_sw_free_tx_resources(sk);
+ if (ctx->conf == TLS_SW_TX ||
+ ctx->conf == TLS_SW_RX ||
+ ctx->conf == TLS_SW_RXTX) {
+ tls_sw_free_resources(sk);
+ }
skip_tx_cleanup:
release_sock(sk);
}
lock_sock(sk);
memcpy(crypto_info_aes_gcm_128->iv,
- ctx->iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE,
+ ctx->tx.iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE,
TLS_CIPHER_AES_GCM_128_IV_SIZE);
- memcpy(crypto_info_aes_gcm_128->rec_seq, ctx->rec_seq,
+ memcpy(crypto_info_aes_gcm_128->rec_seq, ctx->tx.rec_seq,
TLS_CIPHER_AES_GCM_128_REC_SEQ_SIZE);
release_sock(sk);
if (copy_to_user(optval,
return do_tls_getsockopt(sk, optname, optval, optlen);
}
-static int do_tls_setsockopt_tx(struct sock *sk, char __user *optval,
- unsigned int optlen)
+static int do_tls_setsockopt_conf(struct sock *sk, char __user *optval,
+ unsigned int optlen, int tx)
{
struct tls_crypto_info *crypto_info;
struct tls_context *ctx = tls_get_ctx(sk);
int rc = 0;
- int tx_conf;
+ int conf;
if (!optval || (optlen < sizeof(*crypto_info))) {
rc = -EINVAL;
goto out;
}
- crypto_info = &ctx->crypto_send;
+ if (tx)
+ crypto_info = &ctx->crypto_send;
+ else
+ crypto_info = &ctx->crypto_recv;
+
/* Currently we don't support set crypto info more than one time */
if (TLS_CRYPTO_INFO_READY(crypto_info)) {
rc = -EBUSY;
}
/* currently SW is default, we will have ethtool in future */
- rc = tls_set_sw_offload(sk, ctx);
- tx_conf = TLS_SW_TX;
+ if (tx) {
+ rc = tls_set_sw_offload(sk, ctx, 1);
+ if (ctx->conf == TLS_SW_RX)
+ conf = TLS_SW_RXTX;
+ else
+ conf = TLS_SW_TX;
+ } else {
+ rc = tls_set_sw_offload(sk, ctx, 0);
+ if (ctx->conf == TLS_SW_TX)
+ conf = TLS_SW_RXTX;
+ else
+ conf = TLS_SW_RX;
+ }
+
if (rc)
goto err_crypto_info;
- ctx->tx_conf = tx_conf;
+ ctx->conf = conf;
update_sk_prot(sk, ctx);
- ctx->sk_write_space = sk->sk_write_space;
- sk->sk_write_space = tls_write_space;
+ if (tx) {
+ ctx->sk_write_space = sk->sk_write_space;
+ sk->sk_write_space = tls_write_space;
+ } else {
+ sk->sk_socket->ops = &tls_sw_proto_ops;
+ }
goto out;
err_crypto_info:
switch (optname) {
case TLS_TX:
+ case TLS_RX:
lock_sock(sk);
- rc = do_tls_setsockopt_tx(sk, optval, optlen);
+ rc = do_tls_setsockopt_conf(sk, optval, optlen,
+ optname == TLS_TX);
release_sock(sk);
break;
default:
static void build_protos(struct proto *prot, struct proto *base)
{
- prot[TLS_BASE_TX] = *base;
- prot[TLS_BASE_TX].setsockopt = tls_setsockopt;
- prot[TLS_BASE_TX].getsockopt = tls_getsockopt;
- prot[TLS_BASE_TX].close = tls_sk_proto_close;
+ prot[TLS_BASE] = *base;
+ prot[TLS_BASE].setsockopt = tls_setsockopt;
+ prot[TLS_BASE].getsockopt = tls_getsockopt;
+ prot[TLS_BASE].close = tls_sk_proto_close;
- prot[TLS_SW_TX] = prot[TLS_BASE_TX];
+ prot[TLS_SW_TX] = prot[TLS_BASE];
prot[TLS_SW_TX].sendmsg = tls_sw_sendmsg;
prot[TLS_SW_TX].sendpage = tls_sw_sendpage;
+
+ prot[TLS_SW_RX] = prot[TLS_BASE];
+ prot[TLS_SW_RX].recvmsg = tls_sw_recvmsg;
+ prot[TLS_SW_RX].close = tls_sk_proto_close;
+
+ prot[TLS_SW_RXTX] = prot[TLS_SW_TX];
+ prot[TLS_SW_RXTX].recvmsg = tls_sw_recvmsg;
+ prot[TLS_SW_RXTX].close = tls_sk_proto_close;
}
static int tls_init(struct sock *sk)
mutex_unlock(&tcpv6_prot_mutex);
}
- ctx->tx_conf = TLS_BASE_TX;
+ ctx->conf = TLS_BASE;
update_sk_prot(sk, ctx);
out:
return rc;
{
build_protos(tls_prots[TLSV4], &tcp_prot);
+ tls_sw_proto_ops = inet_stream_ops;
+ tls_sw_proto_ops.poll = tls_sw_poll;
+ tls_sw_proto_ops.splice_read = tls_sw_splice_read;
+
tcp_register_ulp(&tcp_tls_ulp_ops);
return 0;
* SOFTWARE.
*/
+#include <linux/sched/signal.h>
#include <linux/module.h>
#include <crypto/aead.h>
+#include <net/strparser.h>
#include <net/tls.h>
+static int tls_do_decryption(struct sock *sk,
+ struct scatterlist *sgin,
+ struct scatterlist *sgout,
+ char *iv_recv,
+ size_t data_len,
+ struct sk_buff *skb,
+ gfp_t flags)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ struct strp_msg *rxm = strp_msg(skb);
+ struct aead_request *aead_req;
+
+ int ret;
+ unsigned int req_size = sizeof(struct aead_request) +
+ crypto_aead_reqsize(ctx->aead_recv);
+
+ aead_req = kzalloc(req_size, flags);
+ if (!aead_req)
+ return -ENOMEM;
+
+ aead_request_set_tfm(aead_req, ctx->aead_recv);
+ aead_request_set_ad(aead_req, TLS_AAD_SPACE_SIZE);
+ aead_request_set_crypt(aead_req, sgin, sgout,
+ data_len + tls_ctx->rx.tag_size,
+ (u8 *)iv_recv);
+ aead_request_set_callback(aead_req, CRYPTO_TFM_REQ_MAY_BACKLOG,
+ crypto_req_done, &ctx->async_wait);
+
+ ret = crypto_wait_req(crypto_aead_decrypt(aead_req), &ctx->async_wait);
+
+ if (ret < 0)
+ goto out;
+
+ rxm->offset += tls_ctx->rx.prepend_size;
+ rxm->full_len -= tls_ctx->rx.overhead_size;
+ tls_advance_record_sn(sk, &tls_ctx->rx);
+
+ ctx->decrypted = true;
+
+ ctx->saved_data_ready(sk);
+
+out:
+ kfree(aead_req);
+ return ret;
+}
+
static void trim_sg(struct sock *sk, struct scatterlist *sg,
int *sg_num_elem, unsigned int *sg_size, int target_size)
{
target_size);
if (target_size > 0)
- target_size += tls_ctx->overhead_size;
+ target_size += tls_ctx->tx.overhead_size;
trim_sg(sk, ctx->sg_encrypted_data,
&ctx->sg_encrypted_num_elem,
target_size);
}
-static int alloc_sg(struct sock *sk, int len, struct scatterlist *sg,
- int *sg_num_elem, unsigned int *sg_size,
- int first_coalesce)
-{
- struct page_frag *pfrag;
- unsigned int size = *sg_size;
- int num_elem = *sg_num_elem, use = 0, rc = 0;
- struct scatterlist *sge;
- unsigned int orig_offset;
-
- len -= size;
- pfrag = sk_page_frag(sk);
-
- while (len > 0) {
- if (!sk_page_frag_refill(sk, pfrag)) {
- rc = -ENOMEM;
- goto out;
- }
-
- use = min_t(int, len, pfrag->size - pfrag->offset);
-
- if (!sk_wmem_schedule(sk, use)) {
- rc = -ENOMEM;
- goto out;
- }
-
- sk_mem_charge(sk, use);
- size += use;
- orig_offset = pfrag->offset;
- pfrag->offset += use;
-
- sge = sg + num_elem - 1;
- if (num_elem > first_coalesce && sg_page(sg) == pfrag->page &&
- sg->offset + sg->length == orig_offset) {
- sg->length += use;
- } else {
- sge++;
- sg_unmark_end(sge);
- sg_set_page(sge, pfrag->page, use, orig_offset);
- get_page(pfrag->page);
- ++num_elem;
- if (num_elem == MAX_SKB_FRAGS) {
- rc = -ENOSPC;
- break;
- }
- }
-
- len -= use;
- }
- goto out;
-
-out:
- *sg_size = size;
- *sg_num_elem = num_elem;
- return rc;
-}
-
static int alloc_encrypted_sg(struct sock *sk, int len)
{
struct tls_context *tls_ctx = tls_get_ctx(sk);
struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
int rc = 0;
- rc = alloc_sg(sk, len, ctx->sg_encrypted_data,
- &ctx->sg_encrypted_num_elem, &ctx->sg_encrypted_size, 0);
+ rc = sk_alloc_sg(sk, len,
+ ctx->sg_encrypted_data, 0,
+ &ctx->sg_encrypted_num_elem,
+ &ctx->sg_encrypted_size, 0);
return rc;
}
struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
int rc = 0;
- rc = alloc_sg(sk, len, ctx->sg_plaintext_data,
- &ctx->sg_plaintext_num_elem, &ctx->sg_plaintext_size,
- tls_ctx->pending_open_record_frags);
+ rc = sk_alloc_sg(sk, len, ctx->sg_plaintext_data, 0,
+ &ctx->sg_plaintext_num_elem, &ctx->sg_plaintext_size,
+ tls_ctx->pending_open_record_frags);
return rc;
}
if (!aead_req)
return -ENOMEM;
- ctx->sg_encrypted_data[0].offset += tls_ctx->prepend_size;
- ctx->sg_encrypted_data[0].length -= tls_ctx->prepend_size;
+ ctx->sg_encrypted_data[0].offset += tls_ctx->tx.prepend_size;
+ ctx->sg_encrypted_data[0].length -= tls_ctx->tx.prepend_size;
aead_request_set_tfm(aead_req, ctx->aead_send);
aead_request_set_ad(aead_req, TLS_AAD_SPACE_SIZE);
aead_request_set_crypt(aead_req, ctx->sg_aead_in, ctx->sg_aead_out,
- data_len, tls_ctx->iv);
+ data_len, tls_ctx->tx.iv);
aead_request_set_callback(aead_req, CRYPTO_TFM_REQ_MAY_BACKLOG,
crypto_req_done, &ctx->async_wait);
rc = crypto_wait_req(crypto_aead_encrypt(aead_req), &ctx->async_wait);
- ctx->sg_encrypted_data[0].offset -= tls_ctx->prepend_size;
- ctx->sg_encrypted_data[0].length += tls_ctx->prepend_size;
+ ctx->sg_encrypted_data[0].offset -= tls_ctx->tx.prepend_size;
+ ctx->sg_encrypted_data[0].length += tls_ctx->tx.prepend_size;
kfree(aead_req);
return rc;
sg_mark_end(ctx->sg_encrypted_data + ctx->sg_encrypted_num_elem - 1);
tls_make_aad(ctx->aad_space, ctx->sg_plaintext_size,
- tls_ctx->rec_seq, tls_ctx->rec_seq_size,
+ tls_ctx->tx.rec_seq, tls_ctx->tx.rec_seq_size,
record_type);
tls_fill_prepend(tls_ctx,
/* Only pass through MSG_DONTWAIT and MSG_NOSIGNAL flags */
rc = tls_push_sg(sk, tls_ctx, ctx->sg_encrypted_data, 0, flags);
if (rc < 0 && rc != -EAGAIN)
- tls_err_abort(sk);
+ tls_err_abort(sk, EBADMSG);
- tls_advance_record_sn(sk, tls_ctx);
+ tls_advance_record_sn(sk, &tls_ctx->tx);
return rc;
}
}
static int zerocopy_from_iter(struct sock *sk, struct iov_iter *from,
- int length)
+ int length, int *pages_used,
+ unsigned int *size_used,
+ struct scatterlist *to, int to_max_pages,
+ bool charge)
{
- struct tls_context *tls_ctx = tls_get_ctx(sk);
- struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
struct page *pages[MAX_SKB_FRAGS];
size_t offset;
ssize_t copied, use;
int i = 0;
- unsigned int size = ctx->sg_plaintext_size;
- int num_elem = ctx->sg_plaintext_num_elem;
+ unsigned int size = *size_used;
+ int num_elem = *pages_used;
int rc = 0;
int maxpages;
while (length > 0) {
i = 0;
- maxpages = ARRAY_SIZE(ctx->sg_plaintext_data) - num_elem;
+ maxpages = to_max_pages - num_elem;
if (maxpages == 0) {
rc = -EFAULT;
goto out;
while (copied) {
use = min_t(int, copied, PAGE_SIZE - offset);
- sg_set_page(&ctx->sg_plaintext_data[num_elem],
+ sg_set_page(&to[num_elem],
pages[i], use, offset);
- sg_unmark_end(&ctx->sg_plaintext_data[num_elem]);
- sk_mem_charge(sk, use);
+ sg_unmark_end(&to[num_elem]);
+ if (charge)
+ sk_mem_charge(sk, use);
offset = 0;
copied -= use;
}
out:
- ctx->sg_plaintext_size = size;
- ctx->sg_plaintext_num_elem = num_elem;
+ *size_used = size;
+ *pages_used = num_elem;
+
return rc;
}
}
required_size = ctx->sg_plaintext_size + try_to_copy +
- tls_ctx->overhead_size;
+ tls_ctx->tx.overhead_size;
if (!sk_stream_memory_free(sk))
goto wait_for_sndbuf;
if (full_record || eor) {
ret = zerocopy_from_iter(sk, &msg->msg_iter,
- try_to_copy);
+ try_to_copy, &ctx->sg_plaintext_num_elem,
+ &ctx->sg_plaintext_size,
+ ctx->sg_plaintext_data,
+ ARRAY_SIZE(ctx->sg_plaintext_data),
+ true);
if (ret)
goto fallback_to_reg_send;
&ctx->sg_encrypted_num_elem,
&ctx->sg_encrypted_size,
ctx->sg_plaintext_size +
- tls_ctx->overhead_size);
+ tls_ctx->tx.overhead_size);
}
ret = memcopy_from_iter(sk, &msg->msg_iter, try_to_copy);
full_record = true;
}
required_size = ctx->sg_plaintext_size + copy +
- tls_ctx->overhead_size;
+ tls_ctx->tx.overhead_size;
if (!sk_stream_memory_free(sk))
goto wait_for_sndbuf;
return ret;
}
-void tls_sw_free_tx_resources(struct sock *sk)
+static struct sk_buff *tls_wait_data(struct sock *sk, int flags,
+ long timeo, int *err)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ struct sk_buff *skb;
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
+
+ while (!(skb = ctx->recv_pkt)) {
+ if (sk->sk_err) {
+ *err = sock_error(sk);
+ return NULL;
+ }
+
+ if (sock_flag(sk, SOCK_DONE))
+ return NULL;
+
+ if ((flags & MSG_DONTWAIT) || !timeo) {
+ *err = -EAGAIN;
+ return NULL;
+ }
+
+ add_wait_queue(sk_sleep(sk), &wait);
+ sk_set_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+ sk_wait_event(sk, &timeo, ctx->recv_pkt != skb, &wait);
+ sk_clear_bit(SOCKWQ_ASYNC_WAITDATA, sk);
+ remove_wait_queue(sk_sleep(sk), &wait);
+
+ /* Handle signals */
+ if (signal_pending(current)) {
+ *err = sock_intr_errno(timeo);
+ return NULL;
+ }
+ }
+
+ return skb;
+}
+
+static int decrypt_skb(struct sock *sk, struct sk_buff *skb,
+ struct scatterlist *sgout)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ char iv[TLS_CIPHER_AES_GCM_128_SALT_SIZE + tls_ctx->rx.iv_size];
+ struct scatterlist sgin_arr[MAX_SKB_FRAGS + 2];
+ struct scatterlist *sgin = &sgin_arr[0];
+ struct strp_msg *rxm = strp_msg(skb);
+ int ret, nsg = ARRAY_SIZE(sgin_arr);
+ char aad_recv[TLS_AAD_SPACE_SIZE];
+ struct sk_buff *unused;
+
+ ret = skb_copy_bits(skb, rxm->offset + TLS_HEADER_SIZE,
+ iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE,
+ tls_ctx->rx.iv_size);
+ if (ret < 0)
+ return ret;
+
+ memcpy(iv, tls_ctx->rx.iv, TLS_CIPHER_AES_GCM_128_SALT_SIZE);
+ if (!sgout) {
+ nsg = skb_cow_data(skb, 0, &unused) + 1;
+ sgin = kmalloc_array(nsg, sizeof(*sgin), sk->sk_allocation);
+ if (!sgout)
+ sgout = sgin;
+ }
+
+ sg_init_table(sgin, nsg);
+ sg_set_buf(&sgin[0], aad_recv, sizeof(aad_recv));
+
+ nsg = skb_to_sgvec(skb, &sgin[1],
+ rxm->offset + tls_ctx->rx.prepend_size,
+ rxm->full_len - tls_ctx->rx.prepend_size);
+
+ tls_make_aad(aad_recv,
+ rxm->full_len - tls_ctx->rx.overhead_size,
+ tls_ctx->rx.rec_seq,
+ tls_ctx->rx.rec_seq_size,
+ ctx->control);
+
+ ret = tls_do_decryption(sk, sgin, sgout, iv,
+ rxm->full_len - tls_ctx->rx.overhead_size,
+ skb, sk->sk_allocation);
+
+ if (sgin != &sgin_arr[0])
+ kfree(sgin);
+
+ return ret;
+}
+
+static bool tls_sw_advance_skb(struct sock *sk, struct sk_buff *skb,
+ unsigned int len)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ struct strp_msg *rxm = strp_msg(skb);
+
+ if (len < rxm->full_len) {
+ rxm->offset += len;
+ rxm->full_len -= len;
+
+ return false;
+ }
+
+ /* Finished with message */
+ ctx->recv_pkt = NULL;
+ kfree_skb(skb);
+ strp_unpause(&ctx->strp);
+
+ return true;
+}
+
+int tls_sw_recvmsg(struct sock *sk,
+ struct msghdr *msg,
+ size_t len,
+ int nonblock,
+ int flags,
+ int *addr_len)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ unsigned char control;
+ struct strp_msg *rxm;
+ struct sk_buff *skb;
+ ssize_t copied = 0;
+ bool cmsg = false;
+ int err = 0;
+ long timeo;
+
+ flags |= nonblock;
+
+ if (unlikely(flags & MSG_ERRQUEUE))
+ return sock_recv_errqueue(sk, msg, len, SOL_IP, IP_RECVERR);
+
+ lock_sock(sk);
+
+ timeo = sock_rcvtimeo(sk, flags & MSG_DONTWAIT);
+ do {
+ bool zc = false;
+ int chunk = 0;
+
+ skb = tls_wait_data(sk, flags, timeo, &err);
+ if (!skb)
+ goto recv_end;
+
+ rxm = strp_msg(skb);
+ if (!cmsg) {
+ int cerr;
+
+ cerr = put_cmsg(msg, SOL_TLS, TLS_GET_RECORD_TYPE,
+ sizeof(ctx->control), &ctx->control);
+ cmsg = true;
+ control = ctx->control;
+ if (ctx->control != TLS_RECORD_TYPE_DATA) {
+ if (cerr || msg->msg_flags & MSG_CTRUNC) {
+ err = -EIO;
+ goto recv_end;
+ }
+ }
+ } else if (control != ctx->control) {
+ goto recv_end;
+ }
+
+ if (!ctx->decrypted) {
+ int page_count;
+ int to_copy;
+
+ page_count = iov_iter_npages(&msg->msg_iter,
+ MAX_SKB_FRAGS);
+ to_copy = rxm->full_len - tls_ctx->rx.overhead_size;
+ if (to_copy <= len && page_count < MAX_SKB_FRAGS &&
+ likely(!(flags & MSG_PEEK))) {
+ struct scatterlist sgin[MAX_SKB_FRAGS + 1];
+ char unused[21];
+ int pages = 0;
+
+ zc = true;
+ sg_init_table(sgin, MAX_SKB_FRAGS + 1);
+ sg_set_buf(&sgin[0], unused, 13);
+
+ err = zerocopy_from_iter(sk, &msg->msg_iter,
+ to_copy, &pages,
+ &chunk, &sgin[1],
+ MAX_SKB_FRAGS, false);
+ if (err < 0)
+ goto fallback_to_reg_recv;
+
+ err = decrypt_skb(sk, skb, sgin);
+ for (; pages > 0; pages--)
+ put_page(sg_page(&sgin[pages]));
+ if (err < 0) {
+ tls_err_abort(sk, EBADMSG);
+ goto recv_end;
+ }
+ } else {
+fallback_to_reg_recv:
+ err = decrypt_skb(sk, skb, NULL);
+ if (err < 0) {
+ tls_err_abort(sk, EBADMSG);
+ goto recv_end;
+ }
+ }
+ ctx->decrypted = true;
+ }
+
+ if (!zc) {
+ chunk = min_t(unsigned int, rxm->full_len, len);
+ err = skb_copy_datagram_msg(skb, rxm->offset, msg,
+ chunk);
+ if (err < 0)
+ goto recv_end;
+ }
+
+ copied += chunk;
+ len -= chunk;
+ if (likely(!(flags & MSG_PEEK))) {
+ u8 control = ctx->control;
+
+ if (tls_sw_advance_skb(sk, skb, chunk)) {
+ /* Return full control message to
+ * userspace before trying to parse
+ * another message type
+ */
+ msg->msg_flags |= MSG_EOR;
+ if (control != TLS_RECORD_TYPE_DATA)
+ goto recv_end;
+ }
+ }
+ } while (len);
+
+recv_end:
+ release_sock(sk);
+ return copied ? : err;
+}
+
+ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos,
+ struct pipe_inode_info *pipe,
+ size_t len, unsigned int flags)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sock->sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ struct strp_msg *rxm = NULL;
+ struct sock *sk = sock->sk;
+ struct sk_buff *skb;
+ ssize_t copied = 0;
+ int err = 0;
+ long timeo;
+ int chunk;
+
+ lock_sock(sk);
+
+ timeo = sock_rcvtimeo(sk, flags & MSG_DONTWAIT);
+
+ skb = tls_wait_data(sk, flags, timeo, &err);
+ if (!skb)
+ goto splice_read_end;
+
+ /* splice does not support reading control messages */
+ if (ctx->control != TLS_RECORD_TYPE_DATA) {
+ err = -ENOTSUPP;
+ goto splice_read_end;
+ }
+
+ if (!ctx->decrypted) {
+ err = decrypt_skb(sk, skb, NULL);
+
+ if (err < 0) {
+ tls_err_abort(sk, EBADMSG);
+ goto splice_read_end;
+ }
+ ctx->decrypted = true;
+ }
+ rxm = strp_msg(skb);
+
+ chunk = min_t(unsigned int, rxm->full_len, len);
+ copied = skb_splice_bits(skb, sk, rxm->offset, pipe, chunk, flags);
+ if (copied < 0)
+ goto splice_read_end;
+
+ if (likely(!(flags & MSG_PEEK)))
+ tls_sw_advance_skb(sk, skb, copied);
+
+splice_read_end:
+ release_sock(sk);
+ return copied ? : err;
+}
+
+unsigned int tls_sw_poll(struct file *file, struct socket *sock,
+ struct poll_table_struct *wait)
+{
+ unsigned int ret;
+ struct sock *sk = sock->sk;
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+
+ /* Grab POLLOUT and POLLHUP from the underlying socket */
+ ret = ctx->sk_poll(file, sock, wait);
+
+ /* Clear POLLIN bits, and set based on recv_pkt */
+ ret &= ~(POLLIN | POLLRDNORM);
+ if (ctx->recv_pkt)
+ ret |= POLLIN | POLLRDNORM;
+
+ return ret;
+}
+
+static int tls_read_size(struct strparser *strp, struct sk_buff *skb)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(strp->sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ char header[tls_ctx->rx.prepend_size];
+ struct strp_msg *rxm = strp_msg(skb);
+ size_t cipher_overhead;
+ size_t data_len = 0;
+ int ret;
+
+ /* Verify that we have a full TLS header, or wait for more data */
+ if (rxm->offset + tls_ctx->rx.prepend_size > skb->len)
+ return 0;
+
+ /* Linearize header to local buffer */
+ ret = skb_copy_bits(skb, rxm->offset, header, tls_ctx->rx.prepend_size);
+
+ if (ret < 0)
+ goto read_failure;
+
+ ctx->control = header[0];
+
+ data_len = ((header[4] & 0xFF) | (header[3] << 8));
+
+ cipher_overhead = tls_ctx->rx.tag_size + tls_ctx->rx.iv_size;
+
+ if (data_len > TLS_MAX_PAYLOAD_SIZE + cipher_overhead) {
+ ret = -EMSGSIZE;
+ goto read_failure;
+ }
+ if (data_len < cipher_overhead) {
+ ret = -EBADMSG;
+ goto read_failure;
+ }
+
+ if (header[1] != TLS_VERSION_MINOR(tls_ctx->crypto_recv.version) ||
+ header[2] != TLS_VERSION_MAJOR(tls_ctx->crypto_recv.version)) {
+ ret = -EINVAL;
+ goto read_failure;
+ }
+
+ return data_len + TLS_HEADER_SIZE;
+
+read_failure:
+ tls_err_abort(strp->sk, ret);
+
+ return ret;
+}
+
+static void tls_queue(struct strparser *strp, struct sk_buff *skb)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(strp->sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+ struct strp_msg *rxm;
+
+ rxm = strp_msg(skb);
+
+ ctx->decrypted = false;
+
+ ctx->recv_pkt = skb;
+ strp_pause(strp);
+
+ strp->sk->sk_state_change(strp->sk);
+}
+
+static void tls_data_ready(struct sock *sk)
+{
+ struct tls_context *tls_ctx = tls_get_ctx(sk);
+ struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
+
+ strp_data_ready(&ctx->strp);
+}
+
+void tls_sw_free_resources(struct sock *sk)
{
struct tls_context *tls_ctx = tls_get_ctx(sk);
struct tls_sw_context *ctx = tls_sw_ctx(tls_ctx);
if (ctx->aead_send)
crypto_free_aead(ctx->aead_send);
+ if (ctx->aead_recv) {
+ if (ctx->recv_pkt) {
+ kfree_skb(ctx->recv_pkt);
+ ctx->recv_pkt = NULL;
+ }
+ crypto_free_aead(ctx->aead_recv);
+ strp_stop(&ctx->strp);
+ write_lock_bh(&sk->sk_callback_lock);
+ sk->sk_data_ready = ctx->saved_data_ready;
+ write_unlock_bh(&sk->sk_callback_lock);
+ release_sock(sk);
+ strp_done(&ctx->strp);
+ lock_sock(sk);
+ }
tls_free_both_sg(sk);
kfree(tls_ctx);
}
-int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx)
+int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx)
{
char keyval[TLS_CIPHER_AES_GCM_128_KEY_SIZE];
struct tls_crypto_info *crypto_info;
struct tls12_crypto_info_aes_gcm_128 *gcm_128_info;
struct tls_sw_context *sw_ctx;
+ struct cipher_context *cctx;
+ struct crypto_aead **aead;
+ struct strp_callbacks cb;
u16 nonce_size, tag_size, iv_size, rec_seq_size;
char *iv, *rec_seq;
int rc = 0;
goto out;
}
- if (ctx->priv_ctx) {
- rc = -EEXIST;
- goto out;
- }
-
- sw_ctx = kzalloc(sizeof(*sw_ctx), GFP_KERNEL);
- if (!sw_ctx) {
- rc = -ENOMEM;
- goto out;
+ if (!ctx->priv_ctx) {
+ sw_ctx = kzalloc(sizeof(*sw_ctx), GFP_KERNEL);
+ if (!sw_ctx) {
+ rc = -ENOMEM;
+ goto out;
+ }
+ crypto_init_wait(&sw_ctx->async_wait);
+ } else {
+ sw_ctx = ctx->priv_ctx;
}
- crypto_init_wait(&sw_ctx->async_wait);
-
ctx->priv_ctx = (struct tls_offload_context *)sw_ctx;
- crypto_info = &ctx->crypto_send;
+ if (tx) {
+ crypto_info = &ctx->crypto_send;
+ cctx = &ctx->tx;
+ aead = &sw_ctx->aead_send;
+ } else {
+ crypto_info = &ctx->crypto_recv;
+ cctx = &ctx->rx;
+ aead = &sw_ctx->aead_recv;
+ }
+
switch (crypto_info->cipher_type) {
case TLS_CIPHER_AES_GCM_128: {
nonce_size = TLS_CIPHER_AES_GCM_128_IV_SIZE;
goto free_priv;
}
- ctx->prepend_size = TLS_HEADER_SIZE + nonce_size;
- ctx->tag_size = tag_size;
- ctx->overhead_size = ctx->prepend_size + ctx->tag_size;
- ctx->iv_size = iv_size;
- ctx->iv = kmalloc(iv_size + TLS_CIPHER_AES_GCM_128_SALT_SIZE, GFP_KERNEL);
- if (!ctx->iv) {
+ cctx->prepend_size = TLS_HEADER_SIZE + nonce_size;
+ cctx->tag_size = tag_size;
+ cctx->overhead_size = cctx->prepend_size + cctx->tag_size;
+ cctx->iv_size = iv_size;
+ cctx->iv = kmalloc(iv_size + TLS_CIPHER_AES_GCM_128_SALT_SIZE,
+ GFP_KERNEL);
+ if (!cctx->iv) {
rc = -ENOMEM;
goto free_priv;
}
- memcpy(ctx->iv, gcm_128_info->salt, TLS_CIPHER_AES_GCM_128_SALT_SIZE);
- memcpy(ctx->iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE, iv, iv_size);
- ctx->rec_seq_size = rec_seq_size;
- ctx->rec_seq = kmalloc(rec_seq_size, GFP_KERNEL);
- if (!ctx->rec_seq) {
+ memcpy(cctx->iv, gcm_128_info->salt, TLS_CIPHER_AES_GCM_128_SALT_SIZE);
+ memcpy(cctx->iv + TLS_CIPHER_AES_GCM_128_SALT_SIZE, iv, iv_size);
+ cctx->rec_seq_size = rec_seq_size;
+ cctx->rec_seq = kmalloc(rec_seq_size, GFP_KERNEL);
+ if (!cctx->rec_seq) {
rc = -ENOMEM;
goto free_iv;
}
- memcpy(ctx->rec_seq, rec_seq, rec_seq_size);
-
- sg_init_table(sw_ctx->sg_encrypted_data,
- ARRAY_SIZE(sw_ctx->sg_encrypted_data));
- sg_init_table(sw_ctx->sg_plaintext_data,
- ARRAY_SIZE(sw_ctx->sg_plaintext_data));
-
- sg_init_table(sw_ctx->sg_aead_in, 2);
- sg_set_buf(&sw_ctx->sg_aead_in[0], sw_ctx->aad_space,
- sizeof(sw_ctx->aad_space));
- sg_unmark_end(&sw_ctx->sg_aead_in[1]);
- sg_chain(sw_ctx->sg_aead_in, 2, sw_ctx->sg_plaintext_data);
- sg_init_table(sw_ctx->sg_aead_out, 2);
- sg_set_buf(&sw_ctx->sg_aead_out[0], sw_ctx->aad_space,
- sizeof(sw_ctx->aad_space));
- sg_unmark_end(&sw_ctx->sg_aead_out[1]);
- sg_chain(sw_ctx->sg_aead_out, 2, sw_ctx->sg_encrypted_data);
-
- if (!sw_ctx->aead_send) {
- sw_ctx->aead_send = crypto_alloc_aead("gcm(aes)", 0, 0);
- if (IS_ERR(sw_ctx->aead_send)) {
- rc = PTR_ERR(sw_ctx->aead_send);
- sw_ctx->aead_send = NULL;
+ memcpy(cctx->rec_seq, rec_seq, rec_seq_size);
+
+ if (tx) {
+ sg_init_table(sw_ctx->sg_encrypted_data,
+ ARRAY_SIZE(sw_ctx->sg_encrypted_data));
+ sg_init_table(sw_ctx->sg_plaintext_data,
+ ARRAY_SIZE(sw_ctx->sg_plaintext_data));
+
+ sg_init_table(sw_ctx->sg_aead_in, 2);
+ sg_set_buf(&sw_ctx->sg_aead_in[0], sw_ctx->aad_space,
+ sizeof(sw_ctx->aad_space));
+ sg_unmark_end(&sw_ctx->sg_aead_in[1]);
+ sg_chain(sw_ctx->sg_aead_in, 2, sw_ctx->sg_plaintext_data);
+ sg_init_table(sw_ctx->sg_aead_out, 2);
+ sg_set_buf(&sw_ctx->sg_aead_out[0], sw_ctx->aad_space,
+ sizeof(sw_ctx->aad_space));
+ sg_unmark_end(&sw_ctx->sg_aead_out[1]);
+ sg_chain(sw_ctx->sg_aead_out, 2, sw_ctx->sg_encrypted_data);
+ }
+
+ if (!*aead) {
+ *aead = crypto_alloc_aead("gcm(aes)", 0, 0);
+ if (IS_ERR(*aead)) {
+ rc = PTR_ERR(*aead);
+ *aead = NULL;
goto free_rec_seq;
}
}
memcpy(keyval, gcm_128_info->key, TLS_CIPHER_AES_GCM_128_KEY_SIZE);
- rc = crypto_aead_setkey(sw_ctx->aead_send, keyval,
+ rc = crypto_aead_setkey(*aead, keyval,
TLS_CIPHER_AES_GCM_128_KEY_SIZE);
if (rc)
goto free_aead;
- rc = crypto_aead_setauthsize(sw_ctx->aead_send, ctx->tag_size);
- if (!rc)
- return 0;
+ rc = crypto_aead_setauthsize(*aead, cctx->tag_size);
+ if (rc)
+ goto free_aead;
+
+ if (!tx) {
+ /* Set up strparser */
+ memset(&cb, 0, sizeof(cb));
+ cb.rcv_msg = tls_queue;
+ cb.parse_msg = tls_read_size;
+
+ strp_init(&sw_ctx->strp, sk, &cb);
+
+ write_lock_bh(&sk->sk_callback_lock);
+ sw_ctx->saved_data_ready = sk->sk_data_ready;
+ sk->sk_data_ready = tls_data_ready;
+ write_unlock_bh(&sk->sk_callback_lock);
+
+ sw_ctx->sk_poll = sk->sk_socket->ops->poll;
+
+ strp_check_rcv(&sw_ctx->strp);
+ }
+
+ goto out;
free_aead:
- crypto_free_aead(sw_ctx->aead_send);
- sw_ctx->aead_send = NULL;
+ crypto_free_aead(*aead);
+ *aead = NULL;
free_rec_seq:
- kfree(ctx->rec_seq);
- ctx->rec_seq = NULL;
+ kfree(cctx->rec_seq);
+ cctx->rec_seq = NULL;
free_iv:
- kfree(ctx->iv);
- ctx->iv = NULL;
+ kfree(ctx->tx.iv);
+ ctx->tx.iv = NULL;
free_priv:
kfree(ctx->priv_ctx);
ctx->priv_ctx = NULL;
static struct pernet_operations unix_net_ops = {
.init = unix_net_init,
.exit = unix_net_exit,
- .async = true,
};
static int __init af_unix_init(void)
static struct pernet_operations cfg80211_pernet_ops = {
.exit = cfg80211_pernet_exit,
- .async = true,
};
static int __init cfg80211_init(void)
struct sk_buff *skb;
struct net *net;
- ASSERT_RTNL();
-
+ down_read(&net_rwsem);
for_each_net(net) {
while ((skb = skb_dequeue(&net->wext_nlevents)))
rtnl_notify(skb, net, 0, RTNLGRP_LINK, NULL,
GFP_KERNEL);
}
+ up_read(&net_rwsem);
}
EXPORT_SYMBOL_GPL(wireless_nlevent_flush);
static struct pernet_operations wext_pernet_ops = {
.init = wext_pernet_init,
.exit = wext_pernet_exit,
- .async = true,
};
static int __init wireless_nlevent_init(void)
/* Process events generated by the wireless layer or the driver. */
static void wireless_nlevent_process(struct work_struct *work)
{
- rtnl_lock();
wireless_nlevent_flush();
- rtnl_unlock();
}
static DECLARE_WORK(wireless_nlevent_work, wireless_nlevent_process);
int __net_init wext_proc_init(struct net *net)
{
/* Create /proc/net/wireless entry */
- if (!proc_create("wireless", S_IRUGO, net->proc_net,
+ if (!proc_create("wireless", 0444, net->proc_net,
&wireless_seq_fops))
return -ENOMEM;
if (!proc_mkdir("x25", init_net.proc_net))
return -ENOMEM;
- if (!proc_create("x25/route", S_IRUGO, init_net.proc_net,
- &x25_seq_route_fops))
+ if (!proc_create("x25/route", 0444, init_net.proc_net,
+ &x25_seq_route_fops))
goto out;
- if (!proc_create("x25/socket", S_IRUGO, init_net.proc_net,
- &x25_seq_socket_fops))
+ if (!proc_create("x25/socket", 0444, init_net.proc_net,
+ &x25_seq_socket_fops))
goto out;
- if (!proc_create("x25/forward", S_IRUGO, init_net.proc_net,
- &x25_seq_forward_fops))
+ if (!proc_create("x25/forward", 0444, init_net.proc_net,
+ &x25_seq_forward_fops))
goto out;
return 0;
*/
#include <linux/bottom_half.h>
+#include <linux/cache.h>
#include <linux/interrupt.h>
#include <linux/slab.h>
#include <linux/module.h>
#define XFRM_TRANS_SKB_CB(__skb) ((struct xfrm_trans_cb *)&((__skb)->cb[0]))
-static struct kmem_cache *secpath_cachep __read_mostly;
+static struct kmem_cache *secpath_cachep __ro_after_init;
static DEFINE_SPINLOCK(xfrm_input_afinfo_lock);
static struct xfrm_input_afinfo const __rcu *xfrm_input_afinfo[AF_INET6 + 1];
struct crypto_comp *tfm;
/* This can be any valid CPU ID so we don't need locking. */
- tfm = __this_cpu_read(*pos->tfms);
+ tfm = this_cpu_read(*pos->tfms);
if (!strcmp(crypto_comp_name(tfm), alg_name)) {
pos->users++;
static struct xfrm_policy_afinfo const __rcu *xfrm_policy_afinfo[AF_INET6 + 1]
__read_mostly;
-static struct kmem_cache *xfrm_dst_cache __read_mostly;
+static struct kmem_cache *xfrm_dst_cache __ro_after_init;
static __read_mostly seqcount_t xfrm_policy_hash_generation;
static void xfrm_init_pmtu(struct xfrm_dst **bundle, int nr);
static int xfrm_get_tos(const struct flowi *fl, int family)
{
const struct xfrm_policy_afinfo *afinfo;
- int tos = 0;
+ int tos;
afinfo = xfrm_policy_get_afinfo(family);
- tos = afinfo ? afinfo->get_tos(fl) : 0;
+ if (!afinfo)
+ return 0;
+
+ tos = afinfo->get_tos(fl);
rcu_read_unlock();
void xfrm_policy_cache_flush(void)
{
struct xfrm_dst *old;
- bool found = 0;
+ bool found = false;
int cpu;
might_sleep();
spin_unlock(&pq->hold_queue.lock);
dst_hold(xfrm_dst_path(dst));
- dst = xfrm_lookup(net, xfrm_dst_path(dst), &fl, sk, 0);
+ dst = xfrm_lookup(net, xfrm_dst_path(dst), &fl, sk, XFRM_LOOKUP_QUEUE);
if (IS_ERR(dst))
goto purge_queue;
while (dst->xfrm) {
const struct xfrm_state *xfrm = dst->xfrm;
+ dst = xfrm_dst_child(dst);
+
if (xfrm->props.mode == XFRM_MODE_TRANSPORT)
continue;
if (xfrm->type->flags & XFRM_TYPE_REMOTE_COADDR)
daddr = xfrm->coaddr;
else if (!(xfrm->type->flags & XFRM_TYPE_LOCAL_COADDR))
daddr = &xfrm->id.daddr;
-
- dst = xfrm_dst_child(dst);
}
return daddr;
}
static struct pernet_operations __net_initdata xfrm_net_ops = {
.init = xfrm_net_init,
.exit = xfrm_net_exit,
- .async = true,
};
void __init xfrm_init(void)
int __net_init xfrm_proc_init(struct net *net)
{
- if (!proc_create("xfrm_stat", S_IRUGO, net->proc_net,
+ if (!proc_create("xfrm_stat", 0444, net->proc_net,
&xfrm_statistics_seq_fops))
return -ENOMEM;
return 0;
} else {
XFRM_SKB_CB(skb)->seq.output.low = oseq + 1;
XFRM_SKB_CB(skb)->seq.output.hi = oseq_hi;
- xo->seq.low = oseq = oseq + 1;
+ xo->seq.low = oseq + 1;
xo->seq.hi = oseq_hi;
oseq += skb_shinfo(skb)->gso_segs;
}
struct xfrm_mgr *km;
struct xfrm_policy *pol = NULL;
+#ifdef CONFIG_COMPAT
+ if (in_compat_syscall())
+ return -EOPNOTSUPP;
+#endif
+
if (!optval && !optlen) {
xfrm_sk_policy_insert(sk, XFRM_POLICY_IN, NULL);
xfrm_sk_policy_insert(sk, XFRM_POLICY_OUT, NULL);
struct nlattr *rt = attrs[XFRMA_REPLAY_ESN_VAL];
struct xfrm_replay_state_esn *rs;
- if (p->flags & XFRM_STATE_ESN) {
- if (!rt)
- return -EINVAL;
+ if (!rt)
+ return (p->flags & XFRM_STATE_ESN) ? -EINVAL : 0;
- rs = nla_data(rt);
+ rs = nla_data(rt);
- if (rs->bmp_len > XFRMA_REPLAY_ESN_MAX / sizeof(rs->bmp[0]) / 8)
- return -EINVAL;
-
- if (nla_len(rt) < (int)xfrm_replay_state_esn_len(rs) &&
- nla_len(rt) != sizeof(*rs))
- return -EINVAL;
- }
+ if (rs->bmp_len > XFRMA_REPLAY_ESN_MAX / sizeof(rs->bmp[0]) / 8)
+ return -EINVAL;
- if (!rt)
- return 0;
+ if (nla_len(rt) < (int)xfrm_replay_state_esn_len(rs) &&
+ nla_len(rt) != sizeof(*rs))
+ return -EINVAL;
/* As only ESP and AH support ESN feature. */
if ((p->id.proto != IPPROTO_ESP) && (p->id.proto != IPPROTO_AH))
static struct pernet_operations xfrm_user_net_ops = {
.init = xfrm_user_net_init,
.exit_batch = xfrm_user_net_exit,
- .async = true,
};
static int __init xfrm_user_init(void)
bool is_cgroup_sk = strncmp(event, "cgroup/sock", 11) == 0;
bool is_sockops = strncmp(event, "sockops", 7) == 0;
bool is_sk_skb = strncmp(event, "sk_skb", 6) == 0;
+ bool is_sk_msg = strncmp(event, "sk_msg", 6) == 0;
size_t insns_cnt = size / sizeof(struct bpf_insn);
enum bpf_prog_type prog_type;
char buf[256];
prog_type = BPF_PROG_TYPE_SOCK_OPS;
} else if (is_sk_skb) {
prog_type = BPF_PROG_TYPE_SK_SKB;
+ } else if (is_sk_msg) {
+ prog_type = BPF_PROG_TYPE_SK_MSG;
} else {
printf("Unknown event '%s'\n", event);
return -1;
if (is_xdp || is_perf_event || is_cgroup_skb || is_cgroup_sk)
return 0;
- if (is_socket || is_sockops || is_sk_skb) {
+ if (is_socket || is_sockops || is_sk_skb || is_sk_msg) {
if (is_socket)
event += 6;
else
memcmp(shname, "socket", 6) == 0 ||
memcmp(shname, "cgroup/", 7) == 0 ||
memcmp(shname, "sockops", 7) == 0 ||
- memcmp(shname, "sk_skb", 6) == 0) {
+ memcmp(shname, "sk_skb", 6) == 0 ||
+ memcmp(shname, "sk_msg", 6) == 0) {
ret = load_and_attach(shname, data->d_buf,
data->d_size);
if (ret != 0)
{
char time_fmt1[] = "Time Enabled: %llu, Time Running: %llu";
char time_fmt2[] = "Get Time Failed, ErrCode: %d";
+ char addr_fmt[] = "Address recorded on event: %llx";
char fmt[] = "CPU-%d period %lld ip %llx";
u32 cpu = bpf_get_smp_processor_id();
struct bpf_perf_event_value value_buf;
else
bpf_trace_printk(time_fmt2, sizeof(time_fmt2), ret);
+ if (ctx->addr != 0)
+ bpf_trace_printk(addr_fmt, sizeof(addr_fmt), ctx->addr);
+
val = bpf_map_lookup_elem(&counts, &key);
if (val)
(*val)++;
/* Intel Instruction Retired */
.config = 0xc0,
};
+ struct perf_event_attr attr_type_raw_lock_load = {
+ .sample_freq = SAMPLE_FREQ,
+ .freq = 1,
+ .type = PERF_TYPE_RAW,
+ /* Intel MEM_UOPS_RETIRED.LOCK_LOADS */
+ .config = 0x21d0,
+ /* Request to record lock address from PEBS */
+ .sample_type = PERF_SAMPLE_ADDR,
+ /* Record address value requires precise event */
+ .precise_ip = 2,
+ };
printf("Test HW_CPU_CYCLES\n");
test_perf_event_all_cpu(&attr_type_hw);
test_perf_event_all_cpu(&attr_type_raw);
test_perf_event_task(&attr_type_raw);
+ printf("Test Lock Load\n");
+ test_perf_event_all_cpu(&attr_type_raw_lock_load);
+ test_perf_event_task(&attr_type_raw_lock_load);
+
printf("*** PASS ***\n");
}
.max_entries = 20,
};
+struct bpf_map_def SEC("maps") sock_map_txmsg = {
+ .type = BPF_MAP_TYPE_SOCKMAP,
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .max_entries = 20,
+};
+
+struct bpf_map_def SEC("maps") sock_map_redir = {
+ .type = BPF_MAP_TYPE_SOCKMAP,
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .max_entries = 1,
+};
+
+struct bpf_map_def SEC("maps") sock_apply_bytes = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .max_entries = 1
+};
+
+struct bpf_map_def SEC("maps") sock_cork_bytes = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .max_entries = 1
+};
+
+struct bpf_map_def SEC("maps") sock_pull_bytes = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .max_entries = 2
+};
+
+
SEC("sk_skb1")
int bpf_prog1(struct __sk_buff *skb)
{
return 0;
}
+
+SEC("sk_msg1")
+int bpf_prog4(struct sk_msg_md *msg)
+{
+ int *bytes, zero = 0, one = 1;
+ int *start, *end;
+
+ bytes = bpf_map_lookup_elem(&sock_apply_bytes, &zero);
+ if (bytes)
+ bpf_msg_apply_bytes(msg, *bytes);
+ bytes = bpf_map_lookup_elem(&sock_cork_bytes, &zero);
+ if (bytes)
+ bpf_msg_cork_bytes(msg, *bytes);
+ start = bpf_map_lookup_elem(&sock_pull_bytes, &zero);
+ end = bpf_map_lookup_elem(&sock_pull_bytes, &one);
+ if (start && end)
+ bpf_msg_pull_data(msg, *start, *end, 0);
+ return SK_PASS;
+}
+
+SEC("sk_msg2")
+int bpf_prog5(struct sk_msg_md *msg)
+{
+ int err1 = -1, err2 = -1, zero = 0, one = 1;
+ int *bytes, *start, *end, len1, len2;
+
+ bytes = bpf_map_lookup_elem(&sock_apply_bytes, &zero);
+ if (bytes)
+ err1 = bpf_msg_apply_bytes(msg, *bytes);
+ bytes = bpf_map_lookup_elem(&sock_cork_bytes, &zero);
+ if (bytes)
+ err2 = bpf_msg_cork_bytes(msg, *bytes);
+ len1 = (__u64)msg->data_end - (__u64)msg->data;
+ start = bpf_map_lookup_elem(&sock_pull_bytes, &zero);
+ end = bpf_map_lookup_elem(&sock_pull_bytes, &one);
+ if (start && end) {
+ int err;
+
+ bpf_printk("sk_msg2: pull(%i:%i)\n",
+ start ? *start : 0, end ? *end : 0);
+ err = bpf_msg_pull_data(msg, *start, *end, 0);
+ if (err)
+ bpf_printk("sk_msg2: pull_data err %i\n",
+ err);
+ len2 = (__u64)msg->data_end - (__u64)msg->data;
+ bpf_printk("sk_msg2: length update %i->%i\n",
+ len1, len2);
+ }
+ bpf_printk("sk_msg2: data length %i err1 %i err2 %i\n",
+ len1, err1, err2);
+ return SK_PASS;
+}
+
+SEC("sk_msg3")
+int bpf_prog6(struct sk_msg_md *msg)
+{
+ int *bytes, zero = 0, one = 1;
+ int *start, *end;
+
+ bytes = bpf_map_lookup_elem(&sock_apply_bytes, &zero);
+ if (bytes)
+ bpf_msg_apply_bytes(msg, *bytes);
+ bytes = bpf_map_lookup_elem(&sock_cork_bytes, &zero);
+ if (bytes)
+ bpf_msg_cork_bytes(msg, *bytes);
+ start = bpf_map_lookup_elem(&sock_pull_bytes, &zero);
+ end = bpf_map_lookup_elem(&sock_pull_bytes, &one);
+ if (start && end)
+ bpf_msg_pull_data(msg, *start, *end, 0);
+ return bpf_msg_redirect_map(msg, &sock_map_redir, zero, 0);
+}
+
+SEC("sk_msg4")
+int bpf_prog7(struct sk_msg_md *msg)
+{
+ int err1 = 0, err2 = 0, zero = 0, one = 1;
+ int *bytes, *start, *end, len1, len2;
+
+ bytes = bpf_map_lookup_elem(&sock_apply_bytes, &zero);
+ if (bytes)
+ err1 = bpf_msg_apply_bytes(msg, *bytes);
+ bytes = bpf_map_lookup_elem(&sock_cork_bytes, &zero);
+ if (bytes)
+ err2 = bpf_msg_cork_bytes(msg, *bytes);
+ len1 = (__u64)msg->data_end - (__u64)msg->data;
+ start = bpf_map_lookup_elem(&sock_pull_bytes, &zero);
+ end = bpf_map_lookup_elem(&sock_pull_bytes, &one);
+ if (start && end) {
+ int err;
+
+ bpf_printk("sk_msg2: pull(%i:%i)\n",
+ start ? *start : 0, end ? *end : 0);
+ err = bpf_msg_pull_data(msg, *start, *end, 0);
+ if (err)
+ bpf_printk("sk_msg2: pull_data err %i\n",
+ err);
+ len2 = (__u64)msg->data_end - (__u64)msg->data;
+ bpf_printk("sk_msg2: length update %i->%i\n",
+ len1, len2);
+ }
+ bpf_printk("sk_msg3: redirect(%iB) err1=%i err2=%i\n",
+ len1, err1, err2);
+ return bpf_msg_redirect_map(msg, &sock_map_redir, zero, 0);
+}
+
+SEC("sk_msg5")
+int bpf_prog8(struct sk_msg_md *msg)
+{
+ void *data_end = (void *)(long) msg->data_end;
+ void *data = (void *)(long) msg->data;
+ int ret = 0, *bytes, zero = 0;
+
+ bytes = bpf_map_lookup_elem(&sock_apply_bytes, &zero);
+ if (bytes) {
+ ret = bpf_msg_apply_bytes(msg, *bytes);
+ if (ret)
+ return SK_DROP;
+ } else {
+ return SK_DROP;
+ }
+ return SK_PASS;
+}
+SEC("sk_msg6")
+int bpf_prog9(struct sk_msg_md *msg)
+{
+ void *data_end = (void *)(long) msg->data_end;
+ void *data = (void *)(long) msg->data;
+ int ret = 0, *bytes, zero = 0;
+
+ bytes = bpf_map_lookup_elem(&sock_cork_bytes, &zero);
+ if (bytes) {
+ if (((__u64)data_end - (__u64)data) >= *bytes)
+ return SK_PASS;
+ ret = bpf_msg_cork_bytes(msg, *bytes);
+ if (ret)
+ return SK_DROP;
+ }
+ return SK_PASS;
+}
+
+SEC("sk_msg7")
+int bpf_prog10(struct sk_msg_md *msg)
+{
+ int *bytes, zero = 0, one = 1;
+ int *start, *end;
+
+ bytes = bpf_map_lookup_elem(&sock_apply_bytes, &zero);
+ if (bytes)
+ bpf_msg_apply_bytes(msg, *bytes);
+ bytes = bpf_map_lookup_elem(&sock_cork_bytes, &zero);
+ if (bytes)
+ bpf_msg_cork_bytes(msg, *bytes);
+ start = bpf_map_lookup_elem(&sock_pull_bytes, &zero);
+ end = bpf_map_lookup_elem(&sock_pull_bytes, &one);
+ if (start && end)
+ bpf_msg_pull_data(msg, *start, *end, 0);
+
+ return SK_DROP;
+}
+
+
char _license[] SEC("license") = "GPL";
--- /dev/null
+#Test a bunch of positive cases to verify basic functionality
+for prog in "--txmsg" "--txmsg_redir" "--txmsg_drop"; do
+for t in "sendmsg" "sendpage"; do
+for r in 1 10 100; do
+ for i in 1 10 100; do
+ for l in 1 10 100; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+ done
+ done
+done
+done
+done
+
+#Test max iov
+t="sendmsg"
+r=1
+i=1024
+l=1
+prog="--txmsg"
+
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+echo $TEST
+$TEST
+sleep 2
+prog="--txmsg_redir"
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+echo $TEST
+$TEST
+
+# Test max iov with 1k send
+
+t="sendmsg"
+r=1
+i=1024
+l=1024
+prog="--txmsg"
+
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+echo $TEST
+$TEST
+sleep 2
+prog="--txmsg_redir"
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+echo $TEST
+$TEST
+sleep 2
+
+# Test apply with 1B
+r=1
+i=1024
+l=1024
+prog="--txmsg_apply 1"
+
+for t in "sendmsg" "sendpage"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply with larger value than send
+r=1
+i=8
+l=1024
+prog="--txmsg_apply 2048"
+
+for t in "sendmsg" "sendpage"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply with apply that never reaches limit
+r=1024
+i=1
+l=1
+prog="--txmsg_apply 2048"
+
+for t in "sendmsg" "sendpage"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply and redirect with 1B
+r=1
+i=1024
+l=1024
+prog="--txmsg_redir --txmsg_apply 1"
+
+for t in "sendmsg" "sendpage"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply and redirect with larger value than send
+r=1
+i=8
+l=1024
+prog="--txmsg_redir --txmsg_apply 2048"
+
+for t in "sendmsg" "sendpage"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply and redirect with apply that never reaches limit
+r=1024
+i=1
+l=1
+prog="--txmsg_apply 2048"
+
+for t in "sendmsg" "sendpage"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test cork with 1B not really useful but test it anyways
+r=1
+i=1024
+l=1024
+prog="--txmsg_cork 1"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test cork with a more reasonable 100B
+r=1
+i=1000
+l=1000
+prog="--txmsg_cork 100"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test cork with larger value than send
+r=1
+i=8
+l=1024
+prog="--txmsg_cork 2048"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test cork with cork that never reaches limit
+r=1024
+i=1
+l=1
+prog="--txmsg_cork 2048"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+r=1
+i=1024
+l=1024
+prog="--txmsg_redir --txmsg_cork 1"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test cork with a more reasonable 100B
+r=1
+i=1000
+l=1000
+prog="--txmsg_redir --txmsg_cork 100"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test cork with larger value than send
+r=1
+i=8
+l=1024
+prog="--txmsg_redir --txmsg_cork 2048"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test cork with cork that never reaches limit
+r=1024
+i=1
+l=1
+prog="--txmsg_cork 2048"
+
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+
+# mix and match cork and apply not really useful but valid programs
+
+# Test apply < cork
+r=100
+i=1
+l=5
+prog="--txmsg_apply 10 --txmsg_cork 100"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Try again with larger sizes so we hit overflow case
+r=100
+i=1000
+l=2048
+prog="--txmsg_apply 4096 --txmsg_cork 8096"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply > cork
+r=100
+i=1
+l=5
+prog="--txmsg_apply 100 --txmsg_cork 10"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Again with larger sizes so we hit overflow cases
+r=100
+i=1000
+l=2048
+prog="--txmsg_apply 8096 --txmsg_cork 4096"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+
+# Test apply = cork
+r=100
+i=1
+l=5
+prog="--txmsg_apply 10 --txmsg_cork 10"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+r=100
+i=1000
+l=2048
+prog="--txmsg_apply 4096 --txmsg_cork 4096"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply < cork
+r=100
+i=1
+l=5
+prog="--txmsg_redir --txmsg_apply 10 --txmsg_cork 100"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Try again with larger sizes so we hit overflow case
+r=100
+i=1000
+l=2048
+prog="--txmsg_redir --txmsg_apply 4096 --txmsg_cork 8096"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Test apply > cork
+r=100
+i=1
+l=5
+prog="--txmsg_redir --txmsg_apply 100 --txmsg_cork 10"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Again with larger sizes so we hit overflow cases
+r=100
+i=1000
+l=2048
+prog="--txmsg_redir --txmsg_apply 8096 --txmsg_cork 4096"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+
+# Test apply = cork
+r=100
+i=1
+l=5
+prog="--txmsg_redir --txmsg_apply 10 --txmsg_cork 10"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+r=100
+i=1000
+l=2048
+prog="--txmsg_redir --txmsg_apply 4096 --txmsg_cork 4096"
+for t in "sendpage" "sendmsg"; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+# Tests for bpf_msg_pull_data()
+for i in `seq 99 100 1600`; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t sendpage -r 16 -i 1 -l 100 \
+ --txmsg --txmsg_start 0 --txmsg_end $i --txmsg_cork 1600"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+for i in `seq 199 100 1600`; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t sendpage -r 16 -i 1 -l 100 \
+ --txmsg --txmsg_start 100 --txmsg_end $i --txmsg_cork 1600"
+ echo $TEST
+ $TEST
+ sleep 2
+done
+
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t sendpage -r 16 -i 1 -l 100 \
+ --txmsg --txmsg_start 1500 --txmsg_end 1600 --txmsg_cork 1600"
+echo $TEST
+$TEST
+sleep 2
+
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t sendpage -r 16 -i 1 -l 100 \
+ --txmsg --txmsg_start 1111 --txmsg_end 1112 --txmsg_cork 1600"
+echo $TEST
+$TEST
+sleep 2
+
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t sendpage -r 16 -i 1 -l 100 \
+ --txmsg --txmsg_start 1111 --txmsg_end 0 --txmsg_cork 1600"
+echo $TEST
+$TEST
+sleep 2
+
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t sendpage -r 16 -i 1 -l 100 \
+ --txmsg --txmsg_start 0 --txmsg_end 1601 --txmsg_cork 1600"
+echo $TEST
+$TEST
+sleep 2
+
+TEST="./sockmap --cgroup /mnt/cgroup2/ -t sendpage -r 16 -i 1 -l 100 \
+ --txmsg --txmsg_start 0 --txmsg_end 1601 --txmsg_cork 1602"
+echo $TEST
+$TEST
+sleep 2
+
+# Run through gamut again with start and end
+for prog in "--txmsg" "--txmsg_redir" "--txmsg_drop"; do
+for t in "sendmsg" "sendpage"; do
+for r in 1 10 100; do
+ for i in 1 10 100; do
+ for l in 1 10 100; do
+ TEST="./sockmap --cgroup /mnt/cgroup2/ -t $t -r $r -i $i -l $l $prog --txmsg_start 1 --txmsg_end 2"
+ echo $TEST
+ $TEST
+ sleep 2
+ done
+ done
+done
+done
+done
+
+# Some specific tests to cover specific code paths
+./sockmap --cgroup /mnt/cgroup2/ -t sendpage \
+ -r 5 -i 1 -l 1 --txmsg_redir --txmsg_cork 5 --txmsg_apply 3
+./sockmap --cgroup /mnt/cgroup2/ -t sendmsg \
+ -r 5 -i 1 -l 1 --txmsg_redir --txmsg_cork 5 --txmsg_apply 3
+./sockmap --cgroup /mnt/cgroup2/ -t sendpage \
+ -r 5 -i 1 -l 1 --txmsg_redir --txmsg_cork 5 --txmsg_apply 5
+./sockmap --cgroup /mnt/cgroup2/ -t sendmsg \
+ -r 5 -i 1 -l 1 --txmsg_redir --txmsg_cork 5 --txmsg_apply 5
#include <sys/time.h>
#include <sys/resource.h>
#include <sys/types.h>
+#include <sys/sendfile.h>
#include <linux/netlink.h>
#include <linux/socket.h>
/* global sockets */
int s1, s2, c1, c2, p1, p2;
+int txmsg_pass;
+int txmsg_noisy;
+int txmsg_redir;
+int txmsg_redir_noisy;
+int txmsg_drop;
+int txmsg_apply;
+int txmsg_cork;
+int txmsg_start;
+int txmsg_end;
+
static const struct option long_options[] = {
{"help", no_argument, NULL, 'h' },
{"cgroup", required_argument, NULL, 'c' },
{"iov_count", required_argument, NULL, 'i' },
{"length", required_argument, NULL, 'l' },
{"test", required_argument, NULL, 't' },
+ {"data_test", no_argument, NULL, 'd' },
+ {"txmsg", no_argument, &txmsg_pass, 1 },
+ {"txmsg_noisy", no_argument, &txmsg_noisy, 1 },
+ {"txmsg_redir", no_argument, &txmsg_redir, 1 },
+ {"txmsg_redir_noisy", no_argument, &txmsg_redir_noisy, 1},
+ {"txmsg_drop", no_argument, &txmsg_drop, 1 },
+ {"txmsg_apply", required_argument, NULL, 'a'},
+ {"txmsg_cork", required_argument, NULL, 'k'},
+ {"txmsg_start", required_argument, NULL, 's'},
+ {"txmsg_end", required_argument, NULL, 'e'},
{0, 0, NULL, 0 }
};
struct timespec end;
};
+struct sockmap_options {
+ int verbose;
+ bool base;
+ bool sendpage;
+ bool data_test;
+ bool drop_expected;
+};
+
+static int msg_loop_sendpage(int fd, int iov_length, int cnt,
+ struct msg_stats *s,
+ struct sockmap_options *opt)
+{
+ bool drop = opt->drop_expected;
+ unsigned char k = 0;
+ FILE *file;
+ int i, fp;
+
+ file = fopen(".sendpage_tst.tmp", "w+");
+ for (i = 0; i < iov_length * cnt; i++, k++)
+ fwrite(&k, sizeof(char), 1, file);
+ fflush(file);
+ fseek(file, 0, SEEK_SET);
+ fclose(file);
+
+ fp = open(".sendpage_tst.tmp", O_RDONLY);
+ clock_gettime(CLOCK_MONOTONIC, &s->start);
+ for (i = 0; i < cnt; i++) {
+ int sent = sendfile(fd, fp, NULL, iov_length);
+
+ if (!drop && sent < 0) {
+ perror("send loop error:");
+ close(fp);
+ return sent;
+ } else if (drop && sent >= 0) {
+ printf("sendpage loop error expected: %i\n", sent);
+ close(fp);
+ return -EIO;
+ }
+
+ if (sent > 0)
+ s->bytes_sent += sent;
+ }
+ clock_gettime(CLOCK_MONOTONIC, &s->end);
+ close(fp);
+ return 0;
+}
+
static int msg_loop(int fd, int iov_count, int iov_length, int cnt,
- struct msg_stats *s, bool tx)
+ struct msg_stats *s, bool tx,
+ struct sockmap_options *opt)
{
struct msghdr msg = {0};
int err, i, flags = MSG_NOSIGNAL;
struct iovec *iov;
+ unsigned char k;
+ bool data_test = opt->data_test;
+ bool drop = opt->drop_expected;
iov = calloc(iov_count, sizeof(struct iovec));
if (!iov)
return errno;
+ k = 0;
for (i = 0; i < iov_count; i++) {
- char *d = calloc(iov_length, sizeof(char));
+ unsigned char *d = calloc(iov_length, sizeof(char));
if (!d) {
fprintf(stderr, "iov_count %i/%i OOM\n", i, iov_count);
}
iov[i].iov_base = d;
iov[i].iov_len = iov_length;
+
+ if (data_test && tx) {
+ int j;
+
+ for (j = 0; j < iov_length; j++)
+ d[j] = k++;
+ }
}
msg.msg_iov = iov;
msg.msg_iovlen = iov_count;
+ k = 0;
if (tx) {
clock_gettime(CLOCK_MONOTONIC, &s->start);
for (i = 0; i < cnt; i++) {
int sent = sendmsg(fd, &msg, flags);
- if (sent < 0) {
+ if (!drop && sent < 0) {
perror("send loop error:");
goto out_errno;
+ } else if (drop && sent >= 0) {
+ printf("send loop error expected: %i\n", sent);
+ errno = -EIO;
+ goto out_errno;
}
- s->bytes_sent += sent;
+ if (sent > 0)
+ s->bytes_sent += sent;
}
clock_gettime(CLOCK_MONOTONIC, &s->end);
} else {
}
s->bytes_recvd += recv;
+
+ if (data_test) {
+ int j;
+
+ for (i = 0; i < msg.msg_iovlen; i++) {
+ unsigned char *d = iov[i].iov_base;
+
+ for (j = 0;
+ j < iov[i].iov_len && recv; j++) {
+ if (d[j] != k++) {
+ errno = -EIO;
+ fprintf(stderr,
+ "detected data corruption @iov[%i]:%i %02x != %02x, %02x ?= %02x\n",
+ i, j, d[j], k - 1, d[j+1], k + 1);
+ goto out_errno;
+ }
+ recv--;
+ }
+ }
+ }
}
clock_gettime(CLOCK_MONOTONIC, &s->end);
}
}
static int sendmsg_test(int iov_count, int iov_buf, int cnt,
- int verbose, bool base)
+ struct sockmap_options *opt)
{
float sent_Bps = 0, recvd_Bps = 0;
int rx_fd, txpid, rxpid, err = 0;
errno = 0;
- if (base)
+ if (opt->base)
rx_fd = p1;
else
rx_fd = p2;
rxpid = fork();
if (rxpid == 0) {
- err = msg_loop(rx_fd, iov_count, iov_buf, cnt, &s, false);
+ if (opt->drop_expected)
+ exit(1);
+
+ if (opt->sendpage)
+ iov_count = 1;
+ err = msg_loop(rx_fd, iov_count, iov_buf,
+ cnt, &s, false, opt);
if (err)
fprintf(stderr,
"msg_loop_rx: iov_count %i iov_buf %i cnt %i err %i\n",
txpid = fork();
if (txpid == 0) {
- err = msg_loop(c1, iov_count, iov_buf, cnt, &s, true);
+ if (opt->sendpage)
+ err = msg_loop_sendpage(c1, iov_buf, cnt, &s, opt);
+ else
+ err = msg_loop(c1, iov_count, iov_buf,
+ cnt, &s, true, opt);
+
if (err)
fprintf(stderr,
"msg_loop_tx: iov_count %i iov_buf %i cnt %i err %i\n",
return err;
}
-static int forever_ping_pong(int rate, int verbose)
+static int forever_ping_pong(int rate, struct sockmap_options *opt)
{
struct timeval timeout;
char buf[1024] = {0};
if (rate)
sleep(rate);
- if (verbose) {
+ if (opt->verbose) {
printf(".");
fflush(stdout);
PING_PONG,
SENDMSG,
BASE,
+ BASE_SENDPAGE,
+ SENDPAGE,
};
int main(int argc, char **argv)
{
- int iov_count = 1, length = 1024, rate = 1, verbose = 0;
+ int iov_count = 1, length = 1024, rate = 1, tx_prog_fd;
struct rlimit r = {10 * 1024 * 1024, RLIM_INFINITY};
int opt, longindex, err, cg_fd = 0;
+ struct sockmap_options options = {0};
int test = PING_PONG;
char filename[256];
- while ((opt = getopt_long(argc, argv, "hvc:r:i:l:t:",
+ while ((opt = getopt_long(argc, argv, ":dhvc:r:i:l:t:",
long_options, &longindex)) != -1) {
switch (opt) {
- /* Cgroup configuration */
+ case 's':
+ txmsg_start = atoi(optarg);
+ break;
+ case 'e':
+ txmsg_end = atoi(optarg);
+ break;
+ case 'a':
+ txmsg_apply = atoi(optarg);
+ break;
+ case 'k':
+ txmsg_cork = atoi(optarg);
+ break;
case 'c':
cg_fd = open(optarg, O_DIRECTORY, O_RDONLY);
if (cg_fd < 0) {
rate = atoi(optarg);
break;
case 'v':
- verbose = 1;
+ options.verbose = 1;
break;
case 'i':
iov_count = atoi(optarg);
case 'l':
length = atoi(optarg);
break;
+ case 'd':
+ options.data_test = true;
+ break;
case 't':
if (strcmp(optarg, "ping") == 0) {
test = PING_PONG;
test = SENDMSG;
} else if (strcmp(optarg, "base") == 0) {
test = BASE;
+ } else if (strcmp(optarg, "base_sendpage") == 0) {
+ test = BASE_SENDPAGE;
+ } else if (strcmp(optarg, "sendpage") == 0) {
+ test = SENDPAGE;
} else {
usage(argv);
return -1;
}
break;
+ case 0:
+ break;
case 'h':
default:
usage(argv);
/* catch SIGINT */
signal(SIGINT, running_handler);
- /* If base test skip BPF setup */
- if (test == BASE)
- goto run;
-
if (load_bpf_file(filename)) {
fprintf(stderr, "load_bpf_file: (%s) %s\n",
filename, strerror(errno));
return 1;
}
+ /* If base test skip BPF setup */
+ if (test == BASE || test == BASE_SENDPAGE)
+ goto run;
+
/* Attach programs to sockmap */
err = bpf_prog_attach(prog_fd[0], map_fd[0],
BPF_SK_SKB_STREAM_PARSER, 0);
goto out;
}
- if (test == PING_PONG)
- err = forever_ping_pong(rate, verbose);
- else if (test == SENDMSG)
- err = sendmsg_test(iov_count, length, rate, verbose, false);
- else if (test == BASE)
- err = sendmsg_test(iov_count, length, rate, verbose, true);
+ /* Attach txmsg program to sockmap */
+ if (txmsg_pass)
+ tx_prog_fd = prog_fd[3];
+ else if (txmsg_noisy)
+ tx_prog_fd = prog_fd[4];
+ else if (txmsg_redir)
+ tx_prog_fd = prog_fd[5];
+ else if (txmsg_redir_noisy)
+ tx_prog_fd = prog_fd[6];
+ else if (txmsg_drop)
+ tx_prog_fd = prog_fd[9];
+ /* apply and cork must be last */
+ else if (txmsg_apply)
+ tx_prog_fd = prog_fd[7];
+ else if (txmsg_cork)
+ tx_prog_fd = prog_fd[8];
else
+ tx_prog_fd = 0;
+
+ if (tx_prog_fd) {
+ int redir_fd, i = 0;
+
+ err = bpf_prog_attach(tx_prog_fd,
+ map_fd[1], BPF_SK_MSG_VERDICT, 0);
+ if (err) {
+ fprintf(stderr,
+ "ERROR: bpf_prog_attach (txmsg): %d (%s)\n",
+ err, strerror(errno));
+ return err;
+ }
+
+ err = bpf_map_update_elem(map_fd[1], &i, &c1, BPF_ANY);
+ if (err) {
+ fprintf(stderr,
+ "ERROR: bpf_map_update_elem (txmsg): %d (%s\n",
+ err, strerror(errno));
+ return err;
+ }
+
+ if (txmsg_redir || txmsg_redir_noisy)
+ redir_fd = c2;
+ else
+ redir_fd = c1;
+
+ err = bpf_map_update_elem(map_fd[2], &i, &redir_fd, BPF_ANY);
+ if (err) {
+ fprintf(stderr,
+ "ERROR: bpf_map_update_elem (txmsg): %d (%s\n",
+ err, strerror(errno));
+ return err;
+ }
+
+ if (txmsg_apply) {
+ err = bpf_map_update_elem(map_fd[3],
+ &i, &txmsg_apply, BPF_ANY);
+ if (err) {
+ fprintf(stderr,
+ "ERROR: bpf_map_update_elem (apply_bytes): %d (%s\n",
+ err, strerror(errno));
+ return err;
+ }
+ }
+
+ if (txmsg_cork) {
+ err = bpf_map_update_elem(map_fd[4],
+ &i, &txmsg_cork, BPF_ANY);
+ if (err) {
+ fprintf(stderr,
+ "ERROR: bpf_map_update_elem (cork_bytes): %d (%s\n",
+ err, strerror(errno));
+ return err;
+ }
+ }
+
+ if (txmsg_start) {
+ err = bpf_map_update_elem(map_fd[5],
+ &i, &txmsg_start, BPF_ANY);
+ if (err) {
+ fprintf(stderr,
+ "ERROR: bpf_map_update_elem (txmsg_start): %d (%s)\n",
+ err, strerror(errno));
+ return err;
+ }
+ }
+
+ if (txmsg_end) {
+ i = 1;
+ err = bpf_map_update_elem(map_fd[5],
+ &i, &txmsg_end, BPF_ANY);
+ if (err) {
+ fprintf(stderr,
+ "ERROR: bpf_map_update_elem (txmsg_end): %d (%s)\n",
+ err, strerror(errno));
+ return err;
+ }
+ }
+ }
+
+ if (txmsg_drop)
+ options.drop_expected = true;
+
+ if (test == PING_PONG)
+ err = forever_ping_pong(rate, &options);
+ else if (test == SENDMSG) {
+ options.base = false;
+ options.sendpage = false;
+ err = sendmsg_test(iov_count, length, rate, &options);
+ } else if (test == SENDPAGE) {
+ options.base = false;
+ options.sendpage = true;
+ err = sendmsg_test(iov_count, length, rate, &options);
+ } else if (test == BASE) {
+ options.base = true;
+ options.sendpage = false;
+ err = sendmsg_test(iov_count, length, rate, &options);
+ } else if (test == BASE_SENDPAGE) {
+ options.base = true;
+ options.sendpage = true;
+ err = sendmsg_test(iov_count, length, rate, &options);
+ } else
fprintf(stderr, "unknown test\n");
out:
bpf_prog_detach2(prog_fd[2], cg_fd, BPF_CGROUP_SOCK_OPS);
echo '\#include <asm-generic/vmlinux.lds.h>'; \
echo '.section .dtb.init.rodata,"a"'; \
echo '.balign STRUCT_ALIGNMENT'; \
- echo '.global __dtb_$(*F)_begin'; \
- echo '__dtb_$(*F)_begin:'; \
+ echo '.global __dtb_$(subst -,_,$(*F))_begin'; \
+ echo '__dtb_$(subst -,_,$(*F))_begin:'; \
echo '.incbin "$<" '; \
- echo '__dtb_$(*F)_end:'; \
- echo '.global __dtb_$(*F)_end'; \
+ echo '__dtb_$(subst -,_,$(*F))_end:'; \
+ echo '.global __dtb_$(subst -,_,$(*F))_end'; \
echo '.balign STRUCT_ALIGNMENT'; \
) > $@
* (Note: it'd be easy to port over the complete mkdep state machine,
* but I don't think the added complexity is worth it)
*/
-/*
- * Note 2: if somebody writes HELLO_CONFIG_BOOM in a file, it will depend onto
- * CONFIG_BOOM. This could seem a bug (not too hard to fix), but please do not
- * fix it! Some UserModeLinux files (look at arch/um/) call CONFIG_BOOM as
- * UML_CONFIG_BOOM, to avoid conflicts with /usr/include/linux/autoconf.h,
- * through arch/um/include/uml-config.h; this fixdep "bug" makes sure that
- * those files will have correct dependencies.
- */
#include <sys/types.h>
#include <sys/stat.h>
static void parse_config_file(const char *p)
{
const char *q, *r;
+ const char *start = p;
while ((p = strstr(p, "CONFIG_"))) {
+ if (p > start && (isalnum(p[-1]) || p[-1] == '_')) {
+ p += 7;
+ continue;
+ }
p += 7;
q = p;
while (*q && (isalnum(*q) || *q == '_'))
{
return str_ends_with(s, len, "include/generated/autoconf.h") ||
str_ends_with(s, len, "include/generated/autoksyms.h") ||
- str_ends_with(s, len, "arch/um/include/uml-config.h") ||
- str_ends_with(s, len, "include/linux/kconfig.h") ||
str_ends_with(s, len, ".ver");
}
if len(sys.argv) < 3:
sys.stderr.write("usage: %s [option] file1 file2\n" % sys.argv[0])
sys.stderr.write("The options are:\n")
- sys.stderr.write("-c cateogrize output based on symbole type\n")
+ sys.stderr.write("-c categorize output based on symbol type\n")
sys.stderr.write("-d Show delta of Data Section\n")
sys.stderr.write("-t Show delta of text Section\n")
sys.exit(-1)
static struct pernet_operations selinux_net_ops = {
.init = selinux_nf_register,
.exit = selinux_nf_unregister,
- .async = true,
};
static int __init selinux_nf_ip_init(void)
{
struct net *net;
- rtnl_lock();
+ down_read(&net_rwsem);
for_each_net(net)
rt_genid_bump_all(net);
- rtnl_unlock();
+ up_read(&net_rwsem);
}
#else
static inline int selinux_xfrm_enabled(void)
static struct pernet_operations smack_net_ops = {
.init = smack_nf_register,
.exit = smack_nf_unregister,
- .async = true,
};
static int __init smack_nf_ip_init(void)
return -ENOMEM;
_snd_pcm_hw_params_any(params);
err = snd_pcm_hw_refine(substream, params);
- format_mask = hw_param_mask_c(params, SNDRV_PCM_HW_PARAM_FORMAT);
- kfree(params);
if (err < 0)
- return err;
+ goto error;
+ format_mask = hw_param_mask_c(params, SNDRV_PCM_HW_PARAM_FORMAT);
for (fmt = 0; fmt < 32; ++fmt) {
if (snd_mask_test(format_mask, fmt)) {
int f = snd_pcm_oss_format_to(fmt);
formats |= f;
}
}
- return formats;
+
+ error:
+ kfree(params);
+ return err < 0 ? err : formats;
}
static int snd_pcm_oss_set_format(struct snd_pcm_oss_file *pcm_oss_file, int format)
if (!client)
return 0;
- snd_seq_delete_all_ports(client);
- snd_seq_queue_client_leave(client->number);
spin_lock_irqsave(&clients_lock, flags);
clienttablock[client->number] = 1;
clienttab[client->number] = NULL;
spin_unlock_irqrestore(&clients_lock, flags);
+ snd_seq_delete_all_ports(client);
+ snd_seq_queue_client_leave(client->number);
snd_use_lock_sync(&client->use_lock);
snd_seq_queue_client_termination(client->number);
if (client->pool)
static int snd_seq_client_enqueue_event(struct snd_seq_client *client,
struct snd_seq_event *event,
struct file *file, int blocking,
- int atomic, int hop)
+ int atomic, int hop,
+ struct mutex *mutexp)
{
struct snd_seq_event_cell *cell;
int err;
return -ENXIO; /* queue is not allocated */
/* allocate an event cell */
- err = snd_seq_event_dup(client->pool, event, &cell, !blocking || atomic, file);
+ err = snd_seq_event_dup(client->pool, event, &cell, !blocking || atomic,
+ file, mutexp);
if (err < 0)
return err;
return -ENXIO;
/* allocate the pool now if the pool is not allocated yet */
+ mutex_lock(&client->ioctl_mutex);
if (client->pool->size > 0 && !snd_seq_write_pool_allocated(client)) {
- mutex_lock(&client->ioctl_mutex);
err = snd_seq_pool_init(client->pool);
- mutex_unlock(&client->ioctl_mutex);
if (err < 0)
- return -ENOMEM;
+ goto out;
}
/* only process whole events */
/* ok, enqueue it */
err = snd_seq_client_enqueue_event(client, &event, file,
!(file->f_flags & O_NONBLOCK),
- 0, 0);
+ 0, 0, &client->ioctl_mutex);
if (err < 0)
break;
written += len;
}
+ out:
+ mutex_unlock(&client->ioctl_mutex);
return written ? written : err;
}
(! snd_seq_write_pool_allocated(client) ||
info->output_pool != client->pool->size)) {
if (snd_seq_write_pool_allocated(client)) {
+ /* is the pool in use? */
+ if (atomic_read(&client->pool->counter))
+ return -EBUSY;
/* remove all existing cells */
snd_seq_pool_mark_closing(client->pool);
- snd_seq_queue_client_leave_cells(client->number);
snd_seq_pool_done(client->pool);
}
client->pool->size = info->output_pool;
if (! cptr->accept_output)
result = -EPERM;
else /* send it */
- result = snd_seq_client_enqueue_event(cptr, ev, file, blocking, atomic, hop);
+ result = snd_seq_client_enqueue_event(cptr, ev, file, blocking,
+ atomic, hop, NULL);
snd_seq_client_unlock(cptr);
return result;
return -EINVAL;
snd_use_lock_use(&f->use_lock);
- err = snd_seq_event_dup(f->pool, event, &cell, 1, NULL); /* always non-blocking */
+ err = snd_seq_event_dup(f->pool, event, &cell, 1, NULL, NULL); /* always non-blocking */
if (err < 0) {
if ((err == -ENOMEM) || (err == -EAGAIN))
atomic_inc(&f->overflow);
*/
static int snd_seq_cell_alloc(struct snd_seq_pool *pool,
struct snd_seq_event_cell **cellp,
- int nonblock, struct file *file)
+ int nonblock, struct file *file,
+ struct mutex *mutexp)
{
struct snd_seq_event_cell *cell;
unsigned long flags;
set_current_state(TASK_INTERRUPTIBLE);
add_wait_queue(&pool->output_sleep, &wait);
spin_unlock_irq(&pool->lock);
+ if (mutexp)
+ mutex_unlock(mutexp);
schedule();
+ if (mutexp)
+ mutex_lock(mutexp);
spin_lock_irq(&pool->lock);
remove_wait_queue(&pool->output_sleep, &wait);
/* interrupted? */
*/
int snd_seq_event_dup(struct snd_seq_pool *pool, struct snd_seq_event *event,
struct snd_seq_event_cell **cellp, int nonblock,
- struct file *file)
+ struct file *file, struct mutex *mutexp)
{
int ncells, err;
unsigned int extlen;
if (ncells >= pool->total_elements)
return -ENOMEM;
- err = snd_seq_cell_alloc(pool, &cell, nonblock, file);
+ err = snd_seq_cell_alloc(pool, &cell, nonblock, file, mutexp);
if (err < 0)
return err;
int size = sizeof(struct snd_seq_event);
if (len < size)
size = len;
- err = snd_seq_cell_alloc(pool, &tmp, nonblock, file);
+ err = snd_seq_cell_alloc(pool, &tmp, nonblock, file,
+ mutexp);
if (err < 0)
goto __error;
if (cell->event.data.ext.ptr == NULL)
void snd_seq_cell_free(struct snd_seq_event_cell *cell);
int snd_seq_event_dup(struct snd_seq_pool *pool, struct snd_seq_event *event,
- struct snd_seq_event_cell **cellp, int nonblock, struct file *file);
+ struct snd_seq_event_cell **cellp, int nonblock,
+ struct file *file, struct mutex *mutexp);
/* return number of unused (free) cells */
static inline int snd_seq_unused_cells(struct snd_seq_pool *pool)
if (f->cells > 0) {
/* drain prioQ */
while (f->cells > 0)
- snd_seq_cell_free(snd_seq_prioq_cell_out(f));
+ snd_seq_cell_free(snd_seq_prioq_cell_out(f, NULL));
}
kfree(f);
return 0;
}
+/* return 1 if the current time >= event timestamp */
+static int event_is_ready(struct snd_seq_event *ev, void *current_time)
+{
+ if ((ev->flags & SNDRV_SEQ_TIME_STAMP_MASK) == SNDRV_SEQ_TIME_STAMP_TICK)
+ return snd_seq_compare_tick_time(current_time, &ev->time.tick);
+ else
+ return snd_seq_compare_real_time(current_time, &ev->time.time);
+}
+
/* dequeue cell from prioq */
-struct snd_seq_event_cell *snd_seq_prioq_cell_out(struct snd_seq_prioq *f)
+struct snd_seq_event_cell *snd_seq_prioq_cell_out(struct snd_seq_prioq *f,
+ void *current_time)
{
struct snd_seq_event_cell *cell;
unsigned long flags;
spin_lock_irqsave(&f->lock, flags);
cell = f->head;
+ if (cell && current_time && !event_is_ready(&cell->event, current_time))
+ cell = NULL;
if (cell) {
f->head = cell->next;
return f->cells;
}
-
-/* peek at cell at the head of the prioq */
-struct snd_seq_event_cell *snd_seq_prioq_cell_peek(struct snd_seq_prioq * f)
-{
- if (f == NULL) {
- pr_debug("ALSA: seq: snd_seq_prioq_cell_in() called with NULL prioq\n");
- return NULL;
- }
- return f->head;
-}
-
-
static inline int prioq_match(struct snd_seq_event_cell *cell,
int client, int timestamp)
{
int snd_seq_prioq_cell_in(struct snd_seq_prioq *f, struct snd_seq_event_cell *cell);
/* dequeue cell from prioq */
-struct snd_seq_event_cell *snd_seq_prioq_cell_out(struct snd_seq_prioq *f);
+struct snd_seq_event_cell *snd_seq_prioq_cell_out(struct snd_seq_prioq *f,
+ void *current_time);
/* return number of events available in prioq */
int snd_seq_prioq_avail(struct snd_seq_prioq *f);
-/* peek at cell at the head of the prioq */
-struct snd_seq_event_cell *snd_seq_prioq_cell_peek(struct snd_seq_prioq *f);
-
/* client left queue */
void snd_seq_prioq_leave(struct snd_seq_prioq *f, int client, int timestamp);
__again:
/* Process tick queue... */
- while ((cell = snd_seq_prioq_cell_peek(q->tickq)) != NULL) {
- if (snd_seq_compare_tick_time(&q->timer->tick.cur_tick,
- &cell->event.time.tick)) {
- cell = snd_seq_prioq_cell_out(q->tickq);
- if (cell)
- snd_seq_dispatch_event(cell, atomic, hop);
- } else {
- /* event remains in the queue */
+ for (;;) {
+ cell = snd_seq_prioq_cell_out(q->tickq,
+ &q->timer->tick.cur_tick);
+ if (!cell)
break;
- }
+ snd_seq_dispatch_event(cell, atomic, hop);
}
-
/* Process time queue... */
- while ((cell = snd_seq_prioq_cell_peek(q->timeq)) != NULL) {
- if (snd_seq_compare_real_time(&q->timer->cur_time,
- &cell->event.time.time)) {
- cell = snd_seq_prioq_cell_out(q->timeq);
- if (cell)
- snd_seq_dispatch_event(cell, atomic, hop);
- } else {
- /* event remains in the queue */
+ for (;;) {
+ cell = snd_seq_prioq_cell_out(q->timeq, &q->timer->cur_time);
+ if (!cell)
break;
- }
+ snd_seq_dispatch_event(cell, atomic, hop);
}
/* free lock */
};
#define param_check_xint param_check_int
-static int power_save = -1;
+static int power_save = CONFIG_SND_HDA_POWER_SAVE_DEFAULT;
module_param(power_save, xint, 0644);
MODULE_PARM_DESC(power_save, "Automatic power-saving timeout "
"(in second, 0 = disable).");
+static bool pm_blacklist = true;
+module_param(pm_blacklist, bool, 0644);
+MODULE_PARM_DESC(pm_blacklist, "Enable power-management blacklist");
+
/* reset the HD-audio controller in power save mode.
* this may give more power-saving, but will take longer time to
* wake up.
val = power_save;
#ifdef CONFIG_PM
- if (val == -1) {
+ if (pm_blacklist) {
const struct snd_pci_quirk *q;
- val = CONFIG_SND_HDA_POWER_SAVE_DEFAULT;
q = snd_pci_quirk_lookup(chip->pci, power_save_blacklist);
if (q && val) {
dev_info(chip->card->dev, "device %04x:%04x is on the power_save blacklist, forcing power_save to 0\n",
SND_PCI_QUIRK(0x1025, 0x054c, "Acer Aspire 3830TG", CXT_FIXUP_ASPIRE_DMIC),
SND_PCI_QUIRK(0x1025, 0x054f, "Acer Aspire 4830T", CXT_FIXUP_ASPIRE_DMIC),
SND_PCI_QUIRK(0x103c, 0x8079, "HP EliteBook 840 G3", CXT_FIXUP_HP_DOCK),
+ SND_PCI_QUIRK(0x103c, 0x807C, "HP EliteBook 820 G3", CXT_FIXUP_HP_DOCK),
+ SND_PCI_QUIRK(0x103c, 0x80FD, "HP ProBook 640 G2", CXT_FIXUP_HP_DOCK),
SND_PCI_QUIRK(0x103c, 0x8174, "HP Spectre x360", CXT_FIXUP_HP_SPECTRE),
SND_PCI_QUIRK(0x103c, 0x8115, "HP Z1 Gen3", CXT_FIXUP_HP_GATE_MIC),
SND_PCI_QUIRK(0x103c, 0x814f, "HP ZBook 15u G3", CXT_FIXUP_MUTE_LED_GPIO),
}
}
+/* disable DAC3 (0x06) selection on NID 0x17 as it has no volume amp control */
+static void alc295_fixup_disable_dac3(struct hda_codec *codec,
+ const struct hda_fixup *fix, int action)
+{
+ if (action == HDA_FIXUP_ACT_PRE_PROBE) {
+ hda_nid_t conn[2] = { 0x02, 0x03 };
+ snd_hda_override_conn_list(codec, 0x17, 2, conn);
+ }
+}
+
/* Hook to update amp GPIO4 for automute */
static void alc280_hp_gpio4_automute_hook(struct hda_codec *codec,
struct hda_jack_callback *jack)
ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY,
ALC255_FIXUP_DELL_SPK_NOISE,
ALC225_FIXUP_DELL1_MIC_NO_PRESENCE,
+ ALC295_FIXUP_DISABLE_DAC3,
ALC280_FIXUP_HP_HEADSET_MIC,
ALC221_FIXUP_HP_FRONT_MIC,
ALC292_FIXUP_TPT460,
ALC233_FIXUP_EAPD_COEF_AND_MIC_NO_PRESENCE,
ALC233_FIXUP_LENOVO_MULTI_CODECS,
ALC294_FIXUP_LENOVO_MIC_LOCATION,
+ ALC225_FIXUP_DELL_WYSE_MIC_NO_PRESENCE,
ALC700_FIXUP_INTEL_REFERENCE,
ALC274_FIXUP_DELL_BIND_DACS,
ALC274_FIXUP_DELL_AIO_LINEOUT_VERB,
ALC298_FIXUP_TPT470_DOCK,
+ ALC255_FIXUP_DUMMY_LINEOUT_VERB,
};
static const struct hda_fixup alc269_fixups[] = {
.chained = true,
.chain_id = ALC298_FIXUP_DELL_AIO_MIC_NO_PRESENCE,
},
+ [ALC295_FIXUP_DISABLE_DAC3] = {
+ .type = HDA_FIXUP_FUNC,
+ .v.func = alc295_fixup_disable_dac3,
+ },
[ALC256_FIXUP_DELL_INSPIRON_7559_SUBWOOFER] = {
.type = HDA_FIXUP_PINS,
.v.pins = (const struct hda_pintbl[]) {
{ }
},
},
+ [ALC225_FIXUP_DELL_WYSE_MIC_NO_PRESENCE] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ { 0x16, 0x0101102f }, /* Rear Headset HP */
+ { 0x19, 0x02a1913c }, /* use as Front headset mic, without its own jack detect */
+ { 0x1a, 0x01a19030 }, /* Rear Headset MIC */
+ { 0x1b, 0x02011020 },
+ { }
+ },
+ .chained = true,
+ .chain_id = ALC269_FIXUP_HEADSET_MODE_NO_HP_MIC
+ },
[ALC700_FIXUP_INTEL_REFERENCE] = {
.type = HDA_FIXUP_VERBS,
.v.verbs = (const struct hda_verb[]) {
.chained = true,
.chain_id = ALC293_FIXUP_LENOVO_SPK_NOISE
},
+ [ALC255_FIXUP_DUMMY_LINEOUT_VERB] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ { 0x14, 0x0201101f },
+ { }
+ },
+ .chained = true,
+ .chain_id = ALC255_FIXUP_DELL1_MIC_NO_PRESENCE
+ },
};
static const struct snd_pci_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1028, 0x0725, "Dell Inspiron 3162", ALC255_FIXUP_DELL_SPK_NOISE),
SND_PCI_QUIRK(0x1028, 0x075b, "Dell XPS 13 9360", ALC256_FIXUP_DELL_XPS_13_HEADPHONE_NOISE),
SND_PCI_QUIRK(0x1028, 0x075d, "Dell AIO", ALC298_FIXUP_SPK_VOLUME),
+ SND_PCI_QUIRK(0x1028, 0x07b0, "Dell Precision 7520", ALC295_FIXUP_DISABLE_DAC3),
SND_PCI_QUIRK(0x1028, 0x0798, "Dell Inspiron 17 7000 Gaming", ALC256_FIXUP_DELL_INSPIRON_7559_SUBWOOFER),
+ SND_PCI_QUIRK(0x1028, 0x080c, "Dell WYSE", ALC225_FIXUP_DELL_WYSE_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x082a, "Dell XPS 13 9360", ALC256_FIXUP_DELL_XPS_13_HEADPHONE_NOISE),
SND_PCI_QUIRK(0x1028, 0x084b, "Dell", ALC274_FIXUP_DELL_AIO_LINEOUT_VERB),
SND_PCI_QUIRK(0x1028, 0x084e, "Dell", ALC274_FIXUP_DELL_AIO_LINEOUT_VERB),
+ SND_PCI_QUIRK(0x1028, 0x0873, "Dell Precision 3930", ALC255_FIXUP_DUMMY_LINEOUT_VERB),
SND_PCI_QUIRK(0x1028, 0x164a, "Dell", ALC293_FIXUP_DELL1_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x1028, 0x164b, "Dell", ALC293_FIXUP_DELL1_MIC_NO_PRESENCE),
SND_PCI_QUIRK(0x103c, 0x1586, "HP", ALC269_FIXUP_HP_MUTE_LED_MIC2),
SND_PCI_QUIRK(0x17aa, 0x2245, "Thinkpad T470", ALC298_FIXUP_TPT470_DOCK),
SND_PCI_QUIRK(0x17aa, 0x2246, "Thinkpad", ALC298_FIXUP_TPT470_DOCK),
SND_PCI_QUIRK(0x17aa, 0x2247, "Thinkpad", ALC298_FIXUP_TPT470_DOCK),
+ SND_PCI_QUIRK(0x17aa, 0x2249, "Thinkpad", ALC292_FIXUP_TPT460),
SND_PCI_QUIRK(0x17aa, 0x224b, "Thinkpad", ALC298_FIXUP_TPT470_DOCK),
SND_PCI_QUIRK(0x17aa, 0x224c, "Thinkpad", ALC298_FIXUP_TPT470_DOCK),
SND_PCI_QUIRK(0x17aa, 0x224d, "Thinkpad", ALC298_FIXUP_TPT470_DOCK),
+ SND_PCI_QUIRK(0x17aa, 0x225d, "Thinkpad T480", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x17aa, 0x30bb, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY),
SND_PCI_QUIRK(0x17aa, 0x30e2, "ThinkCentre AIO", ALC233_FIXUP_LENOVO_LINE2_MIC_HOTKEY),
SND_PCI_QUIRK(0x17aa, 0x310c, "ThinkCentre Station", ALC294_FIXUP_LENOVO_MIC_LOCATION),
{0x12, 0x90a60120},
{0x14, 0x90170110},
{0x21, 0x0321101f}),
- SND_HDA_PIN_QUIRK(0x10ec0289, 0x1028, "Dell", ALC225_FIXUP_DELL1_MIC_NO_PRESENCE,
+ SND_HDA_PIN_QUIRK(0x10ec0289, 0x1028, "Dell", ALC269_FIXUP_DELL4_MIC_NO_PRESENCE,
{0x12, 0xb7a60130},
{0x14, 0x90170110},
{0x21, 0x04211020}),
for (bank = 1; bank < 48; bank++)
acp_set_sram_bank_state(acp_mmio, bank, false);
}
-
- /* Stoney supports 16bit resolution */
- if (asic_type == CHIP_STONEY) {
- val = acp_reg_read(acp_mmio, mmACP_I2S_16BIT_RESOLUTION_EN);
- val |= 0x03;
- acp_reg_write(val, acp_mmio, mmACP_I2S_16BIT_RESOLUTION_EN);
- }
return 0;
}
{
int status;
uint64_t size;
+ u32 val = 0;
struct page *pg;
struct snd_pcm_runtime *runtime;
struct audio_substream_data *rtd;
if (WARN_ON(!rtd))
return -EINVAL;
+ if (adata->asic_type == CHIP_STONEY) {
+ val = acp_reg_read(adata->acp_mmio, mmACP_I2S_16BIT_RESOLUTION_EN);
+ if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK)
+ val |= ACP_I2S_SP_16BIT_RESOLUTION_EN;
+ else
+ val |= ACP_I2S_MIC_16BIT_RESOLUTION_EN;
+ acp_reg_write(val, adata->acp_mmio, mmACP_I2S_16BIT_RESOLUTION_EN);
+ }
size = params_buffer_bytes(params);
status = snd_pcm_lib_malloc_pages(substream, size);
if (status < 0)
#define CAPTURE_END_DMA_DESCR_CH15 7
#define mmACP_I2S_16BIT_RESOLUTION_EN 0x5209
+#define ACP_I2S_MIC_16BIT_RESOLUTION_EN 0x01
+#define ACP_I2S_SP_16BIT_RESOLUTION_EN 0x02
enum acp_dma_priority_level {
/* 0x0 Specifies the DMA channel is given normal priority */
ACP_DMA_PRIORITY_LEVEL_NORMAL = 0x0,
static int hdmi_codec_remove(struct platform_device *pdev)
{
- struct device *dev = &pdev->dev;
- struct hdmi_codec_priv *hcp;
-
- hcp = dev_get_drvdata(dev);
- kfree(hcp->chmap_info);
- snd_soc_unregister_codec(dev);
+ snd_soc_unregister_codec(&pdev->dev);
return 0;
}
.num_reg_defaults = ARRAY_SIZE(rt5651_reg),
.ranges = rt5651_ranges,
.num_ranges = ARRAY_SIZE(rt5651_ranges),
+ .use_single_rw = true,
};
#if defined(CONFIG_OF)
static int sgtl5000_digital_mute(struct snd_soc_dai *codec_dai, int mute)
{
struct snd_soc_codec *codec = codec_dai->codec;
- u16 adcdac_ctrl = SGTL5000_DAC_MUTE_LEFT | SGTL5000_DAC_MUTE_RIGHT;
+ u16 i2s_pwr = SGTL5000_I2S_IN_POWERUP;
- snd_soc_update_bits(codec, SGTL5000_CHIP_ADCDAC_CTRL,
- adcdac_ctrl, mute ? adcdac_ctrl : 0);
+ /*
+ * During 'digital mute' do not mute DAC
+ * because LINE_IN would be muted aswell. We want to mute
+ * only I2S block - this can be done by powering it off
+ */
+ snd_soc_update_bits(codec, SGTL5000_CHIP_DIG_POWER,
+ i2s_pwr, mute ? 0 : i2s_pwr);
return 0;
}
static int sgtl5000_set_bias_level(struct snd_soc_codec *codec,
enum snd_soc_bias_level level)
{
+ struct sgtl5000_priv *sgtl = snd_soc_codec_get_drvdata(codec);
+ int ret;
+
switch (level) {
case SND_SOC_BIAS_ON:
case SND_SOC_BIAS_PREPARE:
case SND_SOC_BIAS_STANDBY:
+ regcache_cache_only(sgtl->regmap, false);
+ ret = regcache_sync(sgtl->regmap);
+ if (ret) {
+ regcache_cache_only(sgtl->regmap, true);
+ return ret;
+ }
+
snd_soc_update_bits(codec, SGTL5000_CHIP_ANA_POWER,
SGTL5000_REFTOP_POWERUP,
SGTL5000_REFTOP_POWERUP);
break;
case SND_SOC_BIAS_OFF:
+ regcache_cache_only(sgtl->regmap, true);
snd_soc_update_bits(codec, SGTL5000_CHIP_ANA_POWER,
SGTL5000_REFTOP_POWERUP, 0);
break;
*/
snd_soc_write(codec, SGTL5000_DAP_CTRL, 0);
+ /* Unmute DAC after start */
+ snd_soc_update_bits(codec, SGTL5000_CHIP_ADCDAC_CTRL,
+ SGTL5000_DAC_MUTE_LEFT | SGTL5000_DAC_MUTE_RIGHT, 0);
+
return 0;
err:
kcontrol->put = wm_coeff_put_acked;
break;
default:
- kcontrol->get = wm_coeff_get;
- kcontrol->put = wm_coeff_put;
-
- ctl->bytes_ext.max = ctl->len;
- ctl->bytes_ext.get = wm_coeff_tlv_get;
- ctl->bytes_ext.put = wm_coeff_tlv_put;
+ if (kcontrol->access & SNDRV_CTL_ELEM_ACCESS_TLV_CALLBACK) {
+ ctl->bytes_ext.max = ctl->len;
+ ctl->bytes_ext.get = wm_coeff_tlv_get;
+ ctl->bytes_ext.put = wm_coeff_tlv_put;
+ } else {
+ kcontrol->get = wm_coeff_get;
+ kcontrol->put = wm_coeff_put;
+ }
break;
}
#define SUN8I_I2S_CHAN_CFG_REG 0x30
#define SUN8I_I2S_CHAN_CFG_RX_SLOT_NUM_MASK GENMASK(6, 4)
-#define SUN8I_I2S_CHAN_CFG_RX_SLOT_NUM(chan) (chan - 1)
+#define SUN8I_I2S_CHAN_CFG_RX_SLOT_NUM(chan) ((chan - 1) << 4)
#define SUN8I_I2S_CHAN_CFG_TX_SLOT_NUM_MASK GENMASK(2, 0)
#define SUN8I_I2S_CHAN_CFG_TX_SLOT_NUM(chan) (chan - 1)
#define X86_FEATURE_SEV ( 7*32+20) /* AMD Secure Encrypted Virtualization */
#define X86_FEATURE_USE_IBPB ( 7*32+21) /* "" Indirect Branch Prediction Barrier enabled */
+#define X86_FEATURE_USE_IBRS_FW ( 7*32+22) /* "" Use IBRS during runtime firmware calls */
/* Virtualization flags: Linux defined, word 8 */
#define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* Intel TPR Shadow */
# SPDX-License-Identifier: GPL-2.0
-prefix = /usr
+include ../scripts/Makefile.include
+
+prefix ?= /usr/local
CC = gcc
LEX = flex
YACC = bison
MAKE = make
+INSTALL ?= install
CFLAGS += -Wall -O2
-CFLAGS += -D__EXPORTED_HEADERS__ -I../../include/uapi -I../../include
+CFLAGS += -D__EXPORTED_HEADERS__ -I$(srctree)/include/uapi -I$(srctree)/include
ifeq ($(srctree),)
srctree := $(patsubst %/,%,$(dir $(CURDIR)))
srctree := $(patsubst %/,%,$(dir $(srctree)))
endif
+ifeq ($(V),1)
+ Q =
+else
+ Q = @
+endif
+
FEATURE_USER = .bpf
FEATURE_TESTS = libbfd disassembler-four-args
FEATURE_DISPLAY = libbfd disassembler-four-args
CFLAGS += -DDISASM_FOUR_ARGS_SIGNATURE
endif
-%.yacc.c: %.y
- $(YACC) -o $@ -d $<
+$(OUTPUT)%.yacc.c: $(srctree)/tools/bpf/%.y
+ $(QUIET_BISON)$(YACC) -o $@ -d $<
-%.lex.c: %.l
- $(LEX) -o $@ $<
+$(OUTPUT)%.lex.c: $(srctree)/tools/bpf/%.l
+ $(QUIET_FLEX)$(LEX) -o $@ $<
-all: bpf_jit_disasm bpf_dbg bpf_asm bpftool
+$(OUTPUT)%.o: $(srctree)/tools/bpf/%.c
+ $(QUIET_CC)$(COMPILE.c) -o $@ $<
-bpf_jit_disasm : CFLAGS += -DPACKAGE='bpf_jit_disasm'
-bpf_jit_disasm : LDLIBS = -lopcodes -lbfd -ldl
-bpf_jit_disasm : bpf_jit_disasm.o
+$(OUTPUT)%.yacc.o: $(OUTPUT)%.yacc.c
+ $(QUIET_CC)$(COMPILE.c) -o $@ $<
+$(OUTPUT)%.lex.o: $(OUTPUT)%.lex.c
+ $(QUIET_CC)$(COMPILE.c) -o $@ $<
-bpf_dbg : LDLIBS = -lreadline
-bpf_dbg : bpf_dbg.o
+PROGS = $(OUTPUT)bpf_jit_disasm $(OUTPUT)bpf_dbg $(OUTPUT)bpf_asm
-bpf_asm : LDLIBS =
-bpf_asm : bpf_asm.o bpf_exp.yacc.o bpf_exp.lex.o
-bpf_exp.lex.o : bpf_exp.yacc.c
+all: $(PROGS) bpftool
-clean: bpftool_clean
- rm -rf *.o bpf_jit_disasm bpf_dbg bpf_asm bpf_exp.yacc.* bpf_exp.lex.*
+$(OUTPUT)bpf_jit_disasm: CFLAGS += -DPACKAGE='bpf_jit_disasm'
+$(OUTPUT)bpf_jit_disasm: $(OUTPUT)bpf_jit_disasm.o
+ $(QUIET_LINK)$(CC) $(CFLAGS) -o $@ $^ -lopcodes -lbfd -ldl
-install: bpftool_install
- install bpf_jit_disasm $(prefix)/bin/bpf_jit_disasm
- install bpf_dbg $(prefix)/bin/bpf_dbg
- install bpf_asm $(prefix)/bin/bpf_asm
+$(OUTPUT)bpf_dbg: $(OUTPUT)bpf_dbg.o
+ $(QUIET_LINK)$(CC) $(CFLAGS) -o $@ $^ -lreadline
+
+$(OUTPUT)bpf_asm: $(OUTPUT)bpf_asm.o $(OUTPUT)bpf_exp.yacc.o $(OUTPUT)bpf_exp.lex.o
+ $(QUIET_LINK)$(CC) $(CFLAGS) -o $@ $^
+
+$(OUTPUT)bpf_exp.lex.c: $(OUTPUT)bpf_exp.yacc.c
+
+clean: bpftool_clean
+ $(call QUIET_CLEAN, bpf-progs)
+ $(Q)rm -rf $(OUTPUT)*.o $(OUTPUT)bpf_jit_disasm $(OUTPUT)bpf_dbg \
+ $(OUTPUT)bpf_asm $(OUTPUT)bpf_exp.yacc.* $(OUTPUT)bpf_exp.lex.*
+ $(call QUIET_CLEAN, core-gen)
+ $(Q)rm -f $(OUTPUT)FEATURE-DUMP.bpf
+
+install: $(PROGS) bpftool_install
+ $(call QUIET_INSTALL, bpf_jit_disasm)
+ $(Q)$(INSTALL) -m 0755 -d $(DESTDIR)$(prefix)/bin
+ $(Q)$(INSTALL) $(OUTPUT)bpf_jit_disasm $(DESTDIR)$(prefix)/bin/bpf_jit_disasm
+ $(call QUIET_INSTALL, bpf_dbg)
+ $(Q)$(INSTALL) $(OUTPUT)bpf_dbg $(DESTDIR)$(prefix)/bin/bpf_dbg
+ $(call QUIET_INSTALL, bpf_asm)
+ $(Q)$(INSTALL) $(OUTPUT)bpf_asm $(DESTDIR)$(prefix)/bin/bpf_asm
bpftool:
- $(MAKE) -C bpftool
+ $(call descend,bpftool)
bpftool_install:
- $(MAKE) -C bpftool install
+ $(call descend,bpftool,install)
bpftool_clean:
- $(MAKE) -C bpftool clean
+ $(call descend,bpftool,clean)
-.PHONY: bpftool FORCE
+.PHONY: all install clean bpftool bpftool_install bpftool_clean
CC = gcc
CFLAGS += -O2
-CFLAGS += -W -Wall -Wextra -Wno-unused-parameter -Wshadow
+CFLAGS += -W -Wall -Wextra -Wno-unused-parameter -Wshadow -Wno-missing-field-initializers
CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ -I$(srctree)/tools/include/uapi -I$(srctree)/tools/include -I$(srctree)/tools/lib/bpf -I$(srctree)/kernel/bpf/
CFLAGS += -DBPFTOOL_VERSION='"$(BPFTOOL_VERSION)"'
LIBS = -lelf -lbfd -lopcodes $(LIBBPF)
CFLAGS += -DDISASM_FOUR_ARGS_SIGNATURE
endif
-include $(wildcard *.d)
+include $(wildcard $(OUTPUT)*.d)
all: $(OUTPUT)bpftool
clean: $(LIBBPF)-clean
$(call QUIET_CLEAN, bpftool)
$(Q)$(RM) $(OUTPUT)bpftool $(OUTPUT)*.o $(OUTPUT)*.d
+ $(call QUIET_CLEAN, core-gen)
+ $(Q)$(RM) $(OUTPUT)FEATURE-DUMP.bpftool
install: $(OUTPUT)bpftool
$(call QUIET_INSTALL, bpftool)
#include "main.h"
+#ifndef BPF_FS_MAGIC
+#define BPF_FS_MAGIC 0xcafe4a11
+#endif
+
void p_err(const char *fmt, ...)
{
va_list ap;
unsigned long address_call_base;
struct kernel_sym *sym_mapping;
__u32 sym_count;
- char scratch_buff[SYM_MAX_NAME];
+ char scratch_buff[SYM_MAX_NAME + 8];
};
void kernel_syms_load(struct dump_data *dd);
BPF_PROG_TYPE_SOCK_OPS,
BPF_PROG_TYPE_SK_SKB,
BPF_PROG_TYPE_CGROUP_DEVICE,
+ BPF_PROG_TYPE_SK_MSG,
};
enum bpf_attach_type {
BPF_SK_SKB_STREAM_PARSER,
BPF_SK_SKB_STREAM_VERDICT,
BPF_CGROUP_DEVICE,
+ BPF_SK_MSG_VERDICT,
__MAX_BPF_ATTACH_TYPE
};
#define BPF_F_RDONLY (1U << 3)
#define BPF_F_WRONLY (1U << 4)
+/* Flag for stack_map, store build_id+offset instead of pointer */
+#define BPF_F_STACK_BUILD_ID (1U << 5)
+
+enum bpf_stack_build_id_status {
+ /* user space need an empty entry to identify end of a trace */
+ BPF_STACK_BUILD_ID_EMPTY = 0,
+ /* with valid build_id and offset */
+ BPF_STACK_BUILD_ID_VALID = 1,
+ /* couldn't get build_id, fallback to ip */
+ BPF_STACK_BUILD_ID_IP = 2,
+};
+
+#define BPF_BUILD_ID_SIZE 20
+struct bpf_stack_build_id {
+ __s32 status;
+ unsigned char build_id[BPF_BUILD_ID_SIZE];
+ union {
+ __u64 offset;
+ __u64 ip;
+ };
+};
+
union bpf_attr {
struct { /* anonymous struct used by BPF_MAP_CREATE command */
__u32 map_type; /* one of enum bpf_map_type */
* int bpf_override_return(pt_regs, rc)
* @pt_regs: pointer to struct pt_regs
* @rc: the return value to set
+ *
+ * int bpf_msg_redirect_map(map, key, flags)
+ * Redirect msg to a sock in map using key as a lookup key for the
+ * sock in map.
+ * @map: pointer to sockmap
+ * @key: key to lookup sock in map
+ * @flags: reserved for future use
+ * Return: SK_PASS
+ *
*/
#define __BPF_FUNC_MAPPER(FN) \
FN(unspec), \
FN(perf_prog_read_value), \
FN(getsockopt), \
FN(override_return), \
- FN(sock_ops_cb_flags_set),
+ FN(sock_ops_cb_flags_set), \
+ FN(msg_redirect_map), \
+ FN(msg_apply_bytes), \
+ FN(msg_cork_bytes), \
+ FN(msg_pull_data),
/* integer value in 'imm' field of BPF_CALL instruction selects which helper
* function eBPF program intends to call
SK_PASS,
};
+/* user accessible metadata for SK_MSG packet hook, new fields must
+ * be added to the end of this structure
+ */
+struct sk_msg_md {
+ void *data;
+ void *data_end;
+};
+
#define BPF_TAG_SIZE 8
struct bpf_prog_info {
#define KVM_TRACE_PAUSE __KVM_DEPRECATED_MAIN_0x07
#define KVM_TRACE_DISABLE __KVM_DEPRECATED_MAIN_0x08
#define KVM_GET_EMULATED_CPUID _IOWR(KVMIO, 0x09, struct kvm_cpuid2)
+#define KVM_GET_MSR_FEATURE_INDEX_LIST _IOWR(KVMIO, 0x0a, struct kvm_msr_list)
/*
* Extension capability list.
#define KVM_CAP_S390_AIS_MIGRATION 150
#define KVM_CAP_PPC_GET_CPU_CHAR 151
#define KVM_CAP_S390_BPB 152
+#define KVM_CAP_GET_MSR_FEATURES 153
#ifdef KVM_CAP_IRQ_ROUTING
BPF_PROG_SEC("lwt_xmit", BPF_PROG_TYPE_LWT_XMIT),
BPF_PROG_SEC("sockops", BPF_PROG_TYPE_SOCK_OPS),
BPF_PROG_SEC("sk_skb", BPF_PROG_TYPE_SK_SKB),
+ BPF_PROG_SEC("sk_msg", BPF_PROG_TYPE_SK_MSG),
};
#undef BPF_PROG_SEC
static int read_retpoline_hints(struct objtool_file *file)
{
- struct section *sec, *relasec;
+ struct section *sec;
struct instruction *insn;
struct rela *rela;
- int i;
- sec = find_section_by_name(file->elf, ".discard.retpoline_safe");
+ sec = find_section_by_name(file->elf, ".rela.discard.retpoline_safe");
if (!sec)
return 0;
- relasec = sec->rela;
- if (!relasec) {
- WARN("missing .rela.discard.retpoline_safe section");
- return -1;
- }
-
- if (sec->len % sizeof(unsigned long)) {
- WARN("retpoline_safe size mismatch: %d %ld", sec->len, sizeof(unsigned long));
- return -1;
- }
-
- for (i = 0; i < sec->len / sizeof(unsigned long); i++) {
- rela = find_rela_by_dest(sec, i * sizeof(unsigned long));
- if (!rela) {
- WARN("can't find rela for retpoline_safe[%d]", i);
+ list_for_each_entry(rela, &sec->rela_list, list) {
+ if (rela->sym->type != STT_SECTION) {
+ WARN("unexpected relocation symbol type in %s", sec->name);
return -1;
}
insn = find_insn(file, rela->sym->sec, rela->addend);
if (!insn) {
- WARN("can't find insn for retpoline_safe[%d]", i);
+ WARN("bad .discard.retpoline_safe entry");
return -1;
}
if (insn->type != INSN_JUMP_DYNAMIC &&
insn->type != INSN_CALL_DYNAMIC) {
- WARN_FUNC("retpoline_safe hint not a indirect jump/call",
+ WARN_FUNC("retpoline_safe hint not an indirect jump/call",
insn->sec, insn->offset);
return -1;
}
SYNOPSIS
--------
[verse]
-'perf kallsyms <options> symbol_name[,symbol_name...]'
+'perf kallsyms' [<options>] symbol_name[,symbol_name...]
DESCRIPTION
-----------
}
}
+ /*
+ * If we have just single event and are sending data
+ * through pipe, we need to force the ids allocation,
+ * because we synthesize event name through the pipe
+ * and need the id for that.
+ */
+ if (data->is_pipe && rec->evlist->nr_entries == 1)
+ rec->opts.sample_id = true;
+
if (record__open(rec) != 0) {
err = -1;
goto out_child;
char buf[64], *vals, *ends;
if (unit == NULL || fmt == NULL) {
- fprintf(out, "%s%s%s%s", csv_sep, csv_sep, csv_sep, csv_sep);
+ fprintf(out, "%s%s", csv_sep, csv_sep);
return;
}
snprintf(buf, sizeof(buf), fmt, val);
evlist__for_each_entry(evlist, counter)
counter->attr.write_backward = false;
opts->overwrite = false;
- ui__warning("fall back to non-overwrite mode\n");
+ pr_debug2("fall back to non-overwrite mode\n");
return 1;
}
bool tail_synthesize;
bool overwrite;
bool ignore_missing_thread;
+ bool sample_id;
unsigned int freq;
unsigned int mmap_pages;
unsigned int auxtrace_mmap_pages;
if (!disasm_line__is_valid_jump(cursor, sym))
return;
+ /*
+ * This first was seen with a gcc function, _cpp_lex_token, that
+ * has the usual jumps:
+ *
+ * │1159e6c: ↓ jne 115aa32 <_cpp_lex_token@@Base+0xf92>
+ *
+ * I.e. jumps to a label inside that function (_cpp_lex_token), and
+ * those works, but also this kind:
+ *
+ * │1159e8b: ↓ jne c469be <cpp_named_operator2name@@Base+0xa72>
+ *
+ * I.e. jumps to another function, outside _cpp_lex_token, which
+ * are not being correctly handled generating as a side effect references
+ * to ab->offset[] entries that are set to NULL, so to make this code
+ * more robust, check that here.
+ *
+ * A proper fix for will be put in place, looking at the function
+ * name right after the '<' token and probably treating this like a
+ * 'call' instruction.
+ */
target = ab->offsets[cursor->ops.target.offset];
+ if (target == NULL) {
+ ui_helpline__printf("WARN: jump target inconsistency, press 'o', ab->offsets[%#x] = NULL\n",
+ cursor->ops.target.offset);
+ return;
+ }
bcursor = browser_line(&cursor->al);
btarget = browser_line(target);
#include "sane_ctype.h"
#include "symbol/kallsyms.h"
+static bool auxtrace__dont_decode(struct perf_session *session)
+{
+ return !session->itrace_synth_opts ||
+ session->itrace_synth_opts->dont_decode;
+}
+
int auxtrace_mmap__mmap(struct auxtrace_mmap *mm,
struct auxtrace_mmap_params *mp,
void *userpg, int fd)
size_t i;
int err;
+ if (auxtrace__dont_decode(session))
+ return 0;
+
list_for_each_entry(auxtrace_index, &session->auxtrace_index, list) {
for (i = 0; i < auxtrace_index->nr; i++) {
ent = &auxtrace_index->entries[i];
return err;
}
-static bool auxtrace__dont_decode(struct perf_session *session)
-{
- return !session->itrace_synth_opts ||
- session->itrace_synth_opts->dont_decode;
-}
-
int perf_event__process_auxtrace_info(struct perf_tool *tool __maybe_unused,
union perf_event *event,
struct perf_session *session)
struct perf_evsel *evsel;
bool use_sample_identifier = false;
bool use_comm_exec;
+ bool sample_id = opts->sample_id;
/*
* Set the evsel leader links before we configure attributes,
* match the id.
*/
use_sample_identifier = perf_can_sample_identifier();
- evlist__for_each_entry(evlist, evsel)
- perf_evsel__set_sample_id(evsel, use_sample_identifier);
+ sample_id = true;
} else if (evlist->nr_entries > 1) {
struct perf_evsel *first = perf_evlist__first(evlist);
use_sample_identifier = perf_can_sample_identifier();
break;
}
+ sample_id = true;
+ }
+
+ if (sample_id) {
evlist__for_each_entry(evlist, evsel)
perf_evsel__set_sample_id(evsel, use_sample_identifier);
}
* States and transits:
*
*
- * OFF--(on)--> READY --(hit)--> HIT
+ * OFF--> ON --> READY --(hit)--> HIT
* ^ |
* | (ready)
* | |
volatile enum {
TRIGGER_ERROR = -2,
TRIGGER_OFF = -1,
- TRIGGER_READY = 0,
- TRIGGER_HIT = 1,
+ TRIGGER_ON = 0,
+ TRIGGER_READY = 1,
+ TRIGGER_HIT = 2,
} state;
const char *name;
};
static inline void trigger_on(struct trigger *t)
{
TRIGGER_WARN_ONCE(t, TRIGGER_OFF);
- t->state = TRIGGER_READY;
+ t->state = TRIGGER_ON;
}
static inline void trigger_ready(struct trigger *t)
CFLAGS += -Wall -O2 -I$(APIDIR) -I$(LIBDIR) -I$(GENDIR) $(GENFLAGS) -I../../../include
LDLIBS += -lcap -lelf -lrt -lpthread
+TEST_CUSTOM_PROGS = $(OUTPUT)/urandom_read
+all: $(TEST_CUSTOM_PROGS)
+
+$(TEST_CUSTOM_PROGS): urandom_read
+
+urandom_read: urandom_read.c
+ $(CC) -o $(TEST_CUSTOM_PROGS) -static $<
+
# Order correspond to 'make run_tests' order
TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test_progs \
test_align test_verifier_log test_dev_cgroup test_tcpbpf_user
test_pkt_md_access.o test_xdp_redirect.o test_xdp_meta.o sockmap_parse_prog.o \
sockmap_verdict_prog.o dev_cgroup.o sample_ret0.o test_tracepoint.o \
test_l4lb_noinline.o test_xdp_noinline.o test_stacktrace_map.o \
- sample_map_ret0.o test_tcpbpf_kern.o
+ sample_map_ret0.o test_tcpbpf_kern.o test_stacktrace_build_id.o \
+ sockmap_tcp_msg_prog.o
# Order correspond to 'make run_tests' order
TEST_PROGS := test_kmod.sh \
$(CLANG) $(CLANG_FLAGS) \
-O2 -target bpf -emit-llvm -c $< -o - | \
$(LLC) -march=bpf -mcpu=$(CPU) -filetype=obj -o $@
+
+EXTRA_CLEAN := $(TEST_CUSTOM_PROGS)
(void *) BPF_FUNC_perf_prog_read_value;
static int (*bpf_override_return)(void *ctx, unsigned long rc) =
(void *) BPF_FUNC_override_return;
+static int (*bpf_msg_redirect_map)(void *ctx, void *map, int key, int flags) =
+ (void *) BPF_FUNC_msg_redirect_map;
+static int (*bpf_msg_apply_bytes)(void *ctx, int len) =
+ (void *) BPF_FUNC_msg_apply_bytes;
+static int (*bpf_msg_cork_bytes)(void *ctx, int len) =
+ (void *) BPF_FUNC_msg_cork_bytes;
+static int (*bpf_msg_pull_data)(void *ctx, int start, int end, int flags) =
+ (void *) BPF_FUNC_msg_pull_data;
/* llvm builtin functions that eBPF C program may use to
* emit BPF_LD_ABS and BPF_LD_IND instructions
(void *) BPF_FUNC_skb_under_cgroup;
static int (*bpf_skb_change_head)(void *, int len, int flags) =
(void *) BPF_FUNC_skb_change_head;
+static int (*bpf_skb_pull_data)(void *, int len) =
+ (void *) BPF_FUNC_skb_pull_data;
/* Scan the ARCH passed in from ARCH env variable (see Makefile) */
#if defined(__TARGET_ARCH_x86)
__u32 lport = skb->local_port;
__u32 rport = skb->remote_port;
__u8 *d = data;
+ __u32 len = (__u32) data_end - (__u32) data;
+ int err;
- if (data + 10 > data_end)
- return skb->len;
+ if (data + 10 > data_end) {
+ err = bpf_skb_pull_data(skb, 10);
+ if (err)
+ return SK_DROP;
+
+ data_end = (void *)(long)skb->data_end;
+ data = (void *)(long)skb->data;
+ if (data + 10 > data_end)
+ return SK_DROP;
+ }
/* This write/read is a bit pointless but tests the verifier and
* strparser handler for read/write pkt data and access into sk
* fields.
*/
+ d = data;
d[7] = 1;
return skb->len;
}
--- /dev/null
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+#include "bpf_util.h"
+#include "bpf_endian.h"
+
+int _version SEC("version") = 1;
+
+#define bpf_printk(fmt, ...) \
+({ \
+ char ____fmt[] = fmt; \
+ bpf_trace_printk(____fmt, sizeof(____fmt), \
+ ##__VA_ARGS__); \
+})
+
+SEC("sk_msg1")
+int bpf_prog1(struct sk_msg_md *msg)
+{
+ void *data_end = (void *)(long) msg->data_end;
+ void *data = (void *)(long) msg->data;
+
+ char *d;
+
+ if (data + 8 > data_end)
+ return SK_DROP;
+
+ bpf_printk("data length %i\n", (__u64)msg->data_end - (__u64)msg->data);
+ d = (char *)data;
+ bpf_printk("hello sendmsg hook %i %i\n", d[0], d[1]);
+
+ return SK_PASS;
+}
+
+char _license[] SEC("license") = "GPL";
.max_entries = 20,
};
+struct bpf_map_def SEC("maps") sock_map_msg = {
+ .type = BPF_MAP_TYPE_SOCKMAP,
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .max_entries = 20,
+};
+
struct bpf_map_def SEC("maps") sock_map_break = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
#include <linux/err.h>
#define SOCKMAP_PARSE_PROG "./sockmap_parse_prog.o"
#define SOCKMAP_VERDICT_PROG "./sockmap_verdict_prog.o"
+#define SOCKMAP_TCP_MSG_PROG "./sockmap_tcp_msg_prog.o"
static void test_sockmap(int tasks, void *data)
{
- int one = 1, map_fd_rx = 0, map_fd_tx = 0, map_fd_break, s, sc, rc;
- struct bpf_map *bpf_map_rx, *bpf_map_tx, *bpf_map_break;
+ struct bpf_map *bpf_map_rx, *bpf_map_tx, *bpf_map_msg, *bpf_map_break;
+ int map_fd_msg = 0, map_fd_rx = 0, map_fd_tx = 0, map_fd_break;
int ports[] = {50200, 50201, 50202, 50204};
int err, i, fd, udp, sfd[6] = {0xdeadbeef};
u8 buf[20] = {0x0, 0x5, 0x3, 0x2, 0x1, 0x0};
- int parse_prog, verdict_prog;
+ int parse_prog, verdict_prog, msg_prog;
struct sockaddr_in addr;
+ int one = 1, s, sc, rc;
struct bpf_object *obj;
struct timeval to;
__u32 key, value;
goto out_sockmap;
}
+ err = bpf_prog_attach(-1, fd, BPF_SK_MSG_VERDICT, 0);
+ if (!err) {
+ printf("Failed invalid msg verdict prog attach\n");
+ goto out_sockmap;
+ }
+
err = bpf_prog_attach(-1, fd, __MAX_BPF_ATTACH_TYPE, 0);
if (!err) {
printf("Failed unknown prog attach\n");
goto out_sockmap;
}
+ err = bpf_prog_detach(fd, BPF_SK_MSG_VERDICT);
+ if (err) {
+ printf("Failed empty msg verdict prog detach\n");
+ goto out_sockmap;
+ }
+
err = bpf_prog_detach(fd, __MAX_BPF_ATTACH_TYPE);
if (!err) {
printf("Detach invalid prog successful\n");
goto out_sockmap;
}
+ err = bpf_prog_load(SOCKMAP_TCP_MSG_PROG,
+ BPF_PROG_TYPE_SK_MSG, &obj, &msg_prog);
+ if (err) {
+ printf("Failed to load SK_SKB msg prog\n");
+ goto out_sockmap;
+ }
+
err = bpf_prog_load(SOCKMAP_VERDICT_PROG,
BPF_PROG_TYPE_SK_SKB, &obj, &verdict_prog);
if (err) {
map_fd_rx = bpf_map__fd(bpf_map_rx);
if (map_fd_rx < 0) {
- printf("Failed to get map fd\n");
+ printf("Failed to get map rx fd\n");
goto out_sockmap;
}
goto out_sockmap;
}
+ bpf_map_msg = bpf_object__find_map_by_name(obj, "sock_map_msg");
+ if (IS_ERR(bpf_map_msg)) {
+ printf("Failed to load map msg from msg_verdict prog\n");
+ goto out_sockmap;
+ }
+
+ map_fd_msg = bpf_map__fd(bpf_map_msg);
+ if (map_fd_msg < 0) {
+ printf("Failed to get map msg fd\n");
+ goto out_sockmap;
+ }
+
bpf_map_break = bpf_object__find_map_by_name(obj, "sock_map_break");
if (IS_ERR(bpf_map_break)) {
printf("Failed to load map tx from verdict prog\n");
goto out_sockmap;
}
+ err = bpf_prog_attach(msg_prog, map_fd_msg, BPF_SK_MSG_VERDICT, 0);
+ if (err) {
+ printf("Failed msg verdict bpf prog attach\n");
+ goto out_sockmap;
+ }
+
err = bpf_prog_attach(verdict_prog, map_fd_rx,
__MAX_BPF_ATTACH_TYPE, 0);
if (!err) {
}
}
+ /* Put sfd[2] (sending fd below) into msg map to test sendmsg bpf */
+ i = 0;
+ err = bpf_map_update_elem(map_fd_msg, &i, &sfd[2], BPF_ANY);
+ if (err) {
+ printf("Failed map_fd_msg update sockmap %i\n", err);
+ goto out_sockmap;
+ }
+
/* Test map send/recv */
for (i = 0; i < 2; i++) {
buf[0] = i;
static int compare_map_keys(int map1_fd, int map2_fd)
{
__u32 key, next_key;
- char val_buf[PERF_MAX_STACK_DEPTH * sizeof(__u64)];
+ char val_buf[PERF_MAX_STACK_DEPTH *
+ sizeof(struct bpf_stack_build_id)];
int err;
err = bpf_map_get_next_key(map1_fd, NULL, &key);
return;
}
+static int extract_build_id(char *build_id, size_t size)
+{
+ FILE *fp;
+ char *line = NULL;
+ size_t len = 0;
+
+ fp = popen("readelf -n ./urandom_read | grep 'Build ID'", "r");
+ if (fp == NULL)
+ return -1;
+
+ if (getline(&line, &len, fp) == -1)
+ goto err;
+ fclose(fp);
+
+ if (len > size)
+ len = size;
+ memcpy(build_id, line, len);
+ build_id[len] = '\0';
+ return 0;
+err:
+ fclose(fp);
+ return -1;
+}
+
+static void test_stacktrace_build_id(void)
+{
+ int control_map_fd, stackid_hmap_fd, stackmap_fd;
+ const char *file = "./test_stacktrace_build_id.o";
+ int bytes, efd, err, pmu_fd, prog_fd;
+ struct perf_event_attr attr = {};
+ __u32 key, previous_key, val, duration = 0;
+ struct bpf_object *obj;
+ char buf[256];
+ int i, j;
+ struct bpf_stack_build_id id_offs[PERF_MAX_STACK_DEPTH];
+ int build_id_matches = 0;
+
+ err = bpf_prog_load(file, BPF_PROG_TYPE_TRACEPOINT, &obj, &prog_fd);
+ if (CHECK(err, "prog_load", "err %d errno %d\n", err, errno))
+ goto out;
+
+ /* Get the ID for the sched/sched_switch tracepoint */
+ snprintf(buf, sizeof(buf),
+ "/sys/kernel/debug/tracing/events/random/urandom_read/id");
+ efd = open(buf, O_RDONLY, 0);
+ if (CHECK(efd < 0, "open", "err %d errno %d\n", efd, errno))
+ goto close_prog;
+
+ bytes = read(efd, buf, sizeof(buf));
+ close(efd);
+ if (CHECK(bytes <= 0 || bytes >= sizeof(buf),
+ "read", "bytes %d errno %d\n", bytes, errno))
+ goto close_prog;
+
+ /* Open the perf event and attach bpf progrram */
+ attr.config = strtol(buf, NULL, 0);
+ attr.type = PERF_TYPE_TRACEPOINT;
+ attr.sample_type = PERF_SAMPLE_RAW | PERF_SAMPLE_CALLCHAIN;
+ attr.sample_period = 1;
+ attr.wakeup_events = 1;
+ pmu_fd = syscall(__NR_perf_event_open, &attr, -1 /* pid */,
+ 0 /* cpu 0 */, -1 /* group id */,
+ 0 /* flags */);
+ if (CHECK(pmu_fd < 0, "perf_event_open", "err %d errno %d\n",
+ pmu_fd, errno))
+ goto close_prog;
+
+ err = ioctl(pmu_fd, PERF_EVENT_IOC_ENABLE, 0);
+ if (CHECK(err, "perf_event_ioc_enable", "err %d errno %d\n",
+ err, errno))
+ goto close_pmu;
+
+ err = ioctl(pmu_fd, PERF_EVENT_IOC_SET_BPF, prog_fd);
+ if (CHECK(err, "perf_event_ioc_set_bpf", "err %d errno %d\n",
+ err, errno))
+ goto disable_pmu;
+
+ /* find map fds */
+ control_map_fd = bpf_find_map(__func__, obj, "control_map");
+ if (CHECK(control_map_fd < 0, "bpf_find_map control_map",
+ "err %d errno %d\n", err, errno))
+ goto disable_pmu;
+
+ stackid_hmap_fd = bpf_find_map(__func__, obj, "stackid_hmap");
+ if (CHECK(stackid_hmap_fd < 0, "bpf_find_map stackid_hmap",
+ "err %d errno %d\n", err, errno))
+ goto disable_pmu;
+
+ stackmap_fd = bpf_find_map(__func__, obj, "stackmap");
+ if (CHECK(stackmap_fd < 0, "bpf_find_map stackmap", "err %d errno %d\n",
+ err, errno))
+ goto disable_pmu;
+
+ assert(system("dd if=/dev/urandom of=/dev/zero count=4 2> /dev/null")
+ == 0);
+ assert(system("./urandom_read if=/dev/urandom of=/dev/zero count=4 2> /dev/null") == 0);
+ /* disable stack trace collection */
+ key = 0;
+ val = 1;
+ bpf_map_update_elem(control_map_fd, &key, &val, 0);
+
+ /* for every element in stackid_hmap, we can find a corresponding one
+ * in stackmap, and vise versa.
+ */
+ err = compare_map_keys(stackid_hmap_fd, stackmap_fd);
+ if (CHECK(err, "compare_map_keys stackid_hmap vs. stackmap",
+ "err %d errno %d\n", err, errno))
+ goto disable_pmu;
+
+ err = compare_map_keys(stackmap_fd, stackid_hmap_fd);
+ if (CHECK(err, "compare_map_keys stackmap vs. stackid_hmap",
+ "err %d errno %d\n", err, errno))
+ goto disable_pmu;
+
+ err = extract_build_id(buf, 256);
+
+ if (CHECK(err, "get build_id with readelf",
+ "err %d errno %d\n", err, errno))
+ goto disable_pmu;
+
+ err = bpf_map_get_next_key(stackmap_fd, NULL, &key);
+ if (CHECK(err, "get_next_key from stackmap",
+ "err %d, errno %d\n", err, errno))
+ goto disable_pmu;
+
+ do {
+ char build_id[64];
+
+ err = bpf_map_lookup_elem(stackmap_fd, &key, id_offs);
+ if (CHECK(err, "lookup_elem from stackmap",
+ "err %d, errno %d\n", err, errno))
+ goto disable_pmu;
+ for (i = 0; i < PERF_MAX_STACK_DEPTH; ++i)
+ if (id_offs[i].status == BPF_STACK_BUILD_ID_VALID &&
+ id_offs[i].offset != 0) {
+ for (j = 0; j < 20; ++j)
+ sprintf(build_id + 2 * j, "%02x",
+ id_offs[i].build_id[j] & 0xff);
+ if (strstr(buf, build_id) != NULL)
+ build_id_matches = 1;
+ }
+ previous_key = key;
+ } while (bpf_map_get_next_key(stackmap_fd, &previous_key, &key) == 0);
+
+ CHECK(build_id_matches < 1, "build id match",
+ "Didn't find expected build ID from the map");
+
+disable_pmu:
+ ioctl(pmu_fd, PERF_EVENT_IOC_DISABLE);
+
+close_pmu:
+ close(pmu_fd);
+
+close_prog:
+ bpf_object__close(obj);
+
+out:
+ return;
+}
+
int main(void)
{
test_pkt_access();
test_obj_name();
test_tp_attach_query();
test_stacktrace_map();
+ test_stacktrace_build_id();
printf("Summary: %d PASSED, %d FAILED\n", pass_cnt, error_cnt);
return error_cnt ? EXIT_FAILURE : EXIT_SUCCESS;
--- /dev/null
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (c) 2018 Facebook
+
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+#ifndef PERF_MAX_STACK_DEPTH
+#define PERF_MAX_STACK_DEPTH 127
+#endif
+
+struct bpf_map_def SEC("maps") control_map = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(__u32),
+ .value_size = sizeof(__u32),
+ .max_entries = 1,
+};
+
+struct bpf_map_def SEC("maps") stackid_hmap = {
+ .type = BPF_MAP_TYPE_HASH,
+ .key_size = sizeof(__u32),
+ .value_size = sizeof(__u32),
+ .max_entries = 10000,
+};
+
+struct bpf_map_def SEC("maps") stackmap = {
+ .type = BPF_MAP_TYPE_STACK_TRACE,
+ .key_size = sizeof(__u32),
+ .value_size = sizeof(struct bpf_stack_build_id)
+ * PERF_MAX_STACK_DEPTH,
+ .max_entries = 128,
+ .map_flags = BPF_F_STACK_BUILD_ID,
+};
+
+/* taken from /sys/kernel/debug/tracing/events/random/urandom_read/format */
+struct random_urandom_args {
+ unsigned long long pad;
+ int got_bits;
+ int pool_left;
+ int input_left;
+};
+
+SEC("tracepoint/random/urandom_read")
+int oncpu(struct random_urandom_args *args)
+{
+ __u32 key = 0, val = 0, *value_p;
+
+ value_p = bpf_map_lookup_elem(&control_map, &key);
+ if (value_p && *value_p)
+ return 0; /* skip if non-zero *value_p */
+
+ /* The size of stackmap and stackid_hmap should be the same */
+ key = bpf_get_stackid(args, &stackmap, BPF_F_USER_STACK);
+ if ((int)key >= 0)
+ bpf_map_update_elem(&stackid_hmap, &key, &val, 0);
+
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
+__u32 _version SEC("version") = 1; /* ignored by tracepoints, required by libbpf.a */
.result = ACCEPT,
.prog_type = BPF_PROG_TYPE_SK_SKB,
},
+ {
+ "direct packet read for SK_MSG",
+ .insns = {
+ BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1,
+ offsetof(struct sk_msg_md, data)),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_1,
+ offsetof(struct sk_msg_md, data_end)),
+ BPF_MOV64_REG(BPF_REG_0, BPF_REG_2),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, 8),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_0, BPF_REG_3, 1),
+ BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_2, 0),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .result = ACCEPT,
+ .prog_type = BPF_PROG_TYPE_SK_MSG,
+ },
+ {
+ "direct packet write for SK_MSG",
+ .insns = {
+ BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1,
+ offsetof(struct sk_msg_md, data)),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_1,
+ offsetof(struct sk_msg_md, data_end)),
+ BPF_MOV64_REG(BPF_REG_0, BPF_REG_2),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, 8),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_0, BPF_REG_3, 1),
+ BPF_STX_MEM(BPF_B, BPF_REG_2, BPF_REG_2, 0),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .result = ACCEPT,
+ .prog_type = BPF_PROG_TYPE_SK_MSG,
+ },
+ {
+ "overlapping checks for direct packet access SK_MSG",
+ .insns = {
+ BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_1,
+ offsetof(struct sk_msg_md, data)),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_3, BPF_REG_1,
+ offsetof(struct sk_msg_md, data_end)),
+ BPF_MOV64_REG(BPF_REG_0, BPF_REG_2),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, 8),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_0, BPF_REG_3, 4),
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_2),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 6),
+ BPF_JMP_REG(BPF_JGT, BPF_REG_1, BPF_REG_3, 1),
+ BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_2, 6),
+ BPF_MOV64_IMM(BPF_REG_0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .result = ACCEPT,
+ .prog_type = BPF_PROG_TYPE_SK_MSG,
+ },
{
"check skb->mark is not writeable by sockets",
.insns = {
--- /dev/null
+#include <stdio.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdlib.h>
+
+#define BUF_SIZE 256
+int main(void)
+{
+ int fd = open("/dev/urandom", O_RDONLY);
+ int i;
+ char buf[BUF_SIZE];
+
+ if (fd < 0)
+ return 1;
+ for (i = 0; i < 4; ++i)
+ read(fd, buf, BUF_SIZE);
+
+ close(fd);
+ return 0;
+}
# connected gateway
CONGW[1]=169.254.1.254
-CONGW[2]=169.254.5.254
+CONGW[2]=169.254.3.254
+CONGW[3]=169.254.5.254
# recursive gateway
RECGW4[1]=169.254.11.254
log_test $? ${exp_rc} "${desc}"
}
+run_ip_mpath()
+{
+ local table="$1"
+ local prefix="$2"
+ local nh1="$3"
+ local nh2="$4"
+ local exp_rc="$5"
+ local desc="$6"
+
+ # dev arg may be empty
+ [ -n "${dev}" ] && dev="dev ${dev}"
+
+ run_cmd ip ro add table "${table}" "${prefix}"/32 \
+ nexthop via ${nh1} nexthop via ${nh2}
+ log_test $? ${exp_rc} "${desc}"
+}
+
valid_onlink_ipv4()
{
# - unicast connected, unicast recursive
log_subsection "VRF ${VRF}"
- run_ip ${VRF_TABLE} ${TEST_NET4[2]}.1 ${CONGW[2]} ${NETIFS[p5]} 0 "unicast connected"
+ run_ip ${VRF_TABLE} ${TEST_NET4[2]}.1 ${CONGW[3]} ${NETIFS[p5]} 0 "unicast connected"
run_ip ${VRF_TABLE} ${TEST_NET4[2]}.2 ${RECGW4[2]} ${NETIFS[p5]} 0 "unicast recursive"
log_subsection "VRF device, PBR table"
- run_ip ${PBR_TABLE} ${TEST_NET4[2]}.3 ${CONGW[2]} ${NETIFS[p5]} 0 "unicast connected"
+ run_ip ${PBR_TABLE} ${TEST_NET4[2]}.3 ${CONGW[3]} ${NETIFS[p5]} 0 "unicast connected"
run_ip ${PBR_TABLE} ${TEST_NET4[2]}.4 ${RECGW4[2]} ${NETIFS[p5]} 0 "unicast recursive"
+
+ # multipath version
+ #
+ log_subsection "default VRF - main table - multipath"
+
+ run_ip_mpath 254 ${TEST_NET4[1]}.5 \
+ "${CONGW[1]} dev ${NETIFS[p1]} onlink" \
+ "${CONGW[2]} dev ${NETIFS[p3]} onlink" \
+ 0 "unicast connected - multipath"
+
+ run_ip_mpath 254 ${TEST_NET4[1]}.6 \
+ "${RECGW4[1]} dev ${NETIFS[p1]} onlink" \
+ "${RECGW4[2]} dev ${NETIFS[p3]} onlink" \
+ 0 "unicast recursive - multipath"
+
+ run_ip_mpath 254 ${TEST_NET4[1]}.7 \
+ "${CONGW[1]} dev ${NETIFS[p1]}" \
+ "${CONGW[2]} dev ${NETIFS[p3]} onlink" \
+ 0 "unicast connected - multipath onlink first only"
+
+ run_ip_mpath 254 ${TEST_NET4[1]}.8 \
+ "${CONGW[1]} dev ${NETIFS[p1]} onlink" \
+ "${CONGW[2]} dev ${NETIFS[p3]}" \
+ 0 "unicast connected - multipath onlink second only"
}
invalid_onlink_ipv4()
log_test $? ${exp_rc} "${desc}"
}
+run_ip6_mpath()
+{
+ local table="$1"
+ local prefix="$2"
+ local opts="$3"
+ local nh1="$4"
+ local nh2="$5"
+ local exp_rc="$6"
+ local desc="$7"
+
+ run_cmd ip -6 ro add table "${table}" "${prefix}"/128 "${opts}" \
+ nexthop via ${nh1} nexthop via ${nh2}
+ log_test $? ${exp_rc} "${desc}"
+}
+
valid_onlink_ipv6()
{
# - unicast connected, unicast recursive, v4-mapped
run_ip6 ${PBR_TABLE} ${TEST_NET6[2]}::4 ${V6ADDRS[p5]/::*}::64 ${NETIFS[p5]} 0 "unicast connected"
run_ip6 ${PBR_TABLE} ${TEST_NET6[2]}::5 ${RECGW6[2]} ${NETIFS[p5]} 0 "unicast recursive"
run_ip6 ${PBR_TABLE} ${TEST_NET6[2]}::6 ::ffff:${TEST_NET4IN6[2]} ${NETIFS[p5]} 0 "v4-mapped"
+
+ # multipath version
+ #
+ log_subsection "default VRF - main table - multipath"
+
+ run_ip6_mpath 254 ${TEST_NET6[1]}::4 "onlink" \
+ "${V6ADDRS[p1]/::*}::64 dev ${NETIFS[p1]}" \
+ "${V6ADDRS[p3]/::*}::64 dev ${NETIFS[p3]}" \
+ 0 "unicast connected - multipath onlink"
+
+ run_ip6_mpath 254 ${TEST_NET6[1]}::5 "onlink" \
+ "${RECGW6[1]} dev ${NETIFS[p1]}" \
+ "${RECGW6[2]} dev ${NETIFS[p3]}" \
+ 0 "unicast recursive - multipath onlink"
+
+ run_ip6_mpath 254 ${TEST_NET6[1]}::6 "onlink" \
+ "::ffff:${TEST_NET4IN6[1]} dev ${NETIFS[p1]}" \
+ "::ffff:${TEST_NET4IN6[2]} dev ${NETIFS[p3]}" \
+ 0 "v4-mapped - multipath onlink"
+
+ run_ip6_mpath 254 ${TEST_NET6[1]}::7 "" \
+ "${V6ADDRS[p1]/::*}::64 dev ${NETIFS[p1]} onlink" \
+ "${V6ADDRS[p3]/::*}::64 dev ${NETIFS[p3]} onlink" \
+ 0 "unicast connected - multipath onlink both nexthops"
+
+ run_ip6_mpath 254 ${TEST_NET6[1]}::8 "" \
+ "${V6ADDRS[p1]/::*}::64 dev ${NETIFS[p1]} onlink" \
+ "${V6ADDRS[p3]/::*}::64 dev ${NETIFS[p3]}" \
+ 0 "unicast connected - multipath onlink first only"
+
+ run_ip6_mpath 254 ${TEST_NET6[1]}::9 "" \
+ "${V6ADDRS[p1]/::*}::64 dev ${NETIFS[p1]}" \
+ "${V6ADDRS[p3]/::*}::64 dev ${NETIFS[p3]} onlink" \
+ 0 "unicast connected - multipath onlink second only"
}
invalid_onlink_ipv6()
log_section "IPv6 onlink"
log_subsection "Valid onlink commands"
valid_onlink_ipv6
+ log_subsection "Invalid onlink commands"
invalid_onlink_ipv6
}
return 0;
}
+static int syscall_available(void)
+{
+ int rc;
+
+ errno = 0;
+ rc = syscall(__NR_subpage_prot, 0, 0, 0);
+
+ return rc == 0 || (errno != ENOENT && errno != ENOSYS);
+}
+
int test_anon(void)
{
unsigned long align;
void *mallocblock;
unsigned long mallocsize;
+ SKIP_IF(!syscall_available());
+
if (getpagesize() != 0x10000) {
fprintf(stderr, "Kernel page size must be 64K!\n");
return 1;
off_t filesize;
int fd;
+ SKIP_IF(!syscall_available());
+
fd = open(file_name, O_RDWR);
if (fd == -1) {
perror("failed to open file");
$(OUTPUT)/tm-syscall: CFLAGS += -I../../../../../usr/include
$(OUTPUT)/tm-tmspr: CFLAGS += -pthread
$(OUTPUT)/tm-vmx-unavail: CFLAGS += -pthread -m64
-$(OUTPUT)/tm-resched-dscr: ../pmu/lib.o
+$(OUTPUT)/tm-resched-dscr: ../pmu/lib.c
$(OUTPUT)/tm-unavailable: CFLAGS += -O0 -pthread -m64 -Wno-error=uninitialized -mvsx
$(OUTPUT)/tm-trap: CFLAGS += -O0 -pthread -m64
struct sigaction trap_sa;
+ SKIP_IF(!have_htm());
+
trap_sa.sa_flags = SA_SIGINFO;
trap_sa.sa_sigaction = trap_signal_handler;
sigaction(SIGTRAP, &trap_sa, NULL);
--- /dev/null
+[
+ {
+ "id": "d959",
+ "name": "Add cBPF action with valid bytecode",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC action flush action bpf",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' index 100",
+ "expExitCode": "0",
+ "verifyCmd": "$TC action get action bpf index 100",
+ "matchPattern": "action order [0-9]*: bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' default-action pipe.*index 100 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC action flush action bpf"
+ ]
+ },
+ {
+ "id": "f84a",
+ "name": "Add cBPF action with invalid bytecode",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action bpf",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action add action bpf bytecode '4,40 0 0 12,31 0 1 2048,6 0 0 262144,6 0 0 0' index 100",
+ "expExitCode": "255",
+ "verifyCmd": "$TC action get action bpf index 100",
+ "matchPattern": "action order [0-9]*: bpf bytecode '4,40 0 0 12,31 0 1 2048,6 0 0 262144,6 0 0 0' default-action pipe.*index 100 ref",
+ "matchCount": "0",
+ "teardown": [
+ "$TC actions flush action bpf"
+ ]
+ },
+ {
+ "id": "e939",
+ "name": "Add eBPF action with valid object-file",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ "printf '#include <linux/bpf.h>\nchar l[] __attribute__((section(\"license\"),used))=\"GPL\"; __attribute__((section(\"action\"),used)) int m(struct __sk_buff *s) { return 2; }' | clang -O2 -x c -c - -target bpf -o _b.o",
+ [
+ "$TC action flush action bpf",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action add action bpf object-file _b.o index 667",
+ "expExitCode": "0",
+ "verifyCmd": "$TC action get action bpf index 667",
+ "matchPattern": "action order [0-9]*: bpf _b.o:\\[action\\] id [0-9]* tag 3b185187f1855c4c default-action pipe.*index 667 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC action flush action bpf",
+ "rm -f _b.o"
+ ]
+ },
+ {
+ "id": "282d",
+ "name": "Add eBPF action with invalid object-file",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ "printf '#include <linux/bpf.h>\nchar l[] __attribute__((section(\"license\"),used))=\"GPL\"; __attribute__((section(\"action\"),used)) int m(struct __sk_buff *s) { s->data = 0x0; return 2; }' | clang -O2 -x c -c - -target bpf -o _c.o",
+ [
+ "$TC action flush action bpf",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action add action bpf object-file _c.o index 667",
+ "expExitCode": "255",
+ "verifyCmd": "$TC action get action bpf index 667",
+ "matchPattern": "action order [0-9]*: bpf _b.o:\\[action\\] id [0-9].*index 667 ref",
+ "matchCount": "0",
+ "teardown": [
+ "$TC action flush action bpf",
+ "rm -f _c.o"
+ ]
+ },
+ {
+ "id": "d819",
+ "name": "Replace cBPF bytecode and action control",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action bpf",
+ 0,
+ 1,
+ 255
+ ],
+ [
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' index 555",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action replace action bpf bytecode '4,40 0 0 12,21 0 1 2054,6 0 0 262144,6 0 0 0' drop index 555",
+ "expExitCode": "0",
+ "verifyCmd": "$TC action get action bpf index 555",
+ "matchPattern": "action order [0-9]*: bpf bytecode '4,40 0 0 12,21 0 1 2054,6 0 0 262144,6 0 0 0' default-action drop.*index 555 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC action flush action bpf"
+ ]
+ },
+ {
+ "id": "6ae3",
+ "name": "Delete cBPF action ",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action bpf",
+ 0,
+ 1,
+ 255
+ ],
+ [
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' index 444",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action delete action bpf index 444",
+ "expExitCode": "0",
+ "verifyCmd": "$TC action get action bpf index 444",
+ "matchPattern": "action order [0-9]*: bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' default-action pipe.*index 444 ref",
+ "matchCount": "0",
+ "teardown": [
+ "$TC action flush action bpf"
+ ]
+ },
+ {
+ "id": "3e0d",
+ "name": "List cBPF actions",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC action flush action bpf",
+ 0,
+ 1,
+ 255
+ ],
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' ok index 101",
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2054,6 0 0 262144,6 0 0 0' drop index 102",
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 33024,6 0 0 262144,6 0 0 0' continue index 103"
+ ],
+ "cmdUnderTest": "$TC action list action bpf",
+ "expExitCode": "0",
+ "verifyCmd": "$TC action list action bpf",
+ "matchPattern": "action order [0-9]*: bpf bytecode",
+ "matchCount": "3",
+ "teardown": [
+ "$TC actions flush action bpf"
+ ]
+ },
+ {
+ "id": "55ce",
+ "name": "Flush BPF actions",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action bpf",
+ 0,
+ 1,
+ 255
+ ],
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' ok index 101",
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2054,6 0 0 262144,6 0 0 0' drop index 102",
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 33024,6 0 0 262144,6 0 0 0' continue index 103"
+ ],
+ "cmdUnderTest": "$TC action flush action bpf",
+ "expExitCode": "0",
+ "verifyCmd": "$TC action list action bpf",
+ "matchPattern": "action order [0-9]*: bpf bytecode",
+ "matchCount": "0",
+ "teardown": [
+ "$TC actions flush action bpf"
+ ]
+ },
+ {
+ "id": "ccc3",
+ "name": "Add cBPF action with duplicate index",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action bpf",
+ 0,
+ 1,
+ 255
+ ],
+ "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' index 4294967295"
+ ],
+ "cmdUnderTest": "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2054,6 0 0 262144,6 0 0 0' index 4294967295",
+ "expExitCode": "255",
+ "verifyCmd": "$TC action get action bpf index 4294967295",
+ "matchPattern": "action order [0-9]*: bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' default-action pipe.*index 4294967295",
+ "matchCount": "1",
+ "teardown": [
+ "$TC action flush action bpf"
+ ]
+ },
+ {
+ "id": "89c7",
+ "name": "Add cBPF action with invalid index",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action bpf",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2054,6 0 0 262144,6 0 0 0' index 4294967296 cookie 12345",
+ "expExitCode": "255",
+ "verifyCmd": "$TC action ls action bpf",
+ "matchPattern": "action order [0-9]*: bpf bytecode '4,40 0 0 12,21 0 1 2048,6 0 0 262144,6 0 0 0' default-action pipe.*cookie 12345",
+ "matchCount": "0",
+ "teardown": [
+ "$TC action flush action bpf"
+ ]
+ },
+ {
+ "id": "7ab9",
+ "name": "Add cBPF action with cookie",
+ "category": [
+ "actions",
+ "bpf"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action bpf",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC action add action bpf bytecode '4,40 0 0 12,21 0 1 2054,6 0 0 262144,6 0 0 0' cookie d0d0d0d0d0d0d0d0",
+ "expExitCode": "0",
+ "verifyCmd": "$TC action list action bpf",
+ "matchPattern": "action order [0-9]*: bpf.*cookie d0d0d0d0d0d0d0",
+ "matchCount": "1",
+ "teardown": [
+ "$TC action flush action bpf"
+ ]
+ }
+]
255
]
],
- "cmdUnderTest": "for i in `seq 1 32`; do cmd=\"action pass index $i \"; args=\"$args$cmd\"; done && $TC actions add $args",
+ "cmdUnderTest": "bash -c \"for i in \\`seq 1 32\\`; do cmd=\\\"action pass index \\$i \\\"; args=\"\\$args\\$cmd\"; done && $TC actions add \\$args\"",
"expExitCode": "0",
"verifyCmd": "$TC actions list action gact",
"matchPattern": "^[ \t]+index [0-9]+ ref",
255
]
],
- "cmdUnderTest": "for i in `seq 1 32`; do cmd=\"action continue index $i cookie aabbccddeeff112233445566778800a1 \"; args=\"$args$cmd\"; done && $TC actions add $args",
+ "cmdUnderTest": "bash -c \"for i in \\`seq 1 32\\`; do cmd=\\\"action continue index \\$i cookie aabbccddeeff112233445566778800a1 \\\"; args=\"\\$args\\$cmd\"; done && $TC actions add \\$args\"",
"expExitCode": "0",
"verifyCmd": "$TC actions list action gact",
"matchPattern": "^[ \t]+index [0-9]+ ref",
1,
255
],
- "for i in `seq 1 32`; do cmd=\"action continue index $i \"; args=\"$args$cmd\"; done && $TC actions add $args"
+ "bash -c \"for i in \\`seq 1 32\\`; do cmd=\\\"action continue index \\$i \\\"; args=\\\"\\$args\\$cmd\\\"; done && $TC actions add \\$args\""
],
- "cmdUnderTest": "for i in `seq 1 32`; do cmd=\"action gact index $i \"; args=\"$args$cmd\"; done && $TC actions del $args",
+ "cmdUnderTest": "bash -c \"for i in \\`seq 1 32\\`; do cmd=\\\"action gact index \\$i \\\"; args=\"\\$args\\$cmd\"; done && $TC actions del \\$args\"",
"expExitCode": "0",
"verifyCmd": "$TC actions list action gact",
"matchPattern": "^[ \t]+index [0-9]+ ref",
"matchCount": "0",
"teardown": []
}
-]
\ No newline at end of file
+]
"$TC actions flush action mirred"
]
},
+ {
+ "id": "8917",
+ "name": "Add mirred mirror action with control pass",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo pass index 1",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 1",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) pass.*index 1 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
+ {
+ "id": "1054",
+ "name": "Add mirred mirror action with control pipe",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo pipe index 15",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 15",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) pipe.*index 15 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
+ {
+ "id": "9887",
+ "name": "Add mirred mirror action with control continue",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo continue index 15",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 15",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) continue.*index 15 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
+ {
+ "id": "e4aa",
+ "name": "Add mirred mirror action with control reclassify",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo reclassify index 150",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 150",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) reclassify.*index 150 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
+ {
+ "id": "ece9",
+ "name": "Add mirred mirror action with control drop",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo drop index 99",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 99",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) drop.*index 99 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
+ {
+ "id": "0031",
+ "name": "Add mirred mirror action with control jump",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo jump 10 index 99",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 99",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) jump 10.*index 99 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
+ {
+ "id": "407c",
+ "name": "Add mirred mirror action with cookie",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo reclassify cookie aa11bb22cc33dd44ee55",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions ls action mirred",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) reclassify.*cookie aa11bb22cc33dd44ee55",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
+ {
+ "id": "8b69",
+ "name": "Add mirred mirror action with maximum index",
+ "category": [
+ "actions",
+ "mirred"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action mirred",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action mirred ingress mirror dev lo pipe index 4294967295",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 4294967295",
+ "matchPattern": "action order [0-9]*: mirred \\(Ingress Mirror to device lo\\) pipe.*index 4294967295",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
{
"id": "a70e",
"name": "Delete mirred mirror action",
"$TC actions flush action police"
]
},
+ {
+ "id": "ddd6",
+ "name": "Add police action with invalid rate value",
+ "category": [
+ "actions",
+ "police"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action police",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action police rate 3tb burst 250k conform-exceed pass/pipe index 5",
+ "expExitCode": "255",
+ "verifyCmd": "$TC actions ls action police",
+ "matchPattern": "action order [0-9]*: police 0x5 rate 3Tb burst 250Kb mtu 2Kb action pass/pipe",
+ "matchCount": "0",
+ "teardown": [
+ "$TC actions flush action police"
+ ]
+ },
+ {
+ "id": "f61c",
+ "name": "Add police action with invalid burst value",
+ "category": [
+ "actions",
+ "police"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action police",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action police rate 3kbit burst 250P conform-exceed pass/pipe index 5",
+ "expExitCode": "255",
+ "verifyCmd": "$TC actions ls action police",
+ "matchPattern": "action order [0-9]*: police 0x5 rate 3Kbit burst 250Pb mtu 2Kb action pass/pipe",
+ "matchCount": "0",
+ "teardown": [
+ "$TC actions flush action police"
+ ]
+ },
+ {
+ "id": "c26f",
+ "name": "Add police action with invalid peakrate value",
+ "category": [
+ "actions",
+ "police"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action police",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action police rate 90kbit burst 10k mtu 2kb peakrate 100T index 1",
+ "expExitCode": "255",
+ "verifyCmd": "$TC actions ls action police",
+ "matchPattern": "action order [0-9]*: police 0x1 rate 90Kbit burst 10Kb mtu 2Kb peakrate 100Tbit",
+ "matchCount": "0",
+ "teardown": [
+ "$TC actions flush action police"
+ ]
+ },
+ {
+ "id": "db04",
+ "name": "Add police action with invalid mtu value",
+ "category": [
+ "actions",
+ "police"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action police",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action police rate 10kbit burst 10k mtu 2Pbit index 1",
+ "expExitCode": "255",
+ "verifyCmd": "$TC actions ls action police",
+ "matchPattern": "action order [0-9]*: police 0x1 rate 10Kbit burst 1Kb mtu 2Pb",
+ "matchCount": "0",
+ "teardown": [
+ "$TC actions flush action police"
+ ]
+ },
+ {
+ "id": "f3c9",
+ "name": "Add police action with cookie",
+ "category": [
+ "actions",
+ "police"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action police",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action police rate 10mbit burst 10k index 1 cookie a1b1c1d1e1f12233bb",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action police index 1",
+ "matchPattern": "action order [0-9]*: police 0x1 rate 10Mbit burst 10Kb mtu 2Kb.*cookie a1b1c1d1e1f12233bb",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action police"
+ ]
+ },
+ {
+ "id": "d190",
+ "name": "Add police action with maximum index",
+ "category": [
+ "actions",
+ "police"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action police",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action police rate 10mbit burst 10k index 4294967295",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action mirred index 4294967295",
+ "matchPattern": "action order [0-9]*: police 0xffffffff rate 10Mbit burst 10Kb mtu 2Kb",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action mirred"
+ ]
+ },
{
"id": "336e",
"name": "Delete police action",
"$TC actions flush action skbedit"
]
},
+ {
+ "id": "464a",
+ "name": "Add skbedit action with control pipe",
+ "category": [
+ "actions",
+ "skbedit"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbedit",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbedit ptype host pipe index 11",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbedit index 11",
+ "matchPattern": "action order [0-9]*: skbedit ptype host pipe.*index 11 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbedit"
+ ]
+ },
+ {
+ "id": "212f",
+ "name": "Add skbedit action with control reclassify",
+ "category": [
+ "actions",
+ "skbedit"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbedit",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbedit mark 56789 reclassify index 90",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbedit index 90",
+ "matchPattern": "action order [0-9]*: skbedit mark 56789 reclassify.*index 90 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbedit"
+ ]
+ },
+ {
+ "id": "0651",
+ "name": "Add skbedit action with control pass",
+ "category": [
+ "actions",
+ "skbedit"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbedit",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbedit queue_mapping 3 pass index 271",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbedit index 271",
+ "matchPattern": "action order [0-9]*: skbedit queue_mapping 3 pass.*index 271 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbedit"
+ ]
+ },
+ {
+ "id": "cc53",
+ "name": "Add skbedit action with control drop",
+ "category": [
+ "actions",
+ "skbedit"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbedit",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbedit queue_mapping 3 drop index 271",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbedit index 271",
+ "matchPattern": "action order [0-9]*: skbedit queue_mapping 3 drop.*index 271 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbedit"
+ ]
+ },
+ {
+ "id": "ec16",
+ "name": "Add skbedit action with control jump",
+ "category": [
+ "actions",
+ "skbedit"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbedit",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbedit priority 8 jump 9 index 2",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbedit index 2",
+ "matchPattern": "action order [0-9]*: skbedit priority :8 jump 9.*index 2 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbedit"
+ ]
+ },
+ {
+ "id": "db54",
+ "name": "Add skbedit action with control continue",
+ "category": [
+ "actions",
+ "skbedit"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbedit",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbedit priority 16 continue index 32",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbedit index 32",
+ "matchPattern": "action order [0-9]*: skbedit priority :16 continue.*index 32 ref",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbedit"
+ ]
+ },
+ {
+ "id": "1055",
+ "name": "Add skbedit action with cookie",
+ "category": [
+ "actions",
+ "skbedit"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbedit",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbedit priority 16 continue index 32 cookie deadbeef",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbedit index 32",
+ "matchPattern": "action order [0-9]*: skbedit priority :16 continue.*index 32 ref.*cookie deadbeef",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbedit"
+ ]
+ },
{
"id": "5172",
"name": "List skbedit actions",
"$TC actions flush action skbmod"
]
},
+ {
+ "id": "6046",
+ "name": "Add skbmod action with control reclassify and cookie",
+ "category": [
+ "actions",
+ "skbmod"
+ ],
+ "setup": [
+ [
+ "$TC actions flush action skbmod",
+ 0,
+ 1,
+ 255
+ ]
+ ],
+ "cmdUnderTest": "$TC actions add action skbmod set smac 00:01:02:03:04:01 reclassify index 1 cookie ddeeffaabb11cc22",
+ "expExitCode": "0",
+ "verifyCmd": "$TC actions get action skbmod index 1",
+ "matchPattern": "action order [0-9]*: skbmod reclassify set smac 00:01:02:03:04:01.*index 1 ref.*cookie ddeeffaabb11cc22",
+ "matchCount": "1",
+ "teardown": [
+ "$TC actions flush action skbmod"
+ ]
+ },
{
"id": "58cb",
"name": "List skbmod actions",
# SPDX-License-Identifier: GPL-2.0
#please run as root
-#we need 256M, below is the size in kB
-needmem=262144
mnt=./huge
exitcode=0
-#get pagesize and freepages from /proc/meminfo
+#get huge pagesize and freepages from /proc/meminfo
while read name size unit; do
if [ "$name" = "HugePages_Free:" ]; then
freepgs=$size
fi
if [ "$name" = "Hugepagesize:" ]; then
- pgsize=$size
+ hpgsize_KB=$size
fi
done < /proc/meminfo
+# Simple hugetlbfs tests have a hardcoded minimum requirement of
+# huge pages totaling 256MB (262144KB) in size. The userfaultfd
+# hugetlb test requires a minimum of 2 * nr_cpus huge pages. Take
+# both of these requirements into account and attempt to increase
+# number of huge pages available.
+nr_cpus=$(nproc)
+hpgsize_MB=$((hpgsize_KB / 1024))
+half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128))
+needmem_KB=$((half_ufd_size_MB * 2 * 1024))
+
#set proper nr_hugepages
-if [ -n "$freepgs" ] && [ -n "$pgsize" ]; then
+if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then
nr_hugepgs=`cat /proc/sys/vm/nr_hugepages`
- needpgs=`expr $needmem / $pgsize`
+ needpgs=$((needmem_KB / hpgsize_KB))
tries=2
while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do
lackpgs=$(( $needpgs - $freepgs ))
echo "---------------------------"
echo "running userfaultfd_hugetlb"
echo "---------------------------"
-# 256MB total huge pages == 128MB src and 128MB dst
-./userfaultfd hugetlb 128 32 $mnt/ufd_test_file
+# Test requires source and destination huge pages. Size of source
+# (half_ufd_size_MB) is passed as argument to test.
+./userfaultfd hugetlb $half_ufd_size_MB 32 $mnt/ufd_test_file
if [ $? -ne 0 ]; then
echo "[FAIL]"
exitcode=1
"int3\n\t"
"vmcode_int80:\n\t"
"int $0x80\n\t"
+ "vmcode_popf_hlt:\n\t"
+ "push %ax\n\t"
+ "popf\n\t"
+ "hlt\n\t"
"vmcode_umip:\n\t"
/* addressing via displacements */
"smsw (2052)\n\t"
extern unsigned char vmcode[], end_vmcode[];
extern unsigned char vmcode_bound[], vmcode_sysenter[], vmcode_syscall[],
- vmcode_sti[], vmcode_int3[], vmcode_int80[], vmcode_umip[],
- vmcode_umip_str[], vmcode_umip_sldt[];
+ vmcode_sti[], vmcode_int3[], vmcode_int80[], vmcode_popf_hlt[],
+ vmcode_umip[], vmcode_umip_str[], vmcode_umip_sldt[];
/* Returns false if the test was skipped. */
static bool do_test(struct vm86plus_struct *v86, unsigned long eip,
(VM86_TYPE(ret) == rettype && VM86_ARG(ret) == retarg)) {
printf("[OK]\tReturned correctly\n");
} else {
- printf("[FAIL]\tIncorrect return reason\n");
+ printf("[FAIL]\tIncorrect return reason (started at eip = 0x%lx, ended at eip = 0x%lx)\n", eip, v86->regs.eip);
nerrs++;
}
v86.regs.ds = load_addr / 16;
v86.regs.es = load_addr / 16;
+ /* Use the end of the page as our stack. */
+ v86.regs.esp = 4096;
+
assert((v86.regs.cs & 3) == 0); /* Looks like RPL = 0 */
/* #BR -- should deliver SIG??? */
v86.regs.eflags &= ~X86_EFLAGS_IF;
do_test(&v86, vmcode_sti - vmcode, VM86_STI, 0, "STI with VIP set");
+ /* POPF with VIP set but IF clear: should not trap */
+ v86.regs.eflags = X86_EFLAGS_VIP;
+ v86.regs.eax = 0;
+ do_test(&v86, vmcode_popf_hlt - vmcode, VM86_UNKNOWN, 0, "POPF with VIP set and IF clear");
+
+ /* POPF with VIP set and IF set: should trap */
+ v86.regs.eflags = X86_EFLAGS_VIP;
+ v86.regs.eax = X86_EFLAGS_IF;
+ do_test(&v86, vmcode_popf_hlt - vmcode, VM86_STI, 0, "POPF with VIP and IF set");
+
+ /* POPF with VIP clear and IF set: should not trap */
+ v86.regs.eflags = 0;
+ v86.regs.eax = X86_EFLAGS_IF;
+ do_test(&v86, vmcode_popf_hlt - vmcode, VM86_UNKNOWN, 0, "POPF with VIP clear and IF set");
+
+ v86.regs.eflags = 0;
+
/* INT3 -- should cause #BP */
do_test(&v86, vmcode_int3 - vmcode, VM86_TRAP, 3, "INT3");
clearhandler(SIGSEGV);
/* Make sure nothing explodes if we fork. */
- if (fork() > 0)
+ if (fork() == 0)
return 0;
return (nerrs == 0 ? 0 : 1);
num_vsyscall_traps++;
}
-static int test_native_vsyscall(void)
+static int test_emulation(void)
{
time_t tmp;
bool is_native;
if (!vtime)
return 0;
- printf("[RUN]\tchecking for native vsyscall\n");
+ printf("[RUN]\tchecking that vsyscalls are emulated\n");
sethandler(SIGTRAP, sigtrap, 0);
set_eflags(get_eflags() | X86_EFLAGS_TF);
vtime(&tmp);
*/
is_native = (num_vsyscall_traps > 1);
- printf("\tvsyscalls are %s (%d instructions in vsyscall page)\n",
+ printf("[%s]\tvsyscalls are %s (%d instructions in vsyscall page)\n",
+ (is_native ? "FAIL" : "OK"),
(is_native ? "native" : "emulated"),
(int)num_vsyscall_traps);
- return 0;
+ return is_native;
}
#endif
nerrs += test_vsys_r();
#ifdef __x86_64__
- nerrs += test_native_vsyscall();
+ nerrs += test_emulation();
#endif
return nerrs ? 1 : 0;
int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu)
{
+ struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
ptimer->cnt_ctl = 0;
kvm_timer_update_state(vcpu);
+ if (timer->enabled && irqchip_in_kernel(vcpu->kvm))
+ kvm_vgic_reset_mapped_irq(vcpu, vtimer->irq.irq);
+
return 0;
}
static_branch_enable(&has_gic_active_state);
}
- kvm_info("virtual timer IRQ%d\n", host_vtimer_irq);
+ kvm_debug("virtual timer IRQ%d\n", host_vtimer_irq);
cpuhp_setup_state(CPUHP_AP_KVM_ARM_TIMER_STARTING,
"kvm/arm/timer:starting", kvm_timer_starting_cpu,
int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
struct kvm_mp_state *mp_state)
{
- vcpu_load(vcpu);
-
if (vcpu->arch.power_off)
mp_state->mp_state = KVM_MP_STATE_STOPPED;
else
mp_state->mp_state = KVM_MP_STATE_RUNNABLE;
- vcpu_put(vcpu);
return 0;
}
{
int ret = 0;
- vcpu_load(vcpu);
-
switch (mp_state->mp_state) {
case KVM_MP_STATE_RUNNABLE:
vcpu->arch.power_off = false;
ret = -EINVAL;
}
- vcpu_put(vcpu);
return ret;
}
struct kvm_device_attr attr;
long r;
- vcpu_load(vcpu);
-
switch (ioctl) {
case KVM_ARM_VCPU_INIT: {
struct kvm_vcpu_init init;
r = -EINVAL;
}
- vcpu_put(vcpu);
return r;
}
* are now visible to the system register interface.
*/
if (!cpu_if->vgic_sre) {
- dsb(st);
+ dsb(sy);
+ isb();
cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
}
*/
BUG_ON((hyp_idmap_start ^ (hyp_idmap_end - 1)) & PAGE_MASK);
- kvm_info("IDMAP page: %lx\n", hyp_idmap_start);
- kvm_info("HYP VA range: %lx:%lx\n",
- kern_hyp_va(PAGE_OFFSET), kern_hyp_va(~0UL));
+ kvm_debug("IDMAP page: %lx\n", hyp_idmap_start);
+ kvm_debug("HYP VA range: %lx:%lx\n",
+ kern_hyp_va(PAGE_OFFSET), kern_hyp_va(~0UL));
if (hyp_idmap_start >= kern_hyp_va(PAGE_OFFSET) &&
hyp_idmap_start < kern_hyp_va(~0UL) &&
/* Loop over all IRQs affected by this read */
for (i = 0; i < len * 8; i++) {
struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+ unsigned long flags;
+ spin_lock_irqsave(&irq->irq_lock, flags);
if (irq_is_pending(irq))
value |= (1U << i);
+ spin_unlock_irqrestore(&irq->irq_lock, flags);
vgic_put_irq(vcpu->kvm, irq);
}
vgic_v2_write_lr(i, 0);
}
+void vgic_v2_set_npie(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
+
+ cpuif->vgic_hcr |= GICH_HCR_NPIE;
+}
+
void vgic_v2_set_underflow(struct kvm_vcpu *vcpu)
{
struct vgic_v2_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v2;
int lr;
unsigned long flags;
- cpuif->vgic_hcr &= ~GICH_HCR_UIE;
+ cpuif->vgic_hcr &= ~(GICH_HCR_UIE | GICH_HCR_NPIE);
for (lr = 0; lr < vgic_cpu->used_lrs; lr++) {
u32 val = cpuif->vgic_lr[lr];
kvm_vgic_global_state.type = VGIC_V2;
kvm_vgic_global_state.max_gic_vcpus = VGIC_V2_MAX_CPUS;
- kvm_info("vgic-v2@%llx\n", info->vctrl.start);
+ kvm_debug("vgic-v2@%llx\n", info->vctrl.start);
return 0;
out:
static bool common_trap;
static bool gicv4_enable;
+void vgic_v3_set_npie(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v3;
+
+ cpuif->vgic_hcr |= ICH_HCR_NPIE;
+}
+
void vgic_v3_set_underflow(struct kvm_vcpu *vcpu)
{
struct vgic_v3_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v3;
int lr;
unsigned long flags;
- cpuif->vgic_hcr &= ~ICH_HCR_UIE;
+ cpuif->vgic_hcr &= ~(ICH_HCR_UIE | ICH_HCR_NPIE);
for (lr = 0; lr < vgic_cpu->used_lrs; lr++) {
u64 val = cpuif->vgic_lr[lr];
return ret;
}
+/**
+ * kvm_vgic_reset_mapped_irq - Reset a mapped IRQ
+ * @vcpu: The VCPU pointer
+ * @vintid: The INTID of the interrupt
+ *
+ * Reset the active and pending states of a mapped interrupt. Kernel
+ * subsystems injecting mapped interrupts should reset their interrupt lines
+ * when we are doing a reset of the VM.
+ */
+void kvm_vgic_reset_mapped_irq(struct kvm_vcpu *vcpu, u32 vintid)
+{
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, vintid);
+ unsigned long flags;
+
+ if (!irq->hw)
+ goto out;
+
+ spin_lock_irqsave(&irq->irq_lock, flags);
+ irq->active = false;
+ irq->pending_latch = false;
+ irq->line_level = false;
+ spin_unlock_irqrestore(&irq->irq_lock, flags);
+out:
+ vgic_put_irq(vcpu->kvm, irq);
+}
+
int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, unsigned int vintid)
{
struct vgic_irq *irq;
vgic_v3_set_underflow(vcpu);
}
+static inline void vgic_set_npie(struct kvm_vcpu *vcpu)
+{
+ if (kvm_vgic_global_state.type == VGIC_V2)
+ vgic_v2_set_npie(vcpu);
+ else
+ vgic_v3_set_npie(vcpu);
+}
+
/* Requires the ap_list_lock to be held. */
-static int compute_ap_list_depth(struct kvm_vcpu *vcpu)
+static int compute_ap_list_depth(struct kvm_vcpu *vcpu,
+ bool *multi_sgi)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_irq *irq;
int count = 0;
+ *multi_sgi = false;
+
DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&vgic_cpu->ap_list_lock));
list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
spin_lock(&irq->irq_lock);
/* GICv2 SGIs can count for more than one... */
- if (vgic_irq_is_sgi(irq->intid) && irq->source)
- count += hweight8(irq->source);
- else
+ if (vgic_irq_is_sgi(irq->intid) && irq->source) {
+ int w = hweight8(irq->source);
+
+ count += w;
+ *multi_sgi |= (w > 1);
+ } else {
count++;
+ }
spin_unlock(&irq->irq_lock);
}
return count;
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_irq *irq;
- int count = 0;
+ int count;
+ bool npie = false;
+ bool multi_sgi;
+ u8 prio = 0xff;
DEBUG_SPINLOCK_BUG_ON(!spin_is_locked(&vgic_cpu->ap_list_lock));
- if (compute_ap_list_depth(vcpu) > kvm_vgic_global_state.nr_lr)
+ count = compute_ap_list_depth(vcpu, &multi_sgi);
+ if (count > kvm_vgic_global_state.nr_lr || multi_sgi)
vgic_sort_ap_list(vcpu);
+ count = 0;
+
list_for_each_entry(irq, &vgic_cpu->ap_list_head, ap_list) {
spin_lock(&irq->irq_lock);
- if (unlikely(vgic_target_oracle(irq) != vcpu))
- goto next;
-
/*
- * If we get an SGI with multiple sources, try to get
- * them in all at once.
+ * If we have multi-SGIs in the pipeline, we need to
+ * guarantee that they are all seen before any IRQ of
+ * lower priority. In that case, we need to filter out
+ * these interrupts by exiting early. This is easy as
+ * the AP list has been sorted already.
*/
- do {
+ if (multi_sgi && irq->priority > prio) {
+ spin_unlock(&irq->irq_lock);
+ break;
+ }
+
+ if (likely(vgic_target_oracle(irq) == vcpu)) {
vgic_populate_lr(vcpu, irq, count++);
- } while (irq->source && count < kvm_vgic_global_state.nr_lr);
-next:
+ if (irq->source) {
+ npie = true;
+ prio = irq->priority;
+ }
+ }
+
spin_unlock(&irq->irq_lock);
if (count == kvm_vgic_global_state.nr_lr) {
}
}
+ if (npie)
+ vgic_set_npie(vcpu);
+
vcpu->arch.vgic_cpu.used_lrs = count;
/* Nuke remaining LRs */
/* we only support 64 kB translation table page size */
#define KVM_ITS_L1E_ADDR_MASK GENMASK_ULL(51, 16)
+/* Requires the irq_lock to be held by the caller. */
static inline bool irq_is_pending(struct vgic_irq *irq)
{
if (irq->config == VGIC_CONFIG_EDGE)
void vgic_v2_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
void vgic_v2_clear_lr(struct kvm_vcpu *vcpu, int lr);
void vgic_v2_set_underflow(struct kvm_vcpu *vcpu);
+void vgic_v2_set_npie(struct kvm_vcpu *vcpu);
int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
int vgic_v2_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
int offset, u32 *val);
void vgic_v3_populate_lr(struct kvm_vcpu *vcpu, struct vgic_irq *irq, int lr);
void vgic_v3_clear_lr(struct kvm_vcpu *vcpu, int lr);
void vgic_v3_set_underflow(struct kvm_vcpu *vcpu);
+void vgic_v3_set_npie(struct kvm_vcpu *vcpu);
void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
void vgic_v3_enable(struct kvm_vcpu *vcpu);
projects / mirror_ubuntu-eoan-kernel.git / commitdiff