[mirror_ubuntu-bionic-kernel.git] / Documentation / networking / s2io.txt

Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver.

Contents
=======
- 1.  Introduction
- 2.  Identifying the adapter/interface
- 3.  Features supported
- 4.  Command line parameters
- 5.  Performance suggestions
- 6.  Available Downloads 


1.	Introduction:
This Linux driver supports Neterion's Xframe I PCI-X 1.0 and
Xframe II PCI-X 2.0 adapters. It supports several features 
such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.
See below for complete list of features.
All features are supported for both IPv4 and IPv6.

2.	Identifying the adapter/interface:
a. Insert the adapter(s) in your system.
b. Build and load driver 
# insmod s2io.ko
c. View log messages
# dmesg | tail -40
You will see messages similar to:
eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
eth4: Device is on 64 bit 133MHz PCIX(M1) bus

The above messages identify the adapter type(Xframe I/II), adapter revision,
driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).
In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed
as well.

To associate an interface with a physical adapter use "ethtool -p <ethX>".
The corresponding adapter's LED will blink multiple times.

3.	Features supported:
a. Jumbo frames. Xframe I/II supports MTU up to 9600 bytes,
modifiable using ip command.

b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit
and receive, TSO.

c. Multi-buffer receive mode. Scattering of packet across multiple
buffers. Currently driver supports 2-buffer mode which yields
significant performance improvement on certain platforms(SGI Altix,
IBM xSeries).

d. MSI/MSI-X. Can be enabled on platforms which support this feature
(IA64, Xeon) resulting in noticeable performance improvement(up to 7%
on certain platforms).

e. Statistics. Comprehensive MAC-level and software statistics displayed
using "ethtool -S" option.

f. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings,
with multiple steering options.

4.  Command line parameters
a. tx_fifo_num
Number of transmit queues
Valid range: 1-8
Default: 1

b. rx_ring_num
Number of receive rings
Valid range: 1-8
Default: 1

c. tx_fifo_len
Size of each transmit queue
Valid range: Total length of all queues should not exceed 8192
Default: 4096

d. rx_ring_sz 
Size of each receive ring(in 4K blocks)
Valid range: Limited by memory on system
Default: 30 

e. intr_type
Specifies interrupt type. Possible values 0(INTA), 2(MSI-X)
Valid values: 0, 2
Default: 2

5.  Performance suggestions
General:
a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)
b. Set TCP windows size to optimal value. 
For instance, for MTU=1500 a value of 210K has been observed to result in 
good performance.
# sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
# sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"
For MTU=9000, TCP window size of 10 MB is recommended.
# sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
# sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"

Transmit performance:
a. By default, the driver respects BIOS settings for PCI bus parameters. 
However, you may want to experiment with PCI bus parameters 
max-split-transactions(MOST) and MMRBC (use setpci command). 
A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.  
It could be different for your hardware.  
Set MMRBC to 4K**.

For example you can set 
For opteron
#setpci -d 17d5:* 62=1d 
For Itanium
#setpci -d 17d5:* 62=3d 

For detailed description of the PCI registers, please see Xframe User Guide.

b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this 
parameter.
c. Turn on TSO(using "ethtool -K")
# ethtool -K <ethX> tso on

Receive performance:
a. By default, the driver respects BIOS settings for PCI bus parameters. 
However, you may want to set PCI latency timer to 248.
#setpci -d 17d5:* LATENCY_TIMER=f8
For detailed description of the PCI registers, please see Xframe User Guide.
b. Use 2-buffer mode. This results in large performance boost on
certain platforms(eg. SGI Altix, IBM xSeries).
c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to 
set/verify this option.
d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network 
device support --->  Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to 
bring down CPU utilization.

** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are 
recommended as safe parameters.
For more information, please review the AMD8131 errata at
http://vip.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/
26310_AMD-8131_HyperTransport_PCI-X_Tunnel_Revision_Guide_rev_3_18.pdf

6. Support
For further support please contact either your 10GbE Xframe NIC vendor (IBM, 
HP, SGI etc.)
Commit	Line	Data
571de88b RA	1	Release notes for Neterion's (Formerly S2io) Xframe I/II PCI-X 10GbE driver.
	2
	3	Contents
	4	=======
	5	- 1. Introduction
	6	- 2. Identifying the adapter/interface
	7	- 3. Features supported
	8	- 4. Command line parameters
	9	- 5. Performance suggestions
	10	- 6. Available Downloads
	11
	12
	13	1. Introduction:
	14	This Linux driver supports Neterion's Xframe I PCI-X 1.0 and
	15	Xframe II PCI-X 2.0 adapters. It supports several features
	16	such as jumbo frames, MSI/MSI-X, checksum offloads, TSO, UFO and so on.
	17	See below for complete list of features.
	18	All features are supported for both IPv4 and IPv6.
	19
	20	2. Identifying the adapter/interface:
	21	a. Insert the adapter(s) in your system.
	22	b. Build and load driver
	23	# insmod s2io.ko
	24	c. View log messages
	25	# dmesg \| tail -40
	26	You will see messages similar to:
	27	eth3: Neterion Xframe I 10GbE adapter (rev 3), Version 2.0.9.1, Intr type INTA
	28	eth4: Neterion Xframe II 10GbE adapter (rev 2), Version 2.0.9.1, Intr type INTA
	29	eth4: Device is on 64 bit 133MHz PCIX(M1) bus
	30
	31	The above messages identify the adapter type(Xframe I/II), adapter revision,
	32	driver version, interface name(eth3, eth4), Interrupt type(INTA, MSI, MSI-X).
	33	In case of Xframe II, the PCI/PCI-X bus width and frequency are displayed
	34	as well.
	35
	36	To associate an interface with a physical adapter use "ethtool -p <ethX>".
	37	The corresponding adapter's LED will blink multiple times.
	38
	39	3. Features supported:
25985edc	40	a. Jumbo frames. Xframe I/II supports MTU up to 9600 bytes,
008a5b07	41	modifiable using ip command.
571de88b RA	42
	43	b. Offloads. Supports checksum offload(TCP/UDP/IP) on transmit
	44	and receive, TSO.
	45
	46	c. Multi-buffer receive mode. Scattering of packet across multiple
	47	buffers. Currently driver supports 2-buffer mode which yields
	48	significant performance improvement on certain platforms(SGI Altix,
	49	IBM xSeries).
	50
	51	d. MSI/MSI-X. Can be enabled on platforms which support this feature
25985edc	52	(IA64, Xeon) resulting in noticeable performance improvement(up to 7%
571de88b RA	53	on certain platforms).
571de88b RA	54
6a3d8aa4	55	e. Statistics. Comprehensive MAC-level and software statistics displayed
571de88b RA	56	using "ethtool -S" option.
571de88b RA	57
6a3d8aa4	58	f. Multi-FIFO/Ring. Supports up to 8 transmit queues and receive rings,
571de88b RA	59	with multiple steering options.
	60
	61	4. Command line parameters
	62	a. tx_fifo_num
	63	Number of transmit queues
	64	Valid range: 1-8
	65	Default: 1
	66
	67	b. rx_ring_num
	68	Number of receive rings
	69	Valid range: 1-8
	70	Default: 1
	71
	72	c. tx_fifo_len
	73	Size of each transmit queue
	74	Valid range: Total length of all queues should not exceed 8192
	75	Default: 4096
	76
	77	d. rx_ring_sz
	78	Size of each receive ring(in 4K blocks)
	79	Valid range: Limited by memory on system
	80	Default: 30
	81
	82	e. intr_type
f471f923 MS	83	Specifies interrupt type. Possible values 0(INTA), 2(MSI-X)
	84	Valid values: 0, 2
	85	Default: 2
571de88b RA	86
	87	5. Performance suggestions
	88	General:
	89	a. Set MTU to maximum(9000 for switch setup, 9600 in back-to-back configuration)
	90	b. Set TCP windows size to optimal value.
	91	For instance, for MTU=1500 a value of 210K has been observed to result in
	92	good performance.
	93	# sysctl -w net.ipv4.tcp_rmem="210000 210000 210000"
	94	# sysctl -w net.ipv4.tcp_wmem="210000 210000 210000"
	95	For MTU=9000, TCP window size of 10 MB is recommended.
	96	# sysctl -w net.ipv4.tcp_rmem="10000000 10000000 10000000"
	97	# sysctl -w net.ipv4.tcp_wmem="10000000 10000000 10000000"
	98
	99	Transmit performance:
	100	a. By default, the driver respects BIOS settings for PCI bus parameters.
	101	However, you may want to experiment with PCI bus parameters
	102	max-split-transactions(MOST) and MMRBC (use setpci command).
	103	A MOST value of 2 has been found optimal for Opterons and 3 for Itanium.
	104	It could be different for your hardware.
	105	Set MMRBC to 4K**.
	106
	107	For example you can set
	108	For opteron
	109	#setpci -d 17d5:* 62=1d
	110	For Itanium
	111	#setpci -d 17d5:* 62=3d
	112
	113	For detailed description of the PCI registers, please see Xframe User Guide.
	114
	115	b. Ensure Transmit Checksum offload is enabled. Use ethtool to set/verify this
	116	parameter.
	117	c. Turn on TSO(using "ethtool -K")
	118	# ethtool -K <ethX> tso on
	119
	120	Receive performance:
	121	a. By default, the driver respects BIOS settings for PCI bus parameters.
	122	However, you may want to set PCI latency timer to 248.
	123	#setpci -d 17d5:* LATENCY_TIMER=f8
	124	For detailed description of the PCI registers, please see Xframe User Guide.
	125	b. Use 2-buffer mode. This results in large performance boost on
670e9f34	126	certain platforms(eg. SGI Altix, IBM xSeries).
571de88b RA	127	c. Ensure Receive Checksum offload is enabled. Use "ethtool -K ethX" command to
	128	set/verify this option.
	129	d. Enable NAPI feature(in kernel configuration Device Drivers ---> Network
	130	device support ---> Ethernet (10000 Mbit) ---> S2IO 10Gbe Xframe NIC) to
	131	bring down CPU utilization.
	132
	133	** For AMD opteron platforms with 8131 chipset, MMRBC=1 and MOST=1 are
	134	recommended as safe parameters.
	135	For more information, please review the AMD8131 errata at
0ea6e611 JM	136	http://vip.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/
0ea6e611 JM	137	26310_AMD-8131_HyperTransport_PCI-X_Tunnel_Revision_Guide_rev_3_18.pdf
571de88b	138
c0589fa7	139	6. Support
571de88b	140	For further support please contact either your 10GbE Xframe NIC vendor (IBM,
c0589fa7	141	HP, SGI etc.)