]>
Commit | Line | Data |
---|---|---|
1da177e4 LT |
1 | SYSFS FILES |
2 | ||
3 | For each InfiniBand device, the InfiniBand drivers create the | |
4 | following files under /sys/class/infiniband/<device name>: | |
5 | ||
6 | node_type - Node type (CA, switch or router) | |
7 | node_guid - Node GUID | |
8 | sys_image_guid - System image GUID | |
9 | ||
10 | In addition, there is a "ports" subdirectory, with one subdirectory | |
11 | for each port. For example, if mthca0 is a 2-port HCA, there will | |
12 | be two directories: | |
13 | ||
14 | /sys/class/infiniband/mthca0/ports/1 | |
15 | /sys/class/infiniband/mthca0/ports/2 | |
16 | ||
17 | (A switch will only have a single "0" subdirectory for switch port | |
18 | 0; no subdirectory is created for normal switch ports) | |
19 | ||
20 | In each port subdirectory, the following files are created: | |
21 | ||
22 | cap_mask - Port capability mask | |
23 | lid - Port LID | |
24 | lid_mask_count - Port LID mask count | |
25 | rate - Port data rate (active width * active speed) | |
26 | sm_lid - Subnet manager LID for port's subnet | |
27 | sm_sl - Subnet manager SL for port's subnet | |
28 | state - Port state (DOWN, INIT, ARMED, ACTIVE or ACTIVE_DEFER) | |
29 | phys_state - Port physical state (Sleep, Polling, LinkUp, etc) | |
30 | ||
31 | There is also a "counters" subdirectory, with files | |
32 | ||
33 | VL15_dropped | |
34 | excessive_buffer_overrun_errors | |
35 | link_downed | |
36 | link_error_recovery | |
37 | local_link_integrity_errors | |
38 | port_rcv_constraint_errors | |
39 | port_rcv_data | |
40 | port_rcv_errors | |
41 | port_rcv_packets | |
42 | port_rcv_remote_physical_errors | |
43 | port_rcv_switch_relay_errors | |
44 | port_xmit_constraint_errors | |
45 | port_xmit_data | |
46 | port_xmit_discards | |
47 | port_xmit_packets | |
48 | symbol_error | |
49 | ||
50 | Each of these files contains the corresponding value from the port's | |
51 | Performance Management PortCounters attribute, as described in | |
52 | section 16.1.3.5 of the InfiniBand Architecture Specification. | |
53 | ||
54 | The "pkeys" and "gids" subdirectories contain one file for each | |
55 | entry in the port's P_Key or GID table respectively. For example, | |
56 | ports/1/pkeys/10 contains the value at index 10 in port 1's P_Key | |
57 | table. | |
58 | ||
b40f4757 CL |
59 | There is an optional "hw_counters" subdirectory that may be under either |
60 | the parent device or the port subdirectories or both. If present, | |
61 | there are a list of counters provided by the hardware. They may match | |
62 | some of the counters in the counters directory, but they often include | |
63 | many other counters. In addition to the various counters, there will | |
64 | be a file named "lifespan" that configures how frequently the core | |
65 | should update the counters when they are being accessed (counters are | |
66 | not updated if they are not being accessed). The lifespan is in milli- | |
67 | seconds and defaults to 10 unless set to something else by the driver. | |
68 | Users may echo a value between 0 - 10000 to the lifespan file to set | |
69 | the length of time between updates in milliseconds. | |
70 | ||
1da177e4 LT |
71 | MTHCA |
72 | ||
73 | The Mellanox HCA driver also creates the files: | |
74 | ||
75 | hw_rev - Hardware revision number | |
76 | fw_ver - Firmware version | |
77 | hca_type - HCA type: "MT23108", "MT25208 (MT23108 compat mode)", | |
78 | or "MT25208" | |
77241056 MM |
79 | |
80 | HFI1 | |
81 | ||
82 | The hfi1 driver also creates these additional files: | |
83 | ||
84 | hw_rev - hardware revision | |
85 | board_id - manufacturing board id | |
86 | tempsense - thermal sense information | |
87 | serial - board serial number | |
88 | nfreectxts - number of free user contexts | |
89 | nctxts - number of allowed contexts (PSM2) | |
90 | chip_reset - diagnostic (root only) | |
91 | boardversion - board version | |
c642cc07 TS |
92 | |
93 | sdma<N>/ - one directory per sdma engine (0 - 15) | |
94 | sdma<N>/cpu_list - read-write, list of cpus for user-process to sdma | |
95 | engine assignment. | |
96 | sdma<N>/vl - read-only, vl the sdma engine maps to. | |
97 | ||
98 | The new interface will give the user control on the affinity settings | |
99 | for the hfi1 device. | |
100 | As an example, to set an sdma engine irq affinity and thread affinity | |
101 | of a user processes to use the sdma engine, which is "near" in terms | |
102 | of NUMA configuration, or physical cpu location, the user will do: | |
103 | ||
104 | echo "3" > /proc/irq/<N>/smp_affinity_list | |
105 | echo "4-7" > /sys/devices/.../sdma3/cpu_list | |
106 | cat /sys/devices/.../sdma3/vl | |
107 | 0 | |
108 | echo "8" > /proc/irq/<M>/smp_affinity_list | |
109 | echo "9-12" > /sys/devices/.../sdma4/cpu_list | |
110 | cat /sys/devices/.../sdma4/vl | |
111 | 1 | |
112 | ||
113 | to make sure that when a process runs on cpus 4,5,6, or 7, | |
114 | and uses vl=0, then sdma engine 3 is selected by the driver, | |
115 | and also the interrupt of the sdma engine 3 is steered to cpu 3. | |
116 | Similarly, when a process runs on cpus 9,10,11, or 12 and sets vl=1, | |
117 | then engine 4 will be selected and the irq of the sdma engine 4 is | |
118 | steered to cpu 8. | |
119 | This assumes that in the above N is the irq number of "sdma3", | |
120 | and M is irq number of "sdma4" in the /proc/interrupts file. | |
121 | ||
77241056 | 122 | ports/1/ |
6c9e50f8 | 123 | CCMgtA/ |
77241056 MM |
124 | cc_settings_bin - CCA tables used by PSM2 |
125 | cc_table_bin | |
6c9e50f8 | 126 | cc_prescan - enable prescaning for faster BECN response |
77241056 MM |
127 | sc2v/ - 32 files (0 - 31) used to translate sl->vl |
128 | sl2sc/ - 32 files (0 - 31) used to translate sl->sc | |
129 | vl2mtu/ - 16 (0 - 15) files used to determine MTU for vl |