]>
Commit | Line | Data |
---|---|---|
9f95a23c | 1 | |
11fdf7f2 TL |
2 | # An Overview of SPDK Applications {#app_overview} |
3 | ||
4 | SPDK is primarily a development kit that delivers libraries and header files for | |
5 | use in other applications. However, SPDK also contains a number of applications. | |
6 | These applications are primarily used to test the libraries, but many are full | |
7 | featured and high quality. The major applications in SPDK are: | |
8 | ||
9 | - @ref iscsi | |
10 | - @ref nvmf | |
11 | - @ref vhost | |
12 | - SPDK Target (a unified application combining the above three) | |
13 | ||
14 | There are also a number of tools and examples in the `examples` directory. | |
15 | ||
16 | The SPDK targets are all based on a common framework so they have much in | |
17 | common. The framework defines a concept called a `subsystem` and all | |
18 | functionality is implemented in various subsystems. Subsystems have a unified | |
19 | initialization and teardown path. | |
20 | ||
21 | # Configuring SPDK Applications {#app_config} | |
22 | ||
23 | ## Command Line Parameters {#app_cmd_line_args} | |
24 | ||
25 | The SPDK application framework defines a set of base command line flags for all | |
26 | applications that use it. Specific applications may implement additional flags. | |
27 | ||
28 | Param | Long Param | Type | Default | Description | |
29 | -------- | ---------------------- | -------- | ---------------------- | ----------- | |
30 | -c | --config | string | | @ref cmd_arg_config_file | |
31 | -d | --limit-coredump | flag | false | @ref cmd_arg_limit_coredump | |
32 | -e | --tpoint-group-mask | integer | 0x0 | @ref cmd_arg_limit_tpoint_group_mask | |
33 | -g | --single-file-segments | flag | | @ref cmd_arg_single_file_segments | |
34 | -h | --help | flag | | show all available parameters and exit | |
35 | -i | --shm-id | integer | | @ref cmd_arg_multi_process | |
36 | -m | --cpumask | CPU mask | 0x1 | application @ref cpu_mask | |
37 | -n | --mem-channels | integer | all channels | number of memory channels used for DPDK | |
38 | -p | --master-core | integer | first core in CPU mask | master (primary) core for DPDK | |
39 | -r | --rpc-socket | string | /var/tmp/spdk.sock | RPC listen address | |
40 | -s | --mem-size | integer | all hugepage memory | @ref cmd_arg_memory_size | |
9f95a23c | 41 | | | --silence-noticelog | flag | | disable notice level logging to `stderr` |
11fdf7f2 | 42 | -u | --no-pci | flag | | @ref cmd_arg_disable_pci_access. |
9f95a23c | 43 | | | --wait-for-rpc | flag | | @ref cmd_arg_deferred_initialization |
11fdf7f2 TL |
44 | -B | --pci-blacklist | B:D:F | | @ref cmd_arg_pci_blacklist_whitelist. |
45 | -W | --pci-whitelist | B:D:F | | @ref cmd_arg_pci_blacklist_whitelist. | |
46 | -R | --huge-unlink | flag | | @ref cmd_arg_huge_unlink | |
9f95a23c TL |
47 | | | --huge-dir | string | the first discovered | allocate hugepages from a specific mount |
48 | -L | --logflag | string | | @ref cmd_arg_debug_log_flags | |
11fdf7f2 TL |
49 | |
50 | ||
51 | ### Configuration file {#cmd_arg_config_file} | |
52 | ||
53 | Historically, the SPDK applications were configured using a configuration file. | |
54 | This is still supported, but is considered deprecated in favor of JSON RPC | |
55 | configuration. See @ref jsonrpc for details. | |
56 | ||
57 | Note that `--config` and `--wait-for-rpc` cannot be used at the same time. | |
58 | ||
59 | ### Limit coredump {#cmd_arg_limit_coredump} | |
60 | ||
61 | By default, an SPDK application will set resource limits for core file sizes | |
62 | to RLIM_INFINITY. Specifying `--limit-coredump` will not set the resource limits. | |
63 | ||
64 | ### Tracepoint group mask {#cmd_arg_limit_tpoint_group_mask} | |
65 | ||
66 | SPDK has an experimental low overhead tracing framework. Tracepoints in this | |
67 | framework are organized into tracepoint groups. By default, all tracepoint | |
68 | groups are disabled. `--tpoint-group-mask` can be used to enable a specific | |
69 | subset of tracepoint groups in the application. | |
70 | ||
71 | Note: Additional documentation on the tracepoint framework is in progress. | |
72 | ||
73 | ### Deferred initialization {#cmd_arg_deferred_initialization} | |
74 | ||
75 | SPDK applications progress through a set of states beginning with `STARTUP` and | |
76 | ending with `RUNTIME`. | |
77 | ||
78 | If the `--wait-for-rpc` parameter is provided SPDK will pause just before starting | |
79 | subsystem initialization. This state is called `STARTUP`. The JSON RPC server is | |
80 | ready but only a small subsystem of commands are available to set up initialization | |
81 | parameters. Those parameters can't be changed after the SPDK application enters | |
82 | `RUNTIME` state. When the client finishes configuring the SPDK subsystems it | |
83 | needs to issue the @ref rpc_start_subsystem_init RPC command to begin the | |
84 | initialization process. After `rpc_start_subsystem_init` returns `true` SPDK | |
85 | will enter the `RUNTIME` state and the list of available commands becomes much | |
86 | larger. | |
87 | ||
88 | To see which RPC methods are available in the current state, issue the | |
9f95a23c | 89 | `rpc_get_methods` with the parameter `current` set to `true`. |
11fdf7f2 TL |
90 | |
91 | For more details see @ref jsonrpc documentation. | |
92 | ||
93 | ### Create just one hugetlbfs file {#cmd_arg_single_file_segments} | |
94 | ||
95 | Instead of creating one hugetlbfs file per page, this option makes SPDK create | |
96 | one file per hugepages per socket. This is needed for @ref virtio to be used | |
97 | with more than 8 hugepages. See @ref virtio_2mb. | |
98 | ||
99 | ### Multi process mode {#cmd_arg_multi_process} | |
100 | ||
101 | When `--shm-id` is specified, the application is started in multi-process mode. | |
102 | Applications using the same shm-id share their memory and | |
103 | [NVMe devices](@ref nvme_multi_process). The first app to start with a given id | |
104 | becomes a primary process, with the rest, called secondary processes, only | |
105 | attaching to it. When the primary process exits, the secondary ones continue to | |
106 | operate, but no new processes can be attached at this point. All processes within | |
107 | the same shm-id group must use the same | |
108 | [--single-file-segments setting](@ref cmd_arg_single_file_segments). | |
109 | ||
110 | ### Memory size {#cmd_arg_memory_size} | |
111 | ||
112 | Total size of the hugepage memory to reserve. If DPDK env layer is used, it will | |
113 | reserve memory from all available hugetlbfs mounts, starting with the one with | |
114 | the highest page size. This option accepts a number of bytes with a possible | |
115 | binary prefix, e.g. 1024, 1024M, 1G. The default unit is megabyte. | |
116 | ||
9f95a23c TL |
117 | Starting with DPDK 18.05.1, it's possible to reserve hugepages at runtime, meaning |
118 | that SPDK application can be started with 0 pre-reserved memory. Unlike hugepages | |
119 | pre-reserved at the application startup, the hugepages reserved at runtime will be | |
120 | released to the system as soon as they're no longer used. | |
121 | ||
11fdf7f2 TL |
122 | ### Disable PCI access {#cmd_arg_disable_pci_access} |
123 | ||
124 | If SPDK is run with PCI access disabled it won't detect any PCI devices. This | |
125 | includes primarily NVMe and IOAT devices. Also, the VFIO and UIO kernel modules | |
126 | are not required in this mode. | |
127 | ||
128 | ### PCI address blacklist and whitelist {#cmd_arg_pci_blacklist_whitelist} | |
129 | ||
130 | If blacklist is used, then all devices with the provided PCI address will be | |
131 | ignored. If a whitelist is used, only whitelisted devices will be probed. | |
132 | `-B` or `-W` can be used more than once, but cannot be mixed together. That is, | |
133 | `-B` and `-W` cannot be used at the same time. | |
134 | ||
135 | ### Unlink hugepage files after initialization {#cmd_arg_huge_unlink} | |
136 | ||
137 | By default, each DPDK-based application tries to remove any orphaned hugetlbfs | |
138 | files during its initialization. This option removes hugetlbfs files of the current | |
139 | process as soon as they're created, but is not compatible with `--shm-id`. | |
140 | ||
141 | ### Debug log {#cmd_arg_debug_log_flags} | |
142 | ||
143 | Enable a specific debug log type. This option can be used more than once. A list of | |
9f95a23c | 144 | all available types is provided in the `--help` output, with `--logflag all` |
11fdf7f2 TL |
145 | enabling all of them. Debug logs are only available in debug builds of SPDK. |
146 | ||
147 | ## CPU mask {#cpu_mask} | |
148 | ||
149 | Whenever the `CPU mask` is mentioned it is a string in one of the following formats: | |
150 | ||
151 | - Case insensitive hexadecimal string with or without "0x" prefix. | |
152 | - Comma separated list of CPUs or list of CPU ranges. Use '-' to define range. | |
153 | ||
154 | ### Example | |
155 | ||
156 | The following CPU masks are equal and correspond to CPUs 0, 1, 2, 8, 9, 10, 11 and 12: | |
157 | ||
158 | ~~~ | |
159 | 0x1f07 | |
160 | 0x1F07 | |
161 | 1f07 | |
162 | [0,1,2,8-12] | |
163 | [0, 1, 2, 8, 9, 10, 11, 12] | |
164 | ~~~ |