]>
Commit | Line | Data |
---|---|---|
9f95a23c | 1 | |
11fdf7f2 TL |
2 | # An Overview of SPDK Applications {#app_overview} |
3 | ||
4 | SPDK is primarily a development kit that delivers libraries and header files for | |
5 | use in other applications. However, SPDK also contains a number of applications. | |
6 | These applications are primarily used to test the libraries, but many are full | |
7 | featured and high quality. The major applications in SPDK are: | |
8 | ||
9 | - @ref iscsi | |
10 | - @ref nvmf | |
11 | - @ref vhost | |
12 | - SPDK Target (a unified application combining the above three) | |
13 | ||
14 | There are also a number of tools and examples in the `examples` directory. | |
15 | ||
16 | The SPDK targets are all based on a common framework so they have much in | |
17 | common. The framework defines a concept called a `subsystem` and all | |
18 | functionality is implemented in various subsystems. Subsystems have a unified | |
19 | initialization and teardown path. | |
20 | ||
21 | # Configuring SPDK Applications {#app_config} | |
22 | ||
23 | ## Command Line Parameters {#app_cmd_line_args} | |
24 | ||
25 | The SPDK application framework defines a set of base command line flags for all | |
26 | applications that use it. Specific applications may implement additional flags. | |
27 | ||
28 | Param | Long Param | Type | Default | Description | |
29 | -------- | ---------------------- | -------- | ---------------------- | ----------- | |
30 | -c | --config | string | | @ref cmd_arg_config_file | |
31 | -d | --limit-coredump | flag | false | @ref cmd_arg_limit_coredump | |
32 | -e | --tpoint-group-mask | integer | 0x0 | @ref cmd_arg_limit_tpoint_group_mask | |
33 | -g | --single-file-segments | flag | | @ref cmd_arg_single_file_segments | |
34 | -h | --help | flag | | show all available parameters and exit | |
35 | -i | --shm-id | integer | | @ref cmd_arg_multi_process | |
36 | -m | --cpumask | CPU mask | 0x1 | application @ref cpu_mask | |
37 | -n | --mem-channels | integer | all channels | number of memory channels used for DPDK | |
38 | -p | --master-core | integer | first core in CPU mask | master (primary) core for DPDK | |
39 | -r | --rpc-socket | string | /var/tmp/spdk.sock | RPC listen address | |
40 | -s | --mem-size | integer | all hugepage memory | @ref cmd_arg_memory_size | |
9f95a23c | 41 | | | --silence-noticelog | flag | | disable notice level logging to `stderr` |
11fdf7f2 | 42 | -u | --no-pci | flag | | @ref cmd_arg_disable_pci_access. |
9f95a23c | 43 | | | --wait-for-rpc | flag | | @ref cmd_arg_deferred_initialization |
11fdf7f2 TL |
44 | -B | --pci-blacklist | B:D:F | | @ref cmd_arg_pci_blacklist_whitelist. |
45 | -W | --pci-whitelist | B:D:F | | @ref cmd_arg_pci_blacklist_whitelist. | |
46 | -R | --huge-unlink | flag | | @ref cmd_arg_huge_unlink | |
9f95a23c TL |
47 | | | --huge-dir | string | the first discovered | allocate hugepages from a specific mount |
48 | -L | --logflag | string | | @ref cmd_arg_debug_log_flags | |
11fdf7f2 | 49 | |
11fdf7f2 TL |
50 | ### Configuration file {#cmd_arg_config_file} |
51 | ||
52 | Historically, the SPDK applications were configured using a configuration file. | |
53 | This is still supported, but is considered deprecated in favor of JSON RPC | |
54 | configuration. See @ref jsonrpc for details. | |
55 | ||
56 | Note that `--config` and `--wait-for-rpc` cannot be used at the same time. | |
57 | ||
58 | ### Limit coredump {#cmd_arg_limit_coredump} | |
59 | ||
60 | By default, an SPDK application will set resource limits for core file sizes | |
61 | to RLIM_INFINITY. Specifying `--limit-coredump` will not set the resource limits. | |
62 | ||
63 | ### Tracepoint group mask {#cmd_arg_limit_tpoint_group_mask} | |
64 | ||
65 | SPDK has an experimental low overhead tracing framework. Tracepoints in this | |
66 | framework are organized into tracepoint groups. By default, all tracepoint | |
67 | groups are disabled. `--tpoint-group-mask` can be used to enable a specific | |
68 | subset of tracepoint groups in the application. | |
69 | ||
70 | Note: Additional documentation on the tracepoint framework is in progress. | |
71 | ||
72 | ### Deferred initialization {#cmd_arg_deferred_initialization} | |
73 | ||
74 | SPDK applications progress through a set of states beginning with `STARTUP` and | |
75 | ending with `RUNTIME`. | |
76 | ||
77 | If the `--wait-for-rpc` parameter is provided SPDK will pause just before starting | |
f67539c2 TL |
78 | framework initialization. This state is called `STARTUP`. The JSON RPC server is |
79 | ready but only a small subset of commands are available to set up initialization | |
11fdf7f2 TL |
80 | parameters. Those parameters can't be changed after the SPDK application enters |
81 | `RUNTIME` state. When the client finishes configuring the SPDK subsystems it | |
f67539c2 TL |
82 | needs to issue the @ref rpc_framework_start_init RPC command to begin the |
83 | initialization process. After `rpc_framework_start_init` returns `true` SPDK | |
11fdf7f2 TL |
84 | will enter the `RUNTIME` state and the list of available commands becomes much |
85 | larger. | |
86 | ||
87 | To see which RPC methods are available in the current state, issue the | |
9f95a23c | 88 | `rpc_get_methods` with the parameter `current` set to `true`. |
11fdf7f2 TL |
89 | |
90 | For more details see @ref jsonrpc documentation. | |
91 | ||
92 | ### Create just one hugetlbfs file {#cmd_arg_single_file_segments} | |
93 | ||
94 | Instead of creating one hugetlbfs file per page, this option makes SPDK create | |
95 | one file per hugepages per socket. This is needed for @ref virtio to be used | |
96 | with more than 8 hugepages. See @ref virtio_2mb. | |
97 | ||
98 | ### Multi process mode {#cmd_arg_multi_process} | |
99 | ||
100 | When `--shm-id` is specified, the application is started in multi-process mode. | |
101 | Applications using the same shm-id share their memory and | |
102 | [NVMe devices](@ref nvme_multi_process). The first app to start with a given id | |
103 | becomes a primary process, with the rest, called secondary processes, only | |
104 | attaching to it. When the primary process exits, the secondary ones continue to | |
105 | operate, but no new processes can be attached at this point. All processes within | |
106 | the same shm-id group must use the same | |
107 | [--single-file-segments setting](@ref cmd_arg_single_file_segments). | |
108 | ||
109 | ### Memory size {#cmd_arg_memory_size} | |
110 | ||
111 | Total size of the hugepage memory to reserve. If DPDK env layer is used, it will | |
112 | reserve memory from all available hugetlbfs mounts, starting with the one with | |
113 | the highest page size. This option accepts a number of bytes with a possible | |
114 | binary prefix, e.g. 1024, 1024M, 1G. The default unit is megabyte. | |
115 | ||
9f95a23c TL |
116 | Starting with DPDK 18.05.1, it's possible to reserve hugepages at runtime, meaning |
117 | that SPDK application can be started with 0 pre-reserved memory. Unlike hugepages | |
118 | pre-reserved at the application startup, the hugepages reserved at runtime will be | |
119 | released to the system as soon as they're no longer used. | |
120 | ||
11fdf7f2 TL |
121 | ### Disable PCI access {#cmd_arg_disable_pci_access} |
122 | ||
123 | If SPDK is run with PCI access disabled it won't detect any PCI devices. This | |
124 | includes primarily NVMe and IOAT devices. Also, the VFIO and UIO kernel modules | |
125 | are not required in this mode. | |
126 | ||
127 | ### PCI address blacklist and whitelist {#cmd_arg_pci_blacklist_whitelist} | |
128 | ||
129 | If blacklist is used, then all devices with the provided PCI address will be | |
130 | ignored. If a whitelist is used, only whitelisted devices will be probed. | |
131 | `-B` or `-W` can be used more than once, but cannot be mixed together. That is, | |
132 | `-B` and `-W` cannot be used at the same time. | |
133 | ||
134 | ### Unlink hugepage files after initialization {#cmd_arg_huge_unlink} | |
135 | ||
136 | By default, each DPDK-based application tries to remove any orphaned hugetlbfs | |
137 | files during its initialization. This option removes hugetlbfs files of the current | |
138 | process as soon as they're created, but is not compatible with `--shm-id`. | |
139 | ||
140 | ### Debug log {#cmd_arg_debug_log_flags} | |
141 | ||
142 | Enable a specific debug log type. This option can be used more than once. A list of | |
9f95a23c | 143 | all available types is provided in the `--help` output, with `--logflag all` |
11fdf7f2 TL |
144 | enabling all of them. Debug logs are only available in debug builds of SPDK. |
145 | ||
146 | ## CPU mask {#cpu_mask} | |
147 | ||
148 | Whenever the `CPU mask` is mentioned it is a string in one of the following formats: | |
149 | ||
150 | - Case insensitive hexadecimal string with or without "0x" prefix. | |
151 | - Comma separated list of CPUs or list of CPU ranges. Use '-' to define range. | |
152 | ||
153 | ### Example | |
154 | ||
155 | The following CPU masks are equal and correspond to CPUs 0, 1, 2, 8, 9, 10, 11 and 12: | |
156 | ||
157 | ~~~ | |
158 | 0x1f07 | |
159 | 0x1F07 | |
160 | 1f07 | |
161 | [0,1,2,8-12] | |
162 | [0, 1, 2, 8, 9, 10, 11, 12] | |
163 | ~~~ |