]>
Commit | Line | Data |
---|---|---|
aa039b0f DM |
1 | include::attributes.txt[] |
2 | [[chapter-storage]] | |
3 | ifdef::manvolnum[] | |
4 | PVE({manvolnum}) | |
5 | ================ | |
6 | ||
7 | NAME | |
8 | ---- | |
9 | ||
10 | pvesm - Proxmox VE Storage Manager | |
11 | ||
12 | ||
13 | SYNOPSYS | |
14 | -------- | |
15 | ||
16 | include::pvesm.1-synopsis.adoc[] | |
17 | ||
18 | DESCRIPTION | |
19 | ----------- | |
20 | endif::manvolnum[] | |
21 | ||
22 | ifndef::manvolnum[] | |
23 | {pve} Storage | |
24 | ============= | |
25 | endif::manvolnum[] | |
26 | ||
27 | The {pve} storage model is very flexible. Virtual machine images | |
28 | can either be stored on one or several local storages, or on shared | |
29 | storage like NFS or iSCSI (NAS, SAN). There are no limits, and you may | |
30 | configure as many storage pools as you like. You can use all | |
31 | storage technologies available for Debian Linux. | |
32 | ||
33 | One major benefit of storing VMs on shared storage is the ability to | |
34 | live-migrate running machines without any downtime, as all nodes in | |
35 | the cluster have direct access to VM disk images. There is no need to | |
36 | copy VM image data, so live migration is very fast in that case. | |
37 | ||
38 | The storage library (package 'libpve-storage-perl') uses a flexible | |
39 | plugin system to provide a common interface to all storage types. This | |
40 | can be easily adopted to include further storage types in future. | |
41 | ||
42 | ||
43 | Storage Types | |
44 | ------------- | |
45 | ||
46 | There are basically two different classes of storage types: | |
47 | ||
48 | Block level storage:: | |
49 | ||
50 | Allows to store large 'raw' images. It is usually not possible to store | |
51 | other files (ISO, backups, ..) on such storage types. Most modern | |
52 | block level storage implementations support snapshots and clones. | |
53 | RADOS, Sheepdog and DRBD are distributed systems, replicating storage | |
54 | data to different nodes. | |
55 | ||
56 | File level storage:: | |
57 | ||
58 | They allow access to a full featured (POSIX) file system. They are | |
59 | more flexible, and allows you to store any content type. ZFS is | |
60 | probably the most advanced system, and it has full support for | |
61 | snapshots and clones. | |
62 | ||
63 | ||
64 | .Available storage types | |
65 | [width="100%",cols="<d,1*m,4*d",options="header"] | |
66 | |=========================================================== | |
67 | |Description |PVE type |Level |Shared|Snapshots|Stable | |
68 | |ZFS (local) |zfspool |file |no |yes |yes | |
69 | |Directory |dir |file |no |no |yes | |
70 | |NFS |nfs |file |yes |no |yes | |
71 | |GlusterFS |glusterfs |file |yes |no |yes | |
72 | |LVM |lvm |block |no |no |yes | |
73 | |LVM-thin |lvmthin |block |no |yes |beta | |
74 | |iSCSI/kernel |iscsi |block |yes |no |yes | |
75 | |iSCSI/libiscsi |iscsidirect |block |yes |no |yes | |
76 | |Ceph/RBD |rbd |block |yes |yes |yes | |
77 | |Sheepdog |sheepdog |block |yes |yes |beta | |
78 | |DRBD9 |drbd |block |yes |yes |beta | |
79 | |ZFS over iSCSI |zfs |block |yes |yes |yes | |
80 | |========================================================= | |
81 | ||
82 | TIP: It is possible to use LVM on top of an iSCSI storage. That way | |
83 | you get a 'shared' LVM storage. | |
84 | ||
85 | Storage Configuration | |
86 | --------------------- | |
87 | ||
88 | All {pve} related storage configuration is stored within a single text | |
89 | file at '/etc/pve/storage.cfg'. As this file is within '/etc/pve/', it | |
90 | gets automatically distributed to all cluster nodes. So all nodes | |
91 | share the same storage configuration. | |
92 | ||
93 | Sharing storage configuration make perfect sense for shared storage, | |
94 | because the same 'shared' storage is accessible from all nodes. But is | |
95 | also useful for local storage types. In this case such local storage | |
96 | is available on all nodes, but it is physically different and can have | |
97 | totally different content. | |
98 | ||
99 | Storage Pools | |
100 | ~~~~~~~~~~~~~ | |
101 | ||
102 | Each storage pool has a `<type>`, and is uniquely identified by its `<STORAGE_ID>`. A pool configuration looks like this: | |
103 | ||
104 | ---- | |
105 | <type>: <STORAGE_ID> | |
106 | <property> <value> | |
107 | <property> <value> | |
108 | ... | |
109 | ---- | |
110 | ||
111 | NOTE: There is one special local storage pool named `local`. It refers to | |
112 | directory '/var/lib/vz' and is automatically generated at installation | |
113 | time. | |
114 | ||
115 | The `<type>: <STORAGE_ID>` line starts the pool definition, which is then | |
116 | followed by a list of properties. Most properties have values, but some of them comes | |
117 | with reasonable default. In that case you can omit the value. | |
118 | ||
119 | .Default storage configuration ('/etc/pve/storage.cfg') | |
120 | ==== | |
121 | dir: local | |
122 | path /var/lib/vz | |
123 | content backup,iso,vztmpl,images,rootdir | |
124 | maxfiles 3 | |
125 | ==== | |
126 | ||
127 | Common Storage Properties | |
128 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | |
129 | ||
130 | A few storage properties are common among differenty storage types. | |
131 | ||
132 | nodes:: | |
133 | ||
134 | List of cluster node names where this storage is | |
135 | usable/accessible. One can use this property to restrict storage | |
136 | access to a limited set of nodes. | |
137 | ||
138 | content:: | |
139 | ||
140 | A storage can support several content types, for example virtual disk | |
141 | images, cdrom iso images, container templates or container root | |
142 | directories. Not all storage types supports all content types. One can set | |
143 | this property to select for what this storage is used for. | |
144 | ||
145 | images::: | |
146 | ||
147 | KVM-Qemu VM images. | |
148 | ||
149 | rootdir::: | |
150 | ||
151 | Allow to store Container data. | |
152 | ||
153 | vztmpl::: | |
154 | ||
155 | Container templates. | |
156 | ||
157 | backup::: | |
158 | ||
159 | Backup files ('vzdump'). | |
160 | ||
161 | iso::: | |
162 | ||
163 | ISO images | |
164 | ||
165 | shared:: | |
166 | ||
167 | Mark storage as shared. | |
168 | ||
169 | disable:: | |
170 | ||
171 | You can use this flag to disable the storage completely. | |
172 | ||
173 | maxfiles:: | |
174 | ||
175 | Maximal number of backup files per VM. Use `0` for unlimted. | |
176 | ||
177 | format:: | |
178 | ||
179 | Default image format (`raw|qcow2|vmdk`) | |
180 | ||
181 | ||
182 | WARNING: It is not advisable to use the same storage pool on different | |
183 | {pve} clusters. Some storage operation needs exclusive access to the | |
184 | storage, so proper locking is required. While this is implemented | |
185 | within an cluster, it does not work between different clusters. | |
186 | ||
187 | ||
188 | Volumes | |
189 | ------- | |
190 | ||
191 | We use a special notation to address storage data. When you allocate | |
192 | data from a storage pool, it returns such volume identifier. A volume | |
193 | is identified by the `<STORAGE_ID>`, followed by a storage type | |
194 | dependent volume name, separated by colon. A valid `<VOLUME_ID>` looks | |
195 | like: | |
196 | ||
197 | local:230/example-image.raw | |
198 | ||
199 | local:iso/debian-501-amd64-netinst.iso | |
200 | ||
201 | local:vztmpl/debian-5.0-joomla_1.5.9-1_i386.tar.gz | |
202 | ||
203 | iscsi-storage:0.0.2.scsi-14f504e46494c4500494b5042546d2d646744372d31616d61 | |
204 | ||
205 | To get the filesystem path for a `<VOLUME_ID>` use: | |
206 | ||
207 | pvesm path <VOLUME_ID> | |
208 | ||
209 | Volume Ownership | |
210 | ~~~~~~~~~~~~~~~~ | |
211 | ||
212 | There exists an ownership relation for 'image' type volumes. Each such | |
213 | volume is owned by a VM or Container. For example volume | |
214 | `local:230/example-image.raw` is owned by VM 230. Most storage | |
215 | backends encodes this ownership information into the volume name. | |
216 | ||
217 | When you remove a VM or Container, the system also remove all | |
218 | associated volumes which are owned by that VM or Container. | |
219 | ||
220 | ||
221 | Using the Command Line Interface | |
222 | -------------------------------- | |
223 | ||
224 | I think it is required to understand the concept behind storage pools | |
225 | and volume identifier, but in real life, you are not forced to do any | |
226 | of those low level operations on the command line. Normally, | |
227 | allocation and removal of volumes is done by the VM and Container | |
228 | management tools. | |
229 | ||
230 | Nevertheless, there is a command line tool called 'pvesm' ({pve} | |
231 | storage manager), which is able to perform common storage management | |
232 | tasks. | |
233 | ||
234 | ||
235 | Examples | |
236 | ~~~~~~~~ | |
237 | ||
238 | Add storage pools | |
239 | ||
240 | pvesm add <TYPE> <STORAGE_ID> <OPTIONS> | |
241 | pvesm add dir <STORAGE_ID> --path <PATH> | |
242 | pvesm add nfs <STORAGE_ID> --path <PATH> --server <SERVER> --export <EXPORT> | |
243 | pvesm add lvm <STORAGE_ID> --vgname <VGNAME> | |
244 | pvesm add iscsi <STORAGE_ID> --portal <HOST[:PORT]> --target <TARGET> | |
245 | ||
246 | Disable storage pools | |
247 | ||
248 | pvesm set <STORAGE_ID> --disable 1 | |
249 | ||
250 | Enable storage pools | |
251 | ||
252 | pvesm set <STORAGE_ID> --disable 0 | |
253 | ||
254 | Change/set storage options | |
255 | ||
256 | pvesm set <STORAGE_ID> <OPTIONS> | |
257 | pvesm set <STORAGE_ID> --shared 1 | |
258 | pvesm set local --format qcow2 | |
259 | pvesm set <STORAGE_ID> --content iso | |
260 | ||
261 | Remove storage pools. This does not delete any data, and does not | |
262 | disconnect or unmount anything. It just removes the storage | |
263 | configuration. | |
264 | ||
265 | pvesm remove <STORAGE_ID> | |
266 | ||
267 | Allocate volumes | |
268 | ||
269 | pvesm alloc <STORAGE_ID> <VMID> <name> <size> [--format <raw|qcow2>] | |
270 | ||
271 | Allocate a 4G volume in local storage. The name is auto-generated if | |
272 | you pass an empty string as `<name>` | |
273 | ||
274 | pvesm alloc local <VMID> '' 4G | |
275 | ||
276 | Free volumes | |
277 | ||
278 | pvesm free <VOLUME_ID> | |
279 | ||
280 | WARNING: This really destroys all volume data. | |
281 | ||
282 | List storage status | |
283 | ||
284 | pvesm status | |
285 | ||
286 | List storage contents | |
287 | ||
288 | pvesm list <STORAGE_ID> [--vmid <VMID>] | |
289 | ||
290 | List volumes allocated by VMID | |
291 | ||
292 | pvesm list <STORAGE_ID> --vmid <VMID> | |
293 | ||
294 | List iso images | |
295 | ||
296 | pvesm list <STORAGE_ID> --iso | |
297 | ||
298 | List container templates | |
299 | ||
300 | pvesm list <STORAGE_ID> --vztmpl | |
301 | ||
302 | Show filesystem path for a volume | |
303 | ||
304 | pvesm path <VOLUME_ID> | |
305 | ||
306 | // backend documentation | |
307 | ||
308 | include::pve-storage-dir.adoc[] | |
309 | ||
310 | include::pve-storage-nfs.adoc[] | |
311 | ||
312 | include::pve-storage-glusterfs.adoc[] | |
313 | ||
314 | include::pve-storage-zfspool.adoc[] | |
315 | ||
316 | include::pve-storage-lvm.adoc[] | |
317 | ||
318 | include::pve-storage-iscsi.adoc[] | |
319 | ||
320 | include::pve-storage-iscsidirect.adoc[] | |
321 | ||
322 | include::pve-storage-rbd.adoc[] | |
323 | ||
324 | ||
325 | ifdef::manvolnum[] | |
326 | include::pve-copyright.adoc[] | |
327 | endif::manvolnum[] | |
328 |