pvesm.adoc

   1 [[chapter-storage]]
   2 ifdef::manvolnum[]
   3 PVE({manvolnum})
   4 ================
   5 include::attributes.txt[]
   6
   7 NAME
   8 ----
   9
  10 pvesm - Proxmox VE Storage Manager
  11
  12
  13 SYNOPSYS
  14 --------
  15
  16 include::pvesm.1-synopsis.adoc[]
  17
  18 DESCRIPTION
  19 -----------
  20 endif::manvolnum[]
  21
  22 ifndef::manvolnum[]
  23 {pve} Storage
  24 =============
  25 include::attributes.txt[]
  26 endif::manvolnum[]
  27
  28 The {pve} storage model is very flexible. Virtual machine images
  29 can either be stored on one or several local storages, or on shared
  30 storage like NFS or iSCSI (NAS, SAN). There are no limits, and you may
  31 configure as many storage pools as you like. You can use all
  32 storage technologies available for Debian Linux.
  33
  34 One major benefit of storing VMs on shared storage is the ability to
  35 live-migrate running machines without any downtime, as all nodes in
  36 the cluster have direct access to VM disk images. There is no need to
  37 copy VM image data, so live migration is very fast in that case.
  38
  39 The storage library (package 'libpve-storage-perl') uses a flexible
  40 plugin system to provide a common interface to all storage types. This
  41 can be easily adopted to include further storage types in future.
  42
  43
  44 Storage Types
  45 -------------
  46
  47 There are basically two different classes of storage types:
  48
  49 Block level storage::
  50
  51 Allows to store large 'raw' images. It is usually not possible to store
  52 other files (ISO, backups, ..) on such storage types. Most modern
  53 block level storage implementations support snapshots and clones.
  54 RADOS, Sheepdog and DRBD are distributed systems, replicating storage
  55 data to different nodes.
  56
  57 File level storage::
  58
  59 They allow access to a full featured (POSIX) file system. They are
  60 more flexible, and allows you to store any content type. ZFS is
  61 probably the most advanced system, and it has full support for
  62 snapshots and clones.
  63
  64
  65 .Available storage types
  66 [width="100%",cols="<d,1*m,4*d",options="header"]
  67 |===========================================================
  68 |Description    |PVE type    |Level |Shared|Snapshots|Stable
  69 |ZFS (local)    |zfspool     |file  |no    |yes      |yes
  70 |Directory      |dir         |file  |no    |no       |yes
  71 |NFS            |nfs         |file  |yes   |no       |yes
  72 |GlusterFS      |glusterfs   |file  |yes   |no       |yes
  73 |LVM            |lvm         |block |no    |no       |yes
  74 |LVM-thin       |lvmthin     |block |no    |yes      |yes
  75 |iSCSI/kernel   |iscsi       |block |yes   |no       |yes
  76 |iSCSI/libiscsi |iscsidirect |block |yes   |no       |yes
  77 |Ceph/RBD       |rbd         |block |yes   |yes      |yes
  78 |Sheepdog       |sheepdog    |block |yes   |yes      |beta
  79 |DRBD9          |drbd        |block |yes   |yes      |beta
  80 |ZFS over iSCSI |zfs         |block |yes   |yes      |yes
  81 |=========================================================
  82
  83 TIP: It is possible to use LVM on top of an iSCSI storage. That way
  84 you get a 'shared' LVM storage.
  85
  86 Storage Configuration
  87 ---------------------
  88
  89 All {pve} related storage configuration is stored within a single text
  90 file at '/etc/pve/storage.cfg'. As this file is within '/etc/pve/', it
  91 gets automatically distributed to all cluster nodes. So all nodes
  92 share the same storage configuration.
  93
  94 Sharing storage configuration make perfect sense for shared storage,
  95 because the same 'shared' storage is accessible from all nodes. But is
  96 also useful for local storage types. In this case such local storage
  97 is available on all nodes, but it is physically different and can have
  98 totally different content.
  99
 100 Storage Pools
 101 ~~~~~~~~~~~~~
 102
 103 Each storage pool has a `<type>`, and is uniquely identified by its `<STORAGE_ID>`. A pool configuration looks like this:
 104
 105 ----
 106 <type>: <STORAGE_ID>
 107         <property> <value>
 108         <property> <value>
 109         ...
 110 ----
 111
 112 NOTE: There is one special local storage pool named `local`. It refers to
 113 the directory '/var/lib/vz' and is automatically generated at installation
 114 time.
 115
 116 The `<type>: <STORAGE_ID>` line starts the pool definition, which is then
 117 followed by a list of properties. Most properties have values, but some of
 118 them come with reasonable default. In that case you can omit the value.
 119
 120 .Default storage configuration ('/etc/pve/storage.cfg')
 121 ----
 122 dir: local
 123         path /var/lib/vz
 124         content iso,vztmpl,backup
 125
 126 lvmthin: local-lvm
 127         thinpool data
 128         vgname pve
 129         content rootdir,images
 130 ----
 131
 132 Common Storage Properties
 133 ~~~~~~~~~~~~~~~~~~~~~~~~~
 134
 135 A few storage properties are common among different storage types.
 136
 137 nodes::
 138
 139 List of cluster node names where this storage is
 140 usable/accessible. One can use this property to restrict storage
 141 access to a limited set of nodes.
 142
 143 content::
 144
 145 A storage can support several content types, for example virtual disk
 146 images, cdrom iso images, container templates or container root
 147 directories. Not all storage types support all content types. One can set
 148 this property to select for what this storage is used for.
 149
 150 images:::
 151
 152 KVM-Qemu VM images.
 153
 154 rootdir:::
 155
 156 Allow to store container data.
 157
 158 vztmpl:::
 159
 160 Container templates.
 161
 162 backup:::
 163
 164 Backup files ('vzdump').
 165
 166 iso:::
 167
 168 ISO images
 169
 170 shared::
 171
 172 Mark storage as shared.
 173
 174 disable::
 175
 176 You can use this flag to disable the storage completely.
 177
 178 maxfiles::
 179
 180 Maximal number of backup files per VM. Use `0` for unlimted.
 181
 182 format::
 183
 184 Default image format (`raw|qcow2|vmdk`)
 185
 186
 187 WARNING: It is not advisable to use the same storage pool on different
 188 {pve} clusters. Some storage operation need exclusive access to the
 189 storage, so proper locking is required. While this is implemented
 190 within a cluster, it does not work between different clusters.
 191
 192
 193 Volumes
 194 -------
 195
 196 We use a special notation to address storage data. When you allocate
 197 data from a storage pool, it returns such a volume identifier. A volume
 198 is identified by the `<STORAGE_ID>`, followed by a storage type
 199 dependent volume name, separated by colon. A valid `<VOLUME_ID>` looks
 200 like:
 201
 202  local:230/example-image.raw
 203
 204  local:iso/debian-501-amd64-netinst.iso
 205
 206  local:vztmpl/debian-5.0-joomla_1.5.9-1_i386.tar.gz
 207
 208  iscsi-storage:0.0.2.scsi-14f504e46494c4500494b5042546d2d646744372d31616d61
 209
 210 To get the filesystem path for a `<VOLUME_ID>` use:
 211
 212  pvesm path <VOLUME_ID>
 213
 214 Volume Ownership
 215 ~~~~~~~~~~~~~~~~
 216
 217 There exists an ownership relation for 'image' type volumes. Each such
 218 volume is owned by a VM or Container. For example volume
 219 `local:230/example-image.raw` is owned by VM 230. Most storage
 220 backends encodes this ownership information into the volume name.
 221
 222 When you remove a VM or Container, the system also removes all
 223 associated volumes which are owned by that VM or Container.
 224
 225
 226 Using the Command Line Interface
 227 --------------------------------
 228
 229 It is recommended to familiarize yourself with the concept behind storage
 230 pools and volume identifiers, but in real life, you are not forced to do any
 231 of those low level operations on the command line. Normally,
 232 allocation and removal of volumes is done by the VM and Container
 233 management tools.
 234
 235 Nevertheless, there is a command line tool called 'pvesm' ({pve}
 236 storage manager), which is able to perform common storage management
 237 tasks.
 238
 239
 240 Examples
 241 ~~~~~~~~
 242
 243 Add storage pools
 244
 245  pvesm add <TYPE> <STORAGE_ID> <OPTIONS>
 246  pvesm add dir <STORAGE_ID> --path <PATH>
 247  pvesm add nfs <STORAGE_ID> --path <PATH> --server <SERVER> --export <EXPORT>
 248  pvesm add lvm <STORAGE_ID> --vgname <VGNAME>
 249  pvesm add iscsi <STORAGE_ID> --portal <HOST[:PORT]> --target <TARGET>
 250
 251 Disable storage pools
 252
 253  pvesm set <STORAGE_ID> --disable 1
 254
 255 Enable storage pools
 256
 257  pvesm set <STORAGE_ID> --disable 0
 258
 259 Change/set storage options
 260
 261  pvesm set <STORAGE_ID> <OPTIONS>
 262  pvesm set <STORAGE_ID> --shared 1
 263  pvesm set local --format qcow2
 264  pvesm set <STORAGE_ID> --content iso
 265
 266 Remove storage pools. This does not delete any data, and does not
 267 disconnect or unmount anything. It just removes the storage
 268 configuration.
 269
 270  pvesm remove <STORAGE_ID>
 271
 272 Allocate volumes
 273
 274  pvesm alloc <STORAGE_ID> <VMID> <name> <size> [--format <raw|qcow2>]
 275
 276 Allocate a 4G volume in local storage. The name is auto-generated if
 277 you pass an empty string as `<name>`
 278
 279  pvesm alloc local <VMID> '' 4G
 280
 281 Free volumes
 282
 283  pvesm free <VOLUME_ID>
 284
 285 WARNING: This really destroys all volume data.
 286
 287 List storage status
 288
 289  pvesm status
 290
 291 List storage contents
 292
 293  pvesm list <STORAGE_ID> [--vmid <VMID>]
 294
 295 List volumes allocated by VMID
 296
 297  pvesm list <STORAGE_ID> --vmid <VMID>
 298
 299 List iso images
 300
 301  pvesm list <STORAGE_ID> --iso
 302
 303 List container templates
 304
 305  pvesm list <STORAGE_ID> --vztmpl
 306
 307 Show filesystem path for a volume
 308
 309  pvesm path <VOLUME_ID>
 310
 311 ifdef::wiki[]
 312
 313 See Also
 314 --------
 315
 316 * link:/index.php/Storage:_Directory[Storage: Directory]
 317
 318 * link:/index.php/Storage:_GlusterFS[Storage: GlusterFS]
 319
 320 * link:/index.php/Storage:_User_Mode_iSCSI[Storage: User Mode iSCSI]
 321
 322 * link:/index.php/Storage:_iSCSI[Storage: iSCSI]
 323
 324 * link:/index.php/Storage:_LVM[Storage: LVM]
 325
 326 * link:/index.php/Storage:_LVM_Thin[Storage: LVM Thin]
 327
 328 * link:/index.php/Storage:_NFS[Storage: NFS]
 329
 330 * link:/index.php/Storage:_RBD[Storage: RBD]
 331
 332 * link:/index.php/Storage:_ZFS[Storage: ZFS]
 333
 334
 335 endif::wiki[]
 336
 337 ifndef::wiki[]
 338
 339 // backend documentation
 340
 341 include::pve-storage-dir.adoc[]
 342
 343 include::pve-storage-nfs.adoc[]
 344
 345 include::pve-storage-glusterfs.adoc[]
 346
 347 include::pve-storage-zfspool.adoc[]
 348
 349 include::pve-storage-lvm.adoc[]
 350
 351 include::pve-storage-lvmthin.adoc[]
 352
 353 include::pve-storage-iscsi.adoc[]
 354
 355 include::pve-storage-iscsidirect.adoc[]
 356
 357 include::pve-storage-rbd.adoc[]
 358
 359
 360
 361 ifdef::manvolnum[]
 362 include::pve-copyright.adoc[]
 363 endif::manvolnum[]
 364
 365 endif::wiki[]
 366