]> git.proxmox.com Git - pve-docs.git/blame - pct.adoc
rework SDN docs a bit
[pve-docs.git] / pct.adoc
CommitLineData
80c0adcb 1[[chapter_pct]]
0c6b782f 2ifdef::manvolnum[]
b2f242ab 3pct(1)
7e2fdb3d 4======
5f09af76
DM
5:pve-toplevel:
6
0c6b782f
DM
7NAME
8----
9
10pct - Tool to manage Linux Containers (LXC) on Proxmox VE
11
12
49a5e11c 13SYNOPSIS
0c6b782f
DM
14--------
15
16include::pct.1-synopsis.adoc[]
17
18DESCRIPTION
19-----------
20endif::manvolnum[]
21
22ifndef::manvolnum[]
23Proxmox Container Toolkit
24=========================
194d2f29 25:pve-toplevel:
0c6b782f 26endif::manvolnum[]
5f09af76 27ifdef::wiki[]
cb84ed18 28:title: Linux Container
5f09af76 29endif::wiki[]
4a2ae9ed 30
14e97811
OB
31Containers are a lightweight alternative to fully virtualized machines (VMs).
32They use the kernel of the host system that they run on, instead of emulating a
33full operating system (OS). This means that containers can access resources on
34the host system directly.
4a2ae9ed 35
6d718b9b
TL
36The runtime costs for containers is low, usually negligible. However, there are
37some drawbacks that need be considered:
4a2ae9ed 38
34694a05 39* Only Linux distributions can be run in containers. It is not possible to run
6d718b9b
TL
40 other Operating Systems like, for example, FreeBSD or Microsoft Windows
41 inside a container.
4a2ae9ed 42
6d718b9b
TL
43* For security reasons, access to host resources needs to be restricted.
44 Containers run in their own separate namespaces. Additionally some syscalls
45 are not allowed within containers.
4a2ae9ed 46
6d718b9b
TL
47{pve} uses https://linuxcontainers.org/[Linux Containers (LXC)] as underlying
48container technology. The ``Proxmox Container Toolkit'' (`pct`) simplifies the
49usage and management of LXC containers.
4a2ae9ed 50
14e97811
OB
51Containers are tightly integrated with {pve}. This means that they are aware of
52the cluster setup, and they can use the same network and storage resources as
53virtual machines. You can also use the {pve} firewall, or manage containers
54using the HA framework.
4a2ae9ed 55
6d718b9b
TL
56Our primary goal is to offer an environment as one would get from a VM, but
57without the additional overhead. We call this ``System Containers''.
4a2ae9ed 58
6d718b9b 59NOTE: If you want to run micro-containers, for example, 'Docker' or 'rkt', it
70a42028 60is best to run them inside a VM.
4a2ae9ed
DM
61
62
99f6ae1a
DM
63Technology Overview
64-------------------
65
66* LXC (https://linuxcontainers.org/)
67
6d718b9b 68* Integrated into {pve} graphical web user interface (GUI)
99f6ae1a
DM
69
70* Easy to use command line tool `pct`
71
72* Access via {pve} REST API
73
6d718b9b 74* 'lxcfs' to provide containerized /proc file system
99f6ae1a 75
6d718b9b 76* Control groups ('cgroups') for resource isolation and limitation
99f6ae1a 77
6d718b9b 78* 'AppArmor' and 'seccomp' to improve security
99f6ae1a 79
14e97811 80* Modern Linux kernels
99f6ae1a
DM
81
82* Image based deployment (templates)
83
6d718b9b 84* Uses {pve} xref:chapter_storage[storage library]
99f6ae1a 85
14e97811 86* Container setup from host (network, DNS, storage, etc.)
99f6ae1a 87
69ab602f 88
80c0adcb 89[[pct_container_images]]
d61bab51
DM
90Container Images
91----------------
92
8c1189b6 93Container images, sometimes also referred to as ``templates'' or
69ab602f 94``appliances'', are `tar` archives which contain everything to run a container.
d61bab51 95
69ab602f
TL
96{pve} itself provides a variety of basic templates for the most common Linux
97distributions. They can be downloaded using the GUI or the `pveam` (short for
98{pve} Appliance Manager) command line utility.
99Additionally, https://www.turnkeylinux.org/[TurnKey Linux] container templates
100are also available to download.
d61bab51 101
2a368b1e
TL
102The list of available templates is updated daily through the 'pve-daily-update'
103timer. You can also trigger an update manually by executing:
3a6fa247 104
14e97811
OB
105----
106# pveam update
107----
3a6fa247 108
14e97811 109To view the list of available images run:
3a6fa247 110
14e97811
OB
111----
112# pveam available
113----
3a6fa247 114
8c1189b6
FG
115You can restrict this large list by specifying the `section` you are
116interested in, for example basic `system` images:
3a6fa247
DM
117
118.List available system images
119----
120# pveam available --section system
14e97811
OB
121system alpine-3.10-default_20190626_amd64.tar.xz
122system alpine-3.9-default_20190224_amd64.tar.xz
123system archlinux-base_20190924-1_amd64.tar.gz
124system centos-6-default_20191016_amd64.tar.xz
125system centos-7-default_20190926_amd64.tar.xz
126system centos-8-default_20191016_amd64.tar.xz
127system debian-10.0-standard_10.0-1_amd64.tar.gz
128system debian-8.0-standard_8.11-1_amd64.tar.gz
129system debian-9.0-standard_9.7-1_amd64.tar.gz
130system fedora-30-default_20190718_amd64.tar.xz
131system fedora-31-default_20191029_amd64.tar.xz
132system gentoo-current-default_20190718_amd64.tar.xz
133system opensuse-15.0-default_20180907_amd64.tar.xz
134system opensuse-15.1-default_20190719_amd64.tar.xz
135system ubuntu-16.04-standard_16.04.5-1_amd64.tar.gz
136system ubuntu-18.04-standard_18.04.1-1_amd64.tar.gz
137system ubuntu-19.04-standard_19.04-1_amd64.tar.gz
138system ubuntu-19.10-standard_19.10-1_amd64.tar.gz
3a6fa247
DM
139----
140
69ab602f 141Before you can use such a template, you need to download them into one of your
2a368b1e
TL
142storages. If you're unsure to which one, you can simply use the `local` named
143storage for that purpose. For clustered installations, it is preferred to use a
144shared storage so that all nodes can access those images.
3a6fa247 145
14e97811
OB
146----
147# pveam download local debian-10.0-standard_10.0-1_amd64.tar.gz
148----
3a6fa247 149
69ab602f
TL
150You are now ready to create containers using that image, and you can list all
151downloaded images on storage `local` with:
24f73a63
DM
152
153----
154# pveam list local
14e97811 155local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz 219.95MB
24f73a63
DM
156----
157
2a368b1e
TL
158TIP: You can also use the {pve} web interface GUI to download, list and delete
159container templates.
160
161`pct` uses them to create a new container, for example:
162
163----
164# pct create 999 local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz
165----
166
69ab602f
TL
167The above command shows you the full {pve} volume identifiers. They include the
168storage name, and most other {pve} commands can use them. For example you can
169delete that image later with:
24f73a63 170
14e97811
OB
171----
172# pveam remove local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz
173----
d61bab51 174
690cd737 175
f3afbb70 176[[pct_settings]]
4f785ca7
DM
177Container Settings
178------------------
179
304eb5a9
EK
180[[pct_general]]
181General Settings
182~~~~~~~~~~~~~~~~
183
1ff5e4e8 184[thumbnail="screenshot/gui-create-ct-general.png"]
2225402c 185
304eb5a9
EK
186General settings of a container include
187
188* the *Node* : the physical server on which the container will run
69ab602f
TL
189* the *CT ID*: a unique number in this {pve} installation used to identify your
190 container
304eb5a9
EK
191* *Hostname*: the hostname of the container
192* *Resource Pool*: a logical group of containers and VMs
193* *Password*: the root password of the container
194* *SSH Public Key*: a public key for connecting to the root account over SSH
195* *Unprivileged container*: this option allows to choose at creation time
69ab602f 196 if you want to create a privileged or unprivileged container.
304eb5a9 197
14e97811
OB
198Unprivileged Containers
199^^^^^^^^^^^^^^^^^^^^^^^
200
69ab602f
TL
201Unprivileged containers use a new kernel feature called user namespaces.
202The root UID 0 inside the container is mapped to an unprivileged user outside
203the container. This means that most security issues (container escape, resource
14e97811
OB
204abuse, etc.) in these containers will affect a random unprivileged user, and
205would be a generic kernel security bug rather than an LXC issue. The LXC team
206thinks unprivileged containers are safe by design.
207
208This is the default option when creating a new container.
209
69ab602f
TL
210NOTE: If the container uses systemd as an init system, please be aware the
211systemd version running inside the container should be equal to or greater than
212220.
14e97811 213
304eb5a9
EK
214
215Privileged Containers
216^^^^^^^^^^^^^^^^^^^^^
217
c02ac25b
TL
218Security in containers is achieved by using mandatory access control 'AppArmor'
219restrictions, 'seccomp' filters and Linux kernel namespaces. The LXC team
220considers this kind of container as unsafe, and they will not consider new
221container escape exploits to be security issues worthy of a CVE and quick fix.
222That's why privileged containers should only be used in trusted environments.
304eb5a9 223
304eb5a9 224
9a5e9443 225[[pct_cpu]]
9a5e9443
DM
226CPU
227~~~
228
1ff5e4e8 229[thumbnail="screenshot/gui-create-ct-cpu.png"]
097aa949 230
14e97811
OB
231You can restrict the number of visible CPUs inside the container using the
232`cores` option. This is implemented using the Linux 'cpuset' cgroup
69ab602f
TL
233(**c**ontrol *group*).
234A special task inside `pvestatd` tries to distribute running containers among
235available CPUs periodically.
236To view the assigned CPUs run the following command:
9a5e9443
DM
237
238----
239# pct cpusets
240 ---------------------
241 102: 6 7
242 105: 2 3 4 5
243 108: 0 1
244 ---------------------
245----
246
14e97811
OB
247Containers use the host kernel directly. All tasks inside a container are
248handled by the host CPU scheduler. {pve} uses the Linux 'CFS' (**C**ompletely
249**F**air **S**cheduler) scheduler by default, which has additional bandwidth
250control options.
9a5e9443
DM
251
252[horizontal]
0725e3c6 253
69ab602f
TL
254`cpulimit`: :: You can use this option to further limit assigned CPU time.
255Please note that this is a floating point number, so it is perfectly valid to
256assign two cores to a container, but restrict overall CPU consumption to half a
257core.
9a5e9443
DM
258+
259----
260cores: 2
261cpulimit: 0.5
262----
263
69ab602f
TL
264`cpuunits`: :: This is a relative weight passed to the kernel scheduler. The
265larger the number is, the more CPU time this container gets. Number is relative
266to the weights of all the other running containers. The default is 1024. You
267can use this setting to prioritize some containers.
9a5e9443
DM
268
269
270[[pct_memory]]
271Memory
272~~~~~~
273
1ff5e4e8 274[thumbnail="screenshot/gui-create-ct-memory.png"]
097aa949 275
9a5e9443
DM
276Container memory is controlled using the cgroup memory controller.
277
278[horizontal]
279
69ab602f
TL
280`memory`: :: Limit overall memory usage. This corresponds to the
281`memory.limit_in_bytes` cgroup setting.
9a5e9443 282
69ab602f
TL
283`swap`: :: Allows the container to use additional swap memory from the host
284swap space. This corresponds to the `memory.memsw.limit_in_bytes` cgroup
285setting, which is set to the sum of both value (`memory + swap`).
9a5e9443 286
4f785ca7
DM
287
288[[pct_mount_points]]
9e44e493
DM
289Mount Points
290~~~~~~~~~~~~
eeecce95 291
1ff5e4e8 292[thumbnail="screenshot/gui-create-ct-root-disk.png"]
097aa949 293
14e97811 294The root mount point is configured with the `rootfs` property. You can
69ab602f
TL
295configure up to 256 additional mount points. The corresponding options are
296called `mp0` to `mp255`. They can contain the following settings:
01639994
FG
297
298include::pct-mountpoint-opts.adoc[]
299
69ab602f
TL
300Currently there are three types of mount points: storage backed mount points,
301bind mounts, and device mounts.
9e44e493 302
5eba0743 303.Typical container `rootfs` configuration
4c3b5c77
DM
304----
305rootfs: thin1:base-100-disk-1,size=8G
306----
307
308
5eba0743 309Storage Backed Mount Points
4c3b5c77 310^^^^^^^^^^^^^^^^^^^^^^^^^^^
01639994 311
9e44e493 312Storage backed mount points are managed by the {pve} storage subsystem and come
eeecce95
WB
313in three different flavors:
314
5eba0743 315- Image based: these are raw images containing a single ext4 formatted file
eeecce95 316 system.
5eba0743 317- ZFS subvolumes: these are technically bind mounts, but with managed storage,
eeecce95
WB
318 and thus allow resizing and snapshotting.
319- Directories: passing `size=0` triggers a special case where instead of a raw
320 image a directory is created.
321
03782251
FG
322NOTE: The special option syntax `STORAGE_ID:SIZE_IN_GB` for storage backed
323mount point volumes will automatically allocate a volume of the specified size
69ab602f
TL
324on the specified storage. For example, calling
325
326----
327pct set 100 -mp0 thin1:10,mp=/path/in/container
328----
329
330will allocate a 10GB volume on the storage `thin1` and replace the volume ID
331place holder `10` with the allocated volume ID, and setup the moutpoint in the
332container at `/path/in/container`
03782251 333
4c3b5c77 334
5eba0743 335Bind Mount Points
4c3b5c77 336^^^^^^^^^^^^^^^^^
01639994 337
9baca183
FG
338Bind mounts allow you to access arbitrary directories from your Proxmox VE host
339inside a container. Some potential use cases are:
340
341- Accessing your home directory in the guest
342- Accessing an USB device directory in the guest
acccc49b 343- Accessing an NFS mount from the host in the guest
9baca183 344
eeecce95 345Bind mounts are considered to not be managed by the storage subsystem, so you
9baca183 346cannot make snapshots or deal with quotas from inside the container. With
eeecce95 347unprivileged containers you might run into permission problems caused by the
9baca183
FG
348user mapping and cannot use ACLs.
349
8c1189b6 350NOTE: The contents of bind mount points are not backed up when using `vzdump`.
eeecce95 351
69ab602f
TL
352WARNING: For security reasons, bind mounts should only be established using
353source directories especially reserved for this purpose, e.g., a directory
354hierarchy under `/mnt/bindmounts`. Never bind mount system directories like
355`/`, `/var` or `/etc` into a container - this poses a great security risk.
9baca183
FG
356
357NOTE: The bind mount source path must not contain any symlinks.
358
359For example, to make the directory `/mnt/bindmounts/shared` accessible in the
360container with ID `100` under the path `/shared`, use a configuration line like
8c1189b6
FG
361`mp0: /mnt/bindmounts/shared,mp=/shared` in `/etc/pve/lxc/100.conf`.
362Alternatively, use `pct set 100 -mp0 /mnt/bindmounts/shared,mp=/shared` to
9baca183 363achieve the same result.
6b707f2c 364
4c3b5c77 365
5eba0743 366Device Mount Points
4c3b5c77 367^^^^^^^^^^^^^^^^^^^
fe154a4f 368
7432d78e
FG
369Device mount points allow to mount block devices of the host directly into the
370container. Similar to bind mounts, device mounts are not managed by {PVE}'s
371storage subsystem, but the `quota` and `acl` options will be honored.
372
373NOTE: Device mount points should only be used under special circumstances. In
374most cases a storage backed mount point offers the same performance and a lot
375more features.
376
69ab602f
TL
377NOTE: The contents of device mount points are not backed up when using
378`vzdump`.
01639994 379
4c3b5c77 380
80c0adcb 381[[pct_container_network]]
f5c351f0
DM
382Network
383~~~~~~~
04c569f6 384
1ff5e4e8 385[thumbnail="screenshot/gui-create-ct-network.png"]
097aa949 386
69ab602f
TL
387You can configure up to 10 network interfaces for a single container.
388The corresponding options are called `net0` to `net9`, and they can contain the
389following setting:
bac8c385
DM
390
391include::pct-network-opts.adoc[]
04c569f6
DM
392
393
139a9019
DM
394[[pct_startup_and_shutdown]]
395Automatic Start and Shutdown of Containers
396~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
397
14e97811
OB
398To automatically start a container when the host system boots, select the
399option 'Start at boot' in the 'Options' panel of the container in the web
400interface or run the following command:
139a9019 401
14e97811
OB
402----
403# pct set CTID -onboot 1
404----
139a9019 405
4dbeb548
DM
406.Start and Shutdown Order
407// use the screenshot from qemu - its the same
1ff5e4e8 408[thumbnail="screenshot/gui-qemu-edit-start-order.png"]
4dbeb548 409
69ab602f
TL
410If you want to fine tune the boot order of your containers, you can use the
411following parameters:
139a9019 412
69ab602f
TL
413* *Start/Shutdown order*: Defines the start order priority. For example, set it
414 to 1 if you want the CT to be the first to be started. (We use the reverse
415 startup order for shutdown, so a container with a start order of 1 would be
416 the last to be shut down)
417* *Startup delay*: Defines the interval between this container start and
418 subsequent containers starts. For example, set it to 240 if you want to wait
419 240 seconds before starting other containers.
139a9019 420* *Shutdown timeout*: Defines the duration in seconds {pve} should wait
69ab602f
TL
421 for the container to be offline after issuing a shutdown command.
422 By default this value is set to 60, which means that {pve} will issue a
423 shutdown request, wait 60s for the machine to be offline, and if after 60s
424 the machine is still online will notify that the shutdown action failed.
139a9019 425
69ab602f
TL
426Please note that containers without a Start/Shutdown order parameter will
427always start after those where the parameter is set, and this parameter only
139a9019
DM
428makes sense between the machines running locally on a host, and not
429cluster-wide.
430
c2c8eb89
DC
431Hookscripts
432~~~~~~~~~~~
433
434You can add a hook script to CTs with the config property `hookscript`.
435
14e97811
OB
436----
437# pct set 100 -hookscript local:snippets/hookscript.pl
438----
c2c8eb89 439
69ab602f
TL
440It will be called during various phases of the guests lifetime. For an example
441and documentation see the example script under
c2c8eb89 442`/usr/share/pve-docs/examples/guest-example-hookscript.pl`.
139a9019 443
bf7f598a
TL
444Security Considerations
445-----------------------
446
447Containers use the kernel of the host system. This exposes an attack surface
448for malicious users. In general, full virtual machines provide better
656d8b21 449isolation. This should be considered if containers are provided to unknown or
bf7f598a
TL
450untrusted people.
451
452To reduce the attack surface, LXC uses many security features like AppArmor,
453CGroups and kernel namespaces.
454
c02ac25b
TL
455AppArmor
456~~~~~~~~
457
bf7f598a
TL
458AppArmor profiles are used to restrict access to possibly dangerous actions.
459Some system calls, i.e. `mount`, are prohibited from execution.
460
461To trace AppArmor activity, use:
462
463----
464# dmesg | grep apparmor
465----
466
c02ac25b
TL
467Although it is not recommended, AppArmor can be disabled for a container. This
468brings security risks with it. Some syscalls can lead to privilege escalation
469when executed within a container if the system is misconfigured or if a LXC or
470Linux Kernel vulnerability exists.
471
472To disable AppArmor for a container, add the following line to the container
473configuration file located at `/etc/pve/lxc/CTID.conf`:
474
475----
76aaaeab 476lxc.apparmor.profile = unconfined
c02ac25b
TL
477----
478
479WARNING: Please note that this is not recommended for production use.
480
481
482// TODO: describe cgroups + seccomp a bit more.
483// TODO: pve-lxc-syscalld
484
485
0892a2c2
TL
486Guest Operating System Configuration
487------------------------------------
488
489{pve} tries to detect the Linux distribution in the container, and modifies
490some files. Here is a short list of things done at container startup:
491
492set /etc/hostname:: to set the container name
493
494modify /etc/hosts:: to allow lookup of the local hostname
495
496network setup:: pass the complete network setup to the container
497
498configure DNS:: pass information about DNS servers
499
500adapt the init system:: for example, fix the number of spawned getty processes
501
502set the root password:: when creating a new container
503
504rewrite ssh_host_keys:: so that each container has unique keys
505
506randomize crontab:: so that cron does not start at the same time on all containers
507
508Changes made by {PVE} are enclosed by comment markers:
509
510----
511# --- BEGIN PVE ---
512<data>
513# --- END PVE ---
514----
515
516Those markers will be inserted at a reasonable location in the file. If such a
517section already exists, it will be updated in place and will not be moved.
518
519Modification of a file can be prevented by adding a `.pve-ignore.` file for it.
520For instance, if the file `/etc/.pve-ignore.hosts` exists then the `/etc/hosts`
521file will not be touched. This can be a simple empty file created via:
522
523----
524# touch /etc/.pve-ignore.hosts
525----
526
527Most modifications are OS dependent, so they differ between different
528distributions and versions. You can completely disable modifications by
529manually setting the `ostype` to `unmanaged`.
530
531OS type detection is done by testing for certain files inside the
3d5c55fc
TL
532container. {pve} first checks the `/etc/os-release` file
533footnote:[/etc/os-release replaces the multitude of per-distribution
534release files https://manpages.debian.org/stable/systemd/os-release.5.en.html].
535If that file is not present, or it does not contain a clearly recognizable
536distribution identifier the following distribution specific release files are
537checked.
0892a2c2
TL
538
539Ubuntu:: inspect /etc/lsb-release (`DISTRIB_ID=Ubuntu`)
540
541Debian:: test /etc/debian_version
542
543Fedora:: test /etc/fedora-release
544
545RedHat or CentOS:: test /etc/redhat-release
546
547ArchLinux:: test /etc/arch-release
548
549Alpine:: test /etc/alpine-release
550
551Gentoo:: test /etc/gentoo-release
552
553NOTE: Container start fails if the configured `ostype` differs from the auto
554detected type.
555
556
b0df9949
TL
557[[pct_container_storage]]
558Container Storage
559-----------------
560
561The {pve} LXC container storage model is more flexible than traditional
562container storage models. A container can have multiple mount points. This
563makes it possible to use the best suited storage for each application.
564
565For example the root file system of the container can be on slow and cheap
566storage while the database can be on fast and distributed storage via a second
567mount point. See section <<pct_mount_points, Mount Points>> for further
568details.
569
570Any storage type supported by the {pve} storage library can be used. This means
571that containers can be stored on local (for example `lvm`, `zfs` or directory),
572shared external (like `iSCSI`, `NFS`) or even distributed storage systems like
573Ceph. Advanced storage features like snapshots or clones can be used if the
574underlying storage supports them. The `vzdump` backup tool can use snapshots to
575provide consistent container backups.
576
577Furthermore, local devices or local directories can be mounted directly using
578'bind mounts'. This gives access to local resources inside a container with
579practically zero overhead. Bind mounts can be used as an easy way to share data
580between containers.
581
582
583FUSE Mounts
584~~~~~~~~~~~
585
586WARNING: Because of existing issues in the Linux kernel's freezer subsystem the
587usage of FUSE mounts inside a container is strongly advised against, as
588containers need to be frozen for suspend or snapshot mode backups.
589
590If FUSE mounts cannot be replaced by other mounting mechanisms or storage
591technologies, it is possible to establish the FUSE mount on the Proxmox host
592and use a bind mount point to make it accessible inside the container.
593
594
595Using Quotas Inside Containers
596~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
597
598Quotas allow to set limits inside a container for the amount of disk space that
599each user can use.
600
601NOTE: This only works on ext4 image based storage types and currently only
602works with privileged containers.
603
604Activating the `quota` option causes the following mount options to be used for
605a mount point:
606`usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0`
607
608This allows quotas to be used like on any other system. You can initialize the
609`/aquota.user` and `/aquota.group` files by running:
610
611----
612# quotacheck -cmug /
613# quotaon /
614----
615
616Then edit the quotas using the `edquota` command. Refer to the documentation of
617the distribution running inside the container for details.
618
619NOTE: You need to run the above commands for every mount point by passing the
620mount point's path instead of just `/`.
621
622
623Using ACLs Inside Containers
624~~~~~~~~~~~~~~~~~~~~~~~~~~~~
625
626The standard Posix **A**ccess **C**ontrol **L**ists are also available inside
627containers. ACLs allow you to set more detailed file ownership than the
628traditional user/group/others model.
629
630
631Backup of Container mount points
632~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
633
634To include a mount point in backups, enable the `backup` option for it in the
635container configuration. For an existing mount point `mp0`
636
637----
638mp0: guests:subvol-100-disk-1,mp=/root/files,size=8G
639----
640
641add `backup=1` to enable it.
642
643----
644mp0: guests:subvol-100-disk-1,mp=/root/files,size=8G,backup=1
645----
646
647NOTE: When creating a new mount point in the GUI, this option is enabled by
648default.
649
650To disable backups for a mount point, add `backup=0` in the way described
651above, or uncheck the *Backup* checkbox on the GUI.
652
653Replication of Containers mount points
654~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
655
656By default, additional mount points are replicated when the Root Disk is
657replicated. If you want the {pve} storage replication mechanism to skip a mount
658point, you can set the *Skip replication* option for that mount point.
659As of {pve} 5.0, replication requires a storage of type `zfspool`. Adding a
660mount point to a different type of storage when the container has replication
661configured requires to have *Skip replication* enabled for that mount point.
662
663
51e33128
FG
664Backup and Restore
665------------------
666
5eba0743 667
2175e37b
FG
668Container Backup
669~~~~~~~~~~~~~~~~
670
69ab602f
TL
671It is possible to use the `vzdump` tool for container backup. Please refer to
672the `vzdump` manual page for details.
8c1189b6 673
51e33128 674
2175e37b
FG
675Restoring Container Backups
676~~~~~~~~~~~~~~~~~~~~~~~~~~~
677
69ab602f
TL
678Restoring container backups made with `vzdump` is possible using the `pct
679restore` command. By default, `pct restore` will attempt to restore as much of
680the backed up container configuration as possible. It is possible to override
681the backed up configuration by manually setting container options on the
682command line (see the `pct` manual page for details).
2175e37b 683
8c1189b6 684NOTE: `pvesm extractconfig` can be used to view the backed up configuration
2175e37b
FG
685contained in a vzdump archive.
686
687There are two basic restore modes, only differing by their handling of mount
688points:
689
4c3b5c77 690
8c1189b6
FG
691``Simple'' Restore Mode
692^^^^^^^^^^^^^^^^^^^^^^^
2175e37b 693
69ab602f
TL
694If neither the `rootfs` parameter nor any of the optional `mpX` parameters are
695explicitly set, the mount point configuration from the backed up configuration
696file is restored using the following steps:
2175e37b
FG
697
698. Extract mount points and their options from backup
699. Create volumes for storage backed mount points (on storage provided with the
69ab602f 700 `storage` parameter, or default local storage if unset)
2175e37b 701. Extract files from backup archive
69ab602f
TL
702. Add bind and device mount points to restored configuration (limited to root
703 user)
2175e37b
FG
704
705NOTE: Since bind and device mount points are never backed up, no files are
706restored in the last step, but only the configuration options. The assumption
707is that such mount points are either backed up with another mechanism (e.g.,
708NFS space that is bind mounted into many containers), or not intended to be
709backed up at all.
710
711This simple mode is also used by the container restore operations in the web
712interface.
713
4c3b5c77 714
8c1189b6
FG
715``Advanced'' Restore Mode
716^^^^^^^^^^^^^^^^^^^^^^^^^
2175e37b
FG
717
718By setting the `rootfs` parameter (and optionally, any combination of `mpX`
8c1189b6 719parameters), the `pct restore` command is automatically switched into an
2175e37b 720advanced mode. This advanced mode completely ignores the `rootfs` and `mpX`
69ab602f
TL
721configuration options contained in the backup archive, and instead only uses
722the options explicitly provided as parameters.
2175e37b 723
69ab602f
TL
724This mode allows flexible configuration of mount point settings at restore
725time, for example:
2175e37b
FG
726
727* Set target storages, volume sizes and other options for each mount point
69ab602f 728 individually
2175e37b
FG
729* Redistribute backed up files according to new mount point scheme
730* Restore to device and/or bind mount points (limited to root user)
731
51e33128 732
8c1189b6 733Managing Containers with `pct`
04c569f6
DM
734------------------------------
735
6d718b9b
TL
736The ``Proxmox Container Toolkit'' (`pct`) is the command line tool to manage
737{pve} containers. It enables you to create or destroy containers, as well as
738control the container execution (start, stop, reboot, migrate, etc.). It can be
739used to set parameters in the config file of a container, for example the
740network configuration or memory limits.
5eba0743 741
04c569f6
DM
742CLI Usage Examples
743~~~~~~~~~~~~~~~~~~
744
69ab602f
TL
745Create a container based on a Debian template (provided you have already
746downloaded the template via the web interface)
04c569f6 747
14e97811
OB
748----
749# pct create 100 /var/lib/vz/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz
750----
04c569f6
DM
751
752Start container 100
753
14e97811
OB
754----
755# pct start 100
756----
04c569f6
DM
757
758Start a login session via getty
759
14e97811
OB
760----
761# pct console 100
762----
04c569f6
DM
763
764Enter the LXC namespace and run a shell as root user
765
14e97811
OB
766----
767# pct enter 100
768----
04c569f6
DM
769
770Display the configuration
771
14e97811
OB
772----
773# pct config 100
774----
04c569f6 775
69ab602f
TL
776Add a network interface called `eth0`, bridged to the host bridge `vmbr0`, set
777the address and gateway, while it's running
04c569f6 778
14e97811
OB
779----
780# pct set 100 -net0 name=eth0,bridge=vmbr0,ip=192.168.15.147/24,gw=192.168.15.1
781----
04c569f6
DM
782
783Reduce the memory of the container to 512MB
784
14e97811
OB
785----
786# pct set 100 -memory 512
787----
0585f29a 788
04c569f6 789
fe57a420
FG
790Obtaining Debugging Logs
791~~~~~~~~~~~~~~~~~~~~~~~~
792
793In case `pct start` is unable to start a specific container, it might be
794helpful to collect debugging output by running `lxc-start` (replace `ID` with
795the container's ID):
796
14e97811
OB
797----
798# lxc-start -n ID -F -l DEBUG -o /tmp/lxc-ID.log
799----
fe57a420 800
69ab602f
TL
801This command will attempt to start the container in foreground mode, to stop
802the container run `pct shutdown ID` or `pct stop ID` in a second terminal.
fe57a420
FG
803
804The collected debug log is written to `/tmp/lxc-ID.log`.
805
806NOTE: If you have changed the container's configuration since the last start
807attempt with `pct start`, you need to run `pct start` at least once to also
808update the configuration used by `lxc-start`.
809
33f50e04
DC
810[[pct_migration]]
811Migration
812---------
813
814If you have a cluster, you can migrate your Containers with
815
14e97811
OB
816----
817# pct migrate <ctid> <target>
818----
33f50e04
DC
819
820This works as long as your Container is offline. If it has local volumes or
14e97811 821mount points defined, the migration will copy the content over the network to
ba021358 822the target host if the same storage is defined there.
33f50e04 823
656d8b21 824Running containers cannot live-migrated due to technical limitations. You can
4c82550d
TL
825do a restart migration, which shuts down, moves and then starts a container
826again on the target node. As containers are very lightweight, this results
827normally only in a downtime of some hundreds of milliseconds.
828
829A restart migration can be done through the web interface or by using the
830`--restart` flag with the `pct migrate` command.
33f50e04 831
69ab602f
TL
832A restart migration will shut down the Container and kill it after the
833specified timeout (the default is 180 seconds). Then it will migrate the
834Container like an offline migration and when finished, it starts the Container
835on the target node.
c7bc47af
DM
836
837[[pct_configuration]]
838Configuration
839-------------
840
69ab602f
TL
841The `/etc/pve/lxc/<CTID>.conf` file stores container configuration, where
842`<CTID>` is the numeric ID of the given container. Like all other files stored
843inside `/etc/pve/`, they get automatically replicated to all other cluster
844nodes.
c7bc47af
DM
845
846NOTE: CTIDs < 100 are reserved for internal purposes, and CTIDs need to be
847unique cluster wide.
848
849.Example Container Configuration
850----
851ostype: debian
852arch: amd64
853hostname: www
854memory: 512
855swap: 512
856net0: bridge=vmbr0,hwaddr=66:64:66:64:64:36,ip=dhcp,name=eth0,type=veth
857rootfs: local:107/vm-107-disk-1.raw,size=7G
858----
859
69ab602f 860The configuration files are simple text files. You can edit them using a normal
da9679b6 861text editor, for example, `vi` or `nano`.
69ab602f
TL
862This is sometimes useful to do small corrections, but keep in mind that you
863need to restart the container to apply such changes.
c7bc47af 864
69ab602f
TL
865For that reason, it is usually better to use the `pct` command to generate and
866modify those files, or do the whole thing using the GUI.
867Our toolkit is smart enough to instantaneously apply most changes to running
da9679b6 868containers. This feature is called ``hot plug'', and there is no need to restart
69ab602f 869the container in that case.
c7bc47af 870
da9679b6 871In cases where a change cannot be hot-plugged, it will be registered as a
69ab602f
TL
872pending change (shown in red color in the GUI).
873They will only be applied after rebooting the container.
14e97811 874
c7bc47af
DM
875
876File Format
877~~~~~~~~~~~
878
69ab602f
TL
879The container configuration file uses a simple colon separated key/value
880format. Each line has the following format:
c7bc47af
DM
881
882-----
883# this is a comment
884OPTION: value
885-----
886
69ab602f
TL
887Blank lines in those files are ignored, and lines starting with a `#` character
888are treated as comments and are also ignored.
c7bc47af 889
69ab602f 890It is possible to add low-level, LXC style configuration directly, for example:
c7bc47af 891
14e97811
OB
892----
893lxc.init_cmd: /sbin/my_own_init
894----
c7bc47af
DM
895
896or
897
14e97811
OB
898----
899lxc.init_cmd = /sbin/my_own_init
900----
c7bc47af 901
14e97811 902The settings are passed directly to the LXC low-level tools.
c7bc47af
DM
903
904
905[[pct_snapshots]]
906Snapshots
907~~~~~~~~~
908
69ab602f
TL
909When you create a snapshot, `pct` stores the configuration at snapshot time
910into a separate snapshot section within the same configuration file. For
911example, after creating a snapshot called ``testsnapshot'', your configuration
912file will look like this:
c7bc47af
DM
913
914.Container configuration with snapshot
915----
916memory: 512
917swap: 512
918parent: testsnaphot
919...
920
921[testsnaphot]
922memory: 512
923swap: 512
924snaptime: 1457170803
925...
926----
927
69ab602f
TL
928There are a few snapshot related properties like `parent` and `snaptime`. The
929`parent` property is used to store the parent/child relationship between
930snapshots. `snaptime` is the snapshot creation time stamp (Unix epoch).
c7bc47af
DM
931
932
933[[pct_options]]
934Options
935~~~~~~~
936
937include::pct.conf.5-opts.adoc[]
938
939
2a11aa70
DM
940Locks
941-----
942
69ab602f
TL
943Container migrations, snapshots and backups (`vzdump`) set a lock to prevent
944incompatible concurrent actions on the affected container. Sometimes you need
945to remove such a lock manually (e.g., after a power failure).
2a11aa70 946
14e97811
OB
947----
948# pct unlock <CTID>
949----
2a11aa70 950
69ab602f
TL
951CAUTION: Only do this if you are sure the action which set the lock is no
952longer running.
2a11aa70 953
fe57a420 954
0c6b782f 955ifdef::manvolnum[]
3bd9d0cf
DM
956
957Files
958------
959
960`/etc/pve/lxc/<CTID>.conf`::
961
962Configuration file for the container '<CTID>'.
963
964
0c6b782f
DM
965include::pve-copyright.adoc[]
966endif::manvolnum[]