]>
Commit | Line | Data |
---|---|---|
1 | [[chapter_pct]] | |
2 | ifdef::manvolnum[] | |
3 | pct(1) | |
4 | ====== | |
5 | :pve-toplevel: | |
6 | ||
7 | NAME | |
8 | ---- | |
9 | ||
10 | pct - Tool to manage Linux Containers (LXC) on Proxmox VE | |
11 | ||
12 | ||
13 | SYNOPSIS | |
14 | -------- | |
15 | ||
16 | include::pct.1-synopsis.adoc[] | |
17 | ||
18 | DESCRIPTION | |
19 | ----------- | |
20 | endif::manvolnum[] | |
21 | ||
22 | ifndef::manvolnum[] | |
23 | Proxmox Container Toolkit | |
24 | ========================= | |
25 | :pve-toplevel: | |
26 | endif::manvolnum[] | |
27 | ifdef::wiki[] | |
28 | :title: Linux Container | |
29 | endif::wiki[] | |
30 | ||
31 | Containers are a lightweight alternative to fully virtualized machines (VMs). | |
32 | They use the kernel of the host system that they run on, instead of emulating a | |
33 | full operating system (OS). This means that containers can access resources on | |
34 | the host system directly. | |
35 | ||
36 | The runtime costs for containers is low, usually negligible. However, there are | |
37 | some drawbacks that need be considered: | |
38 | ||
39 | * Only Linux distributions can be run in Proxmox Containers. It is not possible to run | |
40 | other operating systems like, for example, FreeBSD or Microsoft Windows | |
41 | inside a container. | |
42 | ||
43 | * For security reasons, access to host resources needs to be restricted. | |
44 | Therefore, containers run in their own separate namespaces. Additionally some | |
45 | syscalls (user space requests to the Linux kernel) are not allowed within containers. | |
46 | ||
47 | {pve} uses https://linuxcontainers.org/lxc/introduction/[Linux Containers (LXC)] as its underlying | |
48 | container technology. The ``Proxmox Container Toolkit'' (`pct`) simplifies the | |
49 | usage and management of LXC, by providing an interface that abstracts | |
50 | complex tasks. | |
51 | ||
52 | Containers are tightly integrated with {pve}. This means that they are aware of | |
53 | the cluster setup, and they can use the same network and storage resources as | |
54 | virtual machines. You can also use the {pve} firewall, or manage containers | |
55 | using the HA framework. | |
56 | ||
57 | Our primary goal is to offer an environment that provides the benefits of using a | |
58 | VM, but without the additional overhead. This means that Proxmox Containers can | |
59 | be categorized as ``System Containers'', rather than ``Application Containers''. | |
60 | ||
61 | NOTE: If you want to run application containers, for example, 'Docker' images, it | |
62 | is recommended that you run them inside a Proxmox Qemu VM. This will give you | |
63 | all the advantages of application containerization, while also providing the | |
64 | benefits that VMs offer, such as strong isolation from the host and the ability | |
65 | to live-migrate, which otherwise isn't possible with containers. | |
66 | ||
67 | ||
68 | Technology Overview | |
69 | ------------------- | |
70 | ||
71 | * LXC (https://linuxcontainers.org/) | |
72 | ||
73 | * Integrated into {pve} graphical web user interface (GUI) | |
74 | ||
75 | * Easy to use command line tool `pct` | |
76 | ||
77 | * Access via {pve} REST API | |
78 | ||
79 | * 'lxcfs' to provide containerized /proc file system | |
80 | ||
81 | * Control groups ('cgroups') for resource isolation and limitation | |
82 | ||
83 | * 'AppArmor' and 'seccomp' to improve security | |
84 | ||
85 | * Modern Linux kernels | |
86 | ||
87 | * Image based deployment (templates) | |
88 | ||
89 | * Uses {pve} xref:chapter_storage[storage library] | |
90 | ||
91 | * Container setup from host (network, DNS, storage, etc.) | |
92 | ||
93 | ||
94 | [[pct_container_images]] | |
95 | Container Images | |
96 | ---------------- | |
97 | ||
98 | Container images, sometimes also referred to as ``templates'' or | |
99 | ``appliances'', are `tar` archives which contain everything to run a container. | |
100 | ||
101 | {pve} itself provides a variety of basic templates for the most common Linux | |
102 | distributions. They can be downloaded using the GUI or the `pveam` (short for | |
103 | {pve} Appliance Manager) command line utility. | |
104 | Additionally, https://www.turnkeylinux.org/[TurnKey Linux] container templates | |
105 | are also available to download. | |
106 | ||
107 | The list of available templates is updated daily through the 'pve-daily-update' | |
108 | timer. You can also trigger an update manually by executing: | |
109 | ||
110 | ---- | |
111 | # pveam update | |
112 | ---- | |
113 | ||
114 | To view the list of available images run: | |
115 | ||
116 | ---- | |
117 | # pveam available | |
118 | ---- | |
119 | ||
120 | You can restrict this large list by specifying the `section` you are | |
121 | interested in, for example basic `system` images: | |
122 | ||
123 | .List available system images | |
124 | ---- | |
125 | # pveam available --section system | |
126 | system alpine-3.10-default_20190626_amd64.tar.xz | |
127 | system alpine-3.9-default_20190224_amd64.tar.xz | |
128 | system archlinux-base_20190924-1_amd64.tar.gz | |
129 | system centos-6-default_20191016_amd64.tar.xz | |
130 | system centos-7-default_20190926_amd64.tar.xz | |
131 | system centos-8-default_20191016_amd64.tar.xz | |
132 | system debian-10.0-standard_10.0-1_amd64.tar.gz | |
133 | system debian-8.0-standard_8.11-1_amd64.tar.gz | |
134 | system debian-9.0-standard_9.7-1_amd64.tar.gz | |
135 | system fedora-30-default_20190718_amd64.tar.xz | |
136 | system fedora-31-default_20191029_amd64.tar.xz | |
137 | system gentoo-current-default_20190718_amd64.tar.xz | |
138 | system opensuse-15.0-default_20180907_amd64.tar.xz | |
139 | system opensuse-15.1-default_20190719_amd64.tar.xz | |
140 | system ubuntu-16.04-standard_16.04.5-1_amd64.tar.gz | |
141 | system ubuntu-18.04-standard_18.04.1-1_amd64.tar.gz | |
142 | system ubuntu-19.04-standard_19.04-1_amd64.tar.gz | |
143 | system ubuntu-19.10-standard_19.10-1_amd64.tar.gz | |
144 | ---- | |
145 | ||
146 | Before you can use such a template, you need to download them into one of your | |
147 | storages. If you're unsure to which one, you can simply use the `local` named | |
148 | storage for that purpose. For clustered installations, it is preferred to use a | |
149 | shared storage so that all nodes can access those images. | |
150 | ||
151 | ---- | |
152 | # pveam download local debian-10.0-standard_10.0-1_amd64.tar.gz | |
153 | ---- | |
154 | ||
155 | You are now ready to create containers using that image, and you can list all | |
156 | downloaded images on storage `local` with: | |
157 | ||
158 | ---- | |
159 | # pveam list local | |
160 | local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz 219.95MB | |
161 | ---- | |
162 | ||
163 | TIP: You can also use the {pve} web interface GUI to download, list and delete | |
164 | container templates. | |
165 | ||
166 | `pct` uses them to create a new container, for example: | |
167 | ||
168 | ---- | |
169 | # pct create 999 local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz | |
170 | ---- | |
171 | ||
172 | The above command shows you the full {pve} volume identifiers. They include the | |
173 | storage name, and most other {pve} commands can use them. For example you can | |
174 | delete that image later with: | |
175 | ||
176 | ---- | |
177 | # pveam remove local:vztmpl/debian-10.0-standard_10.0-1_amd64.tar.gz | |
178 | ---- | |
179 | ||
180 | ||
181 | [[pct_settings]] | |
182 | Container Settings | |
183 | ------------------ | |
184 | ||
185 | [[pct_general]] | |
186 | General Settings | |
187 | ~~~~~~~~~~~~~~~~ | |
188 | ||
189 | [thumbnail="screenshot/gui-create-ct-general.png"] | |
190 | ||
191 | General settings of a container include | |
192 | ||
193 | * the *Node* : the physical server on which the container will run | |
194 | * the *CT ID*: a unique number in this {pve} installation used to identify your | |
195 | container | |
196 | * *Hostname*: the hostname of the container | |
197 | * *Resource Pool*: a logical group of containers and VMs | |
198 | * *Password*: the root password of the container | |
199 | * *SSH Public Key*: a public key for connecting to the root account over SSH | |
200 | * *Unprivileged container*: this option allows to choose at creation time | |
201 | if you want to create a privileged or unprivileged container. | |
202 | ||
203 | Unprivileged Containers | |
204 | ^^^^^^^^^^^^^^^^^^^^^^^ | |
205 | ||
206 | Unprivileged containers use a new kernel feature called user namespaces. | |
207 | The root UID 0 inside the container is mapped to an unprivileged user outside | |
208 | the container. This means that most security issues (container escape, resource | |
209 | abuse, etc.) in these containers will affect a random unprivileged user, and | |
210 | would be a generic kernel security bug rather than an LXC issue. The LXC team | |
211 | thinks unprivileged containers are safe by design. | |
212 | ||
213 | This is the default option when creating a new container. | |
214 | ||
215 | NOTE: If the container uses systemd as an init system, please be aware the | |
216 | systemd version running inside the container should be equal to or greater than | |
217 | 220. | |
218 | ||
219 | ||
220 | Privileged Containers | |
221 | ^^^^^^^^^^^^^^^^^^^^^ | |
222 | ||
223 | Security in containers is achieved by using mandatory access control 'AppArmor' | |
224 | restrictions, 'seccomp' filters and Linux kernel namespaces. The LXC team | |
225 | considers this kind of container as unsafe, and they will not consider new | |
226 | container escape exploits to be security issues worthy of a CVE and quick fix. | |
227 | That's why privileged containers should only be used in trusted environments. | |
228 | ||
229 | ||
230 | [[pct_cpu]] | |
231 | CPU | |
232 | ~~~ | |
233 | ||
234 | [thumbnail="screenshot/gui-create-ct-cpu.png"] | |
235 | ||
236 | You can restrict the number of visible CPUs inside the container using the | |
237 | `cores` option. This is implemented using the Linux 'cpuset' cgroup | |
238 | (**c**ontrol *group*). | |
239 | A special task inside `pvestatd` tries to distribute running containers among | |
240 | available CPUs periodically. | |
241 | To view the assigned CPUs run the following command: | |
242 | ||
243 | ---- | |
244 | # pct cpusets | |
245 | --------------------- | |
246 | 102: 6 7 | |
247 | 105: 2 3 4 5 | |
248 | 108: 0 1 | |
249 | --------------------- | |
250 | ---- | |
251 | ||
252 | Containers use the host kernel directly. All tasks inside a container are | |
253 | handled by the host CPU scheduler. {pve} uses the Linux 'CFS' (**C**ompletely | |
254 | **F**air **S**cheduler) scheduler by default, which has additional bandwidth | |
255 | control options. | |
256 | ||
257 | [horizontal] | |
258 | ||
259 | `cpulimit`: :: You can use this option to further limit assigned CPU time. | |
260 | Please note that this is a floating point number, so it is perfectly valid to | |
261 | assign two cores to a container, but restrict overall CPU consumption to half a | |
262 | core. | |
263 | + | |
264 | ---- | |
265 | cores: 2 | |
266 | cpulimit: 0.5 | |
267 | ---- | |
268 | ||
269 | `cpuunits`: :: This is a relative weight passed to the kernel scheduler. The | |
270 | larger the number is, the more CPU time this container gets. Number is relative | |
271 | to the weights of all the other running containers. The default is 1024. You | |
272 | can use this setting to prioritize some containers. | |
273 | ||
274 | ||
275 | [[pct_memory]] | |
276 | Memory | |
277 | ~~~~~~ | |
278 | ||
279 | [thumbnail="screenshot/gui-create-ct-memory.png"] | |
280 | ||
281 | Container memory is controlled using the cgroup memory controller. | |
282 | ||
283 | [horizontal] | |
284 | ||
285 | `memory`: :: Limit overall memory usage. This corresponds to the | |
286 | `memory.limit_in_bytes` cgroup setting. | |
287 | ||
288 | `swap`: :: Allows the container to use additional swap memory from the host | |
289 | swap space. This corresponds to the `memory.memsw.limit_in_bytes` cgroup | |
290 | setting, which is set to the sum of both value (`memory + swap`). | |
291 | ||
292 | ||
293 | [[pct_mount_points]] | |
294 | Mount Points | |
295 | ~~~~~~~~~~~~ | |
296 | ||
297 | [thumbnail="screenshot/gui-create-ct-root-disk.png"] | |
298 | ||
299 | The root mount point is configured with the `rootfs` property. You can | |
300 | configure up to 256 additional mount points. The corresponding options are | |
301 | called `mp0` to `mp255`. They can contain the following settings: | |
302 | ||
303 | include::pct-mountpoint-opts.adoc[] | |
304 | ||
305 | Currently there are three types of mount points: storage backed mount points, | |
306 | bind mounts, and device mounts. | |
307 | ||
308 | .Typical container `rootfs` configuration | |
309 | ---- | |
310 | rootfs: thin1:base-100-disk-1,size=8G | |
311 | ---- | |
312 | ||
313 | ||
314 | Storage Backed Mount Points | |
315 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
316 | ||
317 | Storage backed mount points are managed by the {pve} storage subsystem and come | |
318 | in three different flavors: | |
319 | ||
320 | - Image based: these are raw images containing a single ext4 formatted file | |
321 | system. | |
322 | - ZFS subvolumes: these are technically bind mounts, but with managed storage, | |
323 | and thus allow resizing and snapshotting. | |
324 | - Directories: passing `size=0` triggers a special case where instead of a raw | |
325 | image a directory is created. | |
326 | ||
327 | NOTE: The special option syntax `STORAGE_ID:SIZE_IN_GB` for storage backed | |
328 | mount point volumes will automatically allocate a volume of the specified size | |
329 | on the specified storage. For example, calling | |
330 | ||
331 | ---- | |
332 | pct set 100 -mp0 thin1:10,mp=/path/in/container | |
333 | ---- | |
334 | ||
335 | will allocate a 10GB volume on the storage `thin1` and replace the volume ID | |
336 | place holder `10` with the allocated volume ID, and setup the moutpoint in the | |
337 | container at `/path/in/container` | |
338 | ||
339 | ||
340 | Bind Mount Points | |
341 | ^^^^^^^^^^^^^^^^^ | |
342 | ||
343 | Bind mounts allow you to access arbitrary directories from your Proxmox VE host | |
344 | inside a container. Some potential use cases are: | |
345 | ||
346 | - Accessing your home directory in the guest | |
347 | - Accessing an USB device directory in the guest | |
348 | - Accessing an NFS mount from the host in the guest | |
349 | ||
350 | Bind mounts are considered to not be managed by the storage subsystem, so you | |
351 | cannot make snapshots or deal with quotas from inside the container. With | |
352 | unprivileged containers you might run into permission problems caused by the | |
353 | user mapping and cannot use ACLs. | |
354 | ||
355 | NOTE: The contents of bind mount points are not backed up when using `vzdump`. | |
356 | ||
357 | WARNING: For security reasons, bind mounts should only be established using | |
358 | source directories especially reserved for this purpose, e.g., a directory | |
359 | hierarchy under `/mnt/bindmounts`. Never bind mount system directories like | |
360 | `/`, `/var` or `/etc` into a container - this poses a great security risk. | |
361 | ||
362 | NOTE: The bind mount source path must not contain any symlinks. | |
363 | ||
364 | For example, to make the directory `/mnt/bindmounts/shared` accessible in the | |
365 | container with ID `100` under the path `/shared`, use a configuration line like | |
366 | `mp0: /mnt/bindmounts/shared,mp=/shared` in `/etc/pve/lxc/100.conf`. | |
367 | Alternatively, use `pct set 100 -mp0 /mnt/bindmounts/shared,mp=/shared` to | |
368 | achieve the same result. | |
369 | ||
370 | ||
371 | Device Mount Points | |
372 | ^^^^^^^^^^^^^^^^^^^ | |
373 | ||
374 | Device mount points allow to mount block devices of the host directly into the | |
375 | container. Similar to bind mounts, device mounts are not managed by {PVE}'s | |
376 | storage subsystem, but the `quota` and `acl` options will be honored. | |
377 | ||
378 | NOTE: Device mount points should only be used under special circumstances. In | |
379 | most cases a storage backed mount point offers the same performance and a lot | |
380 | more features. | |
381 | ||
382 | NOTE: The contents of device mount points are not backed up when using | |
383 | `vzdump`. | |
384 | ||
385 | ||
386 | [[pct_container_network]] | |
387 | Network | |
388 | ~~~~~~~ | |
389 | ||
390 | [thumbnail="screenshot/gui-create-ct-network.png"] | |
391 | ||
392 | You can configure up to 10 network interfaces for a single container. | |
393 | The corresponding options are called `net0` to `net9`, and they can contain the | |
394 | following setting: | |
395 | ||
396 | include::pct-network-opts.adoc[] | |
397 | ||
398 | ||
399 | [[pct_startup_and_shutdown]] | |
400 | Automatic Start and Shutdown of Containers | |
401 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
402 | ||
403 | To automatically start a container when the host system boots, select the | |
404 | option 'Start at boot' in the 'Options' panel of the container in the web | |
405 | interface or run the following command: | |
406 | ||
407 | ---- | |
408 | # pct set CTID -onboot 1 | |
409 | ---- | |
410 | ||
411 | .Start and Shutdown Order | |
412 | // use the screenshot from qemu - its the same | |
413 | [thumbnail="screenshot/gui-qemu-edit-start-order.png"] | |
414 | ||
415 | If you want to fine tune the boot order of your containers, you can use the | |
416 | following parameters: | |
417 | ||
418 | * *Start/Shutdown order*: Defines the start order priority. For example, set it | |
419 | to 1 if you want the CT to be the first to be started. (We use the reverse | |
420 | startup order for shutdown, so a container with a start order of 1 would be | |
421 | the last to be shut down) | |
422 | * *Startup delay*: Defines the interval between this container start and | |
423 | subsequent containers starts. For example, set it to 240 if you want to wait | |
424 | 240 seconds before starting other containers. | |
425 | * *Shutdown timeout*: Defines the duration in seconds {pve} should wait | |
426 | for the container to be offline after issuing a shutdown command. | |
427 | By default this value is set to 60, which means that {pve} will issue a | |
428 | shutdown request, wait 60s for the machine to be offline, and if after 60s | |
429 | the machine is still online will notify that the shutdown action failed. | |
430 | ||
431 | Please note that containers without a Start/Shutdown order parameter will | |
432 | always start after those where the parameter is set, and this parameter only | |
433 | makes sense between the machines running locally on a host, and not | |
434 | cluster-wide. | |
435 | ||
436 | Hookscripts | |
437 | ~~~~~~~~~~~ | |
438 | ||
439 | You can add a hook script to CTs with the config property `hookscript`. | |
440 | ||
441 | ---- | |
442 | # pct set 100 -hookscript local:snippets/hookscript.pl | |
443 | ---- | |
444 | ||
445 | It will be called during various phases of the guests lifetime. For an example | |
446 | and documentation see the example script under | |
447 | `/usr/share/pve-docs/examples/guest-example-hookscript.pl`. | |
448 | ||
449 | Security Considerations | |
450 | ----------------------- | |
451 | ||
452 | Containers use the kernel of the host system. This exposes an attack surface | |
453 | for malicious users. In general, full virtual machines provide better | |
454 | isolation. This should be considered if containers are provided to unknown or | |
455 | untrusted people. | |
456 | ||
457 | To reduce the attack surface, LXC uses many security features like AppArmor, | |
458 | CGroups and kernel namespaces. | |
459 | ||
460 | AppArmor | |
461 | ~~~~~~~~ | |
462 | ||
463 | AppArmor profiles are used to restrict access to possibly dangerous actions. | |
464 | Some system calls, i.e. `mount`, are prohibited from execution. | |
465 | ||
466 | To trace AppArmor activity, use: | |
467 | ||
468 | ---- | |
469 | # dmesg | grep apparmor | |
470 | ---- | |
471 | ||
472 | Although it is not recommended, AppArmor can be disabled for a container. This | |
473 | brings security risks with it. Some syscalls can lead to privilege escalation | |
474 | when executed within a container if the system is misconfigured or if a LXC or | |
475 | Linux Kernel vulnerability exists. | |
476 | ||
477 | To disable AppArmor for a container, add the following line to the container | |
478 | configuration file located at `/etc/pve/lxc/CTID.conf`: | |
479 | ||
480 | ---- | |
481 | lxc.apparmor.profile = unconfined | |
482 | ---- | |
483 | ||
484 | WARNING: Please note that this is not recommended for production use. | |
485 | ||
486 | ||
487 | // TODO: describe cgroups + seccomp a bit more. | |
488 | // TODO: pve-lxc-syscalld | |
489 | ||
490 | ||
491 | Guest Operating System Configuration | |
492 | ------------------------------------ | |
493 | ||
494 | {pve} tries to detect the Linux distribution in the container, and modifies | |
495 | some files. Here is a short list of things done at container startup: | |
496 | ||
497 | set /etc/hostname:: to set the container name | |
498 | ||
499 | modify /etc/hosts:: to allow lookup of the local hostname | |
500 | ||
501 | network setup:: pass the complete network setup to the container | |
502 | ||
503 | configure DNS:: pass information about DNS servers | |
504 | ||
505 | adapt the init system:: for example, fix the number of spawned getty processes | |
506 | ||
507 | set the root password:: when creating a new container | |
508 | ||
509 | rewrite ssh_host_keys:: so that each container has unique keys | |
510 | ||
511 | randomize crontab:: so that cron does not start at the same time on all containers | |
512 | ||
513 | Changes made by {PVE} are enclosed by comment markers: | |
514 | ||
515 | ---- | |
516 | # --- BEGIN PVE --- | |
517 | <data> | |
518 | # --- END PVE --- | |
519 | ---- | |
520 | ||
521 | Those markers will be inserted at a reasonable location in the file. If such a | |
522 | section already exists, it will be updated in place and will not be moved. | |
523 | ||
524 | Modification of a file can be prevented by adding a `.pve-ignore.` file for it. | |
525 | For instance, if the file `/etc/.pve-ignore.hosts` exists then the `/etc/hosts` | |
526 | file will not be touched. This can be a simple empty file created via: | |
527 | ||
528 | ---- | |
529 | # touch /etc/.pve-ignore.hosts | |
530 | ---- | |
531 | ||
532 | Most modifications are OS dependent, so they differ between different | |
533 | distributions and versions. You can completely disable modifications by | |
534 | manually setting the `ostype` to `unmanaged`. | |
535 | ||
536 | OS type detection is done by testing for certain files inside the | |
537 | container. {pve} first checks the `/etc/os-release` file | |
538 | footnote:[/etc/os-release replaces the multitude of per-distribution | |
539 | release files https://manpages.debian.org/stable/systemd/os-release.5.en.html]. | |
540 | If that file is not present, or it does not contain a clearly recognizable | |
541 | distribution identifier the following distribution specific release files are | |
542 | checked. | |
543 | ||
544 | Ubuntu:: inspect /etc/lsb-release (`DISTRIB_ID=Ubuntu`) | |
545 | ||
546 | Debian:: test /etc/debian_version | |
547 | ||
548 | Fedora:: test /etc/fedora-release | |
549 | ||
550 | RedHat or CentOS:: test /etc/redhat-release | |
551 | ||
552 | ArchLinux:: test /etc/arch-release | |
553 | ||
554 | Alpine:: test /etc/alpine-release | |
555 | ||
556 | Gentoo:: test /etc/gentoo-release | |
557 | ||
558 | NOTE: Container start fails if the configured `ostype` differs from the auto | |
559 | detected type. | |
560 | ||
561 | ||
562 | [[pct_container_storage]] | |
563 | Container Storage | |
564 | ----------------- | |
565 | ||
566 | The {pve} LXC container storage model is more flexible than traditional | |
567 | container storage models. A container can have multiple mount points. This | |
568 | makes it possible to use the best suited storage for each application. | |
569 | ||
570 | For example the root file system of the container can be on slow and cheap | |
571 | storage while the database can be on fast and distributed storage via a second | |
572 | mount point. See section <<pct_mount_points, Mount Points>> for further | |
573 | details. | |
574 | ||
575 | Any storage type supported by the {pve} storage library can be used. This means | |
576 | that containers can be stored on local (for example `lvm`, `zfs` or directory), | |
577 | shared external (like `iSCSI`, `NFS`) or even distributed storage systems like | |
578 | Ceph. Advanced storage features like snapshots or clones can be used if the | |
579 | underlying storage supports them. The `vzdump` backup tool can use snapshots to | |
580 | provide consistent container backups. | |
581 | ||
582 | Furthermore, local devices or local directories can be mounted directly using | |
583 | 'bind mounts'. This gives access to local resources inside a container with | |
584 | practically zero overhead. Bind mounts can be used as an easy way to share data | |
585 | between containers. | |
586 | ||
587 | ||
588 | FUSE Mounts | |
589 | ~~~~~~~~~~~ | |
590 | ||
591 | WARNING: Because of existing issues in the Linux kernel's freezer subsystem the | |
592 | usage of FUSE mounts inside a container is strongly advised against, as | |
593 | containers need to be frozen for suspend or snapshot mode backups. | |
594 | ||
595 | If FUSE mounts cannot be replaced by other mounting mechanisms or storage | |
596 | technologies, it is possible to establish the FUSE mount on the Proxmox host | |
597 | and use a bind mount point to make it accessible inside the container. | |
598 | ||
599 | ||
600 | Using Quotas Inside Containers | |
601 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
602 | ||
603 | Quotas allow to set limits inside a container for the amount of disk space that | |
604 | each user can use. | |
605 | ||
606 | NOTE: This only works on ext4 image based storage types and currently only | |
607 | works with privileged containers. | |
608 | ||
609 | Activating the `quota` option causes the following mount options to be used for | |
610 | a mount point: | |
611 | `usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0` | |
612 | ||
613 | This allows quotas to be used like on any other system. You can initialize the | |
614 | `/aquota.user` and `/aquota.group` files by running: | |
615 | ||
616 | ---- | |
617 | # quotacheck -cmug / | |
618 | # quotaon / | |
619 | ---- | |
620 | ||
621 | Then edit the quotas using the `edquota` command. Refer to the documentation of | |
622 | the distribution running inside the container for details. | |
623 | ||
624 | NOTE: You need to run the above commands for every mount point by passing the | |
625 | mount point's path instead of just `/`. | |
626 | ||
627 | ||
628 | Using ACLs Inside Containers | |
629 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
630 | ||
631 | The standard Posix **A**ccess **C**ontrol **L**ists are also available inside | |
632 | containers. ACLs allow you to set more detailed file ownership than the | |
633 | traditional user/group/others model. | |
634 | ||
635 | ||
636 | Backup of Container mount points | |
637 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
638 | ||
639 | To include a mount point in backups, enable the `backup` option for it in the | |
640 | container configuration. For an existing mount point `mp0` | |
641 | ||
642 | ---- | |
643 | mp0: guests:subvol-100-disk-1,mp=/root/files,size=8G | |
644 | ---- | |
645 | ||
646 | add `backup=1` to enable it. | |
647 | ||
648 | ---- | |
649 | mp0: guests:subvol-100-disk-1,mp=/root/files,size=8G,backup=1 | |
650 | ---- | |
651 | ||
652 | NOTE: When creating a new mount point in the GUI, this option is enabled by | |
653 | default. | |
654 | ||
655 | To disable backups for a mount point, add `backup=0` in the way described | |
656 | above, or uncheck the *Backup* checkbox on the GUI. | |
657 | ||
658 | Replication of Containers mount points | |
659 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
660 | ||
661 | By default, additional mount points are replicated when the Root Disk is | |
662 | replicated. If you want the {pve} storage replication mechanism to skip a mount | |
663 | point, you can set the *Skip replication* option for that mount point. | |
664 | As of {pve} 5.0, replication requires a storage of type `zfspool`. Adding a | |
665 | mount point to a different type of storage when the container has replication | |
666 | configured requires to have *Skip replication* enabled for that mount point. | |
667 | ||
668 | ||
669 | Backup and Restore | |
670 | ------------------ | |
671 | ||
672 | ||
673 | Container Backup | |
674 | ~~~~~~~~~~~~~~~~ | |
675 | ||
676 | It is possible to use the `vzdump` tool for container backup. Please refer to | |
677 | the `vzdump` manual page for details. | |
678 | ||
679 | ||
680 | Restoring Container Backups | |
681 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
682 | ||
683 | Restoring container backups made with `vzdump` is possible using the `pct | |
684 | restore` command. By default, `pct restore` will attempt to restore as much of | |
685 | the backed up container configuration as possible. It is possible to override | |
686 | the backed up configuration by manually setting container options on the | |
687 | command line (see the `pct` manual page for details). | |
688 | ||
689 | NOTE: `pvesm extractconfig` can be used to view the backed up configuration | |
690 | contained in a vzdump archive. | |
691 | ||
692 | There are two basic restore modes, only differing by their handling of mount | |
693 | points: | |
694 | ||
695 | ||
696 | ``Simple'' Restore Mode | |
697 | ^^^^^^^^^^^^^^^^^^^^^^^ | |
698 | ||
699 | If neither the `rootfs` parameter nor any of the optional `mpX` parameters are | |
700 | explicitly set, the mount point configuration from the backed up configuration | |
701 | file is restored using the following steps: | |
702 | ||
703 | . Extract mount points and their options from backup | |
704 | . Create volumes for storage backed mount points (on storage provided with the | |
705 | `storage` parameter, or default local storage if unset) | |
706 | . Extract files from backup archive | |
707 | . Add bind and device mount points to restored configuration (limited to root | |
708 | user) | |
709 | ||
710 | NOTE: Since bind and device mount points are never backed up, no files are | |
711 | restored in the last step, but only the configuration options. The assumption | |
712 | is that such mount points are either backed up with another mechanism (e.g., | |
713 | NFS space that is bind mounted into many containers), or not intended to be | |
714 | backed up at all. | |
715 | ||
716 | This simple mode is also used by the container restore operations in the web | |
717 | interface. | |
718 | ||
719 | ||
720 | ``Advanced'' Restore Mode | |
721 | ^^^^^^^^^^^^^^^^^^^^^^^^^ | |
722 | ||
723 | By setting the `rootfs` parameter (and optionally, any combination of `mpX` | |
724 | parameters), the `pct restore` command is automatically switched into an | |
725 | advanced mode. This advanced mode completely ignores the `rootfs` and `mpX` | |
726 | configuration options contained in the backup archive, and instead only uses | |
727 | the options explicitly provided as parameters. | |
728 | ||
729 | This mode allows flexible configuration of mount point settings at restore | |
730 | time, for example: | |
731 | ||
732 | * Set target storages, volume sizes and other options for each mount point | |
733 | individually | |
734 | * Redistribute backed up files according to new mount point scheme | |
735 | * Restore to device and/or bind mount points (limited to root user) | |
736 | ||
737 | ||
738 | Managing Containers with `pct` | |
739 | ------------------------------ | |
740 | ||
741 | The ``Proxmox Container Toolkit'' (`pct`) is the command line tool to manage | |
742 | {pve} containers. It enables you to create or destroy containers, as well as | |
743 | control the container execution (start, stop, reboot, migrate, etc.). It can be | |
744 | used to set parameters in the config file of a container, for example the | |
745 | network configuration or memory limits. | |
746 | ||
747 | CLI Usage Examples | |
748 | ~~~~~~~~~~~~~~~~~~ | |
749 | ||
750 | Create a container based on a Debian template (provided you have already | |
751 | downloaded the template via the web interface) | |
752 | ||
753 | ---- | |
754 | # pct create 100 /var/lib/vz/template/cache/debian-10.0-standard_10.0-1_amd64.tar.gz | |
755 | ---- | |
756 | ||
757 | Start container 100 | |
758 | ||
759 | ---- | |
760 | # pct start 100 | |
761 | ---- | |
762 | ||
763 | Start a login session via getty | |
764 | ||
765 | ---- | |
766 | # pct console 100 | |
767 | ---- | |
768 | ||
769 | Enter the LXC namespace and run a shell as root user | |
770 | ||
771 | ---- | |
772 | # pct enter 100 | |
773 | ---- | |
774 | ||
775 | Display the configuration | |
776 | ||
777 | ---- | |
778 | # pct config 100 | |
779 | ---- | |
780 | ||
781 | Add a network interface called `eth0`, bridged to the host bridge `vmbr0`, set | |
782 | the address and gateway, while it's running | |
783 | ||
784 | ---- | |
785 | # pct set 100 -net0 name=eth0,bridge=vmbr0,ip=192.168.15.147/24,gw=192.168.15.1 | |
786 | ---- | |
787 | ||
788 | Reduce the memory of the container to 512MB | |
789 | ||
790 | ---- | |
791 | # pct set 100 -memory 512 | |
792 | ---- | |
793 | ||
794 | ||
795 | Obtaining Debugging Logs | |
796 | ~~~~~~~~~~~~~~~~~~~~~~~~ | |
797 | ||
798 | In case `pct start` is unable to start a specific container, it might be | |
799 | helpful to collect debugging output by running `lxc-start` (replace `ID` with | |
800 | the container's ID): | |
801 | ||
802 | ---- | |
803 | # lxc-start -n ID -F -l DEBUG -o /tmp/lxc-ID.log | |
804 | ---- | |
805 | ||
806 | This command will attempt to start the container in foreground mode, to stop | |
807 | the container run `pct shutdown ID` or `pct stop ID` in a second terminal. | |
808 | ||
809 | The collected debug log is written to `/tmp/lxc-ID.log`. | |
810 | ||
811 | NOTE: If you have changed the container's configuration since the last start | |
812 | attempt with `pct start`, you need to run `pct start` at least once to also | |
813 | update the configuration used by `lxc-start`. | |
814 | ||
815 | [[pct_migration]] | |
816 | Migration | |
817 | --------- | |
818 | ||
819 | If you have a cluster, you can migrate your Containers with | |
820 | ||
821 | ---- | |
822 | # pct migrate <ctid> <target> | |
823 | ---- | |
824 | ||
825 | This works as long as your Container is offline. If it has local volumes or | |
826 | mount points defined, the migration will copy the content over the network to | |
827 | the target host if the same storage is defined there. | |
828 | ||
829 | Running containers cannot live-migrated due to technical limitations. You can | |
830 | do a restart migration, which shuts down, moves and then starts a container | |
831 | again on the target node. As containers are very lightweight, this results | |
832 | normally only in a downtime of some hundreds of milliseconds. | |
833 | ||
834 | A restart migration can be done through the web interface or by using the | |
835 | `--restart` flag with the `pct migrate` command. | |
836 | ||
837 | A restart migration will shut down the Container and kill it after the | |
838 | specified timeout (the default is 180 seconds). Then it will migrate the | |
839 | Container like an offline migration and when finished, it starts the Container | |
840 | on the target node. | |
841 | ||
842 | [[pct_configuration]] | |
843 | Configuration | |
844 | ------------- | |
845 | ||
846 | The `/etc/pve/lxc/<CTID>.conf` file stores container configuration, where | |
847 | `<CTID>` is the numeric ID of the given container. Like all other files stored | |
848 | inside `/etc/pve/`, they get automatically replicated to all other cluster | |
849 | nodes. | |
850 | ||
851 | NOTE: CTIDs < 100 are reserved for internal purposes, and CTIDs need to be | |
852 | unique cluster wide. | |
853 | ||
854 | .Example Container Configuration | |
855 | ---- | |
856 | ostype: debian | |
857 | arch: amd64 | |
858 | hostname: www | |
859 | memory: 512 | |
860 | swap: 512 | |
861 | net0: bridge=vmbr0,hwaddr=66:64:66:64:64:36,ip=dhcp,name=eth0,type=veth | |
862 | rootfs: local:107/vm-107-disk-1.raw,size=7G | |
863 | ---- | |
864 | ||
865 | The configuration files are simple text files. You can edit them using a normal | |
866 | text editor, for example, `vi` or `nano`. | |
867 | This is sometimes useful to do small corrections, but keep in mind that you | |
868 | need to restart the container to apply such changes. | |
869 | ||
870 | For that reason, it is usually better to use the `pct` command to generate and | |
871 | modify those files, or do the whole thing using the GUI. | |
872 | Our toolkit is smart enough to instantaneously apply most changes to running | |
873 | containers. This feature is called ``hot plug'', and there is no need to restart | |
874 | the container in that case. | |
875 | ||
876 | In cases where a change cannot be hot-plugged, it will be registered as a | |
877 | pending change (shown in red color in the GUI). | |
878 | They will only be applied after rebooting the container. | |
879 | ||
880 | ||
881 | File Format | |
882 | ~~~~~~~~~~~ | |
883 | ||
884 | The container configuration file uses a simple colon separated key/value | |
885 | format. Each line has the following format: | |
886 | ||
887 | ----- | |
888 | # this is a comment | |
889 | OPTION: value | |
890 | ----- | |
891 | ||
892 | Blank lines in those files are ignored, and lines starting with a `#` character | |
893 | are treated as comments and are also ignored. | |
894 | ||
895 | It is possible to add low-level, LXC style configuration directly, for example: | |
896 | ||
897 | ---- | |
898 | lxc.init_cmd: /sbin/my_own_init | |
899 | ---- | |
900 | ||
901 | or | |
902 | ||
903 | ---- | |
904 | lxc.init_cmd = /sbin/my_own_init | |
905 | ---- | |
906 | ||
907 | The settings are passed directly to the LXC low-level tools. | |
908 | ||
909 | ||
910 | [[pct_snapshots]] | |
911 | Snapshots | |
912 | ~~~~~~~~~ | |
913 | ||
914 | When you create a snapshot, `pct` stores the configuration at snapshot time | |
915 | into a separate snapshot section within the same configuration file. For | |
916 | example, after creating a snapshot called ``testsnapshot'', your configuration | |
917 | file will look like this: | |
918 | ||
919 | .Container configuration with snapshot | |
920 | ---- | |
921 | memory: 512 | |
922 | swap: 512 | |
923 | parent: testsnaphot | |
924 | ... | |
925 | ||
926 | [testsnaphot] | |
927 | memory: 512 | |
928 | swap: 512 | |
929 | snaptime: 1457170803 | |
930 | ... | |
931 | ---- | |
932 | ||
933 | There are a few snapshot related properties like `parent` and `snaptime`. The | |
934 | `parent` property is used to store the parent/child relationship between | |
935 | snapshots. `snaptime` is the snapshot creation time stamp (Unix epoch). | |
936 | ||
937 | ||
938 | [[pct_options]] | |
939 | Options | |
940 | ~~~~~~~ | |
941 | ||
942 | include::pct.conf.5-opts.adoc[] | |
943 | ||
944 | ||
945 | Locks | |
946 | ----- | |
947 | ||
948 | Container migrations, snapshots and backups (`vzdump`) set a lock to prevent | |
949 | incompatible concurrent actions on the affected container. Sometimes you need | |
950 | to remove such a lock manually (e.g., after a power failure). | |
951 | ||
952 | ---- | |
953 | # pct unlock <CTID> | |
954 | ---- | |
955 | ||
956 | CAUTION: Only do this if you are sure the action which set the lock is no | |
957 | longer running. | |
958 | ||
959 | ||
960 | ifdef::manvolnum[] | |
961 | ||
962 | Files | |
963 | ------ | |
964 | ||
965 | `/etc/pve/lxc/<CTID>.conf`:: | |
966 | ||
967 | Configuration file for the container '<CTID>'. | |
968 | ||
969 | ||
970 | include::pve-copyright.adoc[] | |
971 | endif::manvolnum[] |