+[[chapter_pct]]
ifdef::manvolnum[]
-PVE({manvolnum})
-================
+pct(1)
+======
include::attributes.txt[]
+:pve-toplevel:
NAME
----
pct - Tool to manage Linux Containers (LXC) on Proxmox VE
-SYNOPSYS
+SYNOPSIS
--------
include::pct.1-synopsis.adoc[]
Proxmox Container Toolkit
=========================
include::attributes.txt[]
+:pve-toplevel:
endif::manvolnum[]
-
+ifdef::wiki[]
+:title: Linux Container
+endif::wiki[]
Containers are a lightweight alternative to fully virtualized
VMs. Instead of emulating a complete Operating System (OS), containers
is best to run them inside a VM.
+Technology Overview
+-------------------
+
+* LXC (https://linuxcontainers.org/)
+
+* Integrated into {pve} graphical user interface (GUI)
+
+* Easy to use command line tool `pct`
+
+* Access via {pve} REST API
+
+* lxcfs to provide containerized /proc file system
+
+* AppArmor/Seccomp to improve security
+
+* CRIU: for live migration (planned)
+
+* Use latest available kernels (4.4.X)
+
+* Image based deployment (templates)
+
+* Use {pve} storage library
+
+* Container setup from host (network, DNS, storage, ...)
+
+
Security Considerations
-----------------------
AppArmor, CGroups and PID and user namespaces, which makes containers
usage quite secure. We distinguish two types of containers:
-Privileged containers
+
+Privileged Containers
~~~~~~~~~~~~~~~~~~~~~
Security is done by dropping capabilities, using mandatory access
trusted environment, or when no untrusted task is running as root in
the container.
-Unprivileged containers
+
+Unprivileged Containers
~~~~~~~~~~~~~~~~~~~~~~~
This kind of containers use a new kernel feature called user
-namespaces. The root uid 0 inside the container is mapped to an
+namespaces. The root UID 0 inside the container is mapped to an
unprivileged user outside the container. This means that most security
issues (container escape, resource abuse, ...) in those containers
will affect a random unprivileged user, and so would be a generic
unprivileged containers are safe by design.
-Configuration
--------------
-
-The `/etc/pve/lxc/<CTID>.conf` file stores container configuration,
-where `<CTID>` is the numeric ID of the given container. Like all
-other files stored inside `/etc/pve/`, they get automatically
-replicated to all other cluster nodes.
-
-NOTE: CTIDs < 100 are reserved for internal purposes, and CTIDs need to be
-unique cluster wide.
-
-.Example Container Configuration
-----
-ostype: debian
-arch: amd64
-hostname: www
-memory: 512
-swap: 512
-net0: bridge=vmbr0,hwaddr=66:64:66:64:64:36,ip=dhcp,name=eth0,type=veth
-rootfs: local:107/vm-107-disk-1.raw,size=7G
-----
-
-Those configuration files are simple text files, and you can edit them
-using a normal text editor (`vi`, `nano`, ...). This is sometimes
-useful to do small corrections, but keep in mind that you need to
-restart the container to apply such changes.
-
-For that reason, it is usually better to use the `pct` command to
-generate and modify those files, or do the whole thing using the GUI.
-Our toolkit is smart enough to instantaneously apply most changes to
-running containers. This feature is called "hot plug", and there is no
-need to restart the container in that case.
-
-File Format
-~~~~~~~~~~~
-
-Container configuration files use a simple colon separated key/value
-format. Each line has the following format:
-
- # this is a comment
- OPTION: value
-
-Blank lines in those files are ignored, and lines starting with a `#`
-character are treated as comments and are also ignored.
-
-It is possible to add low-level, LXC style configuration directly, for
-example:
-
- lxc.init_cmd: /sbin/my_own_init
-
-or
-
- lxc.init_cmd = /sbin/my_own_init
-
-Those settings are directly passed to the LXC low-level tools.
-
-Snapshots
-~~~~~~~~~
-
-When you create a snapshot, `pct` stores the configuration at snapshot
-time into a separate snapshot section within the same configuration
-file. For example, after creating a snapshot called ``testsnapshot'',
-your configuration file will look like this:
-
-.Container Configuration with Snapshot
-----
-memory: 512
-swap: 512
-parent: testsnaphot
-...
-
-[testsnaphot]
-memory: 512
-swap: 512
-snaptime: 1457170803
-...
-----
-
-There are a few snapshot related properties like `parent` and
-`snaptime`. The `parent` property is used to store the parent/child
-relationship between snapshots. `snaptime` is the snapshot creation
-time stamp (Unix epoch).
-
-
Guest Operating System Configuration
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+------------------------------------
We normally try to detect the operating system type inside the
container, and then modify some files inside the container to make
NOTE: Container start fails if the configured `ostype` differs from the auto
detected type.
-Options
-~~~~~~~
-
-include::pct.conf.5-opts.adoc[]
-
+[[pct_container_images]]
Container Images
----------------
pct create 999 local:vztmpl/debian-8.0-standard_8.0-1_amd64.tar.gz
-Proxmox itself ships a set of basic templates for most common
+{pve} itself ships a set of basic templates for most common
operating systems, and you can download them using the `pveam` (short
for {pve} Appliance Manager) command line utility. You can also
download https://www.turnkeylinux.org/[TurnKey Linux] containers using
The above command shows you the full {pve} volume identifiers. They include
the storage name, and most other {pve} commands can use them. For
-examply you can delete that image later with:
+example you can delete that image later with:
pveam remove local:vztmpl/debian-8.0-standard_8.0-1_amd64.tar.gz
+[[pct_container_storage]]
Container Storage
-----------------
example, you can use a relatively slow (and thus cheap) storage for
the container root file system. Then you can use a second mount point
to mount a very fast, distributed storage for your database
-application.
+application. See section <<pct_mount_points,Mount Points>> for further
+details.
The second big improvement is that you can use any storage type
supported by the {pve} storage library. That means that you can store
also provide an easy way to share data between different containers.
+FUSE Mounts
+~~~~~~~~~~~
+
+WARNING: Because of existing issues in the Linux kernel's freezer
+subsystem the usage of FUSE mounts inside a container is strongly
+advised against, as containers need to be frozen for suspend or
+snapshot mode backups.
+
+If FUSE mounts cannot be replaced by other mounting mechanisms or storage
+technologies, it is possible to establish the FUSE mount on the Proxmox host
+and use a bind mount point to make it accessible inside the container.
+
+
+Using Quotas Inside Containers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Quotas allow to set limits inside a container for the amount of disk
+space that each user can use. This only works on ext4 image based
+storage types and currently does not work with unprivileged
+containers.
+
+Activating the `quota` option causes the following mount options to be
+used for a mount point:
+`usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0`
+
+This allows quotas to be used like you would on any other system. You
+can initialize the `/aquota.user` and `/aquota.group` files by running
+
+----
+quotacheck -cmug /
+quotaon /
+----
+
+and edit the quotas via the `edquota` command. Refer to the documentation
+of the distribution running inside the container for details.
+
+NOTE: You need to run the above commands for every mount point by passing
+the mount point's path instead of just `/`.
+
+
+Using ACLs Inside Containers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The standard Posix **A**ccess **C**ontrol **L**ists are also available inside containers.
+ACLs allow you to set more detailed file ownership than the traditional user/
+group/others model.
+
+
+[[pct_setting]]
+Container Settings
+------------------
+
+[[pct_cpu]]
+
+CPU
+~~~
+
+You can restrict the number of visible CPUs inside the container using
+the `cores` option. This is implemented using the Linux 'cpuset'
+cgroup (**c**ontrol *group*). A special task inside `pvestatd` tries
+to distribute running containers among available CPUs. You can view
+the assigned CPUs using the following command:
+
+----
+# pct cpusets
+ ---------------------
+ 102: 6 7
+ 105: 2 3 4 5
+ 108: 0 1
+ ---------------------
+----
+
+Containers use the host kernel directly, so all task inside a
+container are handled by the host CPU scheduler. {pve} uses the Linux
+'CFS' (**C**ompletely **F**air **S**cheduler) scheduler by default,
+which has additional bandwidth control options.
+
+[horizontal]
+cpulimit: :: You can use this option to further limit assigned CPU
+time. Please note that this is a floating point number, so it is
+perfectly valid to assign two cores to a container, but restrict
+overall CPU consumption to half a core.
++
+----
+cores: 2
+cpulimit: 0.5
+----
+
+cpuunits: :: This is a relative weight passed to the kernel
+scheduler. The larger the number is, the more CPU time this container
+gets. Number is relative to the weights of all the other running
+containers. The default is 1024. You can use this setting to
+prioritize some containers.
+
+
+[[pct_memory]]
+Memory
+~~~~~~
+
+Container memory is controlled using the cgroup memory controller.
+
+[horizontal]
+
+memory: :: Limit overall memory usage. This corresponds
+to the `memory.limit_in_bytes` cgroup setting.
+
+swap: :: Allows the container to use additional swap memory from the
+host swap space. This corresponds to the `memory.memsw.limit_in_bytes`
+cgroup setting, which is set to the sum of both value (`memory +
+swap`).
+
+
+[[pct_mount_points]]
Mount Points
~~~~~~~~~~~~
Currently there are basically three types of mount points: storage backed
mount points, bind mounts and device mounts.
-.Typical Container `rootfs` configuration
+.Typical container `rootfs` configuration
----
rootfs: thin1:base-100-disk-1,size=8G
----
-Storage backed mount points
+Storage Backed Mount Points
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Storage backed mount points are managed by the {pve} storage subsystem and come
in three different flavors:
-- Image based: These are raw images containing a single ext4 formatted file
+- Image based: these are raw images containing a single ext4 formatted file
system.
-- ZFS Subvolumes: These are technically bind mounts, but with managed storage,
+- ZFS subvolumes: these are technically bind mounts, but with managed storage,
and thus allow resizing and snapshotting.
- Directories: passing `size=0` triggers a special case where instead of a raw
image a directory is created.
-Bind mount points
+Bind Mount Points
^^^^^^^^^^^^^^^^^
Bind mounts allow you to access arbitrary directories from your Proxmox VE host
achieve the same result.
-Device mount points
+Device Mount Points
^^^^^^^^^^^^^^^^^^^
Device mount points allow to mount block devices of the host directly into the
NOTE: The contents of device mount points are not backed up when using `vzdump`.
-FUSE mounts
-~~~~~~~~~~~
-
-WARNING: Because of existing issues in the Linux kernel's freezer
-subsystem the usage of FUSE mounts inside a container is strongly
-advised against, as containers need to be frozen for suspend or
-snapshot mode backups.
-
-If FUSE mounts cannot be replaced by other mounting mechanisms or storage
-technologies, it is possible to establish the FUSE mount on the Proxmox host
-and use a bind mount point to make it accessible inside the container.
-
-
-Using quotas inside containers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Quotas allow to set limits inside a container for the amount of disk
-space that each user can use. This only works on ext4 image based
-storage types and currently does not work with unprivileged
-containers.
-
-Activating the `quota` option causes the following mount options to be
-used for a mount point:
-`usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0`
-
-This allows quotas to be used like you would on any other system. You
-can initialize the `/aquota.user` and `/aquota.group` files by running
-
-----
-quotacheck -cmug /
-quotaon /
-----
-
-and edit the quotas via the `edquota` command. Refer to the documentation
-of the distribution running inside the container for details.
-
-NOTE: You need to run the above commands for every mount point by passing
-the mount point's path instead of just `/`.
-
-
-Using ACLs inside containers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The standard Posix Access Control Lists are also available inside containers.
-ACLs allow you to set more detailed file ownership than the traditional user/
-group/others model.
-
-
+[[pct_container_network]]
Container Network
------------------
+~~~~~~~~~~~~~~~~~
You can configure up to 10 network interfaces for a single
container. The corresponding options are called `net0` to `net9`, and
Backup and Restore
------------------
+
Container Backup
~~~~~~~~~~~~~~~~
...). You can use pct to set parameters in the associated config file,
like network configuration or memory limits.
+
CLI Usage Examples
~~~~~~~~~~~~~~~~~~
Create a container based on a Debian template (provided you have
-already downloaded the template via the webgui)
+already downloaded the template via the web interface)
pct create 100 /var/lib/vz/template/cache/debian-8.0-standard_8.0-1_amd64.tar.gz
pct set 100 -memory 512
-Files
-------
+Obtaining Debugging Logs
+~~~~~~~~~~~~~~~~~~~~~~~~
-`/etc/pve/lxc/<CTID>.conf`::
+In case `pct start` is unable to start a specific container, it might be
+helpful to collect debugging output by running `lxc-start` (replace `ID` with
+the container's ID):
-Configuration file for the container '<CTID>'.
+ lxc-start -n ID -F -l DEBUG -o /tmp/lxc-ID.log
+
+This command will attempt to start the container in foreground mode, to stop the container run `pct shutdown ID` or `pct stop ID` in a second terminal.
+
+The collected debug log is written to `/tmp/lxc-ID.log`.
+
+NOTE: If you have changed the container's configuration since the last start
+attempt with `pct start`, you need to run `pct start` at least once to also
+update the configuration used by `lxc-start`.
+
+
+[[pct_configuration]]
+Configuration
+-------------
+
+The `/etc/pve/lxc/<CTID>.conf` file stores container configuration,
+where `<CTID>` is the numeric ID of the given container. Like all
+other files stored inside `/etc/pve/`, they get automatically
+replicated to all other cluster nodes.
+NOTE: CTIDs < 100 are reserved for internal purposes, and CTIDs need to be
+unique cluster wide.
-Container Advantages
---------------------
+.Example Container Configuration
+----
+ostype: debian
+arch: amd64
+hostname: www
+memory: 512
+swap: 512
+net0: bridge=vmbr0,hwaddr=66:64:66:64:64:36,ip=dhcp,name=eth0,type=veth
+rootfs: local:107/vm-107-disk-1.raw,size=7G
+----
+
+Those configuration files are simple text files, and you can edit them
+using a normal text editor (`vi`, `nano`, ...). This is sometimes
+useful to do small corrections, but keep in mind that you need to
+restart the container to apply such changes.
-* Simple, and fully integrated into {pve}. Setup looks similar to a normal
- VM setup.
+For that reason, it is usually better to use the `pct` command to
+generate and modify those files, or do the whole thing using the GUI.
+Our toolkit is smart enough to instantaneously apply most changes to
+running containers. This feature is called "hot plug", and there is no
+need to restart the container in that case.
-** Storage (ZFS, LVM, NFS, Ceph, ...)
-** Network
+File Format
+~~~~~~~~~~~
-** Authentication
+Container configuration files use a simple colon separated key/value
+format. Each line has the following format:
-** Cluster
+-----
+# this is a comment
+OPTION: value
+-----
-* Fast: minimal overhead, as fast as bare metal
+Blank lines in those files are ignored, and lines starting with a `#`
+character are treated as comments and are also ignored.
-* High density (perfect for idle workloads)
+It is possible to add low-level, LXC style configuration directly, for
+example:
-* REST API
+ lxc.init_cmd: /sbin/my_own_init
-* Direct hardware access
+or
+ lxc.init_cmd = /sbin/my_own_init
-Technology Overview
--------------------
+Those settings are directly passed to the LXC low-level tools.
-- Integrated into {pve} graphical user interface (GUI)
-- LXC (https://linuxcontainers.org/)
+[[pct_snapshots]]
+Snapshots
+~~~~~~~~~
+
+When you create a snapshot, `pct` stores the configuration at snapshot
+time into a separate snapshot section within the same configuration
+file. For example, after creating a snapshot called ``testsnapshot'',
+your configuration file will look like this:
+
+.Container configuration with snapshot
+----
+memory: 512
+swap: 512
+parent: testsnaphot
+...
+
+[testsnaphot]
+memory: 512
+swap: 512
+snaptime: 1457170803
+...
+----
+
+There are a few snapshot related properties like `parent` and
+`snaptime`. The `parent` property is used to store the parent/child
+relationship between snapshots. `snaptime` is the snapshot creation
+time stamp (Unix epoch).
-- cgmanager for cgroup management
-- lxcfs to provive containerized /proc file system
+[[pct_options]]
+Options
+~~~~~~~
+
+include::pct.conf.5-opts.adoc[]
-- apparmor
-- CRIU: for live migration (planned)
+Locks
+-----
-- We use latest available kernels (4.4.X)
+Container migrations, snapshots and backups (`vzdump`) set a lock to
+prevent incompatible concurrent actions on the affected container. Sometimes
+you need to remove such a lock manually (e.g., after a power failure).
-- Image based deployment (templates)
+ pct unlock <CTID>
-- Container setup from host (Network, DNS, Storage, ...)
+CAUTION: Only do that if you are sure the action which set the lock is
+no longer running.
ifdef::manvolnum[]
+
+Files
+------
+
+`/etc/pve/lxc/<CTID>.conf`::
+
+Configuration file for the container '<CTID>'.
+
+
include::pve-copyright.adoc[]
endif::manvolnum[]
projects / pve-docs.git / blobdiff