tree-wide: replace problematic terminology

[mirror_lxc.git] / doc / lxc.container.conf.sgml.in

<!--

lxc: linux Container library

(C) Copyright IBM Corp. 2007, 2008

Authors:
Daniel Lezcano <daniel.lezcano at free.fr>

This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA

-->

<!DOCTYPE refentry PUBLIC @docdtd@ [

<!ENTITY seealso SYSTEM "@builddir@/see_also.sgml">
]>

<refentry>

  <docinfo><date>@LXC_GENERATE_DATE@</date></docinfo>

  <refmeta>
    <refentrytitle>lxc.container.conf</refentrytitle>
    <manvolnum>5</manvolnum>
  </refmeta>

  <refnamediv>
    <refname>lxc.container.conf</refname>

    <refpurpose>
      LXC container configuration file
    </refpurpose>
  </refnamediv>

  <refsect1>
    <title>Description</title>

    <para>
      LXC is the well-known and heavily tested low-level Linux container
      runtime. It is in active development since 2008 and has proven itself in
      critical production environments world-wide. Some of its core contributors
      are the same people that helped to implement various well-known
      containerization features inside the Linux kernel.
    </para>

    <para>
      LXC's main focus is system containers. That is, containers which offer an
      environment as close as possible as the one you'd get from a VM but
      without the overhead that comes with running a separate kernel and
      simulating all the hardware.
    </para>

    <para>
      This is achieved through a combination of kernel security features such as
      namespaces, mandatory access control and control groups.
    </para>

    <para>
      LXC has support for unprivileged containers.  Unprivileged containers are
      containers that are run without any privilege.  This requires support for
      user namespaces in the kernel that the container is run on.  LXC was the
      first runtime to support unprivileged containers after user namespaces
      were merged into the mainline kernel.
    </para>

    <para>
      In essence, user namespaces isolate given sets of UIDs and GIDs. This is
      achieved by establishing a mapping between a range of UIDs and GIDs on the
      host to a different (unprivileged) range of UIDs and GIDs in the
      container. The kernel will translate this mapping in such a way that
      inside the container all UIDs and GIDs appear as you would expect from the
      host whereas on the host these UIDs and GIDs are in fact unprivileged. For
      example, a process running as UID and GID 0 inside the container might
      appear as UID and GID 100000 on the host.  The implementation and working
      details can be gathered from the corresponding user namespace man page.
      UID and GID mappings can be defined with the <option>lxc.idmap</option>
      key.
    </para>

    <para>
      Linux containers are defined with a simple configuration file.  Each
      option in the configuration file has the form <command>key =
      value</command> fitting in one line. The "#" character means the line is a
      comment. List options, like capabilities and cgroups options, can be used
      with no value to clear any previously defined values of that option.
      </para>

    <para>
      LXC namespaces configuration keys use single dots. This means complex
      configuration keys such as <option>lxc.net.0</option> expose various
      subkeys such as <option>lxc.net.0.type</option>,
      <option>lxc.net.0.link</option>, <option>lxc.net.0.ipv6.address</option>, and
      others for even more fine-grained configuration.
    </para>

    <refsect2>
      <title>Configuration</title>
      <para>
        In order to ease administration of multiple related containers, it is
        possible to have a container configuration file cause another file to be
        loaded. For instance, network configuration can be defined in one common
        file which is included by multiple containers. Then, if the containers
        are moved to another host, only one file may need to be updated.
      </para>

      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.include</option>
          </term>
          <listitem>
            <para>
              Specify the file to be included.  The included file must be
              in the same valid lxc configuration file format.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Architecture</title>
      <para>
        Allows one to set the architecture for the container. For example, set a
        32bits architecture for a container running 32bits binaries on a 64bits
        host. This fixes the container scripts which rely on the architecture to
        do some work like downloading the packages.
      </para>

      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.arch</option>
          </term>
          <listitem>
            <para>
              Specify the architecture for the container.
            </para>
            <para>
              Some valid options are
              <option>x86</option>,
              <option>i686</option>,
              <option>x86_64</option>,
              <option>amd64</option>
            </para>
          </listitem>
        </varlistentry>
      </variablelist>

    </refsect2>

    <refsect2>
      <title>Hostname</title>
      <para>
        The utsname section defines the hostname to be set for the container.
        That means the container can set its own hostname without changing the
        one from the system. That makes the hostname private for the container.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.uts.name</option>
          </term>
          <listitem>
            <para>
              specify the hostname for the container
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Halt signal</title>
      <para>
        Allows one to specify signal name or number sent to the container's
        init process to cleanly shutdown the container. Different init systems
        could use different signals to perform clean shutdown sequence. This
        option allows the signal to be specified in kill(1) fashion, e.g.
        SIGPWR, SIGRTMIN+14, SIGRTMAX-10 or plain number. The default signal is
        SIGPWR.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.signal.halt</option>
          </term>
          <listitem>
            <para>
              specify the signal used to halt the container
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Reboot signal</title>
      <para>
        Allows one to specify signal name or number to reboot the container.
        This option allows signal to be specified in kill(1) fashion, e.g.
        SIGTERM, SIGRTMIN+14, SIGRTMAX-10 or plain number. The default signal
        is SIGINT.
          </para>
          <variablelist>
        <varlistentry>
          <term>
            <option>lxc.signal.reboot</option>
          </term>
          <listitem>
            <para>
              specify the signal used to reboot the container
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Stop signal</title>
      <para>
        Allows one to specify signal name or number to forcibly shutdown the
        container. This option allows signal to be specified in kill(1) fashion,
        e.g. SIGKILL, SIGRTMIN+14, SIGRTMAX-10 or plain number.  The default
        signal is SIGKILL.
          </para>
          <variablelist>
        <varlistentry>
          <term>
            <option>lxc.signal.stop</option>
          </term>
          <listitem>
            <para>
              specify the signal used to stop the container
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Init command</title>
      <para>
        Sets the command to use as the init system for the containers.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.execute.cmd</option>
          </term>
          <listitem>
            <para>
              Absolute path from container rootfs to the binary to run by default.  This
              mostly makes sense for <command>lxc-execute</command>.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.init.cmd</option>
          </term>
          <listitem>
            <para>
              Absolute path from container rootfs to the binary to use as init. This
              mostly makes sense for <command>lxc-start</command>. Default is <command>/sbin/init</command>.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Init working directory</title>
      <para>
        Sets the absolute path inside the container as the working directory for the containers.
        LXC will switch to this directory before executing init.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.init.cwd</option>
          </term>
          <listitem>
            <para>
              Absolute path inside the container to use as the working directory.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Init ID</title>
      <para>
        Sets the UID/GID to use for the init system, and subsequent commands.
        Note that using a non-root UID when booting a system container will
        likely not work due to missing privileges. Setting the UID/GID is mostly
        useful when running application containers.

        Defaults to: UID(0), GID(0)
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.init.uid</option>
          </term>
          <listitem>
            <para>
              UID to use for init.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.init.gid</option>
          </term>
          <listitem>
            <para>
              GID to use for init.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Proc</title>
      <para>
        Configure proc filesystem for the container.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.proc.[proc file name]</option>
          </term>
          <listitem>
            <para>
              Specify the proc file name to be set. The file names available
              are those listed under /proc/PID/.
              Example:
            </para>
            <programlisting>
              lxc.proc.oom_score_adj = 10
            </programlisting>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Ephemeral</title>
      <para>
        Allows one to specify whether a container will be destroyed on shutdown.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.ephemeral</option>
          </term>
          <listitem>
            <para>
              The only allowed values are 0 and 1. Set this to 1 to destroy a
              container on shutdown.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Network</title>
      <para>
        The network section defines how the network is virtualized in
        the container. The network virtualization acts at layer
        two. In order to use the network virtualization, parameters
        must be specified to define the network interfaces of the
        container. Several virtual interfaces can be assigned and used
        in a container even if the system has only one physical
        network interface.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.net</option>
          </term>
          <listitem>
            <para>
              may be used without a value to clear all previous network options.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.net.[i].type</option>
          </term>
          <listitem>
            <para>
              specify what kind of network virtualization to be used
              for the container.
              Must be specified before any other option(s) on the net device.
              Multiple networks can be specified by using an additional index
              <option>i</option>
              after all <option>lxc.net.*</option> keys. For example,
              <option>lxc.net.0.type = veth</option> and
              <option>lxc.net.1.type = veth</option> specify two different
              networks of the same type. All keys sharing the same index
              <option>i</option> will be treated as belonging to the same
              network. For example, <option>lxc.net.0.link = br0</option>
              will belong to <option>lxc.net.0.type</option>.
              Currently, the different virtualization types can be:
            </para>

            <para>
              <option>none:</option> will cause the container to share
              the host's network namespace.  This means the host
              network devices are usable in the container.  It also
              means that if both the container and host have upstart as
              init, 'halt' in a container (for instance) will shut down the
              host. Note that unprivileged containers do not work with this
              setting due to an inability to mount sysfs. An unsafe workaround
              would be to bind mount the host's sysfs.
            </para>

            <para>
              <option>empty:</option> will create only the loopback
              interface.
            </para>

            <para>
              <option>veth:</option> a virtual ethernet pair
              device is created with one side assigned to the container
              and the other side on the host.
              <option>lxc.net.[i].veth.mode</option> specifies the
              mode the veth parent will use on the host.
              The accepted  modes are <option>bridge</option> and <option>router</option>.
              The mode defaults to bridge if not specified.
              In <option>bridge</option> mode the host side is attached to a bridge specified by
              the <option>lxc.net.[i].link</option> option.
              If the bridge link is not specified, then the veth pair device
              will be created but not attached to any bridge.
              Otherwise, the bridge has to be created on the system
              before starting the container.
              <command>lxc</command> won't handle any
              configuration outside of the container.
              In <option>router</option> mode static routes are created on the host for the
              container's IP addresses pointing to the host side veth interface.
              Additionally Proxy ARP and Proxy NDP entries are added on the host side veth interface
              for the gateway IPs defined in the container to allow the container to reach the host.
              By default, <command>lxc</command> chooses a name for the
              network device belonging to the outside of the
              container, but if you wish to handle
              this name yourselves, you can tell <command>lxc</command>
              to set a specific name with
              the <option>lxc.net.[i].veth.pair</option> option (except for
              unprivileged containers where this option is ignored for security
              reasons).

              Static routes can be added on the host pointing to the container using the
              <option>lxc.net.[i].veth.ipv4.route</option> and
              <option>lxc.net.[i].veth.ipv6.route</option> options.
              Several lines specify several routes.
              The route is in format x.y.z.t/m, eg. 192.168.1.0/24.

              In <option>bridge</option> mode untagged VLAN membership can be set with the
              <option>lxc.net.[i].veth.vlan.id</option> option. It accepts a special value of 'none' indicating
              that the container port should be removed from the bridge's default untagged VLAN.
              The <option>lxc.net.[i].veth.vlan.tagged.id</option> option can be specified multiple times to set
              the container's bridge port membership to one or more tagged VLANs.
            </para>

            <para>
              <option>vlan:</option> a vlan interface is linked with
              the interface specified by
              the <option>lxc.net.[i].link</option> and assigned to
              the container. The vlan identifier is specified with the
              option <option>lxc.net.[i].vlan.id</option>.
            </para>

            <para>
              <option>macvlan:</option> a macvlan interface is linked
              with the interface specified by
              the <option>lxc.net.[i].link</option> and assigned to
              the container.
              <option>lxc.net.[i].macvlan.mode</option> specifies the
              mode the macvlan will use to communicate between
              different macvlan on the same upper device. The accepted
              modes are <option>private</option>, <option>vepa</option>,
              <option>bridge</option> and <option>passthru</option>.
              In <option>private</option> mode, the device never
              communicates with any other device on the same upper_dev (default).
              In <option>vepa</option> mode, the new Virtual Ethernet Port
              Aggregator (VEPA) mode, it assumes that the adjacent
              bridge returns all frames where both source and
              destination are local to the macvlan port, i.e. the
              bridge is set up as a reflective relay.  Broadcast
              frames coming in from the upper_dev get flooded to all
              macvlan interfaces in VEPA mode, local frames are not
              delivered locally. In <option>bridge</option> mode, it
              provides the behavior of a simple bridge between
              different macvlan interfaces on the same port. Frames
              from one interface to another one get delivered directly
              and are not sent out externally. Broadcast frames get
              flooded to all other bridge ports and to the external
              interface, but when they come back from a reflective
              relay, we don't deliver them again.  Since we know all
              the MAC addresses, the macvlan bridge mode does not
              require learning or STP like the bridge module does. In
              <option>passthru</option> mode, all frames received by
              the physical interface are forwarded to the macvlan
              interface. Only one macvlan interface in <option>passthru</option>
              mode is possible for one physical interface.
            </para>

            <para>
              <option>ipvlan:</option> an ipvlan interface is linked
              with the interface specified by
              the <option>lxc.net.[i].link</option> and assigned to
              the container.
              <option>lxc.net.[i].ipvlan.mode</option> specifies the
              mode the ipvlan will use to communicate between
              different ipvlan on the same upper device. The accepted
              modes are <option>l3</option>, <option>l3s</option> and
              <option>l2</option>. It defaults to <option>l3</option> mode.
              In <option>l3</option> mode TX processing up to L3 happens on the stack instance
              attached to the dependent device and packets are switched to the stack instance of the
              parent device for the L2 processing and routing from that instance will be
              used before packets are queued on the outbound device. In this mode the dependent devices
              will not receive nor can send multicast / broadcast traffic.
              In <option>l3s</option> mode TX processing is very similar to the L3 mode except that
              iptables (conn-tracking) works in this mode and hence it is L3-symmetric (L3s).
              This will have slightly less performance but that shouldn't matter since you are
              choosing this mode over plain-L3 mode to make conn-tracking work.
              In <option>l2</option> mode TX processing happens on the stack instance attached to
              the dependent device and packets are switched and queued to the parent device to send devices
              out. In this mode the dependent devices will RX/TX multicast and broadcast (if applicable) as well.
              <option>lxc.net.[i].ipvlan.isolation</option> specifies the isolation mode.
              The accepted isolation values are <option>bridge</option>,
              <option>private</option> and <option>vepa</option>.
              It defaults to <option>bridge</option>.
              In <option>bridge</option> isolation mode dependent devices can cross-talk among themselves
              apart from talking through the parent device.
              In <option>private</option> isolation mode the port is set in private mode.
              i.e. port won't allow cross communication between dependent devices.
              In <option>vepa</option> isolation mode the port is set in VEPA mode.
              i.e. port will offload switching functionality to the external entity as
              described in 802.1Qbg.
            </para>

            <para>
              <option>phys:</option> an already existing interface
              specified by the <option>lxc.net.[i].link</option> is
              assigned to the container.
            </para>
          </listitem>
          </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].flags</option>
          </term>
          <listitem>
            <para>
              Specify an action to do for the network.
            </para>

            <para><option>up:</option> activates the interface.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].link</option>
          </term>
          <listitem>
            <para>
              Specify the interface to be used for real network traffic.
              </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].l2proxy</option>
          </term>
          <listitem>
            <para>
              Controls whether layer 2 IP neighbour proxy entries will be added to the
              lxc.net.[i].link interface for the IP addresses of the container.
              Can be set to 0 or 1. Defaults to 0.
              When used with IPv4 addresses, the following sysctl values need to be set:
              net.ipv4.conf.[link].forwarding=1
              When used with IPv6 addresses, the following sysctl values need to be set:
              net.ipv6.conf.[link].proxy_ndp=1
              net.ipv6.conf.[link].forwarding=1
              </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].mtu</option>
          </term>
          <listitem>
            <para>
              Specify the maximum transfer unit for this interface.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].name</option>
          </term>
          <listitem>
            <para>
              The interface name is dynamically allocated, but if another name
              is needed because the configuration files being used by the
              container use a generic name, eg. eth0, this option will rename
              the interface in the container.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].hwaddr</option>
          </term>
          <listitem>
            <para>
              The interface mac address is dynamically allocated by default to
              the virtual interface, but in some cases, this is needed to
              resolve a mac address conflict or to always have the same
              link-local ipv6 address.  Any "x" in address will be replaced by
              random value, this allows setting hwaddr templates.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].ipv4.address</option>
          </term>
          <listitem>
            <para>
              Specify the ipv4 address to assign to the virtualized interface.
              Several lines specify several ipv4 addresses. The address is in
              format x.y.z.t/m, eg. 192.168.1.123/24.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].ipv4.gateway</option>
          </term>
          <listitem>
            <para>
              Specify the ipv4 address to use as the gateway inside the
              container. The address is in format x.y.z.t, eg.  192.168.1.123.

              Can also have the special value <option>auto</option>,
              which means to take the primary address from the bridge
              interface (as specified by the
              <option>lxc.net.[i].link</option> option) and use that as
              the gateway. <option>auto</option> is only available when
              using the <option>veth</option>,
              <option>macvlan</option> and <option>ipvlan</option> network types.
              Can also have the special value of <option>dev</option>,
              which means to set the default gateway as a device route.
              This is primarily for use with layer 3 network modes, such as IPVLAN.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].ipv6.address</option>
          </term>
          <listitem>
            <para>
              Specify the ipv6 address to assign to the virtualized
              interface. Several lines specify several ipv6 addresses. The
              address is in format x::y/m, eg.
              2003:db8:1:0:214:1234:fe0b:3596/64
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].ipv6.gateway</option>
          </term>
          <listitem>
            <para>
              Specify the ipv6 address to use as the gateway inside the
              container. The address is in format x::y, eg. 2003:db8:1:0::1

              Can also have the special value <option>auto</option>,
              which means to take the primary address from the bridge
              interface (as specified by the
              <option>lxc.net.[i].link</option> option) and use that as
              the gateway. <option>auto</option> is only available when
              using the <option>veth</option>,
              <option>macvlan</option> and <option>ipvlan</option> network types.
              Can also have the special value of <option>dev</option>,
              which means to set the default gateway as a device route.
              This is primarily for use with layer 3 network modes, such as IPVLAN.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].script.up</option>
          </term>
          <listitem>
            <para>
              Add a configuration option to specify a script to be
              executed after creating and configuring the network used
              from the host side.
            </para>

            <para>
              In addition to the information available to all hooks. The
              following information is provided to the script:
              <itemizedlist>
                <listitem>
                 <para>
                 LXC_HOOK_TYPE: the hook type. This is either 'up' or 'down'.
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_HOOK_SECTION: the section type 'net'.
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_NET_TYPE: the network type. This is one of the valid
                 network types listed here (e.g. 'vlan', 'macvlan', 'ipvlan', 'veth').
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_NET_PARENT: the parent device on the host. This is only
                 set for network types 'mavclan', 'veth', 'phys'.
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_NET_PEER: the name of the peer device on the host. This is
                 only set for 'veth' network types. Note that this information
                 is only available when <option>lxc.hook.version</option> is set
                 to 1.
                  </para>
                </listitem>
              </itemizedlist>

              Whether this information is provided in the form of environment
              variables or as arguments to the script depends on the value of
              <option>lxc.hook.version</option>. If set to 1 then information is
              provided in the form of environment variables. If set to 0
              information is provided as arguments to the script.
            </para>

            <para>
              Standard output from the script is logged at debug level.
              Standard error is not logged, but can be captured by the
              hook redirecting its standard error to standard output.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.net.[i].script.down</option>
          </term>
          <listitem>
            <para>
              Add a configuration option to specify a script to be
              executed before destroying the network used from the
              host side.
            </para>

            <para>
              In addition to the information available to all hooks. The
              following information is provided to the script:
              <itemizedlist>
                <listitem>
                 <para>
                 LXC_HOOK_TYPE: the hook type. This is either 'up' or 'down'.
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_HOOK_SECTION: the section type 'net'.
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_NET_TYPE: the network type. This is one of the valid
                 network types listed here (e.g. 'vlan', 'macvlan', 'ipvlan', 'veth').
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_NET_PARENT: the parent device on the host. This is only
                 set for network types 'mavclan', 'veth', 'phys'.
                  </para>
                </listitem>

                <listitem>
                 <para>
                 LXC_NET_PEER: the name of the peer device on the host. This is
                 only set for 'veth' network types. Note that this information
                 is only available when <option>lxc.hook.version</option> is set
                 to 1.
                  </para>
                </listitem>
              </itemizedlist>

              Whether this information is provided in the form of environment
              variables or as arguments to the script depends on the value of
              <option>lxc.hook.version</option>. If set to 1 then information is
              provided in the form of environment variables. If set to 0
              information is provided as arguments to the script.
            </para>

            <para>
              Standard output from the script is logged at debug level.
              Standard error is not logged, but can be captured by the
              hook redirecting its standard error to standard output.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>New pseudo tty instance (devpts)</title>
      <para>
        For stricter isolation the container can have its own private
        instance of the pseudo tty.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.pty.max</option>
          </term>
          <listitem>
            <para>
              If set, the container will have a new pseudo tty
              instance, making this private to it. The value specifies
              the maximum number of pseudo ttys allowed for a pty
              instance (this limitation is not implemented yet).
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Container system console</title>
      <para>
        If the container is configured with a root filesystem and the
        inittab file is setup to use the console, you may want to specify
        where the output of this console goes.
      </para>
      <variablelist>

        <varlistentry>
          <term>
            <option>lxc.console.buffer.size</option>
          </term>
          <listitem>
            <para>
            Setting this option instructs liblxc to allocate an in-memory
            ringbuffer. The container's console output will be written to the
            ringbuffer. Note that ringbuffer must be at least as big as a
            standard page size. When passed a value smaller than a single page
            size liblxc will allocate a ringbuffer of a single page size. A page
            size is usually 4KB.

            The keyword 'auto' will cause liblxc to allocate a ringbuffer of
            128KB.

            When manually specifying a size for the ringbuffer the value should
            be a power of 2 when converted to bytes. Valid size prefixes are
            'KB', 'MB', 'GB'. (Note that all conversions are based on multiples
            of 1024. That means 'KB' == 'KiB', 'MB' == 'MiB', 'GB' == 'GiB'.
            Additionally, the case of the suffix is ignored, i.e. 'kB', 'KB' and
            'Kb' are treated equally.)
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.console.size</option>
          </term>
          <listitem>
            <para>
            Setting this option instructs liblxc to place a limit on the size of
            the console log file specified in
            <option>lxc.console.logfile</option>. Note that size of the log file
            must be at least as big as a standard page size. When passed a value
            smaller than a single page size liblxc will set the size of log file
            to a single page size. A page size is usually 4KB.

            The keyword 'auto' will cause liblxc to place a limit of 128KB on
            the log file.

            When manually specifying a size for the log file the value should
            be a power of 2 when converted to bytes. Valid size prefixes are
            'KB', 'MB', 'GB'. (Note that all conversions are based on multiples
            of 1024. That means 'KB' == 'KiB', 'MB' == 'MiB', 'GB' == 'GiB'.
            Additionally, the case of the suffix is ignored, i.e. 'kB', 'KB' and
            'Kb' are treated equally.)

            If users want to mirror the console ringbuffer on disk they should set
            <option>lxc.console.size</option> equal to
            <option>lxc.console.buffer.size</option>.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.console.logfile</option>
          </term>
          <listitem>
            <para>
              Specify a path to a file where the console output will be written.
              Note that in contrast to the on-disk ringbuffer logfile this file
              will keep growing potentially filling up the users disks if not
              rotated and deleted. This problem can also be avoided by using the
              in-memory ringbuffer options
              <option>lxc.console.buffer.size</option> and
              <option>lxc.console.buffer.logfile</option>.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.console.rotate</option>
          </term>
          <listitem>
            <para>
              Whether to rotate the console logfile specified in
              <option>lxc.console.logfile</option>. Users can send an API
              request to rotate the logfile. Note that the old logfile will have
              the same name as the original with the suffix ".1" appended.

              Users wishing to prevent the console log file from filling the
              disk should rotate the logfile and delete it if unneeded. This
              problem can also be avoided by using the in-memory ringbuffer
              options <option>lxc.console.buffer.size</option> and
              <option>lxc.console.buffer.logfile</option>.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.console.path</option>
          </term>
          <listitem>
            <para>
              Specify a path to a device to which the console will be
              attached. The keyword 'none' will simply disable the
              console. Note, when specifying 'none' and creating a device node
              for the console in the container at /dev/console or bind-mounting
              the hosts's /dev/console into the container at /dev/console the
              container will have direct access to the hosts's /dev/console.
              This is dangerous when the container has write access to the
              device and should thus be used with caution.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Console through the ttys</title>
      <para>
        This option is useful if the container is configured with a root
        filesystem and the inittab file is setup to launch a getty on the
        ttys. The option specifies the number of ttys to be available for
        the container. The number of gettys in the inittab file of the
        container should not be greater than the number of ttys        specified
        in this option, otherwise the excess getty sessions will die and
        respawn indefinitely giving annoying messages on the console or in
        <filename>/var/log/messages</filename>.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.tty.max</option>
          </term>
          <listitem>
            <para>
              Specify the number of tty to make available to the
              container.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Console devices location</title>
      <para>
        LXC consoles are provided through Unix98 PTYs created on the
        host and bind-mounted over the expected devices in the container.
        By default, they are bind-mounted over <filename>/dev/console</filename>
        and <filename>/dev/ttyN</filename>.  This can prevent package upgrades
        in the guest.  Therefore you can specify a directory location (under
        <filename>/dev</filename> under which LXC will create the files and
        bind-mount over them.  These will then be symbolically linked to
        <filename>/dev/console</filename> and <filename>/dev/ttyN</filename>.
        A package upgrade can then succeed as it is able to remove and replace
        the symbolic links.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.tty.dir</option>
          </term>
          <listitem>
            <para>
              Specify a directory under <filename>/dev</filename>
              under which to create the container console devices. Note that LXC
              will move any bind-mounts or device nodes for /dev/console into
              this directory.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>/dev directory</title>
      <para>
        By default, lxc creates a few symbolic links (fd,stdin,stdout,stderr)
        in the container's <filename>/dev</filename> directory but does not
        automatically create device node entries. This allows the container's
        <filename>/dev</filename> to be set up as needed in the container
        rootfs.  If lxc.autodev is set to 1, then after mounting the container's
        rootfs LXC will mount a fresh tmpfs under <filename>/dev</filename>
        (limited to 500K by default, unless defined in lxc.autodev.tmpfs.size)
        and fill in a minimal set of initial devices.
        This is generally required when starting a container containing
        a "systemd" based "init" but may be optional at other times.  Additional
        devices in the containers /dev directory may be created through the
        use of the <option>lxc.hook.autodev</option> hook.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.autodev</option>
          </term>
          <listitem>
            <para>
              Set this to 0 to stop LXC from mounting and populating a minimal
              <filename>/dev</filename> when starting the container.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.autodev.tmpfs.size</option>
          </term>
          <listitem>
            <para>
              Set this to define the size of the /dev tmpfs.
              The default value is 500000 (500K). If the parameter is used
              but without value, the default value is used.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Mount points</title>
      <para>
        The mount points section specifies the different places to be
        mounted. These mount points will be private to the container
        and won't be visible by the processes running outside of the
        container. This is useful to mount /etc, /var or /home for
        examples.
      </para>
      <para>
        NOTE - LXC will generally ensure that mount targets and relative
        bind-mount sources are properly confined under the container
        root, to avoid attacks involving over-mounting host directories
        and files.  (Symbolic links in absolute mount sources are ignored)
        However, if the container configuration first mounts a directory which
        is under the control of the container user, such as /home/joe, into
        the container at some <filename>path</filename>, and then mounts
        under <filename>path</filename>, then a TOCTTOU attack would be
        possible where the container user modifies a symbolic link under
        their home directory at just the right time.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.mount.fstab</option>
          </term>
          <listitem>
            <para>
              specify a file location in
              the <filename>fstab</filename> format, containing the
              mount information.  The mount target location can and in
              most cases should be a relative path, which will become
              relative to the mounted container root.  For instance,
             </para>
             <programlisting>
             proc proc proc nodev,noexec,nosuid 0 0
             </programlisting>
             <para>
              Will mount a proc filesystem under the container's /proc,
              regardless of where the root filesystem comes from.  This
              is resilient to block device backed filesystems as well as
              container cloning.
             </para>
             <para>
              Note that when mounting a filesystem from an
              image file or block device the third field (fs_vfstype)
              cannot be auto as with
              <citerefentry>
                <refentrytitle>mount</refentrytitle>
                <manvolnum>8</manvolnum>
              </citerefentry>
              but must be explicitly specified.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.mount.entry</option>
          </term>
          <listitem>
            <para>
              Specify a mount point corresponding to a line in the
              fstab format.

              Moreover lxc supports mount propagation, such as rshared or
              rprivate, and adds three additional mount options.
              <option>optional</option> don't fail if mount does not work.
              <option>create=dir</option> or <option>create=file</option>
              to create dir (or file) when the point will be mounted.
              <option>relative</option> source path is taken to be relative to
              the mounted container root. For instance,
             </para>
             <programlisting>
             dev/null proc/kcore none bind,relative 0 0
             </programlisting>
             <para>
              Will expand dev/null to ${<option>LXC_ROOTFS_MOUNT</option>}/dev/null,
              and mount it to proc/kcore inside the container.
             </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.mount.auto</option>
          </term>
          <listitem>
            <para>
              specify which standard kernel file systems should be
              automatically mounted. This may dramatically simplify
              the configuration. The file systems are:
            </para>
            <itemizedlist>
              <listitem>
                <para>
                  <option>proc:mixed</option> (or <option>proc</option>):
                  mount <filename>/proc</filename> as read-write, but
                  remount <filename>/proc/sys</filename> and
                  <filename>/proc/sysrq-trigger</filename> read-only
                  for security / container isolation purposes.
                </para>
              </listitem>
              <listitem>
                <para>
                  <option>proc:rw</option>: mount
                  <filename>/proc</filename> as read-write
                </para>
              </listitem>
              <listitem>
                <para>
                  <option>sys:mixed</option> (or <option>sys</option>):
                  mount <filename>/sys</filename> as read-only but with
                  /sys/devices/virtual/net writable.
                </para>
              </listitem>
              <listitem>
                <para>
                  <option>sys:ro</option>:
                  mount <filename>/sys</filename> as read-only
                  for security / container isolation purposes.
                </para>
              </listitem>
              <listitem>
                <para>
                  <option>sys:rw</option>: mount
                  <filename>/sys</filename> as read-write
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup:mixed</option>:
                  Mount a tmpfs to <filename>/sys/fs/cgroup</filename>,
                  create directories for all hierarchies to which the container
                  is added, create subdirectories in those hierarchies with the
                  name of the cgroup, and bind-mount the container's own cgroup
                  into that directory. The container will be able to write to
                  its own cgroup directory, but not the parents, since they will
                  be remounted read-only.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup:mixed:force</option>:
                  The <option>force</option> option will cause LXC to perform
                  the cgroup mounts for the container under all circumstances.
                  Otherwise it is similar to <option>cgroup:mixed</option>.
                  This is mainly useful when the cgroup namespaces are enabled
                  where LXC will normally leave mounting cgroups to the init
                  binary of the container since it is perfectly safe to do so.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup:ro</option>:
                  similar to <option>cgroup:mixed</option>, but everything will
                  be mounted read-only.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup:ro:force</option>:
                  The <option>force</option> option will cause LXC to perform
                  the cgroup mounts for the container under all circumstances.
                  Otherwise it is similar to <option>cgroup:ro</option>.
                  This is mainly useful when the cgroup namespaces are enabled
                  where LXC will normally leave mounting cgroups to the init
                  binary of the container since it is perfectly safe to do so.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup:rw</option>: similar to
                  <option>cgroup:mixed</option>, but everything will be mounted
                  read-write. Note that the paths leading up to the container's
                  own cgroup will be writable, but will not be a cgroup
                  filesystem but just part of the tmpfs of
                  <filename>/sys/fs/cgroup</filename>
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup:rw:force</option>:
                  The <option>force</option> option will cause LXC to perform
                  the cgroup mounts for the container under all circumstances.
                  Otherwise it is similar to <option>cgroup:rw</option>.
                  This is mainly useful when the cgroup namespaces are enabled
                  where LXC will normally leave mounting cgroups to the init
                  binary of the container since it is perfectly safe to do so.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup</option> (without specifier):
                  defaults to <option>cgroup:rw</option> if the
                  container retains the CAP_SYS_ADMIN capability,
                  <option>cgroup:mixed</option> otherwise.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup-full:mixed</option>:
                  mount a tmpfs to <filename>/sys/fs/cgroup</filename>,
                  create directories for all hierarchies to which
                  the container is added, bind-mount the hierarchies
                  from the host to the container and make everything
                  read-only except the container's own cgroup. Note
                  that compared to <option>cgroup</option>, where
                  all paths leading up to the container's own cgroup
                  are just simple directories in the underlying
                  tmpfs, here
                  <filename>/sys/fs/cgroup/$hierarchy</filename>
                  will contain the host's full cgroup hierarchy,
                  albeit read-only outside the container's own cgroup.
                  This may leak quite a bit of information into the
                  container.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup-full:mixed:force</option>:
                  The <option>force</option> option will cause LXC to perform
                  the cgroup mounts for the container under all circumstances.
                  Otherwise it is similar to <option>cgroup-full:mixed</option>.
                  This is mainly useful when the cgroup namespaces are enabled
                  where LXC will normally leave mounting cgroups to the init
                  binary of the container since it is perfectly safe to do so.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup-full:ro</option>: similar to
                  <option>cgroup-full:mixed</option>, but everything
                  will be mounted read-only.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup-full:ro:force</option>:
                  The <option>force</option> option will cause LXC to perform
                  the cgroup mounts for the container under all circumstances.
                  Otherwise it is similar to <option>cgroup-full:ro</option>.
                  This is mainly useful when the cgroup namespaces are enabled
                  where LXC will normally leave mounting cgroups to the init
                  binary of the container since it is perfectly safe to do so.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup-full:rw</option>: similar to
                  <option>cgroup-full:mixed</option>, but everything
                  will be mounted read-write. Note that in this case,
                  the container may escape its own cgroup. (Note also
                  that if the container has CAP_SYS_ADMIN support
                  and can mount the cgroup filesystem itself, it may
                  do so anyway.)
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup-full:rw:force</option>:
                  The <option>force</option> option will cause LXC to perform
                  the cgroup mounts for the container under all circumstances.
                  Otherwise it is similar to <option>cgroup-full:rw</option>.
                  This is mainly useful when the cgroup namespaces are enabled
                  where LXC will normally leave mounting cgroups to the init
                  binary of the container since it is perfectly safe to do so.
                </para>
              </listitem>

              <listitem>
                <para>
                  <option>cgroup-full</option> (without specifier):
                  defaults to <option>cgroup-full:rw</option> if the
                  container retains the CAP_SYS_ADMIN capability,
                  <option>cgroup-full:mixed</option> otherwise.
                </para>
              </listitem>

            </itemizedlist>
            <para>
              If cgroup namespaces are enabled, then any <option>cgroup</option>
              auto-mounting request will be ignored, since the container can
              mount the filesystems itself, and automounting can confuse the
              container init.
            </para>
            <para>
              Note that if automatic mounting of the cgroup filesystem
              is enabled, the tmpfs under
              <filename>/sys/fs/cgroup</filename> will always be
              mounted read-write (but for the <option>:mixed</option>
              and <option>:ro</option> cases, the individual
              hierarchies,
              <filename>/sys/fs/cgroup/$hierarchy</filename>, will be
              read-only). This is in order to work around a quirk in
              Ubuntu's
              <citerefentry>
                <refentrytitle>mountall</refentrytitle>
                <manvolnum>8</manvolnum>
              </citerefentry>
              command that will cause containers to wait for user
              input at boot if
              <filename>/sys/fs/cgroup</filename> is mounted read-only
              and the container can't remount it read-write due to a
              lack of CAP_SYS_ADMIN.
            </para>
            <para>
              Examples:
            </para>
            <programlisting>
              lxc.mount.auto = proc sys cgroup
              lxc.mount.auto = proc:rw sys:rw cgroup-full:rw
            </programlisting>
          </listitem>
        </varlistentry>

      </variablelist>
    </refsect2>

    <refsect2>
      <title>Root file system</title>
      <para>
        The root file system of the container can be different than that
        of the host system.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.rootfs.path</option>
          </term>
          <listitem>
            <para>
              specify the root file system for the container. It can
              be an image file, a directory or a block device. If not
              specified, the container shares its root file system
              with the host.
            </para>
            <para>
          For directory or simple block-device backed containers,
          a pathname can be used.  If the rootfs is backed by a nbd
          device, then <filename>nbd:file:1</filename> specifies that
          <filename>file</filename> should be attached to a nbd device,
          and partition 1 should be mounted as the rootfs.
          <filename>nbd:file</filename> specifies that the nbd device
          itself should be mounted.  <filename>overlayfs:/lower:/upper</filename>
          specifies that the rootfs should be an overlay with <filename>/upper</filename>
          being mounted read-write over a read-only mount of <filename>/lower</filename>.
          For <filename>overlay</filename> multiple <filename>/lower</filename>
          directories can be specified. <filename>loop:/file</filename> tells lxc to attach
          <filename>/file</filename> to a loop device and mount the loop device.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.rootfs.mount</option>
          </term>
          <listitem>
            <para>
              where to recursively bind <option>lxc.rootfs.path</option>
              before pivoting.  This is to ensure success of the
              <citerefentry>
                <refentrytitle><command>pivot_root</command></refentrytitle>
                <manvolnum>8</manvolnum>
              </citerefentry>
              syscall.  Any directory suffices, the default should
              generally work.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.rootfs.options</option>
          </term>
          <listitem>
            <para>
              Specify extra mount options to use when mounting the rootfs.
                The format of the mount options corresponds to the
                format used in fstab. In addition, LXC supports the custom
                <option>idmap=</option> mount option. This option can be used
                to tell LXC to create an idmapped mount for the container's
                rootfs. This is useful when the user doesn't want to recursively
                chown the rootfs of the container to match the idmapping of the
                user namespace the container is going to use. Instead an
                idmapped mount can be used to handle this.
                The argument for
                <option>idmap=</option>
                can either be a path pointing to a user namespace file that
                LXC will open and use to idmap the rootfs or the special value
                "container" which will instruct LXC to use
                the container's user namespace to idmap the rootfs.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.rootfs.managed</option>
          </term>
          <listitem>
            <para>
              Set this to 0 to indicate that LXC is not managing the
              container storage, then LXC will not modify the
              container storage. The default is 1.
            </para>
          </listitem>
        </varlistentry>

      </variablelist>
    </refsect2>

    <refsect2>
      <title>Control groups ("cgroups")</title>
      <para>
        The control group section contains the configuration for the
        different subsystem. <command>lxc</command> does not check the
        correctness of the subsystem name. This has the disadvantage
        of not detecting configuration errors until the container is
        started, but has the advantage of permitting any future
        subsystem.
      </para>

      <para>
        The kernel implementation of cgroups has changed significantly over the
        years. With Linux 4.5 support for a new cgroup filesystem was added
        usually referred to as "cgroup2" or "unified hierarchy". Since then the
        old cgroup filesystem is usually referred to as "cgroup1" or the
        "legacy hierarchies". Please see the cgroups manual page for a detailed
        explanation of the differences between the two versions.
      </para>

      <para>
        LXC distinguishes settings for the legacy and the unified hierarchy by
        using different configuration key prefixes. To alter settings for
        controllers in a legacy hierarchy the key prefix
        <option>lxc.cgroup.</option> must be used and in order to alter the
        settings for a controller in the unified hierarchy the
        <option>lxc.cgroup2.</option> key must be used. Note that LXC will
        ignore <option>lxc.cgroup.</option> settings on systems that only use
        the unified hierarchy. Conversely, it will ignore
        <option>lxc.cgroup2.</option> options on systems that only use legacy
        hierachies.
      </para>

      <para>
        At its core a cgroup hierarchy is a way to hierarchically organize
        processes. Usually a cgroup hierarchy will have one or more
        "controllers" enabled. A "controller" in a cgroup hierarchy is usually
        responsible for distributing a specific type of system resource along
        the hierarchy. Controllers include the "pids" controller, the "cpu"
        controller, the "memory" controller and others. Some controllers
        however do not fall into the category of distributing a system
        resource, instead they are often referred to as "utility" controllers.
        One utility controller is the device controller. Instead of
        distributing a system resource it allows to manage device access.
      </para>

      <para>
        In the legacy hierarchy the device controller was implemented like most
        other controllers as a set of files that could be written to. These
        files where named "devices.allow" and "devices.deny". The legacy device
        controller allowed the implementation of both "allowlists" and
        "denylists".
      </para>

      <para>
        An allowlist is a device program that by default blocks access to all
        devices. In order to access specific devices "allow rules" for
        particular devices or device classes must be specified. In contrast, a
        denylist is a device program that by default allows access to all
        devices. In order to restrict access to specific devices "deny rules"
        for particular devices or device classes must be specified.
      </para>

      <para>
        In the unified cgroup hierarchy the implementation of the device
        controller has completely changed. Instead of files to read from and
        write to a eBPF program of
        <option>BPF_PROG_TYPE_CGROUP_DEVICE</option> can be attached to a
        cgroup. Even though the kernel implementation has changed completely
        LXC tries to allow for the same semantics to be followed in the legacy
        device cgroup and the unified eBPF-based device controller. The
        following paragraphs explain the semantics for the unified eBPF-based
        device controller.
      </para>

      <para>
        As mentioned the format for specifying device rules for the unified
        eBPF-based device controller is the same as for the legacy cgroup
        device controller; only the configuration key prefix has changed.
        Specifically, device rules for the legacy cgroup device controller are
        specified via <option>lxc.cgroup.devices.allow</option> and
        <option>lxc.cgroup.devices.deny</option> whereas for the
        cgroup2 eBPF-based device controller
        <option>lxc.cgroup.devices.allow</option> and
        <option>lxc.cgroup.devices.deny</option> must be used.
      </para>
      <para>
        <itemizedlist>
          <listitem>
            <para>
              A allowlist device rule
              <programlisting>
                lxc.cgroup2.devices.deny = a
              </programlisting>
              will cause LXC to instruct the kernel to block access to all
              devices by default. To grant access to devices allow device rules
              must be added via the <option>lxc.cgroup2.devices.allow</option>
              key. This is referred to as a "allowlist" device program.
            </para>
          </listitem>

          <listitem>
            <para>
              A denylist device rule
              <programlisting>
                lxc.cgroup2.devices.allow = a
              </programlisting>
              will cause LXC to instruct the kernel to allow access to all
              devices by default. To deny access to devices deny device rules
              must be added via <option>lxc.cgroup2.devices.deny</option> key.
              This is referred to as a "denylist" device program.
            </para>
          </listitem>

          <listitem>
            <para>
              Specifying any of the aformentioned two rules will cause all
              previous rules to be cleared, i.e. the device list will be reset.
            </para>
          </listitem>

          <listitem>
            <para>
            When an allowlist program is requested, i.e. access to all devices
            is blocked by default, specific deny rules for individual devices
            or device classes are ignored.
            </para>
          </listitem>

          <listitem>
            <para>
            When a denylist program is requested, i.e. access to all devices
            is allowed by default, specific allow rules for individual devices
            or device classes are ignored.
            </para>
          </listitem>
        </itemizedlist>
      </para>

      <para>
        For example the set of rules:
        <programlisting>
          lxc.cgroup2.devices.deny = a
          lxc.cgroup2.devices.allow = c *:* m
          lxc.cgroup2.devices.allow = b *:* m
          lxc.cgroup2.devices.allow = c 1:3 rwm
        </programlisting>
        implements an allowlist device program, i.e. the kernel will block
        access to all devices not specifically allowed in this list. This
        particular program states that all character and block devices may be
        created but only /dev/null might be read or written.
      </para>

      <para>
        If we instead switch to the following set of rules:
        <programlisting>
          lxc.cgroup2.devices.allow = a
          lxc.cgroup2.devices.deny = c *:* m
          lxc.cgroup2.devices.deny = b *:* m
          lxc.cgroup2.devices.deny = c 1:3 rwm
        </programlisting>
         then LXC would instruct the kernel to implement a denylist, i.e. the
         kernel will allow access to all devices not specifically denied in
         this list. This particular program states that no character devices or
         block devices might be created and that /dev/null is not allow allowed
         to be read, written, or created.
      </para>

      <para>
         Now consider the same program but followed by a "global rule"
         which determines the type of device program (allowlist or
         denylist) as explained above:
        <programlisting>
          lxc.cgroup2.devices.allow = a
          lxc.cgroup2.devices.deny = c *:* m
          lxc.cgroup2.devices.deny = b *:* m
          lxc.cgroup2.devices.deny = c 1:3 rwm
          lxc.cgroup2.devices.allow = a
        </programlisting>
        The last line will cause LXC to reset the device list without changing
        the type of device program.
      </para>

      <para>
        If we specify:
        <programlisting>
          lxc.cgroup2.devices.allow = a
          lxc.cgroup2.devices.deny = c *:* m
          lxc.cgroup2.devices.deny = b *:* m
          lxc.cgroup2.devices.deny = c 1:3 rwm
          lxc.cgroup2.devices.deny = a
        </programlisting>
        instead then the last line will cause LXC to reset the device list and
        switch from a allowlist program to a denylist program.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.cgroup.[controller name].[controller file]</option>
          </term>
          <listitem>
            <para>
              Specify the control group value to be set on a legacy cgroup
              hierarchy. The controller name is the literal name of the control
              group. The permitted names and the syntax of their values is not
              dictated by LXC, instead it depends on the features of the Linux
              kernel running at the time the container is started, eg.
              <option>lxc.cgroup.cpuset.cpus</option>
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cgroup2.[controller name].[controller file]</option>
          </term>
          <listitem>
            <para>
              Specify the control group value to be set on the unified cgroup
              hierarchy. The controller name is the literal name of the control
              group. The permitted names and the syntax of their values is not
              dictated by LXC, instead it depends on the features of the Linux
              kernel running at the time the container is started, eg.
              <option>lxc.cgroup2.memory.high</option>
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cgroup.dir</option>
          </term>
          <listitem>
            <para>
              specify a directory or path in which the container's cgroup will
              be created. For example, setting
              <option>lxc.cgroup.dir = my-cgroup/first</option> for a container
              named "c1" will create the container's cgroup as a sub-cgroup of
              "my-cgroup". For example, if the user's current cgroup "my-user"
              is located in the root cgroup of the cpuset controller in a
              cgroup v1 hierarchy this would create the cgroup
              "/sys/fs/cgroup/cpuset/my-user/my-cgroup/first/c1" for the
              container. Any missing cgroups will be created by LXC. This
              presupposes that the user has write access to its current cgroup.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cgroup.dir.container</option>
          </term>
          <listitem>
            <para>
              This is similar to <option>lxc.cgroup.dir</option>, but must be
              used together with <option>lxc.cgroup.dir.monitor</option> and
              affects only the container's cgroup path. This option is mutually
              exclusive with <option>lxc.cgroup.dir</option>.
              Note that the final path the container attaches to may be
              extended further by the
              <option>lxc.cgroup.dir.container.inner</option> option.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cgroup.dir.monitor</option>
          </term>
          <listitem>
            <para>
              This is the monitor process counterpart to
              <option>lxc.cgroup.dir.container</option>.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cgroup.dir.monitor.pivot</option>
          </term>
          <listitem>
            <para>
              On container termination the PID of the monitor process is attached to this cgroup.
              This path should not be a subpath of any other configured cgroup dir to ensure
              proper removal of other cgroup paths on container termination.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cgroup.dir.container.inner</option>
          </term>
          <listitem>
            <para>
              Specify an additional subdirectory where the cgroup namespace
              will be created. With this option, the cgroup limits will be
              applied to the outer path specified in
              <option>lxc.cgroup.dir.container</option>, which is not accessible
              from within the container, making it possible to better enforce
              limits for privileged containers in a way they cannot override
              them.
              This only works in conjunction with the
              <option>lxc.cgroup.dir.container</option> and
              <option>lxc.cgroup.dir.monitor</option> options and has otherwise
              no effect.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cgroup.relative</option>
          </term>
          <listitem>
            <para>
              Set this to 1 to instruct LXC to never escape to the
              root cgroup. This makes it easy for users to adhere to
              restrictions enforced by cgroup2 and
              systemd. Specifically, this makes it possible to run LXC
              containers as systemd services.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Capabilities</title>
      <para>
        The capabilities can be dropped in the container if this one
        is run as root.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.cap.drop</option>
          </term>
          <listitem>
            <para>
              Specify the capability to be dropped in the container. A
              single line defining several capabilities with a space
              separation is allowed. The format is the lower case of
              the capability definition without the "CAP_" prefix,
              eg. CAP_SYS_MODULE should be specified as
              sys_module. See
              <citerefentry>
                <refentrytitle><command>capabilities</command></refentrytitle>
                <manvolnum>7</manvolnum>
              </citerefentry>.
              If used with no value, lxc will clear any drop capabilities
              specified up to this point.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.cap.keep</option>
          </term>
          <listitem>
            <para>
              Specify the capability to be kept in the container. All other
              capabilities will be dropped. When a special value of "none" is
              encountered, lxc will clear any keep capabilities specified up
              to this point. A value of "none" alone can be used to drop all
              capabilities.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Namespaces</title>
      <para>
        A namespace can be cloned (<option>lxc.namespace.clone</option>),
        kept (<option>lxc.namespace.keep</option>) or shared
        (<option>lxc.namespace.share.[namespace identifier]</option>).
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.namespace.clone</option>
          </term>
          <listitem>
            <para>
            Specify namespaces which the container is supposed to be created
            with. The namespaces to create are specified as a space separated
            list. Each namespace must correspond to one of the standard
            namespace identifiers as seen in the
            <filename>/proc/PID/ns</filename> directory.
            When <option>lxc.namespace.clone</option> is not explicitly set all
            namespaces supported by the kernel and the current configuration
            will be used.
            </para>

            <para>
            To create a new mount, net and ipc namespace set
            <option>lxc.namespace.clone=mount net ipc</option>.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.namespace.keep</option>
          </term>
          <listitem>
            <para>
            Specify namespaces which the container is supposed to inherit from
            the process that created it. The namespaces to keep are specified as
            a space separated list. Each namespace must correspond to one of the
            standard namespace identifiers as seen in the
            <filename>/proc/PID/ns</filename> directory.
            The <option>lxc.namespace.keep</option> is a
            denylist option, i.e. it is useful when enforcing that containers
            must keep a specific set of namespaces.
            </para>

            <para>
            To keep the network, user and ipc namespace set
            <option>lxc.namespace.keep=user net ipc</option>.
            </para>

            <para>
            Note that sharing pid namespaces will likely not work with most init
            systems.
            </para>

            <para>
            Note that if the container requests a new user namespace and the
            container wants to inherit the network namespace it needs to inherit
            the user namespace as well.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.namespace.share.[namespace identifier]</option>
          </term>
          <listitem>
            <para>
            Specify a namespace to inherit from another container or process.
            The <option>[namespace identifier]</option> suffix needs to be
            replaced with one of the namespaces that appear in the
            <filename>/proc/PID/ns</filename> directory.
            </para>

            <para>
            To inherit the namespace from another process set the
            <option>lxc.namespace.share.[namespace identifier]</option> to the PID of
            the process, e.g. <option>lxc.namespace.share.net=42</option>.
            </para>

            <para>
            To inherit the namespace from another container set the
            <option>lxc.namespace.share.[namespace identifier]</option> to the name of
            the container, e.g. <option>lxc.namespace.share.pid=c3</option>.
            </para>

            <para>
            To inherit the namespace from another container located in a
            different path than the standard liblxc path set the
            <option>lxc.namespace.share.[namespace identifier]</option> to the full
            path to the container, e.g.
            <option>lxc.namespace.share.user=/opt/c3</option>.
            </para>

            <para>
            In order to inherit namespaces the caller needs to have sufficient
            privilege over the process or container.
            </para>

            <para>
            Note that sharing pid namespaces between system containers will
            likely not work with most init systems.
            </para>

            <para>
            Note that if two processes are in different user namespaces and one
            process wants to inherit the other's network namespace it usually
            needs to inherit the user namespace as well.
            </para>

            <para>
            Note that without careful additional configuration of an LSM,
            sharing user+pid namespaces with a task may allow that task to
            escalate privileges to that of the task calling liblxc.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.time.offset.boot</option>
          </term>
          <listitem>
            <para>
            Specify a positive or negative offset for the boottime clock. The
            format accepts hours (h), minutes (m), seconds (s),
            milliseconds (ms), microseconds (us), and nanoseconds (ns).
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.time.offset.monotonic</option>
          </term>
          <listitem>
            <para>
            Specify a positive or negative offset for the monotonic clock. The
            format accepts hours (h), minutes (m), seconds (s),
            milliseconds (ms), microseconds (us), and nanoseconds (ns).
            </para>
          </listitem>
        </varlistentry>

      </variablelist>
    </refsect2>

    <refsect2>
      <title>Resource limits</title>
      <para>
        The soft and hard resource limits for the container can be changed.
        Unprivileged containers can only lower them. Resources which are not
        explicitly specified will be inherited.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.prlimit.[limit name]</option>
          </term>
          <listitem>
            <para>
              Specify the resource limit to be set. A limit is specified as two
              colon separated values which are either numeric or the word
              'unlimited'. A single value can be used as a shortcut to set both
              soft and hard limit to the same value. The permitted names the
              "RLIMIT_" resource names in lowercase without the "RLIMIT_"
              prefix, eg. RLIMIT_NOFILE should be specified as "nofile". See
              <citerefentry>
                <refentrytitle><command>setrlimit</command></refentrytitle>
                <manvolnum>2</manvolnum>
              </citerefentry>.
              If used with no value, lxc will clear the resource limit
              specified up to this point. A resource with no explicitly
              configured limitation will be inherited from the process starting
              up the container.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Sysctl</title>
      <para>
        Configure kernel parameters for the container.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.sysctl.[kernel parameters name]</option>
          </term>
          <listitem>
            <para>
              Specify the kernel parameters to be set. The parameters available
              are those listed under /proc/sys/.
              Note that not all sysctls are namespaced. Changing Non-namespaced
              sysctls will cause the system-wide setting to be modified.
              <citerefentry>
                <refentrytitle><command>sysctl</command></refentrytitle>
                <manvolnum>8</manvolnum>
              </citerefentry>.
              If used with no value, lxc will clear the parameters specified up
              to this point.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Apparmor profile</title>
      <para>
        If lxc was compiled and installed with apparmor support, and the host
        system has apparmor enabled, then the apparmor profile under which the
        container should be run can be specified in the container
        configuration.  The default is <command>lxc-container-default-cgns</command>
        if the host kernel is cgroup namespace aware, or
        <command>lxc-container-default</command> otherwise.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.apparmor.profile</option>
          </term>
          <listitem>
            <para>
              Specify the apparmor profile under which the container should
              be run.  To specify that the container should be unconfined,
              use
            </para>
              <programlisting>lxc.apparmor.profile = unconfined</programlisting>
            <para>
              If the apparmor profile should remain unchanged (i.e. if you
              are nesting containers and are already confined), then use
            </para>
              <programlisting>lxc.apparmor.profile = unchanged</programlisting>
            <para>
              If you instruct LXC to generate the apparmor profile,
              then use
            </para>
              <programlisting>lxc.apparmor.profile = generated</programlisting>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.apparmor.allow_incomplete</option>
          </term>
          <listitem>
            <para>
              Apparmor profiles are pathname based.  Therefore many file
              restrictions require mount restrictions to be effective against
              a determined attacker.  However, these mount restrictions are not
              yet implemented in the upstream kernel.  Without the mount
              restrictions, the apparmor profiles still protect against accidental
              damager.
            </para>
            <para>
              If this flag is 0 (default), then the container will not be
              started if the kernel lacks the apparmor mount features, so that a
              regression after a kernel upgrade will be detected.  To start the
              container under partial apparmor protection, set this flag to 1.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.apparmor.allow_nesting</option>
          </term>
          <listitem>
            <para>
              If set this to 1, causes the following changes. When
              generated apparmor profiles are used, they will contain
              the necessary changes to allow creating a nested
              container. In addition to the usual mount points,
              <filename>/dev/.lxc/proc</filename>
              and <filename>/dev/.lxc/sys</filename> will contain
              procfs and sysfs mount points without the lxcfs
              overlays, which, if generated apparmor profiles are
              being used, will not be read/writable directly.
            </para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term>
            <option>lxc.apparmor.raw</option>
          </term>
          <listitem>
            <para>
              A list of raw AppArmor profile lines to append to the
              profile. Only valid when using generated profiles.
            </para>
          </listitem>
        </varlistentry>

      </variablelist>
    </refsect2>

    <refsect2>
      <title>SELinux context</title>
      <para>
        If lxc was compiled and installed with SELinux support, and the host
        system has SELinux enabled, then the SELinux context under which the
        container should be run can be specified in the container
        configuration.  The default is <command>unconfined_t</command>,
        which means that lxc will not attempt to change contexts.
        See @DATADIR@/lxc/selinux/lxc.te for an example policy and more
        information.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.selinux.context</option>
          </term>
          <listitem>
            <para>
              Specify the SELinux context under which the container should
              be run or <command>unconfined_t</command>. For example
            </para>
            <programlisting>lxc.selinux.context = system_u:system_r:lxc_t:s0:c22</programlisting>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.selinux.context.keyring</option>
          </term>
          <listitem>
            <para>
              Specify the SELinux context under which the container's keyring
              should be created. By default this the same as lxc.selinux.context, or
              the context lxc is executed under if lxc.selinux.context has not been set.
            </para>
            <programlisting>lxc.selinux.context.keyring = system_u:system_r:lxc_t:s0:c22</programlisting>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Kernel Keyring</title>
      <para>
        The Linux Keyring facility is primarily a way for various
        kernel components to retain or cache security data, authentication
        keys, encryption keys, and other data in the kernel. By default lxc
        will create a new session keyring for the started application.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.keyring.session</option>
          </term>
          <listitem>
            <para>
              Disable the creation of new session keyring by lxc. The started
              application will then inherit the current session keyring.
              By default, or when passing the value 1, a new keyring will be created.
            </para>
            <programlisting>lxc.keyring.session = 0</programlisting>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Seccomp configuration</title>
      <para>
        A container can be started with a reduced set of available
        system calls by loading a seccomp profile at startup.  The
        seccomp configuration file must begin with a version number
        on the first line, a policy type on the second line, followed
        by the configuration.
      </para>
      <para>
        Versions 1 and 2 are currently supported.  In version 1, the
        policy is a simple allowlist.  The second line therefore must
        read "allowlist", with the rest of the file containing one (numeric)
        syscall number per line.  Each syscall number is allowlisted,
        while every unlisted number is denylisted for use in the container
      </para>

      <para>
       In version 2, the policy may be denylist or allowlist,
       supports per-rule and per-policy default actions, and supports
       per-architecture system call resolution from textual names.
      </para>
      <para>
       An example denylist policy, in which all system calls are
       allowed except for mknod, which will simply do nothing and
       return 0 (success), looks like:
      </para>

      <programlisting>
      2
      denylist
      mknod errno 0
      ioctl notify
      </programlisting>

      <para>
      Specifying "errno" as action will cause LXC to register a seccomp filter
      that will cause a specific errno to be returned to the caller. The errno
      value can be specified after the "errno" action word.
      </para>

      <para>
      Specifying "notify" as action will cause LXC to register a seccomp
      listener and retrieve a listener file descriptor from the kernel. When a
      syscall is made that is registered as "notify" the kernel will generate a
      poll event and send a message over the file descriptor. The caller can
      read this message, inspect the syscalls including its arguments. Based on
      this information the caller is expected to send back a message informing
      the kernel which action to take. Until that message is sent the kernel
      will block the calling process. The format of the messages to read and
      sent is documented in seccomp itself.
      </para>

      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.seccomp.profile</option>
          </term>
          <listitem>
            <para>
              Specify a file containing the seccomp configuration to
              load before the container starts.
             </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.seccomp.allow_nesting</option>
          </term>
          <listitem>
            <para>
              If this flag is set to 1, then seccomp filters will be stacked
              regardless of whether a seccomp profile is already loaded.
              This allows nested containers to load their own seccomp profile.
              The default setting is 0.
             </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.seccomp.notify.proxy</option>
          </term>
          <listitem>
            <para>
              Specify a unix socket to which LXC will connect and forward
              seccomp events to. The path must be in the form
              unix:/path/to/socket or unix:@socket. The former specifies a
              path-bound unix domain socket while the latter specifies an
              abstract unix domain socket.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.seccomp.notify.cookie</option>
          </term>
          <listitem>
            <para>
              An additional string sent along with proxied seccomp notification
              requests.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>PR_SET_NO_NEW_PRIVS</title>
      <para>
              With PR_SET_NO_NEW_PRIVS active execve() promises not to grant
              privileges to do anything that could not have been done without
              the execve() call (for example, rendering the set-user-ID and
              set-group-ID mode bits, and file capabilities non-functional).
              Once set, this bit cannot be unset. The setting of this bit is
              inherited by children created by fork() and clone(), and preserved
              across execve().
              Note that PR_SET_NO_NEW_PRIVS is applied after the container has
              changed into its intended AppArmor profile or SElinux context.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.no_new_privs</option>
          </term>
          <listitem>
            <para>
              Specify whether the PR_SET_NO_NEW_PRIVS flag should be set for the
              container. Set to 1 to activate.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>UID mappings</title>
      <para>
        A container can be started in a private user namespace with
        user and group id mappings.  For instance, you can map userid
        0 in the container to userid 200000 on the host.  The root
        user in the container will be privileged in the container,
        but unprivileged on the host.  Normally a system container
        will want a range of ids, so you would map, for instance,
        user and group ids 0 through 20,000 in the container to the
        ids 200,000 through 220,000.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.idmap</option>
          </term>
          <listitem>
            <para>
              Four values must be provided.  First a character, either
              'u', or 'g', to specify whether user or group ids are
              being mapped.  Next is the first userid as seen in the
              user namespace of the container.  Next is the userid as
              seen on the host.  Finally, a range indicating the number
              of consecutive ids to map.
             </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Container hooks</title>
      <para>
        Container hooks are programs or scripts which can be executed
        at various times in a container's lifetime.
      </para>
      <para>
        When a container hook is executed, additional information is passed
        along. The <option>lxc.hook.version</option> argument can be used to
        determine if the following arguments are passed as command line
        arguments or through environment variables. The arguments are:
        <itemizedlist>
          <listitem><para> Container name. </para></listitem>
          <listitem><para> Section (always 'lxc'). </para></listitem>
          <listitem><para> The hook type (i.e. 'clone' or 'pre-mount'). </para></listitem>
          <listitem><para> Additional arguments. In the
          case of the clone hook, any extra arguments passed will appear as
          further arguments to the hook.  In the case of the stop hook, paths to
          filedescriptors for each of the container's namespaces along with
          their types are passed. </para></listitem>
        </itemizedlist>
        The following environment variables are set:
        <itemizedlist>
          <listitem><para> LXC_CGNS_AWARE: indicator whether the container is
          cgroup namespace aware. </para></listitem>
          <listitem><para> LXC_CONFIG_FILE: the path to the container
          configuration file. </para></listitem>
          <listitem><para> LXC_HOOK_TYPE: the hook type (e.g. 'clone', 'mount',
          'pre-mount'). Note that the existence of this environment variable is
          conditional on the value of <option>lxc.hook.version</option>. If it
          is set to 1 then LXC_HOOK_TYPE will be set.
          </para></listitem>
          <listitem><para> LXC_HOOK_SECTION: the section type (e.g. 'lxc',
          'net'). Note that the existence of this environment variable is
          conditional on the value of <option>lxc.hook.version</option>. If it
          is set to 1 then LXC_HOOK_SECTION will be set.
          </para></listitem>
          <listitem><para> LXC_HOOK_VERSION: the version of the hooks. This
          value is identical to the value of the container's
          <option>lxc.hook.version</option> config item. If it is set to 0 then
          old-style hooks are used. If it is set to 1 then new-style hooks are
          used. </para></listitem>
          <listitem><para> LXC_LOG_LEVEL: the container's log level. </para></listitem>
          <listitem><para> LXC_NAME: is the container's name. </para></listitem>
          <listitem><para> LXC_[NAMESPACE IDENTIFIER]_NS: path under
          /proc/PID/fd/ to a file descriptor referring to the container's
          namespace. For each preserved namespace type there will be a separate
          environment variable. These environment variables will only be set if
          <option>lxc.hook.version</option> is set to 1. </para></listitem>
          <listitem><para> LXC_ROOTFS_MOUNT: the path to the mounted root filesystem. </para></listitem>
          <listitem><para> LXC_ROOTFS_PATH: this is the lxc.rootfs.path entry
          for the container. Note this is likely not where the mounted rootfs is
          to be found, use LXC_ROOTFS_MOUNT for that. </para></listitem>
          <listitem><para> LXC_SRC_NAME: in the case of the clone hook, this is
          the original container's name. </para></listitem>
        </itemizedlist>
      </para>
      <para>
        Standard output from the hooks is logged at debug level.
        Standard error is not logged, but can be captured by the
        hook redirecting its standard error to standard output.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.version</option>
          </term>
          <listitem>
            <para>
            To pass the arguments in new style via environment variables set to
            1 otherwise set to 0 to pass them as arguments.
            This setting affects all hooks arguments that were traditionally
            passed as arguments to the script. Specifically, it affects the
            container name, section (e.g. 'lxc', 'net') and hook type (e.g.
            'clone', 'mount', 'pre-mount') arguments. If new-style hooks are
            used then the arguments will be available as environment variables.
            The container name will be set in LXC_NAME. (This is set
            independently of the value used for this config item.) The section
            will be set in LXC_HOOK_SECTION and the hook type will be set in
            LXC_HOOK_TYPE.
            It also affects how the paths to file descriptors referring to the
            container's namespaces are passed. If set to 1 then for each
            namespace a separate environment variable LXC_[NAMESPACE
            IDENTIFIER]_NS will be set. If set to 0 then the paths will be
            passed as arguments to the stop hook.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.pre-start</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the host's namespace before the
              container ttys, consoles, or mounts are up.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.pre-mount</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the container's fs namespace but before
              the rootfs has been set up.  This allows for manipulation
              of the rootfs, i.e. to mount an encrypted filesystem.  Mounts
              done in this hook will not be reflected on the host (apart from
              mounts propagation), so they will be automatically cleaned up
              when the container shuts down.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.mount</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the container's namespace after
              mounting has been done, but before the pivot_root.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.autodev</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the container's namespace after
              mounting has been done and after any mount hooks have
              run, but before the pivot_root, if
              <option>lxc.autodev</option> == 1.
              The purpose of this hook is to assist in populating the
              /dev directory of the container when using the autodev
              option for systemd based containers.  The container's /dev
              directory is relative to the
              ${<option>LXC_ROOTFS_MOUNT</option>} environment
              variable available when the hook is run.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.start-host</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the host's namespace after the
              container has been setup, and immediately before starting
              the container init.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.start</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the container's namespace immediately
              before executing the container's init.  This requires the
              program to be available in the container.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.stop</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the host's namespace with references
              to the container's namespaces after the container has been shut
              down. For each namespace an extra argument is passed to the hook
              containing the namespace's type and a filename that can be used to
              obtain a file descriptor to the corresponding namespace, separated
              by a colon. The type is the name as it would appear in the
              <filename>/proc/PID/ns</filename> directory.
              For instance for the mount namespace the argument usually looks
              like <filename>mnt:/proc/PID/fd/12</filename>.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.post-stop</option>
          </term>
          <listitem>
            <para>
              A hook to be run in the host's namespace after the
              container has been shut down.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.clone</option>
          </term>
          <listitem>
            <para>
              A hook to be run when the container is cloned to a new one.
              See <citerefentry><refentrytitle><command>lxc-clone</command></refentrytitle>
              <manvolnum>1</manvolnum></citerefentry> for more information.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.hook.destroy</option>
          </term>
          <listitem>
            <para>
              A hook to be run when the container is destroyed.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
      <title>Container hooks Environment Variables</title>
      <para>
        A number of environment variables are made available to the startup
        hooks to provide configuration information and assist in the
        functioning of the hooks.  Not all variables are valid in all
        contexts.  In particular, all paths are relative to the host system
        and, as such, not valid during the <option>lxc.hook.start</option> hook.
      </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_NAME</option>
          </term>
          <listitem>
            <para>
              The LXC name of the container.  Useful for logging messages
              in common log environments.  [<option>-n</option>]
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_CONFIG_FILE</option>
          </term>
          <listitem>
            <para>
              Host relative path to the container configuration file.  This
              gives the container to reference the original, top level,
              configuration file for the container in order to locate any
              additional configuration information not otherwise made
              available.  [<option>-f</option>]
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_CONSOLE</option>
          </term>
          <listitem>
            <para>
              The path to the console output of the container if not NULL.
              [<option>-c</option>] [<option>lxc.console.path</option>]
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_CONSOLE_LOGPATH</option>
          </term>
          <listitem>
            <para>
              The path to the console log output of the container if not NULL.
              [<option>-L</option>]
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_ROOTFS_MOUNT</option>
          </term>
          <listitem>
            <para>
              The mount location to which the container is initially bound.
              This will be the host relative path to the container rootfs
              for the container instance being started and is where changes
              should be made for that instance.
              [<option>lxc.rootfs.mount</option>]
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_ROOTFS_PATH</option>
          </term>
          <listitem>
            <para>
              The host relative path to the container root which has been
              mounted to the rootfs.mount location.
              [<option>lxc.rootfs.path</option>]
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_SRC_NAME</option>
          </term>
          <listitem>
            <para>
              Only for the clone hook. Is set to the original container name.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_TARGET</option>
          </term>
          <listitem>
            <para>
              Only for the stop hook. Is set to "stop" for a container
              shutdown or "reboot" for a container reboot.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
      <variablelist>
        <varlistentry>
          <term>
            <option>LXC_CGNS_AWARE</option>
          </term>
          <listitem>
            <para>
              If unset, then this version of lxc is not aware of cgroup
              namespaces.  If set, it will be set to 1, and lxc is aware
              of cgroup namespaces.  Note this does not guarantee that
              cgroup namespaces are enabled in the kernel.  This is used
              by the lxcfs mount hook.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>
    <refsect2>
    <title>Logging</title>
    <para>
      Logging can be configured on a per-container basis.  By default,
      depending upon how the lxc package was compiled, container startup
      is logged only at the ERROR level, and logged to a file named after
      the container (with '.log' appended) either under the container path,
      or under @LOGPATH@.
    </para>
    <para>
      Both the default log level and the log file can be specified in the
      container configuration file, overriding the default behavior.  Note
      that the configuration file entries can in turn be overridden by the
      command line options to <command>lxc-start</command>.
    </para>
      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.log.level</option>
          </term>
          <listitem>
            <para>
            The level at which to log.  The log level is an integer in
            the range of 0..8 inclusive, where a lower number means more
            verbose debugging.  In particular 0 = trace, 1 = debug, 2 =
            info, 3 = notice, 4 = warn, 5 = error, 6 = critical, 7 =
            alert, and 8 = fatal.  If unspecified, the level defaults
            to 5 (error), so that only errors and above are logged.
            </para>
            <para>
            Note that when a script (such as either a hook script or a
            network interface up or down script) is called, the script's
            standard output is logged at level 1, debug.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.log.file</option>
          </term>
          <listitem>
            <para>
            The file to which logging info should be written.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.log.syslog</option>
          </term>
          <listitem>
            <para>
            Send logging info to syslog. It respects the log level defined in
            <command>lxc.log.level</command>. The argument should be the syslog
            facility to use, valid ones are: daemon, local0, local1, local2,
            local3, local4, local5, local5, local6, local7.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
    <title>Autostart</title>
    <para>
        The autostart options support marking which containers should be
        auto-started and in what order. These options may be used by LXC tools
        directly or by external tooling provided by the distributions.
    </para>

    <variablelist>
        <varlistentry>
          <term>
            <option>lxc.start.auto</option>
          </term>
          <listitem>
            <para>
              Whether the container should be auto-started.
              Valid values are 0 (off) and 1 (on).
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.start.delay</option>
          </term>
          <listitem>
            <para>
              How long to wait (in seconds) after the container is
              started before starting the next one.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.start.order</option>
          </term>
          <listitem>
            <para>
              An integer used to sort the containers when auto-starting
              a series of containers at once. A lower value means an
              earlier start.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.monitor.unshare</option>
          </term>
          <listitem>
            <para>
              If not zero the mount namespace will be unshared from the host
              before initializing the container (before running any pre-start
              hooks). This requires the CAP_SYS_ADMIN capability at startup.
              Default is 0.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.monitor.signal.pdeath</option>
          </term>
          <listitem>
            <para>
              Set the signal to be sent to the container's init when the lxc
              monitor exits. By default it is set to SIGKILL which will cause
              all container processes to be killed when the lxc monitor process
              dies.
              To ensure that containers stay alive even if lxc monitor dies set
              this to 0.
            </para>
          </listitem>
        </varlistentry>
        <varlistentry>
          <term>
            <option>lxc.group</option>
          </term>
          <listitem>
            <para>
              A multi-value key (can be used multiple times) to put the
              container in a container group. Those groups can then be
              used (amongst other things) to start a series of related
              containers.
            </para>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

    <refsect2>
    <title>Autostart and System Boot</title>
    <para>
          Each container can be part of any number of groups or no group at all.
          Two groups are special. One is the NULL group, i.e. the container does
          not belong to any group. The other group is the "onboot" group.
    </para>

    <para>
          When the system boots with the LXC service enabled, it will first
          attempt to boot any containers with lxc.start.auto == 1 that is a member
          of the "onboot" group. The startup will be in order of lxc.start.order.
          If an lxc.start.delay has been specified, that delay will be honored
          before attempting to start the next container to give the current
          container time to begin initialization and reduce overloading the host
          system. After starting the members of the "onboot" group, the LXC system
          will proceed to boot containers with lxc.start.auto == 1 which are not
          members of any group (the NULL group) and proceed as with the onboot
          group.
    </para>

    </refsect2>

    <refsect2>
      <title>Container Environment</title>
      <para>
        If you want to pass environment variables into the container (that
        is, environment variables which will be available to init and all of
        its descendents), you can use <command>lxc.environment</command>
        parameters to do so.  Be careful that you do not pass in anything
        sensitive; any process in the container which doesn't have its
        environment scrubbed will have these variables available to it, and
        environment variables are always available via
        <command>/proc/PID/environ</command>.
      </para>

      <para>
        This configuration parameter can be specified multiple times; once
        for each environment variable you wish to configure.
      </para>

      <variablelist>
        <varlistentry>
          <term>
            <option>lxc.environment</option>
          </term>
          <listitem>
            <para>
              Specify an environment variable to pass into the container.
              Example:
            </para>
            <programlisting>
              lxc.environment = APP_ENV=production
              lxc.environment = SYSLOG_SERVER=192.0.2.42
            </programlisting>
            <para>
            It is possible to inherit host environment variables by setting the
            name of the variable without a "=" sign. For example:
            </para>
            <programlisting>
              lxc.environment = PATH
            </programlisting>
          </listitem>
        </varlistentry>
      </variablelist>
    </refsect2>

  </refsect1>

  <refsect1>
    <title>Examples</title>
      <para>
        In addition to the few examples given below, you will find
        some other examples of configuration file in @DOCDIR@/examples
      </para>
    <refsect2>
      <title>Network</title>
      <para>This configuration sets up a container to use a veth pair
        device with one side plugged to a bridge br0 (which has been
        configured before on the system by the administrator). The
        virtual network device visible in the container is renamed to
        eth0.</para>
      <programlisting>
        lxc.uts.name = myhostname
        lxc.net.0.type = veth
        lxc.net.0.flags = up
        lxc.net.0.link = br0
        lxc.net.0.name = eth0
        lxc.net.0.hwaddr = 4a:49:43:49:79:bf
        lxc.net.0.ipv4.address = 10.2.3.5/24 10.2.3.255
        lxc.net.0.ipv6.address = 2003:db8:1:0:214:1234:fe0b:3597
      </programlisting>
    </refsect2>

    <refsect2>
      <title>UID/GID mapping</title>
      <para>This configuration will map both user and group ids in the
        range 0-9999 in the container to the ids 100000-109999 on the host.
      </para>
      <programlisting>
        lxc.idmap = u 0 100000 10000
        lxc.idmap = g 0 100000 10000
      </programlisting>
    </refsect2>

    <refsect2>
      <title>Control group</title>
      <para>This configuration will setup several control groups for
      the application, cpuset.cpus restricts usage of the defined cpu,
      cpus.share prioritize the control group, devices.allow makes
      usable the specified devices.</para>
      <programlisting>
        lxc.cgroup.cpuset.cpus = 0,1
        lxc.cgroup.cpu.shares = 1234
        lxc.cgroup.devices.deny = a
        lxc.cgroup.devices.allow = c 1:3 rw
        lxc.cgroup.devices.allow = b 8:0 rw
      </programlisting>
    </refsect2>

    <refsect2>
      <title>Complex configuration</title>
      <para>This example show a complex configuration making a complex
      network stack, using the control groups, setting a new hostname,
      mounting some locations and a changing root file system.</para>
      <programlisting>
        lxc.uts.name = complex
        lxc.net.0.type = veth
        lxc.net.0.flags = up
        lxc.net.0.link = br0
        lxc.net.0.hwaddr = 4a:49:43:49:79:bf
        lxc.net.0.ipv4.address = 10.2.3.5/24 10.2.3.255
        lxc.net.0.ipv6.address = 2003:db8:1:0:214:1234:fe0b:3597
        lxc.net.0.ipv6.address = 2003:db8:1:0:214:5432:feab:3588
        lxc.net.1.type = macvlan
        lxc.net.1.flags = up
        lxc.net.1.link = eth0
        lxc.net.1.hwaddr = 4a:49:43:49:79:bd
        lxc.net.1.ipv4.address = 10.2.3.4/24
        lxc.net.1.ipv4.address = 192.168.10.125/24
        lxc.net.1.ipv6.address = 2003:db8:1:0:214:1234:fe0b:3596
        lxc.net.2.type = phys
        lxc.net.2.flags = up
        lxc.net.2.link = random0
        lxc.net.2.hwaddr = 4a:49:43:49:79:ff
        lxc.net.2.ipv4.address = 10.2.3.6/24
        lxc.net.2.ipv6.address = 2003:db8:1:0:214:1234:fe0b:3297
        lxc.cgroup.cpuset.cpus = 0,1
        lxc.cgroup.cpu.shares = 1234
        lxc.cgroup.devices.deny = a
        lxc.cgroup.devices.allow = c 1:3 rw
        lxc.cgroup.devices.allow = b 8:0 rw
        lxc.mount.fstab = /etc/fstab.complex
        lxc.mount.entry = /lib /root/myrootfs/lib none ro,bind 0 0
        lxc.rootfs.path = dir:/mnt/rootfs.complex
        lxc.rootfs.options = idmap=container
        lxc.cap.drop = sys_module mknod setuid net_raw
        lxc.cap.drop = mac_override
      </programlisting>
    </refsect2>

  </refsect1>

  <refsect1>
    <title>See Also</title>
    <simpara>
      <citerefentry>
        <refentrytitle><command>chroot</command></refentrytitle>
        <manvolnum>1</manvolnum>
      </citerefentry>,

      <citerefentry>
        <refentrytitle><command>pivot_root</command></refentrytitle>
        <manvolnum>8</manvolnum>
      </citerefentry>,

      <citerefentry>
        <refentrytitle><filename>fstab</filename></refentrytitle>
        <manvolnum>5</manvolnum>
      </citerefentry>,

      <citerefentry>
        <refentrytitle><filename>capabilities</filename></refentrytitle>
        <manvolnum>7</manvolnum>
      </citerefentry>
    </simpara>
  </refsect1>

  &seealso;

  <refsect1>
    <title>Author</title>
    <para>Daniel Lezcano <email>daniel.lezcano@free.fr</email></para>
  </refsect1>

</refentry>

<!-- Keep this comment at the end of the file
Local variables:
mode: sgml
sgml-omittag:t
sgml-shorttag:t
sgml-minimize-attributes:nil
sgml-always-quote-attributes:t
sgml-indent-step:2
sgml-indent-data:t
sgml-parent-document:nil
sgml-default-dtd-file:nil
sgml-exposed-tags:nil
sgml-local-catalogs:nil
sgml-local-ecat-files:nil
End:
-->