[mirror_lxc.git] / README

NAME
       lxc - linux containers

QUICK START
       You  are  in a hurry, and you don’t want to read this man page. Ok, without
       warranty, here are the commands to launch a shell inside a container with a
       predefined configuration template, it may work.
        lxc-execute -n foo -f /etc/lxc/lxc-macvlan.conf /bin/bash

OVERVIEW
       The container technology is actively being pushed into the mainstream linux
       kernel. It provides the resource management through the control groups  aka
       process containers and resource isolation through the namespaces.

       The  linux  containers, lxc, aims to use these new functionnalities to pro-
       vide an userspace container object which provides full  resource  isolation
       and resource control for an applications or a system.

       The first objective of this project is to make the life easier for the ker-
       nel developers involved in the containers project and  especially  to  con-
       tinue  working  on  the  Checkpoint/Restart  new features. The lxc is small
       enough to easily manage a container with simple command lines and  complete
       enough to be used for other purposes.

REQUIREMENTS
       The  lxc  relies  on  a  set of functionnalies provided by the kernel which
       needs to be active. Depending of the missing functionnalities the lxc  will
       work with a restricted number of functionnalities or will simply fails.

       The following list gives the kernel features to be enabled in the kernel to
       have the full features container:

                * General
                  * Control Group support
                    -> namespace cgroup subsystem
                    -> cpuset support
                    -> Group CPU scheduler
                    -> control group freeze subsystem
                    -> Basis for grouping tasks (Control Groups)
                    -> Simple CPU accounting
                    -> Resource counters
                    -> Memory resource controllers for Control Groups
                    -> Namespace support
                      -> UTS namespace
                      -> IPC namespace
                      -> User namespace
                      -> Pid namespace
                * Network support
                  -> Networking options
                     -> Network namespace support

       For the moment the easiest way to have all the features in the kernel is to
       use the git tree at:
:

git://git.kernel.org/pub/scm/linux/kernel/git/daveh/linux-2.6-lxc.git
       But the kernel version >= 2.6.27 shipped with the distros,  may  work  with
       lxc, this one will have less functionnalities but enough to be interesting.
       The planned kernel version  which  lxc  should  be  fully  functionnaly  is
       2.6.29.

       Before  using the lxc, your system should be configured with the file capa-
       bilities, otherwise you will need to run the lxc commands as root. The con-
       trol group should be mounted anywhere, eg: mount -t cgroup cgroup /cgroup

FUNCTIONAL SPECIFICATION
       A  container is an object where the configuration is persistent. The appli-
       cation will be launched inside this container and it will use the  configu-
       ration which was previously created.

       How to run an application in a container ?

       Before  running  an  application, you should know what are the resource you
       want to isolate. The default configuration is isolation of  the  pids,  the
       sysv  ipc and the mount points.  If you want to run a simple shell inside a
       container, a basic configuration is needed, especially if you want to share
       the rootfs. If you want to run an application like sshd, you should provide
       a new network stack and a new hostname. If you want to avoid conflicts with
       some  files  eg.  /var/run/httpd.pid,  you  should remount /var/run with an
       empty directory. If you want to avoid the conflicts in all the  cases,  you
       can specify a rootfs for the container. The rootfs can be a directory tree,
       previously bind mounted with the initial rootfs, so you can still use  your
       distro but with your own /etc and /home

       Here is an example of directory tree for sshd:

       [root@lxc sshd]$ tree -d rootfs

       rootfs
       |-- bin
       |-- dev
       |   |-- pts
       |   ‘-- shm
       |       ‘-- network
       |-- etc
       |   ‘-- ssh
       |-- lib
       |-- proc
       |-- root
       |-- sbin
       |-- sys
       |-- usr
       ‘-- var
           |-- empty
           |   ‘-- sshd
           |-- lib
           |   ‘-- empty
           |       ‘-- sshd
           ‘-- run
               ‘-- sshd
       and the mount points file associated with it:

            [root@lxc sshd]$ cat fstab

            /lib /home/root/sshd/rootfs/lib none ro,bind 0 0
            /bin /home/root/sshd/rootfs/bin none ro,bind 0 0
            /usr /home/root/sshd/rootfs/usr none ro,bind 0 0
            /sbin /home/root/sshd/rootfs/sbin none ro,bind 0 0

       How to run a system in a container ?

       Running a system inside a container is paradoxically easier than running an
       application. Why ? Because you don’t have to care about the resources to be
       isolated,  everything  need  to  be  isolated except /dev which needs to be
       remounted in the container rootfs, the other  resources  are  specified  as
       being  isolated  but  without  configuration because the container will set
       them up. eg. the ipv4 address will be setup by the  system  container  init
       scripts. Here is an example of the mount points file:

            [root@lxc debian]$ cat fstab

            /dev /home/root/debian/rootfs/dev none bind 0 0
            /dev/pts /home/root/debian/rootfs/dev/pts  none bind 0 0

       A good idea to have the console is to mount bind this one to our tty, so we
       see the output of the system container booting and we can log to it

            /proc/self/fd/0 /home/root/debian/rootfs/dev/console none bind 0 0

       These examples are avaible in the contrib package located at:
        https://sourceforge.net/projects/lxc/

   CONTAINER LIFE CYCLE
       When the container is created, it contains the  configuration  information.
       When  a  process  is  launched, the container will be starting and running.
       When the last process running inside the container exits, the container  is
       stopped.

       In  case of failure when the container is initialized, it will pass through
       the aborting state.

          ---------
         | STOPPED |<---------------
          ---------                 |
              |                     |
            start                   |
              |                     |
              V                     |
          ----------                |
         | STARTING |--error-       |
          ----------         |      |
              |              |      |
              V              V      |
          ---------    ----------   |
         | RUNNING |  | ABORTING |  |
          ---------    ----------   |
              |              |      |
         no process          |      |
              |              |      |
              V              |      |
          ----------         |      |
         | STOPPING |<-------       |
          ----------                |
              |                     |
               ---------------------

   CONFIGURATION
       The container is configured through a configuration file, the format of the
       configuration file is described in lxc.conf(5)

   CREATING / DESTROYING THE CONTAINERS
       The  container  is created via the lxc-create command. It takes a container
       name as parameter and an optional configuration file. The name is  used  by
       the  different commands to refer to this container. The lxc-destroy command
       will destroy the container object.

              lxc-create -n foo
              lxc-destroy -n foo

   STARTING / STOPPING A CONTAINER
       When the container has been created, it is ready to run  an  application  /
       system.  When  the  application  has  to  be destroyed the container can be
       stopped, that will kill all the processes of the container.

       Running an application inside a container is not exactly the same thing  as
       running a system. For this reason, there is two commands to run an applica-
       tion into a container:

              lxc-execute -n foo [-f config] /bin/bash
              lxc-start -n foo [/bin/bash]

       lxc-execute command will run the specified command into a container but  it
       will  mount  /proc  and autocreate/autodestroy the container if it does not
       exist. It will furthermore create an intermediate process, lxc-init,  which
       is  in  charge to launch the specified command, that allows to support dae-
       mons in the container. In other words, in the container  lxc-init  has  the
       pid 1 and the first process of the application has the pid 2.

       lxc-start  command  will run the specified command into the container doing
       nothing else than using the configuration specified by lxc-create.  The pid
       of  the  first  process is 1. If no command is specified lxc-start will run
       /sbin/init.

       To summarize, lxc-execute is for running an application  and  lxc-start  is
       for running a system.

       If the application is no longer responding, inaccessible and is not able to
       finish by itself, a wild lxc-stop command will kill all  the  processes  in
       the container without pity.
              lxc-stop -n foo

   FREEZE / UNFREEZE A CONTAINER
       Sometime,  it is useful to stop all the processes belonging to a container,
       eg. for job scheduling. The commands:

              lxc-freeze -n foo

       will put all the processes in an ininteruptible state and

              lxc-unfreeze -n foo

       will resume all the tasks.

       This feature is enabled if the cgroup freezer is enabled in the kernel.

   GETTING INFORMATION ABOUT THE CONTAINER
       When there are a lot of containers, it is hard to follow what has been cre-
       ated or destroyed, what is running or what are the pids running into a spe-
       cific container. For this reason, the following commands give this informa-
       tion:

              lxc-ls
              lxc-ps -n foo
              lxc-info -n foo

       lxc-ls  list the containers of the system. The command is a script built on
       top of ls, so it accepts the options of the ls commands, eg:

              lxc-ls -1

       will display the containers list in one column or:

              lxc-ls -l

       will display the containers list and their permissions.

       lxc-ps will display the pids for a specific container. Like lxc-ls,  lxc-ps
       is built on top of ps and accepts the same options, eg:

              lxc-ps -n foo --forest

       will display the process hierarchy for the container ’foo’.

       lxc-info  gives  informations for a specific container, at present only the
       state of the container is displayed.

       Here is an example on how the combination of these commands allow  to  list
       all the containers and retrieve their state.

              for i in $(lxc-ls -1); do
                lxc-info -n $i
              done

       And displaying all the pids of all the containers:
              for i in $(lxc-ls -1); do
                lxc-info -n $i
              done

       And displaying all the pids of all the containers:

              for i in $(lxc-ls -1); do
                lxc-ps -n $i --forest
              done

   MONITORING THE CONTAINERS
       It  is  sometime  useful to track the states of a container, for example to
       monitor it or just to wait for a specific state in a script.

       lxc-monitor command will monitor one or several containers.  The  parameter
       of this command accept a regular expression for example:

              lxc-monitor -n "foo|bar"

       will monitor the states of containers named ’foo’ and ’bar’, and:

              lxc-monitor -n ".*"

       will monitor all the containers.

   SETTING THE CONTROL GROUP FOR A CONTAINER
       The container is tied with the control groups. A control group can be setup
       when the container is running to change or to retrieve its value.

       lxc-cgroup command is used to set or get a control group subsystem which is
       associated  with a container. The subsystem name is handle by the user, the
       command won’t do any syntax checking on name, if the name does not  exists,
       the command will fail.

              lxc-cgroup -n foo cpuset.cpus

       will display the content of this subsystem.

              lxc-cgroup -n foo cpus.share 512

       will set the subsystem to the specified value.

BUGS
       The  lxc  is  still  in  development, so the command syntax and the API can
       change. The version 1.0.0 will be the frozen version.

SEE ALSO
       lxc-create(1), lxc-destroy(1), lxc-start(1),  lxc-execute(1),  lxc-stop(1),
       lxc-monitor(1),  lxc-wait(1),  lxc-cgroup(1),  lxc-ls(1),  lxc-ps(1),  lxc-
       info(1), lxc-freeze(1), lxc-unfreeze(1), lxc.conf(5),

AUTHOR
       Daniel Lezcano <daniel.lezcano@free.fr>
Commit	Line	Data
7f989f01	1	NAME
	2	lxc - linux containers
	3
	4	QUICK START
	5	You are in a hurry, and you don’t want to read this man page. Ok, without
	6	warranty, here are the commands to launch a shell inside a container with a
	7	predefined configuration template, it may work.
	8	lxc-execute -n foo -f /etc/lxc/lxc-macvlan.conf /bin/bash
	9
	10	OVERVIEW
	11	The container technology is actively being pushed into the mainstream linux
	12	kernel. It provides the resource management through the control groups aka
	13	process containers and resource isolation through the namespaces.
	14
	15	The linux containers, lxc, aims to use these new functionnalities to pro-
	16	vide an userspace container object which provides full resource isolation
	17	and resource control for an applications or a system.
	18
	19	The first objective of this project is to make the life easier for the ker-
	20	nel developers involved in the containers project and especially to con-
	21	tinue working on the Checkpoint/Restart new features. The lxc is small
	22	enough to easily manage a container with simple command lines and complete
	23	enough to be used for other purposes.
	24
	25	REQUIREMENTS
	26	The lxc relies on a set of functionnalies provided by the kernel which
	27	needs to be active. Depending of the missing functionnalities the lxc will
	28	work with a restricted number of functionnalities or will simply fails.
	29
	30	The following list gives the kernel features to be enabled in the kernel to
	31	have the full features container:
	32
	33	* General
	34	* Control Group support
	35	-> namespace cgroup subsystem
	36	-> cpuset support
	37	-> Group CPU scheduler
	38	-> control group freeze subsystem
	39	-> Basis for grouping tasks (Control Groups)
	40	-> Simple CPU accounting
	41	-> Resource counters
	42	-> Memory resource controllers for Control Groups
	43	-> Namespace support
	44	-> UTS namespace
	45	-> IPC namespace
	46	-> User namespace
	47	-> Pid namespace
	48	* Network support
	49	-> Networking options
	50	-> Network namespace support
	51
	52	For the moment the easiest way to have all the features in the kernel is to
	53	use the git tree at:
	54	:
5e97c3fc	55
5e97c3fc	56	git://git.kernel.org/pub/scm/linux/kernel/git/daveh/linux-2.6-lxc.git
7f989f01	57	But the kernel version >= 2.6.27 shipped with the distros, may work with
	58	lxc, this one will have less functionnalities but enough to be interesting.
	59	The planned kernel version which lxc should be fully functionnaly is
	60	2.6.29.
	61
	62	Before using the lxc, your system should be configured with the file capa-
	63	bilities, otherwise you will need to run the lxc commands as root. The con-
	64	trol group should be mounted anywhere, eg: mount -t cgroup cgroup /cgroup
	65
	66	FUNCTIONAL SPECIFICATION
	67	A container is an object where the configuration is persistent. The appli-
	68	cation will be launched inside this container and it will use the configu-
	69	ration which was previously created.
	70
	71	How to run an application in a container ?
	72
	73	Before running an application, you should know what are the resource you
	74	want to isolate. The default configuration is isolation of the pids, the
	75	sysv ipc and the mount points. If you want to run a simple shell inside a
	76	container, a basic configuration is needed, especially if you want to share
	77	the rootfs. If you want to run an application like sshd, you should provide
	78	a new network stack and a new hostname. If you want to avoid conflicts with
	79	some files eg. /var/run/httpd.pid, you should remount /var/run with an
	80	empty directory. If you want to avoid the conflicts in all the cases, you
	81	can specify a rootfs for the container. The rootfs can be a directory tree,
	82	previously bind mounted with the initial rootfs, so you can still use your
	83	distro but with your own /etc and /home
	84
	85	Here is an example of directory tree for sshd:
	86
	87	[root@lxc sshd]$ tree -d rootfs
	88
	89	rootfs
	90	\|-- bin
	91	\|-- dev
	92	\| \|-- pts
	93	\| ‘-- shm
	94	\| ‘-- network
	95	\|-- etc
	96	\| ‘-- ssh
	97	\|-- lib
	98	\|-- proc
	99	\|-- root
	100	\|-- sbin
	101	\|-- sys
	102	\|-- usr
	103	‘-- var
	104	\|-- empty
	105	\| ‘-- sshd
	106	\|-- lib
	107	\| ‘-- empty
	108	\| ‘-- sshd
	109	‘-- run
	110	‘-- sshd
	111	and the mount points file associated with it:
	112
	113	[root@lxc sshd]$ cat fstab
	114
	115	/lib /home/root/sshd/rootfs/lib none ro,bind 0 0
	116	/bin /home/root/sshd/rootfs/bin none ro,bind 0 0
	117	/usr /home/root/sshd/rootfs/usr none ro,bind 0 0
	118	/sbin /home/root/sshd/rootfs/sbin none ro,bind 0 0
	119
	120	How to run a system in a container ?
121
122	Running a system inside a container is paradoxically easier than running an
123	application. Why ? Because you don’t have to care about the resources to be
124	isolated, everything need to be isolated except /dev which needs to be
125	remounted in the container rootfs, the other resources are specified as
126	being isolated but without configuration because the container will set
127	them up. eg. the ipv4 address will be setup by the system container init
128	scripts. Here is an example of the mount points file:
129
130	[root@lxc debian]$ cat fstab
131
132	/dev /home/root/debian/rootfs/dev none bind 0 0
133	/dev/pts /home/root/debian/rootfs/dev/pts none bind 0 0
134
135	A good idea to have the console is to mount bind this one to our tty, so we
136	see the output of the system container booting and we can log to it
137
138	/proc/self/fd/0 /home/root/debian/rootfs/dev/console none bind 0 0
139
140	These examples are avaible in the contrib package located at:
141	https://sourceforge.net/projects/lxc/
142
143	CONTAINER LIFE CYCLE
144	When the container is created, it contains the configuration information.
145	When a process is launched, the container will be starting and running.
146	When the last process running inside the container exits, the container is
147	stopped.
148
149	In case of failure when the container is initialized, it will pass through
150	the aborting state.
151
152	---------
153	\| STOPPED \|<---------------
154	--------- \|
155	\| \|
156	start \|
157	\| \|
158	V \|
159	---------- \|
160	\| STARTING \|--error- \|
161	---------- \| \|
162	\| \| \|
163	V V \|
164	--------- ---------- \|
165	\| RUNNING \| \| ABORTING \| \|
166	--------- ---------- \|
167	\| \| \|
168	no process \| \|
169	\| \| \|
170	V \| \|
171	---------- \| \|
172	\| STOPPING \|<------- \|
173	---------- \|
174	\| \|
175	---------------------
176
177	CONFIGURATION
178	The container is configured through a configuration file, the format of the
179	configuration file is described in lxc.conf(5)
180
181	CREATING / DESTROYING THE CONTAINERS
182	The container is created via the lxc-create command. It takes a container
183	name as parameter and an optional configuration file. The name is used by
184	the different commands to refer to this container. The lxc-destroy command
185	will destroy the container object.
186
187	lxc-create -n foo
188	lxc-destroy -n foo
189
190	STARTING / STOPPING A CONTAINER
191	When the container has been created, it is ready to run an application /
192	system. When the application has to be destroyed the container can be
193	stopped, that will kill all the processes of the container.
194
195	Running an application inside a container is not exactly the same thing as
196	running a system. For this reason, there is two commands to run an applica-
197	tion into a container:
5e97c3fc	198
7f989f01	199	lxc-execute -n foo [-f config] /bin/bash
7f989f01	200	lxc-start -n foo [/bin/bash]
5e97c3fc	201
7f989f01	202	lxc-execute command will run the specified command into a container but it
	203	will mount /proc and autocreate/autodestroy the container if it does not
	204	exist. It will furthermore create an intermediate process, lxc-init, which
	205	is in charge to launch the specified command, that allows to support dae-
	206	mons in the container. In other words, in the container lxc-init has the
	207	pid 1 and the first process of the application has the pid 2.
5e97c3fc	208
7f989f01	209	lxc-start command will run the specified command into the container doing
	210	nothing else than using the configuration specified by lxc-create. The pid
	211	of the first process is 1. If no command is specified lxc-start will run
	212	/sbin/init.
5e97c3fc	213
7f989f01	214	To summarize, lxc-execute is for running an application and lxc-start is
7f989f01	215	for running a system.
5e97c3fc	216
7f989f01	217	If the application is no longer responding, inaccessible and is not able to
	218	finish by itself, a wild lxc-stop command will kill all the processes in
	219	the container without pity.
	220	lxc-stop -n foo
5e97c3fc	221
7f989f01	222	FREEZE / UNFREEZE A CONTAINER
	223	Sometime, it is useful to stop all the processes belonging to a container,
	224	eg. for job scheduling. The commands:
848e8927	225
7f989f01	226	lxc-freeze -n foo
5e97c3fc	227
7f989f01	228	will put all the processes in an ininteruptible state and
848e8927	229
7f989f01	230	lxc-unfreeze -n foo
5e97c3fc	231
7f989f01	232	will resume all the tasks.
5e97c3fc	233
7f989f01	234	This feature is enabled if the cgroup freezer is enabled in the kernel.
5e97c3fc	235
7f989f01	236	GETTING INFORMATION ABOUT THE CONTAINER
	237	When there are a lot of containers, it is hard to follow what has been cre-
	238	ated or destroyed, what is running or what are the pids running into a spe-
	239	cific container. For this reason, the following commands give this informa-
	240	tion:
5e97c3fc	241
7f989f01	242	lxc-ls
	243	lxc-ps -n foo
	244	lxc-info -n foo
5e97c3fc	245
7f989f01	246	lxc-ls list the containers of the system. The command is a script built on
7f989f01	247	top of ls, so it accepts the options of the ls commands, eg:
5e97c3fc	248
7f989f01	249	lxc-ls -1
5e97c3fc	250
7f989f01	251	will display the containers list in one column or:
5e97c3fc	252
7f989f01	253	lxc-ls -l
5e97c3fc	254
7f989f01	255	will display the containers list and their permissions.
5e97c3fc	256
7f989f01	257	lxc-ps will display the pids for a specific container. Like lxc-ls, lxc-ps
7f989f01	258	is built on top of ps and accepts the same options, eg:
5e97c3fc	259
7f989f01	260	lxc-ps -n foo --forest
5e97c3fc	261
7f989f01	262	will display the process hierarchy for the container ’foo’.
5e97c3fc	263
7f989f01	264	lxc-info gives informations for a specific container, at present only the
7f989f01	265	state of the container is displayed.
5e97c3fc	266
7f989f01	267	Here is an example on how the combination of these commands allow to list
7f989f01	268	all the containers and retrieve their state.
5e97c3fc	269
7f989f01	270	for i in $(lxc-ls -1); do
	271	lxc-info -n $i
	272	done
5e97c3fc	273
7f989f01	274	And displaying all the pids of all the containers:
	275	for i in $(lxc-ls -1); do
	276	lxc-info -n $i
	277	done
5e97c3fc	278
7f989f01	279	And displaying all the pids of all the containers:
5e97c3fc	280
7f989f01	281	for i in $(lxc-ls -1); do
	282	lxc-ps -n $i --forest
	283	done
5e97c3fc	284
7f989f01	285	MONITORING THE CONTAINERS
	286	It is sometime useful to track the states of a container, for example to
	287	monitor it or just to wait for a specific state in a script.
5e97c3fc	288
7f989f01	289	lxc-monitor command will monitor one or several containers. The parameter
7f989f01	290	of this command accept a regular expression for example:
5e97c3fc	291
7f989f01	292	lxc-monitor -n "foo\|bar"
5e97c3fc	293
7f989f01	294	will monitor the states of containers named ’foo’ and ’bar’, and:
5e97c3fc	295
7f989f01	296	lxc-monitor -n ".*"
5e97c3fc	297
7f989f01	298	will monitor all the containers.
5e97c3fc	299
7f989f01	300	SETTING THE CONTROL GROUP FOR A CONTAINER
	301	The container is tied with the control groups. A control group can be setup
	302	when the container is running to change or to retrieve its value.
5e97c3fc	303
7f989f01	304	lxc-cgroup command is used to set or get a control group subsystem which is
	305	associated with a container. The subsystem name is handle by the user, the
	306	command won’t do any syntax checking on name, if the name does not exists,
	307	the command will fail.
5e97c3fc	308
7f989f01	309	lxc-cgroup -n foo cpuset.cpus
5e97c3fc	310
7f989f01	311	will display the content of this subsystem.
848e8927	312
7f989f01	313	lxc-cgroup -n foo cpus.share 512
848e8927	314
7f989f01	315	will set the subsystem to the specified value.
848e8927	316
7f989f01	317	BUGS
	318	The lxc is still in development, so the command syntax and the API can
	319	change. The version 1.0.0 will be the frozen version.
848e8927	320
7f989f01	321	SEE ALSO
	322	lxc-create(1), lxc-destroy(1), lxc-start(1), lxc-execute(1), lxc-stop(1),
	323	lxc-monitor(1), lxc-wait(1), lxc-cgroup(1), lxc-ls(1), lxc-ps(1), lxc-
	324	info(1), lxc-freeze(1), lxc-unfreeze(1), lxc.conf(5),
5e97c3fc	325
7f989f01	326	AUTHOR
7f989f01	327	Daniel Lezcano <daniel.lezcano@free.fr>