]>
Commit | Line | Data |
---|---|---|
0006ebb4 SF |
1 | |
2 | Aufs4 -- advanced multi layered unification filesystem version 4.x | |
3 | http://aufs.sf.net | |
4 | Junjiro R. Okajima | |
5 | ||
6 | ||
7 | 0. Introduction | |
8 | ---------------------------------------- | |
9 | In the early days, aufs was entirely re-designed and re-implemented | |
10 | Unionfs Version 1.x series. Adding many original ideas, approaches, | |
11 | improvements and implementations, it becomes totally different from | |
12 | Unionfs while keeping the basic features. | |
13 | Recently, Unionfs Version 2.x series begin taking some of the same | |
14 | approaches to aufs1's. | |
15 | Unionfs is being developed by Professor Erez Zadok at Stony Brook | |
16 | University and his team. | |
17 | ||
18 | Aufs4 supports linux-4.0 and later, and for linux-3.x series try aufs3. | |
19 | If you want older kernel version support, try aufs2-2.6.git or | |
20 | aufs2-standalone.git repository, aufs1 from CVS on SourceForge. | |
21 | ||
22 | Note: it becomes clear that "Aufs was rejected. Let's give it up." | |
23 | According to Christoph Hellwig, linux rejects all union-type | |
24 | filesystems but UnionMount. | |
25 | <http://marc.info/?l=linux-kernel&m=123938533724484&w=2> | |
26 | ||
27 | PS. Al Viro seems have a plan to merge aufs as well as overlayfs and | |
28 | UnionMount, and he pointed out an issue around a directory mutex | |
29 | lock and aufs addressed it. But it is still unsure whether aufs will | |
30 | be merged (or any other union solution). | |
31 | <http://marc.info/?l=linux-kernel&m=136312705029295&w=1> | |
32 | ||
33 | ||
34 | 1. Features | |
35 | ---------------------------------------- | |
36 | - unite several directories into a single virtual filesystem. The member | |
37 | directory is called as a branch. | |
38 | - you can specify the permission flags to the branch, which are 'readonly', | |
39 | 'readwrite' and 'whiteout-able.' | |
40 | - by upper writable branch, internal copyup and whiteout, files/dirs on | |
41 | readonly branch are modifiable logically. | |
42 | - dynamic branch manipulation, add, del. | |
43 | - etc... | |
44 | ||
45 | Also there are many enhancements in aufs, such as: | |
46 | - test only the highest one for the directory permission (dirperm1) | |
47 | - copyup on open (coo=) | |
48 | - 'move' policy for copy-up between two writable branches, after | |
49 | checking free space. | |
50 | - xattr, acl | |
51 | - readdir(3) in userspace. | |
52 | - keep inode number by external inode number table | |
53 | - keep the timestamps of file/dir in internal copyup operation | |
54 | - seekable directory, supporting NFS readdir. | |
55 | - whiteout is hardlinked in order to reduce the consumption of inodes | |
56 | on branch | |
57 | - do not copyup, nor create a whiteout when it is unnecessary | |
58 | - revert a single systemcall when an error occurs in aufs | |
59 | - remount interface instead of ioctl | |
60 | - maintain /etc/mtab by an external command, /sbin/mount.aufs. | |
61 | - loopback mounted filesystem as a branch | |
62 | - kernel thread for removing the dir who has a plenty of whiteouts | |
63 | - support copyup sparse file (a file which has a 'hole' in it) | |
64 | - default permission flags for branches | |
65 | - selectable permission flags for ro branch, whether whiteout can | |
66 | exist or not | |
67 | - export via NFS. | |
68 | - support <sysfs>/fs/aufs and <debugfs>/aufs. | |
69 | - support multiple writable branches, some policies to select one | |
70 | among multiple writable branches. | |
71 | - a new semantics for link(2) and rename(2) to support multiple | |
72 | writable branches. | |
73 | - no glibc changes are required. | |
74 | - pseudo hardlink (hardlink over branches) | |
75 | - allow a direct access manually to a file on branch, e.g. bypassing aufs. | |
76 | including NFS or remote filesystem branch. | |
77 | - userspace wrapper for pathconf(3)/fpathconf(3) with _PC_LINK_MAX. | |
78 | - and more... | |
79 | ||
80 | Currently these features are dropped temporary from aufs4. | |
81 | See design/08plan.txt in detail. | |
82 | - nested mount, i.e. aufs as readonly no-whiteout branch of another aufs | |
83 | (robr) | |
84 | - statistics of aufs thread (/sys/fs/aufs/stat) | |
85 | ||
86 | Features or just an idea in the future (see also design/*.txt), | |
87 | - reorder the branch index without del/re-add. | |
88 | - permanent xino files for NFSD | |
89 | - an option for refreshing the opened files after add/del branches | |
90 | - light version, without branch manipulation. (unnecessary?) | |
91 | - copyup in userspace | |
92 | - inotify in userspace | |
93 | - readv/writev | |
94 | ||
95 | ||
96 | 2. Download | |
97 | ---------------------------------------- | |
98 | There are three GIT trees for aufs4, aufs4-linux.git, | |
99 | aufs4-standalone.git, and aufs-util.git. Note that there is no "4" in | |
100 | "aufs-util.git." | |
101 | While the aufs-util is always necessary, you need either of aufs4-linux | |
102 | or aufs4-standalone. | |
103 | ||
104 | The aufs4-linux tree includes the whole linux mainline GIT tree, | |
105 | git://git.kernel.org/.../torvalds/linux.git. | |
106 | And you cannot select CONFIG_AUFS_FS=m for this version, eg. you cannot | |
107 | build aufs4 as an external kernel module. | |
108 | Several extra patches are not included in this tree. Only | |
109 | aufs4-standalone tree contains them. They are described in the later | |
110 | section "Configuration and Compilation." | |
111 | ||
112 | On the other hand, the aufs4-standalone tree has only aufs source files | |
113 | and necessary patches, and you can select CONFIG_AUFS_FS=m. | |
114 | But you need to apply all aufs patches manually. | |
115 | ||
116 | You will find GIT branches whose name is in form of "aufs4.x" where "x" | |
117 | represents the linux kernel version, "linux-4.x". For instance, | |
118 | "aufs4.0" is for linux-4.0. For latest "linux-4.x-rcN", use | |
119 | "aufs4.x-rcN" branch. | |
120 | ||
121 | o aufs4-linux tree | |
122 | $ git clone --reference /your/linux/git/tree \ | |
123 | git://github.com/sfjro/aufs4-linux.git aufs4-linux.git | |
124 | - if you don't have linux GIT tree, then remove "--reference ..." | |
125 | $ cd aufs4-linux.git | |
126 | $ git checkout origin/aufs4.0 | |
127 | ||
128 | Or You may want to directly git-pull aufs into your linux GIT tree, and | |
129 | leave the patch-work to GIT. | |
130 | $ cd /your/linux/git/tree | |
131 | $ git remote add aufs4 git://github.com/sfjro/aufs4-linux.git | |
132 | $ git fetch aufs4 | |
133 | $ git checkout -b my4.0 v4.0 | |
134 | $ (add your local change...) | |
135 | $ git pull aufs4 aufs4.0 | |
136 | - now you have v4.0 + your_changes + aufs4.0 in you my4.0 branch. | |
137 | - you may need to solve some conflicts between your_changes and | |
138 | aufs4.0. in this case, git-rerere is recommended so that you can | |
139 | solve the similar conflicts automatically when you upgrade to 4.1 or | |
140 | later in the future. | |
141 | ||
142 | o aufs4-standalone tree | |
143 | $ git clone git://github.com/sfjro/aufs4-standalone.git aufs4-standalone.git | |
144 | $ cd aufs4-standalone.git | |
145 | $ git checkout origin/aufs4.0 | |
146 | ||
147 | o aufs-util tree | |
148 | $ git clone git://git.code.sf.net/p/aufs/aufs-util aufs-util.git | |
149 | - note that the public aufs-util.git is on SourceForge instead of | |
150 | GitHUB. | |
151 | $ cd aufs-util.git | |
152 | $ git checkout origin/aufs4.0 | |
153 | ||
154 | Note: The 4.x-rcN branch is to be used with `rc' kernel versions ONLY. | |
155 | The minor version number, 'x' in '4.x', of aufs may not always | |
156 | follow the minor version number of the kernel. | |
157 | Because changes in the kernel that cause the use of a new | |
158 | minor version number do not always require changes to aufs-util. | |
159 | ||
160 | Since aufs-util has its own minor version number, you may not be | |
161 | able to find a GIT branch in aufs-util for your kernel's | |
162 | exact minor version number. | |
163 | In this case, you should git-checkout the branch for the | |
164 | nearest lower number. | |
165 | ||
166 | For (an unreleased) example: | |
167 | If you are using "linux-4.10" and the "aufs4.10" branch | |
168 | does not exist in aufs-util repository, then "aufs4.9", "aufs4.8" | |
169 | or something numerically smaller is the branch for your kernel. | |
170 | ||
171 | Also you can view all branches by | |
172 | $ git branch -a | |
173 | ||
174 | ||
175 | 3. Configuration and Compilation | |
176 | ---------------------------------------- | |
177 | Make sure you have git-checkout'ed the correct branch. | |
178 | ||
179 | For aufs4-linux tree, | |
180 | - enable CONFIG_AUFS_FS. | |
181 | - set other aufs configurations if necessary. | |
182 | ||
183 | For aufs4-standalone tree, | |
184 | There are several ways to build. | |
185 | ||
186 | 1. | |
187 | - apply ./aufs4-kbuild.patch to your kernel source files. | |
188 | - apply ./aufs4-base.patch too. | |
189 | - apply ./aufs4-mmap.patch too. | |
190 | - apply ./aufs4-standalone.patch too, if you have a plan to set | |
191 | CONFIG_AUFS_FS=m. otherwise you don't need ./aufs4-standalone.patch. | |
192 | - copy ./{Documentation,fs,include/uapi/linux/aufs_type.h} files to your | |
193 | kernel source tree. Never copy $PWD/include/uapi/linux/Kbuild. | |
194 | - enable CONFIG_AUFS_FS, you can select either | |
195 | =m or =y. | |
196 | - and build your kernel as usual. | |
197 | - install the built kernel. | |
198 | Note: Since linux-3.9, every filesystem module requires an alias | |
199 | "fs-<fsname>". You should make sure that "fs-aufs" is listed in your | |
200 | modules.aliases file if you set CONFIG_AUFS_FS=m. | |
201 | - install the header files too by "make headers_install" to the | |
202 | directory where you specify. By default, it is $PWD/usr. | |
203 | "make help" shows a brief note for headers_install. | |
204 | - and reboot your system. | |
205 | ||
206 | 2. | |
207 | - module only (CONFIG_AUFS_FS=m). | |
208 | - apply ./aufs4-base.patch to your kernel source files. | |
209 | - apply ./aufs4-mmap.patch too. | |
210 | - apply ./aufs4-standalone.patch too. | |
211 | - build your kernel, don't forget "make headers_install", and reboot. | |
212 | - edit ./config.mk and set other aufs configurations if necessary. | |
213 | Note: You should read $PWD/fs/aufs/Kconfig carefully which describes | |
214 | every aufs configurations. | |
215 | - build the module by simple "make". | |
216 | Note: Since linux-3.9, every filesystem module requires an alias | |
217 | "fs-<fsname>". You should make sure that "fs-aufs" is listed in your | |
218 | modules.aliases file. | |
219 | - you can specify ${KDIR} make variable which points to your kernel | |
220 | source tree. | |
221 | - install the files | |
222 | + run "make install" to install the aufs module, or copy the built | |
223 | $PWD/aufs.ko to /lib/modules/... and run depmod -a (or reboot simply). | |
224 | + run "make install_headers" (instead of headers_install) to install | |
225 | the modified aufs header file (you can specify DESTDIR which is | |
226 | available in aufs standalone version's Makefile only), or copy | |
227 | $PWD/usr/include/linux/aufs_type.h to /usr/include/linux or wherever | |
228 | you like manually. By default, the target directory is $PWD/usr. | |
229 | - no need to apply aufs4-kbuild.patch, nor copying source files to your | |
230 | kernel source tree. | |
231 | ||
232 | Note: The header file aufs_type.h is necessary to build aufs-util | |
233 | as well as "make headers_install" in the kernel source tree. | |
234 | headers_install is subject to be forgotten, but it is essentially | |
235 | necessary, not only for building aufs-util. | |
236 | You may not meet problems without headers_install in some older | |
237 | version though. | |
238 | ||
239 | And then, | |
240 | - read README in aufs-util, build and install it | |
241 | - note that your distribution may contain an obsoleted version of | |
242 | aufs_type.h in /usr/include/linux or something. When you build aufs | |
243 | utilities, make sure that your compiler refers the correct aufs header | |
244 | file which is built by "make headers_install." | |
245 | - if you want to use readdir(3) in userspace or pathconf(3) wrapper, | |
246 | then run "make install_ulib" too. And refer to the aufs manual in | |
247 | detail. | |
248 | ||
249 | There several other patches in aufs4-standalone.git. They are all | |
250 | optional. When you meet some problems, they will help you. | |
251 | - aufs4-loopback.patch | |
252 | Supports a nested loopback mount in a branch-fs. This patch is | |
253 | unnecessary until aufs produces a message like "you may want to try | |
254 | another patch for loopback file". | |
255 | - vfs-ino.patch | |
256 | Modifies a system global kernel internal function get_next_ino() in | |
257 | order to stop assigning 0 for an inode-number. Not directly related to | |
258 | aufs, but recommended generally. | |
259 | - tmpfs-idr.patch | |
260 | Keeps the tmpfs inode number as the lowest value. Effective to reduce | |
261 | the size of aufs XINO files for tmpfs branch. Also it prevents the | |
262 | duplication of inode number, which is important for backup tools and | |
263 | other utilities. When you find aufs XINO files for tmpfs branch | |
264 | growing too much, try this patch. | |
265 | - lockdep-debug.patch | |
266 | Because aufs is not only an ordinary filesystem (callee of VFS), but | |
267 | also a caller of VFS functions for branch filesystems, subclassing of | |
268 | the internal locks for LOCKDEP is necessary. LOCKDEP is a debugging | |
269 | feature of linux kernel. If you enable CONFIG_LOCKDEP, then you will | |
270 | need to apply this debug patch to expand several constant values. | |
271 | If don't know what LOCKDEP, then you don't have apply this patch. | |
272 | ||
273 | ||
274 | 4. Usage | |
275 | ---------------------------------------- | |
276 | At first, make sure aufs-util are installed, and please read the aufs | |
277 | manual, aufs.5 in aufs-util.git tree. | |
278 | $ man -l aufs.5 | |
279 | ||
280 | And then, | |
281 | $ mkdir /tmp/rw /tmp/aufs | |
282 | # mount -t aufs -o br=/tmp/rw:${HOME} none /tmp/aufs | |
283 | ||
284 | Here is another example. The result is equivalent. | |
285 | # mount -t aufs -o br=/tmp/rw=rw:${HOME}=ro none /tmp/aufs | |
286 | Or | |
287 | # mount -t aufs -o br:/tmp/rw none /tmp/aufs | |
288 | # mount -o remount,append:${HOME} /tmp/aufs | |
289 | ||
290 | Then, you can see whole tree of your home dir through /tmp/aufs. If | |
291 | you modify a file under /tmp/aufs, the one on your home directory is | |
292 | not affected, instead the same named file will be newly created under | |
293 | /tmp/rw. And all of your modification to a file will be applied to | |
294 | the one under /tmp/rw. This is called the file based Copy on Write | |
295 | (COW) method. | |
296 | Aufs mount options are described in aufs.5. | |
297 | If you run chroot or something and make your aufs as a root directory, | |
298 | then you need to customize the shutdown script. See the aufs manual in | |
299 | detail. | |
300 | ||
301 | Additionally, there are some sample usages of aufs which are a | |
302 | diskless system with network booting, and LiveCD over NFS. | |
303 | See sample dir in CVS tree on SourceForge. | |
304 | ||
305 | ||
306 | 5. Contact | |
307 | ---------------------------------------- | |
308 | When you have any problems or strange behaviour in aufs, please let me | |
309 | know with: | |
310 | - /proc/mounts (instead of the output of mount(8)) | |
311 | - /sys/module/aufs/* | |
312 | - /sys/fs/aufs/* (if you have them) | |
313 | - /debug/aufs/* (if you have them) | |
314 | - linux kernel version | |
315 | if your kernel is not plain, for example modified by distributor, | |
316 | the url where i can download its source is necessary too. | |
317 | - aufs version which was printed at loading the module or booting the | |
318 | system, instead of the date you downloaded. | |
319 | - configuration (define/undefine CONFIG_AUFS_xxx) | |
320 | - kernel configuration or /proc/config.gz (if you have it) | |
321 | - behaviour which you think to be incorrect | |
322 | - actual operation, reproducible one is better | |
323 | - mailto: aufs-users at lists.sourceforge.net | |
324 | ||
325 | Usually, I don't watch the Public Areas(Bugs, Support Requests, Patches, | |
326 | and Feature Requests) on SourceForge. Please join and write to | |
327 | aufs-users ML. | |
328 | ||
329 | ||
330 | 6. Acknowledgements | |
331 | ---------------------------------------- | |
332 | Thanks to everyone who have tried and are using aufs, whoever | |
333 | have reported a bug or any feedback. | |
334 | ||
335 | Especially donators: | |
336 | Tomas Matejicek(slax.org) made a donation (much more than once). | |
337 | Since Apr 2010, Tomas M (the author of Slax and Linux Live | |
338 | scripts) is making "doubling" donations. | |
339 | Unfortunately I cannot list all of the donators, but I really | |
340 | appreciate. | |
341 | It ends Aug 2010, but the ordinary donation URL is still available. | |
342 | <http://sourceforge.net/donate/index.php?group_id=167503> | |
343 | Dai Itasaka made a donation (2007/8). | |
344 | Chuck Smith made a donation (2008/4, 10 and 12). | |
345 | Henk Schoneveld made a donation (2008/9). | |
346 | Chih-Wei Huang, ASUS, CTC donated Eee PC 4G (2008/10). | |
347 | Francois Dupoux made a donation (2008/11). | |
348 | Bruno Cesar Ribas and Luis Carlos Erpen de Bona, C3SL serves public | |
349 | aufs2 GIT tree (2009/2). | |
350 | William Grant made a donation (2009/3). | |
351 | Patrick Lane made a donation (2009/4). | |
352 | The Mail Archive (mail-archive.com) made donations (2009/5). | |
353 | Nippy Networks (Ed Wildgoose) made a donation (2009/7). | |
354 | New Dream Network, LLC (www.dreamhost.com) made a donation (2009/11). | |
355 | Pavel Pronskiy made a donation (2011/2). | |
356 | Iridium and Inmarsat satellite phone retailer (www.mailasail.com), Nippy | |
357 | Networks (Ed Wildgoose) made a donation for hardware (2011/3). | |
358 | Max Lekomcev (DOM-TV project) made a donation (2011/7, 12, 2012/3, 6 and | |
359 | 11). | |
360 | Sam Liddicott made a donation (2011/9). | |
361 | Era Scarecrow made a donation (2013/4). | |
362 | Bor Ratajc made a donation (2013/4). | |
363 | Alessandro Gorreta made a donation (2013/4). | |
364 | POIRETTE Marc made a donation (2013/4). | |
365 | Alessandro Gorreta made a donation (2013/4). | |
366 | lauri kasvandik made a donation (2013/5). | |
367 | "pemasu from Finland" made a donation (2013/7). | |
368 | The Parted Magic Project made a donation (2013/9 and 11). | |
369 | Pavel Barta made a donation (2013/10). | |
370 | Nikolay Pertsev made a donation (2014/5). | |
371 | James B made a donation (2014/7 and 2015/7). | |
372 | Stefano Di Biase made a donation (2014/8). | |
373 | Daniel Epellei made a donation (2015/1). | |
374 | OmegaPhil made a donation (2016/1). | |
375 | Tomasz Szewczyk made a donation (2016/4). | |
376 | James Burry made a donation (2016/12). | |
377 | ||
378 | Thank you very much. | |
379 | Donations are always, including future donations, very important and | |
380 | helpful for me to keep on developing aufs. | |
381 | ||
382 | ||
383 | 7. | |
384 | ---------------------------------------- | |
385 | If you are an experienced user, no explanation is needed. Aufs is | |
386 | just a linux filesystem. | |
387 | ||
388 | ||
389 | Enjoy! | |
390 | ||
391 | # Local variables: ; | |
392 | # mode: text; | |
393 | # End: ; |