]>
Commit | Line | Data |
---|---|---|
a907c907 N |
1 | Written by: Neil Brown |
2 | Please see MAINTAINERS file for where to send questions. | |
7c37fbda NB |
3 | |
4 | Overlay Filesystem | |
5 | ================== | |
6 | ||
7 | This document describes a prototype for a new approach to providing | |
8 | overlay-filesystem functionality in Linux (sometimes referred to as | |
9 | union-filesystems). An overlay-filesystem tries to present a | |
10 | filesystem which is the result over overlaying one filesystem on top | |
11 | of the other. | |
12 | ||
13 | The result will inevitably fail to look exactly like a normal | |
14 | filesystem for various technical reasons. The expectation is that | |
15 | many use cases will be able to ignore these differences. | |
16 | ||
17 | This approach is 'hybrid' because the objects that appear in the | |
18 | filesystem do not all appear to belong to that filesystem. In many | |
19 | cases an object accessed in the union will be indistinguishable | |
20 | from accessing the corresponding object from the original filesystem. | |
21 | This is most obvious from the 'st_dev' field returned by stat(2). | |
22 | ||
23 | While directories will report an st_dev from the overlay-filesystem, | |
65f26738 | 24 | non-directory objects may report an st_dev from the lower filesystem or |
7c37fbda NB |
25 | upper filesystem that is providing the object. Similarly st_ino will |
26 | only be unique when combined with st_dev, and both of these can change | |
27 | over the lifetime of a non-directory object. Many applications and | |
28 | tools ignore these values and will not be affected. | |
29 | ||
65f26738 AG |
30 | In the special case of all overlay layers on the same underlying |
31 | filesystem, all objects will report an st_dev from the overlay | |
32 | filesystem and st_ino from the underlying filesystem. This will | |
33 | make the overlay mount more compliant with filesystem scanners and | |
34 | overlay objects will be distinguishable from the corresponding | |
35 | objects in the original filesystem. | |
36 | ||
7c37fbda NB |
37 | Upper and Lower |
38 | --------------- | |
39 | ||
40 | An overlay filesystem combines two filesystems - an 'upper' filesystem | |
41 | and a 'lower' filesystem. When a name exists in both filesystems, the | |
42 | object in the 'upper' filesystem is visible while the object in the | |
43 | 'lower' filesystem is either hidden or, in the case of directories, | |
44 | merged with the 'upper' object. | |
45 | ||
46 | It would be more correct to refer to an upper and lower 'directory | |
47 | tree' rather than 'filesystem' as it is quite possible for both | |
48 | directory trees to be in the same filesystem and there is no | |
49 | requirement that the root of a filesystem be given for either upper or | |
50 | lower. | |
51 | ||
52 | The lower filesystem can be any filesystem supported by Linux and does | |
53 | not need to be writable. The lower filesystem can even be another | |
54 | overlayfs. The upper filesystem will normally be writable and if it | |
55 | is it must support the creation of trusted.* extended attributes, and | |
56 | must provide valid d_type in readdir responses, so NFS is not suitable. | |
57 | ||
58 | A read-only overlay of two read-only filesystems may use any | |
59 | filesystem type. | |
60 | ||
61 | Directories | |
62 | ----------- | |
63 | ||
64 | Overlaying mainly involves directories. If a given name appears in both | |
65 | upper and lower filesystems and refers to a non-directory in either, | |
66 | then the lower object is hidden - the name refers only to the upper | |
67 | object. | |
68 | ||
69 | Where both upper and lower objects are directories, a merged directory | |
70 | is formed. | |
71 | ||
72 | At mount time, the two directories given as mount options "lowerdir" and | |
73 | "upperdir" are combined into a merged directory: | |
74 | ||
ef94b186 | 75 | mount -t overlay overlay -olowerdir=/lower,upperdir=/upper,\ |
c3c86996 | 76 | workdir=/work /merged |
7c37fbda NB |
77 | |
78 | The "workdir" needs to be an empty directory on the same filesystem | |
79 | as upperdir. | |
80 | ||
81 | Then whenever a lookup is requested in such a merged directory, the | |
82 | lookup is performed in each actual directory and the combined result | |
83 | is cached in the dentry belonging to the overlay filesystem. If both | |
84 | actual lookups find directories, both are stored and a merged | |
85 | directory is created, otherwise only one is stored: the upper if it | |
86 | exists, else the lower. | |
87 | ||
88 | Only the lists of names from directories are merged. Other content | |
89 | such as metadata and extended attributes are reported for the upper | |
90 | directory only. These attributes of the lower directory are hidden. | |
91 | ||
92 | whiteouts and opaque directories | |
93 | -------------------------------- | |
94 | ||
95 | In order to support rm and rmdir without changing the lower | |
96 | filesystem, an overlay filesystem needs to record in the upper filesystem | |
97 | that files have been removed. This is done using whiteouts and opaque | |
98 | directories (non-directories are always opaque). | |
99 | ||
100 | A whiteout is created as a character device with 0/0 device number. | |
101 | When a whiteout is found in the upper level of a merged directory, any | |
102 | matching name in the lower level is ignored, and the whiteout itself | |
103 | is also hidden. | |
104 | ||
105 | A directory is made opaque by setting the xattr "trusted.overlay.opaque" | |
106 | to "y". Where the upper filesystem contains an opaque directory, any | |
107 | directory in the lower filesystem with the same name is ignored. | |
108 | ||
109 | readdir | |
110 | ------- | |
111 | ||
112 | When a 'readdir' request is made on a merged directory, the upper and | |
113 | lower directories are each read and the name lists merged in the | |
114 | obvious way (upper is read first, then lower - entries that already | |
115 | exist are not re-added). This merged name list is cached in the | |
116 | 'struct file' and so remains as long as the file is kept open. If the | |
117 | directory is opened and read by two processes at the same time, they | |
118 | will each have separate caches. A seekdir to the start of the | |
119 | directory (offset 0) followed by a readdir will cause the cache to be | |
120 | discarded and rebuilt. | |
121 | ||
122 | This means that changes to the merged directory do not appear while a | |
123 | directory is being read. This is unlikely to be noticed by many | |
124 | programs. | |
125 | ||
126 | seek offsets are assigned sequentially when the directories are read. | |
127 | Thus if | |
c3c86996 | 128 | |
7c37fbda NB |
129 | - read part of a directory |
130 | - remember an offset, and close the directory | |
131 | - re-open the directory some time later | |
132 | - seek to the remembered offset | |
133 | ||
134 | there may be little correlation between the old and new locations in | |
135 | the list of filenames, particularly if anything has changed in the | |
136 | directory. | |
137 | ||
138 | Readdir on directories that are not merged is simply handled by the | |
139 | underlying directory (upper or lower). | |
140 | ||
a6c60655 MS |
141 | renaming directories |
142 | -------------------- | |
143 | ||
144 | When renaming a directory that is on the lower layer or merged (i.e. the | |
145 | directory was not created on the upper layer to start with) overlayfs can | |
146 | handle it in two different ways: | |
147 | ||
c3c86996 | 148 | 1. return EXDEV error: this error is returned by rename(2) when trying to |
a6c60655 MS |
149 | move a file or directory across filesystem boundaries. Hence |
150 | applications are usually prepared to hande this error (mv(1) for example | |
151 | recursively copies the directory tree). This is the default behavior. | |
152 | ||
c3c86996 | 153 | 2. If the "redirect_dir" feature is enabled, then the directory will be |
a6c60655 MS |
154 | copied up (but not the contents). Then the "trusted.overlay.redirect" |
155 | extended attribute is set to the path of the original location from the | |
156 | root of the overlay. Finally the directory is moved to the new | |
157 | location. | |
7c37fbda NB |
158 | |
159 | Non-directories | |
160 | --------------- | |
161 | ||
162 | Objects that are not directories (files, symlinks, device-special | |
163 | files etc.) are presented either from the upper or lower filesystem as | |
164 | appropriate. When a file in the lower filesystem is accessed in a way | |
165 | the requires write-access, such as opening for write access, changing | |
166 | some metadata etc., the file is first copied from the lower filesystem | |
167 | to the upper filesystem (copy_up). Note that creating a hard-link | |
168 | also requires copy_up, though of course creation of a symlink does | |
169 | not. | |
170 | ||
171 | The copy_up may turn out to be unnecessary, for example if the file is | |
172 | opened for read-write but the data is not modified. | |
173 | ||
174 | The copy_up process first makes sure that the containing directory | |
175 | exists in the upper filesystem - creating it and any parents as | |
176 | necessary. It then creates the object with the same metadata (owner, | |
177 | mode, mtime, symlink-target etc.) and then if the object is a file, the | |
178 | data is copied from the lower to the upper filesystem. Finally any | |
179 | extended attributes are copied up. | |
180 | ||
181 | Once the copy_up is complete, the overlay filesystem simply | |
182 | provides direct access to the newly created file in the upper | |
183 | filesystem - future operations on the file are barely noticed by the | |
184 | overlay filesystem (though an operation on the name of the file such as | |
185 | rename or unlink will of course be noticed and handled). | |
186 | ||
187 | ||
a78d9f0d MS |
188 | Multiple lower layers |
189 | --------------------- | |
190 | ||
191 | Multiple lower layers can now be given using the the colon (":") as a | |
192 | separator character between the directory names. For example: | |
193 | ||
194 | mount -t overlay overlay -olowerdir=/lower1:/lower2:/lower3 /merged | |
195 | ||
6d900f5a MS |
196 | As the example shows, "upperdir=" and "workdir=" may be omitted. In |
197 | that case the overlay will be read-only. | |
198 | ||
199 | The specified lower directories will be stacked beginning from the | |
200 | rightmost one and going left. In the above example lower1 will be the | |
201 | top, lower2 the middle and lower3 the bottom layer. | |
a78d9f0d MS |
202 | |
203 | ||
7c37fbda NB |
204 | Non-standard behavior |
205 | --------------------- | |
206 | ||
207 | The copy_up operation essentially creates a new, identical file and | |
208 | moves it over to the old name. The new file may be on a different | |
209 | filesystem, so both st_dev and st_ino of the file may change. | |
210 | ||
026e5e0c | 211 | Any open files referring to this inode will access the old data. |
7c37fbda | 212 | |
7c37fbda NB |
213 | If a file with multiple hard links is copied up, then this will |
214 | "break" the link. Changes will not be propagated to other names | |
215 | referring to the same inode. | |
216 | ||
a6c60655 MS |
217 | Unless "redirect_dir" feature is enabled, rename(2) on a lower or merged |
218 | directory will fail with EXDEV. | |
2d8f2908 | 219 | |
7c37fbda NB |
220 | Changes to underlying filesystems |
221 | --------------------------------- | |
222 | ||
223 | Offline changes, when the overlay is not mounted, are allowed to either | |
224 | the upper or the lower trees. | |
225 | ||
226 | Changes to the underlying filesystems while part of a mounted overlay | |
227 | filesystem are not allowed. If the underlying filesystem is changed, | |
228 | the behavior of the overlay is undefined, though it will not result in | |
229 | a crash or deadlock. | |
2b7a8f36 MS |
230 | |
231 | Testsuite | |
232 | --------- | |
233 | ||
234 | There's testsuite developed by David Howells at: | |
235 | ||
236 | git://git.infradead.org/users/dhowells/unionmount-testsuite.git | |
237 | ||
238 | Run as root: | |
239 | ||
240 | # cd unionmount-testsuite | |
241 | # ./run --ov |