]>
Commit | Line | Data |
---|---|---|
1da177e4 | 1 | |
5ea626aa | 2 | Overview of the Linux Virtual File System |
1da177e4 | 3 | |
5ea626aa | 4 | Original author: Richard Gooch <rgooch@atnf.csiro.au> |
1da177e4 | 5 | |
cc7d1f8f | 6 | Last updated on October 28, 2005 |
1da177e4 | 7 | |
5ea626aa PE |
8 | Copyright (C) 1999 Richard Gooch |
9 | Copyright (C) 2005 Pekka Enberg | |
1da177e4 | 10 | |
5ea626aa | 11 | This file is released under the GPLv2. |
1da177e4 | 12 | |
1da177e4 | 13 | |
cc7d1f8f PE |
14 | Introduction |
15 | ============ | |
1da177e4 | 16 | |
cc7d1f8f PE |
17 | The Virtual File System (also known as the Virtual Filesystem Switch) |
18 | is the software layer in the kernel that provides the filesystem | |
19 | interface to userspace programs. It also provides an abstraction | |
20 | within the kernel which allows different filesystem implementations to | |
21 | coexist. | |
1da177e4 | 22 | |
cc7d1f8f PE |
23 | VFS system calls open(2), stat(2), read(2), write(2), chmod(2) and so |
24 | on are called from a process context. Filesystem locking is described | |
25 | in the document Documentation/filesystems/Locking. | |
1da177e4 | 26 | |
1da177e4 | 27 | |
cc7d1f8f PE |
28 | Directory Entry Cache (dcache) |
29 | ------------------------------ | |
1da177e4 | 30 | |
cc7d1f8f PE |
31 | The VFS implements the open(2), stat(2), chmod(2), and similar system |
32 | calls. The pathname argument that is passed to them is used by the VFS | |
33 | to search through the directory entry cache (also known as the dentry | |
34 | cache or dcache). This provides a very fast look-up mechanism to | |
35 | translate a pathname (filename) into a specific dentry. Dentries live | |
36 | in RAM and are never saved to disc: they exist only for performance. | |
37 | ||
38 | The dentry cache is meant to be a view into your entire filespace. As | |
39 | most computers cannot fit all dentries in the RAM at the same time, | |
40 | some bits of the cache are missing. In order to resolve your pathname | |
41 | into a dentry, the VFS may have to resort to creating dentries along | |
42 | the way, and then loading the inode. This is done by looking up the | |
43 | inode. | |
44 | ||
45 | ||
46 | The Inode Object | |
47 | ---------------- | |
48 | ||
49 | An individual dentry usually has a pointer to an inode. Inodes are | |
50 | filesystem objects such as regular files, directories, FIFOs and other | |
51 | beasts. They live either on the disc (for block device filesystems) | |
52 | or in the memory (for pseudo filesystems). Inodes that live on the | |
53 | disc are copied into the memory when required and changes to the inode | |
54 | are written back to disc. A single inode can be pointed to by multiple | |
55 | dentries (hard links, for example, do this). | |
56 | ||
57 | To look up an inode requires that the VFS calls the lookup() method of | |
58 | the parent directory inode. This method is installed by the specific | |
59 | filesystem implementation that the inode lives in. Once the VFS has | |
60 | the required dentry (and hence the inode), we can do all those boring | |
61 | things like open(2) the file, or stat(2) it to peek at the inode | |
62 | data. The stat(2) operation is fairly simple: once the VFS has the | |
63 | dentry, it peeks at the inode data and passes some of it back to | |
64 | userspace. | |
65 | ||
66 | ||
67 | The File Object | |
68 | --------------- | |
1da177e4 LT |
69 | |
70 | Opening a file requires another operation: allocation of a file | |
71 | structure (this is the kernel-side implementation of file | |
5ea626aa | 72 | descriptors). The freshly allocated file structure is initialized with |
1da177e4 LT |
73 | a pointer to the dentry and a set of file operation member functions. |
74 | These are taken from the inode data. The open() file method is then | |
75 | called so the specific filesystem implementation can do it's work. You | |
cc7d1f8f PE |
76 | can see that this is another switch performed by the VFS. The file |
77 | structure is placed into the file descriptor table for the process. | |
1da177e4 LT |
78 | |
79 | Reading, writing and closing files (and other assorted VFS operations) | |
80 | is done by using the userspace file descriptor to grab the appropriate | |
cc7d1f8f PE |
81 | file structure, and then calling the required file structure method to |
82 | do whatever is required. For as long as the file is open, it keeps the | |
83 | dentry in use, which in turn means that the VFS inode is still in use. | |
1da177e4 | 84 | |
5ea626aa PE |
85 | |
86 | Registering and Mounting a Filesystem | |
cc7d1f8f | 87 | ===================================== |
1da177e4 | 88 | |
cc7d1f8f PE |
89 | To register and unregister a filesystem, use the following API |
90 | functions: | |
1da177e4 | 91 | |
cc7d1f8f | 92 | #include <linux/fs.h> |
1da177e4 | 93 | |
cc7d1f8f PE |
94 | extern int register_filesystem(struct file_system_type *); |
95 | extern int unregister_filesystem(struct file_system_type *); | |
1da177e4 | 96 | |
cc7d1f8f PE |
97 | The passed struct file_system_type describes your filesystem. When a |
98 | request is made to mount a device onto a directory in your filespace, | |
99 | the VFS will call the appropriate get_sb() method for the specific | |
100 | filesystem. The dentry for the mount point will then be updated to | |
101 | point to the root inode for the new filesystem. | |
1da177e4 | 102 | |
cc7d1f8f PE |
103 | You can see all filesystems that are registered to the kernel in the |
104 | file /proc/filesystems. | |
1da177e4 LT |
105 | |
106 | ||
5ea626aa | 107 | struct file_system_type |
cc7d1f8f | 108 | ----------------------- |
1da177e4 | 109 | |
5ea626aa | 110 | This describes the filesystem. As of kernel 2.6.13, the following |
1da177e4 LT |
111 | members are defined: |
112 | ||
113 | struct file_system_type { | |
114 | const char *name; | |
115 | int fs_flags; | |
5ea626aa PE |
116 | struct super_block *(*get_sb) (struct file_system_type *, int, |
117 | const char *, void *); | |
118 | void (*kill_sb) (struct super_block *); | |
119 | struct module *owner; | |
120 | struct file_system_type * next; | |
121 | struct list_head fs_supers; | |
1da177e4 LT |
122 | }; |
123 | ||
124 | name: the name of the filesystem type, such as "ext2", "iso9660", | |
125 | "msdos" and so on | |
126 | ||
127 | fs_flags: various flags (i.e. FS_REQUIRES_DEV, FS_NO_DCACHE, etc.) | |
128 | ||
5ea626aa | 129 | get_sb: the method to call when a new instance of this |
1da177e4 LT |
130 | filesystem should be mounted |
131 | ||
5ea626aa PE |
132 | kill_sb: the method to call when an instance of this filesystem |
133 | should be unmounted | |
134 | ||
135 | owner: for internal VFS use: you should initialize this to THIS_MODULE in | |
136 | most cases. | |
1da177e4 | 137 | |
5ea626aa PE |
138 | next: for internal VFS use: you should initialize this to NULL |
139 | ||
140 | The get_sb() method has the following arguments: | |
1da177e4 LT |
141 | |
142 | struct super_block *sb: the superblock structure. This is partially | |
5ea626aa PE |
143 | initialized by the VFS and the rest must be initialized by the |
144 | get_sb() method | |
145 | ||
146 | int flags: mount flags | |
147 | ||
148 | const char *dev_name: the device name we are mounting. | |
1da177e4 LT |
149 | |
150 | void *data: arbitrary mount options, usually comes as an ASCII | |
151 | string | |
152 | ||
153 | int silent: whether or not to be silent on error | |
154 | ||
5ea626aa | 155 | The get_sb() method must determine if the block device specified |
1da177e4 LT |
156 | in the superblock contains a filesystem of the type the method |
157 | supports. On success the method returns the superblock pointer, on | |
158 | failure it returns NULL. | |
159 | ||
160 | The most interesting member of the superblock structure that the | |
5ea626aa | 161 | get_sb() method fills in is the "s_op" field. This is a pointer to |
1da177e4 LT |
162 | a "struct super_operations" which describes the next level of the |
163 | filesystem implementation. | |
164 | ||
5ea626aa PE |
165 | Usually, a filesystem uses generic one of the generic get_sb() |
166 | implementations and provides a fill_super() method instead. The | |
167 | generic methods are: | |
168 | ||
169 | get_sb_bdev: mount a filesystem residing on a block device | |
1da177e4 | 170 | |
5ea626aa PE |
171 | get_sb_nodev: mount a filesystem that is not backed by a device |
172 | ||
173 | get_sb_single: mount a filesystem which shares the instance between | |
174 | all mounts | |
175 | ||
176 | A fill_super() method implementation has the following arguments: | |
177 | ||
178 | struct super_block *sb: the superblock structure. The method fill_super() | |
179 | must initialize this properly. | |
180 | ||
181 | void *data: arbitrary mount options, usually comes as an ASCII | |
182 | string | |
183 | ||
184 | int silent: whether or not to be silent on error | |
185 | ||
186 | ||
cc7d1f8f PE |
187 | The Superblock Object |
188 | ===================== | |
189 | ||
190 | A superblock object represents a mounted filesystem. | |
191 | ||
192 | ||
5ea626aa | 193 | struct super_operations |
cc7d1f8f | 194 | ----------------------- |
1da177e4 LT |
195 | |
196 | This describes how the VFS can manipulate the superblock of your | |
5ea626aa | 197 | filesystem. As of kernel 2.6.13, the following members are defined: |
1da177e4 LT |
198 | |
199 | struct super_operations { | |
5ea626aa PE |
200 | struct inode *(*alloc_inode)(struct super_block *sb); |
201 | void (*destroy_inode)(struct inode *); | |
202 | ||
203 | void (*read_inode) (struct inode *); | |
204 | ||
205 | void (*dirty_inode) (struct inode *); | |
206 | int (*write_inode) (struct inode *, int); | |
207 | void (*put_inode) (struct inode *); | |
208 | void (*drop_inode) (struct inode *); | |
209 | void (*delete_inode) (struct inode *); | |
210 | void (*put_super) (struct super_block *); | |
211 | void (*write_super) (struct super_block *); | |
212 | int (*sync_fs)(struct super_block *sb, int wait); | |
213 | void (*write_super_lockfs) (struct super_block *); | |
214 | void (*unlockfs) (struct super_block *); | |
215 | int (*statfs) (struct super_block *, struct kstatfs *); | |
216 | int (*remount_fs) (struct super_block *, int *, char *); | |
217 | void (*clear_inode) (struct inode *); | |
218 | void (*umount_begin) (struct super_block *); | |
219 | ||
220 | void (*sync_inodes) (struct super_block *sb, | |
221 | struct writeback_control *wbc); | |
222 | int (*show_options)(struct seq_file *, struct vfsmount *); | |
223 | ||
224 | ssize_t (*quota_read)(struct super_block *, int, char *, size_t, loff_t); | |
225 | ssize_t (*quota_write)(struct super_block *, int, const char *, size_t, loff_t); | |
1da177e4 LT |
226 | }; |
227 | ||
228 | All methods are called without any locks being held, unless otherwise | |
229 | noted. This means that most methods can block safely. All methods are | |
230 | only called from a process context (i.e. not from an interrupt handler | |
231 | or bottom half). | |
232 | ||
5ea626aa PE |
233 | alloc_inode: this method is called by inode_alloc() to allocate memory |
234 | for struct inode and initialize it. | |
235 | ||
236 | destroy_inode: this method is called by destroy_inode() to release | |
237 | resources allocated for struct inode. | |
238 | ||
1da177e4 | 239 | read_inode: this method is called to read a specific inode from the |
5ea626aa PE |
240 | mounted filesystem. The i_ino member in the struct inode is |
241 | initialized by the VFS to indicate which inode to read. Other | |
242 | members are filled in by this method. | |
243 | ||
244 | You can set this to NULL and use iget5_locked() instead of iget() | |
245 | to read inodes. This is necessary for filesystems for which the | |
246 | inode number is not sufficient to identify an inode. | |
247 | ||
248 | dirty_inode: this method is called by the VFS to mark an inode dirty. | |
1da177e4 LT |
249 | |
250 | write_inode: this method is called when the VFS needs to write an | |
251 | inode to disc. The second parameter indicates whether the write | |
252 | should be synchronous or not, not all filesystems check this flag. | |
253 | ||
254 | put_inode: called when the VFS inode is removed from the inode | |
5ea626aa | 255 | cache. |
1da177e4 LT |
256 | |
257 | drop_inode: called when the last access to the inode is dropped, | |
258 | with the inode_lock spinlock held. | |
259 | ||
5ea626aa | 260 | This method should be either NULL (normal UNIX filesystem |
1da177e4 LT |
261 | semantics) or "generic_delete_inode" (for filesystems that do not |
262 | want to cache inodes - causing "delete_inode" to always be | |
263 | called regardless of the value of i_nlink) | |
264 | ||
5ea626aa | 265 | The "generic_delete_inode()" behavior is equivalent to the |
1da177e4 LT |
266 | old practice of using "force_delete" in the put_inode() case, |
267 | but does not have the races that the "force_delete()" approach | |
268 | had. | |
269 | ||
270 | delete_inode: called when the VFS wants to delete an inode | |
271 | ||
1da177e4 LT |
272 | put_super: called when the VFS wishes to free the superblock |
273 | (i.e. unmount). This is called with the superblock lock held | |
274 | ||
275 | write_super: called when the VFS superblock needs to be written to | |
276 | disc. This method is optional | |
277 | ||
5ea626aa PE |
278 | sync_fs: called when VFS is writing out all dirty data associated with |
279 | a superblock. The second parameter indicates whether the method | |
280 | should wait until the write out has been completed. Optional. | |
281 | ||
cc7d1f8f PE |
282 | write_super_lockfs: called when VFS is locking a filesystem and |
283 | forcing it into a consistent state. This method is currently | |
284 | used by the Logical Volume Manager (LVM). | |
5ea626aa PE |
285 | |
286 | unlockfs: called when VFS is unlocking a filesystem and making it writable | |
287 | again. | |
288 | ||
1da177e4 LT |
289 | statfs: called when the VFS needs to get filesystem statistics. This |
290 | is called with the kernel lock held | |
291 | ||
292 | remount_fs: called when the filesystem is remounted. This is called | |
293 | with the kernel lock held | |
294 | ||
295 | clear_inode: called then the VFS clears the inode. Optional | |
296 | ||
5ea626aa PE |
297 | umount_begin: called when the VFS is unmounting a filesystem. |
298 | ||
299 | sync_inodes: called when the VFS is writing out dirty data associated with | |
300 | a superblock. | |
301 | ||
302 | show_options: called by the VFS to show mount options for /proc/<pid>/mounts. | |
303 | ||
304 | quota_read: called by the VFS to read from filesystem quota file. | |
305 | ||
306 | quota_write: called by the VFS to write to filesystem quota file. | |
307 | ||
1da177e4 LT |
308 | The read_inode() method is responsible for filling in the "i_op" |
309 | field. This is a pointer to a "struct inode_operations" which | |
310 | describes the methods that can be performed on individual inodes. | |
311 | ||
312 | ||
cc7d1f8f PE |
313 | The Inode Object |
314 | ================ | |
315 | ||
316 | An inode object represents an object within the filesystem. | |
317 | ||
318 | ||
5ea626aa | 319 | struct inode_operations |
cc7d1f8f | 320 | ----------------------- |
1da177e4 LT |
321 | |
322 | This describes how the VFS can manipulate an inode in your | |
5ea626aa | 323 | filesystem. As of kernel 2.6.13, the following members are defined: |
1da177e4 LT |
324 | |
325 | struct inode_operations { | |
5ea626aa PE |
326 | int (*create) (struct inode *,struct dentry *,int, struct nameidata *); |
327 | struct dentry * (*lookup) (struct inode *,struct dentry *, struct nameidata *); | |
1da177e4 LT |
328 | int (*link) (struct dentry *,struct inode *,struct dentry *); |
329 | int (*unlink) (struct inode *,struct dentry *); | |
330 | int (*symlink) (struct inode *,struct dentry *,const char *); | |
331 | int (*mkdir) (struct inode *,struct dentry *,int); | |
332 | int (*rmdir) (struct inode *,struct dentry *); | |
333 | int (*mknod) (struct inode *,struct dentry *,int,dev_t); | |
334 | int (*rename) (struct inode *, struct dentry *, | |
335 | struct inode *, struct dentry *); | |
5ea626aa PE |
336 | int (*readlink) (struct dentry *, char __user *,int); |
337 | void * (*follow_link) (struct dentry *, struct nameidata *); | |
338 | void (*put_link) (struct dentry *, struct nameidata *, void *); | |
1da177e4 | 339 | void (*truncate) (struct inode *); |
5ea626aa PE |
340 | int (*permission) (struct inode *, int, struct nameidata *); |
341 | int (*setattr) (struct dentry *, struct iattr *); | |
342 | int (*getattr) (struct vfsmount *mnt, struct dentry *, struct kstat *); | |
343 | int (*setxattr) (struct dentry *, const char *,const void *,size_t,int); | |
344 | ssize_t (*getxattr) (struct dentry *, const char *, void *, size_t); | |
345 | ssize_t (*listxattr) (struct dentry *, char *, size_t); | |
346 | int (*removexattr) (struct dentry *, const char *); | |
1da177e4 LT |
347 | }; |
348 | ||
349 | Again, all methods are called without any locks being held, unless | |
350 | otherwise noted. | |
351 | ||
1da177e4 LT |
352 | create: called by the open(2) and creat(2) system calls. Only |
353 | required if you want to support regular files. The dentry you | |
354 | get should not have an inode (i.e. it should be a negative | |
355 | dentry). Here you will probably call d_instantiate() with the | |
356 | dentry and the newly created inode | |
357 | ||
358 | lookup: called when the VFS needs to look up an inode in a parent | |
359 | directory. The name to look for is found in the dentry. This | |
360 | method must call d_add() to insert the found inode into the | |
361 | dentry. The "i_count" field in the inode structure should be | |
362 | incremented. If the named inode does not exist a NULL inode | |
363 | should be inserted into the dentry (this is called a negative | |
364 | dentry). Returning an error code from this routine must only | |
365 | be done on a real error, otherwise creating inodes with system | |
366 | calls like create(2), mknod(2), mkdir(2) and so on will fail. | |
367 | If you wish to overload the dentry methods then you should | |
368 | initialise the "d_dop" field in the dentry; this is a pointer | |
369 | to a struct "dentry_operations". | |
370 | This method is called with the directory inode semaphore held | |
371 | ||
372 | link: called by the link(2) system call. Only required if you want | |
373 | to support hard links. You will probably need to call | |
374 | d_instantiate() just as you would in the create() method | |
375 | ||
376 | unlink: called by the unlink(2) system call. Only required if you | |
377 | want to support deleting inodes | |
378 | ||
379 | symlink: called by the symlink(2) system call. Only required if you | |
380 | want to support symlinks. You will probably need to call | |
381 | d_instantiate() just as you would in the create() method | |
382 | ||
383 | mkdir: called by the mkdir(2) system call. Only required if you want | |
384 | to support creating subdirectories. You will probably need to | |
385 | call d_instantiate() just as you would in the create() method | |
386 | ||
387 | rmdir: called by the rmdir(2) system call. Only required if you want | |
388 | to support deleting subdirectories | |
389 | ||
390 | mknod: called by the mknod(2) system call to create a device (char, | |
391 | block) inode or a named pipe (FIFO) or socket. Only required | |
392 | if you want to support creating these types of inodes. You | |
393 | will probably need to call d_instantiate() just as you would | |
394 | in the create() method | |
395 | ||
cc7d1f8f PE |
396 | rename: called by the rename(2) system call to rename the object to |
397 | have the parent and name given by the second inode and dentry. | |
398 | ||
1da177e4 LT |
399 | readlink: called by the readlink(2) system call. Only required if |
400 | you want to support reading symbolic links | |
401 | ||
402 | follow_link: called by the VFS to follow a symbolic link to the | |
5ea626aa | 403 | inode it points to. Only required if you want to support |
cc7d1f8f | 404 | symbolic links. This method returns a void pointer cookie |
5ea626aa PE |
405 | that is passed to put_link(). |
406 | ||
407 | put_link: called by the VFS to release resources allocated by | |
cc7d1f8f PE |
408 | follow_link(). The cookie returned by follow_link() is passed |
409 | to to this method as the last parameter. It is used by | |
410 | filesystems such as NFS where page cache is not stable | |
411 | (i.e. page that was installed when the symbolic link walk | |
412 | started might not be in the page cache at the end of the | |
413 | walk). | |
414 | ||
415 | truncate: called by the VFS to change the size of a file. The | |
416 | i_size field of the inode is set to the desired size by the | |
417 | VFS before this method is called. This method is called by | |
418 | the truncate(2) system call and related functionality. | |
5ea626aa PE |
419 | |
420 | permission: called by the VFS to check for access rights on a POSIX-like | |
421 | filesystem. | |
422 | ||
cc7d1f8f PE |
423 | setattr: called by the VFS to set attributes for a file. This method |
424 | is called by chmod(2) and related system calls. | |
5ea626aa | 425 | |
cc7d1f8f PE |
426 | getattr: called by the VFS to get attributes of a file. This method |
427 | is called by stat(2) and related system calls. | |
5ea626aa PE |
428 | |
429 | setxattr: called by the VFS to set an extended attribute for a file. | |
cc7d1f8f PE |
430 | Extended attribute is a name:value pair associated with an |
431 | inode. This method is called by setxattr(2) system call. | |
432 | ||
433 | getxattr: called by the VFS to retrieve the value of an extended | |
434 | attribute name. This method is called by getxattr(2) function | |
435 | call. | |
436 | ||
437 | listxattr: called by the VFS to list all extended attributes for a | |
438 | given file. This method is called by listxattr(2) system call. | |
5ea626aa | 439 | |
cc7d1f8f PE |
440 | removexattr: called by the VFS to remove an extended attribute from |
441 | a file. This method is called by removexattr(2) system call. | |
5ea626aa | 442 | |
5ea626aa | 443 | |
cc7d1f8f PE |
444 | The Address Space Object |
445 | ======================== | |
446 | ||
447 | The address space object is used to identify pages in the page cache. | |
5ea626aa PE |
448 | |
449 | ||
450 | struct address_space_operations | |
cc7d1f8f | 451 | ------------------------------- |
5ea626aa PE |
452 | |
453 | This describes how the VFS can manipulate mapping of a file to page cache in | |
454 | your filesystem. As of kernel 2.6.13, the following members are defined: | |
455 | ||
456 | struct address_space_operations { | |
457 | int (*writepage)(struct page *page, struct writeback_control *wbc); | |
458 | int (*readpage)(struct file *, struct page *); | |
459 | int (*sync_page)(struct page *); | |
460 | int (*writepages)(struct address_space *, struct writeback_control *); | |
461 | int (*set_page_dirty)(struct page *page); | |
462 | int (*readpages)(struct file *filp, struct address_space *mapping, | |
463 | struct list_head *pages, unsigned nr_pages); | |
464 | int (*prepare_write)(struct file *, struct page *, unsigned, unsigned); | |
465 | int (*commit_write)(struct file *, struct page *, unsigned, unsigned); | |
466 | sector_t (*bmap)(struct address_space *, sector_t); | |
467 | int (*invalidatepage) (struct page *, unsigned long); | |
468 | int (*releasepage) (struct page *, int); | |
469 | ssize_t (*direct_IO)(int, struct kiocb *, const struct iovec *iov, | |
470 | loff_t offset, unsigned long nr_segs); | |
471 | struct page* (*get_xip_page)(struct address_space *, sector_t, | |
472 | int); | |
473 | }; | |
474 | ||
475 | writepage: called by the VM write a dirty page to backing store. | |
476 | ||
477 | readpage: called by the VM to read a page from backing store. | |
478 | ||
479 | sync_page: called by the VM to notify the backing store to perform all | |
480 | queued I/O operations for a page. I/O operations for other pages | |
481 | associated with this address_space object may also be performed. | |
482 | ||
483 | writepages: called by the VM to write out pages associated with the | |
484 | address_space object. | |
485 | ||
486 | set_page_dirty: called by the VM to set a page dirty. | |
487 | ||
488 | readpages: called by the VM to read pages associated with the address_space | |
489 | object. | |
1da177e4 | 490 | |
5ea626aa PE |
491 | prepare_write: called by the generic write path in VM to set up a write |
492 | request for a page. | |
1da177e4 | 493 | |
5ea626aa PE |
494 | commit_write: called by the generic write path in VM to write page to |
495 | its backing store. | |
496 | ||
497 | bmap: called by the VFS to map a logical block offset within object to | |
498 | physical block number. This method is use by for the legacy FIBMAP | |
499 | ioctl. Other uses are discouraged. | |
500 | ||
501 | invalidatepage: called by the VM on truncate to disassociate a page from its | |
502 | address_space mapping. | |
503 | ||
504 | releasepage: called by the VFS to release filesystem specific metadata from | |
505 | a page. | |
506 | ||
507 | direct_IO: called by the VM for direct I/O writes and reads. | |
508 | ||
509 | get_xip_page: called by the VM to translate a block number to a page. | |
510 | The page is valid until the corresponding filesystem is unmounted. | |
511 | Filesystems that want to use execute-in-place (XIP) need to implement | |
512 | it. An example implementation can be found in fs/ext2/xip.c. | |
513 | ||
514 | ||
cc7d1f8f PE |
515 | The File Object |
516 | =============== | |
517 | ||
518 | A file object represents a file opened by a process. | |
519 | ||
520 | ||
5ea626aa | 521 | struct file_operations |
cc7d1f8f | 522 | ---------------------- |
1da177e4 LT |
523 | |
524 | This describes how the VFS can manipulate an open file. As of kernel | |
5ea626aa | 525 | 2.6.13, the following members are defined: |
1da177e4 LT |
526 | |
527 | struct file_operations { | |
528 | loff_t (*llseek) (struct file *, loff_t, int); | |
5ea626aa PE |
529 | ssize_t (*read) (struct file *, char __user *, size_t, loff_t *); |
530 | ssize_t (*aio_read) (struct kiocb *, char __user *, size_t, loff_t); | |
531 | ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *); | |
532 | ssize_t (*aio_write) (struct kiocb *, const char __user *, size_t, loff_t); | |
1da177e4 LT |
533 | int (*readdir) (struct file *, void *, filldir_t); |
534 | unsigned int (*poll) (struct file *, struct poll_table_struct *); | |
535 | int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long); | |
5ea626aa PE |
536 | long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); |
537 | long (*compat_ioctl) (struct file *, unsigned int, unsigned long); | |
1da177e4 LT |
538 | int (*mmap) (struct file *, struct vm_area_struct *); |
539 | int (*open) (struct inode *, struct file *); | |
5ea626aa | 540 | int (*flush) (struct file *); |
1da177e4 | 541 | int (*release) (struct inode *, struct file *); |
5ea626aa PE |
542 | int (*fsync) (struct file *, struct dentry *, int datasync); |
543 | int (*aio_fsync) (struct kiocb *, int datasync); | |
544 | int (*fasync) (int, struct file *, int); | |
1da177e4 | 545 | int (*lock) (struct file *, int, struct file_lock *); |
5ea626aa PE |
546 | ssize_t (*readv) (struct file *, const struct iovec *, unsigned long, loff_t *); |
547 | ssize_t (*writev) (struct file *, const struct iovec *, unsigned long, loff_t *); | |
548 | ssize_t (*sendfile) (struct file *, loff_t *, size_t, read_actor_t, void *); | |
549 | ssize_t (*sendpage) (struct file *, struct page *, int, size_t, loff_t *, int); | |
550 | unsigned long (*get_unmapped_area)(struct file *, unsigned long, unsigned long, unsigned long, unsigned long); | |
551 | int (*check_flags)(int); | |
552 | int (*dir_notify)(struct file *filp, unsigned long arg); | |
553 | int (*flock) (struct file *, int, struct file_lock *); | |
1da177e4 LT |
554 | }; |
555 | ||
556 | Again, all methods are called without any locks being held, unless | |
557 | otherwise noted. | |
558 | ||
559 | llseek: called when the VFS needs to move the file position index | |
560 | ||
561 | read: called by read(2) and related system calls | |
562 | ||
5ea626aa PE |
563 | aio_read: called by io_submit(2) and other asynchronous I/O operations |
564 | ||
1da177e4 LT |
565 | write: called by write(2) and related system calls |
566 | ||
5ea626aa PE |
567 | aio_write: called by io_submit(2) and other asynchronous I/O operations |
568 | ||
1da177e4 LT |
569 | readdir: called when the VFS needs to read the directory contents |
570 | ||
571 | poll: called by the VFS when a process wants to check if there is | |
572 | activity on this file and (optionally) go to sleep until there | |
573 | is activity. Called by the select(2) and poll(2) system calls | |
574 | ||
575 | ioctl: called by the ioctl(2) system call | |
576 | ||
5ea626aa PE |
577 | unlocked_ioctl: called by the ioctl(2) system call. Filesystems that do not |
578 | require the BKL should use this method instead of the ioctl() above. | |
579 | ||
580 | compat_ioctl: called by the ioctl(2) system call when 32 bit system calls | |
581 | are used on 64 bit kernels. | |
582 | ||
1da177e4 LT |
583 | mmap: called by the mmap(2) system call |
584 | ||
585 | open: called by the VFS when an inode should be opened. When the VFS | |
5ea626aa PE |
586 | opens a file, it creates a new "struct file". It then calls the |
587 | open method for the newly allocated file structure. You might | |
588 | think that the open method really belongs in | |
589 | "struct inode_operations", and you may be right. I think it's | |
590 | done the way it is because it makes filesystems simpler to | |
591 | implement. The open() method is a good place to initialize the | |
592 | "private_data" member in the file structure if you want to point | |
593 | to a device structure | |
594 | ||
595 | flush: called by the close(2) system call to flush a file | |
1da177e4 LT |
596 | |
597 | release: called when the last reference to an open file is closed | |
598 | ||
599 | fsync: called by the fsync(2) system call | |
600 | ||
601 | fasync: called by the fcntl(2) system call when asynchronous | |
602 | (non-blocking) mode is enabled for a file | |
603 | ||
5ea626aa PE |
604 | lock: called by the fcntl(2) system call for F_GETLK, F_SETLK, and F_SETLKW |
605 | commands | |
606 | ||
607 | readv: called by the readv(2) system call | |
608 | ||
609 | writev: called by the writev(2) system call | |
610 | ||
611 | sendfile: called by the sendfile(2) system call | |
612 | ||
613 | get_unmapped_area: called by the mmap(2) system call | |
614 | ||
615 | check_flags: called by the fcntl(2) system call for F_SETFL command | |
616 | ||
617 | dir_notify: called by the fcntl(2) system call for F_NOTIFY command | |
618 | ||
619 | flock: called by the flock(2) system call | |
620 | ||
1da177e4 LT |
621 | Note that the file operations are implemented by the specific |
622 | filesystem in which the inode resides. When opening a device node | |
623 | (character or block special) most filesystems will call special | |
624 | support routines in the VFS which will locate the required device | |
625 | driver information. These support routines replace the filesystem file | |
626 | operations with those for the device driver, and then proceed to call | |
627 | the new open() method for the file. This is how opening a device file | |
628 | in the filesystem eventually ends up calling the device driver open() | |
5ea626aa | 629 | method. |
1da177e4 LT |
630 | |
631 | ||
5ea626aa PE |
632 | Directory Entry Cache (dcache) |
633 | ============================== | |
634 | ||
1da177e4 LT |
635 | |
636 | struct dentry_operations | |
5ea626aa | 637 | ------------------------ |
1da177e4 LT |
638 | |
639 | This describes how a filesystem can overload the standard dentry | |
640 | operations. Dentries and the dcache are the domain of the VFS and the | |
641 | individual filesystem implementations. Device drivers have no business | |
642 | here. These methods may be set to NULL, as they are either optional or | |
5ea626aa | 643 | the VFS uses a default. As of kernel 2.6.13, the following members are |
1da177e4 LT |
644 | defined: |
645 | ||
646 | struct dentry_operations { | |
5ea626aa | 647 | int (*d_revalidate)(struct dentry *, struct nameidata *); |
1da177e4 LT |
648 | int (*d_hash) (struct dentry *, struct qstr *); |
649 | int (*d_compare) (struct dentry *, struct qstr *, struct qstr *); | |
5ea626aa | 650 | int (*d_delete)(struct dentry *); |
1da177e4 LT |
651 | void (*d_release)(struct dentry *); |
652 | void (*d_iput)(struct dentry *, struct inode *); | |
653 | }; | |
654 | ||
655 | d_revalidate: called when the VFS needs to revalidate a dentry. This | |
656 | is called whenever a name look-up finds a dentry in the | |
657 | dcache. Most filesystems leave this as NULL, because all their | |
658 | dentries in the dcache are valid | |
659 | ||
660 | d_hash: called when the VFS adds a dentry to the hash table | |
661 | ||
662 | d_compare: called when a dentry should be compared with another | |
663 | ||
664 | d_delete: called when the last reference to a dentry is | |
665 | deleted. This means no-one is using the dentry, however it is | |
666 | still valid and in the dcache | |
667 | ||
668 | d_release: called when a dentry is really deallocated | |
669 | ||
670 | d_iput: called when a dentry loses its inode (just prior to its | |
671 | being deallocated). The default when this is NULL is that the | |
672 | VFS calls iput(). If you define this method, you must call | |
673 | iput() yourself | |
674 | ||
675 | Each dentry has a pointer to its parent dentry, as well as a hash list | |
676 | of child dentries. Child dentries are basically like files in a | |
677 | directory. | |
678 | ||
5ea626aa | 679 | |
cc7d1f8f | 680 | Directory Entry Cache API |
1da177e4 LT |
681 | -------------------------- |
682 | ||
683 | There are a number of functions defined which permit a filesystem to | |
684 | manipulate dentries: | |
685 | ||
686 | dget: open a new handle for an existing dentry (this just increments | |
687 | the usage count) | |
688 | ||
689 | dput: close a handle for a dentry (decrements the usage count). If | |
690 | the usage count drops to 0, the "d_delete" method is called | |
691 | and the dentry is placed on the unused list if the dentry is | |
692 | still in its parents hash list. Putting the dentry on the | |
693 | unused list just means that if the system needs some RAM, it | |
694 | goes through the unused list of dentries and deallocates them. | |
695 | If the dentry has already been unhashed and the usage count | |
696 | drops to 0, in this case the dentry is deallocated after the | |
697 | "d_delete" method is called | |
698 | ||
699 | d_drop: this unhashes a dentry from its parents hash list. A | |
5ea626aa | 700 | subsequent call to dput() will deallocate the dentry if its |
1da177e4 LT |
701 | usage count drops to 0 |
702 | ||
703 | d_delete: delete a dentry. If there are no other open references to | |
704 | the dentry then the dentry is turned into a negative dentry | |
705 | (the d_iput() method is called). If there are other | |
706 | references, then d_drop() is called instead | |
707 | ||
708 | d_add: add a dentry to its parents hash list and then calls | |
709 | d_instantiate() | |
710 | ||
711 | d_instantiate: add a dentry to the alias hash list for the inode and | |
712 | updates the "d_inode" member. The "i_count" member in the | |
713 | inode structure should be set/incremented. If the inode | |
714 | pointer is NULL, the dentry is called a "negative | |
715 | dentry". This function is commonly called when an inode is | |
716 | created for an existing negative dentry | |
717 | ||
718 | d_lookup: look up a dentry given its parent and path name component | |
719 | It looks up the child of that given name from the dcache | |
720 | hash table. If it is found, the reference count is incremented | |
721 | and the dentry is returned. The caller must use d_put() | |
722 | to free the dentry when it finishes using it. | |
723 | ||
724 | ||
725 | RCU-based dcache locking model | |
726 | ------------------------------ | |
727 | ||
728 | On many workloads, the most common operation on dcache is | |
729 | to look up a dentry, given a parent dentry and the name | |
730 | of the child. Typically, for every open(), stat() etc., | |
731 | the dentry corresponding to the pathname will be looked | |
732 | up by walking the tree starting with the first component | |
733 | of the pathname and using that dentry along with the next | |
734 | component to look up the next level and so on. Since it | |
735 | is a frequent operation for workloads like multiuser | |
5ea626aa | 736 | environments and web servers, it is important to optimize |
1da177e4 LT |
737 | this path. |
738 | ||
739 | Prior to 2.5.10, dcache_lock was acquired in d_lookup and thus | |
740 | in every component during path look-up. Since 2.5.10 onwards, | |
5ea626aa | 741 | fast-walk algorithm changed this by holding the dcache_lock |
1da177e4 | 742 | at the beginning and walking as many cached path component |
5ea626aa | 743 | dentries as possible. This significantly decreases the number |
1da177e4 | 744 | of acquisition of dcache_lock. However it also increases the |
5ea626aa | 745 | lock hold time significantly and affects performance in large |
1da177e4 LT |
746 | SMP machines. Since 2.5.62 kernel, dcache has been using |
747 | a new locking model that uses RCU to make dcache look-up | |
748 | lock-free. | |
749 | ||
750 | The current dcache locking model is not very different from the existing | |
751 | dcache locking model. Prior to 2.5.62 kernel, dcache_lock | |
752 | protected the hash chain, d_child, d_alias, d_lru lists as well | |
753 | as d_inode and several other things like mount look-up. RCU-based | |
754 | changes affect only the way the hash chain is protected. For everything | |
755 | else the dcache_lock must be taken for both traversing as well as | |
5ea626aa | 756 | updating. The hash chain updates too take the dcache_lock. |
1da177e4 LT |
757 | The significant change is the way d_lookup traverses the hash chain, |
758 | it doesn't acquire the dcache_lock for this and rely on RCU to | |
759 | ensure that the dentry has not been *freed*. | |
760 | ||
761 | ||
762 | Dcache locking details | |
763 | ---------------------- | |
5ea626aa | 764 | |
1da177e4 LT |
765 | For many multi-user workloads, open() and stat() on files are |
766 | very frequently occurring operations. Both involve walking | |
767 | of path names to find the dentry corresponding to the | |
768 | concerned file. In 2.4 kernel, dcache_lock was held | |
769 | during look-up of each path component. Contention and | |
5ea626aa | 770 | cache-line bouncing of this global lock caused significant |
1da177e4 | 771 | scalability problems. With the introduction of RCU |
5ea626aa | 772 | in Linux kernel, this was worked around by making |
1da177e4 LT |
773 | the look-up of path components during path walking lock-free. |
774 | ||
775 | ||
776 | Safe lock-free look-up of dcache hash table | |
777 | =========================================== | |
778 | ||
779 | Dcache is a complex data structure with the hash table entries | |
780 | also linked together in other lists. In 2.4 kernel, dcache_lock | |
781 | protected all the lists. We applied RCU only on hash chain | |
782 | walking. The rest of the lists are still protected by dcache_lock. | |
783 | Some of the important changes are : | |
784 | ||
785 | 1. The deletion from hash chain is done using hlist_del_rcu() macro which | |
786 | doesn't initialize next pointer of the deleted dentry and this | |
787 | allows us to walk safely lock-free while a deletion is happening. | |
788 | ||
789 | 2. Insertion of a dentry into the hash table is done using | |
790 | hlist_add_head_rcu() which take care of ordering the writes - | |
791 | the writes to the dentry must be visible before the dentry | |
5ea626aa | 792 | is inserted. This works in conjunction with hlist_for_each_rcu() |
1da177e4 LT |
793 | while walking the hash chain. The only requirement is that |
794 | all initialization to the dentry must be done before hlist_add_head_rcu() | |
795 | since we don't have dcache_lock protection while traversing | |
796 | the hash chain. This isn't different from the existing code. | |
797 | ||
798 | 3. The dentry looked up without holding dcache_lock by cannot be | |
799 | returned for walking if it is unhashed. It then may have a NULL | |
800 | d_inode or other bogosity since RCU doesn't protect the other | |
801 | fields in the dentry. We therefore use a flag DCACHE_UNHASHED to | |
802 | indicate unhashed dentries and use this in conjunction with a | |
803 | per-dentry lock (d_lock). Once looked up without the dcache_lock, | |
804 | we acquire the per-dentry lock (d_lock) and check if the | |
805 | dentry is unhashed. If so, the look-up is failed. If not, the | |
806 | reference count of the dentry is increased and the dentry is returned. | |
807 | ||
808 | 4. Once a dentry is looked up, it must be ensured during the path | |
809 | walk for that component it doesn't go away. In pre-2.5.10 code, | |
810 | this was done holding a reference to the dentry. dcache_rcu does | |
811 | the same. In some sense, dcache_rcu path walking looks like | |
812 | the pre-2.5.10 version. | |
813 | ||
5ea626aa | 814 | 5. All dentry hash chain updates must take the dcache_lock as well as |
1da177e4 LT |
815 | the per-dentry lock in that order. dput() does this to ensure |
816 | that a dentry that has just been looked up in another CPU | |
817 | doesn't get deleted before dget() can be done on it. | |
818 | ||
819 | 6. There are several ways to do reference counting of RCU protected | |
820 | objects. One such example is in ipv4 route cache where | |
821 | deferred freeing (using call_rcu()) is done as soon as | |
822 | the reference count goes to zero. This cannot be done in | |
823 | the case of dentries because tearing down of dentries | |
824 | require blocking (dentry_iput()) which isn't supported from | |
825 | RCU callbacks. Instead, tearing down of dentries happen | |
826 | synchronously in dput(), but actual freeing happens later | |
827 | when RCU grace period is over. This allows safe lock-free | |
828 | walking of the hash chains, but a matched dentry may have | |
829 | been partially torn down. The checking of DCACHE_UNHASHED | |
830 | flag with d_lock held detects such dentries and prevents | |
831 | them from being returned from look-up. | |
832 | ||
833 | ||
834 | Maintaining POSIX rename semantics | |
835 | ================================== | |
836 | ||
837 | Since look-up of dentries is lock-free, it can race against | |
838 | a concurrent rename operation. For example, during rename | |
839 | of file A to B, look-up of either A or B must succeed. | |
840 | So, if look-up of B happens after A has been removed from the | |
841 | hash chain but not added to the new hash chain, it may fail. | |
842 | Also, a comparison while the name is being written concurrently | |
843 | by a rename may result in false positive matches violating | |
844 | rename semantics. Issues related to race with rename are | |
845 | handled as described below : | |
846 | ||
847 | 1. Look-up can be done in two ways - d_lookup() which is safe | |
848 | from simultaneous renames and __d_lookup() which is not. | |
849 | If __d_lookup() fails, it must be followed up by a d_lookup() | |
850 | to correctly determine whether a dentry is in the hash table | |
851 | or not. d_lookup() protects look-ups using a sequence | |
852 | lock (rename_lock). | |
853 | ||
854 | 2. The name associated with a dentry (d_name) may be changed if | |
855 | a rename is allowed to happen simultaneously. To avoid memcmp() | |
856 | in __d_lookup() go out of bounds due to a rename and false | |
857 | positive comparison, the name comparison is done while holding the | |
858 | per-dentry lock. This prevents concurrent renames during this | |
859 | operation. | |
860 | ||
861 | 3. Hash table walking during look-up may move to a different bucket as | |
862 | the current dentry is moved to a different bucket due to rename. | |
863 | But we use hlists in dcache hash table and they are null-terminated. | |
864 | So, even if a dentry moves to a different bucket, hash chain | |
865 | walk will terminate. [with a list_head list, it may not since | |
866 | termination is when the list_head in the original bucket is reached]. | |
867 | Since we redo the d_parent check and compare name while holding | |
868 | d_lock, lock-free look-up will not race against d_move(). | |
869 | ||
5ea626aa | 870 | 4. There can be a theoretical race when a dentry keeps coming back |
1da177e4 LT |
871 | to original bucket due to double moves. Due to this look-up may |
872 | consider that it has never moved and can end up in a infinite loop. | |
5ea626aa | 873 | But this is not any worse that theoretical livelocks we already |
1da177e4 LT |
874 | have in the kernel. |
875 | ||
876 | ||
877 | Important guidelines for filesystem developers related to dcache_rcu | |
878 | ==================================================================== | |
879 | ||
880 | 1. Existing dcache interfaces (pre-2.5.62) exported to filesystem | |
881 | don't change. Only dcache internal implementation changes. However | |
882 | filesystems *must not* delete from the dentry hash chains directly | |
883 | using the list macros like allowed earlier. They must use dcache | |
884 | APIs like d_drop() or __d_drop() depending on the situation. | |
885 | ||
886 | 2. d_flags is now protected by a per-dentry lock (d_lock). All | |
887 | access to d_flags must be protected by it. | |
888 | ||
889 | 3. For a hashed dentry, checking of d_count needs to be protected | |
890 | by d_lock. | |
891 | ||
892 | ||
893 | Papers and other documentation on dcache locking | |
894 | ================================================ | |
895 | ||
896 | 1. Scaling dcache with RCU (http://linuxjournal.com/article.php?sid=7124). | |
897 | ||
898 | 2. http://lse.sourceforge.net/locking/dcache/dcache.html | |
cc7d1f8f PE |
899 | |
900 | ||
901 | Resources | |
902 | ========= | |
903 | ||
904 | (Note some of these resources are not up-to-date with the latest kernel | |
905 | version.) | |
906 | ||
907 | Creating Linux virtual filesystems. 2002 | |
908 | <http://lwn.net/Articles/13325/> | |
909 | ||
910 | The Linux Virtual File-system Layer by Neil Brown. 1999 | |
911 | <http://www.cse.unsw.edu.au/~neilb/oss/linux-commentary/vfs.html> | |
912 | ||
913 | A tour of the Linux VFS by Michael K. Johnson. 1996 | |
914 | <http://www.tldp.org/LDP/khg/HyperNews/get/fs/vfstour.html> | |
915 | ||
916 | A small trail through the Linux kernel by Andries Brouwer. 2001 | |
917 | <http://www.win.tue.nl/~aeb/linux/vfs/trail.html> |