]>
Commit | Line | Data |
---|---|---|
1 | ||
2 | LOCK STATISTICS | |
3 | ||
4 | - WHAT | |
5 | ||
6 | As the name suggests, it provides statistics on locks. | |
7 | ||
8 | - WHY | |
9 | ||
10 | Because things like lock contention can severely impact performance. | |
11 | ||
12 | - HOW | |
13 | ||
14 | Lockdep already has hooks in the lock functions and maps lock instances to | |
15 | lock classes. We build on that. The graph below shows the relation between | |
16 | the lock functions and the various hooks therein. | |
17 | ||
18 | __acquire | |
19 | | | |
20 | lock _____ | |
21 | | \ | |
22 | | __contended | |
23 | | | | |
24 | | <wait> | |
25 | | _______/ | |
26 | |/ | |
27 | | | |
28 | __acquired | |
29 | | | |
30 | . | |
31 | <hold> | |
32 | . | |
33 | | | |
34 | __release | |
35 | | | |
36 | unlock | |
37 | ||
38 | lock, unlock - the regular lock functions | |
39 | __* - the hooks | |
40 | <> - states | |
41 | ||
42 | With these hooks we provide the following statistics: | |
43 | ||
44 | con-bounces - number of lock contention that involved x-cpu data | |
45 | contentions - number of lock acquisitions that had to wait | |
46 | wait time min - shortest (non-0) time we ever had to wait for a lock | |
47 | max - longest time we ever had to wait for a lock | |
48 | total - total time we spend waiting on this lock | |
49 | acq-bounces - number of lock acquisitions that involved x-cpu data | |
50 | acquisitions - number of times we took the lock | |
51 | hold time min - shortest (non-0) time we ever held the lock | |
52 | max - longest time we ever held the lock | |
53 | total - total time this lock was held | |
54 | ||
55 | From these number various other statistics can be derived, such as: | |
56 | ||
57 | hold time average = hold time total / acquisitions | |
58 | ||
59 | These numbers are gathered per lock class, per read/write state (when | |
60 | applicable). | |
61 | ||
62 | It also tracks 4 contention points per class. A contention point is a call site | |
63 | that had to wait on lock acquisition. | |
64 | ||
65 | - USAGE | |
66 | ||
67 | Look at the current lock statistics: | |
68 | ||
69 | ( line numbers not part of actual output, done for clarity in the explanation | |
70 | below ) | |
71 | ||
72 | # less /proc/lock_stat | |
73 | ||
74 | 01 lock_stat version 0.3 | |
75 | 02 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
76 | 03 class name con-bounces contentions waittime-min waittime-max waittime-total acq-bounces acquisitions holdtime-min holdtime-max holdtime-total | |
77 | 04 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |
78 | 05 | |
79 | 06 &mm->mmap_sem-W: 233 538 18446744073708 22924.27 607243.51 1342 45806 1.71 8595.89 1180582.34 | |
80 | 07 &mm->mmap_sem-R: 205 587 18446744073708 28403.36 731975.00 1940 412426 0.58 187825.45 6307502.88 | |
81 | 08 --------------- | |
82 | 09 &mm->mmap_sem 487 [<ffffffff8053491f>] do_page_fault+0x466/0x928 | |
83 | 10 &mm->mmap_sem 179 [<ffffffff802a6200>] sys_mprotect+0xcd/0x21d | |
84 | 11 &mm->mmap_sem 279 [<ffffffff80210a57>] sys_mmap+0x75/0xce | |
85 | 12 &mm->mmap_sem 76 [<ffffffff802a490b>] sys_munmap+0x32/0x59 | |
86 | 13 --------------- | |
87 | 14 &mm->mmap_sem 270 [<ffffffff80210a57>] sys_mmap+0x75/0xce | |
88 | 15 &mm->mmap_sem 431 [<ffffffff8053491f>] do_page_fault+0x466/0x928 | |
89 | 16 &mm->mmap_sem 138 [<ffffffff802a490b>] sys_munmap+0x32/0x59 | |
90 | 17 &mm->mmap_sem 145 [<ffffffff802a6200>] sys_mprotect+0xcd/0x21d | |
91 | 18 | |
92 | 19 ............................................................................................................................................................................................... | |
93 | 20 | |
94 | 21 dcache_lock: 621 623 0.52 118.26 1053.02 6745 91930 0.29 316.29 118423.41 | |
95 | 22 ----------- | |
96 | 23 dcache_lock 179 [<ffffffff80378274>] _atomic_dec_and_lock+0x34/0x54 | |
97 | 24 dcache_lock 113 [<ffffffff802cc17b>] d_alloc+0x19a/0x1eb | |
98 | 25 dcache_lock 99 [<ffffffff802ca0dc>] d_rehash+0x1b/0x44 | |
99 | 26 dcache_lock 104 [<ffffffff802cbca0>] d_instantiate+0x36/0x8a | |
100 | 27 ----------- | |
101 | 28 dcache_lock 192 [<ffffffff80378274>] _atomic_dec_and_lock+0x34/0x54 | |
102 | 29 dcache_lock 98 [<ffffffff802ca0dc>] d_rehash+0x1b/0x44 | |
103 | 30 dcache_lock 72 [<ffffffff802cc17b>] d_alloc+0x19a/0x1eb | |
104 | 31 dcache_lock 112 [<ffffffff802cbca0>] d_instantiate+0x36/0x8a | |
105 | ||
106 | This excerpt shows the first two lock class statistics. Line 01 shows the | |
107 | output version - each time the format changes this will be updated. Line 02-04 | |
108 | show the header with column descriptions. Lines 05-18 and 20-31 show the actual | |
109 | statistics. These statistics come in two parts; the actual stats separated by a | |
110 | short separator (line 08, 13) from the contention points. | |
111 | ||
112 | The first lock (05-18) is a read/write lock, and shows two lines above the | |
113 | short separator. The contention points don't match the column descriptors, | |
114 | they have two: contentions and [<IP>] symbol. The second set of contention | |
115 | points are the points we're contending with. | |
116 | ||
117 | The integer part of the time values is in us. | |
118 | ||
119 | View the top contending locks: | |
120 | ||
121 | # grep : /proc/lock_stat | head | |
122 | &inode->i_data.tree_lock-W: 15 21657 0.18 1093295.30 11547131054.85 58 10415 0.16 87.51 6387.60 | |
123 | &inode->i_data.tree_lock-R: 0 0 0.00 0.00 0.00 23302 231198 0.25 8.45 98023.38 | |
124 | dcache_lock: 1037 1161 0.38 45.32 774.51 6611 243371 0.15 306.48 77387.24 | |
125 | &inode->i_mutex: 161 286 18446744073709 62882.54 1244614.55 3653 20598 18446744073709 62318.60 1693822.74 | |
126 | &zone->lru_lock: 94 94 0.53 7.33 92.10 4366 32690 0.29 59.81 16350.06 | |
127 | &inode->i_data.i_mmap_lock: 79 79 0.40 3.77 53.03 11779 87755 0.28 116.93 29898.44 | |
128 | &q->__queue_lock: 48 50 0.52 31.62 86.31 774 13131 0.17 113.08 12277.52 | |
129 | &rq->rq_lock_key: 43 47 0.74 68.50 170.63 3706 33929 0.22 107.99 17460.62 | |
130 | &rq->rq_lock_key#2: 39 46 0.75 6.68 49.03 2979 32292 0.17 125.17 17137.63 | |
131 | tasklist_lock-W: 15 15 1.45 10.87 32.70 1201 7390 0.58 62.55 13648.47 | |
132 | ||
133 | Clear the statistics: | |
134 | ||
135 | # echo 0 > /proc/lock_stat |