]>
Commit | Line | Data |
---|---|---|
f2836352 JT |
1 | Guidance for writing policies |
2 | ============================= | |
3 | ||
4 | Try to keep transactionality out of it. The core is careful to | |
5 | avoid asking about anything that is migrating. This is a pain, but | |
6 | makes it easier to write the policies. | |
7 | ||
8 | Mappings are loaded into the policy at construction time. | |
9 | ||
10 | Every bio that is mapped by the target is referred to the policy. | |
11 | The policy can return a simple HIT or MISS or issue a migration. | |
12 | ||
13 | Currently there's no way for the policy to issue background work, | |
492d48db | 14 | e.g. to start writing back dirty blocks that are going to be evicted |
f2836352 JT |
15 | soon. |
16 | ||
17 | Because we map bios, rather than requests it's easy for the policy | |
18 | to get fooled by many small bios. For this reason the core target | |
19 | issues periodic ticks to the policy. It's suggested that the policy | |
20 | doesn't update states (eg, hit counts) for a block more than once | |
21 | for each tick. The core ticks by watching bios complete, and so | |
22 | trying to see when the io scheduler has let the ios run. | |
23 | ||
24 | ||
25 | Overview of supplied cache replacement policies | |
26 | =============================================== | |
27 | ||
bccab6a0 MS |
28 | multiqueue (mq) |
29 | --------------- | |
f2836352 | 30 | |
9ed84698 | 31 | This policy is now an alias for smq (see below). |
f2836352 | 32 | |
9ed84698 | 33 | The following tunables are accepted, but have no effect: |
01911c19 | 34 | |
78e03d69 JT |
35 | 'sequential_threshold <#nr_sequential_ios>' |
36 | 'random_threshold <#nr_random_ios>' | |
37 | 'read_promote_adjustment <value>' | |
38 | 'write_promote_adjustment <value>' | |
39 | 'discard_promote_adjustment <value>' | |
f2836352 | 40 | |
bccab6a0 MS |
41 | Stochastic multiqueue (smq) |
42 | --------------------------- | |
43 | ||
44 | This policy is the default. | |
45 | ||
46 | The stochastic multi-queue (smq) policy addresses some of the problems | |
47 | with the multiqueue (mq) policy. | |
48 | ||
49 | The smq policy (vs mq) offers the promise of less memory utilization, | |
50 | improved performance and increased adaptability in the face of changing | |
492d48db | 51 | workloads. smq also does not have any cumbersome tuning knobs. |
bccab6a0 MS |
52 | |
53 | Users may switch from "mq" to "smq" simply by appropriately reloading a | |
54 | DM table that is using the cache target. Doing so will cause all of the | |
55 | mq policy's hints to be dropped. Also, performance of the cache may | |
56 | degrade slightly until smq recalculates the origin device's hotspots | |
57 | that should be cached. | |
58 | ||
59 | Memory usage: | |
492d48db | 60 | The mq policy used a lot of memory; 88 bytes per cache block on a 64 |
bccab6a0 MS |
61 | bit machine. |
62 | ||
67721046 | 63 | smq uses 28bit indexes to implement its data structures rather than |
bccab6a0 | 64 | pointers. It avoids storing an explicit hit count for each block. It |
492d48db | 65 | has a 'hotspot' queue, rather than a pre-cache, which uses a quarter of |
bccab6a0 MS |
66 | the entries (each hotspot block covers a larger area than a single |
67 | cache block). | |
68 | ||
492d48db | 69 | All this means smq uses ~25bytes per cache block. Still a lot of |
bccab6a0 MS |
70 | memory, but a substantial improvement nontheless. |
71 | ||
72 | Level balancing: | |
492d48db MS |
73 | mq placed entries in different levels of the multiqueue structures |
74 | based on their hit count (~ln(hit count)). This meant the bottom | |
75 | levels generally had the most entries, and the top ones had very | |
76 | few. Having unbalanced levels like this reduced the efficacy of the | |
bccab6a0 MS |
77 | multiqueue. |
78 | ||
492d48db MS |
79 | smq does not maintain a hit count, instead it swaps hit entries with |
80 | the least recently used entry from the level above. The overall | |
bccab6a0 MS |
81 | ordering being a side effect of this stochastic process. With this |
82 | scheme we can decide how many entries occupy each multiqueue level, | |
83 | resulting in better promotion/demotion decisions. | |
84 | ||
85 | Adaptability: | |
492d48db | 86 | The mq policy maintained a hit count for each cache block. For a |
67721046 | 87 | different block to get promoted to the cache its hit count has to |
492d48db | 88 | exceed the lowest currently in the cache. This meant it could take a |
bccab6a0 | 89 | long time for the cache to adapt between varying IO patterns. |
bccab6a0 | 90 | |
492d48db | 91 | smq doesn't maintain hit counts, so a lot of this problem just goes |
bccab6a0 MS |
92 | away. In addition it tracks performance of the hotspot queue, which |
93 | is used to decide which blocks to promote. If the hotspot queue is | |
94 | performing badly then it starts moving entries more quickly between | |
95 | levels. This lets it adapt to new IO patterns very quickly. | |
96 | ||
97 | Performance: | |
492d48db | 98 | Testing smq shows substantially better performance than mq. |
bccab6a0 | 99 | |
8735a813 HM |
100 | cleaner |
101 | ------- | |
102 | ||
103 | The cleaner writes back all dirty blocks in a cache to decommission it. | |
104 | ||
f2836352 JT |
105 | Examples |
106 | ======== | |
107 | ||
108 | The syntax for a table is: | |
109 | cache <metadata dev> <cache dev> <origin dev> <block size> | |
110 | <#feature_args> [<feature arg>]* | |
111 | <policy> <#policy_args> [<policy arg>]* | |
112 | ||
113 | The syntax to send a message using the dmsetup command is: | |
114 | dmsetup message <mapped device> 0 sequential_threshold 1024 | |
115 | dmsetup message <mapped device> 0 random_threshold 8 | |
116 | ||
117 | Using dmsetup: | |
118 | dmsetup create blah --table "0 268435456 cache /dev/sdb /dev/sdc \ | |
119 | /dev/sdd 512 0 mq 4 sequential_threshold 1024 random_threshold 8" | |
120 | creates a 128GB large mapped device named 'blah' with the | |
121 | sequential threshold set to 1024 and the random_threshold set to 8. |