]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | Erasure Coded pool |
2 | ================== | |
3 | ||
4 | Purpose | |
5 | ------- | |
6 | ||
7 | Erasure-coded pools require less storage space compared to replicated | |
8 | pools. The erasure-coding support has higher computational requirements and | |
9 | only supports a subset of the operations allowed on an object (for instance, | |
10 | partial write is not supported). | |
11 | ||
12 | Use cases | |
13 | --------- | |
14 | ||
15 | Cold storage | |
16 | ~~~~~~~~~~~~ | |
17 | ||
18 | An erasure-coded pool is created to store a large number of 1GB | |
19 | objects (imaging, genomics, etc.) and 10% of them are read per | |
20 | month. New objects are added every day and the objects are not | |
21 | modified after being written. On average there is one write for 10,000 | |
22 | reads. | |
23 | ||
24 | A replicated pool is created and set as a cache tier for the | |
25 | erasure coded pool. An agent demotes objects (i.e. moves them from the | |
26 | replicated pool to the erasure-coded pool) if they have not been | |
27 | accessed in a week. | |
28 | ||
b32b8144 | 29 | The erasure-coded pool CRUSH rule targets hardware designed for |
7c673cae | 30 | cold storage with high latency and slow access time. The replicated |
b32b8144 | 31 | pool CRUSH rule targets faster hardware to provide better response |
7c673cae FG |
32 | times. |
33 | ||
34 | Cheap multidatacenter storage | |
35 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
36 | ||
37 | Ten datacenters are connected with dedicated network links. Each | |
38 | datacenter contains the same amount of storage with no power-supply | |
39 | backup and no air-cooling system. | |
40 | ||
b32b8144 | 41 | An erasure-coded pool is created with a CRUSH rule that will |
7c673cae FG |
42 | ensure no data loss if at most three datacenters fail |
43 | simultaneously. The overhead is 50% with erasure code configured to | |
44 | split data in six (k=6) and create three coding chunks (m=3). With | |
45 | replication the overhead would be 400% (four replicas). | |
46 | ||
47 | Interface | |
48 | --------- | |
49 | ||
50 | Set up an erasure-coded pool:: | |
51 | ||
9f95a23c | 52 | $ ceph osd pool create ecpool erasure |
7c673cae | 53 | |
b32b8144 | 54 | Set up an erasure-coded pool and the associated CRUSH rule ``ecrule``:: |
7c673cae | 55 | |
b32b8144 | 56 | $ ceph osd crush rule create-erasure ecrule |
9f95a23c | 57 | $ ceph osd pool create ecpool erasure default ecrule |
7c673cae | 58 | |
b32b8144 | 59 | Set the CRUSH failure domain to osd (instead of host, which is the default):: |
7c673cae FG |
60 | |
61 | $ ceph osd erasure-code-profile set myprofile \ | |
224ce89b | 62 | crush-failure-domain=osd |
7c673cae FG |
63 | $ ceph osd erasure-code-profile get myprofile |
64 | k=2 | |
9f95a23c | 65 | m=2 |
7c673cae FG |
66 | plugin=jerasure |
67 | technique=reed_sol_van | |
224ce89b | 68 | crush-failure-domain=osd |
9f95a23c | 69 | $ ceph osd pool create ecpool erasure myprofile |
7c673cae FG |
70 | |
71 | Control the parameters of the erasure code plugin:: | |
72 | ||
73 | $ ceph osd erasure-code-profile set myprofile \ | |
9f95a23c | 74 | k=3 m=2 |
7c673cae FG |
75 | $ ceph osd erasure-code-profile get myprofile |
76 | k=3 | |
9f95a23c | 77 | m=2 |
7c673cae FG |
78 | plugin=jerasure |
79 | technique=reed_sol_van | |
9f95a23c | 80 | $ ceph osd pool create ecpool erasure myprofile |
7c673cae FG |
81 | |
82 | Choose an alternate erasure code plugin:: | |
83 | ||
84 | $ ceph osd erasure-code-profile set myprofile \ | |
85 | plugin=example technique=xor | |
86 | $ ceph osd erasure-code-profile get myprofile | |
87 | k=2 | |
9f95a23c | 88 | m=2 |
7c673cae FG |
89 | plugin=example |
90 | technique=xor | |
91 | $ ceph osd pool create ecpool 12 12 erasure \ | |
92 | myprofile | |
93 | ||
94 | Display the default erasure code profile:: | |
95 | ||
96 | $ ceph osd erasure-code-profile ls | |
97 | default | |
98 | $ ceph osd erasure-code-profile get default | |
99 | k=2 | |
9f95a23c | 100 | m=2 |
7c673cae FG |
101 | plugin=jerasure |
102 | technique=reed_sol_van | |
103 | ||
104 | Create a profile to set the data to be distributed on six OSDs (k+m=6) and sustain the loss of three OSDs (m=3) without losing data:: | |
105 | ||
106 | $ ceph osd erasure-code-profile set myprofile k=3 m=3 | |
107 | $ ceph osd erasure-code-profile get myprofile | |
108 | k=3 | |
109 | m=3 | |
110 | plugin=jerasure | |
111 | technique=reed_sol_van | |
112 | $ ceph osd erasure-code-profile ls | |
113 | default | |
114 | myprofile | |
115 | ||
116 | Remove a profile that is no longer in use (otherwise it will fail with EBUSY):: | |
117 | ||
118 | $ ceph osd erasure-code-profile ls | |
119 | default | |
120 | myprofile | |
121 | $ ceph osd erasure-code-profile rm myprofile | |
122 | $ ceph osd erasure-code-profile ls | |
123 | default | |
124 | ||
b32b8144 | 125 | Set the rule to ssd (instead of default):: |
7c673cae FG |
126 | |
127 | $ ceph osd erasure-code-profile set myprofile \ | |
224ce89b | 128 | crush-root=ssd |
7c673cae FG |
129 | $ ceph osd erasure-code-profile get myprofile |
130 | k=2 | |
9f95a23c | 131 | m=2 |
7c673cae FG |
132 | plugin=jerasure |
133 | technique=reed_sol_van | |
224ce89b | 134 | crush-root=ssd |
7c673cae | 135 |