]>
Commit | Line | Data |
---|---|---|
1 | Erasure Coded pool | |
2 | ================== | |
3 | ||
4 | Purpose | |
5 | ------- | |
6 | ||
7 | Erasure-coded pools require less storage space compared to replicated | |
8 | pools. The erasure-coding support has higher computational requirements and | |
9 | only supports a subset of the operations allowed on an object (for instance, | |
10 | partial write is not supported). | |
11 | ||
12 | Use cases | |
13 | --------- | |
14 | ||
15 | Cold storage | |
16 | ~~~~~~~~~~~~ | |
17 | ||
18 | An erasure-coded pool is created to store a large number of 1GB | |
19 | objects (imaging, genomics, etc.) and 10% of them are read per | |
20 | month. New objects are added every day and the objects are not | |
21 | modified after being written. On average there is one write for 10,000 | |
22 | reads. | |
23 | ||
24 | A replicated pool is created and set as a cache tier for the | |
25 | erasure coded pool. An agent demotes objects (i.e. moves them from the | |
26 | replicated pool to the erasure-coded pool) if they have not been | |
27 | accessed in a week. | |
28 | ||
29 | The erasure-coded pool CRUSH rule targets hardware designed for | |
30 | cold storage with high latency and slow access time. The replicated | |
31 | pool CRUSH rule targets faster hardware to provide better response | |
32 | times. | |
33 | ||
34 | Cheap multidatacenter storage | |
35 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
36 | ||
37 | Ten datacenters are connected with dedicated network links. Each | |
38 | datacenter contains the same amount of storage with no power-supply | |
39 | backup and no air-cooling system. | |
40 | ||
41 | An erasure-coded pool is created with a CRUSH rule that will | |
42 | ensure no data loss if at most three datacenters fail | |
43 | simultaneously. The overhead is 50% with erasure code configured to | |
44 | split data in six (k=6) and create three coding chunks (m=3). With | |
45 | replication the overhead would be 400% (four replicas). | |
46 | ||
47 | Interface | |
48 | --------- | |
49 | ||
50 | Set up an erasure-coded pool:: | |
51 | ||
52 | $ ceph osd pool create ecpool 12 12 erasure | |
53 | ||
54 | Set up an erasure-coded pool and the associated CRUSH rule ``ecrule``:: | |
55 | ||
56 | $ ceph osd crush rule create-erasure ecrule | |
57 | $ ceph osd pool create ecpool 12 12 erasure \ | |
58 | default ecrule | |
59 | ||
60 | Set the CRUSH failure domain to osd (instead of host, which is the default):: | |
61 | ||
62 | $ ceph osd erasure-code-profile set myprofile \ | |
63 | crush-failure-domain=osd | |
64 | $ ceph osd erasure-code-profile get myprofile | |
65 | k=2 | |
66 | m=1 | |
67 | plugin=jerasure | |
68 | technique=reed_sol_van | |
69 | crush-failure-domain=osd | |
70 | $ ceph osd pool create ecpool 12 12 erasure myprofile | |
71 | ||
72 | Control the parameters of the erasure code plugin:: | |
73 | ||
74 | $ ceph osd erasure-code-profile set myprofile \ | |
75 | k=3 m=1 | |
76 | $ ceph osd erasure-code-profile get myprofile | |
77 | k=3 | |
78 | m=1 | |
79 | plugin=jerasure | |
80 | technique=reed_sol_van | |
81 | $ ceph osd pool create ecpool 12 12 erasure \ | |
82 | myprofile | |
83 | ||
84 | Choose an alternate erasure code plugin:: | |
85 | ||
86 | $ ceph osd erasure-code-profile set myprofile \ | |
87 | plugin=example technique=xor | |
88 | $ ceph osd erasure-code-profile get myprofile | |
89 | k=2 | |
90 | m=1 | |
91 | plugin=example | |
92 | technique=xor | |
93 | $ ceph osd pool create ecpool 12 12 erasure \ | |
94 | myprofile | |
95 | ||
96 | Display the default erasure code profile:: | |
97 | ||
98 | $ ceph osd erasure-code-profile ls | |
99 | default | |
100 | $ ceph osd erasure-code-profile get default | |
101 | k=2 | |
102 | m=1 | |
103 | plugin=jerasure | |
104 | technique=reed_sol_van | |
105 | ||
106 | Create a profile to set the data to be distributed on six OSDs (k+m=6) and sustain the loss of three OSDs (m=3) without losing data:: | |
107 | ||
108 | $ ceph osd erasure-code-profile set myprofile k=3 m=3 | |
109 | $ ceph osd erasure-code-profile get myprofile | |
110 | k=3 | |
111 | m=3 | |
112 | plugin=jerasure | |
113 | technique=reed_sol_van | |
114 | $ ceph osd erasure-code-profile ls | |
115 | default | |
116 | myprofile | |
117 | ||
118 | Remove a profile that is no longer in use (otherwise it will fail with EBUSY):: | |
119 | ||
120 | $ ceph osd erasure-code-profile ls | |
121 | default | |
122 | myprofile | |
123 | $ ceph osd erasure-code-profile rm myprofile | |
124 | $ ceph osd erasure-code-profile ls | |
125 | default | |
126 | ||
127 | Set the rule to ssd (instead of default):: | |
128 | ||
129 | $ ceph osd erasure-code-profile set myprofile \ | |
130 | crush-root=ssd | |
131 | $ ceph osd erasure-code-profile get myprofile | |
132 | k=2 | |
133 | m=1 | |
134 | plugin=jerasure | |
135 | technique=reed_sol_van | |
136 | crush-root=ssd | |
137 |