]>
Commit | Line | Data |
---|---|---|
9f95a23c TL |
1 | Essentials (tl;dr) |
2 | ================== | |
3 | ||
4 | This chapter presents essential information that every Ceph developer needs | |
5 | to know. | |
6 | ||
7 | Leads | |
8 | ----- | |
9 | ||
1e59de90 TL |
10 | The Ceph project was created by Sage Weil and is led by the Ceph Leadership |
11 | Team (CLT). In addition, each major project component has its own lead. The | |
12 | following table shows all the leads and their nicks on `GitHub`_: | |
9f95a23c TL |
13 | |
14 | .. _github: https://github.com/ | |
15 | ||
16 | ========= ================ ============= | |
17 | Scope Lead GitHub nick | |
18 | ========= ================ ============= | |
19 | Ceph Sage Weil liewegas | |
20 | RADOS Neha Ojha neha-ojha | |
21 | RGW Yehuda Sadeh yehudasa | |
22 | RGW Matt Benjamin mattbenjamin | |
20effc67 | 23 | RBD Ilya Dryomov dis |
1e59de90 | 24 | CephFS Venky Shankar vshankar |
20effc67 | 25 | Dashboard Ernesto Puerta epuertat |
9f95a23c TL |
26 | MON Joao Luis jecluis |
27 | Build/Ops Ken Dreyer ktdreyer | |
f67539c2 | 28 | Docs Zac Dover zdover23 |
9f95a23c TL |
29 | ========= ================ ============= |
30 | ||
31 | The Ceph-specific acronyms in the table are explained in | |
32 | :doc:`/architecture`. | |
33 | ||
34 | History | |
35 | ------- | |
36 | ||
37 | See the `History chapter of the Wikipedia article`_. | |
38 | ||
39 | .. _`History chapter of the Wikipedia article`: https://en.wikipedia.org/wiki/Ceph_%28software%29#History | |
40 | ||
41 | Licensing | |
42 | --------- | |
43 | ||
44 | Ceph is free software. | |
45 | ||
46 | Unless stated otherwise, the Ceph source code is distributed under the | |
47 | terms of the LGPL2.1 or LGPL3.0. For full details, see the file | |
48 | `COPYING`_ in the top-level directory of the source-code tree. | |
49 | ||
50 | .. _`COPYING`: | |
51 | https://github.com/ceph/ceph/blob/master/COPYING | |
52 | ||
53 | Source code repositories | |
54 | ------------------------ | |
55 | ||
56 | The source code of Ceph lives on `GitHub`_ in a number of repositories below | |
57 | the `Ceph "organization"`_. | |
58 | ||
59 | .. _`Ceph "organization"`: https://github.com/ceph | |
60 | ||
f67539c2 | 61 | A working knowledge of git_ is essential to make a meaningful contribution to the project as a developer. |
9f95a23c TL |
62 | |
63 | .. _git: https://git-scm.com/doc | |
64 | ||
65 | Although the `Ceph "organization"`_ includes several software repositories, | |
66 | this document covers only one: https://github.com/ceph/ceph. | |
67 | ||
68 | Redmine issue tracker | |
69 | --------------------- | |
70 | ||
71 | Although `GitHub`_ is used for code, Ceph-related issues (Bugs, Features, | |
72 | Backports, Documentation, etc.) are tracked at http://tracker.ceph.com, | |
73 | which is powered by `Redmine`_. | |
74 | ||
75 | .. _Redmine: http://www.redmine.org | |
76 | ||
77 | The tracker has a Ceph project with a number of subprojects loosely | |
78 | corresponding to the various architectural components (see | |
79 | :doc:`/architecture`). | |
80 | ||
81 | Mere `registration`_ in the tracker automatically grants permissions | |
82 | sufficient to open new issues and comment on existing ones. | |
83 | ||
84 | .. _registration: http://tracker.ceph.com/account/register | |
85 | ||
86 | To report a bug or propose a new feature, `jump to the Ceph project`_ and | |
87 | click on `New issue`_. | |
88 | ||
89 | .. _`jump to the Ceph project`: http://tracker.ceph.com/projects/ceph | |
90 | .. _`New issue`: http://tracker.ceph.com/projects/ceph/issues/new | |
91 | ||
39ae355f TL |
92 | Slack |
93 | ----- | |
94 | ||
95 | Ceph's Slack is https://ceph-storage.slack.com/. | |
96 | ||
f67539c2 | 97 | .. _mailing-list: |
9f95a23c | 98 | |
f67539c2 TL |
99 | Mailing lists |
100 | ------------- | |
101 | ||
102 | Ceph Development Mailing List | |
20effc67 | 103 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
9f95a23c TL |
104 | The ``dev@ceph.io`` list is for discussion about the development of Ceph, |
105 | its interoperability with other technology, and the operations of the | |
106 | project itself. | |
107 | ||
f67539c2 TL |
108 | The email discussion list for Ceph development is open to all. Subscribe by |
109 | sending a message to ``dev-request@ceph.io`` with the following line in the | |
110 | body of the message:: | |
9f95a23c TL |
111 | |
112 | subscribe ceph-devel | |
113 | ||
9f95a23c | 114 | |
f67539c2 | 115 | Ceph Client Patch Review Mailing List |
20effc67 | 116 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
9f95a23c TL |
117 | The ``ceph-devel@vger.kernel.org`` list is for discussion and patch review |
118 | for the Linux kernel Ceph client component. Note that this list used to | |
f67539c2 TL |
119 | be an all-encompassing list for developers. When searching the archives, |
120 | remember that this list contains the generic devel-ceph archives before mid-2018. | |
9f95a23c | 121 | |
f67539c2 TL |
122 | Subscribe to the list covering the Linux kernel Ceph client component by sending |
123 | a message to ``majordomo@vger.kernel.org`` with the following line in the body | |
124 | of the message:: | |
9f95a23c TL |
125 | |
126 | subscribe ceph-devel | |
127 | ||
9f95a23c | 128 | |
f67539c2 | 129 | Other Ceph Mailing Lists |
20effc67 | 130 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
9f95a23c TL |
131 | |
132 | There are also `other Ceph-related mailing lists`_. | |
133 | ||
134 | .. _`other Ceph-related mailing lists`: https://ceph.com/irc/ | |
135 | ||
f67539c2 TL |
136 | .. _irc: |
137 | ||
9f95a23c TL |
138 | |
139 | IRC | |
140 | --- | |
141 | ||
f67539c2 TL |
142 | In addition to mailing lists, the Ceph community also communicates in real time |
143 | using `Internet Relay Chat`_. | |
9f95a23c TL |
144 | |
145 | .. _`Internet Relay Chat`: http://www.irchelp.org/ | |
146 | ||
20effc67 TL |
147 | The Ceph community gathers in the #ceph channel of the Open and Free Technology |
148 | Community (OFTC) IRC network. | |
149 | ||
150 | Created in 1988, Internet Relay Chat (IRC) is a relay-based, real-time chat | |
151 | protocol. It is mainly designed for group (many-to-many) communication in | |
152 | discussion forums called channels, but also allows one-to-one communication via | |
153 | private message. On IRC you can talk to many other members using Ceph, on | |
154 | topics ranging from idle chit-chat to support questions. Though a channel might | |
155 | have many people in it at any one time, they might not always be at their | |
156 | keyboard; so if no-one responds, just wait around and someone will hopefully | |
157 | answer soon enough. | |
158 | ||
159 | Registration | |
160 | ~~~~~~~~~~~~ | |
161 | ||
162 | If you intend to use the IRC service on a continued basis, you are advised to | |
163 | register an account. Registering gives you a unique IRC identity and allows you | |
164 | to access channels where unregistered users have been locked out for technical | |
165 | reasons. | |
166 | ||
2a845540 TL |
167 | See ``the official OFTC (Open and Free Technology Community) documentation's |
168 | registration instructions | |
169 | <https://www.oftc.net/Services/#register-your-account>`` to learn how to | |
170 | register your IRC account. | |
20effc67 TL |
171 | |
172 | Channels | |
173 | ~~~~~~~~ | |
174 | ||
175 | To connect to the OFTC IRC network, download an IRC client and configure it to | |
176 | connect to ``irc.oftc.net``. Then join one or more of the channels. Discussions | |
177 | inside #ceph are logged and archives are available online. | |
178 | ||
179 | Here are the real-time discussion channels for the Ceph community: | |
180 | ||
181 | - #ceph | |
182 | - #ceph-devel | |
183 | - #cephfs | |
184 | - #ceph-dashboard | |
185 | - #ceph-orchestrators | |
186 | - #sepia | |
187 | ||
9f95a23c | 188 | |
f67539c2 TL |
189 | .. _submitting-patches: |
190 | ||
9f95a23c TL |
191 | Submitting patches |
192 | ------------------ | |
193 | ||
194 | The canonical instructions for submitting patches are contained in the | |
195 | file `CONTRIBUTING.rst`_ in the top-level directory of the source-code | |
196 | tree. There may be some overlap between this guide and that file. | |
197 | ||
198 | .. _`CONTRIBUTING.rst`: | |
2a845540 | 199 | https://github.com/ceph/ceph/blob/main/CONTRIBUTING.rst |
9f95a23c TL |
200 | |
201 | All newcomers are encouraged to read that file carefully. | |
202 | ||
203 | Building from source | |
204 | -------------------- | |
205 | ||
206 | See instructions at :doc:`/install/build-ceph`. | |
207 | ||
208 | Using ccache to speed up local builds | |
209 | ------------------------------------- | |
f67539c2 TL |
210 | `ccache`_ can make the process of rebuilding the ceph source tree faster. |
211 | ||
212 | Before you use `ccache`_ to speed up your rebuilds of the ceph source tree, | |
213 | make sure that your source tree is clean and will produce no build failures. | |
214 | When you have a clean source tree, you can confidently use `ccache`_, secure in | |
215 | the knowledge that you're not using a dirty tree. | |
216 | ||
217 | Old build artifacts can cause build failures. You might introduce these | |
218 | artifacts unknowingly when switching from one branch to another. If you see | |
219 | build errors when you attempt a local build, follow the procedure below to | |
220 | clean your source tree. | |
9f95a23c | 221 | |
f67539c2 | 222 | Cleaning the Source Tree |
20effc67 | 223 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
9f95a23c | 224 | |
f67539c2 | 225 | .. prompt:: bash $ |
9f95a23c | 226 | |
20effc67 | 227 | ninja clean |
f67539c2 TL |
228 | |
229 | .. note:: The following commands will remove everything in the source tree | |
230 | that isn't tracked by git. Make sure to back up your log files | |
231 | and configuration options before running these commands. | |
9f95a23c | 232 | |
f67539c2 | 233 | .. prompt:: bash $ |
9f95a23c | 234 | |
f67539c2 | 235 | git clean -fdx; git submodule foreach git clean -fdx |
9f95a23c | 236 | |
f67539c2 | 237 | Building Ceph with ccache |
20effc67 TL |
238 | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
239 | ||
f67539c2 TL |
240 | ``ccache`` is available as a package in most distros. To build ceph with |
241 | ccache, run the following command. | |
9f95a23c | 242 | |
f67539c2 | 243 | .. prompt:: bash $ |
9f95a23c | 244 | |
f67539c2 | 245 | cmake -DWITH_CCACHE=ON .. |
9f95a23c | 246 | |
f67539c2 | 247 | Using ccache to Speed Up Build Times |
20effc67 TL |
248 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
249 | ||
f67539c2 TL |
250 | ``ccache`` can be used for speeding up all builds of the system. For more |
251 | details, refer to the `run modes`_ section of the ccache manual. The default | |
252 | settings of ``ccache`` can be displayed with the ``ccache -s`` command. | |
253 | ||
254 | .. note:: We recommend overriding the ``max_size``. The default is 10G. | |
255 | Use a larger value, like 25G. Refer to the `configuration`_ section | |
256 | of the ccache manual for more information. | |
9f95a23c TL |
257 | |
258 | To further increase the cache hit rate and reduce compile times in a | |
f67539c2 TL |
259 | development environment, set the version information and build timestamps to |
260 | fixed values. This makes it unnecessary to rebuild the binaries that contain | |
261 | this information. | |
9f95a23c TL |
262 | |
263 | This can be achieved by adding the following settings to the ``ccache`` | |
264 | configuration file ``ccache.conf``:: | |
265 | ||
266 | sloppiness = time_macros | |
267 | run_second_cpp = true | |
268 | ||
269 | Now, set the environment variable ``SOURCE_DATE_EPOCH`` to a fixed value (a | |
270 | UNIX timestamp) and set ``ENABLE_GIT_VERSION`` to ``OFF`` when running | |
f67539c2 TL |
271 | ``cmake``: |
272 | ||
273 | .. prompt:: bash $ | |
9f95a23c | 274 | |
f67539c2 TL |
275 | export SOURCE_DATE_EPOCH=946684800 |
276 | cmake -DWITH_CCACHE=ON -DENABLE_GIT_VERSION=OFF .. | |
9f95a23c TL |
277 | |
278 | .. note:: Binaries produced with these build options are not suitable for | |
279 | production or debugging purposes, as they do not contain the correct build | |
280 | time and git version information. | |
281 | ||
282 | .. _`ccache`: https://ccache.samba.org/ | |
283 | .. _`run modes`: https://ccache.samba.org/manual.html#_run_modes | |
284 | .. _`configuration`: https://ccache.samba.org/manual.html#_configuration | |
285 | ||
286 | Development-mode cluster | |
287 | ------------------------ | |
288 | ||
289 | See :doc:`/dev/quick_guide`. | |
290 | ||
291 | Kubernetes/Rook development cluster | |
292 | ----------------------------------- | |
293 | ||
294 | See :ref:`kubernetes-dev` | |
295 | ||
f67539c2 TL |
296 | .. _backporting: |
297 | ||
9f95a23c TL |
298 | Backporting |
299 | ----------- | |
300 | ||
2a845540 | 301 | All bugfixes should be merged to the ``main`` branch before being |
9f95a23c TL |
302 | backported. To flag a bugfix for backporting, make sure it has a |
303 | `tracker issue`_ associated with it and set the ``Backport`` field to a | |
304 | comma-separated list of previous releases (e.g. "hammer,jewel") that you think | |
305 | need the backport. | |
306 | The rest (including the actual backporting) will be taken care of by the | |
307 | `Stable Releases and Backports`_ team. | |
308 | ||
309 | .. _`tracker issue`: http://tracker.ceph.com/ | |
310 | .. _`Stable Releases and Backports`: http://tracker.ceph.com/projects/ceph-releases/wiki | |
311 | ||
2a845540 TL |
312 | Dependabot |
313 | ---------- | |
314 | ||
315 | Dependabot is a GitHub bot that scans the dependencies in the repositories for | |
316 | security vulnerabilities (CVEs). If a fix is available for a discovered CVE, | |
317 | Dependabot creates a pull request to update the dependency. | |
318 | ||
319 | Dependabot also indicates the compatibility score of the upgrade. This score is | |
320 | based on the number of CI failures that occur in other GitHub repositories | |
321 | where the fix was applied. | |
322 | ||
323 | With some configuration, Dependabot can perform non-security updates (for | |
324 | example, it can upgrade to the latest minor version or patch version). | |
325 | ||
326 | Dependabot supports `several languages and package managers | |
327 | <https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/about-dependabot-version-updates#supported-repositories-and-ecosystems>`_. | |
328 | As of July 2022, the Ceph project receives alerts only from pip (based on the | |
329 | `requirements.txt` files) and npm (`package*.json`). It is possible to extend | |
330 | these alerts to git submodules, Golang, and Java. As of July 2022, there is no | |
331 | support for C++ package managers such as vcpkg, conan, C++20 modules. | |
332 | ||
333 | Many of the dependencies discovered by Dependabot will best be updated | |
334 | elsewhere than the Ceph Github repository (distribution packages, for example, | |
335 | will be a better place to update some of the dependencies). Nonetheless, the | |
336 | list of new and existing vulnerabilities generated by Dependabot will be | |
337 | useful. | |
338 | ||
339 | `Here is an example of a Dependabot pull request. | |
340 | <https://github.com/ceph/ceph/pull/46998>`_ | |
341 | ||
9f95a23c TL |
342 | Guidance for use of cluster log |
343 | ------------------------------- | |
344 | ||
345 | If your patches emit messages to the Ceph cluster log, please consult | |
f67539c2 | 346 | this: :doc:`/dev/logging`. |