]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | # Ceph - a scalable distributed storage system |
2 | ||
aee94f69 | 3 | See https://ceph.com/ for current information about Ceph. |
7c673cae FG |
4 | |
5 | ||
6 | ## Contributing Code | |
7 | ||
aee94f69 TL |
8 | Most of Ceph is dual-licensed under the LGPL version 2.1 or 3.0. Some |
9 | miscellaneous code is either public domain or licensed under a BSD-style | |
10 | license. | |
7c673cae | 11 | |
aee94f69 TL |
12 | The Ceph documentation is licensed under Creative Commons Attribution Share |
13 | Alike 3.0 (CC-BY-SA-3.0). | |
7c673cae | 14 | |
aee94f69 TL |
15 | Some headers included in the `ceph/ceph` repository are licensed under the GPL. |
16 | See the file `COPYING` for a full inventory of licenses by file. | |
17 | ||
18 | All code contributions must include a valid "Signed-off-by" line. See the file | |
19 | `SubmittingPatches.rst` for details on this and instructions on how to generate | |
20 | and submit patches. | |
21 | ||
22 | Assignment of copyright is not required to contribute code. Code is | |
7c673cae FG |
23 | contributed under the terms of the applicable license. |
24 | ||
25 | ||
26 | ## Checking out the source | |
27 | ||
05a536ef TL |
28 | Clone the ceph/ceph repository from github by running the following command on |
29 | a system that has git installed: | |
7c673cae FG |
30 | |
31 | git clone git@github.com:ceph/ceph | |
32 | ||
05a536ef TL |
33 | Alternatively, if you are not a github user, you should run the following |
34 | command on a system that has git installed: | |
7c673cae | 35 | |
1e59de90 | 36 | git clone https://github.com/ceph/ceph.git |
7c673cae | 37 | |
aee94f69 TL |
38 | When the `ceph/ceph` repository has been cloned to your system, run the |
39 | following commands to move into the cloned `ceph/ceph` repository and to check | |
40 | out the git submodules associated with it: | |
7c673cae | 41 | |
aee94f69 | 42 | cd ceph |
7c673cae FG |
43 | git submodule update --init --recursive |
44 | ||
45 | ||
46 | ## Build Prerequisites | |
47 | ||
05a536ef TL |
48 | *section last updated 27 Jul 2023* |
49 | ||
50 | Make sure that ``curl`` is installed. The Debian and Ubuntu ``apt`` command is | |
51 | provided here, but if you use a system with a different package manager, then | |
52 | you must use whatever command is the proper counterpart of this one: | |
53 | ||
54 | apt install curl | |
55 | ||
56 | Install Debian or RPM package dependencies by running the following command: | |
7c673cae FG |
57 | |
58 | ./install-deps.sh | |
59 | ||
05a536ef TL |
60 | Install the ``python3-routes`` package: |
61 | ||
62 | apt install python3-routes | |
63 | ||
7c673cae FG |
64 | |
65 | ## Building Ceph | |
66 | ||
05a536ef TL |
67 | These instructions are meant for developers who are compiling the code for |
68 | development and testing. To build binaries that are suitable for installation | |
aee94f69 TL |
69 | we recommend that you build `.deb` or `.rpm` packages, or refer to |
70 | ``ceph.spec.in`` or ``debian/rules`` to see which configuration options are | |
71 | specified for production builds. | |
7c673cae | 72 | |
aee94f69 TL |
73 | To build Ceph, make sure that you are in the top-level `ceph` directory that |
74 | contains `do_cmake.sh` and `CONTRIBUTING.rst` and run the following commands: | |
7c673cae FG |
75 | |
76 | ./do_cmake.sh | |
77 | cd build | |
20effc67 | 78 | ninja |
7c673cae | 79 | |
aee94f69 TL |
80 | ``do_cmake.sh`` by default creates a "debug build" of Ceph, which can be up to |
81 | five times slower than a non-debug build. Pass | |
82 | ``-DCMAKE_BUILD_TYPE=RelWithDebInfo`` to ``do_cmake.sh`` to create a non-debug | |
83 | build. | |
84 | ||
85 | [Ninja](https://ninja-build.org/) is the buildsystem used by the Ceph project | |
86 | to build test builds. The number of jobs used by `ninja` is derived from the | |
87 | number of CPU cores of the building host if unspecified. Use the `-j` option to | |
88 | limit the job number if the build jobs are running out of memory. If you | |
89 | attempt to run `ninja` and receive a message that reads `g++: fatal error: | |
90 | Killed signal terminated program cc1plus`, then you have run out of memory. | |
91 | Using the `-j` option with an argument appropriate to the hardware on which the | |
92 | `ninja` command is run is expected to result in a successful build. For example, | |
93 | to limit the job number to 3, run the command `ninja -j 3`. On average, each | |
94 | `ninja` job run in parallel needs approximately 2.5 GiB of RAM. | |
95 | ||
96 | This documentation assumes that your build directory is a subdirectory of the | |
97 | `ceph.git` checkout. If the build directory is located elsewhere, point | |
98 | `CEPH_GIT_DIR` to the correct path of the checkout. Additional CMake args can | |
99 | be specified by setting ARGS before invoking ``do_cmake.sh``. See [cmake | |
100 | options](#cmake-options) for more details. For example: | |
11fdf7f2 TL |
101 | |
102 | ARGS="-DCMAKE_C_COMPILER=gcc-7" ./do_cmake.sh | |
7c673cae | 103 | |
aee94f69 | 104 | To build only certain targets, run a command of the following form: |
7c673cae | 105 | |
20effc67 | 106 | ninja [target name] |
7c673cae FG |
107 | |
108 | To install: | |
109 | ||
20effc67 | 110 | ninja install |
7c673cae FG |
111 | |
112 | ### CMake Options | |
113 | ||
114 | If you run the `cmake` command by hand, there are many options you can | |
20effc67 | 115 | set with "-D". For example, the option to build the RADOS Gateway is |
7c673cae FG |
116 | defaulted to ON. To build without the RADOS Gateway: |
117 | ||
20effc67 | 118 | cmake -DWITH_RADOSGW=OFF [path to top-level ceph directory] |
7c673cae FG |
119 | |
120 | Another example below is building with debugging and alternate locations | |
121 | for a couple of external dependencies: | |
122 | ||
f67539c2 | 123 | cmake -DLEVELDB_PREFIX="/opt/hyperleveldb" \ |
20effc67 | 124 | -DCMAKE_INSTALL_PREFIX=/opt/ceph -DCMAKE_C_FLAGS="-Og -g3 -gdwarf-4" \ |
11fdf7f2 | 125 | .. |
7c673cae | 126 | |
1e59de90 TL |
127 | Ceph has several bundled dependencies such as Boost, RocksDB and Arrow. By |
128 | default, cmake will build these bundled dependencies from source instead of | |
129 | using libraries that are already installed on the system. You can opt-in to | |
130 | using these system libraries, provided they meet the minimum version required | |
131 | by Ceph, with cmake options like `WITH_SYSTEM_BOOST`: | |
132 | ||
133 | cmake -DWITH_SYSTEM_BOOST=ON [...] | |
134 | ||
7c673cae FG |
135 | To view an exhaustive list of -D options, you can invoke `cmake` with: |
136 | ||
11fdf7f2 | 137 | cmake -LH |
7c673cae | 138 | |
20effc67 | 139 | If you often pipe `ninja` to `less` and would like to maintain the |
7c673cae FG |
140 | diagnostic colors for errors and warnings (and if your compiler |
141 | supports it), you can invoke `cmake` with: | |
142 | ||
20effc67 | 143 | cmake -DDIAGNOSTICS_COLOR=always ... |
7c673cae FG |
144 | |
145 | Then you'll get the diagnostic colors when you execute: | |
146 | ||
20effc67 | 147 | ninja | less -R |
7c673cae FG |
148 | |
149 | Other available values for 'DIAGNOSTICS_COLOR' are 'auto' (default) and | |
150 | 'never'. | |
151 | ||
152 | ||
153 | ## Building a source tarball | |
154 | ||
155 | To build a complete source tarball with everything needed to build from | |
156 | source and/or build a (deb or rpm) package, run | |
157 | ||
158 | ./make-dist | |
159 | ||
160 | This will create a tarball like ceph-$version.tar.bz2 from git. | |
161 | (Ensure that any changes you want to include in your working directory | |
162 | are committed to git.) | |
163 | ||
164 | ||
165 | ## Running a test cluster | |
166 | ||
aee94f69 TL |
167 | From the `ceph/` directory, run the following commands to launch a test Ceph |
168 | cluster: | |
7c673cae FG |
169 | |
170 | cd build | |
20effc67 | 171 | ninja vstart # builds just enough to run vstart |
11fdf7f2 | 172 | ../src/vstart.sh --debug --new -x --localhost --bluestore |
7c673cae FG |
173 | ./bin/ceph -s |
174 | ||
aee94f69 | 175 | Most Ceph commands are available in the `bin/` directory. For example: |
7c673cae | 176 | |
7c673cae | 177 | ./bin/rbd create foo --size 1000 |
aee94f69 | 178 | ./bin/rados -p foo bench 30 write |
7c673cae | 179 | |
aee94f69 TL |
180 | To shut down the test cluster, run the following command from the `build/` |
181 | directory: | |
7c673cae FG |
182 | |
183 | ../src/stop.sh | |
184 | ||
aee94f69 | 185 | Use the sysvinit script to start or stop individual daemons: |
7c673cae FG |
186 | |
187 | ./bin/init-ceph restart osd.0 | |
188 | ./bin/init-ceph stop | |
189 | ||
190 | ||
191 | ## Running unit tests | |
192 | ||
193 | To build and run all tests (in parallel using all processors), use `ctest`: | |
194 | ||
195 | cd build | |
20effc67 | 196 | ninja |
7c673cae FG |
197 | ctest -j$(nproc) |
198 | ||
199 | (Note: Many targets built from src/test are not run using `ctest`. | |
20effc67 | 200 | Targets starting with "unittest" are run in `ninja check` and thus can |
7c673cae FG |
201 | be run with `ctest`. Targets starting with "ceph_test" can not, and should |
202 | be run by hand.) | |
203 | ||
204 | When failures occur, look in build/Testing/Temporary for logs. | |
205 | ||
206 | To build and run all tests and their dependencies without other | |
207 | unnecessary targets in Ceph: | |
208 | ||
11fdf7f2 | 209 | cd build |
20effc67 | 210 | ninja check -j$(nproc) |
7c673cae FG |
211 | |
212 | To run an individual test manually, run `ctest` with -R (regex matching): | |
213 | ||
214 | ctest -R [regex matching test name(s)] | |
215 | ||
216 | (Note: `ctest` does not build the test it's running or the dependencies needed | |
217 | to run it) | |
218 | ||
219 | To run an individual test manually and see all the tests output, run | |
220 | `ctest` with the -V (verbose) flag: | |
221 | ||
222 | ctest -V -R [regex matching test name(s)] | |
223 | ||
20effc67 | 224 | To run tests manually and run the jobs in parallel, run `ctest` with |
c07f9fc5 | 225 | the `-j` flag: |
7c673cae FG |
226 | |
227 | ctest -j [number of jobs] | |
228 | ||
229 | There are many other flags you can give `ctest` for better control | |
230 | over manual test execution. To view these options run: | |
231 | ||
232 | man ctest | |
233 | ||
234 | ||
235 | ## Building the Documentation | |
236 | ||
237 | ### Prerequisites | |
238 | ||
239 | The list of package dependencies for building the documentation can be | |
c07f9fc5 | 240 | found in `doc_deps.deb.txt`: |
7c673cae FG |
241 | |
242 | sudo apt-get install `cat doc_deps.deb.txt` | |
243 | ||
244 | ### Building the Documentation | |
245 | ||
246 | To build the documentation, ensure that you are in the top-level | |
c07f9fc5 | 247 | `/ceph` directory, and execute the build script. For example: |
7c673cae FG |
248 | |
249 | admin/build-doc | |
250 | ||
20effc67 TL |
251 | ## Reporting Issues |
252 | ||
253 | To report an issue and view existing issues, please visit https://tracker.ceph.com/projects/ceph. |