]>
Commit | Line | Data |
---|---|---|
e228b1e6 NP |
1 | Copyright 2010 Nicolas Palix <npalix@diku.dk> |
2 | Copyright 2010 Julia Lawall <julia@diku.dk> | |
3 | Copyright 2010 Gilles Muller <Gilles.Muller@lip6.fr> | |
4 | ||
5 | ||
6 | Getting Coccinelle | |
7 | ~~~~~~~~~~~~~~~~~~~~ | |
8 | ||
ec97946e NP |
9 | The semantic patches included in the kernel use features and options |
10 | which are provided by Coccinelle version 1.0.0-rc11 and above. | |
11 | Using earlier versions will fail as the option names used by | |
12 | the Coccinelle files and coccicheck have been updated. | |
e228b1e6 | 13 | |
ec97946e | 14 | Coccinelle is available through the package manager |
e228b1e6 NP |
15 | of many distributions, e.g. : |
16 | ||
ec97946e NP |
17 | - Debian |
18 | - Fedora | |
19 | - Ubuntu | |
e228b1e6 NP |
20 | - OpenSUSE |
21 | - Arch Linux | |
22 | - NetBSD | |
23 | - FreeBSD | |
24 | ||
25 | ||
26 | You can get the latest version released from the Coccinelle homepage at | |
27 | http://coccinelle.lip6.fr/ | |
28 | ||
32af0898 NP |
29 | Information and tips about Coccinelle are also provided on the wiki |
30 | pages at http://cocci.ekstranet.diku.dk/wiki/doku.php | |
31 | ||
e228b1e6 NP |
32 | Once you have it, run the following command: |
33 | ||
34 | ./configure | |
35 | make | |
36 | ||
37 | as a regular user, and install it with | |
38 | ||
39 | sudo make install | |
40 | ||
e228b1e6 NP |
41 | Using Coccinelle on the Linux kernel |
42 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
43 | ||
44 | A Coccinelle-specific target is defined in the top level | |
45 | Makefile. This target is named 'coccicheck' and calls the 'coccicheck' | |
46 | front-end in the 'scripts' directory. | |
47 | ||
78a95b9b | 48 | Four basic modes are defined: patch, report, context, and org. The mode to |
e228b1e6 NP |
49 | use is specified by setting the MODE variable with 'MODE=<mode>'. |
50 | ||
32af0898 NP |
51 | 'patch' proposes a fix, when possible. |
52 | ||
e228b1e6 NP |
53 | 'report' generates a list in the following format: |
54 | file:line:column-column: message | |
55 | ||
e228b1e6 NP |
56 | 'context' highlights lines of interest and their context in a |
57 | diff-like style.Lines of interest are indicated with '-'. | |
58 | ||
59 | 'org' generates a report in the Org mode format of Emacs. | |
60 | ||
32af0898 | 61 | Note that not all semantic patches implement all modes. For easy use |
78a95b9b | 62 | of Coccinelle, the default mode is "report". |
e228b1e6 | 63 | |
78a95b9b | 64 | Two other modes provide some common combinations of these modes. |
e228b1e6 | 65 | |
78a95b9b | 66 | 'chain' tries the previous modes in the order above until one succeeds. |
e228b1e6 | 67 | |
78a95b9b NP |
68 | 'rep+ctxt' runs successively the report mode and the context mode. |
69 | It should be used with the C option (described later) | |
70 | which checks the code on a file basis. | |
e228b1e6 | 71 | |
78a95b9b NP |
72 | Examples: |
73 | To make a report for every semantic patch, run the following command: | |
e228b1e6 | 74 | |
78a95b9b NP |
75 | make coccicheck MODE=report |
76 | ||
77 | To produce patches, run: | |
78 | ||
79 | make coccicheck MODE=patch | |
e228b1e6 NP |
80 | |
81 | ||
82 | The coccicheck target applies every semantic patch available in the | |
32af0898 | 83 | sub-directories of 'scripts/coccinelle' to the entire Linux kernel. |
e228b1e6 | 84 | |
32af0898 | 85 | For each semantic patch, a commit message is proposed. It gives a |
e228b1e6 NP |
86 | description of the problem being checked by the semantic patch, and |
87 | includes a reference to Coccinelle. | |
88 | ||
89 | As any static code analyzer, Coccinelle produces false | |
90 | positives. Thus, reports must be carefully checked, and patches | |
91 | reviewed. | |
92 | ||
26e56720 BS |
93 | To enable verbose messages set the V= variable, for example: |
94 | ||
95 | make coccicheck MODE=report V=1 | |
96 | ||
c930a1b2 LR |
97 | Coccinelle parallelization |
98 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
99 | ||
90d06a46 KC |
100 | By default, coccicheck tries to run as parallel as possible. To change |
101 | the parallelism, set the J= variable. For example, to run across 4 CPUs: | |
102 | ||
103 | make coccicheck MODE=report J=4 | |
104 | ||
c930a1b2 LR |
105 | As of Coccinelle 1.0.2 Coccinelle uses Ocaml parmap for parallelization, |
106 | if support for this is detected you will benefit from parmap parallelization. | |
107 | ||
108 | When parmap is enabled coccicheck will enable dynamic load balancing by using | |
109 | '--chunksize 1' argument, this ensures we keep feeding threads with work | |
110 | one by one, so that we avoid the situation where most work gets done by only | |
111 | a few threads. With dynamic load balancing, if a thread finishes early we keep | |
112 | feeding it more work. | |
113 | ||
114 | When parmap is enabled, if an error occurs in Coccinelle, this error | |
115 | value is propagated back, the return value of the 'make coccicheck' | |
116 | captures this return value. | |
e228b1e6 NP |
117 | |
118 | Using Coccinelle with a single semantic patch | |
119 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
120 | ||
121 | The optional make variable COCCI can be used to check a single | |
122 | semantic patch. In that case, the variable must be initialized with | |
123 | the name of the semantic patch to apply. | |
124 | ||
125 | For instance: | |
126 | ||
127 | make coccicheck COCCI=<my_SP.cocci> MODE=patch | |
128 | or | |
129 | make coccicheck COCCI=<my_SP.cocci> MODE=report | |
130 | ||
131 | ||
f95ab209 GD |
132 | Controlling Which Files are Processed by Coccinelle |
133 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
134 | By default the entire kernel source tree is checked. | |
135 | ||
136 | To apply Coccinelle to a specific directory, M= can be used. | |
137 | For example, to check drivers/net/wireless/ one may write: | |
32af0898 | 138 | |
f95ab209 | 139 | make coccicheck M=drivers/net/wireless/ |
ed621cc4 | 140 | |
32af0898 NP |
141 | To apply Coccinelle on a file basis, instead of a directory basis, the |
142 | following command may be used: | |
143 | ||
144 | make C=1 CHECK="scripts/coccicheck" | |
145 | ||
146 | To check only newly edited code, use the value 2 for the C flag, i.e. | |
147 | ||
148 | make C=2 CHECK="scripts/coccicheck" | |
149 | ||
78a95b9b NP |
150 | In these modes, which works on a file basis, there is no information |
151 | about semantic patches displayed, and no commit message proposed. | |
152 | ||
32af0898 NP |
153 | This runs every semantic patch in scripts/coccinelle by default. The |
154 | COCCI variable may additionally be used to only apply a single | |
155 | semantic patch as shown in the previous section. | |
156 | ||
78a95b9b | 157 | The "report" mode is the default. You can select another one with the |
32af0898 NP |
158 | MODE variable explained above. |
159 | ||
be1fa900 LR |
160 | Debugging Coccinelle SmPL patches |
161 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
162 | ||
163 | Using coccicheck is best as it provides in the spatch command line | |
164 | include options matching the options used when we compile the kernel. | |
165 | You can learn what these options are by using V=1, you could then | |
166 | manually run Coccinelle with debug options added. | |
167 | ||
168 | Alternatively you can debug running Coccinelle against SmPL patches | |
169 | by asking for stderr to be redirected to stderr, by default stderr | |
170 | is redirected to /dev/null, if you'd like to capture stderr you | |
171 | can specify the DEBUG_FILE="file.txt" option to coccicheck. For | |
172 | instance: | |
173 | ||
174 | rm -f cocci.err | |
175 | make coccicheck COCCI=scripts/coccinelle/free/kfree.cocci MODE=report DEBUG_FILE=cocci.err | |
176 | cat cocci.err | |
177 | ||
5c384dba LR |
178 | You can use SPFLAGS to add debugging flags, for instance you may want to |
179 | add both --profile --show-trying to SPFLAGS when debugging. For instance | |
180 | you may want to use: | |
181 | ||
182 | rm -f err.log | |
183 | export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci | |
184 | make coccicheck DEBUG_FILE="err.log" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd/arizona-irq.c | |
185 | ||
186 | err.log will now have the profiling information, while stdout will | |
187 | provide some progress information as Coccinelle moves forward with | |
188 | work. | |
189 | ||
be1fa900 LR |
190 | DEBUG_FILE support is only supported when using coccinelle >= 1.2. |
191 | ||
dd951fc1 LR |
192 | .cocciconfig support |
193 | ~~~~~~~~~~~~~~~~~~~~~~ | |
194 | ||
195 | Coccinelle supports reading .cocciconfig for default Coccinelle options that | |
196 | should be used every time spatch is spawned, the order of precedence for | |
197 | variables for .cocciconfig is as follows: | |
198 | ||
199 | o Your current user's home directory is processed first | |
200 | o Your directory from which spatch is called is processed next | |
201 | o The directory provided with the --dir option is processed last, if used | |
202 | ||
203 | Since coccicheck runs through make, it naturally runs from the kernel | |
204 | proper dir, as such the second rule above would be implied for picking up a | |
205 | .cocciconfig when using 'make coccicheck'. | |
206 | ||
207 | 'make coccicheck' also supports using M= targets.If you do not supply | |
208 | any M= target, it is assumed you want to target the entire kernel. | |
209 | The kernel coccicheck script has: | |
210 | ||
211 | if [ "$KBUILD_EXTMOD" = "" ] ; then | |
212 | OPTIONS="--dir $srctree $COCCIINCLUDE" | |
213 | else | |
214 | OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" | |
215 | fi | |
216 | ||
217 | KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases | |
218 | the spatch --dir argument is used, as such third rule applies when whether M= | |
219 | is used or not, and when M= is used the target directory can have its own | |
220 | .cocciconfig file. When M= is not passed as an argument to coccicheck the | |
221 | target directory is the same as the directory from where spatch was called. | |
222 | ||
223 | If not using the kernel's coccicheck target, keep the above precedence | |
224 | order logic of .cocciconfig reading. If using the kernel's coccicheck target, | |
225 | override any of the kernel's .coccicheck's settings using SPFLAGS. | |
226 | ||
227 | We help Coccinelle when used against Linux with a set of sensible defaults | |
228 | options for Linux with our own Linux .cocciconfig. This hints to coccinelle | |
229 | git can be used for 'git grep' queries over coccigrep. A timeout of 200 | |
230 | seconds should suffice for now. | |
231 | ||
232 | The options picked up by coccinelle when reading a .cocciconfig do not appear | |
233 | as arguments to spatch processes running on your system, to confirm what | |
234 | options will be used by Coccinelle run: | |
235 | ||
236 | spatch --print-options-only | |
237 | ||
238 | You can override with your own preferred index option by using SPFLAGS. Take | |
239 | note that when there are conflicting options Coccinelle takes precedence for | |
240 | the last options passed. Using .cocciconfig is possible to use idutils, however | |
241 | given the order of precedence followed by Coccinelle, since the kernel now | |
242 | carries its own .cocciconfig, you will need to use SPFLAGS to use idutils if | |
243 | desired. See below section "Additional flags" for more details on how to use | |
244 | idutils. | |
245 | ||
ed621cc4 NP |
246 | Additional flags |
247 | ~~~~~~~~~~~~~~~~~~ | |
248 | ||
249 | Additional flags can be passed to spatch through the SPFLAGS | |
8e826ad5 LR |
250 | variable. This works as Coccinelle respects the last flags |
251 | given to it when options are in conflict. | |
ed621cc4 | 252 | |
78a95b9b | 253 | make SPFLAGS=--use-glimpse coccicheck |
dd951fc1 LR |
254 | |
255 | Coccinelle supports idutils as well but requires coccinelle >= 1.0.6. | |
256 | When no ID file is specified coccinelle assumes your ID database file | |
257 | is in the file .id-utils.index on the top level of the kernel, coccinelle | |
258 | carries a script scripts/idutils_index.sh which creates the database with | |
259 | ||
260 | mkid -i C --output .id-utils.index | |
261 | ||
262 | If you have another database filename you can also just symlink with this | |
263 | name. | |
264 | ||
78a95b9b | 265 | make SPFLAGS=--use-idutils coccicheck |
ed621cc4 | 266 | |
dd951fc1 LR |
267 | Alternatively you can specify the database filename explicitly, for |
268 | instance: | |
269 | ||
270 | make SPFLAGS="--use-idutils /full-path/to/ID" coccicheck | |
271 | ||
ed621cc4 | 272 | See spatch --help to learn more about spatch options. |
32af0898 | 273 | |
78a95b9b NP |
274 | Note that the '--use-glimpse' and '--use-idutils' options |
275 | require external tools for indexing the code. None of them is | |
276 | thus active by default. However, by indexing the code with | |
277 | one of these tools, and according to the cocci file used, | |
278 | spatch could proceed the entire code base more quickly. | |
279 | ||
e228b1e6 NP |
280 | Proposing new semantic patches |
281 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
282 | ||
283 | New semantic patches can be proposed and submitted by kernel | |
284 | developers. For sake of clarity, they should be organized in the | |
32af0898 | 285 | sub-directories of 'scripts/coccinelle/'. |
e228b1e6 NP |
286 | |
287 | ||
288 | Detailed description of the 'report' mode | |
289 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
290 | ||
291 | 'report' generates a list in the following format: | |
292 | file:line:column-column: message | |
293 | ||
294 | Example: | |
295 | ||
296 | Running | |
297 | ||
9dcf7990 | 298 | make coccicheck MODE=report COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
299 | |
300 | will execute the following part of the SmPL script. | |
301 | ||
302 | <smpl> | |
303 | @r depends on !context && !patch && (org || report)@ | |
304 | expression x; | |
305 | position p; | |
306 | @@ | |
307 | ||
308 | ERR_PTR@p(PTR_ERR(x)) | |
309 | ||
310 | @script:python depends on report@ | |
311 | p << r.p; | |
312 | x << r.x; | |
313 | @@ | |
314 | ||
315 | msg="ERR_CAST can be used with %s" % (x) | |
316 | coccilib.report.print_report(p[0], msg) | |
317 | </smpl> | |
318 | ||
319 | This SmPL excerpt generates entries on the standard output, as | |
320 | illustrated below: | |
321 | ||
322 | /home/user/linux/crypto/ctr.c:188:9-16: ERR_CAST can be used with alg | |
323 | /home/user/linux/crypto/authenc.c:619:9-16: ERR_CAST can be used with auth | |
324 | /home/user/linux/crypto/xts.c:227:9-16: ERR_CAST can be used with alg | |
325 | ||
326 | ||
327 | Detailed description of the 'patch' mode | |
328 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
329 | ||
330 | When the 'patch' mode is available, it proposes a fix for each problem | |
331 | identified. | |
332 | ||
333 | Example: | |
334 | ||
335 | Running | |
9dcf7990 | 336 | make coccicheck MODE=patch COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
337 | |
338 | will execute the following part of the SmPL script. | |
339 | ||
340 | <smpl> | |
341 | @ depends on !context && patch && !org && !report @ | |
342 | expression x; | |
343 | @@ | |
344 | ||
345 | - ERR_PTR(PTR_ERR(x)) | |
346 | + ERR_CAST(x) | |
347 | </smpl> | |
348 | ||
349 | This SmPL excerpt generates patch hunks on the standard output, as | |
350 | illustrated below: | |
351 | ||
352 | diff -u -p a/crypto/ctr.c b/crypto/ctr.c | |
353 | --- a/crypto/ctr.c 2010-05-26 10:49:38.000000000 +0200 | |
354 | +++ b/crypto/ctr.c 2010-06-03 23:44:49.000000000 +0200 | |
355 | @@ -185,7 +185,7 @@ static struct crypto_instance *crypto_ct | |
356 | alg = crypto_attr_alg(tb[1], CRYPTO_ALG_TYPE_CIPHER, | |
357 | CRYPTO_ALG_TYPE_MASK); | |
358 | if (IS_ERR(alg)) | |
359 | - return ERR_PTR(PTR_ERR(alg)); | |
360 | + return ERR_CAST(alg); | |
361 | ||
362 | /* Block size must be >= 4 bytes. */ | |
363 | err = -EINVAL; | |
364 | ||
365 | Detailed description of the 'context' mode | |
366 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
367 | ||
368 | 'context' highlights lines of interest and their context | |
369 | in a diff-like style. | |
370 | ||
371 | NOTE: The diff-like output generated is NOT an applicable patch. The | |
372 | intent of the 'context' mode is to highlight the important lines | |
373 | (annotated with minus, '-') and gives some surrounding context | |
374 | lines around. This output can be used with the diff mode of | |
375 | Emacs to review the code. | |
376 | ||
377 | Example: | |
378 | ||
379 | Running | |
9dcf7990 | 380 | make coccicheck MODE=context COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
381 | |
382 | will execute the following part of the SmPL script. | |
383 | ||
384 | <smpl> | |
385 | @ depends on context && !patch && !org && !report@ | |
386 | expression x; | |
387 | @@ | |
388 | ||
389 | * ERR_PTR(PTR_ERR(x)) | |
390 | </smpl> | |
391 | ||
392 | This SmPL excerpt generates diff hunks on the standard output, as | |
393 | illustrated below: | |
394 | ||
395 | diff -u -p /home/user/linux/crypto/ctr.c /tmp/nothing | |
396 | --- /home/user/linux/crypto/ctr.c 2010-05-26 10:49:38.000000000 +0200 | |
397 | +++ /tmp/nothing | |
398 | @@ -185,7 +185,6 @@ static struct crypto_instance *crypto_ct | |
399 | alg = crypto_attr_alg(tb[1], CRYPTO_ALG_TYPE_CIPHER, | |
400 | CRYPTO_ALG_TYPE_MASK); | |
401 | if (IS_ERR(alg)) | |
402 | - return ERR_PTR(PTR_ERR(alg)); | |
403 | ||
404 | /* Block size must be >= 4 bytes. */ | |
405 | err = -EINVAL; | |
406 | ||
407 | Detailed description of the 'org' mode | |
408 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
409 | ||
410 | 'org' generates a report in the Org mode format of Emacs. | |
411 | ||
412 | Example: | |
413 | ||
414 | Running | |
9dcf7990 | 415 | make coccicheck MODE=org COCCI=scripts/coccinelle/api/err_cast.cocci |
e228b1e6 NP |
416 | |
417 | will execute the following part of the SmPL script. | |
418 | ||
419 | <smpl> | |
420 | @r depends on !context && !patch && (org || report)@ | |
421 | expression x; | |
422 | position p; | |
423 | @@ | |
424 | ||
425 | ERR_PTR@p(PTR_ERR(x)) | |
426 | ||
427 | @script:python depends on org@ | |
428 | p << r.p; | |
429 | x << r.x; | |
430 | @@ | |
431 | ||
432 | msg="ERR_CAST can be used with %s" % (x) | |
433 | msg_safe=msg.replace("[","@(").replace("]",")") | |
434 | coccilib.org.print_todo(p[0], msg_safe) | |
435 | </smpl> | |
436 | ||
437 | This SmPL excerpt generates Org entries on the standard output, as | |
438 | illustrated below: | |
439 | ||
440 | * TODO [[view:/home/user/linux/crypto/ctr.c::face=ovl-face1::linb=188::colb=9::cole=16][ERR_CAST can be used with alg]] | |
441 | * TODO [[view:/home/user/linux/crypto/authenc.c::face=ovl-face1::linb=619::colb=9::cole=16][ERR_CAST can be used with auth]] | |
442 | * TODO [[view:/home/user/linux/crypto/xts.c::face=ovl-face1::linb=227::colb=9::cole=16][ERR_CAST can be used with alg]] |