]>
Commit | Line | Data |
---|---|---|
f7c9fe4b MCC |
1 | .. _development_early_stage: |
2 | ||
3 | Early-stage planning | |
4 | ==================== | |
75b02146 JC |
5 | |
6 | When contemplating a Linux kernel development project, it can be tempting | |
7 | to jump right in and start coding. As with any significant project, | |
8 | though, much of the groundwork for success is best laid before the first | |
9 | line of code is written. Some time spent in early planning and | |
10 | communication can save far more time later on. | |
11 | ||
12 | ||
f7c9fe4b MCC |
13 | Specifying the problem |
14 | ---------------------- | |
75b02146 JC |
15 | |
16 | Like any engineering project, a successful kernel enhancement starts with a | |
17 | clear description of the problem to be solved. In some cases, this step is | |
18 | easy: when a driver is needed for a specific piece of hardware, for | |
19 | example. In others, though, it is tempting to confuse the real problem | |
20 | with the proposed solution, and that can lead to difficulties. | |
21 | ||
22 | Consider an example: some years ago, developers working with Linux audio | |
23 | sought a way to run applications without dropouts or other artifacts caused | |
24 | by excessive latency in the system. The solution they arrived at was a | |
25 | kernel module intended to hook into the Linux Security Module (LSM) | |
26 | framework; this module could be configured to give specific applications | |
27 | access to the realtime scheduler. This module was implemented and sent to | |
28 | the linux-kernel mailing list, where it immediately ran into problems. | |
29 | ||
30 | To the audio developers, this security module was sufficient to solve their | |
31 | immediate problem. To the wider kernel community, though, it was seen as a | |
32 | misuse of the LSM framework (which is not intended to confer privileges | |
33 | onto processes which they would not otherwise have) and a risk to system | |
34 | stability. Their preferred solutions involved realtime scheduling access | |
35 | via the rlimit mechanism for the short term, and ongoing latency reduction | |
36 | work in the long term. | |
37 | ||
38 | The audio community, however, could not see past the particular solution | |
39 | they had implemented; they were unwilling to accept alternatives. The | |
40 | resulting disagreement left those developers feeling disillusioned with the | |
41 | entire kernel development process; one of them went back to an audio list | |
42 | and posted this: | |
43 | ||
44 | There are a number of very good Linux kernel developers, but they | |
45 | tend to get outshouted by a large crowd of arrogant fools. Trying | |
46 | to communicate user requirements to these people is a waste of | |
47 | time. They are much too "intelligent" to listen to lesser mortals. | |
48 | ||
49 | (http://lwn.net/Articles/131776/). | |
50 | ||
51 | The reality of the situation was different; the kernel developers were far | |
52 | more concerned about system stability, long-term maintenance, and finding | |
53 | the right solution to the problem than they were with a specific module. | |
54 | The moral of the story is to focus on the problem - not a specific solution | |
55 | - and to discuss it with the development community before investing in the | |
56 | creation of a body of code. | |
57 | ||
58 | So, when contemplating a kernel development project, one should obtain | |
59 | answers to a short set of questions: | |
60 | ||
61 | - What, exactly, is the problem which needs to be solved? | |
62 | ||
63 | - Who are the users affected by this problem? Which use cases should the | |
64 | solution address? | |
65 | ||
66 | - How does the kernel fall short in addressing that problem now? | |
67 | ||
68 | Only then does it make sense to start considering possible solutions. | |
69 | ||
70 | ||
f7c9fe4b MCC |
71 | Early discussion |
72 | ---------------- | |
75b02146 JC |
73 | |
74 | When planning a kernel development project, it makes great sense to hold | |
75 | discussions with the community before launching into implementation. Early | |
76 | communication can save time and trouble in a number of ways: | |
77 | ||
78 | - It may well be that the problem is addressed by the kernel in ways which | |
79 | you have not understood. The Linux kernel is large and has a number of | |
80 | features and capabilities which are not immediately obvious. Not all | |
81 | kernel capabilities are documented as well as one might like, and it is | |
82 | easy to miss things. Your author has seen the posting of a complete | |
83 | driver which duplicated an existing driver that the new author had been | |
84 | unaware of. Code which reinvents existing wheels is not only wasteful; | |
85 | it will also not be accepted into the mainline kernel. | |
86 | ||
87 | - There may be elements of the proposed solution which will not be | |
88 | acceptable for mainline merging. It is better to find out about | |
89 | problems like this before writing the code. | |
90 | ||
91 | - It's entirely possible that other developers have thought about the | |
92 | problem; they may have ideas for a better solution, and may be willing | |
93 | to help in the creation of that solution. | |
94 | ||
95 | Years of experience with the kernel development community have taught a | |
96 | clear lesson: kernel code which is designed and developed behind closed | |
97 | doors invariably has problems which are only revealed when the code is | |
98 | released into the community. Sometimes these problems are severe, | |
99 | requiring months or years of effort before the code can be brought up to | |
100 | the kernel community's standards. Some examples include: | |
101 | ||
102 | - The Devicescape network stack was designed and implemented for | |
103 | single-processor systems. It could not be merged into the mainline | |
104 | until it was made suitable for multiprocessor systems. Retrofitting | |
105 | locking and such into code is a difficult task; as a result, the merging | |
106 | of this code (now called mac80211) was delayed for over a year. | |
107 | ||
108 | - The Reiser4 filesystem included a number of capabilities which, in the | |
109 | core kernel developers' opinion, should have been implemented in the | |
110 | virtual filesystem layer instead. It also included features which could | |
111 | not easily be implemented without exposing the system to user-caused | |
112 | deadlocks. The late revelation of these problems - and refusal to | |
113 | address some of them - has caused Reiser4 to stay out of the mainline | |
114 | kernel. | |
115 | ||
116 | - The AppArmor security module made use of internal virtual filesystem | |
117 | data structures in ways which were considered to be unsafe and | |
5c050fb9 JC |
118 | unreliable. This concern (among others) kept AppArmor out of the |
119 | mainline for years. | |
75b02146 JC |
120 | |
121 | In each of these cases, a great deal of pain and extra work could have been | |
122 | avoided with some early discussion with the kernel developers. | |
123 | ||
124 | ||
f7c9fe4b MCC |
125 | Who do you talk to? |
126 | ------------------- | |
75b02146 JC |
127 | |
128 | When developers decide to take their plans public, the next question will | |
129 | be: where do we start? The answer is to find the right mailing list(s) and | |
130 | the right maintainer. For mailing lists, the best approach is to look in | |
131 | the MAINTAINERS file for a relevant place to post. If there is a suitable | |
132 | subsystem list, posting there is often preferable to posting on | |
133 | linux-kernel; you are more likely to reach developers with expertise in the | |
134 | relevant subsystem and the environment may be more supportive. | |
135 | ||
136 | Finding maintainers can be a bit harder. Again, the MAINTAINERS file is | |
137 | the place to start. That file tends to not always be up to date, though, | |
138 | and not all subsystems are represented there. The person listed in the | |
139 | MAINTAINERS file may, in fact, not be the person who is actually acting in | |
140 | that role currently. So, when there is doubt about who to contact, a | |
141 | useful trick is to use git (and "git log" in particular) to see who is | |
142 | currently active within the subsystem of interest. Look at who is writing | |
143 | patches, and who, if anybody, is attaching Signed-off-by lines to those | |
144 | patches. Those are the people who will be best placed to help with a new | |
145 | development project. | |
146 | ||
5c050fb9 JC |
147 | The task of finding the right maintainer is sometimes challenging enough |
148 | that the kernel developers have added a script to ease the process: | |
149 | ||
f7c9fe4b MCC |
150 | :: |
151 | ||
5c050fb9 JC |
152 | .../scripts/get_maintainer.pl |
153 | ||
154 | This script will return the current maintainer(s) for a given file or | |
155 | directory when given the "-f" option. If passed a patch on the | |
156 | command line, it will list the maintainers who should probably receive | |
157 | copies of the patch. There are a number of options regulating how hard | |
158 | get_maintainer.pl will search for maintainers; please be careful about | |
159 | using the more aggressive options as you may end up including developers | |
160 | who have no real interest in the code you are modifying. | |
161 | ||
75b02146 JC |
162 | If all else fails, talking to Andrew Morton can be an effective way to |
163 | track down a maintainer for a specific piece of code. | |
164 | ||
165 | ||
f7c9fe4b MCC |
166 | When to post? |
167 | ------------- | |
75b02146 JC |
168 | |
169 | If possible, posting your plans during the early stages can only be | |
170 | helpful. Describe the problem being solved and any plans that have been | |
171 | made on how the implementation will be done. Any information you can | |
172 | provide can help the development community provide useful input on the | |
173 | project. | |
174 | ||
175 | One discouraging thing which can happen at this stage is not a hostile | |
176 | reaction, but, instead, little or no reaction at all. The sad truth of the | |
177 | matter is (1) kernel developers tend to be busy, (2) there is no shortage | |
178 | of people with grand plans and little code (or even prospect of code) to | |
179 | back them up, and (3) nobody is obligated to review or comment on ideas | |
5c050fb9 JC |
180 | posted by others. Beyond that, high-level designs often hide problems |
181 | which are only reviewed when somebody actually tries to implement those | |
182 | designs; for that reason, kernel developers would rather see the code. | |
183 | ||
184 | If a request-for-comments posting yields little in the way of comments, do | |
185 | not assume that it means there is no interest in the project. | |
186 | Unfortunately, you also cannot assume that there are no problems with your | |
187 | idea. The best thing to do in this situation is to proceed, keeping the | |
188 | community informed as you go. | |
75b02146 JC |
189 | |
190 | ||
f7c9fe4b MCC |
191 | Getting official buy-in |
192 | ----------------------- | |
75b02146 JC |
193 | |
194 | If your work is being done in a corporate environment - as most Linux | |
195 | kernel work is - you must, obviously, have permission from suitably | |
196 | empowered managers before you can post your company's plans or code to a | |
197 | public mailing list. The posting of code which has not been cleared for | |
198 | release under a GPL-compatible license can be especially problematic; the | |
199 | sooner that a company's management and legal staff can agree on the posting | |
200 | of a kernel development project, the better off everybody involved will be. | |
201 | ||
202 | Some readers may be thinking at this point that their kernel work is | |
203 | intended to support a product which does not yet have an officially | |
204 | acknowledged existence. Revealing their employer's plans on a public | |
205 | mailing list may not be a viable option. In cases like this, it is worth | |
206 | considering whether the secrecy is really necessary; there is often no real | |
207 | need to keep development plans behind closed doors. | |
208 | ||
209 | That said, there are also cases where a company legitimately cannot | |
210 | disclose its plans early in the development process. Companies with | |
211 | experienced kernel developers may choose to proceed in an open-loop manner | |
212 | on the assumption that they will be able to avoid serious integration | |
213 | problems later. For companies without that sort of in-house expertise, the | |
214 | best option is often to hire an outside developer to review the plans under | |
215 | a non-disclosure agreement. The Linux Foundation operates an NDA program | |
216 | designed to help with this sort of situation; more information can be found | |
217 | at: | |
218 | ||
219 | http://www.linuxfoundation.org/en/NDA_program | |
220 | ||
221 | This kind of review is often enough to avoid serious problems later on | |
222 | without requiring public disclosure of the project. |