]>
Commit | Line | Data |
---|---|---|
13cf67c4 XL |
1 | ## Turning Our Single-Threaded Server into a Multithreaded Server |
2 | ||
3 | Right now, the server will process each request in turn, meaning it won’t | |
4 | process a second connection until the first is finished processing. If the | |
5 | server received more and more requests, this serial execution would be less and | |
6 | less optimal. If the server receives a request that takes a long time to | |
7 | process, subsequent requests will have to wait until the long request is | |
8 | finished, even if the new requests can be processed quickly. We’ll need to fix | |
9 | this, but first, we’ll look at the problem in action. | |
10 | ||
11 | ### Simulating a Slow Request in the Current Server Implementation | |
12 | ||
13 | We’ll look at how a slow-processing request can affect other requests made to | |
14 | our current server implementation. Listing 20-10 implements handling a request | |
15 | to */sleep* with a simulated slow response that will cause the server to sleep | |
16 | for 5 seconds before responding. | |
17 | ||
18 | <span class="filename">Filename: src/main.rs</span> | |
19 | ||
74b04a01 XL |
20 | ```rust,no_run |
21 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-10/src/main.rs:here}} | |
13cf67c4 XL |
22 | ``` |
23 | ||
24 | <span class="caption">Listing 20-10: Simulating a slow request by recognizing | |
25 | */sleep* and sleeping for 5 seconds</span> | |
26 | ||
27 | This code is a bit messy, but it’s good enough for simulation purposes. We | |
28 | created a second request `sleep`, whose data our server recognizes. We added an | |
29 | `else if` after the `if` block to check for the request to */sleep*. When that | |
30 | request is received, the server will sleep for 5 seconds before rendering the | |
31 | successful HTML page. | |
32 | ||
33 | You can see how primitive our server is: real libraries would handle the | |
34 | recognition of multiple requests in a much less verbose way! | |
35 | ||
36 | Start the server using `cargo run`. Then open two browser windows: one for | |
37 | *http://127.0.0.1:7878/* and the other for *http://127.0.0.1:7878/sleep*. If | |
38 | you enter the */* URI a few times, as before, you’ll see it respond quickly. | |
39 | But if you enter */sleep* and then load */*, you’ll see that */* waits until | |
40 | `sleep` has slept for its full 5 seconds before loading. | |
41 | ||
42 | There are multiple ways we could change how our web server works to avoid | |
43 | having more requests back up behind a slow request; the one we’ll implement is | |
44 | a thread pool. | |
45 | ||
46 | ### Improving Throughput with a Thread Pool | |
47 | ||
48 | A *thread pool* is a group of spawned threads that are waiting and ready to | |
49 | handle a task. When the program receives a new task, it assigns one of the | |
50 | threads in the pool to the task, and that thread will process the task. The | |
51 | remaining threads in the pool are available to handle any other tasks that come | |
52 | in while the first thread is processing. When the first thread is done | |
53 | processing its task, it’s returned to the pool of idle threads, ready to handle | |
54 | a new task. A thread pool allows you to process connections concurrently, | |
55 | increasing the throughput of your server. | |
56 | ||
57 | We’ll limit the number of threads in the pool to a small number to protect us | |
58 | from Denial of Service (DoS) attacks; if we had our program create a new thread | |
59 | for each request as it came in, someone making 10 million requests to our | |
60 | server could create havoc by using up all our server’s resources and grinding | |
61 | the processing of requests to a halt. | |
62 | ||
63 | Rather than spawning unlimited threads, we’ll have a fixed number of threads | |
64 | waiting in the pool. As requests come in, they’ll be sent to the pool for | |
65 | processing. The pool will maintain a queue of incoming requests. Each of the | |
66 | threads in the pool will pop off a request from this queue, handle the request, | |
67 | and then ask the queue for another request. With this design, we can process | |
68 | `N` requests concurrently, where `N` is the number of threads. If each thread | |
69 | is responding to a long-running request, subsequent requests can still back up | |
70 | in the queue, but we’ve increased the number of long-running requests we can | |
71 | handle before reaching that point. | |
72 | ||
73 | This technique is just one of many ways to improve the throughput of a web | |
74 | server. Other options you might explore are the fork/join model and the | |
75 | single-threaded async I/O model. If you’re interested in this topic, you can | |
76 | read more about other solutions and try to implement them in Rust; with a | |
77 | low-level language like Rust, all of these options are possible. | |
78 | ||
79 | Before we begin implementing a thread pool, let’s talk about what using the | |
80 | pool should look like. When you’re trying to design code, writing the client | |
81 | interface first can help guide your design. Write the API of the code so it’s | |
82 | structured in the way you want to call it; then implement the functionality | |
83 | within that structure rather than implementing the functionality and then | |
84 | designing the public API. | |
85 | ||
86 | Similar to how we used test-driven development in the project in Chapter 12, | |
87 | we’ll use compiler-driven development here. We’ll write the code that calls the | |
88 | functions we want, and then we’ll look at errors from the compiler to determine | |
89 | what we should change next to get the code to work. | |
90 | ||
91 | #### Code Structure If We Could Spawn a Thread for Each Request | |
92 | ||
93 | First, let’s explore how our code might look if it did create a new thread for | |
94 | every connection. As mentioned earlier, this isn’t our final plan due to the | |
95 | problems with potentially spawning an unlimited number of threads, but it is a | |
96 | starting point. Listing 20-11 shows the changes to make to `main` to spawn a | |
97 | new thread to handle each stream within the `for` loop. | |
98 | ||
99 | <span class="filename">Filename: src/main.rs</span> | |
100 | ||
101 | ```rust,no_run | |
74b04a01 | 102 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-11/src/main.rs:here}} |
13cf67c4 XL |
103 | ``` |
104 | ||
105 | <span class="caption">Listing 20-11: Spawning a new thread for each | |
106 | stream</span> | |
107 | ||
108 | As you learned in Chapter 16, `thread::spawn` will create a new thread and then | |
109 | run the code in the closure in the new thread. If you run this code and load | |
110 | */sleep* in your browser, then */* in two more browser tabs, you’ll indeed see | |
111 | that the requests to */* don’t have to wait for */sleep* to finish. But as we | |
112 | mentioned, this will eventually overwhelm the system because you’d be making | |
113 | new threads without any limit. | |
114 | ||
115 | #### Creating a Similar Interface for a Finite Number of Threads | |
116 | ||
117 | We want our thread pool to work in a similar, familiar way so switching from | |
118 | threads to a thread pool doesn’t require large changes to the code that uses | |
119 | our API. Listing 20-12 shows the hypothetical interface for a `ThreadPool` | |
120 | struct we want to use instead of `thread::spawn`. | |
121 | ||
122 | <span class="filename">Filename: src/main.rs</span> | |
123 | ||
74b04a01 XL |
124 | ```rust,ignore,does_not_compile |
125 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-12/src/main.rs:here}} | |
13cf67c4 XL |
126 | ``` |
127 | ||
128 | <span class="caption">Listing 20-12: Our ideal `ThreadPool` interface</span> | |
129 | ||
130 | We use `ThreadPool::new` to create a new thread pool with a configurable number | |
131 | of threads, in this case four. Then, in the `for` loop, `pool.execute` has a | |
132 | similar interface as `thread::spawn` in that it takes a closure the pool should | |
133 | run for each stream. We need to implement `pool.execute` so it takes the | |
134 | closure and gives it to a thread in the pool to run. This code won’t yet | |
135 | compile, but we’ll try so the compiler can guide us in how to fix it. | |
136 | ||
137 | #### Building the `ThreadPool` Struct Using Compiler Driven Development | |
138 | ||
139 | Make the changes in Listing 20-12 to *src/main.rs*, and then let’s use the | |
140 | compiler errors from `cargo check` to drive our development. Here is the first | |
141 | error we get: | |
142 | ||
143 | ```text | |
74b04a01 | 144 | {{#include ../listings/ch20-web-server/listing-20-12/output.txt}} |
13cf67c4 XL |
145 | ``` |
146 | ||
147 | Great! This error tells us we need a `ThreadPool` type or module, so we’ll | |
148 | build one now. Our `ThreadPool` implementation will be independent of the kind | |
149 | of work our web server is doing. So, let’s switch the `hello` crate from a | |
150 | binary crate to a library crate to hold our `ThreadPool` implementation. After | |
151 | we change to a library crate, we could also use the separate thread pool | |
152 | library for any work we want to do using a thread pool, not just for serving | |
153 | web requests. | |
154 | ||
155 | Create a *src/lib.rs* that contains the following, which is the simplest | |
156 | definition of a `ThreadPool` struct that we can have for now: | |
157 | ||
158 | <span class="filename">Filename: src/lib.rs</span> | |
159 | ||
160 | ```rust | |
74b04a01 | 161 | {{#rustdoc_include ../listings/ch20-web-server/no-listing-01-define-threadpool-struct/src/lib.rs}} |
13cf67c4 XL |
162 | ``` |
163 | ||
164 | Then create a new directory, *src/bin*, and move the binary crate rooted in | |
165 | *src/main.rs* into *src/bin/main.rs*. Doing so will make the library crate the | |
166 | primary crate in the *hello* directory; we can still run the binary in | |
167 | *src/bin/main.rs* using `cargo run`. After moving the *main.rs* file, edit it | |
168 | to bring the library crate in and bring `ThreadPool` into scope by adding the | |
169 | following code to the top of *src/bin/main.rs*: | |
170 | ||
171 | <span class="filename">Filename: src/bin/main.rs</span> | |
172 | ||
173 | ```rust,ignore | |
74b04a01 | 174 | {{#rustdoc_include ../listings/ch20-web-server/no-listing-01-define-threadpool-struct/src/bin/main.rs:here}} |
13cf67c4 XL |
175 | ``` |
176 | ||
177 | This code still won’t work, but let’s check it again to get the next error that | |
178 | we need to address: | |
179 | ||
180 | ```text | |
74b04a01 | 181 | {{#include ../listings/ch20-web-server/no-listing-01-define-threadpool-struct/output.txt}} |
13cf67c4 XL |
182 | ``` |
183 | ||
184 | This error indicates that next we need to create an associated function named | |
185 | `new` for `ThreadPool`. We also know that `new` needs to have one parameter | |
186 | that can accept `4` as an argument and should return a `ThreadPool` instance. | |
187 | Let’s implement the simplest `new` function that will have those | |
188 | characteristics: | |
189 | ||
190 | <span class="filename">Filename: src/lib.rs</span> | |
191 | ||
192 | ```rust | |
74b04a01 | 193 | {{#rustdoc_include ../listings/ch20-web-server/no-listing-02-impl-threadpool-new/src/lib.rs:here}} |
13cf67c4 XL |
194 | ``` |
195 | ||
196 | We chose `usize` as the type of the `size` parameter, because we know that a | |
197 | negative number of threads doesn’t make any sense. We also know we’ll use this | |
198 | 4 as the number of elements in a collection of threads, which is what the | |
9fa01778 XL |
199 | `usize` type is for, as discussed in the [“Integer Types”][integer-types]<!-- |
200 | ignore --> section of Chapter 3. | |
13cf67c4 XL |
201 | |
202 | Let’s check the code again: | |
203 | ||
204 | ```text | |
74b04a01 | 205 | {{#include ../listings/ch20-web-server/no-listing-02-impl-threadpool-new/output.txt}} |
13cf67c4 XL |
206 | ``` |
207 | ||
74b04a01 XL |
208 | Now the error occurs because we don’t have an `execute` method on `ThreadPool`. |
209 | Recall from the [“Creating a Similar Interface for a Finite Number of | |
9fa01778 XL |
210 | Threads”](#creating-a-similar-interface-for-a-finite-number-of-threads)<!-- |
211 | ignore --> section that we decided our thread pool should have an interface | |
212 | similar to `thread::spawn`. In addition, we’ll implement the `execute` function | |
213 | so it takes the closure it’s given and gives it to an idle thread in the pool | |
214 | to run. | |
13cf67c4 XL |
215 | |
216 | We’ll define the `execute` method on `ThreadPool` to take a closure as a | |
9fa01778 XL |
217 | parameter. Recall from the [“Storing Closures Using Generic Parameters and the |
218 | `Fn` Traits”][storing-closures-using-generic-parameters-and-the-fn-traits]<!-- | |
219 | ignore --> section in Chapter 13 that we can take closures as parameters with | |
13cf67c4 XL |
220 | three different traits: `Fn`, `FnMut`, and `FnOnce`. We need to decide which |
221 | kind of closure to use here. We know we’ll end up doing something similar to | |
222 | the standard library `thread::spawn` implementation, so we can look at what | |
223 | bounds the signature of `thread::spawn` has on its parameter. The documentation | |
224 | shows us the following: | |
225 | ||
226 | ```rust,ignore | |
227 | pub fn spawn<F, T>(f: F) -> JoinHandle<T> | |
228 | where | |
229 | F: FnOnce() -> T + Send + 'static, | |
230 | T: Send + 'static | |
231 | ``` | |
232 | ||
233 | The `F` type parameter is the one we’re concerned with here; the `T` type | |
234 | parameter is related to the return value, and we’re not concerned with that. We | |
235 | can see that `spawn` uses `FnOnce` as the trait bound on `F`. This is probably | |
236 | what we want as well, because we’ll eventually pass the argument we get in | |
237 | `execute` to `spawn`. We can be further confident that `FnOnce` is the trait we | |
238 | want to use because the thread for running a request will only execute that | |
239 | request’s closure one time, which matches the `Once` in `FnOnce`. | |
240 | ||
241 | The `F` type parameter also has the trait bound `Send` and the lifetime bound | |
242 | `'static`, which are useful in our situation: we need `Send` to transfer the | |
243 | closure from one thread to another and `'static` because we don’t know how long | |
244 | the thread will take to execute. Let’s create an `execute` method on | |
245 | `ThreadPool` that will take a generic parameter of type `F` with these bounds: | |
246 | ||
247 | <span class="filename">Filename: src/lib.rs</span> | |
248 | ||
249 | ```rust | |
74b04a01 | 250 | {{#rustdoc_include ../listings/ch20-web-server/no-listing-03-define-execute/src/lib.rs:here}} |
13cf67c4 XL |
251 | ``` |
252 | ||
253 | We still use the `()` after `FnOnce` because this `FnOnce` represents a closure | |
e74abb32 | 254 | that takes no parameters and returns the unit type `()`. Just like function |
13cf67c4 XL |
255 | definitions, the return type can be omitted from the signature, but even if we |
256 | have no parameters, we still need the parentheses. | |
257 | ||
258 | Again, this is the simplest implementation of the `execute` method: it does | |
259 | nothing, but we’re trying only to make our code compile. Let’s check it again: | |
260 | ||
261 | ```text | |
74b04a01 | 262 | {{#include ../listings/ch20-web-server/no-listing-03-define-execute/output.txt}} |
13cf67c4 XL |
263 | ``` |
264 | ||
74b04a01 XL |
265 | It compiles! But note that if you try `cargo run` and make a request in the |
266 | browser, you’ll see the errors in the browser that we saw at the beginning of | |
267 | the chapter. Our library isn’t actually calling the closure passed to `execute` | |
268 | yet! | |
13cf67c4 XL |
269 | |
270 | > Note: A saying you might hear about languages with strict compilers, such as | |
271 | > Haskell and Rust, is “if the code compiles, it works.” But this saying is not | |
272 | > universally true. Our project compiles, but it does absolutely nothing! If we | |
273 | > were building a real, complete project, this would be a good time to start | |
274 | > writing unit tests to check that the code compiles *and* has the behavior we | |
275 | > want. | |
276 | ||
277 | #### Validating the Number of Threads in `new` | |
278 | ||
74b04a01 XL |
279 | We aren’t doing anything with the parameters to `new` and `execute`. Let’s |
280 | implement the bodies of these functions with the behavior we want. To start, | |
281 | let’s think about `new`. Earlier we chose an unsigned type for the `size` | |
282 | parameter, because a pool with a negative number of threads makes no sense. | |
283 | However, a pool with zero threads also makes no sense, yet zero is a perfectly | |
284 | valid `usize`. We’ll add code to check that `size` is greater than zero before | |
285 | we return a `ThreadPool` instance and have the program panic if it receives a | |
286 | zero by using the `assert!` macro, as shown in Listing 20-13. | |
13cf67c4 XL |
287 | |
288 | <span class="filename">Filename: src/lib.rs</span> | |
289 | ||
290 | ```rust | |
74b04a01 | 291 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-13/src/lib.rs:here}} |
13cf67c4 XL |
292 | ``` |
293 | ||
294 | <span class="caption">Listing 20-13: Implementing `ThreadPool::new` to panic if | |
295 | `size` is zero</span> | |
296 | ||
297 | We’ve added some documentation for our `ThreadPool` with doc comments. Note | |
298 | that we followed good documentation practices by adding a section that calls | |
299 | out the situations in which our function can panic, as discussed in Chapter 14. | |
300 | Try running `cargo doc --open` and clicking the `ThreadPool` struct to see what | |
301 | the generated docs for `new` look like! | |
302 | ||
303 | Instead of adding the `assert!` macro as we’ve done here, we could make `new` | |
304 | return a `Result` like we did with `Config::new` in the I/O project in Listing | |
305 | 12-9. But we’ve decided in this case that trying to create a thread pool | |
306 | without any threads should be an unrecoverable error. If you’re feeling | |
307 | ambitious, try to write a version of `new` with the following signature to | |
308 | compare both versions: | |
309 | ||
310 | ```rust,ignore | |
311 | pub fn new(size: usize) -> Result<ThreadPool, PoolCreationError> { | |
312 | ``` | |
313 | ||
314 | #### Creating Space to Store the Threads | |
315 | ||
316 | Now that we have a way to know we have a valid number of threads to store in | |
317 | the pool, we can create those threads and store them in the `ThreadPool` struct | |
318 | before returning it. But how do we “store” a thread? Let’s take another look at | |
319 | the `thread::spawn` signature: | |
320 | ||
321 | ```rust,ignore | |
322 | pub fn spawn<F, T>(f: F) -> JoinHandle<T> | |
323 | where | |
324 | F: FnOnce() -> T + Send + 'static, | |
325 | T: Send + 'static | |
326 | ``` | |
327 | ||
328 | The `spawn` function returns a `JoinHandle<T>`, where `T` is the type that the | |
329 | closure returns. Let’s try using `JoinHandle` too and see what happens. In our | |
330 | case, the closures we’re passing to the thread pool will handle the connection | |
331 | and not return anything, so `T` will be the unit type `()`. | |
332 | ||
333 | The code in Listing 20-14 will compile but doesn’t create any threads yet. | |
334 | We’ve changed the definition of `ThreadPool` to hold a vector of | |
335 | `thread::JoinHandle<()>` instances, initialized the vector with a capacity of | |
336 | `size`, set up a `for` loop that will run some code to create the threads, and | |
337 | returned a `ThreadPool` instance containing them. | |
338 | ||
339 | <span class="filename">Filename: src/lib.rs</span> | |
340 | ||
341 | ```rust,ignore,not_desired_behavior | |
74b04a01 | 342 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-14/src/lib.rs:here}} |
13cf67c4 XL |
343 | ``` |
344 | ||
345 | <span class="caption">Listing 20-14: Creating a vector for `ThreadPool` to hold | |
346 | the threads</span> | |
347 | ||
348 | We’ve brought `std::thread` into scope in the library crate, because we’re | |
349 | using `thread::JoinHandle` as the type of the items in the vector in | |
350 | `ThreadPool`. | |
351 | ||
352 | Once a valid size is received, our `ThreadPool` creates a new vector that can | |
353 | hold `size` items. We haven’t used the `with_capacity` function in this book | |
354 | yet, which performs the same task as `Vec::new` but with an important | |
355 | difference: it preallocates space in the vector. Because we know we need to | |
356 | store `size` elements in the vector, doing this allocation up front is slightly | |
357 | more efficient than using `Vec::new`, which resizes itself as elements are | |
358 | inserted. | |
359 | ||
360 | When you run `cargo check` again, you’ll get a few more warnings, but it should | |
361 | succeed. | |
362 | ||
363 | #### A `Worker` Struct Responsible for Sending Code from the `ThreadPool` to a Thread | |
364 | ||
365 | We left a comment in the `for` loop in Listing 20-14 regarding the creation of | |
366 | threads. Here, we’ll look at how we actually create threads. The standard | |
367 | library provides `thread::spawn` as a way to create threads, and | |
368 | `thread::spawn` expects to get some code the thread should run as soon as the | |
369 | thread is created. However, in our case, we want to create the threads and have | |
370 | them *wait* for code that we’ll send later. The standard library’s | |
371 | implementation of threads doesn’t include any way to do that; we have to | |
372 | implement it manually. | |
373 | ||
374 | We’ll implement this behavior by introducing a new data structure between the | |
375 | `ThreadPool` and the threads that will manage this new behavior. We’ll call | |
376 | this data structure `Worker`, which is a common term in pooling | |
377 | implementations. Think of people working in the kitchen at a restaurant: the | |
378 | workers wait until orders come in from customers, and then they’re responsible | |
379 | for taking those orders and filling them. | |
380 | ||
381 | Instead of storing a vector of `JoinHandle<()>` instances in the thread pool, | |
382 | we’ll store instances of the `Worker` struct. Each `Worker` will store a single | |
383 | `JoinHandle<()>` instance. Then we’ll implement a method on `Worker` that will | |
384 | take a closure of code to run and send it to the already running thread for | |
385 | execution. We’ll also give each worker an `id` so we can distinguish between | |
386 | the different workers in the pool when logging or debugging. | |
387 | ||
388 | Let’s make the following changes to what happens when we create a `ThreadPool`. | |
389 | We’ll implement the code that sends the closure to the thread after we have | |
390 | `Worker` set up in this way: | |
391 | ||
392 | 1. Define a `Worker` struct that holds an `id` and a `JoinHandle<()>`. | |
393 | 2. Change `ThreadPool` to hold a vector of `Worker` instances. | |
394 | 3. Define a `Worker::new` function that takes an `id` number and returns a | |
395 | `Worker` instance that holds the `id` and a thread spawned with an empty | |
396 | closure. | |
397 | 4. In `ThreadPool::new`, use the `for` loop counter to generate an `id`, create | |
398 | a new `Worker` with that `id`, and store the worker in the vector. | |
399 | ||
400 | If you’re up for a challenge, try implementing these changes on your own before | |
401 | looking at the code in Listing 20-15. | |
402 | ||
403 | Ready? Here is Listing 20-15 with one way to make the preceding modifications. | |
404 | ||
405 | <span class="filename">Filename: src/lib.rs</span> | |
406 | ||
407 | ```rust | |
74b04a01 | 408 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-15/src/lib.rs:here}} |
13cf67c4 XL |
409 | ``` |
410 | ||
411 | <span class="caption">Listing 20-15: Modifying `ThreadPool` to hold `Worker` | |
412 | instances instead of holding threads directly</span> | |
413 | ||
414 | We’ve changed the name of the field on `ThreadPool` from `threads` to `workers` | |
415 | because it’s now holding `Worker` instances instead of `JoinHandle<()>` | |
416 | instances. We use the counter in the `for` loop as an argument to | |
417 | `Worker::new`, and we store each new `Worker` in the vector named `workers`. | |
418 | ||
419 | External code (like our server in *src/bin/main.rs*) doesn’t need to know the | |
420 | implementation details regarding using a `Worker` struct within `ThreadPool`, | |
421 | so we make the `Worker` struct and its `new` function private. The | |
422 | `Worker::new` function uses the `id` we give it and stores a `JoinHandle<()>` | |
423 | instance that is created by spawning a new thread using an empty closure. | |
424 | ||
425 | This code will compile and will store the number of `Worker` instances we | |
426 | specified as an argument to `ThreadPool::new`. But we’re *still* not processing | |
427 | the closure that we get in `execute`. Let’s look at how to do that next. | |
428 | ||
429 | #### Sending Requests to Threads via Channels | |
430 | ||
431 | Now we’ll tackle the problem that the closures given to `thread::spawn` do | |
432 | absolutely nothing. Currently, we get the closure we want to execute in the | |
433 | `execute` method. But we need to give `thread::spawn` a closure to run when we | |
434 | create each `Worker` during the creation of the `ThreadPool`. | |
435 | ||
436 | We want the `Worker` structs that we just created to fetch code to run from a | |
437 | queue held in the `ThreadPool` and send that code to its thread to run. | |
438 | ||
439 | In Chapter 16, you learned about *channels*—a simple way to communicate between | |
440 | two threads—that would be perfect for this use case. We’ll use a channel to | |
441 | function as the queue of jobs, and `execute` will send a job from the | |
442 | `ThreadPool` to the `Worker` instances, which will send the job to its thread. | |
443 | Here is the plan: | |
444 | ||
445 | 1. The `ThreadPool` will create a channel and hold on to the sending side of | |
446 | the channel. | |
447 | 2. Each `Worker` will hold on to the receiving side of the channel. | |
448 | 3. We’ll create a new `Job` struct that will hold the closures we want to send | |
449 | down the channel. | |
450 | 4. The `execute` method will send the job it wants to execute down the sending | |
451 | side of the channel. | |
452 | 5. In its thread, the `Worker` will loop over its receiving side of the channel | |
453 | and execute the closures of any jobs it receives. | |
454 | ||
455 | Let’s start by creating a channel in `ThreadPool::new` and holding the sending | |
456 | side in the `ThreadPool` instance, as shown in Listing 20-16. The `Job` struct | |
457 | doesn’t hold anything for now but will be the type of item we’re sending down | |
458 | the channel. | |
459 | ||
460 | <span class="filename">Filename: src/lib.rs</span> | |
461 | ||
462 | ```rust | |
74b04a01 | 463 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-16/src/lib.rs:here}} |
13cf67c4 XL |
464 | ``` |
465 | ||
466 | <span class="caption">Listing 20-16: Modifying `ThreadPool` to store the | |
467 | sending end of a channel that sends `Job` instances</span> | |
468 | ||
469 | In `ThreadPool::new`, we create our new channel and have the pool hold the | |
470 | sending end. This will successfully compile, still with warnings. | |
471 | ||
472 | Let’s try passing a receiving end of the channel into each worker as the thread | |
473 | pool creates the channel. We know we want to use the receiving end in the | |
474 | thread that the workers spawn, so we’ll reference the `receiver` parameter in | |
475 | the closure. The code in Listing 20-17 won’t quite compile yet. | |
476 | ||
477 | <span class="filename">Filename: src/lib.rs</span> | |
478 | ||
479 | ```rust,ignore,does_not_compile | |
74b04a01 | 480 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-17/src/lib.rs:here}} |
13cf67c4 XL |
481 | ``` |
482 | ||
483 | <span class="caption">Listing 20-17: Passing the receiving end of the channel | |
484 | to the workers</span> | |
485 | ||
486 | We’ve made some small and straightforward changes: we pass the receiving end of | |
487 | the channel into `Worker::new`, and then we use it inside the closure. | |
488 | ||
489 | When we try to check this code, we get this error: | |
490 | ||
491 | ```text | |
74b04a01 | 492 | {{#include ../listings/ch20-web-server/listing-20-17/output.txt}} |
13cf67c4 XL |
493 | ``` |
494 | ||
495 | The code is trying to pass `receiver` to multiple `Worker` instances. This | |
496 | won’t work, as you’ll recall from Chapter 16: the channel implementation that | |
497 | Rust provides is multiple *producer*, single *consumer*. This means we can’t | |
498 | just clone the consuming end of the channel to fix this code. Even if we could, | |
499 | that is not the technique we would want to use; instead, we want to distribute | |
500 | the jobs across threads by sharing the single `receiver` among all the workers. | |
501 | ||
502 | Additionally, taking a job off the channel queue involves mutating the | |
503 | `receiver`, so the threads need a safe way to share and modify `receiver`; | |
504 | otherwise, we might get race conditions (as covered in Chapter 16). | |
505 | ||
506 | Recall the thread-safe smart pointers discussed in Chapter 16: to share | |
507 | ownership across multiple threads and allow the threads to mutate the value, we | |
508 | need to use `Arc<Mutex<T>>`. The `Arc` type will let multiple workers own the | |
509 | receiver, and `Mutex` will ensure that only one worker gets a job from the | |
510 | receiver at a time. Listing 20-18 shows the changes we need to make. | |
511 | ||
512 | <span class="filename">Filename: src/lib.rs</span> | |
513 | ||
514 | ```rust | |
74b04a01 | 515 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-18/src/lib.rs:here}} |
13cf67c4 XL |
516 | ``` |
517 | ||
518 | <span class="caption">Listing 20-18: Sharing the receiving end of the channel | |
519 | among the workers using `Arc` and `Mutex`</span> | |
520 | ||
521 | In `ThreadPool::new`, we put the receiving end of the channel in an `Arc` and a | |
522 | `Mutex`. For each new worker, we clone the `Arc` to bump the reference count so | |
523 | the workers can share ownership of the receiving end. | |
524 | ||
525 | With these changes, the code compiles! We’re getting there! | |
526 | ||
527 | #### Implementing the `execute` Method | |
528 | ||
529 | Let’s finally implement the `execute` method on `ThreadPool`. We’ll also change | |
530 | `Job` from a struct to a type alias for a trait object that holds the type of | |
9fa01778 XL |
531 | closure that `execute` receives. As discussed in the [“Creating Type Synonyms |
532 | with Type Aliases”][creating-type-synonyms-with-type-aliases]<!-- ignore --> | |
533 | section of Chapter 19, type aliases allow us to make long types shorter. Look | |
534 | at Listing 20-19. | |
13cf67c4 XL |
535 | |
536 | <span class="filename">Filename: src/lib.rs</span> | |
537 | ||
538 | ```rust | |
74b04a01 | 539 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-19/src/lib.rs:here}} |
13cf67c4 XL |
540 | ``` |
541 | ||
542 | <span class="caption">Listing 20-19: Creating a `Job` type alias for a `Box` | |
543 | that holds each closure and then sending the job down the channel</span> | |
544 | ||
545 | After creating a new `Job` instance using the closure we get in `execute`, we | |
546 | send that job down the sending end of the channel. We’re calling `unwrap` on | |
547 | `send` for the case that sending fails. This might happen if, for example, we | |
548 | stop all our threads from executing, meaning the receiving end has stopped | |
549 | receiving new messages. At the moment, we can’t stop our threads from | |
550 | executing: our threads continue executing as long as the pool exists. The | |
551 | reason we use `unwrap` is that we know the failure case won’t happen, but the | |
552 | compiler doesn’t know that. | |
553 | ||
554 | But we’re not quite done yet! In the worker, our closure being passed to | |
555 | `thread::spawn` still only *references* the receiving end of the channel. | |
556 | Instead, we need the closure to loop forever, asking the receiving end of the | |
557 | channel for a job and running the job when it gets one. Let’s make the change | |
558 | shown in Listing 20-20 to `Worker::new`. | |
559 | ||
560 | <span class="filename">Filename: src/lib.rs</span> | |
561 | ||
74b04a01 XL |
562 | ```rust |
563 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-20/src/lib.rs:here}} | |
13cf67c4 XL |
564 | ``` |
565 | ||
566 | <span class="caption">Listing 20-20: Receiving and executing the jobs in the | |
567 | worker’s thread</span> | |
568 | ||
569 | Here, we first call `lock` on the `receiver` to acquire the mutex, and then we | |
570 | call `unwrap` to panic on any errors. Acquiring a lock might fail if the mutex | |
571 | is in a *poisoned* state, which can happen if some other thread panicked while | |
572 | holding the lock rather than releasing the lock. In this situation, calling | |
573 | `unwrap` to have this thread panic is the correct action to take. Feel free to | |
574 | change this `unwrap` to an `expect` with an error message that is meaningful to | |
575 | you. | |
576 | ||
577 | If we get the lock on the mutex, we call `recv` to receive a `Job` from the | |
578 | channel. A final `unwrap` moves past any errors here as well, which might occur | |
579 | if the thread holding the sending side of the channel has shut down, similar to | |
580 | how the `send` method returns `Err` if the receiving side shuts down. | |
581 | ||
582 | The call to `recv` blocks, so if there is no job yet, the current thread will | |
583 | wait until a job becomes available. The `Mutex<T>` ensures that only one | |
584 | `Worker` thread at a time is trying to request a job. | |
585 | ||
13cf67c4 XL |
586 | With the implementation of this trick, our thread pool is in a working state! |
587 | Give it a `cargo run` and make some requests: | |
588 | ||
74b04a01 XL |
589 | <!-- manual-regeneration |
590 | cd listings/ch20-web-server/listing-20-20 | |
591 | cargo run | |
592 | make some requests to 127.0.0.1:7878 | |
593 | Can't automate because the output depends on making requests | |
594 | --> | |
595 | ||
13cf67c4 XL |
596 | ```text |
597 | $ cargo run | |
598 | Compiling hello v0.1.0 (file:///projects/hello) | |
74b04a01 | 599 | warning: field is never read: `workers` |
13cf67c4 XL |
600 | --> src/lib.rs:7:5 |
601 | | | |
602 | 7 | workers: Vec<Worker>, | |
603 | | ^^^^^^^^^^^^^^^^^^^^ | |
604 | | | |
74b04a01 | 605 | = note: `#[warn(dead_code)]` on by default |
13cf67c4 | 606 | |
74b04a01 XL |
607 | warning: field is never read: `id` |
608 | --> src/lib.rs:48:5 | |
13cf67c4 | 609 | | |
74b04a01 | 610 | 48 | id: usize, |
13cf67c4 | 611 | | ^^^^^^^^^ |
13cf67c4 | 612 | |
74b04a01 XL |
613 | warning: field is never read: `thread` |
614 | --> src/lib.rs:49:5 | |
13cf67c4 | 615 | | |
74b04a01 | 616 | 49 | thread: thread::JoinHandle<()>, |
13cf67c4 | 617 | | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
13cf67c4 | 618 | |
74b04a01 XL |
619 | Finished dev [unoptimized + debuginfo] target(s) in 1.40s |
620 | Running `target/debug/main` | |
13cf67c4 XL |
621 | Worker 0 got a job; executing. |
622 | Worker 2 got a job; executing. | |
623 | Worker 1 got a job; executing. | |
624 | Worker 3 got a job; executing. | |
625 | Worker 0 got a job; executing. | |
626 | Worker 2 got a job; executing. | |
627 | Worker 1 got a job; executing. | |
628 | Worker 3 got a job; executing. | |
629 | Worker 0 got a job; executing. | |
630 | Worker 2 got a job; executing. | |
631 | ``` | |
632 | ||
633 | Success! We now have a thread pool that executes connections asynchronously. | |
634 | There are never more than four threads created, so our system won’t get | |
635 | overloaded if the server receives a lot of requests. If we make a request to | |
636 | */sleep*, the server will be able to serve other requests by having another | |
637 | thread run them. | |
638 | ||
9fa01778 XL |
639 | > Note: if you open */sleep* in multiple browser windows simultaneously, they |
640 | > might load one at a time in 5 second intervals. Some web browsers execute | |
641 | > multiple instances of the same request sequentially for caching reasons. This | |
642 | > limitation is not caused by our web server. | |
13cf67c4 XL |
643 | |
644 | After learning about the `while let` loop in Chapter 18, you might be wondering | |
60c5eb7d | 645 | why we didn’t write the worker thread code as shown in Listing 20-21. |
13cf67c4 XL |
646 | |
647 | <span class="filename">Filename: src/lib.rs</span> | |
648 | ||
649 | ```rust,ignore,not_desired_behavior | |
74b04a01 | 650 | {{#rustdoc_include ../listings/ch20-web-server/listing-20-21/src/lib.rs:here}} |
13cf67c4 XL |
651 | ``` |
652 | ||
60c5eb7d | 653 | <span class="caption">Listing 20-21: An alternative implementation of |
13cf67c4 XL |
654 | `Worker::new` using `while let`</span> |
655 | ||
656 | This code compiles and runs but doesn’t result in the desired threading | |
657 | behavior: a slow request will still cause other requests to wait to be | |
658 | processed. The reason is somewhat subtle: the `Mutex` struct has no public | |
659 | `unlock` method because the ownership of the lock is based on the lifetime of | |
660 | the `MutexGuard<T>` within the `LockResult<MutexGuard<T>>` that the `lock` | |
661 | method returns. At compile time, the borrow checker can then enforce the rule | |
662 | that a resource guarded by a `Mutex` cannot be accessed unless we hold the | |
663 | lock. But this implementation can also result in the lock being held longer | |
664 | than intended if we don’t think carefully about the lifetime of the | |
74b04a01 XL |
665 | `MutexGuard<T>`. Because the values in the `while let` expression remain in |
666 | scope for the duration of the block, the lock remains held for the duration of | |
667 | the call to `job()`, meaning other workers cannot receive jobs. | |
668 | ||
669 | By using `loop` instead and acquiring the lock without assigning to a variable, | |
670 | the temporary `MutexGuard` returned from the `lock` method is dropped as soon | |
671 | as the `let job` statement ends. This ensures that the lock is held during the | |
672 | call to `recv`, but it is released before the call to `job()`, allowing | |
673 | multiple requests to be serviced concurrently. | |
9fa01778 XL |
674 | |
675 | [creating-type-synonyms-with-type-aliases]: | |
676 | ch19-04-advanced-types.html#creating-type-synonyms-with-type-aliases | |
677 | [integer-types]: ch03-02-data-types.html#integer-types | |
678 | [storing-closures-using-generic-parameters-and-the-fn-traits]: | |
679 | ch13-01-closures.html#storing-closures-using-generic-parameters-and-the-fn-traits |