Auto merge of #7838 - ehuss:fix-memory-rustc-output, r=alexcrichton

Avoid buffering large amounts of rustc output.

If `rustc` prints out a lot of information (such as with `RUSTC_LOG`, or a huge number of diagnostics), cargo would buffer up large amounts of that in memory.  For normal builds, this would happen if the terminal does not print fast enough. For "fresh" replay, *everything* was being buffered.

There are two issues:
1. There is no back-pressure on the mpsc queue.  If messages come in faster than they can be processed, it grows without bounds.
2. The cache-replay code runs in the "fresh" code path which does not spawn a thread. Thus the main thread was blocked and unable to process `Message`s while the replay is happening.

The solution here is to use a bounded queue, and to always spawn a thread for the "fresh" case.

The main concern here is performance.  Previously the "fresh" jobs avoided spawning a thread to improve performance.  I did a fair bit of profiling to understand the impact, using projects with anywhere from 100 to 500 units.  On my macOS machine, I found spawning a thread to be slightly faster (1-5%).  On Linux and Windows, it was generally about 0 to 5% slower.  It might be helpful for others to profile it on their own system.

I'm on the fence for the cost/benefit here.  It seems generally good to reduce memory usage, but the slight performance hit is disappointing.  I tried several other approaches to fix this, all with worse trade offs (I can discuss them if interested).

Fixes #6197