Thread Pools
-
Creating a thread requires a lot of work
- Create an execution stack for the thread
- Call a system API
- The operating system creates internal data to manage the thread
- The scheduler executes the thread
- A context switch occurs to run the thread
-
Creating a new thread can take 10.000 times as long as calling a function directly
-
Is there any way we can "recycle" threads?
- Reduce or avoid this overhead
Thread Pool Motivation
- We want to make full use of all our processor cores
- Every core should be running one of our threads
- Except perhaps for
main() - And the operating system
- Except perhaps for
- Difficult to achieve with
std::async()- Need to keep track of the number of threads started
Typing Pool
-
Before computers, all business correspondence had to be typed
- Senior managers had a dedicated secretary
- Other employees used a typing pool
-
Each typist is working on a letter
-
A new work item arrives
- It is added to a pile of pending work
- A typist becomes free
- The typist takes the next item and starts working on it
Thread Pool Structure
-
Container of C++ thread objects
- Has a fixed size
- Usually matched to the number of cores on the machine
- Found by calling
std::thread::hardware_concurrency() - Subtract 2 from the result (main thread + OS)
-
A queue of tasks
- A thread takes a task off the queue
- It performs the task
- Then it takes the next task from the queue
-
Tasks represented as callable objects
Advantages of Thread Pool
-
No scaling concerns
- The thread pool will automatically use all the available cores
-
Makes efficient use of resources
- Threads are always busy
- Provided there is work for them to do
-
Works best with short, simple tasks where
- The time taken to create a thread causes a significant delay
- The task does not block
Disadvantages of Thread Pool
-
Requires a concurrent queue or similar
- Not directly supported in C++
-
Overhead
- Must add and remove task functions in a thread safe way