c++ - How do I automatically add threads to a pool based on the computational needs of the program? -

we have c++ program which, depending on way user configures it, may cpu bound or io bound. purpose of loose coupling program configuration, i'd have thread pool automatically realize when program benefit more threads (i.e. cpu bound). nice if realized when i/o bound , reduced number of workers, bonus (i.e. i'd happy automatically grows without automatic shrinkage).

we use boost if there's there can use it. realize solution platform specific, we're interested in windows , linux, tertiary interest in os x or other *nix.

short answer: use distinct fixed-size thread pools cpu intensive operations , ios. in addition pool sizes, further regulation of number of active threads done bounded-buffer (producer/consumer) synchronizes computer , io steps of workflow.

for compute- , data-intensive problems bottlenecks moving target between different resources (e.g. cpu vs io), can useful make clear distinction between thread , thread, particularly, first approximation:

• a thread created use more cpu cycles ("cpu thread")
• a thread created handle asynchronous io operation ("io thread")

more generally, threads should segregated type of resources need. aim should ensure single thread doesn't use more 1 resource (e.g. avoiding switching between reading data , processing data in same thread). when tread uses more 1 resource, should split , 2 resulting threads should synchronized through bounded-buffer.

typically there should many cpu threads needed saturate instruction pipelines of cores available on system. ensure that, have "cpu thread pool" many threads dedicated computational work only. boost:: or std::thread::hardware_concurrency() if can trusted. when application needs less, there unused threads in cpu thread pool. when needs more, work queued. instead of "cpu thread pool", use c++11 std::async need implement thread throttling mechanism selection of synchronization tools (e.g. counting semaphore).

in addition "cpu thread pool", there can thread pool (or several other thread pools) dedicated asynchronous io operations. in case, seems io resource contention potentially concern. if that's case (e.g. local hard drive) maximum number of threads should controlled (e.g. @ 2 read , 2 write threads on local hard drive). conceptually same cpu threads , should have 1 fixed size thread pool reading , 1 writing. unfortunately, there not primitive available decide on size of these thread pools (measuring might simple though, if io patterns regular). if resource contention not issue (e.g. nas or small http requests) boost::asio or c++11 std::async better option thread pool; in case, thread throttling can entirely left bounded-buffers.