slow-work

SLOW WORK ITEM EXECUTION THREAD POOL

The slow work item execution thread pool is a pool of threads for performing things that take a relatively long time, such as making mkdir calls. Typically, when processing something, these items will spend a lot of time blocking a thread on I/O, thus making that thread unavailable for doing other work.

The standard workqueue model is unsuitable for this class of work item as that limits the owner to a single thread or a single thread per CPU. For some tasks, however, more threads - or fewer - are required. There is just one pool per system. It contains no threads unless something wants to use it - and that something must register its interest first. When the pool is active, the number of threads it contains is dynamic, varying between a maximum and minimum setting, depending on the load.

CLASSES OF WORK ITEM

This pool support two classes of work items:

  • Slow work items.
  • Very slow work items.

The former are expected to finish much quicker than the latter.

An operation of the very slow class may do a batch combination of several lookups, mkdirs, and a create for instance.

An operation of the ordinarily slow class may, for example, write stuff or expand files, provided the time taken to do so isn't too long.

Operations of both types may sleep during execution, thus tying up the thread loaned to it.

THREAD-TO-CLASS ALLOCATION

Not all the threads in the pool are available to work on very slow work items. The number will be between one and one fewer than the number of active threads. This is configurable.

All the threads are available to work on ordinarily slow work items, but a percentage of the threads will prefer to work on very slow work items.

The configuration ensures that at least one thread will be available to work on very slow work items, and at least one thread will be available that won't work on very slow work items at all.