engine::TaskProcessor or task processor is a thread pool on which the tasks (engine::Task, engine::TaskWithresult) are executed.

Creation

Task processors are configured via the static config file and are created at the start of the component system. Example:

# yaml
components_manager:
  coro_pool:
    initial_size: 5000
    max_size: 50000
  default_task_processor: main-task-processor
  event_thread_pool:
    threads: 2
  task_processors:
    bg-task-processor:
      thread_name: bg-worker
      worker_threads: 2
      os-scheduling: idle
      task-processor-queue: global-task-queue
      task-trace:
        every: 1000
        max-context-switch-count: 1000
        logger: tracer
    fs-task-processor:
      thread_name: fs-worker
      worker_threads: 2
    main-task-processor:
      thread_name: main-worker
      worker_threads: 16
    monitor-task-processor:
      thread_name: monitor
      worker_threads: 2
  components:
    manager-controller:  # Nothing

Any amount of task processors could be created with any names.

How to use

utils::Async and engine::AsyncNoSpan start a new task on a provided as a first argument task processor. If the task processor is not provided utils::Async and engine::AsyncNoSpan use the task processor that runs the current task (engine::current_task::GetTaskProcessor()).

A task processor could be obtained from components::ComponentContext in the constructor of the component. References to task processors outlive the component system tear-down, they are safe to use from within any components:

#include <userver/components/component.hpp>
#include <userver/dynamic_config/storage/component.hpp>
#include <userver/dynamic_config/value.hpp>
#include <userver/utils/async.hpp>
#include <userver/yaml_config/merge_schemas.hpp>
 
namespace myservice::smth {
 
Component::Component(const components::ComponentConfig& config, const components::ComponentContext& context)
    : components::ComponentBase(config, context),
      config_(
          // Searching for some component to initialize members
          context.FindComponent<components::DynamicConfig>().GetSource()  // getting "client" from a component
      ) {
    // Reading config values from static config
    [[maybe_unused]] auto url = config["some-url"].As<std::string>();
    const auto fs_tp_name = config["fs-task-processor"].As<std::string>();
 
    // Starting a task on a separate task processor from config
    auto& fs_task_processor = context.GetTaskProcessor(fs_tp_name);
    utils::Async(fs_task_processor, "my-component/fs-work", [] { /*...*/ }).Get();
    // ...
}
 
}  // namespace myservice::smth

Warning: If a blocking system call (for example, one that reads a file in a synchronous way) runs on main-task-processor, then the thread and the processor core are idle until the system call ends. As a result, the throughput of the service temporarily decreases. To prevent this from happening, use userver provided primitives or if the primitive is missing, run the blocking system call on a separate task processor.

Task processors intentionally hide their internals and member functions, so there's no way to call any of the task processor members directly.

Common Task Processors

In static configuration we use different names for task processors. The name does not affect the task processor behavior, only gives a hint on its usage for the developer.

main-task-processor

This is usually the default task processor for CPU bound tasks. It is used to

start the component system, call constructors of the components;
for initialization of the caches (if not specified otherwise);
for accepting incoming requests and processing them;
do all the other CPU bound things;

Note: congestion_control::Component at the moment monitors only the task processor that starts the component system. Handlers that run on other tasks processors are not controlled.

fs-task-processor

Task processor for blocking calls. For example, for functions from blocking namespaces, low-level blocking system calls or third-party library code that does blocking calls.

Note: Functions from the userver framework that are not in the blocking namespaces or do not have blocking in their name are non-blocking! For example, there's no need to run engine::io::Socket::ReadAll() in fs-task-processor, use the main-task-processor instead.

A common usage pattern for this task processor looks like:

// lib_sample synchronously reads some of /etc/* files.
auto result = engine::AsyncNoSpan(fs_task_processor_, [preset_name]() {
  return lib_sample::quick_check_config_preset(preset_name);
}).Get();

Or:

auto result = engine::Async(fs_task_processor_, "torrent/validate", [path]() {
  auto files = libtorrent::discover_files(path);  // searches FS
  return libtorrent::validate_hashes(files);      // reads files and computes hashes
}).Get();

single-threaded-task-processors

Used for running task that do not permit concurrent execution (for example V8 or other interpreters). components::SingleThreadedTaskProcessors usually starts those task processors.

monitor-task-processor

This task processor is used for diagnostic and administration handlers and tasks. Separate task processor helps to control service under heavy load or get info from a server with deadlocked threads in the main-task-processor, e.g. server::handlers::InspectRequests handler.

Moving CPU-bound tasks to a separate task processor

Warning: Test and load-test your service, the feature may do things worse.

Some background tasks can slow down handles even if those tasks don't execute a blocking wait:

Multiple background tasks can start running at the same time
Worse if those background tasks are CPU-heavy and don't call engine::Yield
It can also be a problem if parallel tasks are launched in caches. Then one cache is able to fill the entire task processor with tasks

As a workaround for the issue the heavy background tasks could be moved to a separate bg-task-processor task processor.

There are multiple ways to prevent background tasks from consuming 100% of CPU cores:

Allocate the minimum possible number of threads for background task processors, less than CPU cores.
Experiment with the os-scheduling static option to give lower priority for background tasks.

Make sure that tasks execute faster than they arrive.

⇦ Logging and Tracing | Periodics and DistLocks ⇨