engine::TaskProcessor or task processor is a thread pool on which the tasks (engine::Task, engine::TaskWithresult) are executed.

Creation

Task processors are configured via the static config file and are created at the start of the component system. Example:

# yaml
components_manager:
  coro_pool:
    initial_size: 5000
    max_size: 50000
  default_task_processor: main-task-processor
  event_thread_pool:
    threads: 2
  task_processors:
    bg-task-processor:
      thread_name: bg-worker
      worker_threads: 2
      os-scheduling: idle
      task-trace:
        every: 1000
        max-context-switch-count: 1000
        logger: tracer
    fs-task-processor:
      thread_name: fs-worker
      worker_threads: 2
    main-task-processor:
      thread_name: main-worker
      worker_threads: 16
    monitor-task-processor:
      thread_name: monitor
      worker_threads: 2
  components:
    manager-controller:  # Nothing

How to use

utils::Async and engine::AsyncNoSpan start a new task on a provided as a first argument task processor. If the task processor is not provided utils::Async and engine::AsyncNoSpan use the task processor that runs the current task (engine::current_task::GetTaskProcessor()).

A task processor could be obtained from components::ComponentContext in the constructor of the component. References to task processors outlive the component system tear-down, they are safe to use from within any components:

#include <userver/components/component.hpp>
#include <userver/dynamic_config/storage/component.hpp>
#include <userver/dynamic_config/value.hpp>
#include <userver/utils/async.hpp>
#include <userver/yaml_config/merge_schemas.hpp>
 
namespace myservice::smth {
 
Component::Component(const components::ComponentConfig& config,
                     const components::ComponentContext& context)
    : components::LoggableComponentBase(config, context),
      config_(
          // Searching for some component to initialize members
          context.FindComponent<components::DynamicConfig>()
              .GetSource()  // getting "client" from a component
      ) {
  // Reading config values from static config
  [[maybe_unused]] auto url = config["some-url"].As<std::string>();
  const auto fs_tp_name = config["fs-task-processor"].As<std::string>();
 
  // Starting a task on a separate task processor from config
  auto& fs_task_processor = context.GetTaskProcessor(fs_tp_name);
  utils::Async(fs_task_processor, "my-component/fs-work", [] { /*...*/ }).Get();
  // ...
}
 
}  // namespace myservice::smth

Warning: If a blocking system call (for example, one that reads a file in a synchronous way) runs on main-task-processor, then the thread and the processor core are idle until the system call ends. As a result, the throughput of the service temporarily decreases. To prevent this from happening, use userver provided primitives or if the primitive is missing, run the blocking system call on a separate task processor.

Common Task Processors

In static configuration we use different names for task processors. The name does not affect the task processor behavior, only gives a hint on its usage for the developer.

main-task-processor

This is usually the default task processor for CPU bound tasks. It is used to

start the component system, call constructors of the components;
for initialization of the caches (if not specified otherwise);
for accepting incoming requests and processing them;
do all the other CPU bound things;

Note: congestion_control::Component at the moment monitors only the task processor that starts the component system. Handlers that run on other tasks processors are not controlled.

fs-task-processor

Task processor for blocking calls. For example, for functions from blocking namespaces, low-level blocking system calls or third-party library code that does blocking calls.

Note: Functions from the userver framework that are not in the blocking namespaces or do not have blocking in their name are non-blocking! For example, there's no need to run engine::io::Socket::ReadAll() in fs-task-processor, use the main-task-processor instead.

A common usage pattern for this task processor looks like:

// lib_sample synchronously reads some of /etc/* files.
auto result = engine::AsyncNoSpan(fs_task_processor_, [preset_name]() {
  return lib_sample::quick_check_config_preset(preset_name);
}).Get();

Or:

auto result = engine::Async(fs_task_processor_, "torrent/validate", [path]() {
  auto files = libtorrent::discover_files(path);  // searches FS
  return libtorrent::validate_hashes(files);      // reads files and computes hashes
}).Get();

single-threaded-task-processors

Used for running task that do not permit concurrent execution (for example V8 or other interpreters). components::SingleThreadedTaskProcessors usually starts those task processors.

monitor-task-processor

This task processor is used for diagnostic and administration handlers and tasks. Separate task processor helps to control service under heavy load or get info from a server with deadlocked threads in the main-task-processor, e.g. server::handlers::InspectRequests handler.

Moving CPU-bound tasks to a separate task processor

Warning: Test and load-test your service, the feature may do things worse.

Some background tasks can slow down handles even if those tasks don't execute a blocking wait:

Multiple background tasks can start running at the same time
Worse if those background tasks are CPU-heavy and don't call engine::Yield
It can also be a problem if parallel tasks are launched in caches. Then one cache is able to fill the entire task processor with tasks

As a workaround for the issue the heavy background tasks could be moved to a separate bg-task-processor task processor.

There are multiple ways to prevent background tasks from consuming 100% of CPU cores:

Allocate the minimum possible number of threads for background task processors, less than CPU cores.
Experiment with the os-scheduling static option to give lower priority for background tasks.

Make sure that tasks execute faster than they arrive.

⇦ Logging and Tracing | Unit Tests and Benchmarks ⇨