userver: The Basics
All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages Concepts
The Basics

Restrictions

Usage of catch (...) without throw; should be avoided as the framework may use exceptions not derived from std::exception to manage some resources. Usage of catch with explicit exception type specification (like std::exception or std::runtime_error) is fine without throw;.

🐙 userver uses its own coroutine scheduler, which is unknown to the C++ standard library, as well as to the libc/pthreads. The standard library for synchronization often uses mutexes, other synchronization primitives and event waiting mechanisms that block the current thread. When using userver, this results in the current thread not being able to be used to execute other coroutines. As a result, the number of threads executing coroutines decreases. This can lead to a huge performance drops and increased latencies.

For the reasons described above, the use of synchronization primitives or IO operations of the C++ standard library and libc in the main task processor should be avoided in high-load applications. The same goes for all functions and classes that use blocking IO operations or synchronization primitives.

⚠️🐙❗ Instead of the standard primitives, you need to use the primitives from the userver:

Standard primitive Replacement from userver
thread_local It depends, but do not use standard thread_local
std::this_thread::sleep_for() engine::SleepFor()
std::this_thread::sleep_until() engine::SleepUntil()
std::mutex engine::Mutex
std::shared_mutex engine::SharedMutex
std::condition_variable engine::ConditionVariable
std::future<T> engine::TaskWithResult<T> or engine::Future
std::async() utils::Async()
std::thread utils::Async()
std::counting_semaphore engine::Semaphore
network sockets engine::io::Socket
std::filesystem:: fs::* (but not fs::blocking::*!)
std::cout LOG_INFO()
std::cerr LOG_WARNING() and LOG_ERROR()

An overview of the main synchronization mechanisms is available on a separate page.

Note that if your application is not meant for high-load and does not require low-latency, then it may be fine to run all the code on the same task processor.


⚠️🐙❗ If you want to run code that uses standard synchronization primitives (for example, code from a third-party library), then this code should be run in a separate engine::TaskProcessor to avoid starvation of main task processors. See Guide on TaskProcessor Usage for more info.


Tasks

The asynchronous task (engine::Task, engine::TaskWithResult) can return a result (possibly in form of an exception) or return nothing. In any case, the task has the semantics of future, i.e. you can wait for it and get the result from it.

To create a task call the utils::Async function. It accepts the name of a task, and the user-defined function to execute:

auto task = utils::Async("my_job", [some, &captures] {
return Use(some, captures);
});
// do something ...
auto result = task.Get();

Like std::async, you can call an existing function asynchronously, passing it some args:

auto task = utils::Async("my_job", SomeFunc, copied_arg, std::move(moved_arg), std::ref(ref_arg));
// do something ...
auto result = task.Get();

Flavors of Async

There are multiple orthogonal parameters of the task being started. Use this specific overload by default (utils::Async).

By engine::TaskProcessor:

  • By default, task processor of the current task is used.
  • Custom task processor can be passed as the first or second function parameter (see function signatures).

By shared-ness:

  • By default, functions return engine::TaskWithResult, which can be awaited from 1 task at once. This is a reasonable choice for most cases.
  • Functions from utils::Shared*Async* and engine::Shared*AsyncNoSpan families return engine::SharedTaskWithResult, which can be awaited from multiple tasks at the same time, at the cost of some overhead.

By engine::TaskBase::Importance ("critical-ness"):

  • By default, functions can be cancelled due to engine::TaskProcessor overload. Also, if the task is cancelled before being started, it will not run at all.
  • If the whole service's health (not just one request) depends on the task being run, then functions from utils::*CriticalAsync* and engine::*CriticalAsyncNoSpan* families can be used. There, execution of the function is guaranteed to start regardless of engine::TaskProcessor load limits

By tracing::Span:

  • Functions from utils::*Async* family (which you should use by default) create tracing::Span with inherited trace_id and link, a new span_id and the specified stopwatch_name, which ensures that logs from the task are categorized correctly and will not get lost.
  • Functions from engine::*AsyncNoSpan* family create span-less tasks:
    • A possible usage scenario is to create a task that will mostly wait in the background and do various unrelated work every now and then. In this case it might make sense to trace execution of work items, but not of the task itself.
    • Its usage can (albeit very rarely) be justified to squeeze some nanoseconds of performance where no logging is expected. But beware! Using tracing::Span::CurrentSpan() will trigger asserts and lead to UB in production.

By the propagation of engine::TaskInheritedVariable instances:

  • Functions from utils::*Async* family (which you should use by default) inherit all task-inherited variables from the parent task.
  • Functions from engine::*AsyncNoSpan* family do not inherit any task-inherited variables.

By deadline: some utils::*Async* functions accept an engine::Deadline parameter. If the deadline expires, the task is cancelled. See *Async* function signatures for details.

Scoping of tasks

A task is only allowed to run within the lifetime of its engine::TaskWithResult handle. If the control flow escapes the task definition scope while the task is running, it is cancelled and awaited in the task's destructor:

std::string FrobnicateBoth(std::string_view first, std::string_view second) {
auto first_task = utils::Async("frobnicate_first", Frobnicate, first);
auto second_task = utils::Async("frobnicate_second", Frobnicate, second);
if (SomethingBadHappened()) throw std::runtime_error("Nope");
return first_task.Get() + second_task.Get();
}

If an exception is thrown before the tasks are finished, they will be cancelled and awaited. In general, those tasks will be awaited anyway and will not keep running in the background indefinitely.

This is the backbone of structured concurrency in userver.

To make the task keep running in the background:

See also
concurrent::BackgroundTaskStorage

For more details on task cancellations:

See also
task_cancellation_intro

Lifetime of captures

Note
To launch background tasks, which are not awaited in the local scope, use concurrent::BackgroundTaskStorage.

When launching a task, it's important to ensure that it will not access its lambda captures after they are destroyed. Plain data captured by value (including by move) is always safe. By-reference captures and objects that store references inside are always something to be aware of. Of course, copying the world will degrade performance, so let's see how to ensure lifetime safety with captured references.

Task objects are automatically cancelled and awaited on destruction, if not already finished. The lifetime of the task object is what determines when the task may be running and accessing its captures. The task can only safely capture by reference objects that outlive the task object.

When the task is just stored in a new local variable and is not moved or returned from a function, capturing anything is safe:

int x{};
int y{};
// It's recommended to write out captures explicitly when launching tasks.
auto task = utils::Async("frobnicate", [&x, &y] {
// Capturing anything defined before the `task` variable is safe.
Use(x, y);
});
// ...
task.Get();

A more complicated example, where the task is moved into a container:

// Variables are destroyed in the reverse definition order: y, tasks, x.
int x{};
std::vector<engine::TaskWithResult<void>> tasks;
int y{};
tasks.push_back(utils::Async("frobnicate", [&x, &y] {
// Capturing x is safe, because `tasks` outlives `x`.
Use(x);
// BUG! The task may keep running for some time after `y` is destroyed.
Use(y);
}));

The bug above can be fixed by placing the declaration of tasks after y.

In the case above people often think that calling .Get() in appropriate places solves the problem. It does not! If an exception is thrown somewhere before .Get(), then the variables' definition order is the source of truth.

Same guidelines apply when tasks are stored in classes or structs: the task object must be defined below everything that it accesses:

private:
Foo foo_;
// Can access foo_ but not bar_.
engine::TaskWithResult<void> task_;
Bar bar_;

Generally, it's a good idea to put task objects as low as possible in the list of class members.

Although, tasks are rarely stored in classes on practice, concurrent::BackgroundTaskStorage is typically used for that purpose.

Components and their clients can always be safely captured by reference:

See also
Component system

Waiting

The code inside the coroutine may want to wait for an external event - a response from the database, a response from the HTTP client, the arrival of a certain time. If a coroutine wants to wait, it tells the engine that it wants to suspend its execution, and another coroutine starts executing on the current thread of the operating system instead. As a result, the thread is not idle, but reused by other users. After an external event occurs, the coroutine will be scheduled and executed.

f();
engine::SleepFor(std::chrono::seconds(60)); // voluntarily giving the current thread to other coroutines
g(); // The thread has returned to us

Task cancellation

A task can be notified that it needs to discard its progress and finish early. Once cancelled, the task remains cancelled until its completion. Cancelling a task permanently interrupts most awaiting operations in that task.

Ways to cancel a task

Cancellation can occur:

  • by an explicit request;
  • due to the end of the task object lifetime;
  • at coroutine engine shutdown (affects tasks launched via engine::Task::Detach);
  • due to the lack of resources.

To cancel a task explicitly, call the engine::TaskBase::RequestCancel() or engine::TaskBase::SyncCancel() method. It cancels only a single task and does not directly affect the subtasks that were created by the canceled task.

Another way to cancel a task it to drop the engine::TaskWithResult without awaiting it, e.g. by returning from the function that stored it in a local variable or by letting an exception fly out.

void Child() {
throw std::runtime_error("This exception will be swallowed in the task destructor");
}
void SomeOtherWork() { throw std::runtime_error("Something went wrong"); }
void Parent() {
auto child_task = utils::Async("child", Child);
// Now the current function proceeds to do some other work. Suppose it throws an exception.
// `child_task` is destroyed during stack unwinding, and the destructor
// cancels and awaits `child_task`. Its exception is swallowed in the destructor.
SomeOtherWork();
// After we've done our work, we'd expect to merge in child_task's result.
child_task.Get();
}

Tasks can be cancelled due to engine::TaskProcessor overload, if configured. This is a last-ditch effort to avoid OOM due to a spam of tasks. Read more in utils::Async and engine::TaskBase::Importance. Tasks started with engine::CriticalAsync are excepted from cancellations due to TaskProcessor overload.

How the task sees its cancellation

Unlike C++20 coroutines, userver does not have a magical way to kill a task. The cancellation will somehow be signaled to the synchronization primitive being waited on, then it will go through the logic of the user's function, then the function will somehow complete.

How some synchronization primitives react to cancellations:

Some synchronization primitives deliberately ignore cancellations, notably:

Most clients throw a client-specific exception on cancellation. Please explore the docs of the client you are using to find out how it reacts to cancellations. Typically, there is a special exception type thrown in case of cancellations, e.g. clients::http::CancelException.

How the outside world sees the task's cancellation

The general theme is that a task's completion upon cancellation is still a completion. The task's function will ultimately return or throw something, and that is what the parent task will receive in engine::TaskWithResult::Get or engine::TaskBase::Wait.

If the cancellation is due to the parent task being cancelled, then its engine::TaskWithResult::Get or engine::TaskBase::Wait will throw an engine::WaitInterruptedException, leaving the child task running (for now), so the parent task will likely not have a chance to observe the child task's completion status. Usually the stack unwinding in the parent task then destroys the engine::Task handle, which causes it to be cancelled and awaited.

void Parent() {
auto child_task = utils::Async("child", Child);
// Cancel ourselves for the sake of a simple example. On practice, Parent's parent will cancel it.
// The cancellation will be visible at the next waiting operation.
try {
child_task.Get();
FAIL() << "The line above should have thrown";
} catch (const engine::WaitInterruptedException& /*ex*/) {
// Cancelling Parent does not magically cancel any other tasks...
engine::SleepFor(std::chrono::milliseconds{10});
EXPECT_FALSE(child_task.IsFinished());
throw;
}
// ...It typically happens because `child_task` exits the scope.
}

If the child task got cancelled without the parent being cancelled, then:

  • engine::TaskWithResult::Get will return or throw whatever the child task has returned or thrown, which is practically meaningless (because why else would someone cancel a task?);
  • engine::TaskBase::Wait will return upon completion;
  • engine::TaskBase::IsFinished will return true upon completion;
  • engine::TaskBase::GetStatus will return engine::TaskBase::Status::kCancelled upon completion.

There is one extra quirk: if the task is cancelled before being started, then only the functor's destructor will be run by default. See details in utils::Async. In this case engine::TaskWithResult::Get will throw engine::TaskCancelledException.

Tasks launched via utils::CriticalAsync are always started, even if cancelled before entering the function. The cancellation will take effect immediately after the function starts:

bool task_was_run = false;
auto task = utils::CriticalAsync("sleep", [&task_was_run] {
task_was_run = true;
});
task.RequestCancel();
// It will actually typically only take a few microseconds for the task to complete.
// Check that the cancellation interrupted the sleep.
EXPECT_TRUE(task.IsFinished());
EXPECT_TRUE(task_was_run);

Lifetime of a cancelled task

Note that the destructor of engine::Task cancels and waits for task to finish if the task has not finished yet. Use concurrent::BackgroundTaskStorage to continue task execution out of scope.

The invariant that the task only runs within the lifetime of the engine::Task handle or concurrent::BackgroundTaskStorage is the backbone of structured concurrency in userver, see utils::Async and concurrent::BackgroundTaskStorage for details.

Utilities that interact with cancellations

The user is provided with several mechanisms to control the behavior of the application in case of cancellation: