Usage of catch (...)
without throw;
should be avoided as the framework may use exceptions not derived from std::exception
to manage some resources. Usage of catch
with explicit exception type specification (like std::exception
or std::runtime_error
) is fine without throw;
.
🐙 userver uses its own coroutine scheduler, which is unknown to the C++ standard library, as well as to the libc/pthreads. The standard library for synchronization often uses mutexes, other synchronization primitives and event waiting mechanisms that block the current thread. When using userver, this results in the current thread not being able to be used to execute other coroutines. As a result, the number of threads executing coroutines decreases. This can lead to a huge performance drops and increased latencies.
For the reasons described above, the use of synchronization primitives or IO operations of the C++ standard library and libc in the main task processor should be avoided in high-load applications. The same goes for all functions and classes that use blocking IO operations or synchronization primitives.
⚠️🐙❗ Instead of the standard primitives, you need to use the primitives from the userver:
Standard primitive | Replacement from userver |
---|---|
thread_local | It depends, but do not use standard thread_local |
std::this_thread::sleep_for() | engine::SleepFor() |
std::this_thread::sleep_until() | engine::SleepUntil() |
std::mutex | engine::Mutex |
std::shared_mutex | engine::SharedMutex |
std::condition_variable | engine::ConditionVariable |
std::future<T> | engine::TaskWithResult<T> or engine::Future |
std::async() | utils::Async() |
std::thread | utils::Async() |
std::counting_semaphore | engine::Semaphore |
network sockets | engine::io::Socket |
std::filesystem:: | fs::* (but not fs::blocking::* !) |
std::cout | LOG_INFO() |
std::cerr | LOG_WARNING() and LOG_ERROR() |
An overview of the main synchronization mechanisms is available on a separate page.
Note that if your application is not meant for high-load and does not require low-latency, then it may be fine to run all the code on the same task processor.
⚠️🐙❗ If you want to run code that uses standard synchronization primitives (for example, code from a third-party library), then this code should be run in a separate engine::TaskProcessor
to avoid starvation of main task processors. See Guide on TaskProcessor Usage for more info.
The asynchronous task (engine::Task, engine::TaskWithResult) can return a result (possibly in form of an exception) or return nothing. In any case, the task has the semantics of future, i.e. you can wait for it and get the result from it.
To create a task call the utils::Async function. It accepts the name of a task, and the user-defined function to execute:
Like std::async
, you can call an existing function asynchronously, passing it some args:
There are multiple orthogonal parameters of the task being started. Use this specific overload by default (utils::Async).
By engine::TaskProcessor:
By shared-ness:
utils::Shared*Async*
and engine::Shared*AsyncNoSpan
families return engine::SharedTaskWithResult, which can be awaited from multiple tasks at the same time, at the cost of some overhead.By engine::TaskBase::Importance ("critical-ness"):
utils::*CriticalAsync*
and engine::*CriticalAsyncNoSpan*
families can be used. There, execution of the function is guaranteed to start regardless of engine::TaskProcessor load limitsBy tracing::Span:
utils::*Async*
family (which you should use by default) create tracing::Span with inherited trace_id
and link
, a new span_id
and the specified stopwatch_name
, which ensures that logs from the task are categorized correctly and will not get lost.engine::*AsyncNoSpan*
family create span-less tasks:By the propagation of engine::TaskInheritedVariable instances:
utils::*Async*
family (which you should use by default) inherit all task-inherited variables from the parent task.engine::*AsyncNoSpan*
family do not inherit any task-inherited variables.By deadline: some utils::*Async*
functions accept an engine::Deadline parameter. If the deadline expires, the task is cancelled. See *Async*
function signatures for details.
A task is only allowed to run within the lifetime of its engine::TaskWithResult handle. If the control flow escapes the task definition scope while the task is running, it is cancelled and awaited in the task's destructor:
If an exception is thrown before the tasks are finished, they will be cancelled and awaited. In general, those tasks will be awaited anyway and will not keep running in the background indefinitely.
This is the backbone of structured concurrency in userver.
To make the task keep running in the background:
For more details on task cancellations:
When launching a task, it's important to ensure that it will not access its lambda captures after they are destroyed. Plain data captured by value (including by move) is always safe. By-reference captures and objects that store references inside are always something to be aware of. Of course, copying the world will degrade performance, so let's see how to ensure lifetime safety with captured references.
Task objects are automatically cancelled and awaited on destruction, if not already finished. The lifetime of the task object is what determines when the task may be running and accessing its captures. The task can only safely capture by reference objects that outlive the task object.
When the task is just stored in a new local variable and is not moved or returned from a function, capturing anything is safe:
A more complicated example, where the task is moved into a container:
The bug above can be fixed by placing the declaration of tasks
after y
.
In the case above people often think that calling .Get()
in appropriate places solves the problem. It does not! If an exception is thrown somewhere before .Get()
, then the variables' definition order is the source of truth.
Same guidelines apply when tasks are stored in classes or structs: the task object must be defined below everything that it accesses:
Generally, it's a good idea to put task objects as low as possible in the list of class members.
Although, tasks are rarely stored in classes on practice, concurrent::BackgroundTaskStorage is typically used for that purpose.
Components and their clients can always be safely captured by reference:
The code inside the coroutine may want to wait for an external event - a response from the database, a response from the HTTP client, the arrival of a certain time. If a coroutine wants to wait, it tells the engine that it wants to suspend its execution, and another coroutine starts executing on the current thread of the operating system instead. As a result, the thread is not idle, but reused by other users. After an external event occurs, the coroutine will be scheduled and executed.
A task can be notified that it needs to discard its progress and finish early. Once cancelled, the task remains cancelled until its completion. Cancelling a task permanently interrupts most awaiting operations in that task.
Cancellation can occur:
engine::Task::Detach
);To cancel a task explicitly, call the engine::TaskBase::RequestCancel()
or engine::TaskBase::SyncCancel()
method. It cancels only a single task and does not directly affect the subtasks that were created by the canceled task.
Another way to cancel a task it to drop the engine::TaskWithResult
without awaiting it, e.g. by returning from the function that stored it in a local variable or by letting an exception fly out.
Tasks can be cancelled due to engine::TaskProcessor
overload, if configured. This is a last-ditch effort to avoid OOM due to a spam of tasks. Read more in utils::Async
and engine::TaskBase::Importance
. Tasks started with engine::CriticalAsync
are excepted from cancellations due to TaskProcessor
overload.
Unlike C++20 coroutines, userver does not have a magical way to kill a task. The cancellation will somehow be signaled to the synchronization primitive being waited on, then it will go through the logic of the user's function, then the function will somehow complete.
How some synchronization primitives react to cancellations:
engine::TaskWithResult::Get
and engine::TaskBase::Wait
throw engine::WaitIterruptedException
, which typically leads to the destruction of the child task during stack unwinding, cancelling and awaiting it;engine::ConditionVariable::Wait
and engine::Future::wait
return a status code;engine::SingleConsumerEvent::WaitForEvent
returns false
;engine::SingleConsumerEvent::WaitForEventFor
returns false
and needs an additional engine::current_task::ShouldCancel()
check;engine::InterruptibleSleepFor
needs an additional engine::current_task::ShouldCancel()
check;engine::CancellableSemaphore
returns false
or throws engine::SemaphoreLockCancelledError
.Some synchronization primitives deliberately ignore cancellations, notably:
engine::Mutex
;engine::Semaphore
(use engine::CancellableSemaphore
to support cancellations);engine::SleepFor
(use engine::InterruptibleSleepFor
to support cancellations).Most clients throw a client-specific exception on cancellation. Please explore the docs of the client you are using to find out how it reacts to cancellations. Typically, there is a special exception type thrown in case of cancellations, e.g. clients::http::CancelException
.
The general theme is that a task's completion upon cancellation is still a completion. The task's function will ultimately return or throw something, and that is what the parent task will receive in engine::TaskWithResult::Get
or engine::TaskBase::Wait
.
If the cancellation is due to the parent task being cancelled, then its engine::TaskWithResult::Get
or engine::TaskBase::Wait
will throw an engine::WaitInterruptedException
, leaving the child task running (for now), so the parent task will likely not have a chance to observe the child task's completion status. Usually the stack unwinding in the parent task then destroys the engine::Task
handle, which causes it to be cancelled and awaited.
If the child task got cancelled without the parent being cancelled, then:
engine::TaskWithResult::Get
will return or throw whatever the child task has returned or thrown, which is practically meaningless (because why else would someone cancel a task?);engine::TaskBase::Wait
will return upon completion;engine::TaskBase::IsFinished
will return true
upon completion;engine::TaskBase::GetStatus
will return engine::TaskBase::Status::kCancelled
upon completion.There is one extra quirk: if the task is cancelled before being started, then only the functor's destructor will be run by default. See details in utils::Async
. In this case engine::TaskWithResult::Get
will throw engine::TaskCancelledException
.
Tasks launched via utils::CriticalAsync
are always started, even if cancelled before entering the function. The cancellation will take effect immediately after the function starts:
Note that the destructor of engine::Task
cancels and waits for task to finish if the task has not finished yet. Use concurrent::BackgroundTaskStorage
to continue task execution out of scope.
The invariant that the task only runs within the lifetime of the engine::Task
handle or concurrent::BackgroundTaskStorage
is the backbone of structured concurrency in userver, see utils::Async
and concurrent::BackgroundTaskStorage
for details.
The user is provided with several mechanisms to control the behavior of the application in case of cancellation:
engine::current_task::CancellationPoint()
– if the task is canceled, calling this function throws an exception that is not caught during normal exception handling (not inherited from std::exception
). This will result in stack unwinding with normal destructor calls for all local objects. The parent task will receive engine::TaskCancelledException
from engine::TaskWithResult::Get
. ⚠️🐙❗ Catching this exception results in UB, your code should not have catch (...)
without throw;
in the handler body!engine::current_task::ShouldCancel()
and engine::current_task::IsCancelRequested()
– predicates that return true
if the task is canceled:engine::current_task::ShouldCancel()
. It reports that a cancellation was requested for the task and the cancellation was not blocked (see below);engine::current_task::IsCancelRequested()
notifies that the task was canceled even if cancellation was blocked; effectively ignoring caller's requests to complete the task regardless of cancellation.engine::TaskCancellationBlocker
– scope guard, preventing cancellation in the current task. As long as it is alive all the blocking calls are not interrupted, engine::current_task::CancellationPoint
throws no exceptions, engine::current_task::ShouldCancel
returns false
. ⚠️🐙❗ Disabling cancellation does not affect the return value of engine::current_task::IsCancelRequested()
.