We keep an eye on all the issues and feature requests at the github.com/userver-framework, at the English-speaking and Russian-speaking Telegram support chats. All the good ideas are discussed, big and important ones go to the Roadmap. We also have our in-house feature requests, those could be also found in Roadmap.
Important or interesting features go to the changelog as they get implemented. Note that there's also a Security Changelog.
Logging now supports fmt formatting in macro LOG_INFO("User {} logged in from {}", user_id, ip_address); and lambda formatting. See Logging and Tracing for more info.
PostgreSQL driver now can disable all the statements logging via static config option statement-log-mode
ClickHouse driver now supports doubles in the queries.
YDB now can be used with GCC compiler, not only Clang. YDB still requires C++20 support
components::Redis in sentinel mode now supports selection of database index via database_index in secdist config. Many thanks to Tikhon Sergienko for the PR!
PostgreSQL cache (shadow replicas) traits now support kOrderBy. With DISTINCT ON expression in kQuery it allows to store only slices of data. Many thanks to Dmitry Kopturov for the PR!
Added a userver-create-service script for creation of a new service. Github service templates are now deprecated.
Notify on state change when it actually happens in Redis Standalone. Many thanks to Nikolay Pervushin for the PR!
Modernized testsuite code, including removal of event_loop usages and usage of asyncio-socket.
Hidden thread_id and task_id for INFO+ logger levels to make the logs shorter.
Stack usage monitor now could be disabled in test via USERVER_GTEST_ENABLE_STACK_USAGE_MONITOR=1 environment variable.
gRPC
Added support for debug_redact and added a recommendation to use [debug_redact = true]; instead of [(userver.field).secret = true];
Server and client logging tags are now consistent.
More improvements, docs and samples for middlewares. See gRPC for more info.
Build:
Fixed floating point escaping compilation in ClickHouse driver. Many thanks to Ksenia-C for the PR!
Fix compilation errors for curl 8.13. Many thanks to Konstantin Goncharik for the PR!
Fix compilation errors for fmt 11. Many thanks to Konstantin Goncharik for the PR!
Added USERVER_ENABLE_DEBUG_INFO_COMPRESSION build option, that was later changed to USERVER_DEBUG_INFO_COMPRESSION. Many thanks to Konstantin Goncharik for the PRs! Compression detection algorithm was improved to check the linker compression support.
Added Debian-12 dependencies and build instructions.
Export only files that are not ignored by git for Conan sources. Many thanks to c0rt for the PR!
Added docs and samples for Eval* functions of Redis driver.
Release v2.8
sharding_strategy of the components::Redis now supports RedisStandalone configuration, that may be useful for tests or unimportant caches. Many thanks to Aleksey Ignatiev for the PR!
Added logging::JsonString to explicitly describe in type system that a string contains loggable JSON. Many thanks to akhoroshev for the PR!
Deadline Propagation for PostgreSQL is now enabled by default and can be controlled via deadline-propagation-enabled static option of the components::Postgres.
Erroneous attempt to log function address is now captured at build time.
Testing
Easy grpc mock registration in testsuite.
Daemon-scoped fixtures implemented via @pytest.mark.uservice_oneshot.
Build
Debug symbols of userver libraries are now compressed with zstd if the toolset supports it, leading to smaller binaries size.
Docker images now use zstd compression too.
Dropped CI testing on Ubuntu 20.04 which lifetime almost ended.
Improved build type matching for installed userver. Many thanks to Aleksey Ignatiev for the PR!
Release v2.7
Logging in JSON format was implemented. See static option format at components::Logging.
utils::regex now uses Re2 under the hood, leading to at least x2 faster regular expression matching and guaranteed absence of backtracking. Updating is highly recommended.
Mongo connection state checking algorithms was adjusted to work well on small RPS.
Conan packages now support all the userver features. Conan package build now reuses the CMake install targets and CMake config files.
Full feature support for MacOS, including testing and Conan package build and usage on that platform.
Added support for TLS certificate chains. See tls.cert static option at components::Server. Many thanks to aklyuchev for the PR!
Chaotic exceptions now do not depend on JSON. Thanks to Artyom for the PR!
Added ubuntu-22.04-userver-pg-dev image with all the tools for development. Planning to switch to Ubuntu-24.04 and leave only 2 containers: with build dependencies to build userver, and with prebuild userver.
Added missing fmt11 headers. Thanks to Pavel Sidorovich for the PR!
Added USERVER_USE_STATIC_LIBS to link third-party libraries statically.
Support pacman epoch in CMake version detection. Many thanks to Konstantin Goncharik for the PR.
storages::secdist::Secdist is now automatically reloaded for Mongo, Redis and PostgreSQL databases if the secdist file was changed. Now changing the connection parameters in file does not require service restart.
Public parts of the Redis driver were moved out from impl/ directory and placed into storages::redis:: namespace. If you were relying on the old paths, see ./scripts/migrate_from_legacy_redis_ns.sh script to ease migration.
Shortened testsuite logs were made more functional by providing HTTP URL info.
Removed old gRPC interface for server handlers as was promised in previous release notes.
gRPC client interfaces were changed to be more user friendly. For example, for HelloWorld method in protobuf we generate the old HelloWorld function along with the new AsyncHelloWorld and SyncHelloWorld functions. AsyncHelloWorld returns a ugrpc::client::ResponseFuture that can be used to retrieve the request result later in code. SyncHelloWorld retrieves the response from the future and returns the response itself. Consider replacing:
HelloWorld(x).Finish() with SyncHelloWorld(x)
auto res = HelloWorld(x); /* a lot of code */; res.Finish(); with auto res = AsyncHelloWorld(x); /* a lot of code*/; res.Get(); In next release we will remove the old HelloWorld and will rename SyncHelloWorld into HelloWorld.
GT and LT flags support in Redis ZADD. Thanks to Nikolay Pervushin for the PR!
Reduced condition in OTLP, thanks to Dudnik Pavel.
Build:
Simplified Profile Guided Optimization (PGO) gathering and usage due to new USERVER_PGO_GENERATE and USERVER_PGO_USE CMake options. See Configure, Build and Install for more info.
MacOS now can build the userver as a Conan package.
Build flags were reorganized to use a new USERVER_BUILD_ALL_LIBRARIES CMake option. See Build options for more info.
Source directory now can contain spaces.
Correctly set grpc-reflection found flag. Thanks to Nikita for the PR!
Fixed USERVER_CHAOTIC_FORMAT option for CMake build. Thanks to Konstantin Goncharik for the PR.
Optimized reconfiguration in CMake giving up to 60% time save (6-20 seconds).
Initial work towards embedding GDB pretty-printers to userver binaries.
Mongo now has the full functionality for diagnostics out-of-the box, without mongo-c library patches.
Simplified contributing by removing the annoying bot that checks for explicit agreement to CLA. Creating an issue or sending a PR already means agreement with CLA. Added notes to PR and Issue creation to highlight that.
Basic support for HTTP/2 body streaming.
Kafka support in testsuite implemented. See Functional tests section at Kafka service tutorial.
gRPC:
Safe new interface for gRPC server handlers. Old interface will be removed in next release.
Added support for TLS in gRPC.
Added ugrpc::server::middlewares::field_mask::Component for masking and trimming messages. Many thanks to TTPO100AJIEX for the work!
gRPC clients now allow configuring channels count for particular methods via dedicated-channel-counts static config option.
OpenTelemetry protocol (OTLP) now can optionally do only logging or only tracing. Thanks to TertiumOrganum1 for the PR!
The framework now accepts OTLP headers for tracing by default and puts those headers for new requests.
PostgreSQL span names are now a little bit more informative. Thanks to TertiumOrganum1 for the PR!
Kafka now has a client.id static option. Many thanks to Nikolay Pervushin for the PR.
PostgreSQL type errors become more informative. Thanks to farmovit for the report!
Optimizations:
HTTP/2 server implementation now does not copy data to send, saving CPU and RAM.
HTTP/2 now relies on open-addressing unordered map from nghttp2, leading to faster stream lookup.
Kafka consumer now does not block a task processor thread, allowing multiple consumers to share the same OS thread. Consume cycle now can be treated as an asynchronous non-blocking event loop.
Kafka producer delivery acknowledgments processing is now done in parallel, leading to better scalability. Also it does not block the OS thread when waiting for new delivery acknowledgments.
Internals of all the Sockets became smaller in size, saving some RAM.
The scheduler implementation now could be adjusted for each task_processor via task-processor-queue static option. A more efficient work-stealing-task-queue was introduced. Many thanks to Egor Bocharov for the PR!
Added storages::postgres::TimePointWithoutTz for more explicit declaration of intent. Direct work with std::chrono::system_clock is now deprecated in PostgreSQL driver.
Validation of static config schemas now understands minItems and maxItems for arrays. Many thanks to eparoshin for the PR.
Websockets now have case insensitive check of headers. Thanks to Alexander Enaldiev for the PR!
gRPC server now shows aggregated grpc.server.total metrics
More samples and docs.
Optimizations:
IO events are now uniformly distributed between ev threads. This leads to better performance on high loads in default configurations. Even number of ev threads now works as good as odd number of threads.
IO watchers now always start asynchronously, leading to x2 less CPU consumption for each start+stop operation. As a result ev threads of HTTP client and Redis driver now use less CPU.
Timer events with reachable deadlines now are deferred if that does not affect latencies. This gives ~5% RPS improvement for service_template.
Build
Find*.cmake files are not generated any more, leading to simpler code base and faster configure times.
Fixed incorrect handling of dots in chaotic paths. Thanks to Alexander Chernov for the PR!
MacOS build options are now part of the CMake files, leading to less boilerplate while compiling for that platform. Many thanks to Daniil Shvalov for the PR.
Kafka driver is now enabled in Conan. Many thanks to Aleksandr Gusev for the PR.
Conan related build fixes. Thanks to Alex for the PR.
Improved the ability to unit test of gRPC clients and servers, Redis and Mongo databases, logs. Samples and docs were improved.
Implemented feedback forms and likes/dislikes for each documentation page. Feedback would be appreciated! Many thanks to Fedor Alekseev for the PR and to MariaGrinchenko for the buttons design!
find_package(userver) now implicitly calls userver_setup_environment(), includes all the helper CMake userver scripts, making the configuration simpler. Added diagnostics and fix-it hints for some of the CMake missuses.
In docs Ctrl+k hotkey now focuses on Search input. Many thanks to Fedor Alekseev for the PR!
ODR-violations are now avoided if the userver is built with different standard version than the service.
YDB driver now can be built on modern Clang compilers in C++20 and above modes.
Redis now allows to subscribe to master instances.
Improved logging of failures in testsuite.
Optimizations:
rcu::Variable was optimized to use asymmetric fences and concurrent::StripedCounter. x1000 performance improvement for some edge cases, x3 improvement for the generic use case.
Internal TaskCounter now uses concurrent::StripedCounter to reduce contention on atomics on each async task construction and destruction.
Adjusted the default spinning count in scheduler to better fit modern hardware and typical loads.
~5% faster tasks queue overload detection logic. Many thanks to Egor Bocharov for the PR!
Docs, tests and build
Fixed build of utils/rand.hpp related source files. Thanks to Nikita for the PR!
Improved logic of Telegram support chats URL selection. Many thanks to Mingaripov Niyaz for the PR!
Fixed multiple @snippet links in docs. Many thanks to Mingaripov Niyaz for the PR!
Fixed flaky ThreadLocal.SafeThreadLocalWorks test. Many thanks to Egor Bocharov for the PR!
April 2024
Initial CPack support. Now userver-all.deb packages could be build via:
New Docker images based on Ubuntu-22.04 with weekly builds and publishing:
ghcr.io/userver-framework/ubuntu-22.04-userver-pg:latest - image with preinstalled userver deb package and PostgreSQL database. All the service templates now use it for tests
ghcr.io/userver-framework/ubuntu-22.04-userver:latest - image with preinstalled userver deb package. Good starting container for developing servers that do not use POstgreSQL.
ghcr.io/userver-framework/ubuntu-22.04-userver-base:latest - an image with only the build dependencies to build userver. Good for development of userver itself. More info at Configure, Build and Install
All the service templates were moved to a new components naming with :: (for example userver::core) and now attempt to find an installed userver. Now in CMake component names with :: work regardless of the framework installation (CPM, add_subdirectory, or find_package).
A secret value for the digest nonce generating now could be provided in http_server_digest_auth_secret key via components::Secdist. Many thanks to Mingaripov Niyaz for the PR!
ugrpc::server::ServerComponent now has a unix-socket-path static option to listen on unix domain socket path instead of an inet socket.
All the formats::*::ValueBuilder now support std::string_view. Thanks to Андрей Будиловский for the bug report!
Switched from unmaintained http_parser to a 156% faster llhttp.
Implemented a concurrent::StripedCounter that allows to have a per-cpu data in user space with kernel-provided transactional guarantees. Works at least x2 faster than std::atomic on a single thread and scales linearly (unlike std::atomic). As a result gives more than x10 performance improvement under heavy contention.
An explicit control over cache invalidations in testsuite. As a result, less caches are invalidated between tests and the overall time to run tests goes down significantly.
Components start was optimized to copy less containers during component dependencies detection.
Per each incoming HTTP request:
one less std::chrono::steady_clock::now() call
one less StrCaseHash construction (2 calls into std::uniform_int_distribution<std::uint64_t> over std::mt19937)
HTTP Client now copies less std::shared_ptr variables in implementation.
Optimized up to an order of magnitude the user types query in PostgreSQL driver.
pytest_userver.client.Client.capture_logs() now accepts log_level to filter out messages with lower log level in the service itself and minimize CPU and memory consumption during tests.
Added missing dependency for Arch based distros. Thanks to VScdr for the PR!
February 2024
PostgreSQL driver now keeps processing the current queries and transactions after encountering a "prepared statement already exists" event.
Implemented a new landing page to make userver even more nice&shiny! The whole project was done by 👨🏻💻 frontend developer Fedor Alekseev; 🧑🏼🎨 designers hellenisaeva and MariaGrinchenko; 👨🏻💼 manager Oleg Komarov. Many thanks for the awesome work!
HTTP components::Server now has a static configuration option address to select network interface to listen to.
Flush headers before starting to produce the body parts in HTTP streaming handlers, so a client can react to the request earlier. Thanks to akhoroshev for the PR!
Added tracing::TagScope for comfortable work with local scope tracing tags. Many thanks to nfrmtk for the PR!
INCLUDE_DIRECTORIES is now used in userver_add_grpc_library. Thanks to Nikita for the PR!
New storages::postgres::QueryQueue class, mostly for executing multiple SELECT statements and retrieving the results in one roundtrip.
PostgreSQL driver now caches the low-level database (D)escribe responses, workarounds the libpq implementation and does not request query describe information on each request. This results in about ~2 times less data transmitted for select statements that return multiple columns, less CPU consumption for the database server and for the application itself.
gRPC deadline propagation now uses less now() calls and measures time more precisely.
About 16% faster HttpResponse::SetHeader due to using a vectorization friendly algorithm.
Improved Redis driver start-up time.
dynamic_config::DocsMap::Parse() and dynamic configs retrieval copies less strings.
Build:
New Docker container for developing userver based projects ghcr.io/userver-framework/ubuntu-22.04-userver-base:latest. It contains all the build dependencies preinstalled and a proper setup of PPAs with databases, compilers and tools.
Switched to a modern python venv instead of the older virtualenv.
Use system provided Boost.Stacktrace if it is available.
More references and info for the metrics documentation.
More documentation for dynamic_config::ValueDict __default__ behavior, and for dynamic configs.
January 2024
full-update-jitter default value is now full-update-interval / 10. This leads to more responsive databases on full update on multiple instances of the service.
server::handlers::ServerMonitor now has a format static config option, simplifying userver based services setup with unified agent.
Allow creating custom implicit options. Thanks to trenin17 for the PR!
Limit JSON parsing depth to some sane value. Thanks to Kirill Morozov for the PR!
Requirements are not code generated any more and the default requirements used on userver_testsuite_add_simple() CMake function call. add_grpc_library() renamed to userver_add_grpc_library().
USERVER_PIP_USE_SYSTEM_PACKAGES and USERVER_PIP_OPTIONS CMake options now allow building the userver without internet connection. Thanks to Nikita for the PR!
Curl library now downloaded automatically if no suitable version found.
Added missing check for Mongo in Conan. Thanks to Pavel Talashchenko for the PR!
Fixed sanitizers error message in CMake. Thanks to Kirill Pavlov for the PR!
Updated MacOS dependencies. Thanks to Kirill Morozov for the PRs!
Added PostgreSQL 16 detection. Thanks to linuxnyasha for the PR!
Optimizations:
Optimized Http::Cookie::ToString(). Thanks to Илья Оплачкин for the PR!
Documentation:
Documented Secdist formats and added notes on possible environment variables usage instead of secdist.
Layout of reference pages changes. Now descriptions are at the top of the page rather than being after the list of methods and members.
PostgreSQL now has the following static config options:
check-user-types option to prevent service from starting if database is not ready for work (for example: some migrations were not applied).
discard-all-on-connect - to force running DISCARD ALL on new connections, which could be useful for some PostgreSQLs smart-proxies that reuse the same connections.
Boost.PFR 2.2.0 is now in use. The door for compile time reflection pull requests is now open!
More metrics and fallbacks for logging errors.
More tests for utils::datetime::MockSteadyNow. Thanks to Aleksej Kamenev for the PR!
components::CachingComponentBase now has a static config option alert-on-failing-to-update-times to fire an alert if the cache update failed specified amount of times in a row.
Optimizations:
Caching of dl_phdr_info items is not ON by default, leading up to multiple seconds faster exception handling under heavy load when parts of the executable still being on hard drive rather than in memory. Use USERVER_DISABLE_PHDR_CACHE CMake option to disable it, if the framework reports dlopen usage after component start and there's no way to avoid it.
Optimized discarding logs by log level from 4ns to 2ns. Added CMake option USERVER_FEATURE_ERASE_LOG_WITH_LEVEL to totally eliminate all the CPU and binary size overhead from logging.
Parsing inside storages::postgres::ResultSet now does not compute parsing-failure diagnostic information if parsing is successful.
Return PostgreSQL connection to pool earlier, before updating the statistics.
fs::RewriteFileContents() now does not fsync directories, dumping caches now also do not fsync directories leading to better performance while still properly restoring after server power-off.
PostgreSQL typed parsing was optimized to not copy std::shared_ptrs.
TESTPOINT() and TESTPOINT_CALLBACK() now produce less instructions and guaranteed to not throw it the testpoints are disabled.
Samples were significantly simplified, more static configuration options now have good defaults and do not require explicit setup.
Configure, Build and Install now contains information on how to build service templates. Information on how to build the framework tests was moved to scripts/docs/en/userver/build/userver.md
tracing::DefaultTracingManagerLocator now has incoming-format and new-requests-format static config options to configure the tracing headers to receive and send.
RabbitMQ now provides a Get() command to retrieve a single message. Thanks to Vladislav Nepogodin for the PR!
Optimizations:
Coroutines working set was reduced, leading to significantly less memory usage under most of the loads that do not use all the coroutines from pool at once.
Improved experience with metrics. Added Prometheus and Graphite metric formats. Provided a modern simple interface to write and test metrics.
Added chaos tests for drivers
Implemented streaming API for the HTTP
Improved documentation, added more samples and descriptions, improved search.
Numerous optimizations.
Numerous build improvements, including Conan and Docker support.
Optimized and improved features that were available at the Beta announcement:
gRPC client and server
Mongo driver
Redis driver
PostgreSQL
HTTP server and client
Logging and Tracing
... and many other features.
Detailed descriptions could be found below.
Beta (September 2023)
WebSockets server and TLS server are now implemented as part of the HTTP server
PostgreSQL pipelining is now implemented and turned on by default, leading to improved response times.
Mongo Congestion Control is implemented and turned on by default, leading to better stability of the service in case of high load on database.
Initial logger is now initialized from the component config, leading to a more simple code and setup. The --initial-logger option now does nothing and will be removed soon.
std::bitset<N>, std::array<bool, N>, userver::utils::Flags<Enum> and integral values are now mapped to PostgreSQL bit and bit varying types. Thanks to dsaa27 and bshirokov for the PRs!
Signifiacally optimized JSON DOM access to a non-existing element.
Logging now does not lock a mutex in fs-task-processor on each log record.
Up to 3 times faster logging due to multiple minor tweaks with data copy and memory prereserving.
The new asynchronous engine::io::sys::linux::Inotify driver for Linux allows subscriptions to filesystem events. Thanks to it the fs::FsCacheClient now reacts faster.
Non-coroutine userver-universal CMake target was refactored and is now used by the whole framework as a basic dependency. Compile times dropped down drastically for building the whole framework from scratch.
MacOS build instructions were enhanced, thanks to Daniil Shvalov for the PR!
Conan build was fixed, thanks to Yuri Bogomolov for the PR.
Numerous build and configure fixes.
Beta (June 2023)
Static configs of the service now can retrieve environment variables via the #env syntax. See yaml_config::YamlConfig for more examples.
New testsuite plugins userver_config_http_client and userver_config_testsuite_support turned on by default to increase timeouts in tests and make the functional tests more reliable.
clients::http::Request now does less atomic counters increments/decrements while the request is being built.
mlock_debug_info static configuration option of components::ManagerControllerComponent is now on by default. It improves responsiveness of the service under heavy load on low memory and bad hard drives.
engine::io::Socket now has an additional SendAll overload that accepts const struct iovec* list, std::size_t list_size for implementing low-level vector sends. Mongo driver now uses the new function, resulting in smaller CPU and memory consumption.
clients::http::Form::AddContent now instead of const std::string& parameters accepts std::string_view parameters that allow to copy less data.
TESTPOINT and other testpoint related macro now imply zero overhead if testpoints were disabled in static config.
More functions of formats::json::ValueBuilder now accept std::string_view, resulting in less std::string constructions and better performance.
TSKV escaping was optimized via SIMD, resulting in up to x10 speedup on long logs.
mlock_debug_info static configuration option of components::ManagerControllerComponent now allows to lock exception unwinding information in memory. It improves responsiveness of the service under heavy load on low memory and bad hard drives.
Experimental support for HTTP "Baggage" header is implemented, including verification, forwarding from HTTP handlers to client, baggage manipulation. See baggage::BaggageManagerComponent for more info.
Redis driver now supports non-queued variants for pubsub.
engine::Task now does not have a virtual destructor. New engine::TaskBase based hierarchy does not use RTTI, resulting in smaller binaries.
Mongo driver does not capture stack traces in release builds in case of errors. The error path become slightly faster, server is more responsive in case of Mongo problems.
Build:
Improved support for Conan 2.0, many thanks to Anton for the PR.
.gitattributes now handles line endings automatically for files detected as text. This simplifies WSL builds. Thanks to Anatoly Shirokov for the PR.
PostgreSQL libs selection is now possible in CMake if the platform has multiple versions installed, see PostgreSQL versions for more info.
Improved support for Arch Linux, many thanks to Konstantin Goncharik for the PR.
Multiple improvements for docs, including mockserver clarifications from Victor Makarov.
Beta (March 2023)
Now logging of particular lines could be controlled by dynamic config. See USERVER_LOG_DYNAMIC_DEBUG for more info.
Getting the default logger now takes only a single atomic read. LOG_* macro now do two RMW atomic operations less and do not use RCU, that could lead to a dynamic memory allocation in rare cases.
PostgreSQL driver now does much less atomic operations due to wider usage of std::move on the internal std::shared_ptr.
Now the "help wanted" issues at github have additional tags "good first issue" and "big", to help you to choose between a good starting issue and a big feature. See Releases, Trunk-based Development and Pull Requests for more info.
Cmake function userver_testsuite_add now can pass arguments to virtualenv, thanks to Дмитрий Изволов for the PR.
Improved hello_service run instruction, thanks to Svirex for the PR.
Better Python3 detection, thanks to Дмитрий Изволов for the PR.
Task processors now have an os-scheduling static config option and a usage guide.
Added a pg_service_template service template that uses userver the userver framework with PostgreSQL
In template services, it is now possible to deploy the environment and run the service in one command: make service-start-debug or make service-start-release.
Added userver::os_signals::Component, which is used for handling OS signals.
