We keep an eye on all the issues and feature requests at the github.com/userver-framework, at the English-speaking and Russian-speaking Telegram support chats. All the good ideas are discussed, big and important ones go to the Roadmap. We also have our in-house feature requests, those could be also found in Roadmap.
Important or interesting features go to the changelog as they get implemented. Note that there's also a Security Changelog.
Changelog news also go to the userver framework news channel.
Roadmap
- ✔️ Codegen parsers and serializers by JSON schema
- ✔️ HTTP 2.0 server support
- ✔️ Improve OpenTelemetry Protocol (OTLP) support.
- ✔️ Improve Kafka driver.
- 👨💻 gRPC simplification and functionality improvement.
- 👨💻 Logging format customization, including JSON logging.
- 👨💻 Secdist simplification and functionality improvement.
- 👨💻 Add retry budget or retry circuit breaker for clients.
- Improved MacOS build support.
- Add web interface to the uservice-dynconf
- Generate full-blown accessories for OpenAPI:
Changelog
Release v2.5
- Added S3 API client s3api::Client. Many thanks to v-for-vandal for the work!
- Added gRPC reflection library. Many thanks to v-for-vandal for the work!
- Added Kill Switch functionality. Many thanks to Aksenov Anton for the work!
- Congestion Control turned on by default.
- Initial work towards embedding GDB pretty-printers to userver binaries.
- Mongo now has the full functionality for diagnostics out-of-the box, without mongo-c library patches.
- Simplified contributing by removing the annoying bot that checks for explicit agreement to CLA. Creating an issue or sending a PR already means agreement with CLA. Added notes to PR and Issue creation to highlight that.
- Basic support for HTTP/2 body streaming.
- Kafka support in testsuite implemented. See Functional tests section at Kafka service tutorial.
- gRPC:
- Safe new interface for gRPC server handlers. Old interface will be removed in next release.
- Added support for TLS in gRPC.
- Added ugrpc::server::middlewares::field_mask::Component for masking and trimming messages. Many thanks to TTPO100AJIEX for the work!
- gRPC clients now allow configuring channels count for particular methods via
dedicated-channel-counts
static config option.
Optimizations:
Build:
- Added
userver_module()
CMake function to simplify configuration of new drivers that are being added to userver.
- Added missing
fmt/ranges.h
includes. Thanks to Vasilii Kuziakin and to SidorovichPavel for the PRs!
- Proper use of
PROTOBUF_PROTOC
in CMake. Thanks to Nikita for the PR!
- Added support for builds in paths that contain whitespaces and other special symbols.
- Added CI build tests for Ubuntu 24.04 and MacOS.
- Switched to Conan v2. Many thanks to Anton for the PR! Also use modern versions of third party libraries in Conan.
Documentation and diagnostics:
- A whole new build dedicated section was added to the docs instead of the old "Configure, Build and Install" page.
- Improved schemes validation messages, including config validation messages because no schema is written.
- Disambiguated diagnostic messages for component system.
- Better log messages for the dist locks.
- Better docs for gRPC middlewares and gRPC logs at gRPC.
- Added topology and heartbeats logs and metrics for Mongo.
- Clarified docs on PostgreSQL data types with timezones. See uPg: Supported data types.
- Added Kafka service tutorial.
- Logging at runtime, Congestion Control documentation rewrite.
Release v2.4
- Added USERVER_LOG_REQUEST_HEADERS_WHITELIST to control the HTTP headers to log.
- OpenTelemetry protocol (OTLP) now can optionally do only logging or only tracing. Thanks to TertiumOrganum1 for the PR!
- The framework now accepts OTLP headers for tracing by default and puts those headers for new requests.
- PostgreSQL span names are now a little bit more informative. Thanks to TertiumOrganum1 for the PR!
- Kafka now has a
client.id
static option. Many thanks to Nikolay Pervushin for the PR.
- PostgreSQL type errors become more informative. Thanks to farmovit for the report!
- Optimizations:
- HTTP/2 server implementation now does not copy data to send, saving CPU and RAM.
- HTTP/2 now relies on open-addressing unordered map from nghttp2, leading to faster stream lookup.
- Kafka consumer now does not block a task processor thread, allowing multiple consumers to share the same OS thread. Consume cycle now can be treated as an asynchronous non-blocking event loop.
- Kafka producer delivery acknowledgments processing is now done in parallel, leading to better scalability. Also it does not block the OS thread when waiting for new delivery acknowledgments.
- Internals of all the Sockets became smaller in size, saving some RAM.
- gRPC:
- Build, Install and CI:
- OTLP build is now supported in Conan. Thanks to Amina Ramazanova for the PR!
- Chaotic now exposes less headers, leading to faster build times.
- Fixed compilation on modern Boost.UUID. Thanks to Alexander Botev for the PR!
- Added
dependabot
to CI and updated the dependencies. Thanks to Dzmitry Ivaniuk for the PR!
- Added missing
#include
. Thanks to Nikita for the PR!
- Removed outdated defines in the core. Thanks to Sergey Kazmin for the PR!
- Install now does not put third party headers into the top level include directory. Multiple unused files are now not installed.
- Started the work to enable builds in directories with whitespace in names.
- Documentation:
Release v2.3
- Initial HTTP 2.0 server support is now implemented. Use
handler-defaults.http_version
static config option of components::Server to enable.
- Logger for OpenTelemetry protocol was implemented. Could be enabled via
USERVER_FEATURE_OTLP
CMake option. See the docs for more info.
- Client address in handler now could be retrieved via server::http::HttpRequest::GetRemoteAddress(). Many thanks to Daniil Shvalov for the PR.
- The scheduler implementation now could be adjusted for each task_processor via
task-processor-queue
static option. A more efficient work-stealing-task-queue
was introduced. Many thanks to Egor Bocharov for the PR!
- Added storages::postgres::TimePointWithoutTz for more explicit declaration of intent. Direct work with std::chrono::system_clock is now deprecated in PostgreSQL driver.
- Validation of static config schemas now understands
minItems
and maxItems
for arrays. Many thanks to eparoshin for the PR.
- Websockets now have case insensitive check of headers. Thanks to Alexander Enaldiev for the PR!
- Added engine::io::Socket::ReadNoblock() function to check if there's a pending data and read it if any. server::websocket::WebSocketConnection now has a TryRecv() function to receive a message if its first bytes already came. Thanks to Alexander Enaldiev for the PR!
#env
, #file
and #fallback
now could be used in config_vars
file. See yaml_config::YamlConfig for more info. Thanks to Artyom Samuylik for the PR.
- gRPC
- Sensitive data now could be hidden in logs via applying a
[(userver.field).secret = true]
option to a protobuf field in schema.
- Generic server now could be implemented via ugrpc::server::GenericServiceBase. Generic client ugrpc::client::GenericClient was also implemented. The functionality of generic client/server is useful for writing gRPC proxies.
- gRPC server now shows aggregated
grpc.server.total
metrics
- More samples and docs.
- Optimizations:
- IO events are now uniformly distributed between ev threads. This leads to better performance on high loads in default configurations. Even number of ev threads now works as good as odd number of threads.
- IO watchers now always start asynchronously, leading to x2 less CPU consumption for each start+stop operation. As a result ev threads of HTTP client and Redis driver now use less CPU.
- Timer events with reachable deadlines now are deferred if that does not affect latencies. This gives ~5% RPS improvement for
service_template
.
- Build
Find*.cmake
files are not generated any more, leading to simpler code base and faster configure times.
- Fixed incorrect handling of dots in chaotic paths. Thanks to Alexander Chernov for the PR!
- MacOS build options are now part of the CMake files, leading to less boilerplate while compiling for that platform. Many thanks to Daniil Shvalov for the PR.
- Kafka driver is now enabled in Conan. Many thanks to Aleksandr Gusev for the PR.
- Conan related build fixes. Thanks to Alex for the PR.
Release v2.2
- Added codegen parsers and serializers by JSON schema
- Improved the ability to unit test of gRPC clients and servers, Redis and Mongo databases, logs. Samples and docs were improved.
- Implemented feedback forms and likes/dislikes for each documentation page. Feedback would be appreciated! Many thanks to Fedor Alekseev for the PR and to MariaGrinchenko for the buttons design!
- Added docs on YDB.
- Mobile header view and docs layout was improved. Many thanks to Fedor Alekseev for the PRs.
- engine::subprocess::ProcessStarter::Exec now can lookup binaries via
PATH
variable.
- Fixed gRPC generation for nested namespaces with repetitions. Many thanks to nfrmtk for the PR!
- Handle both websocket and plain HTTP requests for the same path. Many thanks to Hovard Smith for the PR!
- Support setting client + CA certs in RabbitMQ. Many thanks to Alexey Dyumin for the PR!
- yaml_config::YamlConfig now can read files via
#file
. Now the static config of the service could refer to other files.
- Added support of bit operations to Redis.
- PostgreSQL driver now works with AWS Aurora.
- Added quick start for beginners to Configure, Build and Install. Many thanks to Fedor Alekseev for the PR.
- Improved path to sources trimming for Conan builds. Many thanks to Kirill for the PR!
- Multiple minor improvements to build, packaging, docs and testing.
Release v2.1 (May 2024)
Release v2.0
Big new features since the v1.0.0:
- Simplified dynamic configs and embedded defaults into the code.
- Added PostgreSQL connection pools auto-configuration.
- Added YDB driver and basic Kafka driver.
- LISTEN/NOTIFY support for PostgreSQL
- New landing page for the website
- Significantly reduced network data transmission for PostgreSQL
- Supported
install
in CMake and CPack packaging.
- Implemented middlewares for HTTP server, most of the HTTP server functionality was moved to middlewares.
- Improved documentation, added more samples and descriptions.
- Numerous optimizations and build improvements.
Detailed descriptions could be found below.
Binary Ubuntu 22.04 amd64 package could be found at userver Releases.
May 2024 (v2.0-rc)
- Kafka driver is now documented, compiles and works. Thanks to Fedor for the work!
- Added utils::numeric_cast for safe casting of integers of different width.
- The userver framework is now available at Yandex Cloud Marketplace.
- YDB driver now can be built on modern Clang compilers in C++20 and above modes.
- Redis now allows to subscribe to master instances.
- Improved logging of failures in testsuite.
- Optimizations:
- rcu::Variable was optimized to use asymmetric fences and concurrent::StripedCounter. x1000 performance improvement for some edge cases, x3 improvement for the generic use case.
- Internal
TaskCounter
now uses concurrent::StripedCounter to reduce contention on atomics on each async task construction and destruction.
- Adjusted the default spinning count in scheduler to better fit modern hardware and typical loads.
- ~5% faster tasks queue overload detection logic. Many thanks to Egor Bocharov for the PR!
- Docs, tests and build
- Fixed build of utils/rand.hpp related source files. Thanks to Nikita for the PR!
- Improved logic of Telegram support chats URL selection. Many thanks to Mingaripov Niyaz for the PR!
- Fixed multiple
@snippet
links in docs. Many thanks to Mingaripov Niyaz for the PR!
- Fixed flaky
ThreadLocal.SafeThreadLocalWorks
test. Many thanks to Egor Bocharov for the PR!
April 2024
- Initial CPack support. Now
userver-all.deb
packages could be build via: git clone --depth 1 https://github.com/userver-framework/userver.git
docker run --rm -it --network ip6net -v $(pwd):/home/user -w /home/user/userver \
--entrypoint bash ghcr.io/userver-framework/ubuntu-22.04-userver-base:latest ./scripts/docker/run_as_user.sh \
./scripts/build_and_install_all.sh
We'll start publishing the packages soon.
- New Docker images based on Ubuntu-22.04 with weekly builds and publishing:
- ghcr.io/userver-framework/ubuntu-22.04-userver-pg:latest - image with preinstalled userver deb package and PostgreSQL database. All the service templates now use it for tests
- ghcr.io/userver-framework/ubuntu-22.04-userver:latest - image with preinstalled userver deb package. Good starting container for developing servers that do not use POstgreSQL.
- ghcr.io/userver-framework/ubuntu-22.04-userver-base:latest - an image with only the build dependencies to build userver. Good for development of userver itself. More info at Configure, Build and Install
- All the service templates were moved to a new components naming with
::
(for example userver::core
) and now attempt to find an installed userver. Now in CMake component names with ::
work regardless of the framework installation (CPM, add_subdirectory
, or find_package
).
- Added userver/utils/hedged_request.hpp with a bunch of helper classes to perform hedged requests.
- Improved testsuite logging for functional tests.
- Consumers of concurrent queues now have a Reset() method. Thanks to akhoroshev for the PR!
- Fixed gRPC builds and runs with ASAN. Thanks to Nikita for the PR!
- Published early work on Kafka driver. API is not stable, build scripts, improvements and samples to come. Thanks to Fedor for the work!
- Published early work on RocksDB driver. API is not stable, build scripts, improvements and samples to come. Thanks to Kirill for the work!
- Optimizations:
- WriteAll for TLS became up to 7 times faster if multiple small chunks of data are written. Thanks to Илья Оплачкин and VScdr for the PR!
- Binary sizes were reduced if building without LTO. All the binaries linked with userver became about 1MB smaller.
- Implemented asymmetric thread fences. This opens the door for optimizations of rcu::Variable and internals of the scheduler.
- Caches in testsuite are now invalidated concurrently, leading to tens of seconds speedup for services with multiple caches and tests.
- concurrent::StripedCounter became 8 times more memory efficient.
- Build and docs:
March 2024
- Installation via
cmake --install
was implemented. See userver_install for more info.
- Implemented Http server middlewares
- USERVER_LOG_DYNAMIC_DEBUG now provides fine granted runtime control over logging.
- Now all the modern versions of
libmongoc
are supported in userver-mongo
.
- POSTGRES_CONNLIMIT_MODE_AUTO_ENABLED is now
on
by default.
- A secret value for the digest nonce generating now could be provided in
http_server_digest_auth_secret
key via components::Secdist. Many thanks to Mingaripov Niyaz for the PR!
- ugrpc::server::ServerComponent now has a
unix-socket-path
static option to listen on unix domain socket path instead of an inet socket.
- All the
formats::*::ValueBuilder
now support std::string_view. Thanks to Андрей Будиловский for the bug report!
- clients::http::Form is now movable and slightly more efficient. Thanks to Alexandr Kondratev for the PR!
- Redis now supports
SSUBSCRIBE
and removes dead nodes.
- engine::WaitAny() now can wait for an engine::io::Socket/engine::io::TlsWrapper to become readable or writable. For example:
engine::WaitAny(socket.GetReadableBase(), task1, tls_socket.GetWritableBase(), future1);
- New tracing::Span::MakeRootSpan() helper function.
- Optimizations:
- Switched from unmaintained
http_parser
to a 156% faster llhttp
.
- Implemented a concurrent::StripedCounter that allows to have a per-cpu data in user space with kernel-provided transactional guarantees. Works at least x2 faster than std::atomic on a single thread and scales linearly (unlike std::atomic). As a result gives more than x10 performance improvement under heavy contention.
- An explicit control over cache invalidations in testsuite. As a result, less caches are invalidated between tests and the overall time to run tests goes down significantly.
- Components start was optimized to copy less containers during component dependencies detection.
- Per each incoming HTTP request:
- one less
std::chrono::steady_clock::now()
call
- one less
StrCaseHash
construction (2 calls into std::uniform_int_distribution<std::uint64_t>
over std::mt19937
)
- one less
dynamic_config::Snapshot
construction (at least one atomic CAS)
- HTTP Client now copies less
std::shared_ptr
variables in implementation.
- Optimized up to an order of magnitude the user types query in PostgreSQL driver.
- pytest_userver.client.Client.capture_logs() now accepts
log_level
to filter out messages with lower log level in the service itself and minimize CPU and memory consumption during tests.
- Up to two times faster utils::statistics::RecentPeriod statistics in PostgreSQL driver due to switching to a utils::datetime::SteadyCoarseClock.
- Build:
- Workarounds for the Protobuf >= 5.26.0
- Removed redundant semicolon, thanks to Oleksii Demchenko for the PR.
- utils::FastScopeGuard now uses proper traits. Thanks to Илья Оплачкин for the PR and to Artyom Kolpakov for the report!
- Added missing dependency for Arch based distros. Thanks to VScdr for the PR!
February 2024
- PostgreSQL driver now keeps processing the current queries and transactions after encountering a "prepared statement already exists" event.
- Implemented a new landing page to make userver even more nice&shiny! The whole project was done by 👨🏻💻 frontend developer Fedor Alekseev; 🧑🏼🎨 designers hellenisaeva and MariaGrinchenko; 👨🏻💼 manager Oleg Komarov. Many thanks for the awesome work!
- HTTP components::Server now has a static configuration option
address
to select network interface to listen to.
- gRPC futures now can be used efficiently with engine::WaitAny().
- Flush headers before starting to produce the body parts in HTTP streaming handlers, so a client can react to the request earlier. Thanks to akhoroshev for the PR!
- Added tracing::TagScope for comfortable work with local scope tracing tags. Many thanks to nfrmtk for the PR!
INCLUDE_DIRECTORIES
is now used in userver_add_grpc_library
. Thanks to Nikita for the PR!
- New storages::postgres::QueryQueue class, mostly for executing multiple
SELECT
statements and retrieving the results in one roundtrip.
- POSTGRES_TOPOLOGY_SETTINGS to control maximum allowed replication lag.
- Optimizations:
- PostgreSQL driver now caches the low-level database (D)escribe responses, workarounds the libpq implementation and does not request query describe information on each request. This results in about ~2 times less data transmitted for select statements that return multiple columns, less CPU consumption for the database server and for the application itself.
- On Boost 1.74 and newer the engine::SingleConsumerEvent::IsReady() now uses atomic load instead of a much less efficient atomic DWCAS.
- gRPC deadline propagation now uses less
now()
calls and measures time more precisely.
- About 16% faster HttpResponse::SetHeader due to using a vectorization friendly algorithm.
- Improved Redis driver start-up time.
- dynamic_config::DocsMap::Parse() and dynamic configs retrieval copies less strings.
- Build:
- New Docker container for developing userver based projects
ghcr.io/userver-framework/ubuntu-22.04-userver-base:latest
. It contains all the build dependencies preinstalled and a proper setup of PPAs with databases, compilers and tools.
- Switched to a modern python
venv
instead of the older virtualenv
.
- Use system provided Boost.Stacktrace if it is available.
- Documentation and Diagnostics:
January 2024
December 2023
- LISTEN/NOTIFY support for PostgreSQL. See storages::postgres::Cluster::Listen() and storages::postgres::NotifyScope for more information.
- components::Server now has
tls.ca
option for client-side certificates check.
- Caches now have a
full-update-jitter
option to avoid simultaneous full updates in different servers.
- Fault injection into transactions. See pytest_userver.sql.RegisteredTrx for sample usage.
- Improved diagnostics for improper Secdist usage.
- Added storages::postgres::ResultSet::AsOptionalSingleRow. Thanks to Jiawei He for the PR!
- Build:
- Big rewrite of CMake logic. Now
userver_setup_environment()
function is used to setup the environment:
- before:
include(third_party/userver/cmake/SetupEnvironment.cmake)
add_subdirectory(third_party/userver)
- after:
add_subdirectory(third_party/userver)
userver_setup_environment()
Requirements are not code generated any more and the default requirements used on userver_testsuite_add_simple()
CMake function call. add_grpc_library()
renamed to userver_add_grpc_library()
.
USERVER_PIP_USE_SYSTEM_PACKAGES
and USERVER_PIP_OPTIONS
CMake options now allow building the userver without internet connection. Thanks to Nikita for the PR!
- Curl library now downloaded automatically if no suitable version found.
- Added missing check for Mongo in Conan. Thanks to Pavel Talashchenko for the PR!
- Fixed sanitizers error message in CMake. Thanks to Kirill Pavlov for the PR!
- Updated MacOS dependencies. Thanks to Kirill Morozov for the PRs!
- Added PostgreSQL 16 detection. Thanks to linuxnyasha for the PR!
- Optimizations:
- Optimized Http::Cookie::ToString(). Thanks to Илья Оплачкин for the PR!
- Documentation:
- Documented Secdist formats and added notes on possible environment variables usage instead of secdist.
- Layout of reference pages changes. Now descriptions are at the top of the page rather than being after the list of methods and members.
- Documented Mongo pool metrics at Service Statistics and Metrics (Prometheus/Graphite/...).
- New gRPC middleware sample.
- Switched to Doxygen 1.10 with our search patches upstreamed
November 2023
- formats::json::Schema and validation function for it formats::json::Validate were implemented. Many thanks to Aleksej Kamenev for the PR!
- PostgreSQL driver now auto detects the max pool size, see uPg: Automatic connection limit
- server::http::HttpRequest:GetMethod() now can return server::http::HttpMethod::kHEAD. Thanks to Kirill Zimnikov for the bugreport and PR!
- PostgreSQL now has the following static config options:
check-user-types
option to prevent service from starting if database is not ready for work (for example: some migrations were not applied).
discard-all-on-connect
- to force running DISCARD ALL
on new connections, which could be useful for some PostgreSQLs smart-proxies that reuse the same connections.
- Boost.PFR 2.2.0 is now in use. The door for compile time reflection pull requests is now open!
- More metrics and fallbacks for logging errors.
- More tests for utils::datetime::MockSteadyNow. Thanks to Aleksej Kamenev for the PR!
- components::CachingComponentBase now has a static config option
alert-on-failing-to-update-times
to fire an alert if the cache update failed specified amount of times in a row.
- Optimizations:
- Caching of
dl_phdr_info
items is not ON by default, leading up to multiple seconds faster exception handling under heavy load when parts of the executable still being on hard drive rather than in memory. Use USERVER_DISABLE_PHDR_CACHE
CMake option to disable it, if the framework reports dlopen
usage after component start and there's no way to avoid it.
- Optimized discarding logs by log level from 4ns to 2ns. Added CMake option
USERVER_FEATURE_ERASE_LOG_WITH_LEVEL
to totally eliminate all the CPU and binary size overhead from logging.
- Implemented utils::SmallString::resize_and_overwrite() function as in C++23 std::string. Thanks to Илья Оплачкин for the PR.
- Up to an order of magnitude faster generation of HTTP response header via utils::SmallString usage. Thanks to Илья Оплачкин for the PR.
- Faster logging of durations in tracing::Span, tracing::ScopeTime and friends.
- Parsing inside storages::postgres::ResultSet now does not compute parsing-failure diagnostic information if parsing is successful.
- Return PostgreSQL connection to pool earlier, before updating the statistics.
- fs::RewriteFileContents() now does not
fsync
directories, dumping caches now also do not fsync
directories leading to better performance while still properly restoring after server power-off.
- PostgreSQL typed parsing was optimized to not copy std::shared_ptrs.
- TESTPOINT() and TESTPOINT_CALLBACK() now produce less instructions and guaranteed to not throw it the testpoints are disabled.
- Documentation:
- gRPC now has a deeper explanation of middlewares
- New Dynamic config page and related samples.
- Samples were significantly simplified, more static configuration options now have good defaults and do not require explicit setup.
- Configure, Build and Install now contains information on how to build service templates. Information on how to build the framework tests was moved to scripts/docs/en/userver/build/userver.md
- Documented the server::handlers::ImplicitOptions.
October 2023
- Added from-code alerts via alerts::Storage client and alerts::StorageComponent component.
- Added utils::statistics::Histogram.
- HTTP client now uses RATE metrics were possible.
- tracing::DefaultTracingManagerLocator now has
incoming-format
and new-requests-format
static config options to configure the tracing headers to receive and send.
- Clickhouse now supports Array(T) type
- Added
max-ttl-sec
to the POSTGRES_CONNECTION_SETTINGS dynamic config to force connections reopening.
- RabbitMQ now provides a Get() command to retrieve a single message. Thanks to Vladislav Nepogodin for the PR!
- Optimizations:
- Coroutines working set was reduced, leading to significantly less memory usage under most of the loads that do not use all the coroutines from pool at once.
- ugrpc::server::ServerComponent now provides a
completion-queue-count
to scale better under heavy load.
- Primary address string is now cached in HTTP server connection, leading to minor performance improvement.
- cache::NWayLRU now works faster due to improved hashing logic.
- Internal per-task timer size was reduced by 176 bytes.
- Removed rcu::Variable usage on hot path of tracing::Span destruction, optimized writing of Opentracing files.
- Dynamic configs were changed:
- dynamic_config::Key now requires config defaults at construction.
dynamic-config-fallbacks
component was merged into dynamic-config
, see components::DynamicConfig for more info.
- utils::DaemonMain now has
--print-dynamic-config-defaults
program option that prints all the in-code defaults of the dynamic configs.
- In-code defaults now could be overridden in static config of components::DynamicConfig via
defaults
option.
USERVER_CHECK_AUTH_IN_HANDLERS
was removed.
- Documentation:
- Build:
- Multiple build fixes and improvements for gRPC, including gRPC download via CPM.
- Conan was updated, thanks to Anton for the PR.
Release v1.0.0
Big new features since the Beta announcement:
- Implemented WebSockets server
- Added MySQL driver
- RabbitMQ drived was added
- Implemented TLS server
- Enabled PostgreSQL pipelining
- Implemented and enabled Deadline Propagation
- Improved experience with metrics. Added Prometheus and Graphite metric formats. Provided a modern simple interface to write and test metrics.
- Added chaos tests for drivers
- Implemented streaming API for the HTTP
- Improved documentation, added more samples and descriptions, improved search.
- Numerous optimizations.
- Numerous build improvements, including Conan and Docker support.
Optimized and improved features that were available at the Beta announcement:
- gRPC client and server
- Mongo driver
- Redis driver
- PostgreSQL
- HTTP server and client
- Logging and Tracing
- ... and many other features.
Detailed descriptions could be found below.
Beta (September 2023)
- WebSockets server and TLS server are now implemented as part of the HTTP server
- PostgreSQL pipelining is now implemented and turned on by default, leading to improved response times.
- Mongo Congestion Control is implemented and turned on by default, leading to better stability of the service in case of high load on database.
- Initial logger is now initialized from the component config, leading to a more simple code and setup. The
--initial-logger
option now does nothing and will be removed soon.
- Added a `userver_testsuite_add_simple()` CMake function to simplify testsuite CMake configuration.
- Expanded list of HTTP codes, thanks to Vladimir Popov for the PR!
- Projects from Yandex Schools were updated by the original authors. Thanks to bshirokov, Almaz Shagiev, Konstantin Antoniadi, Mingaripov Niyaz, Ilya Golosov and all the participants for the the great work and PRs!
- Build:
- New versions of
yaml-cpp
library are now supported. Thanks to Nikita for the bug report!
- Supported compilation with fmt 10.1.0. Thanks to Vladislav Nepogodin for the PRs!
- Fixed unused result warning. Thanks to Vladislav Nepogodin for the PR!
- Fixed use of deprecated API. Thanks to Vladislav Nepogodin for the PR!
- Fixed build with GCC-13 libstdc++. Thanks to Vladislav Nepogodin for the PR!
- Fixed MacOS Protobuf discovery. Thanks to Pavel Talashchenko for the PR!
- Fixed multiple other build warnings and issues.
Beta (August 2023)
- Deadline Propagation is now implemented and documented.
- Projects from Yandex Schools:
- Documentation was redesigned by hellenisaeva and MariaGrinchenko; new design was made up by Fedor Alekseev, fleshr, Anna Zagrebaylo, Michael Talalaev; the whole process was managed by Oleg Komarov with feedback from marketing specialist makachusha. Many thanks to all the participants for the great work.
- Implemented initial API for RFC 7616 - HTTP Digest Access Authentication, thanks to Almaz Shagiev, Konstantin Antoniadi, Ilya Golosov, and Niyaz for multiple PRs!
- std::bitset<N>, std::array<bool, N>, userver::utils::Flags<Enum> and integral values are now mapped to PostgreSQL bit and bit varying types. Thanks to dsaa27 and bshirokov for the PRs!
- Initial
Realmedium
sample was implemented and moved into a separate repository at userver-framework/realmedium_sample. Thanks to berholz, rumxcola, Konstantin Artemev, GasikPasik, Anna Volkova, Nikita Semenov, Daniil Boriskin, VVendor for the PR!
- Metrics for CPU usage of particular threads of task processors are now available via new engine::TaskProcessorsLoadMonitor component.
- gRPC and HTTP handler metrics were switched to utils::statistics::Rate, leading to better handling of service start/stop by metrics storage.
- server::Server now returns HTTP header "Content-Type: application/octet-stream" if the content type was not set.
- Implemented
ToString(utils::StrongTypedef<floating point>)
, thanks to Daniil Shvalov for the PR.
- Optimizations:
- Signifiacally optimized JSON DOM access to a non-existing element.
- Logging now does not lock a mutex in fs-task-processor on each log record.
- Up to 3 times faster logging due to multiple minor tweaks with data copy and memory prereserving.
- The new asynchronous engine::io::sys::linux::Inotify driver for Linux allows subscriptions to filesystem events. Thanks to it the fs::FsCacheClient now reacts faster.
- Docs:
- Build:
- Dropped spdlog dependency
- CPM library is now used for managing third-party dependencies.
- Fixed cryptopp compilation error, thanks to Daniil Shvalov for the PR!
- Fixed -Wsign-conversion -Wctad-maybe-unsupported warnings in utils::TrivialBiMap, thanks to darktime78 for the PR!
Beta (July 2023)
- server::http::CustomHandlerException now allows to provide a custom HTTP status that is not mapped to protocol-agnostic server::handlers::HandlerErrorCode.
- Non-coroutine
userver-universal
CMake target was refactored and is now used by the whole framework as a basic dependency. Compile times dropped down drastically for building the whole framework from scratch.
- Added a non-coroutine usage example Non-Coroutine Console Tool.
- It is now possible to subscribe to
SIGUSR1
and SIGUSR2
in the same class. Thanks to Beshkent for the bug report.
- Dynamic config management commands were added to
uctl
tool.
- Optimizations:
- storages::redis::SubscribeClient subscriptions were optimized to do less dynamic allocations on new message and use a faster lock-free queue.
- Escaping of tags for logging is now done at compile-time by default, which gives up to 30% speedup in some cases.
- Internal asynchronous logging logic was rewritten, leading to better scalability and more than 25% speedup.
- HTTP client connection is now preserved on deadline, leading to less connections being reopened.
- Cleaned up and added docs for redis::CommandControl.
- MacOS build instructions were enhanced, thanks to Daniil Shvalov for the PR!
- Conan build was fixed, thanks to Yuri Bogomolov for the PR.
- Numerous build and configure fixes.
Beta (June 2023)
- Static configs of the service now can retrieve environment variables via the
#env
syntax. See yaml_config::YamlConfig for more examples.
- New testsuite plugins userver_config_http_client and userver_config_testsuite_support turned on by default to increase timeouts in tests and make the functional tests more reliable.
- gRPC now can write access logs, see
access-tskv-logger
static option of ugrpc::server::ServerComponent.
- Now the waiter is allowed to destroy the engine::SingleConsumerEvent immediately after exiting WaitForEvent, if the wait succeeded.
- New dynamic_config_fallback_patch fixture could be used to replace some dynamic config values specifically for testsuite tests.
- Fixed utils::SmallString tests, thanks to Chingiz Sabdenov for the PR!
- Optimizations:
- Counters in the tasks were optimized. New thread local counters use less CPU and scale better on huge amounts of coroutines.
- engine::TaskProcessor optimized layout with interference shielded atomics uses less CPU and scales better on huge amounts of coroutines.
- Minor optimizations for HTTP clients URL manipulations.
- tracing::Span generation of ID was made faster.
- clients::http::Request now does less atomic counters increments/decrements while the request is being built.
mlock_debug_info
static configuration option of components::ManagerControllerComponent is now on by default. It improves responsiveness of the service under heavy load on low memory and bad hard drives.
- engine::io::Socket now has an additional
SendAll
overload that accepts const struct iovec* list, std::size_t list_size
for implementing low-level vector sends. Mongo driver now uses the new function, resulting in smaller CPU and memory consumption.
- clients::http::Form::AddContent now instead of
const std::string&
parameters accepts std::string_view
parameters that allow to copy less data.
- Docs and diagnostics:
- Build:
- Multiple build fixes for gRPC targets.
- Clickhouse for Conan was added, thanks to Anton for the PR.
- CMake build flag
USERVER_FEATURE_UBOOST_CORO
can be used to use system boost::context.
Beta (May 2023)
- New scripts/uctl/uctl console script for administration of the running service was added.
- Improved compile times by removing multiple includes from userver headers including templating the serializers of different formats.
- Implemented ugrpc::server::HealthComponent handler, a gRPC alternative to server::handlers::Ping.
- gRPC server and clients now support middlewares - a customization plugins that could be shared by different handlers.
- Invalid implementations of CacheUpdateTrait::Update are now detected and logged.
- Optimizations:
- Significant improvements in HTTP handling due to new http::headers::HeaderMap usage instead of std::unordered_map.
- Improved performance of utils::TrivialBiMap by an order of magnitude for enum-to-enum mappings, thanks to Vlad Tytskiy for the bug report!
- utils::FromString now uses std::from_chars for better performance.
- TESTPOINT and other testpoint related macro now imply zero overhead if testpoints were disabled in static config.
- More functions of formats::json::ValueBuilder now accept std::string_view, resulting in less std::string constructions and better performance.
- TSKV escaping was optimized via SIMD, resulting in up to x10 speedup on long logs.
mlock_debug_info
static configuration option of components::ManagerControllerComponent now allows to lock exception unwinding information in memory. It improves responsiveness of the service under heavy load on low memory and bad hard drives.
- Docs:
- Build:
- Arch Linux instructions were improved, thanks to Kirill Zimnikov for the PR!
- Fixed Conan based builds, thanks to Anton for the PR.
- Clickhouse-cpp version was raised to 2.4.0, thanks to Kirill Zimnikov for the PR!
- Fixed build on libstdc++ from GCC-13, thanks to Kirill Zimnikov for the PR!
- Fixed benchmarks build on non x86 targets.
- Rewrite of Protobuf and gRPC locating logic.
Beta (April 2023)
- MySQL driver was added, see MySQL Driver.
- Experimental support for HTTP "Baggage" header is implemented, including verification, forwarding from HTTP handlers to client, baggage manipulation. See baggage::BaggageManagerComponent for more info.
- Redis driver now supports non-queued variants for pubsub.
- Redis driver now supports read-only transactions.
- utils::FilterBloom was merged in along with initial SLRU cache implementations. The work is a part of the backend development school course project by Leonid Faktorovich, Alexandr Starovoytov, Ruslan, Egor Chistyakov from PR #262.
- HTTP request decompression is now ON by default in server::handlers::HandlerBase.
- dynamic_config::Source now allows subscribing to a particular dynamic config variable changes.
- Initial support for utils::statistics::Rate metrics type.
- Human-readable "pretty" format (utils::statistics::ToPrettyFormat) for metrics output was added to the server::handlers::ServerMonitor.
- Optimizations:
- Don't issue tail
writev
for empty io_vec
on bulk socket writes.
- All the userver metrics are now written via the fast utils::statistics::Writer.
- x2-x50 faster serialization of unique maps into formats::json::ValueBuilder.
- utils::StrIcaseHash became slightly faster.
- engine::Task now does not have a virtual destructor. New engine::TaskBase based hierarchy does not use RTTI, resulting in smaller binaries.
- Mongo driver does not capture stack traces in release builds in case of errors. The error path become slightly faster, server is more responsive in case of Mongo problems.
- Build:
- Improved support for Conan 2.0, many thanks to Anton for the PR.
.gitattributes
now handles line endings automatically for files detected as text. This simplifies WSL builds. Thanks to Anatoly Shirokov for the PR.
- PostgreSQL libs selection is now possible in CMake if the platform has multiple versions installed, see PostgreSQL versions for more info.
- Improved support for Arch Linux, many thanks to Konstantin Goncharik for the PR.
- Multiple improvements for docs, including mockserver clarifications from Victor Makarov.
Beta (March 2023)
- Now logging of particular lines could be controlled by dynamic config. See USERVER_LOG_DYNAMIC_DEBUG for more info.
- HTTP headers that contain the tracing context now could be customized, both for handlers and HttpClient, by feeding tracing::TracingManagerBase implementation into tracing::DefaultTracingManagerLocator (docs to come soon).
- User defined literals for different formats are now available at formats::literals
- Added crypto::CmsSigner and crypto::CmsVerifier as per RFC 5652.
- cache::LruMap and cache::LruSet now work with non-movable types.
- BSON<>JSON conversions now supported via json_value.ConvertTo<formats::bson::Value>() and bson_value.ConvertTo<formats::json::Value>().
- engine::CancellableSemaphore was implemented.
- Optimizations:
- Mongo driver switched to a faster utils::statistics::Writer.
- utils::Async functions now make 1 dynamic allocation less, thanks to Ivan Trofimov for the PR.
- Getting the default logger now takes only a single atomic read. LOG_* macro now do two RMW atomic operations less and do not use RCU, that could lead to a dynamic memory allocation in rare cases.
- PostgreSQL driver now does much less atomic operations due to wider usage of std::move on the internal std::shared_ptr.
- Added storages::postgres::Transaction::ExecuteDecomposeBulk function for fast insertion of C++ array of structures as arrays of values.
- Str[I]caseHash now uses a 5-10 times faster SipHash13
- Redis driver now does an asynchronous DNS resolving, amount of heavy system calls dropped down noticeably.
- Build changes:
- CMake option
USERVER_OPEN_SOURCE_BUILD
was removed as the build is always the same for in-house and public environments.
- CMake option
USERVER_FEATURE_SPDLOG_TCP_SINK
was removed as now the implementation of the sink does not rely on spdlog implementation.
- Configuration step was made much faster.
- Makefile was simplified and only up-to-date targets were left.
- Added a script to prepare docker build, see Building Docker container for userver for more info.
- Scripts for generating CMakeLists were simplified and cleared from internal stuff.
- Added missing dependencies to scripts/docs/en/deps_ubuntu_.md20_04 and sorted all the dependencies, thanks to Anatoly Shirokov for the PR.
- Statistics and metrics now do additional lifetime checks in debug builds to catch improper usages.
- Push functions of concurrent::MpscQueue and other queues now have a
[[nodiscard]]
for compile time misuse detection.
- Significant improvement of Redis server-side errors diagnostic.
- Improved diagnostics for distributed locking.
- Fixed numerous typos in docs and samples.
Beta (February 2023)
- tracing::Span now logs the location where it was constructed.
- Now the string<>enum utils::TrivialBiMap mappings could be used within storages::postgres::io::CppToUserPg. See pg_enum for more info.
- components::Logging now provides options for configuring the task processors to do the asynchronous logging.
- Redis driver now supports the
geosearch
and unlink
commands.
- DNS resolvers were switched to asynchronous mode by default.
- UDP chaos proxy was implemented, see chaos.UdpGate
- C++ Standard now could be explicitly controlled via the CMake flag
CMAKE_CXX_STANDARD
.
- Optimizations:
- Many metrics were moved to a faster utils::statistics::Writer.
- gRPC now allows concurrent execution of 1 Read and 1 Write on the same Bidirectional stream.
- Python dependencies for build are now automatically installed, thanks to Pavel Shuvalov for the PR!
- Added a Conan option to disable LTO, thanks to Oleg Burchakov for the PR!
- Diagnostic messages and docs were improved, including:
- In case of a typo in static config name the fixed name is now reported
error-injection
static option for components::Postgres was documented
- Fixed typos, thanks to Ch0p1k3 for the PR!
Beta (January 2023)
- Unknown/mistyped values in static configs are now reported by default. Static configs validation now could use
minimum
and maximum
.
- gRPC clients now have ReadAsync() functions, that return a future and allow to request multiple results from different RPCs at the same moment.
- ugrpc::server::Server now can return a vector of gRPC service names.
- To aid in asynchronous drivers development the engine::io::FdPoller is now a part of the public API.
- Added a blazing fast utils::TrivialBiMap.
- HTTP Streaming is now considered production ready, see HTTP, HTTPS, WebSocket for docs.
- Testsuite fixtures were improved:
- Optimizations:
- Now the engine does less random number generator invocations for HTTP handling.
- Logging of tracing::Span became x2 faster
- IntrusiveMpscQueue is now used for engine internals, IO operations now schedule faster.
- Writing HTTP headers became faster, thanks to Ivan Trofimov for the PR.
- utils::TrivialBiMap is now used wherever it is possible.
- Metrics:
- FreeBSD build fixes.
- Multiple documentation and diagnostics improvements.
- Added
ToStringView(HttpMethod)
function, thanks to Фёдор Барков for the PR.
- Added more engine::Yield tests, thanks to Ivan Trofimov for the PR.
Beta (December 2022)
- Added logs colorization to the testsuite plugins, thanks to Victor Makarov for the PR.
- Multiple big improvements in framework testing:
- improved unit tests re-entrance
- multiple new chaos and metrics tests
- improved testsuite diagnostics
- Added a Custom Authorization/Authentication via PostgreSQL sample.
- Added an option
set_tracing_headers
to disable HTTP tracing headers, thanks to Ivan Trofimov for the PR.
- Fixed race in RabbitMQ sample, thanks to Ivan Trofimov for the PR.
- Fixed PostgreSQL testing at GithubCI, thanks to Ivan Trofimov for the PR.
Beta (November 2022)
Beta (October 2022)
- Experimental support for Conan packaging, many thanks to Anton for the PR.
- Prometheus and Graphite metrics formats were added, see Service Statistics and Metrics (Prometheus/Graphite/...) for details.
- Initial support for chaos testing was added, see Chaos Testing for more info.
- Generic Escape implementation for ranges was added to Clickhouse, thanks to Ivan Trofimov for the PR.
- TLS/SSL support for Redis.
- Multiple optimizations from Ivan Trofimov:
- utils::datetime::WallCoarseClock and its usage in the framework core;
- HTTP "Date" header caching;
- Significant reduction of syscalls count during any recv operations;
- Server::GetRequestsView not initialized if it is not required;
- utils::encoding::ToHex became faster;
- Marking response as ready became faster.
- Better diagnostics for CoroPool initialization failure, thanks to Ivan Trofimov for the PR.
- New handler server::handlers::OnLogRotate.
- Multiple optimizations, including:
- Faster async cancellations for PostgreSQL;
- Avoid using dynamic_cast in multiple places;
- Avoid calling
std::chrono::steady_clock::now()
in multiple places.
- gRPC mockserver support (see gRPC client and service).
- gRPC now provides an efficient API for async execution of requests without additional
utils::Async
invocations.
- Build fixes for older platforms, thanks to Yuri Bogomolov for the PR.
- components::Logging now can output logs to UNIX sockets.
- Now the "help wanted" issues at github have additional tags "good first issue" and "big", to help you to choose between a good starting issue and a big feature. See Releases, Trunk-based Development and Pull Requests for more info.
Beta (September 2022)
- Ivan Trofimov implemented the RabbitMQ driver.
- Added navigation to the next and previous page in docs, thanks to multiple feature requests in Telegram support chat.
- Improved Task::Detach docs and added recommendation to use concurrent::BackgroundTaskStorage instead, thanks to Ivan Trofimov for the bugreport.
- Added
start-
targets for the samples, to simplify experimenting with them.
- Docs now support dark theme out of the box.
- Fixed CMake issue with
-DUSERVER_FEATURE_TESTSUITE=0
, thanks to Георгий Попов for the bugreport.
- Fixed build on Arch Linux, thanks to Mikhail K. for the bugreport.
- Fixed building in virtual environment on Windows, thanks to sabudilovskiy for the bug report.
- Fixed building with
-std=gnu++20
, thanks to Георгий Попов for the PR.
- Improved package version detection in CMake via
pkg-config
.
- Added a
USERVER_FEATURE_UTEST
flag for disabling utest and ubench target builds, thanks to Anton for the PR.
- Simplified gRPC component registration and usage.
- Added an ability to turn on gRPCs ChannelZ.
- Added evalsha/script load commands for Redis driver.
Beta (August 2022)
- Added server::handlers::HttpHandlerStatic handler to serve static pages.
- Added navigation to previous and next page in docs.
- Optimized internals:
- WaitListLight now never calls std::this_thread::yield().
- More lightweight queues are now used in HTTP server.
- Smaller critical section in TaskContext::Sleep(), improved performance for many the synchronization primitives.
- std::unique_ptr now holds the payload in engine::Task, rather than std::shared_ptr. Thanks to the Stas Zvyagin for the idea and draft PR.
- Simplified and optimized FdControl, resulting in less CPU and memory usage for sockets, pipes and TLS.
- Fixed typos in tests, thanks to Георгий Попов for the PR.
- Removed suspicious operator, thanks to the PatriotRossii for the bugreport.
- Fixed CentOS 7.8 builds, many thanks to jinguoli for the bugreport and fix ideas.
- Fixed Gentoo builds, many thanks to SanyaNya for the PR.
- Fixed default DB values in uservice-dynconf, many thanks to skene2321 for the bugreport.
- Added engine::io::WritableBase and engine::io::RwBase, thanks to Stas Zvyagin for the idea.
- Added components::TcpAcceptorBase with new tutorials TCP half-duplex server with static configs validation and TCP full-duplex server with metrics and Spans, thanks to Stas Zvyagin for the idea and usage samples at https://github.com/szvyagin-gj/unetwork.
- Fixed comparison operator for UserScope, thanks to PatriotRossii for the PR.
- Add CryptoPP version download during CryptoPP installation, thanks to Konstantin for the PR.
- Added more documentation on Non FIFO queues, thanks to Ivan Trofimov for the report.
- Added missing std::atomic into TaskProcessor, thanks to Ivan Trofimov for the report.
- Fixed Boost version detection on MacOs, thanks to Konstantin for the PR.
- Added Fedora 36 support, thanks to Benjamin Conlan for the PR.
- Improved statistics for gRPC.
- Vector versions of engine::io::Socket::SendAll were added and used to optimize CPU and memory consumption during HTTP response sends.
Pre announce (May-Jul 2022)
- Fixed engine::io::TlsWrapper retries, thanks to Ivan Trofimov for the report.
- Fixed missing
const
in utils::DaemonMain function, thanks to Denis Chernikov for the PR.
- Cmake function
userver_testsuite_add
now can pass arguments to virtualenv
, thanks to Дмитрий Изволов for the PR.
- Improved hello_service run instruction, thanks to Svirex for the PR.
- Better Python3 detection, thanks to Дмитрий Изволов for the PR.
- Task processors now have an
os-scheduling
static config option and @md_en_userver_task_processors_guide "a usage guide".
- Added a pg_service_template service template that uses userver the userver framework with PostgreSQL
- In template services, it is now possible to deploy the environment and run the service in one command:
make service-start-debug
or make service-start-release
.
- Added userver::os_signals::Component, which is used for handling OS signals.
- You can now allow skipping the component in the static config file by specializing the components::kConfigFileMode, see the documentation.
- The PostgreSQL driver now requires explicit serialization methods when working with enum.
- Optimized CPU consumption for handlers that do not log requests or responses.
- utils::Async() now can be invoked from non-coroutine thread (in that case do not forget to use engine::Task::BlockingWait() to wait for it). tracing::Span construction became faster. Thanks to Ivan Trofimov for the report.
- Improved MacOS support, thanks to Evgeny Medvedev.
- Docker support: base image for development, docker-compose.yaml for the userver with build and test targets. See Configure, Build and Install
- Docs improved: removed internal links; added Feature Comparison with other Frameworks, Supported Platforms, Security Changelog, Profiling context switches, Driver Writing Guide, @md_en_userver_task_processors_guide, Handling OS signals and Roadmap and Changelog.
- AArch64 build supported. Tests pass successfully
- HTTP headers hashing not vulnerable to HashDOS any more, thanks to Ivan Trofimov for the report.
- engine::WaitAny now can wait for engine::Future, including futures that are signaled by engine::Promise from non-coroutine environment.
- Optimized the PostgreSql driver, thanks to Dmitry Sokolov for the idea.
- Arch Linux is now properly supported, thanks to Denis Sheremet and Konstantin Goncharik.
- Published a service to manage dynamic configs of other userver-based services.
- Now it is possible to enable logging a particular LOG_XXX by its source location, see server::handlers::DynamicDebugLog for more details.
- Added a wrapper class utils::NotNull and aliases utils::UniqueRef and utils::SharedRef.
- LTSV-format is now available for logs via components::Logging static option