🐙 userver has built-in support for functional service tests using Yandex.Taxi Testsuite. Testsuite is based on pytest and allows developers to test their services in isolated environment. It starts service binary with minimal database and all external services mocked then allows developer to call service handlers and test their result.
SERVICE_TARGET, required CMake name of the target service to test. Used as suffix for testsuite- and start- CMake target names.
WORKING_DIRECTORY, pytest working directory. Default is ${CMAKE_CURRENT_SOURCE_DIR}.
PYTEST_ARGS, list of extra arguments passed to pytest.
PYTHONPATH, list of directories to be prepended to PYTHONPATH.
REQUIREMENTS, list of requirements.txt files used to populate virtualenv.
PYTHON_BINARY, path to existing Python binary.
VIRTUALENV_ARGS, list of extra arguments passed to virtualenv.
PRETTY_LOGS, set to OFF to disable pretty printing.
Some of the most useful arguments for PYTEST_ARGS:
Argument
Description
-v
Increase verbosity
-s
Do not intercept stdout and stderr
-x
Stop on first error
-k EXPRESSION
Filter tests by expression
--service-logs-pretty
Enable logs coloring
--service-logs-pretty-disable
Disable logs coloring
--service-log-level=LEVEL
Set the log level for the service. Possible values: trace, debug, info, warning, error, critical
--service-wait
With this argument the testsuite will wait for the service start by user. For example under gdb. Testsuite outputs a hint on starting the service
-rf
Show a summary of failed tests
CMake integration via <tt>userver_testsuite_add_simple()</tt>
userver_testsuite_add_simple() is a version of userver_testsuite_add() that makes some assumptions of the project structure. It should be invoked from the service's CMakeLists.txt as follows:
userver_testsuite_add_simple(
PYTHON_BINARY "${TESTSUITE_PYTHON_BINARY}"
)
It supports the following file structure (and a few others):
userver_testsuite_add() registers a ctest target with name testsuite-${SERVICE_TARGET}. Run all project tests with ctest command or use filters to run specific tests:
ctest -V -R testsuite-my-project # SERVICE_TARGET argument is used
Direct run
To run pytest directly userver_testsuite_add() creates a testsuite runner script that could be found in corresponding binary directory. This may be useful to run a single testcase, to start the testsuite with gdb or to start the testsuite with extra pytest arguments:
Now you can open a new terminal window and run this command in it or if you use an IDE you can find the corresponding CMake target and add arg --config /.../config.yaml. After that it will be possible to set breakpoints and start target with debug.
pytest_userver
By default pytest_userver plugin is included in python path. It provides basic testsuite support for userver service. To use it add it to your pytest_plugins in root conftest.py:
The plugins match the userver cmake targets. For example, if the service links with userver-core its tests should use the pytest_userver.plugins.core plugin.
Please note that the testsuite support must be disabled in production environment. Testsuite sets the testsuite-enabled variable into true when runs the service. In the example above this variable controls whether or tests-control component is loaded.
pytest_userver modifies static configs config.yaml and config_vars.yaml passed to pytest before starting the userver based service.
To apply additional modifications to the static config files declare USERVER_CONFIG_HOOKS variable in your pytest-plugin with a list of functions or pytest-fixtures that should be run before config is written to disk. USERVER_CONFIG_HOOKS values are collected across different files and all the collected functions and fixtures are applied.
Note that auto_client_deps fixture already knows about the userver supported databases and clients, so usually you do not need to manually register any dependencies.
Mockserver
Mockserver allows to mock external HTTP handlers. It starts its own HTTP server that receives HTTP traffic from the service being tested. And allows to install custom HTTP handlers within testsuite. In order to use it all HTTP clients must be pointed to mockserver address.
It provides TESTPOINT() and family of TESTPOINT_CALLBACK() macros that do nothing in production environment and only work when run under testsuite. Under testsuite they only make sense when corresponding testsuite handler is installed.
All testpoints has their own name that is used to call named testsuite handler. Second argument is formats::json::Value() instance that is only evaluated under testsuite.
In order to eliminate unnecessary testpoint requests userver keeps track of testpoints that have testsuite handlers installed. Usually testpoint handlers are declared before first call to service_client which implicitly updates userver's list of testpoint. Sometimes it might be required to manually update server state. This can be achieved using service_client.update_server_state() method e.g.:
@testpoint('injection-point')
def injection_point(data):
return {'value': 'injected'}
await service_client.update_server_state()
assert injection_point.times_called == 0
Accessing testpoint userver is not aware of will raise an exception:
Testsuite can be used to test logs written by service. To achieve this the testsuite starts a simple logs capture TCP server and tells the service to replicate logs to it on per test basis.
Example usage:
async def test_select(service_client):
asyncwith service_client.capture_logs() as capture:
Testsuite tasks facility allows to register a custom function and call it by name from testsuite. It's useful for testing components that perform periodic job not related to its own HTTP handler.
Different monitoring systems and time series databases have different limitations. To make sure that the metrics of your service could be used on most of the popular systems, there is special action in server::handlers::TestsControl.
To use it you could just write the following test:
Testsuite provides a way to start standalone service with all mocks and database started. This can be done by adding --service-runner-mode flag to pytest, e.g.:
