Before you start
Make sure that you can compile and run core tests and read a basic example Writing your first HTTP server.
Step by step guide
Microservices that have state often work with database to store their data and replicate that state across instances of the microservice. In this tutorial we will write a service that is a simple key-value storage on top of Redis database. The service would have the following Rest API:
- HTTP POST by path
/v1/key-value with query parameters key and value stores the provided key and value or 409 Conflict if such key already exists
- HTTP GET by path
/v1/key-value with query parameter key returns the value if it exists or 404 Not Found if it is missing
- HTTP DELETE by path
/v1/key-value with query parameter key deletes the key if it exists and returns number of deleted keys (cannot be more than 1, since keys are unique in Redis database)
HTTP handler component
Like in Writing your first HTTP server we create a component for handling HTTP requests:
#include <userver/storages/secdist/provider_component.hpp>
namespace samples::redis {
public:
static constexpr std::string_view kName = "handler-script";
const override;
private:
storages::redis::ClientPtr redis_client_;
storages::redis::CommandControl redis_cc_;
};
public:
static constexpr std::string_view kName = "handler-key-value";
const override;
private:
std::string DeleteValue(std::string_view key) const;
storages::redis::ClientPtr redis_client_;
storages::redis::CommandControl redis_cc_;
};
}
Note that the component holds a storages::redis::ClientPtr - a client to the Redis database. That client is thread safe, you can use it concurrently from different threads and tasks.
Initializing the database
To access the database from our new component we need to find the Redis component and request a client to a specific cluster by its name. After that we are ready to make requests.
: server::handlers::HttpHandlerBase(config, context),
redis_client_{context.FindComponent<
components::Redis>(
"key-value-database").GetClient(
"taxi-tmp")},
redis_cc_{
std::chrono::seconds{15},
std::chrono::seconds{60}, 4} {}
KeyValue::HandleRequestThrow
In this sample we use a single handler to deal with all the HTTP methods. The KeyValue::HandleRequestThrow member function mostly dispatches the request to one of the member functions that actually implement the key-value storage logic:
std::string
const {
const auto& key = request.
GetArg(
"key");
if (key.empty()) {
}
switch (request.GetMethod()) {
case server::http::HttpMethod::kGet:
return GetValue(key, request);
case server::http::HttpMethod::kPost:
return PostValue(key, request);
case server::http::HttpMethod::kDelete:
return DeleteValue(key);
default:
fmt::format("Unsupported method {}", request.GetMethod())});
}
}
- Warning
Handle* functions are invoked concurrently on the same instance of the handler class. In this sample the KeyValue component only uses the thread safe DB client. In more complex cases synchronization primitives should be used or data must not be mutated.
KeyValue::GetValue
Executing a query to the Redis database is as simple as calling the corresponding method of storages::redis::ClientPtr.
Note that some methods return an optional result, which must be checked. Here it can indicate a missing key value.
const auto result = redis_client_->Get(std::string{key}, redis_cc_).Get();
if (!result) {
return {};
}
return *result;
}
KeyValue::PostValue
Here we use storages::redis::Client::SetIfNotExist() to ensure not to change already existing keys.
const auto& value = request.
GetArg(
"value");
const auto result = redis_client_->SetIfNotExist(std::string{key}, value, redis_cc_).Get();
if (!result) {
return {};
}
return std::string{value};
}
KeyValue::DeleteValue
Note that mutating queries are automatically executed on a master instance.
std::string KeyValue::DeleteValue(std::string_view key) const {
const auto result = redis_client_->Del(std::string{key}, redis_cc_).Get();
return std::to_string(result);
}
Static config
Static configuration of service is quite close to the configuration from Writing your first HTTP server except for the handler and DB:
# yaml
components_manager:
components: # Configuring components that were registered via component_list
handler-key-value:
path: /v1/key-value # Registering handler by URL '/v1/key-value'.
method: GET,POST,DELETE # GET, POST and DELETE requests only.
task_processor: main-task-processor # Run it on CPU bound task processor
handler-script:
path: /v1/script # Registering handler by URL '/v1/key-value'.
method: POST # GET, POST and DELETE requests only.
task_processor: main-task-processor # Run it on CPU bound task processor
key-value-database:
groups:
- config_name: taxi-tmp # Key to lookup in secdist configuration
db: taxi-tmp # Name to refer to the cluster in components::Redis::GetClient()
subscribe_groups: # Array of redis clusters to work with in subscribe mode
thread_pools:
redis_thread_pool_size: 8
sentinel_thread_pool_size: 1
# /// [Sample secdist static config]
# yaml
default-secdist-provider: # Component that loads configuration of hosts and passwords
config: /etc/redis_service/secure_data.json # Values are supposed to be stored in this file
missing-ok: true # ... but if the file is missing it is still ok
environment-secrets-key: SECDIST_CONFIG # ... values will be loaded from this environment value
# /// [Sample secdist static config]
testsuite-support:
server:
# ...
int main()
Finally, after writing down the dynamic config values into file at dynamic-config-fallbacks.fallback-path, we add our component to the components::MinimalServerComponentList(), and start the server with static config kStaticConfig.
int main(int argc, char* argv[]) {
.
Append<samples::redis::KeyValue>()
.Append<samples::redis::EvalSha>()
.Append<components::DefaultSecdistProvider>()
.Append<components::TestsuiteSupport>()
}
Build and Run
To build the sample, execute the following build steps at the userver root directory:
mkdir build_release
cd build_release
cmake -DCMAKE_BUILD_TYPE=Release ..
make userver-samples-redis_service
The sample could be started by running make start-userver-samples-redis_service. The command would invoke testsuite start target that sets proper paths in the configuration files, prepares and starts the DB, and starts the service.
To start the service manually start the DB server and run ./samples/redis_service/userver-samples-redis_service -c </path/to/static_config.yaml>.
Now you can send a request to your service from another terminal:
bash
$ curl -X POST 'localhost:8088/v1/key-value?key=hello&value=world' -i
HTTP/1.1 201 Created
Date: Wed, 27 Oct 2021 16:45:13 UTC
Content-Type: text/html
X-YaSpanId: 015fb0becd2926ef
X-YaRequestId: 7830671d7dd2462ba9043db532c2b82a
Server: userver/2.0 (20211027123413; rv:c1879aa03)
X-YaTraceId: d7422d7bcdc9493997fc687f8be24883
Connection: keep-alive
Content-Length: 5
world
$ curl -X DELETE 'localhost:8088/v1/key-value?key=hello&value=world' -i
HTTP/1.1 200 OK
Date: Wed, 27 Oct 2021 16:46:35 UTC
Content-Type: text/html
X-YaSpanId: e83698e2ef8cc729
X-YaRequestId: ffbaacae38e64bb588affa10b928b759
Server: userver/2.0 (20211027123413; rv:c1879aa03)
X-YaTraceId: cd3e6acc299742739bb22c795b6ef3a7
Connection: keep-alive
Content-Length: 1
1
Unit tests
Unit tests for the service could be implemented with one of UTEST macros in the following way:
auto client = GetClient();
client->Rpush("sample_list", "a", {}).Get();
client->Rpush("sample_list", "b", {}).Get();
const auto length = client->Llen("sample_list", {}).Get();
EXPECT_EQ(length, 2);
}
Functional testing
Functional tests for the service could be implemented using the testsuite. To do that you have to:
- Prepare the pytest by importing the pytest_userver.plugins.redis plugin:
import json
import pytest
pytest_plugins = ['pytest_userver.plugins.redis']
Add the Redis Secdist settings info to the service environment variable:
@pytest.fixture(scope='session')
def service_env(redis_sentinels):
secdist_config = {
'redis_settings': {
'taxi-tmp': {
'password': '',
'sentinels': redis_sentinels,
'shards': [{'name': 'test_master0'}],
},
},
}
return {'SECDIST_CONFIG': json.dumps(secdist_config)}
The auto_client_deps fixture already knows about the redis_store fixture, so there's no need to override the extra_client_deps fixture.
For details on Redis Secdist format, see components::Redis.
- Write the test:
async def test_redis(service_client):
response = await service_client.delete('/v1/key-value?key=hello')
assert response.status == 200
response = await service_client.post('/v1/key-value?key=hello&value=world')
assert response.status == 201
assert 'text/plain' in response.headers['Content-Type']
assert response.text == 'world'
response = await service_client.request('GET', '/v1/key-value?key=hello')
assert response.status == 200
assert 'text/plain' in response.headers['Content-Type']
assert response.text == 'world'
Full sources
See the full example:
