MongoDB service

Table of Contents

• Before you start
• Step by step guide
• HTTP handler component
• Translations::InsertNew
• Translations::ReturnDiff
• Static config
• Dynamic config
• int main()
• Build and Run
• Functional testing
• Full sources

Before you start

Make sure that you can compile and run core tests and read a basic example Writing your first HTTP server.

Step by step guide

In this tutorial we will write a service that stores history of translation changes and returns the most recent translations. MongoDB would be used as a database. The service would have the following Rest API:

• HTTP PATCH by path '/v1/translations' with query parameters 'key', 'lang' and 'value' updates a translation.
• HTTP GET by path '/v1/translations' with query parameter 'last_update' returns unique translations that were added after the 'last_update'.

HTTP handler component

Like in Writing your first HTTP server we create a component for handling HTTP requests:

namespace samples::mongodb {
 
class Translations final : public server::handlers::HttpHandlerBase {
 public:
  static constexpr std::string_view kName = "handler-translations";
 
  Translations(const components::ComponentConfig& config,
               const components::ComponentContext& context)
      : HttpHandlerBase(config, context),
        pool_(context.FindComponent<components::Mongo>("mongo-tr").GetPool()) {}
 
  std::string HandleRequestThrow(
      const server::http::HttpRequest& request,
      server::request::RequestContext&) const override {
    if (request.GetMethod() == server::http::HttpMethod::kPatch) {
      InsertNew(request);
      return {};
    } else {
      return ReturnDiff(request);
    }
  }
 
 private:
  void InsertNew(const server::http::HttpRequest& request) const;
  std::string ReturnDiff(const server::http::HttpRequest& request) const;
 
  storages::mongo::PoolPtr pool_;
};
 
}  // namespace samples::mongodb

Note that the component holds a storages::mongo::PoolPtr - a client to the Mongo. That client is thread safe, you can use it concurrently from different threads and tasks.

Translations::InsertNew

In the Translations::InsertNew function we get the request arguments and form a BSON document for insertion.

void Translations::InsertNew(const server::http::HttpRequest& request) const {
  const auto& key = request.GetArg("key");
  const auto& lang = request.GetArg("lang");
  const auto& value = request.GetArg("value");
 
  using formats::bson::MakeDoc;
  auto transl = pool_->GetCollection("translations");
  transl.InsertOne(MakeDoc("key", key, "lang", lang, "value", value));
  request.SetResponseStatus(server::http::HttpStatus::kCreated);
}

There are different ways to form a document, see Formats (JSON, YAML, BSON, ...).

Translations::ReturnDiff

MongoDB queries are just BSON documents. Each mongo document has an implicit _id field that stores the document creation time. Knowing that, we can use formats::bson::Oid::MakeMinimalFor() to find all the documents that were added after update_time. Query sorts the documents by modification times (by _id), so when the results are written into formats::json::ValueBuilder latter writes rewrite previous data for the same key.

std::string Translations::ReturnDiff(
    const server::http::HttpRequest& request) const {
  auto time_point = std::chrono::system_clock::time_point{};
  if (request.HasArg("last_update")) {
    const auto& update_time = request.GetArg("last_update");
    time_point = utils::datetime::Stringtime(update_time);
  }
 
  using formats::bson::MakeDoc;
  namespace options = storages::mongo::options;
  auto transl = pool_->GetCollection("translations");
  auto cursor = transl.Find(
      MakeDoc("_id",
              MakeDoc("$gte", formats::bson::Oid::MakeMinimalFor(time_point))),
      options::Sort{std::make_pair("_id", options::Sort::kAscending)});
 
  if (!cursor) {
    return "{}";
  }
 
  formats::json::ValueBuilder vb;
  auto content = vb["content"];
 
  formats::bson::Value last;
  for (const auto& doc : cursor) {
    const auto key = doc["key"].As<std::string>();
    const auto lang = doc["lang"].As<std::string>();
    content[key][lang] = doc["value"].As<std::string>();
 
    last = doc;
  }
 
  vb["update_time"] = utils::datetime::Timestring(
      last["_id"].As<formats::bson::Oid>().GetTimePoint());
 
  return ToString(vb.ExtractValue());
}

See MongoDB for MongoDB hints and more usage samples.

Static config

Static configuration of service is quite close to the configuration from Writing your first HTTP server except for the handler and DB:

# yaml
components_manager:
    components:
        mongo-tr:     # Matches component registration and component retrieval strings
            dbconnection: mongodb://localhost:27217/admin
 
        dns-client:                     # Asynchronous DNS component
            fs-task-processor: fs-task-processor
 
        handler-translations:
            path: /v1/translations
            method: GET,PATCH
            task_processor: main-task-processor
 
        server:
            # ...

There are more static options for the MongoDB component configuration, all of them are described at components::Mongo.

Dynamic config

We are not planning to get new dynamic config values in this sample. Because of that we just write the defaults to the fallback file of the components::DynamicConfigFallbacks component: samples/mongo_service/dynamic_config_fallback.json

All the values are described in a separate section Dynamic configs .

A production ready service would dynamically retrieve the above options at runtime from a configuration service. See Writing your own configs server for insights on how to change the above options on the fly, without restarting the service.

int main()

Finally, after writing down the dynamic config values into file at dynamic-config-fallbacks.fallback-path, we add our component to the components::MinimalServerComponentList(), and start the server with static configuration kStaticConfig.

int main(int argc, char* argv[]) {
  const auto component_list = components::MinimalServerComponentList()
                                  .Append<clients::dns::Component>()
                                  .Append<components::Mongo>("mongo-tr")
                                  .Append<samples::mongodb::Translations>();
  return utils::DaemonMain(argc, argv, component_list);
}

Build and Run

To build the sample, execute the following build steps at the userver root directory:

mkdir build_release
cd build_release
cmake -DCMAKE_BUILD_TYPE=Release ..
make userver-samples-mongo_service

The sample could be started by running make start-userver-samples-mongo_service. The command would invoke testsuite start target that sets proper paths in the configuration files, prepares and starts the DB, and starts the service.

To start the service manually start the DB server and run ./samples/mongo_service/userver-samples-mongo_service -c </path/to/static_config.yaml> (do not forget to prepare the configuration files!).

Now you can send a request to your service from another terminal:

bash
$ curl -X PATCH 'http://localhost:8090/v1/translations?key=hello&lang=ru&value=Привки'
$ curl -X PATCH 'http://localhost:8090/v1/translations?key=hello&lang=ru&value=Дратути'
$ curl -X PATCH 'http://localhost:8090/v1/translations?key=hello&lang=ru&value=Здрасьте'
$ curl -s http://localhost:8090/v1/translations?last_update=2021-11-01T12:00:00Z | jq
{
  "content": {
    "hello": {
      "ru": "Дратути"
    },
    "wellcome": {
      "ru": "Здрасьте"
    }
  },
  "update_time": "2021-12-20T10:17:37.249767773+00:00"
}

Functional testing

Functional tests for the service could be implemented using the testsuite. To do that you have to:

• Turn on the pytest_userver.plugins.mongo plugin and provide Mongo settings info for the testsuite:

  import pytest
   
  pytest_plugins = ['pytest_userver.plugins.mongo']
   
  MONGO_COLLECTIONS = {
      'translations': {
          'settings': {
              'collection': 'translations',
              'connection': 'admin',
              'database': 'admin',
          },
          'indexes': [],
      },
  }
   
   
  @pytest.fixture(scope='session')
  def mongodb_settings():
      return MONGO_COLLECTIONS

  The pytest_userver.plugins.service_client.auto_client_deps() fixture already known about the mongodb fixture, so there's no need to override the extra_client_deps() fixture.
• Write the test:

  async def test_mongo(service_client):
      data = {
          ('hello', 'ru', 'Привет'),
          ('hello', 'en', 'hello'),
          ('welcome', 'ru', 'Добро пожаловать'),
          ('welcome', 'en', 'Welcome'),
      }
      for key, lang, value in data:
          response = await service_client.patch(
              '/v1/translations',
              params={'key': key, 'lang': lang, 'value': value},
          )
          assert response.status == 201
   
      response = await service_client.get('/v1/translations')
      assert response.status_code == 200
      assert response.json()['content'] == {
          'hello': {'en': 'hello', 'ru': 'Привет'},
          'welcome': {'ru': 'Добро пожаловать', 'en': 'Welcome'},
      }

Full sources

See the full example:

• samples/mongo_service/mongo_service.cpp
• samples/mongo_service/static_config.yaml
• samples/mongo_service/dynamic_config_fallback.json
• samples/mongo_service/CMakeLists.txt
• samples/mongo_service/tests/conftest.py
• samples/mongo_service/tests/test_mongo.py

---

⇦ PostgreSQL service | Redis service ⇨