HTTP Flatbuf handler and requests

Before you start

Make sure that you can compile and run core tests and read a basic example Writing your first HTTP server.

Step by step guide

JSON is a nice format, but it does not suit well for high-load applications.

This tutorial shows you how to send and receive Flatbuffers over HTTP using userver.

In this sample we use the samples/flatbuf_service/flatbuffer_schema.fbs Flatbuffers scheme and compile it via the ‘flatc --cpp --gen-object-api --filename-suffix ’.fbs' flatbuffer_schema.fbs` command.

HTTP Flatbuffer handler component

There are two ways to write a handler that deals with Flatbuffers:

• We could do that by creating a component derived from server::handlers::HttpHandlerBase as in the Writing your first HTTP server example. In that case we would need
  • to manually parse the input
  • and manually to serialize the result to a std::string that holds the bytes of a Flatbuffers.
• Or we could just take a server::handlers::HttpHandlerFlatbufBase that does all the above steps for us.

We are going to take the second approach. All the Flatbuffers related action happens in the HandleRequestFlatbufThrow method:

#include <userver/server/handlers/http_handler_flatbuf_base.hpp>
#include "flatbuffer_schema.fbs.h"
 
namespace samples::fbs_handle {
 
class FbsSumEcho final
    : public server::handlers::HttpHandlerFlatbufBase<fbs::SampleRequest,
                                                      fbs::SampleResponse> {
 public:
  // `kName` is used as the component name in static config
  static constexpr std::string_view kName = "handler-fbs-sample";
 
  // Component is valid after construction and is able to accept requests
  FbsSumEcho(const components::ComponentConfig& config,
             const components::ComponentContext& context)
      : HttpHandlerFlatbufBase(config, context) {}
 
  fbs::SampleResponse::NativeTableType HandleRequestFlatbufThrow(
      const server::http::HttpRequest& /*request*/,
      const fbs::SampleRequest::NativeTableType& fbs_request,
      server::request::RequestContext&) const override {
    fbs::SampleResponse::NativeTableType res;
    res.sum = fbs_request.arg1 + fbs_request.arg2;
    res.echo = fbs_request.data;
    return res;
  }
};
 
}  // namespace samples::fbs_handle

HTTP Flatbuffer request

A clients::http::Client is needed to make HTTP requests. It could be obtained from the components::HttpClient component.

class FbsRequest final : public components::LoggableComponentBase {
 public:
  static constexpr std::string_view kName = "fbs-request";
 
  FbsRequest(const components::ComponentConfig& config,
             const components::ComponentContext& context)
      : LoggableComponentBase(config, context),
        http_client_{
            context.FindComponent<components::HttpClient>().GetHttpClient()},
        task_{utils::Async("requests", [this]() { KeepRequesting(); })} {}
 
  void KeepRequesting() const;
 
 private:
  clients::http::Client& http_client_;
  engine::TaskWithResult<void> task_;
};

After that, we just send the data and validate the response:

void FbsRequest::KeepRequesting() const {
  // Fill the payload data
  fbs::SampleRequest::NativeTableType payload;
  payload.arg1 = 20;
  payload.arg2 = 22;
  payload.data = "Hello word";
 
  // Serialize the payload into a std::string
  flatbuffers::FlatBufferBuilder fbb;
  auto ret_fbb = fbs::SampleRequest::Pack(fbb, &payload);
  fbb.Finish(ret_fbb);
  std::string data(reinterpret_cast<const char*>(fbb.GetBufferPointer()),
                   fbb.GetSize());
 
  // Send it
  const auto response = http_client_.CreateRequest()
                            .post("http://localhost:8084/fbs", std::move(data))
                            .timeout(std::chrono::seconds(1))
                            .retry(10)
                            .perform();
 
  // Response code should be 200 (use proper error handling in real code!)
  UASSERT_MSG(response->IsOk(), "Sample should work well in tests");
 
  // Verify and deserialize response
  const auto body = response->body_view();
  const auto* response_fb =
      flatbuffers::GetRoot<fbs::SampleResponse>(body.data());
  flatbuffers::Verifier verifier(reinterpret_cast<const uint8_t*>(body.data()),
                                 body.size());
  UASSERT_MSG(response_fb->Verify(verifier), "Broken flatbuf in sample");
  fbs::SampleResponse::NativeTableType result;
  response_fb->UnPackTo(&result);
 
  // Make sure that the response is the expected one for sample
  UASSERT_MSG(result.sum == 42, "Sample should work well in tests");
  UASSERT_MSG(result.echo == payload.data, "Sample should work well in tests");
}

Build and Run

To build the sample, execute the following build steps at the userver root directory:

bash
mkdir build_release
cd build_release
cmake -DCMAKE_BUILD_TYPE=Release ..
make userver-samples-flatbuf_service

The sample could be started by running make start-userver-samples-flatbuf_service. The command would invoke testsuite start target that sets proper paths in the configuration files and starts the service.

To start the service manually run ./samples/flatbuf_service/userver-samples-flatbuf_service -c </path/to/static_config.yaml>.

Now you can send a request to your server from another terminal:

bash
$ echo "100000000c00180000000800100004000c00000014000000140000000000000016000000000000000a00000048656c6c6f20776f72640000" \
      | xxd -r -p | curl --data-binary "@-" http://localhost:8084/fbs -v --output /dev/null
* TCP_NODELAY set
* Connected to localhost (::1) port 8084 (#0)
> POST /fbs HTTP/1.1
> Host: localhost:8084
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 56
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 56 out of 56 bytes
< HTTP/1.1 200 OK
< Date: Wed, 16 Jun 2021 12:52:22 UTC
< Content-Type: text/html
< X-YaRequestId: c8b2aee7ca5f4165ad25119b1850e778
< Server: userver/2.0 (20210616074040; rv:2c78282ea)
< X-YaTraceId: 8f4765a7176e41d28c3c6a677f00193e
< Connection: keep-alive
< Content-Length: 48
<
* Connection #0 to host localhost left intact

Functional testing

Naive functional tests for the service could be implemented using the testsuite in the following way:

async def test_flatbuf(service_client):
    body = bytearray.fromhex(
        '100000000c00180000000800100004000c000000140000001400000000000000'
        '16000000000000000a00000048656c6c6f20776f72640000',
    )
    response = await service_client.post('/fbs', data=body)
    assert response.status == 200

Do not forget to add the plugin in conftest.py:

# Adding a plugin from userver/testsuite/pytest_plugins/
pytest_plugins = ['pytest_userver.plugins.core']

Full sources

See the full example:

• samples/flatbuf_service/flatbuf_service.cpp
• samples/flatbuf_service/flatbuffer_schema.fbs
• samples/flatbuf_service/static_config.yaml
• samples/flatbuf_service/CMakeLists.txt
• samples/flatbuf_service/tests/test_flatbuf.py

---

⇦ Pre-caching data from HTTP remote | gRPC client and service ⇨