Http server middlewares

Introduction

Middleware is software that's assembled into an app pipeline to handle requests and responses. Each component:

• Chooses whether to pass the request to the next component in the pipeline.
• Can perform work before and after the next component in the pipeline.

🐙 userver provides interfaces to implement these pipeline pieces and to configure them as well as the resulting pipeline for the http server. The general idea behind middlewares is no different from that in other web-frameworks like Django, ASP.NET Core, Gin, Spring, Axum etc., but the exact implementation details differ here and there.

If you've already made yourself familiar with how middlewares are implemented in 🐙 userver you may want to skip straight to Usage and configuration.

High level design

Every http request handled by the components::Server is processed in a number of steps:

1. The request is parsed from the byte stream coming from a network connection
2. The request is traced, accounted for, decompressed/rate-limited/authorized/etc if needed, parsed into application-requested format (say, JSON), logged if configured to do so, and a bunch of other steps
3. The request is routed into corresponding server::handlers::HttpHandlerBase
4. The response created by the handler is traced, accounted for, logged if configured to do so... you get the idea
5. The response is serialized into byte stream and sent to the wire

Steps 2. and 4. are implemented by a number of logically-independent middlewares, which allows us to tune/enhance/rearrange things without going into massive refactoring, and allows our users to adjust the pipeline as they see fit.

An important detail of the middlewares implementation is that technically middleware is not a Component, but rather a Client: given M middlewares and H handlers, there will be M x H instances of middlewares in total, M instances for each handler. Such arguably more complicated approach is justified by the configurability-freedom it provides: instead of somehow adjusting the middleware behavior in the code (based on a route, for example), one can configure the middleware at per-handler basis in exactly the same way most things in userver are configured: by a static config.

Middlewares very well might want to interact with the component system of userver, be that for sharing some common resource (say, global rate-limiting) or for performing database/http queries. Since a middleware is not a component, it doesn't have a way to lookup it's dependencies from the components::ComponentContext itself, but rather the dependencies should be injected into a middleware instance. This is where a MiddlewareFactory comes into play.

Every Middleware is paired with a corresponding Factory, which is a Component, and is responsible for creating middleware instances and injecting required dependencies into these instances. It goes like this: when a handler is constructed, it looks up all the factories specified by the pipeline configuration, requests each factory to create a middleware instance, and then builds the resulting pipeline from these instances. So, to emphasize once again: a Middleware instance is not a Component but rather a Client, and there might be multiple instances of the same Middleware type, but a MiddlewareFactory is a Component, hence is a singleton.

For reference, this is the userver-provided default middlewares pipeline:

MiddlewaresList DefaultPipeline() {
    return {
        // Metrics should go before everything else, basically.
        std::string{builtin::kHandlerMetrics},
        // Tracing should go before UnknownExceptionsHandlingMiddleware because it
        // adds some headers, which otherwise might be cleared
        std::string{builtin::kTracing},
        // Ditto
        std::string{builtin::kSetAcceptEncoding},
 
        // Every exception caught here is transformed into Http500 without
        // context.
        // All middlewares except for the most obscure ones should go below.
        std::string{builtin::kUnknownExceptionsHandling},
 
        // Should be self-explanatory
        std::string{builtin::kRateLimit},
        std::string{builtin::kBaggage},
        std::string{builtin::kAuth},
        std::string{builtin::kDecompression},
 
        // Transforms CustomHandlerException into response as specified by the
        // exception, transforms std::exception into Http500 without context.
        // Middlewares that throw CustomHandlerException on error should go below.
        // Middlewares that call HttpHandlerBase::HandleCustomHandlerException or
        // fill the response manually on error (which is faster) should go above.
        std::string{builtin::kExceptionsHandling},
 
        // DeadlinePropagation should go after ExceptionsHandlingMiddleware
        // if the request threw an std::exception and was canceled by deadline
        // propagation,
        // then it must be handled differently.
        std::string{builtin::kDeadlinePropagation},
    };
}

Caveats

In general, one should be very careful when modifying the response after the downstream part of the pipeline completed, and the reason for that is simple: the modification could be overwritten by the upstream part. This is especially apparent when exceptions are involved: consider the case when an exception is propagating through the pipeline, and a middleware is adding a header to the response. Depending on the exception type and where in the pipeline middleware is put, the header may or may not be overridden by the ExceptionsHandling middleware.

Due to this, response-modifying middlewares should be either put before ExceptionsHandling middleware for their changes to reliably take effect if the downstream pipeline threw, or a middleware should handle downstream exceptions itself. Middlewares also should never throw on their own: it's ok to let the downstream exception pass through, but throwing from the middlewares logic could lead to some nonsensical responses being sent to the client, up to violating the http specification.

The default userver-provided middlewares pipeline handles all these cases and is guaranteed to still have all the tracing headers and a meaningful status code (500, likely) present for the response even if the user-built part of the middleware pipeline threw some random exception, but if you reorder or hijack the default pipeline, you are on your own.

Usage and configuration

This is how a minimal implementation of a middleware looks like:

class NoopMiddleware final : public server::middlewares::HttpMiddlewareBase {
private:
    void HandleRequest(server::http::HttpRequest& request, server::request::RequestContext& context) const override {
        Next(request, context);
    }
};

It doesn't have any logic in it and just passes the execution to the downstream. This is how a factory for this middleware looks:

class NoopMiddlewareFactory final : public server::middlewares::HttpMiddlewareFactoryBase {
public:
    static constexpr std::string_view kName{"noop-middleware"};
 
    using HttpMiddlewareFactoryBase::HttpMiddlewareFactoryBase;
 
private:
    std::unique_ptr<server::middlewares::HttpMiddlewareBase>
    Create(const server::handlers::HttpHandlerBase&, yaml_config::YamlConfig) const override {
        return std::make_unique<NoopMiddleware>();
    }
};

which feels too verbose for the amount of logic the code performs, so we have a shortcut version, which does the same and also passes the handler into the middleware constructor. Given the middleware that performs some logic

class SomeServerMiddleware final : public server::middlewares::HttpMiddlewareBase {
public:
    // This will be used as a kName for the SimpleHttpMiddlewareFactory
    static constexpr std::string_view kName{"server-middleware"};
 
    // Handler isn't interesting to us, but we could use it if needed.
    // Or we could implement the factory ourselves instead of using
    // SimpleHttpMiddlewareFactory, and pass whatever parameters we want.
    explicit SomeServerMiddleware(const server::handlers::HttpHandlerBase&) {}
 
private:
    void HandleRequest(server::http::HttpRequest& request, server::request::RequestContext& context) const override {
        Next(request, context);
        request.GetHttpResponse().SetHeader(kCustomServerHeader, "1");
    }
 
    static constexpr http::headers::PredefinedHeader kCustomServerHeader{"X-Some-Server-Header"};
};

the factory implementation is just this:

Do not forget to add components configs:

        noop-middleware: {}
 
        server-middleware: {}

Global middleware configuration

Normally, the process of configuring a middleware is the same as configuring any other component, see Component system

As a component, a MiddlewareFactory takes (config, context) parameters in its constructor. It can parse some fields from config and store them in the component. Then it can pass this configuration (references are OK) to each Middleware created in its Create method.

All used config fields should be described in MyMiddlewareFactory::GetStaticConfigSchema.

Global configuration should be preferred to per-handler configuration, because the latter leads to copy-pasta in configs. For some options, it's a good idea to implement both global and per-handler configuration.

Per-handler middleware configuration

Basically, the whole point of having MiddlewareFactory-ies separated from Middleware-s, is to have a possibility to configure a middleware at a per-handler basis. In the snippet above that's what "handler-middleware.header-value" is for: given the middleware (which actually resembles pretty close to how tracing headers are set to the response in userver)

class SomeHandlerMiddleware final : public server::middlewares::HttpMiddlewareBase {
public:
    static constexpr std::string_view kName{"handler-middleware"};
 
    SomeHandlerMiddleware(const server::handlers::HttpHandlerBase&, yaml_config::YamlConfig middleware_config)
        : header_value_{middleware_config["header-value"].As<std::string>()} {}
 
private:
    void HandleRequest(server::http::HttpRequest& request, server::request::RequestContext& context) const override {
        // We will set the header no matter whether downstream succeeded or not.
        //
        // Note that in presence of exceptions other that
        // CustomHandlerException-derived ones, this header would be removed by
        // default userver-provided exceptions handling middleware. If this is
        // undesirable, the middleware should be earlier in the pipeline, or the
        // default exceptions handling behavior could be overridden.
        const utils::ScopeGuard set_header_scope{
            [this, &request] { request.GetHttpResponse().SetHeader(kCustomHandlerHeader, header_value_); }};
 
        Next(request, context);
    }
 
    static constexpr http::headers::PredefinedHeader kCustomHandlerHeader{"X-Some-Handler-Header"};
 
    const std::string header_value_;
};

and the factory implementation

class SomeHandlerMiddlewareFactory final : public server::middlewares::HttpMiddlewareFactoryBase {
public:
    static constexpr std::string_view kName{SomeHandlerMiddleware::kName};
 
    using HttpMiddlewareFactoryBase::HttpMiddlewareFactoryBase;
 
private:
    std::unique_ptr<server::middlewares::HttpMiddlewareBase>
    Create(const server::handlers::HttpHandlerBase& handler, yaml_config::YamlConfig middleware_config) const override {
        return std::make_unique<SomeHandlerMiddleware>(handler, std::move(middleware_config));
    }
 
    yaml_config::Schema GetMiddlewareConfigSchema() const override {
        return formats::yaml::FromString(R"(
type: object
description: Config for this particular middleware
additionalProperties: false
properties:
    header-value:
        type: string
        description: header value to set for responses
)")
            .As<yaml_config::Schema>();
    }
};

one can configure the middleware behavior (header value, in this particular case) in the handler's static config.

If a global configuration is desired (that is, for every middleware instance there is), the easiest way to achieve that would be to have a configuration in the Factory config, and for Factory to pass the configuration into the Middleware constructor. This takes away the possibility to declare a Factory as a SimpleHttpMiddlewareFactory, but we find this tradeoff acceptable (after all, if a middleware needs a configuration it isn't that "Simple" already).

Do not forget to add components configs:

Pipelines configuration

Now, after we have a middleware and its factory implemented, it would be nice to actually use the middleware in the pipeline. 🐙 userver provides two interfaces for configuring middleware pipelines: one for a server-wide configuration, and one for a more granular per-handler configuration.

Server-wide middleware pipeline

The server-wide pipeline is server::middlewares::PipelineBuilder. In its simple form, it takes server::middlewares::DefaultPipeline and appends the given middlewares to it, which looks like this:

        # yaml
        default-server-middleware-pipeline-builder:
            append:
              - server-middleware     # Or we could implement the same in the code, consider it a shortcut.

If a more sophisticated behavior is desired, derive from server::middlewares::PipelineBuilder and override its BuildPipeline method. Then set the custom pipeline component's name in the config of components::Server:

# yaml
        server:
            # ...
            middleware-pipeline-builder: custom-pipeline-builder

Remember that messing with the default userver-provided pipeline is error-prone and leaves you on your own.

Custom per-handler middleware pipelines

To configure the pipeline at a per-handler basis 🐙 userver provides server::middlewares::HandlerPipelineBuilder interface. By default, it returns the server-wide pipeline without any modifications to it. To change the behavior one should derive from it, override the BuildPipeline method and specify the builder as the pipeline-builder for the handler. For example:

class CustomHandlerPipelineBuilder final : public server::middlewares::HandlerPipelineBuilder {
public:
    using HandlerPipelineBuilder::HandlerPipelineBuilder;
 
    server::middlewares::MiddlewaresList BuildPipeline(server::middlewares::MiddlewaresList server_middleware_pipeline
    ) const override {
        // We could do any kind of transformation here.
        // For the sake of example (and what we assume to be the most common case),
        // we just add some middleware to the pipeline.
        auto& pipeline = server_middleware_pipeline;
        pipeline.emplace_back(SomeHandlerMiddleware::kName);
        pipeline.emplace_back(NoopMiddlewareFactory::kName);
 
        return pipeline;
    }
};

and to use the class as a pipeline builder we should append it to the ComponentList

          .Append<samples::http_middlewares::CustomHandlerPipelineBuilder>(
              "custom-handler-pipeline-builder")

and specify as a pipeline-builder for the handler (notice the middlewares.pipeline-builder section):

        # yaml
        handler-with-custom-middlewares:
            path: /custom-hello
            method: GET
            task_processor: main-task-processor
            middlewares:
                pipeline-builder: custom-handler-pipeline-builder
                handler-middleware:
                    header-value: some_value