gRPC client middleware implementation

Two main classes

There are two main interfaces for implementing a middleware:

1. ugrpc::client::MiddlewareBase. Class that implements the main logic of a middleware.
2. ugrpc::client::SimpleMiddlewareFactoryComponent short-cut for simple cases without static options.
   • Or ugrpc::client::MiddlewareFactoryComponentBase to declare static options.

MiddlewareBase

ugrpc::client::MiddlewareBase

PreStartCall and PostFinish

Methods ugrpc::client::MiddlewareBase::PreStartCall and ugrpc::client::MiddlewareBase::PostFinish are called for each RPC attempt.

PreStartCall is called before the first message is sent in each attempt. PostFinish is called after each attempt completes, regardless of how it completes (success, error, cancellation, abandonment, or network error).

For unary RPCs with retries enabled, both PreStartCall and PostFinish will be called multiple times - once per retry attempt.

PreStartCall hooks are called in the direct middlewares order. PostFinish hooks are called in the reversed order.

Streaming RPCs can have multiple requests and responses, but PreStartCall and PostFinish are called once per RPC in any case.

For more information about the middlewares order:

See also

gRPC middlewares order.

PostFinish completion status

The PostFinish hook receives a ugrpc::client::CompletionStatus parameter, which is a utils::expected<grpc::Status, SpecialCaseCompletionType>.

This means the completion can be one of:

• Normal completion (grpc::Status): The RPC completed with a standard gRPC status (success or error)
• Special case completion (ugrpc::client::SpecialCaseCompletionType): The RPC completed in an exceptional way:

      kNetworkError,
      kTimeoutDeadlinePropagated,
      kCancelled,
      kAbandoned

Per-call (RPC) hooks implementation example

class AuthMiddleware final : public ugrpc::client::MiddlewareBase {
public:
    // Name of a middleware-factory that creates this middleware.
    static constexpr std::string_view kName = "grpc-auth-client";
 
    // 'Auth' is a group for authentication. See middlewares groups for more information.
    static inline const auto
        kDependency = middlewares::MiddlewareDependencyBuilder().InGroup<middlewares::groups::Auth>();
 
    AuthMiddleware();
 
    ~AuthMiddleware() override;
 
    void PreStartCall(ugrpc::client::MiddlewareCallContext& context) const override;
};
 
// This component creates Middleware. Name of the component is 'Middleware::kName'.
// In this case we use a short-cut for defining a middleware-factory, but you can create your own factory by
// inheritance from 'ugrpc::client::MiddlewareFactoryComponentBase'
using AuthComponent = ugrpc::client::SimpleMiddlewareFactoryComponent<AuthMiddleware>;
 

void ApplyCredentials(::grpc::ClientContext& context) { context.AddMetadata(kKey, kCredentials); }
 
AuthMiddleware::AuthMiddleware() = default;
 
AuthMiddleware::~AuthMiddleware() = default;
 
void AuthMiddleware::PreStartCall(ugrpc::client::MiddlewareCallContext& context) const {
    ApplyCredentials(context.GetClientContext());
}
 

Register the Middleware component in the component system.

int main(int argc, char* argv[]) {
    const auto component_list =
        components::MinimalServerComponentList()
            .AppendComponentList(ugrpc::server::DefaultComponentList())
            .AppendComponentList(ugrpc::client::DefaultComponentList())
            .Append<ugrpc::client::ClientFactoryComponent>()
            .Append<components::TestsuiteSupport>()
            .Append<congestion_control::Component>()
            .Append<samples::grpc::auth::GreeterClient>()
            .Append<samples::grpc::auth::GreeterServiceComponent>()
            .Append<samples::grpc::auth::GreeterHttpHandler>()
            .Append<samples::grpc::auth::server::AuthComponent>()
            .Append<samples::grpc::auth::server::MetaFilterComponent>()
            .Append<samples::grpc::auth::client::AuthComponent>()
            .Append<samples::grpc::auth::client::ChaosComponent>();
 
    return utils::DaemonMain(argc, argv, component_list);
}

The static YAML config.

        grpc-auth-client:
 
        grpc-client-middlewares-pipeline:
            middlewares:
                grpc-auth-client:
                    enabled: true  # register the middleware in the pipeline

PreSendMessage and PostRecvMessage

PreSendMessage hooks are called in the order of middlewares. PostRecvMessage hooks are called in the reverse order of middlewares.

For more information about the middlewares order:

See also

gRPC middlewares order.

These hooks are called on each message.

PreSendMessage:

• unary: is called exactly once
• stream: is called 0, 1 or more

PostRecvMessage:

• unary: is called 0 or 1 (0 if service doesn't return a message)
• stream: is called 0, 1 or more

Per-message hooks implementation example

class Middleware final : public MiddlewareBase {
public:
    explicit Middleware(const Settings& settings);
 
    void PreStartCall(MiddlewareCallContext& context) const override;
 
    void PreSendMessage(MiddlewareCallContext& context, const google::protobuf::Message& message) const override;
 
    void PostRecvMessage(MiddlewareCallContext& context, const google::protobuf::Message& message) const override;
 
    void PostFinish(MiddlewareCallContext& context, const CompletionStatus& result) const override;
 
private:
    Settings settings_;
};

void Middleware::PreSendMessage(MiddlewareCallContext& context, const google::protobuf::Message& message) const {
    auto& span = context.GetSpan();
 
    const SpanLogger logger{span, settings_.log_level};
    logger.Log(settings_.msg_log_level, [&](auto& log_helper) {
        logging::LogExtra extra{
            {ugrpc::impl::kTypeTag, "request"},
            {ugrpc::impl::kBodyTag, GetMessageForLogging(message, settings_)},
            {ugrpc::impl::kMessageMarshalledLenTag, message.ByteSizeLong()},
        };
        log_helper
            << (IsSingleRequestMethod(context.GetRpcType()) ? "gRPC request" : "gRPC request stream message")
            << std::move(extra);
    });
}
 
void Middleware::PostRecvMessage(MiddlewareCallContext& context, const google::protobuf::Message& message) const {
    const SpanLogger logger{context.GetSpan(), settings_.log_level};
    logger.Log(settings_.msg_log_level, [&](auto& log_helper) {
        logging::LogExtra extra{
            {ugrpc::impl::kTypeTag, "response"},
            {ugrpc::impl::kBodyTag, GetMessageForLogging(message, settings_)},
            {ugrpc::impl::kMessageMarshalledLenTag, message.ByteSizeLong()},
        };
        log_helper
            << (IsSingleResponseMethod(context.GetRpcType()) ? "gRPC response" : "gRPC response stream message")
            << std::move(extra);
    });
}

The static YAML config and component registration are identical as in the example above. So, let's not focus on this.

MiddlewareFactoryComponent

We use a simple short-cut ugrpc::client::SimpleMiddlewareFactoryComponent in the example above. To declare static config options of your middleware see gRPC middlewares configuration.

Exceptions and errors in middlewares

To fully understand what happens when middleware hooks fail, you should understand the middlewares order:

See also

grpc_client_middlewares_order.

All exceptions are rethrown to the user code from client's RPC creating methods, Read / Write (for streaming), and from methods that return the RPC status.

Note that if an exception occurs, the middleware pipeline is stopped and subsequent middleware hooks are not called.

Using static config options in middlewares

There are two ways to implement a middleware component. You can see above ugrpc::client::SimpleMiddlewareFactoryComponent. This component is needed for simple cases without static config options of a middleware.

Note

In that case, kName and kDependency (middlewares::MiddlewareDependencyBuilder) must be in a middleware class (as shown above).

If you want to use static config options for your middleware, use ugrpc::client::MiddlewareFactoryComponentBase.

See also

gRPC middlewares configuration.

To override static config options of a middleware per a client see grpc_middlewares_config_override.

Using dynamic config values in middlewares

If your middleware needs a dynamic config value, do NOT do your own FindComponent<components::DynamicConfig>() in your middleware factory component, and do NOT depend, even transitively, on a component that does so itself (e.g. do not add components::DynamicConfig, or a component that depends on it, as a dependency of your middleware factory component). Instead, store ugrpc::client::ClientInfo::config_source, passed to CreateMiddleware, as a field of your middleware object, and read the config through it (.GetSnapshot()[key]) inside the middleware hooks (PreStartCall/PreSendMessage/PostRecvMessage/PostFinish).

CreateMiddleware is called once per client, so the resulting config_source is the same one that the rest of the client uses. This is important because a client's ClientFactoryComponent may be configured with use-constant-dynamic-configs: true (a "light" gRPC client without a blocking dependency on components::DynamicConfig, see ugrpc::client::ClientFactoryComponent) — in that case config_source is constant and does not go through components::DynamicConfig at all. A middleware that bypasses ClientInfo::config_source and does its own FindComponent<components::DynamicConfig>() (directly or transitively, through some other component it depends on) would silently reintroduce a blocking dependency for such a client.

Middlewares order

Before starting to read specifics of client middlewares ordering:

See also

gRPC middlewares order.

There are simple cases above: we just set Auth group for one middleware.

Here we say that all client middlewares are located in these groups.

PreCore group is called firstly, then Logging and so forth...

⇦ gRPC client middlewares | gRPC middlewares order ⇨