userver: kafka::ConsumerComponent Class Reference
Loading...
Searching...
No Matches
kafka::ConsumerComponent Class Referencefinal

#include <userver/kafka/consumer_component.hpp>

Detailed Description

Apache Kafka Consumer client component.

Static configuration example:

# yaml
kafka-consumer: # Kafka consumer component name must start with kafka-consumer
group_id: test-group # Consumers group current consumer will join on start
topics: # List of topics consumer starts to listen
- test-topic-1
- test-topic-2
auto_offset_reset: smallest # Strategy for reading topics without committed offsets
max_batch_size: 5 # Number of messages consumers waits until user-callback is called
security_protocol: PLAINTEXT # Broker connection settings

Secdist format

A Kafka alias in secdist is described as a JSON object kafka_settings, containing credentials of Kafka brokers.

@pytest.fixture(scope='session')
def service_env(kafka_secdist) -> dict:
"""kafka_secist fixture generates the secdist config"""
return {'SECDIST_CONFIG': kafka_secdist}

Static options:

Name Description Default value
client_id Client identifier. May be an arbitrary string userver
group_id consumer group id (name)
topics list of topics consumer subscribes
max_batch_size maximum number of messages consumer waits for new message before calling a callback 1
poll_timeout maximum amount of time consumer waits for messages for new messages before calling a callback 1s
max_callback_duration duration user callback must fit not to be kicked from the consumer group 5m
restart_after_failure_delay time consumer suspends execution if user-callback fails 10s
auto_offset_reset action to take when there is no initial offset in offset store smallest
env_pod_name environment variable to substitute {pod_name} substring in group_id none
security_protocol protocol used to communicate with brokers
sasl_mechanisms SASL mechanism to use for authentication none
ssl_ca_location File or directory path to CA certificate(s) for verifying the broker's key none
topic_metadata_refresh_interval period of time at which topic and broker metadata is refreshed 5m
metadata_max_age metadata cache max age 15
rd_kafka_custom_options a map of librdkafka library additional options '{}'

Definition at line 59 of file consumer_component.hpp.

+ Inheritance diagram for kafka::ConsumerComponent:

Public Member Functions

 ConsumerComponent (const components::ComponentConfig &config, const components::ComponentContext &context)
 
ConsumerScope GetConsumer ()
 
ComponentHealth GetComponentHealth () const override
 
void OnLoadingCancelled () override
 
void OnAllComponentsLoaded () override
 
void OnAllComponentsAreStopping () override
 

Static Public Member Functions

static yaml_config::Schema GetStaticConfigSchema ()
 

Static Public Attributes

static constexpr std::string_view kName = "kafka-consumer"
 

Protected Types

using LoggableComponentBase = ComponentBase
 Legacy alias, use ComponentBase instead.
 

Member Typedef Documentation

◆ LoggableComponentBase

using components::ComponentBase::LoggableComponentBase = ComponentBase
protectedinherited

Legacy alias, use ComponentBase instead.

Definition at line 69 of file component_base.hpp.

Member Function Documentation

◆ GetComponentHealth()

ComponentHealth components::ComponentBase::GetComponentHealth ( ) const
inlineoverridevirtualinherited

Override this function to inform the world of the state of your component.

Warning
The function is called concurrently from multiple threads.

Reimplemented from components::RawComponentBase.

Definition at line 35 of file component_base.hpp.

◆ OnAllComponentsAreStopping()

void components::ComponentBase::OnAllComponentsAreStopping ( )
inlineoverridevirtualinherited

Component may use this function to stop doing work before the stop of the components that depend on it.

Base components may override it and make final to do some work before the derived object constructor is called. Don't use it otherwise.

Reimplemented from components::RawComponentBase.

Reimplemented in urabbitmq::ConsumerComponentBase, and components::Server.

Definition at line 60 of file component_base.hpp.

◆ OnAllComponentsLoaded()

void components::ComponentBase::OnAllComponentsLoaded ( )
inlineoverridevirtualinherited

Component may use this function to finalize registration of other components that depend on it (for example, handler components register in server component, and the latter uses OnAllComponentsLoaded() to start processing requests).

Base components may override it and make final to do some work after the derived object constructor is called. Don't use it otherwise.

Reimplemented from components::RawComponentBase.

Reimplemented in urabbitmq::ConsumerComponentBase, components::Server, and server::handlers::Ping.

Definition at line 53 of file component_base.hpp.

◆ OnLoadingCancelled()

void components::ComponentBase::OnLoadingCancelled ( )
inlineoverridevirtualinherited

Called once if the creation of any other component failed. If the current component expects some other component to take any action with the current component, this call is a signal that such action may never happen due to components loading was cancelled. Application components might not want to override it.

Reimplemented from components::RawComponentBase.

Definition at line 44 of file component_base.hpp.

Member Data Documentation

◆ kName

constexpr std::string_view kafka::ConsumerComponent::kName = "kafka-consumer"
staticconstexpr

Definition at line 61 of file consumer_component.hpp.


The documentation for this class was generated from the following file: