kafka::ProducerComponent Class Reference

#include <userver/kafka/producer_component.hpp>

Detailed Description

Apache Kafka Producer client component.

Static configuration example:

        # yaml
        kafka-producer:                   # Kafka producer component name must start with kafka-consumer
            delivery_timeout: 10s         # Timeout for message delivery
            queue_buffering_max: 100ms    # Time, producer waits before sending a message
            enable_idempotence: true      # Whether to enable idempotence mode
            security_protocol: PLAINTEXT  # Broker connection settings

Secdist format

A Kafka alias in secdist is described as a JSON object kafka_settings, containing credentials of Kafka brokers.

@pytest.fixture(scope='session')
def service_env(kafka_secdist) -> dict:
    """
    Note: kafka_secist fixture generates the secdist config
 
    Expected secdist format is:
    "kafka_settings": {
        "<kafka-component-name>": {
            "brokers": "<brokers comma-separated endpoint list>",
            "username": "SASL2 username (may be empty if use PLAINTEXT)",
            "password": "SASL2 password (may be empty if use PLAINTEXT)"
        }
    }
    """
 
    return {'SECDIST_CONFIG': kafka_secdist}

Static options:

Name	Description	Default value
client_id	Client identifier. May be an arbitrary string	userver
delivery_timeout	time a produced message waits for successful delivery	–
queue_buffering_max	delay to wait for messages to be transmitted to broker	–
enable_idempotence	whether to make producer idempotent	false
queue_buffering_max_messages	maximum number of messages waiting for delivery	100000
queue_buffering_max_kbytes	maximum size of messages waiting for delivery	1048576
message_max_bytes	maximum size of message	1000000
message_send_max_retries	maximum number of send request retries until delivery_timeout reached	2147483647
retry_backoff	backoff time before retrying send request, exponentially increases after each retry	100
retry_backoff_max	backoff upper bound	1000
security_protocol	protocol used to communicate with brokers	–
sasl_mechanisms	SASL mechanism to use for authentication	none
ssl_ca_location	file or directory path to CA certificate(s) for verifying the broker's key	none
rd_kafka_custom_options	a map of librdkafka library additional options	'{}'

Examples

samples/kafka_service/src/producer_handler.cpp.

Definition at line 51 of file producer_component.hpp.

Inheritance diagram for kafka::ProducerComponent:

Public Member Functions
	ProducerComponent (const components::ComponentConfig &config, const components::ComponentContext &context)
const Producer &	GetProducer ()
	Returns a producer instance reference.
ComponentHealth	GetComponentHealth () const override
void	OnLoadingCancelled () override
void	OnAllComponentsLoaded () override
void	OnAllComponentsAreStopping () override

Static Public Member Functions
static yaml_config::Schema	GetStaticConfigSchema ()

Static Public Attributes
static constexpr std::string_view	kName {"kafka-producer"}
	The default name of kafka::ProducerComponent component.

Protected Types
using	LoggableComponentBase = ComponentBase
	Legacy alias, use ComponentBase instead.

Member Typedef Documentation

LoggableComponentBase

using components::ComponentBase::LoggableComponentBase = ComponentBase

protectedinherited

Legacy alias, use ComponentBase instead.

Definition at line 67 of file component_base.hpp.

Member Function Documentation

GetComponentHealth()

ComponentHealth components::ComponentBase::GetComponentHealth ( ) const

inlineoverridevirtualinherited

Override this function to inform the world of the state of your component.

Warning

The function is called concurrently from multiple threads.

Reimplemented from components::RawComponentBase.

Reimplemented in server::handlers::Restart.

Definition at line 35 of file component_base.hpp.

GetProducer()

const Producer & kafka::ProducerComponent::GetProducer

(

)

Returns a producer instance reference.

See also

kafka::Producer

OnAllComponentsAreStopping()

void components::ComponentBase::OnAllComponentsAreStopping ( )

inlineoverridevirtualinherited

Component may use this function to stop doing work before the stop of the components that depend on it.

Base components may override it and make final to do some work before the derived object constructor is called. Don't use it otherwise.

Reimplemented from components::RawComponentBase.

Reimplemented in urabbitmq::ConsumerComponentBase, and components::Server.

Definition at line 58 of file component_base.hpp.

OnAllComponentsLoaded()

void components::ComponentBase::OnAllComponentsLoaded ( )

inlineoverridevirtualinherited

Component may use this function to finalize registration of other components that depend on it (for example, handler components register in server component, and the latter uses OnAllComponentsLoaded() to start processing requests).

Base components may override it and make final to do some work after the derived object constructor is called. Don't use it otherwise.

Reimplemented from components::RawComponentBase.

Reimplemented in urabbitmq::ConsumerComponentBase, components::Server, and server::handlers::Ping.

Definition at line 51 of file component_base.hpp.

OnLoadingCancelled()

void components::ComponentBase::OnLoadingCancelled ( )

inlineoverridevirtualinherited

Called once if the creation of any other component failed. If the current component expects some other component to take any action with the current component, this call is a signal that such action may never happen due to components loading was cancelled. Application components might not want to override it.

Reimplemented from components::RawComponentBase.

Definition at line 42 of file component_base.hpp.

---

The documentation for this class was generated from the following file:

• userver/kafka/producer_component.hpp