Detailed Description

Statistics writers, metric types, and histogram helpers.

Classes
class	BaseFormatBuilder
class	BusyMarker
	A RAII-style guard to account code block execution time in utils::statistics::BusyStorage. Aware of recursive invocations in the same thread. More...
class	BusyStorage
class	Entry
	Statistics registration holder, used to unregister a metric source before it is destroyed. More...
class	Histogram
	A histogram with a dynamically-allocated array of buckets. More...
class	HistogramAggregator
	Used to aggregate multiple utils::statistics::Histogram metrics. More...
class	HistogramView
	The non-owning reader API for "histogram" metrics. More...
class	Label
	Label name+value storage. More...
class	LabelsSpan
	View over a continuous range of LabelView. More...
class	LabelView
	Non owning label name+value storage. More...
class	MetricQueryError
	Thrown by statistics::Snapshot queries on unexpected metrics states. More...
class	MetricsStorage
	Storage of metrics registered with MetricTag<Metric>. More...
class	MetricTag
	Metric description and registration in a declarative way. More...
class	MetricValue
	The value of a metric. More...
class	MinMaxAvg
	Class for concurrent safe calculation of minimum, maximum and average over series of values. More...
class	MonotonicByLabelStorage
	Thread-safe monotonic storage of metrics indexed by label values. More...
class	Percentile
	Class allows easy calculation of percentiles. More...
struct	Rate
	Rate metrics (or "counter" metrics) are metrics that only monotonically increase or are reset to zero on restart. Some monitoring systems give them special treatment with regard to maintaining proper non-negative derivative. More...
class	RateCounter
	Atomic counter of type Rate with relaxed memory ordering by default. More...
class	RecentPeriod
	Class maintains circular buffer of Counters. More...
class	RelaxedCounter
	Atomic counter of type T with relaxed memory ordering. More...
class	Request
	Class describing the request for metrics data. More...
class	Snapshot
	A snapshot of metrics dumped via Writer. More...
struct	StatisticsRequest
	Used in legacy statistics extenders. More...
class	Storage
class	StripedRateCounter
	Atomic counter of type Rate with relaxed memory ordering, increased memory consumption and decreased contention. More...
struct	Warning
class	Writer
	Class for writing metrics that is provided by utils::statistics::Storage. More...

Typedefs
using	MetricsStoragePtr = std::shared_ptr<MetricsStorage>
using	WriterFuncRef = utils::function_ref<void(Writer&) const>
	Non-owning reference to a function that writes metrics via Writer.
using	PortabilityWarnings = std::unordered_map<WarningCode, std::vector<Warning>>
using	ExtenderFunc = std::function<formats::json::ValueBuilder(const StatisticsRequest&)>
using	WriterFunc = std::function<void(Writer&)>

Enumerations

enum class WarningCode {
kInf ,
kNan ,
kHistogramBucketsCount ,
kLabelsCount ,
kReservedLabelApplication ,
kReservedLabelCluster ,
kReservedLabelGroup ,
kReservedLabelHost ,
kReservedLabelProject ,
kReservedLabelSensor ,
kReservedLabelService ,
kLabelNameLength ,
kLabelValueLength ,
kPathLength ,
kLabelNameMismatch
}

Functions
void	DumpMetric (Writer &writer, const Histogram &histogram)
	Metric serialization support for Histogram.
bool	operator== (HistogramView lhs, HistogramView rhs) noexcept
	Compares equal if bounds are close and values are equal.
bool	operator< (const LabelView &x, const LabelView &y) noexcept
bool	operator== (const LabelView &x, const LabelView &y) noexcept
bool	operator< (const Label &x, const Label &y) noexcept
bool	operator== (const Label &x, const Label &y) noexcept
Rate	operator+ (Rate first, Rate second) noexcept
void	DumpMetric (Writer &writer, const RateCounter &value)
void	ResetMetric (RateCounter &value)
template<class Metric>
void	DumpMetric (Writer &writer, const std::atomic< Metric > &m)
void	VisitMetrics (WriterFuncRef func, BaseFormatBuilder &out, const Request &request={})
	Low-level connecting function that dumps metrics from func into out.
void	VisitMetrics (const HasWriterSupport auto &metric, BaseFormatBuilder &out, const Request &request={})
void	PrintTo (const Snapshot &data, std::ostream *)
	Support for gtest diagnostics for utils::statistics::Snapshot.
void	PrintTo (MetricValue value, std::ostream *)
	Support for gtest diagnostics for utils::statistics::MetricValue.
void	PrintTo (Rate value, std::ostream *)
	Support for gtest diagnostics for utils::statistics::Rate.
void	RegisterWriterScope (const components::ComponentContext &, std::string common_prefix, WriterFunc func, std::vector< Label > add_labels={})
	Add a writer function to Storage from components::StatisticsStorage. It automatically calls utils::statistics::Storage::RegisterWriter() just after the component construction and utils::statistics::Entry::Unregister() just before the component destructor.
std::string	DurationToString (std::chrono::seconds duration)
std::string	ToGraphiteFormat (const Storage &statistics, const Request &statistics_request={})
	Output statistics in Graphite format with tags (labels).
std::string	ToGraphiteFormat (WriterFuncRef writer, const Request &statistics_request={})
void	DumpMetric (Writer &writer, const HistogramAggregator &histogram)
	Metric serialization support for HistogramAggregator.
std::string	ToJsonFormat (const Storage &statistics, const Request &statistics_request={})
std::string	ToJsonFormat (WriterFuncRef writer, const Request &statistics_request={})
void	SolomonSkip (formats::json::ValueBuilder &stats_node)
	Marks statistics node to be excluded from Solomon sensor name.
void	SolomonSkip (formats::json::ValueBuilder &&stats_node)
	Marks statistics node to be excluded from Solomon sensor name.
void	SolomonRename (formats::json::ValueBuilder &stats_node, const std::string &new_name)
	Replaces a node in Solomon sensor name.
void	SolomonRename (formats::json::ValueBuilder &&stats_node, const std::string &new_name)
	Replaces a node in Solomon sensor name.
void	SolomonLabelValue (formats::json::ValueBuilder &stats_node, const std::string &label_name)
void	SolomonLabelValue (formats::json::ValueBuilder &&stats_node, const std::string &label_name)
void	SolomonChildrenAreLabelValues (formats::json::ValueBuilder &stats_node, const std::string &label_name)
void	SolomonChildrenAreLabelValues (formats::json::ValueBuilder &&stats_node, const std::string &label_name)
template<typename ValueType, typename AverageType>
auto	Serialize (const MinMaxAvg< ValueType, AverageType > &mma, formats::serialize::To< formats::json::Value >)
template<typename ValueType, typename AverageType>
void	DumpMetric (Writer &writer, const MinMaxAvg< ValueType, AverageType > &value)
std::string	GetPercentileFieldName (double perc)
template<size_t M, typename Counter, size_t ExtraBuckets, size_t ExtraBucketSize>
void	DumpMetric (Writer &writer, const Percentile< M, Counter, ExtraBuckets, ExtraBucketSize > &perc, std::initializer_list< double > percents={0, 50, 90, 95, 98, 99, 99.6, 99.9, 100})
template<typename T>
formats::json::ValueBuilder	PercentileToJson (const T &perc, std::initializer_list< double > percents)
template<typename T>
formats::json::ValueBuilder	PercentileToJson (const T &perc)
formats::json::Value	Serialize (const std::vector< Label > &labels, formats::serialize::To< formats::json::Value >)
formats::json::Value	Serialize (const Warning &entry, formats::serialize::To< formats::json::Value >)
std::string_view	ToString (WarningCode code)
formats::json::Value	Serialize (const PortabilityWarnings &info, formats::serialize::To< formats::json::Value >)
PortabilityWarnings	GetPortabilityWarnings (const Storage &statistics, const Request &request)
PortabilityWarnings	GetPortabilityWarnings (WriterFuncRef writer, const Request &request)
std::string	ToPrettyFormat (const Storage &statistics, const Request &statistics_request={})
	Output statistics in a human-readable, compact format, useful for visual inspection.
std::string	ToPrettyFormat (WriterFuncRef writer, const Request &statistics_request={})
std::string	ToPrometheusFormat (const Storage &statistics, const Request &request={})
	Output statistics in Prometheus format, each metric has gauge type.
std::string	ToPrometheusFormat (WriterFuncRef writer, const Request &request={})
std::string	ToPrometheusFormatUntyped (const Storage &statistics, const Request &request={})
	Output statistics in Prometheus format, without metric types.
std::string	ToPrometheusFormatUntyped (WriterFuncRef writer, const Request &request={})
template<typename Counter, typename Result, typename Timer>
void	DumpMetric (Writer &writer, const RecentPeriod< Counter, Result, Timer > &recent_period)
	Writer support for RecentPeriod
template<typename Counter, typename Result, typename Timer>
void	ResetMetric (RecentPeriod< Counter, Result, Timer > &recent_period)
	Reset support for RecentPeriod.
template<typename T>
void	DumpMetric (Writer &writer, const RelaxedCounter< T > &value)
template<typename T>
void	ResetMetric (RelaxedCounter< T > &value)
std::string	ToSolomonFormat (const Storage &statistics, const std::unordered_map< std::string, std::string > &common_labels, const Request &statistics_request={})
std::string	ToSolomonFormat (WriterFuncRef writer, const std::unordered_map< std::string, std::string > &common_labels, const Request &statistics_request={})
void	DumpMetric (Writer &writer, const Storage &storage)
	Dumps Writer-based metrics registered in storage.
void	DumpMetric (Writer &writer, const StripedRateCounter &value)
void	ResetMetric (StripedRateCounter &value)

Variables
constexpr auto	kDefaultMaxPeriod = std::chrono::seconds(60)
constexpr auto	kDefaultEpochDuration = std::chrono::seconds(5)

Typedef Documentation

◆ ExtenderFunc

using utils::statistics::ExtenderFunc = std::function<formats::json::ValueBuilder(const StatisticsRequest&)>

Definition at line 27 of file storage.hpp.

◆ MetricsStoragePtr

typedef std::shared_ptr< MetricsStorage > utils::statistics::MetricsStoragePtr = std::shared_ptr<MetricsStorage>

Definition at line 27 of file fwd.hpp.

◆ PortabilityWarnings

using utils::statistics::PortabilityWarnings = std::unordered_map<WarningCode, std::vector<Warning>>

Definition at line 54 of file portability_info.hpp.

◆ WriterFunc

using utils::statistics::WriterFunc = std::function<void(Writer&)>

Definition at line 29 of file storage.hpp.

◆ WriterFuncRef

using utils::statistics::WriterFuncRef = utils::function_ref<void(Writer&) const>

Non-owning reference to a function that writes metrics via Writer.

Definition at line 215 of file writer.hpp.

Enumeration Type Documentation

◆ WarningCode

enum class utils::statistics::WarningCode

strong

Definition at line 31 of file portability_info.hpp.

Function Documentation

◆ DumpMetric() [1/6]

template<typename ValueType, typename AverageType>

void utils::statistics::DumpMetric	(	Writer &	writer,
		const MinMaxAvg< ValueType, AverageType > &	value )

Definition at line 152 of file min_max_avg.hpp.

◆ DumpMetric() [2/6]

template<size_t M, typename Counter, size_t ExtraBuckets, size_t ExtraBucketSize>

void utils::statistics::DumpMetric	(	Writer &	writer,
		const Percentile< M, Counter, ExtraBuckets, ExtraBucketSize > &	perc,
		std::initializer_list< double >	percents = {0, 50, 90, 95, 98, 99, 99.6, 99.9, 100} )

Definition at line 217 of file percentile.hpp.

◆ DumpMetric() [3/6]

template<typename Counter, typename Result, typename Timer>

void utils::statistics::DumpMetric	(	Writer &	writer,
		const RecentPeriod< Counter, Result, Timer > &	recent_period )

Writer support for RecentPeriod

Definition at line 187 of file recentperiod.hpp.

◆ DumpMetric() [4/6]

template<typename T>

void utils::statistics::DumpMetric	(	Writer &	writer,
		const RelaxedCounter< T > &	value )

Definition at line 76 of file relaxed_counter.hpp.

◆ DumpMetric() [5/6]

template<class Metric>

void utils::statistics::DumpMetric	(	Writer &	writer,
		const std::atomic< Metric > &	m )

Definition at line 218 of file writer.hpp.

◆ DumpMetric() [6/6]

void utils::statistics::DumpMetric	(	Writer &	writer,
		const Storage &	storage )

Dumps Writer-based metrics registered in storage.

Legacy JSON extenders are skipped; Storage::VisitMetrics dumps those as well.

◆ GetPortabilityWarnings()

PortabilityWarnings utils::statistics::GetPortabilityWarnings	(	const Storage &	statistics,
		const Request &	request )

Output portability info for statistics.

See also: Functional service tests (testsuite)

◆ operator+()

Rate utils::statistics::operator+	(	Rate	first,
		Rate	second )

inlinenoexcept

Definition at line 58 of file rate.hpp.

◆ PercentileToJson() [1/2]

template<typename T>

formats::json::ValueBuilder utils::statistics::PercentileToJson ( const T & perc )

Definition at line 35 of file percentile_format_json.hpp.

◆ PercentileToJson() [2/2]

template<typename T>

formats::json::ValueBuilder utils::statistics::PercentileToJson	(	const T &	perc,
		std::initializer_list< double >	percents )

Definition at line 20 of file percentile_format_json.hpp.

◆ PrintTo()

void utils::statistics::PrintTo	(	const Snapshot &	data,
		std::ostream *	)

Support for gtest diagnostics for utils::statistics::Snapshot.

Warning: Never check the printed value programmatically! The string is not stable and may change, depending on the version of stdlib and userver.

Valid usage:

EXPECT_EQ(..., ...) << testing::PrintToString(snapshot);

Invalid usage:

std::ostringstream stream;
PrintTo(snapshot, &stream);
EXPECT_EQ(stream.str(), ...);

◆ RegisterWriterScope()

void utils::statistics::RegisterWriterScope	(	const components::ComponentContext &	,
		std::string	common_prefix,
		WriterFunc	func,
		std::vector< Label >	add_labels = {} )

Add a writer function to Storage from components::StatisticsStorage. It automatically calls utils::statistics::Storage::RegisterWriter() just after the component construction and utils::statistics::Entry::Unregister() just before the component destructor.

See also: Storage::RegisterWriter.

◆ ResetMetric() [1/2]

template<typename Counter, typename Result, typename Timer>

void utils::statistics::ResetMetric ( RecentPeriod< Counter, Result, Timer > & recent_period )

Reset support for RecentPeriod.

Definition at line 193 of file recentperiod.hpp.

◆ ResetMetric() [2/2]

template<typename T>

void utils::statistics::ResetMetric ( RelaxedCounter< T > & value )

Definition at line 81 of file relaxed_counter.hpp.

◆ Serialize() [1/2]

template<typename ValueType, typename AverageType>

auto utils::statistics::Serialize	(	const MinMaxAvg< ValueType, AverageType > &	mma,
		formats::serialize::To< formats::json::Value >	)

Definition at line 146 of file min_max_avg.hpp.

◆ Serialize() [2/2]

formats::json::Value utils::statistics::Serialize	(	const PortabilityWarnings &	info,
		formats::serialize::To< formats::json::Value >	)

JSON serialization for the PortabilityInfo in the following format:

{
  "warning_code": [
    {
      "error_message": "Human readable message",
      "path": "foo-bar",
      "labels": {
        "some-label": "label-value",
        "some-other-label": "other-label-value",
      }
    }
  ],
  "another_error_id": [
    {
      "path": "foo.bar",
      "labels": {
        "another-label": "another-value"
      }
    },
  ]
}

◆ SolomonChildrenAreLabelValues() [1/2]

void utils::statistics::SolomonChildrenAreLabelValues	(	formats::json::ValueBuilder &&	stats_node,
		const std::string &	label_name )

Moves statistics node children names to label values for Solomon sensors.

Deprecated: Use utils::statistics::Writer instead.

◆ SolomonChildrenAreLabelValues() [2/2]

void utils::statistics::SolomonChildrenAreLabelValues	(	formats::json::ValueBuilder &	stats_node,
		const std::string &	label_name )

Moves statistics node children names to label values for Solomon sensors.

Deprecated: Use utils::statistics::Writer instead.

◆ SolomonLabelValue() [1/2]

void utils::statistics::SolomonLabelValue	(	formats::json::ValueBuilder &&	stats_node,
		const std::string &	label_name )

Moves statistic node name to label value for Solomon sensor.

Deprecated: Use utils::statistics::Writer instead.

Warning: Cannot be applied to leaf nodes.

◆ SolomonLabelValue() [2/2]

void utils::statistics::SolomonLabelValue	(	formats::json::ValueBuilder &	stats_node,
		const std::string &	label_name )

Moves statistic node name to label value for Solomon sensor.

Deprecated: Use utils::statistics::Writer instead.

Warning: Cannot be applied to leaf nodes.

◆ SolomonRename() [1/2]

void utils::statistics::SolomonRename	(	formats::json::ValueBuilder &&	stats_node,
		const std::string &	new_name )

Replaces a node in Solomon sensor name.

Deprecated: Use utils::statistics::Writer instead.

Warning: Cannot be applied to leaf nodes.

◆ SolomonRename() [2/2]

void utils::statistics::SolomonRename	(	formats::json::ValueBuilder &	stats_node,
		const std::string &	new_name )

Replaces a node in Solomon sensor name.

Deprecated: Use utils::statistics::Writer instead.

Warning: Cannot be applied to leaf nodes.

◆ SolomonSkip() [1/2]

void utils::statistics::SolomonSkip ( formats::json::ValueBuilder && stats_node )

Marks statistics node to be excluded from Solomon sensor name.

Deprecated: Use utils::statistics::Writer instead.

Warning: Cannot be applied to leaf nodes.

◆ SolomonSkip() [2/2]

void utils::statistics::SolomonSkip ( formats::json::ValueBuilder & stats_node )

Marks statistics node to be excluded from Solomon sensor name.

Deprecated: Use utils::statistics::Writer instead.

Warning: Cannot be applied to leaf nodes.

◆ ToJsonFormat()

std::string utils::statistics::ToJsonFormat	(	const Storage &	statistics,
		const Request &	statistics_request = {} )

Output statistics in JSON format:

{
  "metric-path": [
    {
      "type": "GAUGE",
      "value": 42,
      "labels": {
        "some-label": "label-value",
        "some-other-label": "other-label-value",
      }
    },
    {
      "type": "RATE",
      "value": 43,
      "labels": {
        "another-label": "another-value"
      }
    },
  ]
}

◆ ToPrettyFormat()

std::string utils::statistics::ToPrettyFormat	(	const Storage &	statistics,
		const Request &	statistics_request = {} )

Output statistics in a human-readable, compact format, useful for visual inspection.

Warning: This format is unstable, we reserve the right to change it at any point. Don't use it in scripts, use utils::statistics::ToJsonFormat instead!

The current version of this unstable format is:

metric-path: some-label=label-value, other-label=other-label-value\tRATE\t42

◆ ToSolomonFormat()

std::string utils::statistics::ToSolomonFormat	(	const Storage &	statistics,
		const std::unordered_map< std::string, std::string > &	common_labels,
		const Request &	statistics_request = {} )

Output statistics in Solomon format with tags (labels). It's JSON format in the form

{
  "commonLabels": {
    "application": "some-service-name"
  },
  "metrics": [
    {
      "labels": {
        "sensor": "cpu_time_sec"
      },
      "value": 34155.69
    },
    {
      "labels": {
        "sensor": "postgresql.replication-lag.min",
        "postgresql_database": "some_pg_db",
        "postgresql_database_shard": "shard_0",
        "postgresql_cluster_host_type": "slaves",
        "postgresql_instance": "some_host:6432"
      },
      "value": 0
    },
    {
      "labels": {
        "sensor": "http.handler.in-flight",
        "http_path": "/ping",
        "http_handler": "handler-ping"
      },
      "value": 1
    },
    {
      "labels": {
        "sensor": "cache.full.time.last-update-duration-ms",
        "cache_name": "some-cache-name"
      },
      "value": 23
    }
  ]
}

◆ VisitMetrics() [1/2]

void utils::statistics::VisitMetrics	(	const HasWriterSupport auto &	metric,
		BaseFormatBuilder &	out,
		const Request &	request = {} )

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

Dumps metric via writer = metric (DumpMetric / built-in Writer support).

Definition at line 247 of file writer.hpp.

◆ VisitMetrics() [2/2]

void utils::statistics::VisitMetrics	(	WriterFuncRef	func,
		BaseFormatBuilder &	out,
		const Request &	request = {} )

Low-level connecting function that dumps metrics from func into out.

Calls func with a Writer. Each metric written through that Writer is forwarded to out.HandleMetric. request filters the metrics and may attach extra labels; request.prefix is a match filter, not a path to write under.

utils::statistics::Storage::VisitMetrics uses this function to dump Writer-based metrics. It is used to implement formats such as:

Prometheus
Graphite
JSON
pretty format
Solomon
portability info

Parameters

func	writes metrics to the provided Writer
out	receives each written metric via BaseFormatBuilder::HandleMetric
request	metric filter and extra labels

Variable Documentation

◆ kDefaultEpochDuration

auto utils::statistics::kDefaultEpochDuration = std::chrono::seconds(5)

constexpr

Definition at line 17 of file common.hpp.

◆ kDefaultMaxPeriod

auto utils::statistics::kDefaultMaxPeriod = std::chrono::seconds(60)

constexpr

Definition at line 15 of file common.hpp.

Detailed Description

Classes

Typedefs

Enumerations

Functions

Variables

Typedef Documentation

◆ ExtenderFunc

◆ MetricsStoragePtr

◆ PortabilityWarnings

◆ WriterFunc

◆ WriterFuncRef

Enumeration Type Documentation

◆ WarningCode

Function Documentation

◆ DumpMetric() [1/6]

◆ DumpMetric() [2/6]

◆ DumpMetric() [3/6]

◆ DumpMetric() [4/6]

◆ DumpMetric() [5/6]

◆ DumpMetric() [6/6]

◆ GetPortabilityWarnings()

◆ operator+()

◆ PercentileToJson() [1/2]

◆ PercentileToJson() [2/2]

◆ PrintTo()

◆ RegisterWriterScope()

◆ ResetMetric() [1/2]

◆ ResetMetric() [2/2]

◆ Serialize() [1/2]

◆ Serialize() [2/2]

◆ SolomonChildrenAreLabelValues() [1/2]

◆ SolomonChildrenAreLabelValues() [2/2]

◆ SolomonLabelValue() [1/2]

◆ SolomonLabelValue() [2/2]

◆ SolomonRename() [1/2]

◆ SolomonRename() [2/2]

◆ SolomonSkip() [1/2]

◆ SolomonSkip() [2/2]

◆ ToJsonFormat()

◆ ToPrettyFormat()

◆ ToSolomonFormat()

◆ VisitMetrics() [1/2]

◆ VisitMetrics() [2/2]

Variable Documentation

◆ kDefaultEpochDuration

◆ kDefaultMaxPeriod