Add support for per-broker and per-topic partition metrics

[allait]

Adds a new KafkaMetrics struct that can export global, per-broker and per-topic/partition statistics from librdkafka as swift-metrics. New struct replaces KafkaConfiguration.*Metrics structs that only supported a subset of top-level metrics and required creating a Gauge instance for each one.

Motivation

Most of the useful librdkafka metrics like consumer lag, latency and round-trip times require a dynamic set of Gauges so weren't supported and don't combine well with the existing KafkaConfiguration.*Metrics interfaces.

Modifications

Add a mapping for most of librdkafka stats in RDKafkaStatistics

Adds broker, topic, partition and consumer-group level properties for parsing librdkafka statistics output.

This covers most of the fields exported in librdkafka statistics events, skipping some internal ones (e.g. internal timestamp) that are unlikely to be useful as metrics.

Renames existing properties to have a more consistent naming convention: using total as a suffix to closer match the metric names recommended by Prometheus docs and dropping repeated ...KafkaBroker....

Add KafkaMetrics to convert librdkafka statistics to swift-metrics

KafkaMetrics is going to replace metrics structs from KafkaConfiguration.

Instead of expecting users to initialize metric instances it creates them on demand using the provided configuration and label prefix.

Note: all metric instances are created on-demand, we're not holding a cache of metric handlers as the list of broker and topic statistics might change between updates. Statistics interval can only be configured in increments of 1 second, but this might still have a performance impact when there are many brokers/partitions involved.

Replace KafkaConfiguration.*Metrics with KafkaMetrics in configs

Update clients to use KafkaMetrics

Drop KafkaConfiguration metrics property and structs

Supporting both old and new implementations at the same time would require an additional public protocol. With the goal of removing old configuration structs before 1.0 we drop the .metrics field from them instead.

Update KafkaMetricsTests

Add tests for per-broker and per-partition metrics and make the tests run concurrently by specifying the metrics factory.

Result

Most of librdkafka statistics (excluding some internal ones) are exported as swift-metrics Gauges.