Repository files navigation

NOTE: For a more recently developed collector with more output flexibility and support, please evaluate usage of the followingTelegraf plugins for your use case:cisco_telemetry_mdt andcisco_telemetry_gnmi.

A Model-Driven Telemetry collector based on the open-source toolpipeline including enhancements and bug fixes.

pipeline-gnmi is a Model-Driven Telemetry (MDT) collector based on the open-source toolpipeline which has gNMI support and fixes for maintainability (e.g. Go modules) and compatibility (e.g. Kafka version support). It supports MDT from IOS XE, IOS XR, and NX-OS enabling end-to-end Cisco MDT collection for DIY operators.

The original pipeline README is includedhere for reference.

pipeline-gnmi is written in Go and targets Go 1.11+. Windows and MacOS/Darwin support is experimental.

1. pipeline-gnmi binaries may be downloaded fromReleases
2. Built from source:

git clone https://github.com/cisco-ie/pipeline-gnmicd pipeline-gnmimake build

3. Acquired viago get github.com/cisco-ie/pipeline-gnmi to be located in$GOPATH/bin

pipeline configuration support is maintained and detailed in theoriginal README. Sample configuration is supplied aspipeline.conf.

This project introduces support forgNMI.gNMI is a standardized and cross-platform protocol for network management and telemetry. gNMI does not require prior sensor path configuration on the target device, merely enabling gRPC/gNMI is enough. Sensor paths are requested by the collector (e.g. pipeline). Subscription type (interval, on-change, target-defined) can be specified per path.

Filtering of retrieved sensor values can be done directly at the input stage through selectors in the configuration file,by defining all the sensor paths that should be stored in a TSDB or forwarded via Kafka.Regular metrics filtering through metrics.json files is ignored and not implemented, due to the lack of user-friendliness of the configuration.

[mygnmirouter]stage = xport_inputtype = gnmiserver = 10.49.234.114:57777# Sensor Path to subscribe to. No configuration on the device necessary# Appending an @ with a parameter specifies subscription type:#   @x where x is a positive number indicates a fixed interval, e.g. @10 -> every 10 seconds#   @change indicates only changes should be reported#   omitting @ and parameter will do a target-specific subscriptions (not universally supported)#path1 = Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters@10#path2 = /interfaces/interface/state@change# Whitelist the actual sensor values we are interested in (1 per line) and drop the rest.# This replaces metrics-based filtering for gNMI input - which is not implemented.# Note: Specifying one or more selectors will drop all other sensor values and is applied for all paths.#select1 = Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters/packets-sent#select2 = Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters/packets-received# Suppress redundant messages (minimum hearbeat interval)# If set and 0 or positive, redundant messages should be suppressed by the server# If greater than 0, the number of seconds after which a measurement should be sent, even if no change has occured#heartbeat_interval = 0tls = falseusername = ciscopassword = ...

This project supports Kafka 2.x by requiring the Kafka version (kafkaversion) to be specified in the config file stage. This is a requirement of the underlying Kafka library and ensures that the library is communicating with the Kafka brokers effectively.

[kafkaconsumer]topic=mdtconsumergroup=pipeline-gnmitype=kafkastage=xport_inputbrokers=kafka-host:9092encoding=gpbdatachanneldepth=1000kafkaversion=2.1.0

This project has improved Docker support. The Dockerfile uses multi-stage builds andbuilds Pipeline from scratch. The configuration file can now be created from environment variables directly,e.g.

PIPELINE_default_id=pipelinePIPELINE_mygnmirouter_stage=xport_inputPIPELINE_mygnmirouter_type=gnmi

is translated into a pipeline.conf with following contents:

[default]id = pipeline[mygnmirouter]stage = xport_inputtype = gnmi

If the special variable_password is used, the value is encrypted using the pipeline RSA key before being written tothepassword option. Similarly_secret can be used, then the value is read from the file whose name is given asvalue, encrypted using the pipeline RSA key and then written aspassword option. If the Pipeline RSA key is notgiven or does not exist it is created upon creation of the container.

Additionally, existing replays of sensor data can be fed in efficiently using xz-compressed files.

pipeline-gnmi is licensed withApache License, Version 2.0, per pipeline.

For support, please open aGitHub Issue or emailcisco-ie@cisco.com.

Chris Cassar for implementingpipeline used by anyone interested in MDT, Steven Barth for gNMI plugin development, and the Cisco teams implementing MDT support in the platforms.