Repository files navigation

A command line tool to interact with an Apache Kafka cluster.

kafkactl is a command line tool for people that work withApache Kafka.It can be used toquery information (e.g., brokers, topics, messages, consumers, etc.)orcreate,update, anddelete resources in the Kafka cluster. The command canbe used toconsume andproduce messages and has native support for Kafka messagesthat are encoded usingProtocol Buffers orApache Avro.Finally, kafkactl implements more advanced behaviour on top of these primitives,e.g. toreplay messages on the same or another cluster.

You can either install a pre-compiled binary or compile from source.

Download the pre-compiled binaries from thereleases page and copythem into your$PATH.

If you haveGo installed, you can fetch the latest code and compile anexecutable binary using the following command:

go install github.com/fgrosse/kafkactl@latest

kafkactl is intended to be used as CLI tool on your local machine. It is bothuseful in day to day operations on the shell as well in automation and scripting.The tool is split into multiple commands, each with its own help output. You cansee each commands usage by setting the--help flag.

$ kafkactl --help       kafkactl is a command line tool to interact with an Apache Kafka cluster.Usage:  kafkactl [command]Managing configuration  config      Manage the kafkactl configuration  context     Switch between different configuration contextsResource operations  create      Create resources in the Kafka cluster  delete      Delete resources in the Kafka cluster  get         Display resources in the Kafka cluster  update      Update resources in the Kafka clusterConsuming & Producing messages  consume     Consume messages from a Kafka topic and print them to stdout  produce     Read messages from stdin and write them to a Kafka topic  replay      Read messages from a Kafka topic and append them to the end of a topicAdditional Commands:  completion  Generate the autocompletion script for the specified shellFlags:      --config string    path to kafkactl config file (default "/home/fgrosse/.config/kafkactl/config.yml")      --context string   the name of the kafkactl context to use (defaults to "current_context" field from config file)  -h, --help             help for kafkactl  -v, --verbose          enable verbose outputUse "kafkactl [command] --help" for more information about a command.

The first thing you need to do after installingkafkactl is to set up aconfiguration context. Each context contains all information to connect to aset of Kafka brokers which form a Kafka cluster.

kafkactl config add "my-context" --broker example.com:9092

This will add and activate a new configuration context named "my-context".Each subsequent command that interacts with Kafka will be directed towards thebrokers of the currently active configuration context. This is similar to howkubectl manages different contexts to talk to multiple Kubernetes clusters.

You can alsoadd,delete andrename configuration contexts as well asprintthe full kafkactl configuration usingkafkactl config and its sub-commands.If you want to learn more, try passing the--help flag.

Now you can run your firstkafkactl command to query information from the cluster:

$ kafkactl get brokers                     ID      ADDRESS      ROLE1       kafka1:9092  controller  2       kafka2:9093              3       kafka3:9094$ kafkactl get topicsNAME    PARTITIONS  REPLICATION  RETENTIONtest-1  2           1            2 weeks   test-2  4           3            7 days    test-3  10          3            12 hours

Apart frombrokers andtopics, you can also query information about the Kafkaclusterconfiguration, all knownconsumer groups and fetch individualmessagesusingkafkactl get.

All sub commands ofkafkactl get default to printing information in a humanfriendly way (e.g. as a table). They also have an--output flag which allows forother output encoding that is more suitable for scripting (e.g. JSON) and oftencontains more information than what fits into the tabular format.

You can usekafkactl to create and delete topics and update configuration.Please refer to the correspondingkafkactl --help output for more informationand examples.

There are multiple applications that call themselveskafkactl. All of themhave been developed independently but with similar feature sets.

This incarnation ofkafkactl was created atFraugster in September 2017.It was a useful tool for many years and we decided to keep it around even after Fraugsterceased to exist, mainly because we are very used to it and maybe for sentimental reasons.

Otherkafkactl implementations come with similar features (e.g. Protobuf & Avro support,managing configuration with kubectl-like contexts). We list them here, so you canpick the tool that serves your use case best:

• deviceinsight/kafkactl
• jbvmio/kafkactl

• sarama - a Go library for Apache Kafka
• confluent-kafka-go - Confluent's Apache Kafka Golang client
• goavro - a library that encodes and decodes Avro data
• cobra - a library to build powerful CLI applications
• viper - configuration with fangs
• protoreflect - reflection for Go Protocol Buffers
• testify - A simple unit test library
• and more

Please readCONTRIBUTING.md for details on our code ofconduct and on the process for submitting pull requests to this repository.

We useSemVer for versioning.All significant (e.g. breaking) changes are documented in theCHANGELOG.md.A list of all available versions can be found at thereleases page.

• Friedrich Große -Initial work -fgrosse
• Various folks atFraugster
  • Julius Bachnick -juliusbachnick
  • Andreas Krennmair -akrennmair
  • Stefan Warman -warmans
  • Stefan Koshiw -panamafrancis
  • Oleg Prozorov -oleg
  • and more...

See also the list ofcontributors who participated in this project.

This project is licensed under the BSD-3-Clause License - see theLICENSE file for details.