Repository files navigation

This project is starting to be maintained by John Ward, the current status is that I am bringing the project up to date with the latest dependencies, removing deprecated Rust code and adjusting the tests.

Welcome to kafka-rust's new home:https://github.com/kafka-rust

• This library is primarily documented through examples in itsAPI documentation.
• Documentation about Kafka itself can be found atits project home page.

This crate works with Cargo and is oncrates.io. The API is currentlyunder heavy movement although we do follow semantic versioning (butexpect the version number to grow quickly).

[dependencies]kafka ="0.10"

To build kafka-rust the usualcargo build should suffice. The cratesupports various features which can be turned off at compile time.See kafka-rust'sCargo.toml andcargo's documentation.

kafka-rust is tested for compatibility with a select few Kafka versions from 0.8.2 to 3.1.0. However,not all features from Kafka 0.9 and newer are supported yet.

As mentioned, thecargo generated documentation contains some examples.Further, standalone, compilable example programs are provided in theexamples directory of the repository.

This is a higher-level consumer API for Kafka and is provided by themodulekafka::consumer. It provides convenient offset managementsupport on behalf of a specified group. This is the API a clientapplication of this library wants to use for receiving messages fromKafka.

This is a higher-level producer API for Kafka and is provided by themodulekafka::producer. It provides convenient automatic partitionassignment capabilities through partitioners. This is the API aclient application of this library wants to use for sending messagesto Kafka.

KafkaClient in thekafka::client module is the central point ofthis API. However, this is a mid-level abstraction for Kafka rathersuitable for building higher-level APIs. Applications typically wantto use the already mentionedConsumer andProducer.Nevertheless, the main features orKafkaClient are:

• Loading metadata
• Fetching topic offsets
• Sending messages
• Fetching messages
• Committing a consumer group's offsets
• Fetching a consumer group's offsets

There's still a lot of room for improvement onkafka-rust.Not everything works right at the moment, and testing coverage could be better.Use it in production at your own risk. Have a look at theissue tracker and feel freeto contribute by reporting new problems or contributing to existingones. Any constructive feedback is warmly welcome!

As usually with open source, don't hesitate to fork the repo andsubmit a pull request if you see something to be changed. We'll behappy to seekafka-rust improving over time.

When working locally, the integration tests require that you must haveDocker (1.10.0+) and docker-compose (1.6.0+) installed and run the tests via theincludedrun-all-tests script in thetests directory. See therun-all-testsscript itself for details on its usage.

Note unless otherwise explicitly stated in the documentation, thislibrary will ignore requests to topics which it doesn't know about.In particular it will not try to retrieve messages fromnon-existing/unknown topics. (This behavior is very likely to changein future version of this library.)

Given a local kafka server installation you can create topics with thefollowing command (wherekafka-topics.sh is part of the Kafkadistribution):

kafka-topics.sh --topic my-topic --create --zookeeper localhost:2181 --partitions 1 --replication-factor 1

Zookeeper will be removed in the next major kafka release. Using--bootstrap-server to be more ready.

kafka-topics.sh --topic my-topic --create --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

See alsoKafka's quickstart guidefor more information.

• rust-rdkafka is an emerging alternative Kafka client library for Rust based onlibrdkafka. rust-rdkafka provides a safe Rust interface to librdkafka.