Repository files navigation

Dockerfile forApache Kafka

The image is available directly fromhttps://registry.hub.docker.com/

##Pre-Requisites

• install docker-composehttps://docs.docker.com/compose/install/
• modify theKAFKA_ADVERTISED_HOST_NAME indocker-compose.yml to match your docker host IP (Note: Do not use localhost or 127.0.0.1 as the host ip if you want to run multiple brokers.)
• if you want to customise any Kafka parameters, simply add them as environment variables indocker-compose.yml, e.g. in order to increase themessage.max.bytes parameter set the environment toKAFKA_MESSAGE_MAX_BYTES: 2000000. To turn off automatic topic creation setKAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'

##Usage

Start a cluster:

• docker-compose up -d

Add more brokers:

• docker-compose scale kafka=3

Destroy a cluster:

• docker-compose stop

##Note

The defaultdocker-compose.yml should be seen as a starting point. By default each broker will get a new port number and broker id on restart. Depending on your use case this might not be desirable. If you need to use specific ports and broker ids, modify the docker-compose configuration accordingly, e.g.docker-compose-single-broker.yml:

• docker-compose -f docker-compose-single-broker.yml up

##Broker IDs

If you don't specify a broker id in your docker-compose file, it will automatically be generated (seehttps://issues.apache.org/jira/browse/KAFKA-1070. This allows scaling up and down. In this case it is recommended to use the--no-recreate option of docker-compose to ensure that containers are not re-created and thus keep their names and ids.

##Automatically create topics

If you want to have kafka-docker automatically create topics in Kafka duringcreation, aKAFKA_CREATE_TOPICS environment variable can beadded indocker-compose.yml.

Here is an example snippet fromdocker-compose.yml:

    environment:      KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1"

Topic 1 will have 1 partition and 3 replicas,Topic 2 will have 1 partition and 1 replica.

##Advertised hostname

You can configure the advertised hostname in different ways

1. explicitly, usingKAFKA_ADVERTISED_HOST_NAME
2. via a command, usingHOSTNAME_COMMAND, e.g.HOSTNAME_COMMAND: "route -n | awk '/UG[ \t]/{print $$2}'"

When using commands, make sure you review the "Variable Substitution" section inhttps://docs.docker.com/compose/compose-file/

IfKAFKA_ADVERTISED_HOST_NAME is specified, it takes presendence overHOSTNAME_COMMAND

##Tutorial

http://wurstmeister.github.io/kafka-docker/