Repository files navigation

Weno longer develop this project. If you'd like to take over and continue its maintenance, please fork this repository and let us know by Github issues.

RegisterMarathon Tasks asConsul Services for service discovery.

marathon-consul takes information provided by theMarathon event bus andforwards it to Consul agents. It also re-syncs all the information from Marathonto Consul on startup and repeats it with given interval.

Note: In the future release Event Bus (callbacks) will be considered deprecated and eventuallyremoved in favor of Event Stream (SSE).Right now marathon-consul is supporting both solutions.SSE is provided as experimental feature, disabled by default (more).

This project is based on

• mesos-consul
• marathon-consul

• CiscoCloud/marathon-consul copies application information to Consul KV whileallegro/marathon-consul registers tasks as Consul services(it is more similar to CiscoCloud/mesos-consul)
• CiscoCloud/mesos-consul uses polling while allegro/marathon-consul usesMarathon's event busto detect changes
• CiscoCloud/marathon-consul is no longer developed(seecomment)

To simply compile and run the source code:

go run main.go [options]

To run the tests:

make test

To build the binary:

make build

Binary distribution ofmarathon-consul can be downloaded directly fromthe releases page.Download the build dedicated to your OS. After unpacking the archive, runmarathon-consul binary. You can also add someoptions, for example:

marathon-consul --marathon-location=marathon.service.consul:8080 --sync-interval=5m --log-level=debug

If you are a Debian/Ubuntu user, you can easily installmarathon-consul as adeb package usingAPT package manager. Bothupstart andsystemd service managers are supported.All releases are published asdeb packages toour repository at Bintray.

To installmarathon-consul withapt-get, simply follow the instructions:

# add our public key to aptcurl -s https://bintray.com/user/downloadSubjectPublicKey?username=allegro| sudo apt-key add -# add the repository urlecho"deb http://dl.bintray.com/v1/content/allegro/deb /"| sudo tee /etc/apt/sources.list.d/marathon-consul.list# update apt cachesudo apt-get -y update# install latest release of marathon-consulsudo apt-get -qy install marathon-consul

Run it withservice marathon-consul start. The configuration file is located at/etc/marathon-consul.d/config.json.

To build docker image run

make docker

Then you can run it with

docker run -d -P allegro/marathon-consul [options]

If you want to use SSL you will need to expose cert store to Docker. The Following line is only example,your cert store might be different depending on your system.

docker run'/etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt' -P  allegro/marathon-consul

• marathon-consul should be installed on all Marathon masters

• Consul Agents should be available on every Mesos agent.
• Tasks will be registered at the Mesos slave they run on.

• Task labels are used by marathon-consul to register tasks in Consul.
• Only tasks which are labeled asconsul will be registered in Consul.If theconsul label is left blank like"consul": "", the task will be registered with the app nameby default. A different name can be provided as the label's value, e.g."consul": "customName". As an exception to this rule, for backward compatibility with the0.3.x branch, a value oftrue is resolved to the default name.

  "id": "my-new-app",  "labels": {    "consul": ""  }

• Only services with tag specified byconsul-tag property will be maintained. By default,"consul-tag": "marathon" is automatically added when the task is registered.
• Labels with avalue oftag are converted to consul-tags, and appear in Consul as ServiceTags.
• For example, we can set these tags in an app definition like:

  "id": "my-new-app",  "labels": {    "consul": "",    "varnish": "tag",    "metrics": "tag"  }

• If marathon-consul registers the app with Consul, we can then query Consul and see these tags appear:

curl -X GET http://localhost:8500/v1/catalog/service/my-new-app..."ServiceName": "my-new-app","ServiceTags": [  "marathon",  "varnish",  "metrics",  "marathon-task:my-new-app.6a95bb03-6ad3-11e6-beaf-080027a7aca0"],

• Every service registration contains an additional tagmarathon-task specifying the Marathon task id related to this registration.
• If there are multiple ports in use for the same app, note that only the first one will be registered by marathon-consul in Consul.

If you need to register your task under multiple ports, refer toAdvanced usage section below.

• At least one HTTP healthcheck should be defined for a task. The task is registered when Marathon marks it as alive.
• The provided HTTP healthcheck will be transferred to Consul.
• Seethisfor more details.

Healthchecks commands are registered in Consul with a simple variable substitution.

• $HOST is replaced by task hostname.
• $PORT0, $PORT1... are replaced by port number defined in task.

Using a special shell syntax can break this variable substitution ($HOST, ${PORT0} ...)

• The scheduled Marathon-consul sync may run in two modes:
  • Only on node that is the currentMarathon-leader,marathon-leader parameter should be set tohostname:port the current node appears in the Marathon cluster.This mode isenabled by default and themarathon-leader property is set to the hostname resolved by OS.Note that there is a difference betweenmarathon-leader andmarathon-location:marathon-leader is used fornode leadership detection (should be set to cluster-wide node name), whilemarathon-location is used forconnection purpose (may be set tolocalhost)
  • On every node,sync-force parameter should be set totrue
• If marathon-consul fails on startup sync and you see following error"Can't get Consul services: No Consul client available in agents cache"it may be caused by empty consul agents cache. If this occurs try configuring--consul-local-agent-host to Consul Master or Consul agent.

If you need to map your Marathon task into multiple service registrations in Consul, you can configure marathon-consulvia Marathon'sportDefinitions:

  "id": "my-new-app",  "labels": {    "consul": "",    "common-tag": "tag"  },  "portDefinitions": [    {      "labels": {        "consul": "my-app-custom-name"      }    },    {      "labels": {        "consul": "my-app-other-name",        "specific-tag": "tag"      }    }  ]

This configuration would result in two service registrations:

curl -X GET http://localhost:8500/v1/catalog/service/my-app-custom-name..."ServiceName": "my-app-custom-name","ServiceTags": [  "marathon",  "common-tag",  "marathon-task:my-new-app.6a95bb03-6ad3-11e6-beaf-080027a7aca0"],"ServicePort": 31292,...curl -X GET http://localhost:8500/v1/catalog/service/my-app-other-name..."ServiceName": "my-app-other-name","ServiceTags": [  "marathon",  "common-tag",  "specific-tag",  "marathon-task:my-new-app.6a95bb03-6ad3-11e6-beaf-080027a7aca0"],"ServicePort": 31293,...

If any port definition contains theconsul label, then advanced configuration mode is enabled. As a result, only the portscontaining this label are registered, under the name specified as the label's value – with empty value resolved to default name.Names don't have to be unique – you can have multiple registrations under the same name, but on different ports,perhaps with different tags. Note that theconsul label still needs to be present in the top-level application labels, eventhough its value won't have any effect.

Tags configured in the top-level application labels will be added to all registrations. Tags configured in the port definitionlabels will be added only to corresponding registrations.

All registrations share the samemarathon-task tag.

Until 1.x.x marathon-consul would register services in Consul with registration id equal to related Marathon task id. Since 1.x.x registration ids are different andan additional tag,marathon-task, is added to each registration.

If you update marathon-consul from version 0.x.x to 1.x.x, expect the synchronization phase during the first startup toreregister all healthy services managed by marathon-consul to the new format. Unhealthy services will get deregistered in the process.

While using SSE please consider:

• SSE is using Web module config for queues, event sizes, in the future will be moved to sse module,
• SSE is using marathon-leader config for determining current leader, when this value match leader returned by marathon (/v2/leader endpoint)then SSE is started on this instance,
• when enabled SSE is spawning its own own set of workers and separated dispatcher,
• be advised to disable marathon callback subscription when enabling SSE, otherwise it might result in doubling registers and deregisers.

Marathon-Consul does not support HTTP callbacks.Marathondeprecated support for HTTP callbacks in 1.4.This mechanism is no longer available starting fromMarathon 1.5.

The following section describes known limitations inmarathon-consul.

• In Marathon when a deployment changing the application's service name (by changing itslabels) is being stopped, it changes app's configuration anyway.This means we loose the link between the app and the services registered with the old name in Consul.Later on, if another deployment takes place, new services are registered with a new name, the old ones are not being deregistered though.A scheduled sync is required to wipe them out.

To release new version of marathon-consul follow steps:

1. Commit all changes you need for release to master branch.
2. git checkout master
3. git checkout -b release/<version> e.g.,git checkout -b release/1.3.1
4. make version v=<version> e.g.,make version v=1.3.1
5. git push
6. Create pull request from branchrelease/<version> to master.
7. Once pull request gets merged put tag on this commit (remember to update your master withgit pull)git tag <version> -f e.g.,git tag 1.3.1 -f. Create annotated tag with release notes in it.
8. Travis will automatically prepare github release from tag on master. Go there and update release notes.
9. Copy github release to bintray.

Marathon-consul is released under the Apache 2.0 license (seeLICENSE)