This is a protocol specification for Fluentdforward input/output plugins. This protocol is also used by fluent-logger software, and many other software in ecosystem (e.g., Docker logging driver for Fluentd).

This protocol version is v1. This spec is supported by Fluentdv0.14 (v0.14.5 and later), andv1.Protocol version v0 is supported byv0.10 andv0.12, and protocol versionv1 is compatible withv0 (All implementations which supportsv1 also supportsv0).

• September 6, 2016: the first release of this spec.
• September 12, 2016: fix inconsistent specification about EventTime.

• Handshake phase added
• "compress" option and CompressedPackedForward mode added
• SSL/TLS transport layer handling added

This specification describes the fluentd forward protocol, which is used to authenticate/authorize clients/servers, and to transport events from hosts to hosts over network.

The keywords "MUST", "MUST NOT", "SHOULD", "SHOULD NOT" and "MAY" in this document are to be interpreted as described inRFC 2119. The following terms are also used:

• event: a virtual set of tag, time and record. tag MAY be shared by 2 or more events in actual representation if these events have same tag.
• client: an endpoint that sends events.out_forward plugin is the reference client implementation shipped with the fluentd distribution.
• server: an endpoint that receives events.in_forward plugin is the reference server implementation shipped with the fluentd distribution.
• connection: a TCP connection between two endpoints.
• msgpack: a light weight binary representation of serialized objects.

• Client MAY send UDP packets to the same port of connection, to check existence of Servers.
• Server SHOULD listen the same port of data transport connection in UDP, and SHOULD respond to source address when heartbeat message arrived.

UDP heartbeat message SHOULD be a byte of0x00 in both of Client and Server.

• Server MUST listen any TCP port for data transport connection, and MAY be able to establish SSL/TLS session on that connection.
• Client MUST be able to connect to servers via TCP, and MAY connect to servers using SSL/TLS.

Connections have two phases on each sessions.

1. Handshake(optional): In this phase, servers and clients send messages each other to establish connections after authentication and authorization. It can be skipped when server doesn't require it.
2. Transport: In this phase, clients send events to servers, and servers send response messages if client require it.

Once handshake phase completes, clients can use that connection in transport phase until disconnection.

Handshake phase is optional. If server doesn't require authentication/authorization, connection becomes transport phase immediately, and client can send events without any preparation.

• Client sends a msgpackarray which contains handshake messages, or one or more events to the server through connections.
• Client MUST respond to the server for authentication/authorization messages in handshake phase by sendingPING described below.
• Client MUST disconnect the connection if authentication operation fails.
• Client MUST choose a carrier mode for events from four:Message,Forward,PackedForward andCompressedPackedForward described below.
• Client MAY send a msgpacknil value for heartbeat health check usage without any event record payload.
• Client MAY send multiple msgpackarrays of events in transport phase on a single connection, continuously.

• Server MUST receive msgpackarrays which contains handshake messages, or one or more events, from the client through connections.
• Server MAY disconnect the connection when it has network/domain ACL (access control list) and the client is not allowed by ACL.
• Server MAY sendHELO andPONG messages in handshake phase to the client for authentication and authorization.
• Server MUST disconnect the connection if authentication/authorization operation fails.
• Server MUST detect the carrier mode by inspecting the second element of the array.
• Server SHOULD ignore any request value other thanarray format.
• Note: In addition to the four carrier modes,in_forward plugin also accepts JSON representation of a single event for convenience. It detect it by the first byte of the request. It is unavailable with authentication/authorization.

Handshake messages are to establish connections with assurance for:

• Server and client share the same key (shared_key string) for the connection.
• Client has correct username and password for the connection (if required by server).

All handshake messages are msgpackarrays which contains values described below. The first element of these messages MUST show the types of messages.

• Authentication: Both server and client authenticate the peer node by having sameshared_key string. (required)
• Authorization: Server authorize client by having validusername andpassword pair. (optional)

Some values are used to implement authentication/authorization, and these are shared by server and client in handshake phase.

• nonce is generated by server, sent inHELO and used to generateshared_key_hexdigest inPING andPONG
• shared_key_salt is generated by client, send inPING and used to generateshared_key_hexdigest inPING andPONG
• auth salt is generated by server, sent inHELO and used to generatepassword (hexdigest) inPING

HELO message is sent from server to client.

• type is a string, equals toHELO.
• options is key-value pairs to tell options enabled in the connection to the client.

HELOoptions has some keys and values described below.

["HELO",  {"nonce":"xxxxxxx","auth":"yyyyyyyy","keepalive":true  }]

NOTE: HELO is inspired by the command to start SMTP sessions.

PING message is sent from client to server after HELO.

• type is a string, equals toPING.
• client_hostname is a string represents the hostname of client.
• shared_key_salt is a binary string of salt to generateshared_key_digest.
• shared_key_hexdigest is a hex string of SHA512 digest ofshared_key_salt,client_hostname,nonce andshare_key.
• username is a username string for the connection, or empty string if authentication is not required.
• password is a hex string of SHA512 digest ofauth salt,username and raw password string, or empty string if authentication is not required.

["PING","client.hostname.example.com","xxxxxxx_shared_key_salt","deadbeef_shared_key_hex_digest","a_user_name","c0ffee_password_hex_digest"]

PONG message is sent from server to client after PING.

• type is a string, equals toPONG.
• auth_result is a boolean value to represent result of authentication/authorization (true means success).
• reason is a strong to show the reason why client is disallowed, or empty string ifauth_result is true.
• server_hostname is a string represents the hostname of server.
• shared_key_hexdigest is a hex string of SHA512 digest ofshared_key_salt,server_hostname,nonce andshared_key.

["PONG",true,"","server.hostname.example.com","deadbeef_shared_key_hex_digest"]

Ifauth_result is false, server will disconnect the connection. Otherwise, client can use the connection in transport phase.

Once the connection becomes transport phase, client can send events to servers, in one event mode of modes described below.

It carries just a event.

• tag is a string separated with '.' (e.g. myapp.access) to categorize events.
• time is a EventTime value (described below), or a number of seconds since Unix epoch.
• record is key-value pairs of the event record.
• option is optional key-value pairs, to bring data to control servers' behavior.

["tag.name",1441588984,  {"message":"bar"},  {"option":"optional"}]

NOTE: EventTime is formatted into Integer in json format.

It carries a series of events as a msgpackarray on a single request.

["tag.name",  [    [1441588984, {"message":"foo"}],    [1441588985, {"message":"bar"}],    [1441588986, {"message":"baz"}]  ],  {"option":"optional"}]

It carries a series of events as a msgpack binary on a single request.

• entries is a binary chunk ofMessagePackEventStream which contains multiple raw msgpack representations ofEntry.
• Client SHOULD send aMessagePackEventStream as msgpackbin format as its binary representation.
• Client MAY send aMessagePackEventStream as msgpackstr format for compatibility reasons.
• Server MUST accept both formats ofbin andstr.
• Server MAY decode individual events on demand but MAY NOT do right after request arrival. It means it MAY costs less, compared toForward mode, when decoding is not needed by any plugins.
• Note:out_forward plugin sends events by thePackedForward mode. It encloses event records with msgpackstr format instead ofbin format for a backward compatibility reason.

["tag.name","<<MessagePackEventStream>>",  {"option":"optional"}]

Note for v2 protocol: PackedForward messages should be sent inbin format.

It carries a series of events as a msgpack binary, compressed by gzip, on a single request. The supported compression algorithm is only gzip.

• entries is a gzipped binary chunk ofMessagePackEventStream, which MAY be a concatenated binary of multiple gzip binary strings.
• Client MUST send an option withcompressed key with the valuegzip.
• Client MUST send a gzipped chunk as msgpackbin format.
• Server MUST acceptbin format.
• Server MAY decompress and decode individual events on demand but MAY NOT do right after request arrival. It means it MAY costs less, compared toForward mode, when decoding is not needed by any plugins.

["tag.name","<<CompressedMessagePackEventStream>>",  {"compressed":"gzip"}]

Entry is anarray representation of pairs of time and record, used inForward,PackedForward andCompressedPackedForward mode.

It carries an optional meta data for the request.

• Client MAY send key-value pairs of options.
• Server MAY just ignore any options given.
• size: Clients MAY send thesize option to show the number of event records in an entries by an integer as a value. Server can know the number of events without unpacking entries (especially for PackedForward and CompressedPackedForward mode).
• chunk: Clients MAY send thechunk option to confirm the server receives event records. The value is a string of Base64 representation of 128 bitsunique_id which is an ID of a set of events.
• compressed: Clients MUST send thecompressed option with valuegzip to tell servers that entries isCompressedPackedForward. Other values will be ignored.

{"chunk":"p8n9gmxTQVC8/nh2wlKKeQ==","size":4097}{"chunk":"p8n9gmxTQVC8/nh2wlKKeQ==","size":1023,"compressed":"gzip"}

• Server SHOULD close the connection silently with no response when thechunk option is not sent.
• ack: Server MUST respondack when thechunk option is sent by client. Theack response value MUST be the same value given bychunk option from client. Client SHOULD retry to send events later when the request has achunk but the response has noack.

{"ack":"p8n9gmxTQVC8/nh2wlKKeQ=="}

EventTime uses msgpack extension format of type 0 to carry nanosecond precision oftime.

• Client MAY sendEventTime instead of plain integer representation of second since unix epoch.
• Server SHOULD accept both formats of integer and EventTime.
• Binary representation ofEventTime may befixext orext(with length 8).

+-------+----+----+----+----+----+----+----+----+----+|     1 |  2 |  3 |  4 |  5 |  6 |  7 |  8 |  9 | 10|+-------+----+----+----+----+----+----+----+----+----+|    D7 | 00 | second from epoch |     nanosecond|+-------+----+----+----+----+----+----+----+----+----+|fixext8|type| 32bits integer BE | 32bits integer BE|+-------+----+----+----+----+----+----+----+----+----++--------+----+----+----+----+----+----+----+----+----+----+|      1 |  2 |  3 |  4 |  5 |  6 |  7 |  8 |  9 | 10 | 11|+--------+----+----+----+----+----+----+----+----+----+----+|     C7 | 08 | 00 | second from epoch |     nanosecond|+--------+----+----+----+----+----+----+----+----+----+----+|   ext8 | len|type| 32bits integer BE | 32bits integer BE|+--------+----+----+----+----+----+----+----+----+----+----+

• Name? means thatName is optional.
• Name* means thatName can occur zero or more times.
• <<Name>> means binary msgpack representation ofName.
• [ A, B, C ] means an array.
• nil,string,integer andobject means as it is.

Connection ::= <<Request>>*Request ::= Message | Forward | PackedForward | nilMessage ::= [ Tag, Time, Record, Option? ]Forward ::= [ Tag, MultiEventStream, Option? ]MultiEventStream ::= [ Event* ]PackedForward ::= [ Tag, MessagePackEventStream, Option? ]MessagePackEventStream ::= <<Event>>*Event ::= [ Time, Record ]Tag ::= stringTime ::= integer | EventTimeRecord ::= objectOption ::= object