Repository files navigation

Non-blocking Redis client with focus on performance and robustness.

This fork is the official continuation ofwooga/eredis.It includes several improvements, such asTLS support, Sentinel support and TCP error handling corrections.SeeCHANGELOG.md for details.

Note: This client connects to a standalone Redis node, optionally using Sentinel.ForRedis Cluster, you neederedis_cluster orecredis.

Supported Redis features:

• Any command, througheredis:q/2,3
• Transactions
• Pipelining
• Authentication & multiple DBs
• Pubsub
• TLS
• Sentinel support

Generated API documentation:doc/eredis.md

Published documentation can also be found onhexdocs.

If you have Redis running on localhost with default settings, like:

docker run --rm --net=host redis:latest

you may copy and paste the following into a shell to try out Eredis:

git clone https://github.com/Nordix/eredis.gitcd eredisrebar3 shell{ok, C} = eredis:start_link().{ok, <<"OK">>} = eredis:q(C, ["SET", "foo", "bar"]).{ok, <<"bar">>} = eredis:q(C, ["GET", "foo"]).

To connect to a Redis instance listening on a Unix domain socket:

{ok,C1}=eredis:start_link({local,"/var/run/redis.sock"},0).

To connect to a Redis instance using TLS:

Options= [{tls, [{cacertfile,"ca.crt"},                  {certfile,"client.crt"},                  {keyfile,"client.key"}]}],{ok,C2}=eredis:start_link("127.0.0.1",?TLS_PORT,Options),

To connect to a Redis sentinel cluster:

SentinelOptions= [{master_group,mymaster},                   {endpoints, [{"127.0.0.1",26379}]}],{ok,C1}=eredis:start_link([{sentinel,SentinelOptions}]).

MSET and MGET:

KeyValuePairs= ["key1","value1","key2","value2","key3","value3"].{ok, <<"OK">>}=eredis:q(C, ["MSET" |KeyValuePairs]).{ok,Values}=eredis:q(C, ["MGET" | ["key1","key2","key3"]]).

HASH

HashObj= ["id","objectId","message","message","receiver","receiver","status","read"].{ok, <<"OK">>}=eredis:q(C, ["HMSET","key" |HashObj]).{ok,Values}=eredis:q(C, ["HGETALL","key"]).

LIST

eredis:q(C, ["LPUSH","keylist","value"]).eredis:q(C, ["RPUSH","keylist","value"]).eredis:q(C, ["LRANGE","keylist",0,-1]).

Transactions:

{ok, <<"OK">>}=eredis:q(C, ["MULTI"]).{ok, <<"QUEUED">>}=eredis:q(C, ["SET","foo","bar"]).{ok, <<"QUEUED">>}=eredis:q(C, ["SET","bar","baz"]).{ok, [<<"OK">>, <<"OK">>]}=eredis:q(C, ["EXEC"]).

Pipelining:

P1= [["SET",a,"1"],      ["LPUSH",b,"3"],      ["LPUSH",b,"2"]].[{ok, <<"OK">>}, {ok, <<"1">>}, {ok, <<"2">>}]=eredis:qp(C,P1).

Pubsub:

1>eredis_sub:sub_example().received {subscribed,<<"foo">>,<0.34.0>}{<0.34.0>,<0.37.0>}2>eredis_sub:pub_example().received {message,<<"foo">>,<<"bar">>,<0.34.0>}ok3>

Pattern Subscribe:

1>eredis_sub:psub_example().received {subscribed,<<"foo*">>,<0.33.0>}{<0.33.0>,<0.36.0>}2>eredis_sub:ppub_example().received {pmessage,<<"foo*">>,<<"foo123">>,<<"bar">>,<0.33.0>}ok3>

Eredis has one main function to interact with redis, which iseredis:q(Client::pid(), Command::iolist()). The response will eitherbe{ok, Value::binary() | [binary()]} or{error, Message::binary()}. The value is always the exact value returned byRedis, without any type conversion. If Redis returns a list of values,this list is returned in the exact same order without any typeconversion.

To send multiple requests to redis in a batch, aka. pipeliningrequests, you may useeredis:qp(Client::pid(), [Command::iolist()]). This function returns{ok, [Value::binary()]}where the values are the redis responses in the same order as thecommands you provided.

To start the client, usestart_link/1 or one of its variants.start_link/1takes the following options (proplist):

• host: DNS name or IP address as string; or unix domain socket as{local, Path} (available in OTP 19+)
• port: integer, default is 6379
• database: integer or 0 for default database, default: 0
• username: string, default: no username
• password: string, default: no password
• reconnect_sleep: integer of milliseconds to sleep between reconnect attempts, default: 100
• connect_timeout: timeout value in milliseconds to use in the connect, default: 5000
• socket_options: proplist ofgen_tcpoptions used when connecting the socket, default is?SOCKET_OPTS.It's possible to specify{active, N} or{active, true}.Activeonce andfalse are not supported. The default is N = 10.
• tls: enable TLS by providing a list ofoptions used when establishing the TLSconnection, default is off.It's possible to specify{active, N} or{active, true} in TLS options.Activeonce andfalse are not supported. The default is N = 10.Note that{active, N} with TLS requires OTP 21.3 or later.

Commands are pipelined automatically so multiple processes can share the sameEredis connection instance. Althoughq/2,3 andqp/2,3 are blocking until theresponse is returned, Eredis is not blocked.

  Process A          Process B          Eredis        TCP/TLS socket     |                  |                  |          (Redis server)     | q(Pid, Command1) |                  |                 |     |------------------------------------>|---------------->|     |                  | q(Pid, Command2) |                 |     |                  |----------------->|---------------->|     |                  |                  |                 |    ...                ...                ...               ...     |                  |                  |                 |     |                  |                  |      Response 1 |     |<------------------------------------|<----------------|     |                  |                  |      Response 2 |     |                  |<-----------------|<----------------|

When Eredis for some reason loses the connection to Redis, Erediswill keep trying to reconnect until a connection is successfullyestablished, which includes theAUTH andSELECT calls. The sleeptime between attempts to reconnect can be set in theeredis:start_link/1 call.

As long as the connection is down, Eredis will respond to any requestimmediately with{error, no_connection} without actually trying toconnect. This serves as a kind of circuit breaker and prevents astampede of clients just waiting for a failed connection attempt orgen_server:call timeout.

Note: If Eredis is starting up and cannot connect, it will failimmediately with{connection_error, Reason}.

Thanks to Dave Peticolas (jdavisp3), eredis supportspubsub.eredis_sub offers a separate client that will forwardchannel messages from Redis to an Erlang process in an "active-once"pattern similar to gen_tcp sockets. After every message received, thecontrolling process must acknowledge receipt usingeredis_sub:ack_message/1.

If the controlling process does not process messages fast enough,eredis will queue the messages up to a certain queue size controlledby configuration. When the max size is reached, eredis will eitherdrop messages or crash, also based on configuration.

Subscriptions are managed usingeredis_sub:subscribe/2 anderedis_sub:unsubscribe/2. When Redis acknowledges the change insubscription, a message is sent to the controlling process for eachchannel. Then, eredis sends a message on the form{subscribed, Channel, ClientPid} to the controlling process, which must be acked usingeredis_sub:ack_message/1.

eredis also supports Pattern Subscribe usingeredis_sub:psubscribe/2anderedis_sub:punsubscribe/2. As with normal subscriptions, a messageis sent to the controlling process for each channel and eredis sends amessage on the form{subscribed, Channel, ClientPid} to thecontrolling process, which must be acked usingeredis_sub:ack_message/1.

The controlling process is also notified in case of reconnection attempts orfailures. Seetest/eredis_pubsub_SUITE.erl for examples.

Starting with version 1.5, eredis automatically resubscribes after reconnect.Then, the controlling process will receive{subscribed, Channel, ClientPid}messages again for every channel and pattern subscribed to. These must also beacked.

Here is a list of all the messages that are sent to the controlling process.Some messages don't need to be acked, but it does not harm doing so.

• {eredis_connected, Pid} when the socket to Redis is establishedand authenticated. Doesn't need to be acked.

• {eredis_disconnected, Pid} when the connection to Redis has been lost.Doesn't need to be acked.

• {eredis_reconnect_attempt, Pid} at every reconnect attempt, when theconnection to Redis has been lost. Doesn't need to be acked.

• {eredis_reconnect_failed, Pid, {error, {connection_error, Reason}}} afterevery failed reconnect attempt. Doesn't need to be acked.

• {message, Channel, Message, Pid} for every incoming message on subscribedchannels. Needs to be acked usingeredis_sub:ack_message/1.

• {pmessage, Pattern, Channel, Message, Pid} for every incoming message onchannels subscribed to by pattern, using usingeredis_sub:psubscribe/2.Needs to be acked usingeredis_sub:ack_message/1.

• {subscribed, Channel, Pid} when a subscription has been confirmed by Redis.Channel is either a channel (subscribe) or a pattern (psubscribe). Needs to beacked usingeredis_sub:ack_message/1.

• {unsubscribed, Channel, Pid} when a subscription has been removed fromRedis. Channel is either a channel (unsubscribe) or a pattern (punsubscribe).Needs to be acked usingeredis_sub:ack_message/1.

Eredis also implements the AUTH and SELECT calls for you. When theclient is started with something else than default values for passwordand database, it will issue theAUTH andSELECT commandsappropriately, even when reconnecting after a timeout.

Usinglasp-bench you maybenchmark Eredis on your own hardware using the provided config anddriver. Seepriv/basho_bench_driver_eredis.config andsrc/basho_bench_driver_eredis.erl.

Testcase summary from our daily runs:

• eredis
• eredis_pipeline

Theeredis-benchmark repo runsa daily job that produces above graphs. It also contains the scriptrun-tests.sh that might help you with the needed steps when setting up thebenchmark testing on your own.

Eredis uses the same queueing mechanism as Erldis.eredis:q/2 usesgen_server:call/2 to do a blocking call to the clientgen_server. The client will immediately send the request to Redis, addthe caller to the queue and reply withnoreply. This frees thegen_server up to accept new requests and parse responses as they comeon the socket.

When data is received on the socket, we calleredis_parser:parse/2until it returns a value, we then usegen_server:reply/2 to reply tothe first process waiting in the queue.

This queueing mechanism works because Redis guarantees that theresponse will be in the same order as the requests.

The socket is set to{active, N} with N = 10 by default. This is configurableby including{active, N} in socket options or TLS options (if TLS is used). Nmust be an integer ortrue. Activeonce andfalse are not supported.

The response parser is capable of parsing partial data and continuing when moredata arrives on the socket.

EUnit tests currently require a locally running instance of Redis.

rebar3 eunit

Xref, dialyzer and elvis should result in no errors.

rebar3 xrefrebar3 dialyzerelvis rock

When the parser is accumulating data, a new binary is generated forevery call toparse/2. This might create binaries that will bereference counted. This could be improved by replacing it with aniolist.

When parsing bulk replies, the parser knows the size of the bulk. If thebulk is big and would come in many chunks, this could be improved byhaving the client explicitly usegen_tcp:recv/2 to fetch the entirebulk at once.

This is a fork of the original Eredis. Eredis was created by Knut Nesheim, withinspiration from the earlier Erldis.

Although this project is almost a complete rewrite, many patterns arethe same as you find in Erldis, most notably the queueing of requests.

create_multibulk/1 andto_binary/1 were taken verbatim from Erldis.