Repository files navigation

A Java library designed to build a very fast queuing system that guaranteesat least once and at most once delivery. To achieve this objective, RedisRQS uses Redis as the data store and supports a 1:1 Publisher to Subscriber relationship. When a message arrives, RedisRQS guarantees that only one Subscriber will ever receive a message for processing regardless of the number of Subscribers listening for the same topic in the queue. This should allow a system to scale out the number of processors based on work load.

The way this system handles queuing and dequeuing messages is via Lua scripts. This provides an additional guarantee that each method call to RedisRQS will be a single network call to Redis. Furthermore, RedisRQS also guarantees that messages will never be lost due to a Subscriber unable to finish processing or crashing while processing a message. The queue sweeper will periodically put messages back in to the queue for processing when it runs. You can change the frequency of the sweep in the options.

It is important to note that at this time, there is no support for poison messages. If a message is incomplete or will always crash a Subscriber, the message (poison) will be put back in the pending queue for processing by another Subscriber. This is on the roadmap and will be made available at a future date. As all things open source, your contributions are welcome.

When using this library, three lists will be created:

• Pending: namedredisrqs:pending
• Working: namedredisrqs:working
• Values: namedredisrqs:values

Note: There is aNodeJS version of this library as well.

Add the library as a dependency in your pom.xml file:

<dependency>  <groupId>com.amirkhawaja</groupId>  <artifactId>redisrqs-java</artifactId>  <version>1.0.2</version></dependency>

At this time, this library is not made available in Maven Central. You can build this locally by issuing the command:

$> make package

Alternatively, you can use the following Maven command:

$> mvn clean compile package

Either of these two steps will create ajar file you can include in your project.

The following is an example of how you can use this library:

privatefinalstaticStringCONNECTION_URI ="redis://localhost:6379/1";privatestaticvoidmain(String[]args) {finalHashMap<String,String>options =newHashMap<>();options.put("sweepInterval","60000");// Sweep the queue every 1-minutefinalcom.amirkhawaja.RedisQueuequeue =newcom.amirkhawaja.RedisQueue(CONNECTION_URI,options);finalStringdata ="This is a test";finalStringtopic ="Test Topic";// Queue the messagequeue.enqueue(topic,data);// Dequeue the messagefinalcom.amirkhawaja.models.Messagemessage =queue.dequeue();// Do some work...then remove the message from the queuequeue.release(message.getUuid());// Dispose the queue when you are donequeue.close();}

When a message is queued it is put in this list. This list will only containthe UUID of the message. The actual message value lives in the Values list.

This is where the currently processed message(s) will be stored.

This is where the actual message will live. Each message is identifiedby a UUID. When a message is de-queued, only the UUID is moved from thePending list to the Working list.

• 1.0.2: Published to Maven Central.
• 1.0.0: Initial public release.