Repository files navigation

• Source:https://github.com/youzan/nsq
• Issues:https://github.com/youzan/nsq/issues
• Mailing List:nsq-users@googlegroups.com
• IRC: #nsq on freenode
• Docs:http://nsq.io
• Twitter:@nsqio,@absolute8511

NSQ is a realtime distributed messaging platform designed to operate at scale, handlingbillions of messages per day.

It promotesdistributed anddecentralized topologies without single points of failure,enabling fault tolerance and high availability coupled with a reliable message deliveryguarantee. Seefeatures & guarantees.

Operationally,NSQ is easy to configure and deploy (all parameters are specified on the commandline and compiled binaries have no runtime dependencies). For maximum flexibility, it is agnostic todata format (messages can be JSON, MsgPack, Protocol Buffers, or anything else). Official Go andPython libraries are available out of the box (as well as many otherclientlibraries) and, if you're interested in building your own, there's aprotocolspec.

We publishbinary releases for linux and darwin.

NOTE: master is ourdevelopment branch and may not be stable at all times.

This fork has been moved tohttps://github.com/youzan/nsq

This fork add many features missing in the original and has been used in Youzan.com.Features:

• Replication
• HA
• Auto balance and migrate
• Partition
• Delivery in Order
• Consume history data
• Trace the message life-cycle
• Performance improvement

This fork need the etcd cluster, so to deploy we need set the configure for the nsqd and nsqlookupd.For details please refer to the sample configure files under contrib directory.

Other deploy can be the same as the Official.

有赞NSQ用户指南

有赞NSQ部署运维指南

JavaSDK使用指南

重构概览

重构实现细节

客户端SDK实现

扩展消息设计

Most client SDK can be used to communication with the new server without using the new features and for new features in thisfork the new client SDK should be used.

Golang:https://github.com/youzan/go-nsq

Java:https://github.com/youzan/nsqJavaSDK

PHP:https://github.com/youzan/php-nsq-client

In order to manager the topics by nsqlookupd, we removed the create api on the nsqd. So in order to create a new topic,you need create the topic in the nsqadmin orsend the create api to the nsqlookupd as below:

curl -X POST "http://127.0.0.1:4161/topic/create?topic=xxx&partition_num=2&replicator=3&syncdisk=1000"Explain:partition_num=2 : topic partition numberreplicator=3: topic replica numbersyncdisk=1000: fsync disk every syncdisk messagesfor ordered topic the query param "orderedmulti=true" is needed.

The new server introduced the partitions but in order to keep compatible with the old clients, each node can holdonly one partition of the topic. So the partition_num*replicator should not be great than the nsqd nodes.For the ordered topic, we can create as much as we need partitions. (replicator <= number of nsqd node)Also, the first channel for this topic should be created after the topic is created, otherwise this topic is notallowed to be written.

This is useful to debug on the fly.

nsqd: curl -X POST "http://127.0.0.1:4151/loglevel/set?loglevel=3"nsqlookupd: curl -X POST "http://127.0.0.1:4161/loglevel/set?loglevel=3"

The log is much more detail when the log level increased.

curl -X POST -d "timestamp:xxxxxxxxx" "http://127.0.0.1:4151/channel/setoffset?topic=xxx&partition=xx&channel=xxx"Post body explain:timestamp:xxxx (the seconds since 1970-1-1)virtual_queue:xxx (the queue offset position from the begin)msgcount:xxx (the message count from the begin)

For multi nsqd nodes, each node for each partition should be done using the api above.

please see the other documents under the doc

Help us keep NSQ open and inclusive. Please read and follow ourCode of Conduct.

NSQ was designed and developed by Matt Reiferson (@imsnakes) and Jehiah Czebotar(@jehiah) but wouldn't have been possible without the support ofbitly and all ourcontributors.

Logo created by Wolasi Konu (@kisalow).