Repository files navigation

A distributed generator to create unique IDs with ease; inspired byTwitter's Snowflake. Blog post about this project can be foundhere.

Apart from being a crystal of snow, a snowflake is a form of unique identifier which is being used in distributed computing. It has specific parts and is 64 bits long in binary. I simply named my type of snowflake, aSpaceflake, as it does not compose of the same parts of a Twitter Snowflake and is being used forProject Absence and other projects of myself.

A Spaceflake is structured like the following:

A Spaceflake Network is a very basic concept where you have multipleindependent nodes that themselves consist of multiple workers. These workers are the ones that can generate a Spaceflake.

Ideally a Spaceflake Network represents your entire application, or company. Each node represents a single server or application within the company, and each worker represents a single process which can generate a Spaceflake for a specific purpose. This way you can easily identify where a Spaceflake was generated by looking at its node ID and worker ID.

In the end you are free to use them as you wish, just make sure you use these nodes and workers to be able to identify the Spaceflake.

An example network is structured like the followingWe can considerNode 1 as being the API/backend of your application. TheWorker (ID: 1) would be responsible for generating Spaceflakes for user IDs. TheWorker (ID: 2) would be responsible for generating Spaceflakes for blog post IDs.

TheNode 2 might be responsible for the logs of your components, and the log ID generated would be generated by theWorker (ID: 1) from that node.

• A Spaceflake network can hold up to31 nodes and31 workers per node. So you can have up to961 workers in total in a single network that will generate Spaceflakes.
• Asingle worker can generate up to4095 Spaceflakes per millisecond.
• Asingle node with31 workers can generate up to126'945 Spaceflakes per millisecond.
• Asingle network with31 nodes and31 workers per node can generate up to3'935'295 Spaceflakes per millisecond.

A very basic example on using the library is by using the generatorwithout nodes and worker objects, though this is not recommended and using nodes and workers is better.

package mainimport ("fmt""github.com/kkrypt0nn/spaceflake")funcmain() {node:=spaceflake.NewNode(1)worker:=node.NewWorker()sf,err:=worker.GenerateSpaceflake()iferr!=nil {panic(err)}fmt.Println(sf.Decompose())// map[id:<Spaceflake> nodeID:1 sequence:1 time:<timestamp> workerID:1]}

Some other examples:

• Bulk generation: Generate multiple Spaceflakes at once.
• Generation with nodes and workers: Generate a Spaceflake by creating a node and worker object.
• Generation with settings: Generate a Spaceflakes with specific settings(Not recommended, consider using nodes and workers)

If you want to use this library for one of your projects, you can install it like any other Go library

go get github.com/kkrypt0nn/spaceflake

📜 TL;DR: If you use Spaceflakes in an API, return them as astring, not a number.

Since Spaceflakes are big numbers, it is most likely that if you use them for an API that returns a JSON you will need to return the Spaceflake as a string, otherwise you will lose some precision and it will alter the value of, most likely, the sequence of the Spaceflake. Example:

{"id":144328692659220480// ID actually generated in Go: 144328692659220481}

The difference between the two numbers is not that big in the example above, though it plays a big role. The difference is not always the same, so you can't subtract. JavaScript, for example, sees no difference between both of these numbers:

console.log(144328692659220480==144328692659220481)// true

You can get the Spaceflake as a string and convert to auint64 data type, when needed, in your Go code using the following:

spaceflakeID:="144328692659220481"// Will be the value returned by the APIid,_:=strconv.ParseUint(spaceflakeID,10,64)sequence:=spaceflake.ParseSequence(id)

📜 TL;DR: The sequence is not truly random, it is based on the time; and if you generate lots of Spaceflake in the same millisecond, there is a chance that two Spaceflakes will result to the same. Usingnodes and workers is highly recommended.

When generating lots of Spaceflakes in a really short time and without using a worker, there is a chance that the same ID is generated twice. Consider making your program sleep for 1 millisecond or test around between the generations, example:

funcGenerateLotsOfSpaceflakes() {spaceflakes:=map[uint64]*Spaceflake{}settings:=NewGeneratorSettings()fori:=0;i<1000;i++ {sf,err:=Generate(settings)iferr!=nil {t.Error(err)}ifspaceflakes[sf.ID()]!=nil {panic(err)}spaceflakes[sf.ID()]=sf// When using random there is a chance that the sequence will be twice the same due to Go's speed, hence using a worker is better. We wait a millisecond to make sure it's different.time.Sleep(1*time.Millisecond)}}

In that case it is recommended to use the workers, as they do not use a random value as a sequence number, but an incrementing value. Another option would be to use thebulk generator to create lots ofunique Spaceflakes at once.

As a last resort you can replace the sequence with a better random number generator using the following:

settings.Sequence=...// Replace with your number generator

This library was made with 💜 by Krypton and is under theMIT license.