Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Implements distributed sequence#5583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
emmanuel-keller merged 42 commits intomainfromemmanuel/unique_distributed_sequence
Apr 29, 2025

Conversation

emmanuel-keller
Copy link
Collaborator

@emmanuel-kelleremmanuel-keller commentedFeb 24, 2025
edited
Loading

Thank you for submitting this pull request. We really appreciate you spending the time to work on SurrealDB. 🚀 🎉

What is the motivation?

The implementation of distributed sequences in SurrealDB meets the critical need for generating reliable, monotonically increasing numeric identifiers across a cluster of nodes without excessive coordination overhead, allowing applications to utilize sequences for unique IDs, counters, and ordered operations with optimal performance in distributed environments.

What does this change do?

Distributed Sequence Generator for SurrealDB.

This code provides reliable, monotonically increasing numeric sequences in both single‑node and clustered SurrealDB deployments (multiple compute nodes backed by TiKV). It uses a batch‑allocation strategy to minimise coordination while guaranteeing global uniqueness.

Key features:

  • Batch allocation: Nodes request ranges of sequence values at once, reducing network chatter and coordination overhead.
  • Node ownership tagging: Every batch is tagged with the requesting node’s UUID to prevent overlap between nodes.
  • Durable Persistence: Sequence metadata is stored in the underlying key-value store to survive restarts and network partitions.
  • Concurrent, thread‑safe access: A DashMap caches active sequences, allowing lock‑free reads on the hot path.
  • Exponential back‑off with full jitter: When a batch‑allocation attempt fails, the node retries with an exponential delay that includes full jitter to avoid thundering‑herd effects across the cluster.
  • Automatic cleanup: Listens for namespace and database‑removal events and purges the corresponding sequence state.

How it works

The sequence implementation avoids contention by having each node reserve a range of sequence values, allowing it to serve multiple requests locally without requiring distributed coordination for every request. When a node exhausts its allocated range, it acquires a new batch from the distributed store.

To create a sequence:

DEFINE SEQUENCE mySeq;DEFINE SEQUENCE mySeq2 BATCH1000 START100 TIMEOUT 5s;

It is possible to modify the timeout of the sequence:

ALTERSEQUENCE mySeq TIMEOUT 5s;

Once a sequence is created, it is possible to retrieve the consecutive values:

RETURN sequence::nextval('mySeq');RETURN sequence::nextval('mySeq');

Will return:

01

To remove a sequence:

REMOVE SEQUENCE mySeq;

INFO FOR DB returns now defined sequences:

INFO FOR DB
{accesses:{},analyzers:{},apis:{},buckets:{},configs:{},functions:{},models:{},params:{},sequences:{seq1:'DEFINE SEQUENCE seq1 BATCH 1000 START 0',seq2:'DEFINE SEQUENCE seq2 BATCH 250 START -25',seq3:'DEFINE SEQUENCE seq3 BATCH 1000 START 0 TIMEOUT 5s'},tables:{},users:{}}

What is your testing strategy?

Github action.
Tests have been added to language tests and unit tests.

Is this related to any issues?

  • No related issues

Does this change need documentation?

Have you read the Contributing Guidelines?

@emmanuel-kelleremmanuel-keller changed the titleImplements DefineSequenceStatementImplements concurrent sequenceFeb 24, 2025
@emmanuel-kelleremmanuel-keller changed the titleImplements concurrent sequenceImplements distributed sequenceFeb 25, 2025
Fixes language tests
@emmanuel-kelleremmanuel-keller marked this pull request as ready for reviewApril 16, 2025 18:43
kearfy
kearfy previously approved these changesApr 28, 2025
@emmanuel-kelleremmanuel-keller added this pull request to themerge queueApr 29, 2025
Merged via the queue intomain with commitff5061bApr 29, 2025
33 checks passed
@emmanuel-kelleremmanuel-keller deleted the emmanuel/unique_distributed_sequence branchApril 29, 2025 08:38
@rushmoremrushmorem added this to thev3.0.0-alpha.2 milestoneApr 29, 2025
emmanuel-keller added a commit that referenced this pull requestMay 2, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@gssergegssergegsserge approved these changes

@tobiemhtobiemhAwaiting requested review from tobiemhtobiemh is a code owner

@kearfykearfyAwaiting requested review from kearfy

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v3.0.0-alpha.2
Development

Successfully merging this pull request may close these issues.

4 participants
@emmanuel-keller@gsserge@kearfy@rushmorem

[8]ページ先頭

©2009-2025 Movatter.jp