Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up

Clojure(Script) client for Declarative Dataflow.

License

NotificationsYou must be signed in to change notification settings

sixthnormal/clj-3df

Repository files navigation

Clojars Project

This is alpha-quality software, not feature-complete, and not yetready for use in production services.

3DF is best thought of as a pub/sub system in which subscriptions canbe arbitrary Datalog expressions. Subscribers register queries withthe broker, and data sources (such as Kafka, Datomic, or any othersource-of-truth) publish new data to it. All subscriber queriesaffected by incoming data will be notified with a diff, describing howtheir results have changed. The Datalog implementation is modeledafterDatomic's querylanguage and aims tosupport the same set of features.

3DF does this efficiently, thanks to being built on top ofdifferentialdataflows. Inparticular, Differential Dataflow will only compute changes, ratherthan execute a computation from scratch.

This repository contains the Clojure client for 3DF. The broker iswritten in Rust and can be found in theDeclarative DifferentialDataflows repository.

How it works

The 3DF client compiles Datalog expressions into an intermediaterepresentation that can be synthesised into a differential dataflow onthe server. This dataflow is then registered and executed across anynumber of workers. Whenever query results change due to new dataentering the system, the server will push the neccessary changes via aWebSocket connection.

For example, consider a subscriber created the following subscription:

(exec! conn  (query db"user inbox"     '[:find ?msg ?content:where       [?msg:msg/recipient"me@nikolasgoebel.com"]      [?msg:msg/content ?content]]))

and a new message arrives in the system.

[{:msg/receipient"me@nikolasgoebel.com":msg/content"Hello!"}]

Then the server will push the following results to the subscriber:

[[[<msg-id>"Hello!"]+1]]

If at some later point in time, this message was retracted

[[:db/retractEntity <msg-id>]]

the server would again notify the subscriber, this time indicating theretraction:

[[[<msg-id>"Hello!"]-1]]

This guarantees, that subscribers maintaining any form of functionallyderived information will always have a consistent view of the data.

Query Language Features

  • Implicit joins and unions,and /or operators
  • Stratified negation
  • Parameterized queries
  • Rules, self-referential / mutually recursive rules
  • Aggregation (min, max, count, etc...)
  • Grouping via:with
  • Basic predicates (<=, <, >, >=, =, not=)
  • As-of queries
  • More find specifications (e.g. collection, scalar)
  • Pull queries
  • Queries across many heterogeneous data sources

Please also have a look at the open issues to get a sense for whatwe're working on.

Non-Features

3DF is neither concerned with durability nor with consistency in theACID sense. It is intended to be used in combination with aconsistent, durable source-of-truth such asDatomic orKafka.

Consequently, 3DF will accept whatever tuples it is supplied with. Forexample, whereas in Datomic two subsequent transactions on an emptydatabase

(d/transact conn [[:db/add123:user/balance1000]])...(d/transact conn [[:db/add123:user/balance1500]])

would result in the following sets of datoms being added into thedatabase:

[[123:user/balance1000 <tx1>true]]...[[123:user/balance1000 <tx2>false] [123:user/balance1500 <tx2>true]]

3DF will by itself not take any previous information into account ontransactions. Again, 3DF is intended to be fed data from a system likeDatomic, which would ensure that transactions produce consistenttuples.

License

Copyright © 2018 Nikolas Göbel

Licensed under Eclipse Public License (seeLICENSE).

Releases

No releases published

Packages

No packages published

Languages


[8]ページ先頭

©2009-2025 Movatter.jp