Repository files navigation

A quotation-based Scala DSL for scalable data analysis.

Our goal is to improve developer productivity by hiding parallelism aspects behind a high-level,declarative API which maximises reuse of native Scala syntax and constructs.

Emma supports state-of-the-art dataflow engines such asApache Flink andApache Spark as backend co-processors.

DSLs for scalable data analysis are embedded through types.In contrast, Emma isbased on quotations (similar toQuill).This approach has two benefits.

First, it allows to reuse Scala-native, declarative constructs in the DSL.Quoted Scala syntax such asfor-comprehensions,case-classes, andpattern matchingare thereby lifted to an intermediate representation calledEmma Core.

Second, it allows toanalyze and optimize Emma Core terms holistically.Subterms of typeDataBag[A] are thereby transformed and off-loaded to a parallel dataflow engine such as Apache Flink or Apache Spark.

Theemma-examples module contains examples from various fields.

• Graph Analysis
  • Connected Components
  • Triangle Enumeration
  • Transitive Closure
• Supervised Learning
  • Naive Bayses Classification
• Unsupervised Learning
  • k-Means Clustering
• Text Processing
  • Word Count

Checkemma-language.org for further information.

• JDK 7+ (preferably JDK 8)
• Maven 3

Run

mvn clean package -DskipTests

to build Emma without running any tests.

For more advanced build options including integration tests for the target runtimes please see the"Building Emma" section in the Wiki.