- Notifications
You must be signed in to change notification settings - Fork6
VTL Engine and Editor
License
EUPL-1.2, Unknown licenses found
Licenses found
vpinna80/VTL
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
An engine and editor for theValidation and Transformation Language,written in Java, Javascript and R.
Usage info, documentation and examples are available at theproject wiki.
All contributions are welcome!
Please take a moment to read theGuideline.
A complete description of the project modules and the build process will be available soon.
To build this project, you will need:
- A Windows or Linux machine;
Building on a MacOS machine should be possible by now butit is untested.
- JDK >= 11;
- Apache Maven 3.9.4;
- GNU R >= 4.3.2 for building RVTL package;
- Configured internet connection (to downloadnode.js).
To build the project, launch the command:
mvn [-P with-r,with-sdmx,with-spark,with-jupyter,with-cli] clean package
Each artifact will be generated inside thetarget
folder of each module.The optional maven profiles allow you to build any of the provided VTL bundles forthe different front-ends capable of communicating with the VTL engine.
If you want to build the editor and the R package along with the engine, activate thewith-r
maven profile. Before you do that, make sure to installrJava
package in yourR installation. Then you may need to configure your internet connection in Maven settingsin order to download Node.js and npm packages. You will also need to installREngine.jar
into your local maven repository. It can be done with the following command:
Un*x:mvn install:install-file -DgroupId=org.rosuda.JRI -DartifactId=Rengine -Dversion=$R_VERSION -Dfile=$R_LIBRARY/java/jri/REngine.jar -Dpackaging=jarWindowsmvn install:install-file -DgroupId=org.rosuda.JRI -DartifactId=Rengine -Dversion=%R_VERSION% -Dfile=%R_LIBRARY%/java/jri/REngine.jar -Dpackaging=jar
Please set the environment variablesR_VERSION
andR_LIBRARY
accordinglyto your environment settings, or directly replace them in the command above.The R package, ready for installation in R (with install.packages),will be located there. Moreover, each bundle created while this profile isactive will contain support for the R environment.
VTL E&E is fully integrated with sdmx.io Metadata Registry. In order to enable support inbuilt bundles for SDMX interactions, such as using DSD and datasets, activate thewith-sdmx
maven profile. To build you also need to have previously built and installed the metadataregistry maven artifacts, available from sdmx.io web site.
If you want to build with support for the Spark evironment, activate thewith-spark
maven profile. Each bundle listed below created while this profile isactive will contain the Spark environment (Note that the Spark libraries are notincluded in the bundle and must be made available at runtime in your CLASSPATH).
If you want to build the Jupyter notebook kernel along with the engine, also activate thewith-jupyter
maven profile. The java-native jupyter kernel will be packaged as anexecutable jar that can be installed into your jupyter with the commandjava -jar vtl-jupyter-x.x.x-complete.jar
. Please make sure that your Python environmentis able to launch java, and that java version requirement is satisfied.
If you want to build the command line interface to VTL along with the engine, activate thewith-cli
maven profile. An executable JAR file ready to be deployed in your platformwill be packaged during thepackage
maven lifecycle phase in your build.
The Project is an ongoing development effort at Bank of Italy, with thepurpose of demonstrating the feasibility and usefulness of VTL in a realscenario, such as translating validation rules exchanged by entitiesoperating in the ESCB. Thus,
some VTL operators and statements offer only limited functionality,and some have to be implemented yet.
The following table shows the current implementation status:
operator | scalar | dataset-level | component-level |
---|---|---|---|
define datapoint ruleset | ✘ | ||
define hierarchical ruleset | ✔ | ||
define operator | ✔ | ✔ | ✔ |
call defined operator | ✔ | ✔ | ✔ |
persistent assignment | ✔ | ✔ | ✔ |
membership (#) | ✔ | ||
eval | ✘ | ✘ | ✘ |
cast | ✔ | ✔ | |
inner_join | ✔ | ||
left_join | ✔ | ||
full_join | ✘ | ||
cross_join | ✔ | ||
concat (||) | ✔ | ✔ | ✔ |
l-r-trim | ✔ | ✔ | ✔ |
upper,lower | ✔ | ✔ | ✔ |
substr | ✔ | ✔ | ✔ |
replace | ✔ | ✔ | ✔ |
instr | ✔ | ✔ | ✔ |
length | ✔ | ✔ | ✔ |
arithmetic | ✔ | ✔ | ✔ |
mod | ✔ | ✔ | ✔ |
round | ✔ | ✔ | ✔ |
trunc | ✔ | ✔ | ✔ |
ceil | ✔ | ✔ | ✔ |
floor | ✔ | ✔ | ✔ |
abs | ✔ | ✔ | ✔ |
exp | ✔ | ✔ | ✔ |
ln | ✔ | ✔ | ✔ |
power | ✔ | ✔ | ✔ |
log | ✔ | ✔ | ✔ |
sqrt | ✔ | ✔ | ✔ |
comparisons (=, >, <, <>) | ✔ | ✔ | ✔ |
between | ✔ | ✔ | ✔ |
in/not_in | ✘ | ✔ | |
match_characters | ✔ | ✘ | ✔ |
isnull | ✔ | ✔ | ✔ |
exists_in | ✔ | ||
boolean operators | ✔ | ✔ | ✔ |
period_indicator | ✔ | ✔ | ✔ |
fill_time_series | ✔ | ||
flow_to_stock | ✔ | ||
stock_to_flow | ✔ | ||
timeshift | ✔ | ||
timeagg | ✔ | ✔ | ✔ |
current_date | ✔ | ✔ | |
union | ✔ |
About
VTL Engine and Editor