Repository files navigation

Metis is an open scientific framework, materials data organizer, and collaborative online platform for the nanotechnology research. It was designed for the offline physical and online virtual autonomous laboratories dealing with the materials science. Metis is an AI-ready solution, aiming to bring the recent advances of computer science into a rather conservative area of new materials development and quality control. Metis currently focuses on the X-ray powder diffraction and atomistic simulations. Its development was started in 2021 in BASF (Ludwigshafen am Rhein, Germany) by Bernd Hinrichsen and Evgeny Blokhin.

This is the third part of the whole Metis infra:GUI ⇄BFF ⇄backend.

Metis backend presents minimalistic utility toolset in materials informatics and Flask-based CRUD-server for the nano-scale crystalline data, XRPD measurements, and cloud simulations.

The basic requirements arePython,Numpy, andPostgreSQL. Note that the Numpy depends on the low-level Fortran numeric system libraries, which might be absent in your system. The Python dependencyxylib-py requires C++ Boost stack to compile, install it on Debian withapt-get install libboost-all-dev.

A scientific cloud scheduleryascheduler is required for a separate cloud orchestration. An optional requirement isAiiDA, which can be linked to the cloud scheduler. The AiiDA is a Python framework for the complex scientific workflows, requiring PostgreSQL database andRabbitMQ message broker.

Thus,metis-backend consists of the 3 independent parts, each using PostgreSQL:

• metis-backend Python server per se
• aiida_core workflow engine
• yascheduler cloud scheduler

Optionally, a frontend server isNginx (conf/nginx.conf goes to/etc/nginx), and all these guys are controlled by theSupervisor daemon (conf/supervisord.conf goes to/etc/supervisor).

Refer toconf/install.sh for installation of Nginx, PostgreSQL, RabbitMQ, and Supervisor, as well as configuring them. Runconf/install.sh and then modify global optionsconf/env.ini. Alternatively, feel free to install each component on your own (or all together in a container, see below).

The AiiDA can be installed and configured separately. First, a possibility forssh-ing into a localhost should be ensured:

ssh-keygen -t rsacat~/.ssh/id_rsa.pub>>~/.ssh/authorized_keysssh$USER@localhost

Then ayascheduler plugin is configured (conf/yascheduler.conf goes to/etc/yascheduler). After that runyainit to start ayascheduler service. Then, if you have already runconf/install.sh script, it should be enought to do:

pip install aiida_corebash conf/aiida_setup.sh

Then setup your virtual env, if needed, and install the Python requirements as simply aspip install -r requirements.txt.

Finally, apply the database schema:/data/pg/bin/psql -U postgres -d metis -f schema/schema.sql.

Runsupervisorctl status to see all the services in Supervisor.

One by one, all the parts are managed as follows:

• metis-backend server is started withindex.py
• yascheduler is started simply withyascheduler command
• AiiDA is managed withverdi, e.g.verdi process list orverdi node show
• AiiDA daemon is started separately withverdi daemon start
• PostgreSQL database(s) can be seen with/data/pg/bin/psql -U postgres -l
• RabbitMQ is controlled withrabbitmqctl status
• (Nginx can be added to Supervisor as well depending on your taste)

A scriptscripts/assign_phases.py should be run regularly to organize users' uploaded data, e.g. in cron job scheduler

*/5 * * * * /path/to/metis-backend/scripts/assign_phases.py

This is an experimental feature intended primarily for the development and testing.

It is assumed that you have themetis-backend,metis-bff, andmetis-guirepositories cloned on the same level. Also, you needdocker (orpodmanandpodman-compose) installed.

Now, you can rundocker compose up (podman-compose up) inmetis-backenddirectory. This should start all dependencies and services.

metis-gui should be available athttp://localhost:10000/

metis-bff should be available athttp://localhost:3000/

• CIF
• POSCAR
• Optimade JSON

• XY and XYE patterns (TSV-alike)
• Bruker's RAW (binary)
• Bruker's Topas CLI modeling
• Synchrotron HDF5 NeXus (binary)

Copyright 2021-2023 BASF SE

BSD 3-Clause