Repository files navigation

A data quality dashboard for library catalogues

This application provides a web interface for searching, browsing, anddisplayingQA Catalogue data fromlibrary catalogues for quality analysis and statistics of metadata.

Catalog information is made available from CSV files, JSON files, SQLitedatabase, images, and aSolr searchindex. The reports are made accessible using a Web browser and has beenimplemented by PHP and JavaScript.

• Installation
  • Download
  • Setup
  • Configuration
• Customization
  • Templates
  • Catalogue class
• Translation
• Maintainers
• Contributing
• License

In the following:

• $DATADIR denotes the base output directory of data analysis with QA Catalogue
• $APPDIR denotes the directory where QA Catalogue Web is installed
• $CATALOG denotes thecatalogue class (such as loc, bl, k10plus...)

Analyse your catalog using theQA Catalogue Backend)and the result will be saved in$DATADIR/$CATALOG and a Solr index.

Install this software into a web server with PHP enabled such as (Apache or Nginx withPHP-FPM).

Create a temporary directory and download the current version to an applicationdirectory served by your webserver (here we use/var/www/html/$APPDIR):

mkdir tmpcd tmpwget https://github.com/pkiraly/qa-catalogue-web/archive/master.zipunzip master.zipmv metadata-marc-web-master /var/www/html/$APPDIR

or clone the git repository:

git clone https://github.com/pkiraly/qa-catalogue-web.git /var/www/html/$APPDIRcd /var/www/html/$APPDIR# optionally checkout a tagged releasegit checkout v0.7.0

Requirements:

sudo apt install locales gettext php-sqlite3 php-yaml php-curl composersudo locale-gen en_GB.UTF-8sudo locale-gen de_DE.UTF-8sudo locale-gen pt_BR.UTF-8sudo locale-gen hu_HU.UTF-8

Change into the application directory:

cd /var/www/html/$APPDIR

Install PHP dependencies and create required cache directories and permissions:

composer install

The script will further tell you to change ownership of directories if theapplication is served aswwww-data but installed as another user.

If you also want to locally change the source code (seecontributing),allowwww-data to get the current state of the working directory:

sudo -u www-data git config --global --add safe.directory "$PWD"

A configuration file in INI format is required. Prepare configuration file:

echo"dir=$DATADIR"> configuration.cnf

Thecatalogue class should explicitly be specified:

echo"catalogue=$CATALOG">> configuration.cnf

Additional configuration parameters are listed below. Data type and version number when the parameter was introduced are given in parentheses:

• id (string) the machine name of the data directory. By default, it comes from the URL as the path of the application(qa-catalogue). With this parameter the administrator can overwrite the path. Note: this parameter was calleddbpreviously. For compatibility reason we will supportdb as well for some time.

The following parameters can be set either for all catalogues (parameter=value) or for each individual catalogue (parameter[id]=...).

• dir (string) the base output directory of data analysis with QA Catalogue
• catalogue (string) thecatalogue class given in lowercase.The valuecatalogue can be used for generic catalogue. Set to value ofid by default.
• default-tab (string) the tab which will be displayed when no tab is selected. Thiswill be the tab which will be opened by the root URL (the landing page). If no default-tab has been set,start will be used. The possible values are:start (default),data,completeness,issues,functions,classifications,authorities,serials,tt-completeness,shelf-ready-completeness,shacl,network,terms,pareto,history,timeline,settings,about,record-issues,histogram,functional-analysis-histogram,control-fields,download,collocations.
• display-... (bool) show or hide selected tabs, for instancedisplay-issues,display-shacl.By default all tabs are shown if approriate data is available. Onlyshacl andnetwork are disabled by default.
• indexName (string) name of the Solr index of a particular catalogue, if it is different fromthe name of the catalogue or the URL path.
• dirName (string) name of the data directory of a particular catalogue, if it is different fromthe name of the catalogue or the URL path.
• versions (bool) denotes if there are versions for a catalogue. Possible values: true (there areversions), false (there are no versions). Default:false.
• templates (string) directory with additional, custom Smarty templates forcustomization. Default:config.
• mainSolrEndpoint (string, v0.8.0) the URL of the main Solr endpoint. Default:http://localhost:8983/solr/.
• solrForScoresUrl (string, v0.8.0) the URL of the Solr core that isused for storing the results of validation. Usually this index is merged to the main core, so this property is neededon the special case when the validation is not merged. Default:http://localhost:8983/solr/.In multi-tenant mode you can specify it for a particular catalogue withmainSolrEndpoint[<id>]. (Its previousname wassolrEndpoint4ValidationResults.)
• showAdvancedSearchForm (bool, v.0.8.0) show or hide advanced search form. Default:false
• logHandler (string) where to log to. Allowed values arefile (log to a file specified by thelogFile parameters=, anderror_log (default, log to the webserver's error log - for Apache it is/var/log/apache2/error.log)).
• logFile (string, v0.8.0) a path of the log file iflogHandler is set tofile. Make sure the fileis writeable by the webserver. Default:logs/qa-catalogue.log.
• logLevel (string, v0.8.0) the minimal logging level. Possible values are:DEBUG,INFO,NOTICE,WARNING,ERROR,CRITICAL,ALERT,EMERGENCY. If you setWARNING you will not receiveDEBUG,INFO,andNOTICE messages. Default:WARNING.
• label (string) name of the library catalogue
• url (string) link to library catalogue homepage
• schema (string) metadata schema type (MARC21 as default,PICA orUNIMARC)
• language (string) default language of the user interface
• linkTemplate (string) URL template to link into the library catalogue ({id} will bereplaced by the trimmed record identifier). This parameter does not work for all cataloueg classes.

The parameterinclude can be used to include another configuration file and merge its parameters into the base configuration. Recursive inclusion is not supported and include is ignored if the referenced file does not exist.

Multiple catalogues can be configured in one file. To do so, add an individualcatalogueid in brackets after a configuration field to override its defaultvalue. For instance ifversions=false is a general setting, whileversions[loc]=false is a library specific setting.

Example:

id=bvbdisplay-network=0indexName[bvb]=bayerndirName[bvb]=bayern

setup additional directories and permissions:

sudo chgrp www-data -R _smarty cacheln -s$DATADIR/[catalogue]/img images/[catalogue]

On Apache webserver add these lines to its configuration (/etc/apache2/sites-available/000-default.conf):

<Directory/var/www/html/$CATALOG>AllowOverride AllOrder allow,deny  allowfrom all</Directory>

If you don't want the application to check version number from local gitrepository, create a (possibly empty) configuration fileversion.ini.The file can also be generated withcomposer run git-version.

echo"catalogue=$CATALOG">> configuration.cnf

Finally you can access the application athttp://localhost/$APPDIR.

Some parts of the web interface can be customized with local files in directoryconfig (not existing by default) or another directory configured withparametertemplates:

• config/header.en.tpl andconfig/header.de.tpl shown at bottom of each page(right after the<body> tag)
• config/footer.en.tpl andconfig/footer.de.tpl shown at bottom of each page(right before the</body> tag)
• config/about.en.tpl andconfig/about.de.tpl: Additional information shownin the "about" tab.

The name, catalogue link and the record level catalogue link are differentper libraries. The tool has prepared for a number of libraries, but there'shigh chance, that you would like to apply it for another library.

Basic properties and defaultconfiguration settings are definedin a catalogue class which extends the generic classCatalogue. Such a class enablesyou to define additional changes that go beyond simple configuration settings.

Here is an example of a custom catalogue class:

class Gentextends Catalogue {protected$name ='gent';protected$label ='Universiteitsbibliotheek Gent';protected$url ='https://lib.ugent.be/';protected$marcVersion ='GENT';functiongetOpacLink($id,$record) {return'https://lib.ugent.be/catalog/rug01:' .trim($id);  }}

$url contains an URL of the catalogue in the library website,$marcVersionis the abbreviation of MARC version used in the analyses.

Please create a new file in the directoryclasses/catalogue. You do not haveto do any other registration. The convention is that the name of the classis the first upper case form of the name property (Gent - gent,Cerl - cerl)etc. The later should fit the data directory name, the Solr index name, andeither the application path or thecatalogue property of theconfiguration.$url contains an URL of the catalogue in the librarywebsite,$marcVersion is the abbreviation of MARC version used in theanalyses.

Now the user interface is translatable via gettext methods. We providetwo language files (German and English). In.tpl files you can add translatable text as

{_('translatable text')}

_ is a built-in alias for the PHP functiongettext. If there are variables in thetranslated string, in the.tpl file you should use the_t function, defined by the project,like this:

{_t('number of records: %d', $number_of_records)}

You should add the translations intolocale/de_DE/LC_MESSAGES/messages.po as

msgid "translatable text"msgstr "übersetzbarer Text"

and intolocale/en_GB/LC_MESSAGES/messages.po as

msgid "translatable text"msgstr "translatable text"

If the message string contain variables, use sprintf compatible placeholders,such as '%d' for integers, '%s' for strings, '%f' for floating point numbers etc.

msgid "Found <span id=\"numFound\">%s</span> records"msgstr "Found <span id=\"numFound\">%s</span> records"

Of course the message identifier could be different, and dense but nowI think that it is more understandable (so translatable) this way. Whenyou add a translation please add a comment to denote which page the originaltext appears, such as

# completenessmsgid "by document types"msgstr "nach Dokumenttypen"

Once you have done, you should generate the.mo files from the.po files with the following command:

composer run translate

If translations have been changed, Webserver or FastCGI respectively may need to be restarted to clear the translation cache, e.g.

sudo service apache2 restart

Please let us know if you would like to see more languages supported.

Troubleshouting: if the translation would not work you can check if a givenlanguage (locale) is available in your system. In Linux you can check it with

locale -a

If the locale (e.g. 'de_DE.UTF-8') is not available, you can install it with

locale-gen de_DE.UTF-8

The tool supports the following languages:

• English (en, en_GB.UTF-8, en_GB.utf8)
• German (de, de_DE.UTF-8, de_DE.utf8)
• Brazilian Portuguese (pt, pt_BR.UTF-8, pt_BR.utf8)
• Hungarian (hu, hu_HU.UTF-8, hu_HU.utf8)

Note: translation is in a very early phase, not all the text of the user interface are translatable, and translations are not equally covering the original texts.

QA Catalogue Web is developed by Péter Király and Jakob Voß.

Please notify us if you would like to use it. Happy searching!

QA Catalogue Web is managed in a public git repository atpkiraly/qa-catalogue-web.Contributions are welcome!

To ensure code quality run the following tasks:

• composer run lint for PHP linting (detect syntax errors)
• composer run analyses for static code analysis with PHPStan (detect type errors, unused code etc.)
• composer run test for unit tests
• composer run checks to run all of the tasks listed above

GNU General Public License - GPLv3