🌐 harmonydata.ac.uk

You can also joinour Discord server!

• Psychologists and social scientists often have to match items in different questionnaires, such as "I often feel anxious" and "Feeling nervous, anxious or afraid".
• This is calledharmonisation.
• Harmonisation is a time consuming and subjective process.
• Going through long PDFs of questionnaires and putting the questions into Excel is no fun.
• EnterHarmony, a tool that usesnatural language processing and generative AI models to help researchers harmonise questionnaire items, evenin different languages.

💁You can run the walkthrough Python notebook inGoogle Colab with a single click:

🇷You can also download an R markdown notebook to run in R Studio:

🇷You can run the walkthrough R notebook in Google Colab with a single click:

Harmony is a tool using AI which allows you to compare items from questionnaires and identify similar content. You can try Harmony athttps://app.harmonydata.ac.uk and you can read our blog athttps://harmonydata.ac.uk/blog/.

The source code for Harmony is on Github athttps://github.com/harmonydata/harmony.

Here's a walkthrough video on how you can use Harmony online at harmonydata.ac.uk. Click to view:

• 🖱️ The Harmony app which is running athttps://harmonydata.ac.uk/app is in this repo:https://github.com/harmonydata/app.
• 👨‍💻 The Harmony Python library source code is here:https://github.com/harmonydata/harmony.
• 🇷 The Harmony R library source code is here:https://github.com/harmonydata/harmony_r.
• 💻 The Harmony API source code is here:https://github.com/harmonydata/harmonyapi.
• 📰 The code for training the PDF extraction is here:https://github.com/harmonydata/pdf-questionnaire-extraction
• 📔 Finally, the source code of the Harmony static blog athttps://harmonydata.ac.uk is in this repo:https://github.com/harmonydata/harmonydata.github.io. It is hosted withGithub Pages.

Information about Harmony's server setup and deployment isin the private repo harmony_deployment_ulster_private.

The Harmony project is a data harmonisation project that usesNatural Language Processing to help researchers make better use of existing data from different studies by supporting them with the harmonisation of various measures and items used in different studies.

Harmony is a collaboration project betweenUlster University,University College London, theUniversidade Federal de Santa Maria, andFast Data Science. Harmony has been funded byWellcome as part of theWellcome Data Prize in Mental Health and byEconomic and Social Research Council (ESRC).

The core team at Harmony is made up of:

• Dr Bettina Moltrecht, PhD (UCL)
• Dr Eoin McElroy (University of Ulster)
• Dr George Ploubidis (UCL)
• Dr Mauricio Scopel Hoffmann (Universidade Federal de Santa Maria, Brazil)
• Thomas Wood (Fast Data Science)

You can contact us athttps://harmonydata.ac.uk/contact/.

McElroy, E., Moltrecht, B., Ploubidis, G.B., Scopel Hoffman, M., Wood, T.A., Harmony [Computer software], Version 1.0, accessed athttps://app.harmonydata.ac.uk. Ulster University (2022)

If you upload a questionnaire or instrument, Harmony does not store or save it. You can read more on ourPrivacy Policy page.

Harmony passes the text of each questionnaire item through a neural network called Sentence-BERT, in order to convert it into a vector. The similarity of two texts is then measured as the similarity between their vectors. Two identical texts have a similarity of 100% while two completely different texts have a similarity of 0%. You can read more in thistechnical blog post and you can evendownload and run Harmony’s source code.

Harmony was able to reconstruct the matches of the questionnaire harmonisation tool developed by McElroy et al in 2020 with the following AUC scores: childhood 81%, adulthood 77%. Harmony was able to match the questions of the English and Portuguese GAD-7 instruments with AUC 100%. You can read more inthis blog post.

The numbers are the cosine similarity of document vectors. The cosine similarity of two vectors can range from -1 to 1 based on the angle between the two vectors being compared. We have converted these to percentages. We have also used a preprocessing stage to convert positive sentences to negative and vice-versa (e.g. I feel anxious -> I do not feel anxious). If the match between two sentences improves once this preprocessing has been applied, then the items are assigned a negative similarity.

At this time Harmony does not give p-values. But you can interpret the percentage matches like correlation coefficients. In future we hope to provide more statistical data to Harmony’s users.

• Thomas Wood (Fast Data Science)

You can cite ourvalidation paper:

McElroy, Wood, Bond, Mulvenna, Shevlin, Ploubidis, Scopel Hoffmann, Moltrecht,Using natural language processing to facilitate the harmonisation of mental health questionnaires: a validation study using real-world data. BMC Psychiatry 24, 530 (2024),https://doi.org/10.1186/s12888-024-05954-2

A BibTeX entry for LaTeX users is

{{< rawhtml >}}

@article{mcelroy2024using,  title={Using natural language processing to facilitate the harmonisation of mental health questionnaires: a validation study using real-world data},  author={McElroy, Eoin and Wood, Thomas and Bond, Raymond and Mulvenna, Maurice and Shevlin, Mark and Ploubidis, George B and Hoffmann, Mauricio Scopel and Moltrecht, Bettina},  journal={BMC psychiatry},  volume={24},  number={1},  pages={530},  year={2024},  publisher={Springer}}

{{< /rawhtml >}}

Repositories