- Notifications
You must be signed in to change notification settings - Fork2
On the Robustness and Discriminative Power of Information Retrieval Metrics for Top-N Recommendation
License
dvalcarce/evalMetrics
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Source code of the experiments of:
Daniel Valcarce,Alejandro Bellogín,Javier Parapar,Pablo Castells:On the Robustness and Discriminative Power of IR Metrics for Top-N Recommendation. InProceedings of the 12th ACM Conference on Recommender Systems, RecSys 2018, Vancouver, Canada, 2-7 October, 2018. DOI10.1145/3240323.3240347.
The code of the experiments can be found in the following Jupyter notebooks:
correlation-among-metrics.ipynb: measures ranking correlations among metrics.discrimination-analysis.ipynb: measures discriminative power of metrics.pop-correlation.ipynb: measures robustness to popularity bias of metrics.sparse-correlation.ipynb: measures robustness to sparsity bias of metrics.
We ran 21 recommender systems on three datasets (BeerAdvocate, LibraryThing and MovieLens 1M). The output of these recommenders was evaluated usingrec_eval tool. We also measured statistically significant improvements using permutation test. The output of both tools can be found indata.
The code was implemented by Daniel Valcarce of theInformation Retrieval Lab of theUniversity of A Coruña during his stay at theInformation Retrieval Group of theUniversidad Autónoma de Madrid. If you have any comment or question, do not hesitate to write an email to daniel [DOT] valcarce [AT] udc [DOT] es.