Repository files navigation

Docs

irspack is a Python package for recommender systems based on implicit feedback, designed to be used by practitioners.

Some of its features include:

• Efficient parameter tuning enabled by C++/Eigen implementations of core recommender algorithms andoptuna.
  • In particular, if an early stopping scheme is available, optuna can prune out unpromising trial based on the intermediate validation scores.
• Various utility functions, including
  • ID/index mapping utilities
  • Fast, multithreaded argsort for batch recommendation retrieval
  • Efficient and configurable evaluation of recommender system performance

In most cases, you can install the pre-build binaries via

pip install irspack

The binaries have been compiled to use AVX instruction. If you want to use AVX2/AVX512 or your environment does not support AVX (like Rosetta 2 on Apple M1), install it from source like

CFLAGS="-march=native" pip install git+https://github.com/tohtsky/irspack.git

In that case, you must have a decent version of C++ compiler (with C++11 support). If it doesn't work, feel free to make an issue!

I have also prepared a wrapper class (BPRFMRecommender) to train/optimize BPR/warp loss Matrix factorization implemented inlightfm. To use it you have to installlightfm separately, e.g. by

pip install lightfm

If you want to use Mult-VAE, you'll need the following additional (pip-installable) packages:

• jax
• jaxlib
  • If you want to use GPU, follow the installation guidehttps://github.com/google/jax#installation
• dm-haiku
• optax

To begin with, we represent the user/item interaction as ascipy.sparse matrix. Then we can feed it into recommender classes:

importnumpyasnpimportscipy.sparseasspsfromirspackimportIALSRecommender,df_to_sparsefromirspack.datasetimportMovieLens100KDataManagerdf=MovieLens100KDataManager().read_interaction()# Convert pandas.Dataframe into scipy's sparse matrix.# The i'th row of `X_interaction` corresponds to `unique_user_id[i]`# and j'th column of `X_interaction` corresponds to `unique_movie_id[j]`.X_interaction,unique_user_id,unique_movie_id=df_to_sparse(df,'userId','movieId')recommender=IALSRecommender(X_interaction)recommender.learn()# for user 0 (whose userId is unique_user_id[0]),# compute the masked score (i.e., already seen items have the score of negative infinity)# of items.recommender.get_score_remove_seen([0])

To evaluate the performance of a recommenderm we have to split the dataset to train and validation sets:

fromirspack.splitimportrowwise_train_test_splitfromirspack.evaluationimportEvaluator# Random splitX_train,X_val=rowwise_train_test_split(X_interaction,test_ratio=0.2,random_state=0)evaluator=Evaluator(ground_truth=X_val)recommender=IALSRecommender(X_train)recommender.learn()evaluator.get_score(recommender)

This will print something like

{'appeared_item':435.0,'entropy':5.160409123151053,'gini_index':0.9198367595008214,'hit':0.40084835630965004,'map':0.013890322881619916,'n_items':1682.0,'ndcg':0.07867240014767263,'precision':0.06797454931071051,'recall':0.03327028758587522,'total_user':943.0,'valid_user':943.0}

Now that we can evaluate the recommenders' performance against the validation set, we can useoptuna-backed hyperparameter optimization.

best_params,trial_dfs=IALSRecommender.tune(X_train,evaluator,n_trials=20)# maximal ndcg around 0.43 ~ 0.45trial_dfs["ndcg@10"].max()

Of course, we have to hold-out another interaction set for test, and measure the performance of tuned recommender against the test set.

Seeexamples/ for more complete examples.

• more benchmark dataset