Repository files navigation

Code implementation for paperLearning to Explain: Generating Stable Explanations Fast at ACL 2021, by Xuelin Situ, Ingrid Zukerman, Cecile Paris, Sameen Maruf and Reza Haffari.

• Python version >= 3.6.8
• PyTorch version >= 1.7.0
• HuggingFace transformers version >= 1.2.0
• LIME >= 0.1.1.36
• shap == 0.29.3

1. Collect explanations from different baselines >>preprocess.collect_base_explanations.py

2. Train L2E explainer (also refer to folder hyperparameters) >>learning2explain.py

3. Find neighbours for each test example (for stability evaluation):

   • For IMDB_R >>evaluation.find_neighbours_imbdr.py
   • For other datasets >>evaluation.find_neighbours.py

4. Faithfulness evaluation:

   • Prediction based >>evaluation.compare_faithfulness_agreement.py
   • Confidence based >>evaluation.compare_faithfulness.py
   • Prcision/Recall (for IMDB_R only) >>evaluation.compare_imdbr_faithfulness.py

5. Stability evaluation:

   • For IMDB_R >>evaluation.compare_imdbr_stability.py
   • For other datasets >>evaluation.compare_stability.py

6. Efficiency evaluation >>compare_efficiency.py