Hi!

I have tried to generate explanations for NER models but I haven't found any description of how to implement this, so I have tried to extrapolate what is been already done for other tasks, such as multi-class classification.

I have followed the approach described in theCaptum tutorial to interpret BERT in which a custom forward function that allows to access a particular position of the prediction using theposition input argument is used and the attributions are computed with respect to theBertEmbeddings using thecaptum.attr.LayerIntegratedGradients class.

I have generated two types of visualizations:

1. Attributions considering each token (position) and its corresponding predicted NER label: this allows to see how the rest of the sentence influenced each token to be classified into a particular NER class.
2. Attributions considering a single token and all the possible NER classes: this allows to see how the rest of the sentence influenced a certain token to be classified into a class and not into any of the others.

A notebook with these two visualizations can be found inthis repo. I would like to know if my approach makes sense, or, if not, what I should do to correctly visualize explanations for NER models. Once I am sure my approach is right, I would like to contribute toTransformers Interpret implementing the support for NER models.

Thank you in advance for the help!