Repository files navigation

Zennit (Zennitexplainsneuralnetworksintorch) is ahigh-level framework in Python using Pytorch for explaining/exploring neuralnetworks. Its design philosophy is intended to provide high customizability andintegration as a standardized solution for applying rule-based attributionmethods in research, with a strong focus on Layerwise Relevance Propagation(LRP). Zennit strictly requires models to use Pytorch'storch.nn.Modulestructure (including activation functions).

Zennit is currently under active development, but should be mostly stable.

If you find Zennit useful for your research, please consider citing our relatedpaper:

@article{anders2021software,      author  = {Anders, Christopher J. and                 Neumann, David and                 Samek, Wojciech and                 Müller, Klaus-Robert and                 Lapuschkin, Sebastian},      title   = {Software for Dataset-wide XAI: From Local Explanations to Global Insights with {Zennit}, {CoRelAy}, and {ViRelAy}},      journal = {CoRR},      volume  = {abs/2106.13200},      year    = {2021},}

The latest documentation is hosted atzennit.readthedocs.io.

To install directly from PyPI using pip, use:

$ pip install zennit

Alternatively, install from a manually cloned repository to try out the examples:

$ git clone https://github.com/chr5tphr/zennit.git$ pip install ./zennit

At its heart, Zennit registers hooks at Pytorch's Module level, to modify thebackward pass to produce rule-based attributions like LRP (instead of the usualgradient). All rules are implemented as hooks(zennit/rules.py) and most use the LRP basisBasicHook (zennit/core.py).

Composites (zennit/composites.py) are a wayof choosing the right hook for the right layer. In addition to the abstractNameMapComposite, which assigns hooks to layers by name, andLayerMapComposite, which assigns hooks to layers based on their Type, thereexist explicitComposites, some of which areEpsilonGammaBox (ZBox ininput,Epsilon in dense,Gamma in convolutions) orEpsilonPlus (Epsilonin dense,ZPlus in convolutions). All composites may be used by directlyimporting fromzennit.composites, or by using their snake-case name as keyforzennit.composites.COMPOSITES.

Canonizers (zennit/canonizers.py) temporarilytransform models into a canonical form, if required, likeSequentialMergeBatchNorm, which automatically detects and merges BatchNormlayers followed by linear layers in sequential networks, orAttributeCanonizer, which temporarily overwrites attributes of applicablemodules, e.g. to handle the residual connection in ResNet-Bottleneck modules.

Attributors (zennit/attribution.py) directlyexecute the necessary steps to apply certain attribution methods, like thesimpleGradient,SmoothGrad orOcclusion. An optionalComposite maybe passed, which will be applied during theAttributor's execution tocompute the modified gradient, or hybrid methods.

Using all of these components, an LRP-type attribution for VGG16 withbatch-norm layers with respect to label 0 may be computed using:

importtorchfromtorchvision.modelsimportvgg16_bnfromzennit.compositesimportEpsilonGammaBoxfromzennit.canonizersimportSequentialMergeBatchNormfromzennit.attributionimportGradientdata=torch.randn(1,3,224,224)model=vgg16_bn()canonizers= [SequentialMergeBatchNorm()]composite=EpsilonGammaBox(low=-3.,high=3.,canonizers=canonizers)withGradient(model=model,composite=composite)asattributor:out,relevance=attributor(data,torch.eye(1000)[[0]])

A similar setup usingthe example scriptproduces the following attribution heatmaps:

For more details and examples, have a look at ourdocumentation.

More heatmaps of various attribution methods for VGG16 and ResNet50, allgenerated usingshare/example/feed_forward.py, can be foundbelow.

SeeCONTRIBUTING.md for detailed instructions on how to contribute.

Zennit is licensed under the GNU LESSER GENERAL PUBLIC LICENSE VERSION 3 ORLATER -- see theLICENSE,COPYING andCOPYING.LESSER files for details.