Vertex Explainable AI provides built-in visualization capabilities for your image data.You can configure visualizations for custom-trained image models.

When you request an explanation on an image classification model, you get thepredicted class along with an image overlay showing which pixelsor regions contributed to the prediction.

The following images show visualizations on a husky image. The leftvisualization uses the integrated gradients method and highlights areas ofpositive attribution. The right visualization uses an XRAI method with a colorgradient indicating areas of lesser (blue) and greater (yellow) influence inmaking a positive prediction.

The type of data you're working with can influence whether you use anintegrated gradients or XRAI approach to visualizing your explanations.

• XRAI tends to be better with natural images and provides a better high-levelsummary of insights, like showing that positive attribution is related to theshape of a dog's face.
• Integrated gradients (IG) tends to provide details at the pixel level and isuseful for uncovering more granular attributions.

Learn more about the attribution methods in the Vertex Explainable AIOverview page.

Get started

Configure visualization when youcreate aModel resource that supportsVertex Explainable AI, or when youoverridetheModel'sExplanationSpec.

To configure visualization for your model, populate thevisualization field oftheInputMetadata message corresponding to the featurethat you want to visualize. In this configuration message, you can includeoptions such as the type of overlay used, which attributions are highlighted,color, and more. All settings are optional.

Visualization options

The default and recommended settings depend on the attribution method(integrated gradients or XRAI). The following list describes configurationoptions and how you might use them. For a full list of options, see theAPI reference for theVisualization message.

• type: The type of visualization used:OUTLINES orPIXELS. Thetypefield defaults toOUTLINES, which shows regions of attribution. To showper-pixel attribution, set the field toPIXELS.

• polarity: The directionality of the highlighted attributions.positive isset by default, which highlights areas with the highest positiveattributions. This means highlighting pixels thatwere most influential to the model's positive prediction.Setting polarity tonegative highlights areas that lead the model to notpredicting the positive class. Using a negative polarity can be useful fordebugging your model by identifying false negative regions. You can also setpolarity toboth which shows positive and negative attributions.

• clip_percent_upperbound: Excludes attributions above the specifiedpercentilefrom the highlighted areas. Using the clip parameters together can be usefulfor filtering out noise and making it easier to see areas of strongattribution.

• clip_percent_lowerbound: Excludes attributions below the specifiedpercentilefrom the highlighted areas.

• color_map: The color scheme used for the highlighted areas. Default ispink_green for integrated gradients, which shows positive attributions ingreen and negative in pink. For XRAI visualizations, the color map is agradient. The XRAI default isviridis which highlights the most influentialregions in yellow and the least influential in blue.

  For a full list of possible values, see theAPI reference for theVisualization message.

• overlay_type: How the original image is displayed in the visualization.Adjusting the overlay can help increase visual clarity if the original imagemakes it difficult to view the visualization.

  For a full list of possible values, see theAPI reference for theVisualization message.

Example configurations

To get started, below are sampleVisualization configurations that you can useas a starting point and images that show a range of settings applied.

Integrated gradients

For integrated gradients, you may need to adjust the clip values if theattribution areas are too noisy.

visualization:{"type":"OUTLINES","polarity":"positive","clip_percent_lowerbound":70,"clip_percent_upperbound":99.9,"color_map":"pink_green","overlay_type":"grayscale"}

The following visualizations use both theoutlines andpixelstypes. The columns labeled "Highly predictive only," "Moderately predictive,"and "Almost all" are examples of clipping at different levels that can helpfocus your visualization.

XRAI

For XRAI visualizations, we recommend starting with no clip values forXRAI because the overlay uses a gradient to show areas of high and lowattribution.

visualization:{"type":"PIXELS","polarity":"positive","clip_percent_lowerbound":0,"clip_percent_upperbound":100,"color_map":"viridis","overlay_type":"grayscale"}

The following image is an XRAI visualization that uses the default viridiscolor map and a range of overlay types. The areas in yellow indicate the mostinfluential regions that contributed positively to the prediction.

What's next

• For details about other Vertex Explainable AI configuration options, readConfiguringexplanations for custom-trained models.