teex.saliencyMap package#

teex.saliencyMap.data module#

Module for synthetic and real datasets with available ground truth saliency map explanations. Also contains methods and classes for saliency map data manipulation.

All of the datasets must be instanced first. Then, when sliced, they all return the observations, labels and ground truth explanations, respectively. Note that all real-world datasets implement the delete_data method, which allows to delete all of their downloaded internal data. In this module, images and explanations are represented by the PIL.Image.Image class.

class teex.saliencyMap.data.CUB200[source]#

Bases: _ClassificationDataset

The CUB-200-2011 Classification Dataset. 11788 observations with 200 different classes. From

Wah, Branson, Welinder, Perona, & Belongie. (2022). CUB-200-2011 (1.0) [Data set]. CaltechDATA. https://doi.org/10.22002/D1.20098

get_class_observations(classId: int)[source]#

Get all observations from a particular class given its index.

Args:
classId (int): Class index. It can be consulted from the attribute

CUB200.classMap

Returns:

imgs (list): Images pertaining to the specified class. labels (list): Int labels pertaining to the specified class. exps (list): Explanations pertaining to the specified class.

class teex.saliencyMap.data.Kahikatea[source]#

Bases: _ClassificationDataset

Binary classification dataset from [Y. Jia et al. (2021) Studying and Exploiting the Relationship Between Model Accuracy and Explanation Quality, ECML-PKDD 2021].

This dataset contains images for Kahikatea (an endemic tree in New Zealand) classification. Positive examples (in which Kahikatea trees can be identified) are annotated with true explanations such that the Kahikatea trees are highlighted. If an image belongs to the negative class, None is provided as an explanation.

Example:

>>> kDataset = Kahikatea()
>>> img, label, exp = kDataset[1]

where img is a PIL Image, label is an int and exp is a PIL Image. When a slice is performed, obs, label and exp are lists of the objects described above.

get_class_observations(classId: int) → list[source]#

Get observations of a particular class.

Parameters:

classId (int) – Class ID. See attribute classMap.

Returns:

Observations of the specified type.

Return type:

list

class teex.saliencyMap.data.OxfordIIIT[source]#

Bases: _ClassificationDataset

The Oxford-IIIT Pet Dataset. 7347 images from 37 categories with approximately 200 images per class. From

O. M. Parkhi, A. Vedaldi, A. Zisserman and C. V. Jawahar, “Cats and dogs,” 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 3498-3505, doi: 10.1109/CVPR.2012.6248092.

get_class_observations(classId: int) → list[source]#

Get all observations from a particular class given its index.

Args:
classId (int): Class index. It can be consulted from the attribute

classMap.

Returns:

imgs (np.ndarray): Images pertaining to the specified class. labels (np.ndarray): Int labels pertaining to the specified class. exps (np.ndarray): Explanations pertaining to the specified class.

class teex.saliencyMap.data.SenecaSM(nSamples=1000, imageH=32, imageW=32, patternH=16, patternW=16, cellH=4, cellW=4, patternProp=0.5, fillPct=0.4, colorDev=0.1, randomState=888)[source]#

Bases: _SyntheticDataset

Synthetic dataset with available saliency map explanations.

Images and g.t. explanations generated following the procedure presented in [Evaluating local explanation methods on ground truth, Riccardo Guidotti, 2021]. The g.t. explanations are binary ndarray masks of shape (imageH, imageW) that indicate the position of the pattern in an image (zero array if the pattern is not present) and are generated The generated RGB images belong to one class if they contain a certain generated pattern and to the other if not. The images are composed of homogeneous cells of size (cellH, cellW), which in turn compose a certain pattern of shape (patternH, patternW) that is inserted on some of the generated images.

From this class one can also obtain a trained transparent model (instance of TransparentImageClassifier).

When sliced, this object will return

  • X (ndarray) of shape (nSamples, imageH, imageW, 3) or (imageH, imageW, 3). Generated image data.

  • y (ndarray) of shape (nSamples,) or int. Image labels. 1 if an image contains the pattern and 0 otherwise.

  • explanations (ndarray) of shape (nSamples, imageH, imageW) or (imageH, imageW). Ground truth explanations.

Parameters:

  • nSamples (int) – number of images to generate.

  • imageH (int) – height in pixels of the images. Must be multiple of cellH.

  • imageW (int) – width in pixels of the images. Must be multiple of cellW.

  • patternH (int) – height in pixels of the pattern. Must be <= imageH and multiple of cellH.

  • patternW (int) – width in pixels of the pattern. Must be <= imageW and multiple of cellW.

  • cellH (int) – height in pixels of each cell.

  • cellW (int) – width in pixels of each cell.

  • patternProp (float) – ([0, 1]) percentage of appearance of the pattern in the dataset.

  • fillPct (float) – ([0, 1]) percentage of cells filled (not black) in each image.

  • colorDev (float) – ([0, 0.5]) maximum val summed to 0 valued channels and minimum val substracted to 1 valued channels of filled cells. If 0, each cell will be completely red, green or blue. If > 0, colors may be a mix of the three channels (one ~1, the other two ~0).

  • randomState (int) – random seed.

class teex.saliencyMap.data.TransparentImageClassifier[source]#

Bases: _BaseClassifier

Used on the higher level data generation class SenecaSM (use that and get it from there preferably).

Transparent, pixel-based classifier with pixel (features) importances as explanations. Predicts the class of the images based on whether they contain a certain specified pattern or not. Class 1 if they contain the pattern, 0 otherwise. To be trained only a pattern needs to be fed. Follows the sklean API. Presented in [Evaluating local explanation methods on ground truth, Riccardo Guidotti, 2021].

explain(obs: ndarray) → ndarray[source]#

Explain observations’ predictions with binary masks (pixel importance arrays).

Parameters:

obs (np.ndarray) – array of n images as ndarrays.

Returns:

list with n binary masks as explanations.

fit(pattern: ndarray, cellH: int = 1, cellW: int = 1) → None[source]#

Fits the model.

predict(obs: ndarray) → ndarray[source]#

Predicts the class for each observation.

Parameters:

obs (np.ndarray) – array of n images as ndarrays of np.float32 type.

Returns:

array of n predicted labels.

predict_proba(obs: ndarray) → ndarray[source]#

Predicts probability that each observation belongs to class 1 or 0. Probability of class 1 will be 1 if the image contains the pattern and 0 otherwise.

Parameters:

obs (np.ndarray) – array of n images as ndarrays.

Returns:

array of n probability tuples of length 2.

teex.saliencyMap.data.binarize_rgb_mask(img, bgValue='high') → array[source]#

Binarizes a RGB binary mask, letting the background (negative class) be 0. Use this function when the image to binarize has a very defined background.

Parameters:

  • img – (ndarray) of shape (imageH, imageW, 3), RGB mask to binarize.

  • bgValue – (str) Intensity of the negative class of the image to binarize: {‘high’, ‘low’}

Returns:

(ndarray) a binary mask.

teex.saliencyMap.data.delete_sm_data() → None[source]#

Removes from internal storage all downloaded Saliency Map datasets. See the delete_data method of all Saliency Map datasets to delete only their corresponding data.

teex.saliencyMap.data.rgb_to_grayscale(img)[source]#

Transforms a 3 channel RGB image into a grayscale image (1 channel).

Parameters:

img (np.ndarray) – of shape (imageH, imageW, 3)

Return np.ndarray:

of shape (imageH, imageW)

teex.saliencyMap.eval module#

Module for evaluation of saliency map explanations.

teex.saliencyMap.eval.saliency_map_scores(gts, sMaps, metrics=None, binThreshold=0.01, gtBackgroundVals='high', average=True)[source]#

Quality metrics for saliency map explanations, where each pixel is considered as a feature. Computes different scores of a saliency map explanation w.r.t. its ground truth explanation (a binary mask).

Parameters:

  • gts (np.ndarray) –

    ground truth RGB or binary mask/s. Accepted shapes are

    • (imageH, imageW) A single grayscale mask, where each pixel should be 1 if it is part of the salient class and 0 otherwise.

    • (imageH, imageW, 3) A single RGB mask, where pixels that do not contain the salient class are all either black (all channels set to 0) or white (all channels set to max.).

    • (nSamples, imageH, imageW) Multiple grayscale masks, where for each where, in each image, each pixel should be 1 if it is part of the salient class and 0 otherwise.

    • (nSamples, imageH, imageW, 3) Multiple RGB masks, where for each image, pixels that do not contain the salient class are all either black (all channels set to 0) or white (all channels set to max.).

    If the g.t. masks are RGB they will be binarized (see param gtBackground to specify the color of the pixels that pertain to the non-salient class).

  • sMaps (np.ndarray) –

    grayscale saliency map explanation/s ([0, 1] or [-1, 1] normalised). Supported shapes are

    • (imageH, imageW) A single explanation

    • (nSamples, imageH, imageW) Multiple explanations

  • metrics –

    (str / array-like of str, default=[‘auc’]) Quality metric/s to compute. Available:

    • ’auc’: ROC AUC score. The val of each pixel of each saliency map in sMaps is considered as a prediction probability of the pixel pertaining to the salient class.

    • ’fscore’: F1 Score.

    • ’prec’: Precision Score.

    • ’rec’: Recall score.

    • ’cs’: Cosine Similarity.

    For ‘fscore’, ‘prec’, ‘rec’ and ‘cs’, the saliency maps in sMaps are binarized (see param binThreshold).

  • binThreshold (float) – (in [0, 1]) pixels of images in sMaps with a val bigger than this will be set to 1 and 0 otherwise when binarizing for the computation of ‘fscore’, ‘prec’, ‘rec’ and ‘auc’.

  • gtBackgroundVals (str) – Only used when provided ground truth explanations are RGB. Color of the background of the g.t. masks ‘low’ if pixels in the mask representing the non-salient class are dark, ‘high’ otherwise).

  • average (bool) – (default True) Used only if gts and sMaps contain multiple observations. Should the computed metrics be averaged across all of the samples?

Returns:

specified metric/s in the original order. Can be of shape

  • (n_metrics,) if only one image has been provided in both gts and sMaps or when both are contain multiple observations and average=True.

  • (n_metrics, n_samples) if gts and sMaps contain multiple observations and average=False.

Return type:

np.ndarray