Class Activation Map Localization

Enivironment

• OS : Linux
• Python : 3.7.6
• CPU : i7-6700HQ ~2.60GHz
• GPU : GTX 1060 6GB

Preprocessing

Transform sequence :

normalize = transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
        transformList = []
        transformList.append(transforms.Resize(imageSize))
        transformList.append(transforms.ToTensor())
        transformList.append(normalize)   

where imageSize = 224.

Model

This Class Activation Map is generated using a single DenseNet-121 based model. The model pipeline is as shown below:

Algorithm and Explanation

CAM algorithm used here works on similar lines of what has been shown above.

• We take in a image and pass it through the transformation pipeline and finally flatten it, to feed it to the convolutional densenet121 base.

• output = self.model(input.cuda()) is the output of the final relu Activation layer of DenseNet.

• self.weights = list(self.model.parameters())[-2] are the 1024 weights of the second-last layer of the model convolutional base.

• The heatmap generated will be the weighted average of the 2D Sub-Tensors of the 4D Output Tensor over the entire convolutional base output. So, the final heatmap is => heatmap += self.weights[i] * output[0,i,:,:] with i ranging from 0 to len(self.weights) which is 1024

• Finally we complete the average and make sure that its a float32 image.

  npHeatmap = heatmap.cpu().data.numpy()
  cam = npHeatmap / np.max(npHeatmap)

• We have the heatmap now 😄

Finally Some Example Images 🤓

References

• This repo here helped me a lot.

• This paper about Learning Deep Features for Discriminative Localization here explains the algorithm.