Microsoft AdversarialCat TF

AdversarialCat TF

artificial-intelligencernnganmicrosoft-for-beginnerslessonsAImicrosoft-AI-For-Beginnersmachine-learning08-TransferLearningdeep-learning4-ComputerVisioncomputer-visioncnnNLP

alph-notebooks/microsoft-AI-For-Beginners / AdversarialCat_TF.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

How Neural Network sees a Cat

A neural network pre-trained on ImageNet is capable of recognizing any of 1000 different classes of objects, such as cats of different breeds. It would be interesting to see, what does the ideal siamese cat looks like for a neural network.

Of course, you can replace siamese cat with any other ImageNet class.

To start, let's load VGG network:

[1]

2022-06-17 13:28:58.931467: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-06-17 13:28:59.603736: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 15401 MB memory:  -> device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0001:00:00.0, compute capability: 6.0

Optimizing for Result

To visualize the ideal cat, we will start with a random noise image, and will try to use the gradient descent optimization technique to adjust the image to make a network recognize a cat.

Optimization Loop

Here is our starting image:

[2]

<matplotlib.image.AxesImage at 0x7f90a0277ac0>

We use normalize function to bring our values into 0-1 range.

If we call our VGG network on this image, we will get more or less random distribution of probabilities:

[3]

2022-06-17 13:29:02.100818: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8303
2022-06-17 13:29:02.570980: I tensorflow/core/platform/default/subprocess.cc:304] Start cannot spawn child process: No such file or directory

Predicted class: 669 (mosquito net)
Probability of predicted class = 0.05466596782207489

Even though it may look like the probability of one of the classes is much higher than the others, it is still very low - look at the scale to see that actual probability is still around 5%.

Now let's chose one target category (eg., siamese cat), and start adjusting the image using gradient descent. If $x$ is the input image, and $V$ is the VGG network, we will calculate the loss function $\mathcal{L} = \mathcal{L}(c,V(x))$ (where $c$ is the target category), and adjust $x$ using the following formula:

x^{(i+1)} = x^{(i)} - \eta{\partial \mathcal{L}\over\partial x}

Loss function would be cross-entropy loss, because we are comparing two probability distributions. In our case, because the class is represented by a number, and not by one-hot encoded vector, we will use sparse categorical cross-entropy.

We will repeat this process for several epochs, printing the image as we go.

It is better to execute this code on GPU-enabled compute, or reduce the number of epochs in order to minimize waiting time.

[4]

Epoch: 900, loss: 0.5220473408699036

[5]

Predicted class: 284 (Siamese cat, Siamese)
Probability of predicted class = 0.6449142098426819

We now have obtained an image that looks like a cat for a neural network, even though it still looks like a noise for us. If we optimize for a little bit longer - we are likeley to get the image of ideal noisy cat, which has probability close to 1.

Making Sense of Noise

This noise does not make a lot of sense for us, but most probably it contains a lot of low-level filters that are typical for a cat. However, because there are very many ways to optimize input for the ideal result, the optimization algorithm is not motivated to find patterns that are visually comprehensible.

To make this look a little bit less like a noise, we can introduce an additional term to the loss function - variation loss. It measures how similar neighboring pixels of the image are. If we add this term to our loss function, it will force the optimizer to find solutions with less noise, and thus having more recognizable details.

In practice, we need to balance between cross-entropy loss and variation loss to obtain good results. In our function, we introduce some numeric coefficients, and you can play with them and observe how image changes.

[6]

Epoch: 900, loss: [27.257]

This is the ideal image of a cat for our neural network, and we can also see some of the familiar features, such as eyes and ears. There are many of them, which makes neural network even more certain that this is a cat.

[7]

Predicted class: 284 (Siamese cat, Siamese)
Probability of predicted class = 0.9201651215553284

Let's also see how some other object looks like for the VGG:

[8]

Epoch: 900, loss: [29.59]

Adversarial Attacks

Since ideal cat image can look like a random noise, it suggests that we can maybe tweak any image in a little way so that it changes it's class. Let's experiment with this a little bit. We will start with an image of a dog:

[9]

<matplotlib.image.AxesImage at 0x7f8fd816e0a0>

We can see that this image is clearly recognized as a dog:

[10]

Predicted class: 171 (Italian greyhound)
Probability of predicted class = 0.9281901121139526

Now, we will use this image a starting point, and try to optimize it to become a cat:

[11]

Epoch: 90, loss: 0.15769274532794952

[12]

Predicted class: 284 (Siamese cat, Siamese)
Probability of predicted class = 0.8651191592216492

So, this image above is a perfect cat, from the point of view of VGG network!

Experimenting with ResNet

Let's now see how this same image is classified by a different model, say, ResNet:

[13]

Since we used model as a global variable, from now on all functions will use ResNet instead of VGG

[14]

Predicted class: 111 (nematode, nematode worm, roundworm)
Probability of predicted class = 0.13089127838611603

Apparenlty, the result is quite different. This is quite expected, because when optimizing for a cat we took into account the nature of VGG network, it's low-level filters, etc. Since ResNet has different filters, it gives different results. This gives us the idea of how we can protect ourselves from adversarial attacks - by using ensemble of different models.

Let's see how the ideal zebra looks like for ResNet:

[15]

Epoch: 450, loss: [46.166]

[16]

Predicted class: 340 (zebra)
Probability of predicted class = 0.8876020312309265

This picture is quite different, which tells us that the architecture of a neural network probably plays quite an important role in the way it recognizes objects.

Task: Try to perform adversarial attach on ResNet, and compare the results.

Using Different Optimizers

In our example, we have been using the simplest optimization technique - gradient descent. However, Keras framework contains different built-in optimizers, and we can use them instead of gradient descent. This will require very little change to our code - we will replace the part where we adjust input image x.assign_sub(eta*grads) with a call to apply_gradients function of the optimizer:

[17]

Epoch: 900, loss: [41.451]

Conclusion

We were able to visualize the ideal image of a cat (as well as any other objects) within pre-trained CNN, using gradient descent optimization to adjust the input image instead of weights. The main trick to get the image that makes some sense was to use variation loss as an additional loss function, which enforces the image to look smoother.