如何使用我自己的图片使用 FGSM 生成对抗样本？

Question

我正在尝试使用 FGSM 生成对抗性示例，我使用的代码框架来自 Google Colab 代码（ https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials /generation/adversarial_fgsm.ipynb#scrollTo=wpYrQ4OQSYWk ）。 我的 Jupyter Book 的内核信息是 Python3.7。 但是，当我尝试使用自己的图片生成对抗样本时，编译一直失败。 实际上，我唯一改变的部分是“image_path = 'cat.jpg'”。 好吧，我在google上搜索了错误，但似乎没有类似的情况。 因此，请您帮忙，非常感谢！ 这是我的代码：

  import tensorflow as tf
import matplotlib as mpl
import matplotlib.pyplot as plt


mpl.rcParams['figure.figsize'] = (8, 8)
mpl.rcParams['axes.grid'] = False

pretrained_model = tf.keras.applications.MobileNetV2(include_top=True,
                                                     weights='imagenet')
pretrained_model.trainable = False

# ImageNet labels
decode_predictions = tf.keras.applications.mobilenet_v2.decode_predictions

# Helper function to preprocess the image so that it can be inputted in MobileNetV2
def preprocess(image):
  image = tf.cast(image, tf.float32)
  image = tf.image.resize(image, (224, 224))
  image = tf.keras.applications.mobilenet_v2.preprocess_input(image)
  image = image[None, ...]
  return image

# Helper function to extract labels from probability vector
def get_imagenet_label(probs):
  return decode_predictions(probs, top=1)[0][0]

image_path = 'cat.jpg'
image_raw = tf.io.read_file(image_path)
image = tf.image.decode_image(image_raw)

image = preprocess(image)
image_probs = pretrained_model.predict(image)

plt.figure()
plt.imshow(image[0]*0.5+0.5) # To change [-1, 1] to [0,1]
_, image_class, class_confidence = get_imagenet_label(image_probs)
plt.title('{} : {:.2f}% Confidence'.format(image_class, class_confidence*100))
plt.show()

loss_object = tf.keras.losses.CategoricalCrossentropy()

def create_adversarial_pattern(input_image, input_label):
  with tf.GradientTape() as tape:
    tape.watch(input_image)
    prediction = pretrained_model(input_image)
    loss = loss_object(input_label, prediction)

  # Get the gradients of the loss w.r.t to the input image.
  gradient = tape.gradient(loss, input_image)
  # Get the sign of the gradients to create the perturbation
  signed_grad = tf.sign(gradient)
  return signed_grad

# Get the input label of the image.
labrador_retriever_index = 208
label = tf.one_hot(labrador_retriever_index, image_probs.shape[-1])
label = tf.reshape(label, (1, image_probs.shape[-1]))

perturbations = create_adversarial_pattern(image, label)
plt.imshow(perturbations[0]*0.5+0.5); # To change [-1, 1] to [0,1]

def display_images(image, description):
  _, label, confidence = get_imagenet_label(pretrained_model.predict(image))
  plt.figure()
  plt.imshow(image[0]*0.5+0.5)
  plt.title('{} \n {} : {:.2f}% Confidence'.format(description,
                                                   label, confidence*100))
  plt.show()

epsilons = [0, 0.02, 0.2, 0.4, 0.8, 0.9, 1.0]
descriptions = [('Epsilon = {:0.3f}'.format(eps) if eps else 'Input')
                for eps in epsilons]

for i, eps in enumerate(epsilons):
  adv_x = image + eps*perturbations
  adv_x = tf.clip_by_value(adv_x, -1, 1)
  display_images(adv_x, descriptions[i])

错误是：

>     ---------------------------------------------------------------------------
_FallbackException                        Traceback (most recent call last)
D:\anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\ops\gen_io_ops.py in read_file(filename, name)
    601         _ctx._context_handle, _ctx._thread_local_data.device_name, "ReadFile",
--> 602         name, _ctx._post_execution_callbacks, filename)
    603       return _result

_FallbackException: This function does not handle the case of the path where all inputs are not already EagerTensors.

During handling of the above exception, another exception occurred:

UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-18-4edeac21c176> in <module>
     27 
     28 image_path = 'cat.jpg'
---> 29 image_raw = tf.io.read_file(image_path)
     30 image = tf.image.decode_image(image_raw)
     31 

D:\anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\ops\gen_io_ops.py in read_file(filename, name)
    605       try:
    606         return read_file_eager_fallback(
--> 607             filename, name=name, ctx=_ctx)
    608       except _core._SymbolicException:
    609         pass  # Add nodes to the TensorFlow graph.

D:\anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\ops\gen_io_ops.py in read_file_eager_fallback(filename, name, ctx)
    654   _attrs = None
    655   _result = _execute.execute(b"ReadFile", 1, inputs=_inputs_flat,
--> 656                              attrs=_attrs, ctx=_ctx, name=name)
    657   _execute.record_gradient(
    658       "ReadFile", _inputs_flat, _attrs, _result, name)

D:\anaconda3\envs\tensorflow\lib\site-packages\tensorflow_core\python\eager\execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     59     tensors = pywrap_tensorflow.TFE_Py_Execute(ctx._handle, device_name,
     60                                                op_name, inputs, attrs,
---> 61                                                num_outputs)
     62   except core._NotOkStatusException as e:
     63     if name is not None:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 57: invalid continuation byte

Answer 1

我用我的一些图片测试了你的代码，一切正常。

这只是关于您的图像编码。 使用其他一些图像再次运行或更改图像编码。