显示带有边框的图像时出现问题

Question

I have some code which is meant to simply draw a bounding box on an image, and save it out as a new image. However, when I run it I get a TypeError: Cannot handle this data type from pillow.

The code is

# Tests that the processed data file can be read correctly and has correct bounding boxes

import tensorflow as tf
import numpy as np
from PIL import Image

def read_processed_data(filename, num_show):
    """ Reads in the processed data file and displays the
        given number of images, along with the bounding boxes.
    """
    with open(filename, 'r') as f:
        i = 0

        while i < num_show:
            for line in f:
                filename = line.rstrip()
                next_line = f.readline()
                num_faces = int(next_line.rstrip())
                face_num = 0

                #while face_num < num_faces:
                bb_line = f.readline().rstrip()
                y1, x1, y2, x2 = bb_line.split(',')

                box = [y1, x1, y2, x2]
                box = tf.cast(box, tf.float32)

                return box, filename

with tf.Session() as sess:
    bb, fn = read_processed_data("processed.txt", 1)
    image = tf.image.decode_image(tf.read_file(fn))
    img = image.eval()
    print(img.shape)

    img_show = np.asarray(img)
    Image.fromarray(img_show).save("test_no_bb.jpg") # Works

    bb_image = tf.image.draw_bounding_boxes(img, bb)
    print(bb_image.shape)
    bb_image = tf.cast(bb_image, tf.uint8)
    bb_img_jpeg = tf.image.encode_jpeg(bb_image)
    bb_image_np = np.asarray(bb_img_jpeg)
    Image.fromarray(bb_image_np).save("test.jpg") # Does not work

test_no_bb.jpg gets created fine, but when I reach Image.fromarray(bb_image_np).save("test.jpg") , I get the aforementioned type error.

I have searched the web all over to no avail, and TensorFlow's documentation on this is lacking. The shape of the bb_image is correct, and the output of bb (the coordinates of the bounding box) is also correct so I am at a loss.

Any help is greatly appreciated.

Answer 1

Look at the doc of https://www.tensorflow.org/api_docs/python/tf/image/encode_jpeg , it returns encoded (ie, compressed, including necessary format headers etc..) image serialized into the tensor string.

First, you may need to get the actual data from the tensor, ie, something like this data = bb_img_jpeg.eval() , this should end up having a Python string type (please check that). Then you write this to a file:

with open('test.jpg', 'wb') as f: f.write(data)

Note that I am not sure if the returned string data is string of bytes or chars, in the latter case you should convert it to bytes using data.encode().

Alternatively you may skip the jpeg encoding of TF and leave it to PIL, ie, img_bb = bb_image.eval() , convert that to an array and save it as JPEG using PIL in a similar way you did in the first half of your example.