如何使用张量流预测图像中所有先前像素的下一个像素

Question

Just as a project, I wanted to see if it would be possible to predict the next pixel in an image given all previous pixels.
For example: lets say I have an image with x pixels. Given the first y pixels, I want to be able to somewhat accurately predict the y+1th pixel. How should I go about solving this problem.

Answer 1

You are looking for some kind of generative model. RNNs are commonly used, and there's a great blog post here demonstrating character-by-character text generation.

The same principle can be applied to any ordered sequence. You talk about an image as being a sequence of pixels, but images have an intrinsic 2D structure (3 if you include color) that would be lost if you took the exact same approach as text generation. A couple of ideas:

1. Use tensorflow's GridLSTMCell s
2. Treat a column of pixels as a single element of the sequence and predict column-by-column (or row by row)
3. Combine idea 2 with some 1D convolutions along the column/row
4. Use features from a section of an image as the seed to a generative adversarial network . See this repository for basic implementations.