使用图像 package 并发的奇怪异常行为

Question

The program im trying to get working is a generator for images of 1D cellular automate and it needs to be robust enough to handle extremely large simulations on orders of several millions of individual cells so multi-threading the image generation process is necessary. I chose Go for this reason because go-routines were going to make the issue of dividing work for the CPU much easier and efficient. Now because writing each cell with a individual go-routine would not be very performant at all i decided to create a function that calls the image object and is responsible for generating an entire row of cells instead. This function is referencing a 2D array object containing a bitsliced ( see this ) array of all the cells to be drawn hence the many loops however this is not important to the issue at hand. What the program is supposed to do is simply read all the individual bits and write a square to the image rectangle in the correct position denoting the presence of a cell (based on the variable pSize noting the side length of the square). Here is that function...

func renderRow(wg *sync.WaitGroup, img *image.RGBA, i int, pSize int) {
    defer wg.Done()
    var lpc = 0
    for j := 0; j < 64; j++ {
        for k := range sim[i] {
            for l := lpc * pSize; l <= (lpc*pSize)+pSize; l++ {
                for m := i * pSize; m <= (i*pSize)+pSize; m++ {
                    if getBit(sim[i][k], j) == 1 {
                        img.Set(l, m, black)
                    } else {
                        img.Set(l, m, white)
                    }
                }
            }
            lpc++
        }
    }
}

Now im happy to say that this function here performs just as expected when run sequentially on one thread. Here is the non parallel function call (ignoring the waitgroup)

img = image.NewRGBA(image.Rectangle{Min: upLeft, Max: lowRight})

for i := range sim {
    renderRow(&wg, img, i, pSize)
}

f, _ := os.Create("export/image.png")
_ = png.Encode(f, img)

Now on the other hand when we make the simple change to a concurrent implementation the output has several individual pixel errors and seems to shrink and extend certain rows randomly as the amount of errors changes with each run. Here's the concurrent function call. Here's the concurrent function call...

img = image.NewRGBA(image.Rectangle{Min: upLeft, Max: lowRight})

for i := range sim {
    go renderRow(&wg, img, i, pSize) // TODO make multithreaded again
}

wg.Wait()

f, _ := os.Create("export/image.png")
_ = png.Encode(f, img)

Now what does the output look like for these two respective implementations? using these starting conditions {0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1} and a evolution space of 11 (pSize 2 ). We get this as output from the single thread implementation...

Now if you zoom in on that image you'll find all the squares all evenly spaced vertically and horizontally with no anomalies. However now lets take a look at the concurrent output.

This version seems to have several anomalies many rows have been shrunk there are individual pixel errors in many places and although it follows the general pattern of the simulation correctly it is most certainly not visually pleasing. While i was investigating this issue i looked for issues related to concurrency and so i thought that perhaps a dynamic allocation of the pixel array in the image package might be causing conflicts of some sort and so i investigated img.Set() which looks like this...

func (p *NRGBA) Set(x, y int, c color.Color) {
    if !(Point{x, y}.In(p.Rect)) {
        return
    }
    i := p.PixOffset(x, y)
    c1 := color.NRGBAModel.Convert(c).(color.NRGBA)
    s := p.Pix[i : i+4 : i+4] // Small cap improves performance, see https://golang.org/issue/27857
    s[0] = c1.R
    s[1] = c1.G
    s[2] = c1.B
    s[3] = c1.A
}

However when i look at this it seems to make no sense. As it appears that img.Pix element is storing all the pixel data in a sequential 1D array of integers representing colors but the .Set() function immediately returns if the (x,y) elements passed to it are already found in the.Pix slice. But whats even more strange is what appears to be some sort of implicit assignment (which iv'e never seen in Go) where 4 elements of the.Pix slice are taken out representing an individual pixel's color and assigned to s . And the strangest part being that s , c1 and i are never referenced again, returned, or stored in memory simply thrown to garbage collection. But somehow this function appears to work sequentially so i just decided to let it do its thing and take a look at what the differences were in the .Pix slice between the concurrent and non concurrent implementations.

Now here's the links to four paste bins, they contain the img.Pix objects data for 2 separate trials arranged with each row belonging to an individual pixel's colors starting from the top left of each image and moving down. The reason for two trials is to verify consistency for the single threaded approach which appears to be consistent but as you can observe by going to a website like diffchecker.com is that both the multi threaded tests show differences between them and the single threaded output.

Multithreaded Test 1

Single-threaded Test 1

Multithreaded Test 2

Single-threaded Test 2

Now here I'll share some observations about this data.

• There are differences and different quantities of differences between the different multi-threaded and the single-threaded tests
• there are identical quantities of additions and deletions between single thread and multithread implying that all the data is present and that its simply in the wrong order.

Now these observations may imply that as we call the Set function threads are colliding with each other on certain indices in the Pix array but from looking at the set function every single pixel is supposed to have a distinct place in the array which is preallocated based on the length and width of the provided rectangle which should make ordering absolute and collisions impossible between threads. Heres the function thats responsible for creating the image object...

// NewRGBA returns a new RGBA image with the given bounds.
func NewRGBA(r Rectangle) *RGBA {
    return &RGBA{
        Pix:    make([]uint8, pixelBufferLength(4, r, "RGBA")),
        Stride: 4 * r.Dx(),
        Rect:   r,
    }
}

So all in all I really have no idea whats going on. There seems to be some weird behaviors arising from the image package as multiple go-routines access the same slice but since the indices of the slice are theoretically absolute (meaning unique for each variable) there shouldn't be any ordering issues. The only possible issue i could think of is that the slice despite being defined in the manner it was is somehow being resized by that set function or at least shifted around causing collisions. Any help figuring out whats going wrong or any theories about what might be causing the problem are greatly appreciated. Cheers!

Answer 1

The code above produces many race conflicts arising from go-routines attempting to write to the same pixel coordinate in the.Pix object. The fix was within the renderRow function where the calculations for the width and height of the current pixel were overlapping on each iteration due to <= instead of '<'. Moral of the story is use -race to look for collisions and always look for overwrites or concurrent reads of the same variable. Credit to @rustyx.