对于带有间隙的网格数据，是否有 np.reshape 的替代方法？

Question

I have a dataset which is gridded data which has been flattened into single columns of x, y and z data, but I would like to get it back into a grid to run a 2D fourier transform on it. Unfortunately there are some gaps in the data around the ages of the grid.

For regular grids, using np.reshape would work to put this data back into a grid, but since there are gaps this will not work. Is there a similar function which will allow me to reshape and add in nans?

An example dataset (although my data has over 80,000 entries).

x = np.array([0, 1, 2 , 3, -1, 0, 1, 2, -1, 0, 1, -3, -2,-1,0])
y = np.array([0,1,2, 3, 1, 2,3,4, 3,4, 5, 3, 4,5,6])

which I would like to get into the form:

x = array([[ 0.,  1.,  2.,  3.],
   [-1.,  0.,  1.,  2.],
   [nan, -1.,  0.,  1.],
   [-3., -2., -1.,  0.]])

y = array([[ 0.,  1.,  2.,  3.],
       [ 1.,  2.,  3.,  4.],
       [nan,  3.,  4.,  5.],
       [ 3.,  4.,  5.,  6.]])

This data set plotted as a scatter graph

Answer 1

Here is a partial solution. Currently it assumes that data is missing from the left or right hand side of the grid (IE it doesn't yet handle top and bottom). It also assumes that the data increases across columns (if otherwise, you just need to swap a > for a < and the bodies of the if / else statements), and that the change across rows is less than the change from the first column to the last column (if this is not true then you have a harder problem on your hands). Depending on how you data is generated, you may also want to replace the == with a condition to test whether or not the two values being compared are relatively close/less than the differences between values at the end of the rows.

import numpy as np

x = np.array([0, 1, 2 , 3, -1, 0, 1, 2, -1, 0, 1, -3, -2,-1,0])

def vector_to_array(x, verbose=False):
    # Reformat into list of lists
    x_grid_list = [[x[0]]]
    for i in x[1:]:
        if i > x_grid_list[-1][-1]:
            x_grid_list[-1].append(i)
        else:
            x_grid_list.append([i])
    
    # Calculate width and height
    height = len(x_grid_list)
    width = max(len(i) for i in x_grid_list)

    # Fill in missing data
    for row_ind in range(1, len(x_grid_list[1:])):
        row = x_grid_list[row_ind]
        if len(row) < width:
            if row[0] == x_grid_list[row_ind - 1][0]:
                # Missing data is on the left hand side
                x_grid_list[row_ind] = [np.nan] + row
            else:
                # Missing data is on the right hand side
                x_grid_list[row_ind] = row + [np.nan]

    # Convert to np array, print if verbose, and return
    x_array = np.array(x_grid_list)
    if verbose:
        print(x)
        print(x_grid_list)
        print(x_array)
    return x_array

x = np.array([0, 1, 2 , 3, -1, 0, 1, 2, -1, 0, 1, -3, -2,-1,0])
vector_to_array(x, True)
print("*" * 50)
x = np.array([0, 1, 2 , 3, -1, 0, 1, 2, -2, -1, 0, -3, -2,-1,0])
vector_to_array(x, True)

Output:

[ 0  1  2  3 -1  0  1  2 -1  0  1 -3 -2 -1  0]
[[0, 1, 2, 3], [-1, 0, 1, 2], [nan, -1, 0, 1], [-3, -2, -1, 0]]
[[ 0.  1.  2.  3.]
 [-1.  0.  1.  2.]
 [nan -1.  0.  1.]
 [-3. -2. -1.  0.]]
**************************************************
[ 0  1  2  3 -1  0  1  2 -2 -1  0 -3 -2 -1  0]
[[0, 1, 2, 3], [-1, 0, 1, 2], [-2, -1, 0, nan], [-3, -2, -1, 0]]
[[ 0.  1.  2.  3.]
 [-1.  0.  1.  2.]
 [-2. -1.  0. nan]
 [-3. -2. -1.  0.]]