Python: Combining 2D arrays with 1 common column that has different values

Question

I want to combine two arrays which represent a curve where the variable is column 1, however the column 0 values do not always match:

import numpy as np
arr1= np.array([(12,1003),(17,900),(20,810)])
arr2= np.array([(10,1020),(17,902),(19,870),(21,750)])

I want to combine these into one array where the column 0 is combined and both column 1s are stacked with gaps where there is no value for the corresponding column 0 value, something like this:

arr3=np.array([((10,None,1020),(12,1003,None),(17,900,902),(19,None,870),(20,810,None),(21,None,750))])

The reason for this is that I want to be able to get mean values of the second column for each array but they are not at exactly the same column 0 value so the idea of creating this array is to then interpolate to replace all the None values, then create mean values from column 1 and 2 and have an extra column to represent that.

I have used numPy for everything else so far but obviously have got stuck with the np.column_stack function as it needs lists of the same length and also will be blind to stacking based on values from column o. Lastly I do not want to create a fit for the data as the actual data is non-linear and possibily not consistent so a fit will not work and interpolation seems like the most accurate method.

There may be an answer already but due to me not knowing how to describe it well I can't find it. Also I am relatively new to python so please don't make any assumptions about my knowledge other than it is very little.

Thank you.

Answer 1

will this help ??

import pandas
import numpy as np

arr1= np.array([(12,1003),(17,900),(20,810)])
arr2= np.array([(10,1020),(17,902),(19,870),(21,750)])

d1 = pandas.DataFrame(arr1)
d2 = pandas.DataFrame(arr2)

d1.columns = d2.columns  = ['t','v']
d3 =  pandas.DataFrame(np.array(d1.merge(d2, on='t',how='outer')))
print d3.values

# use d3.as_matrix() to convert to numpy array 

output

[[   12.  1003.    nan]
 [   17.   900.   902.]
 [   20.   810.    nan]
 [   10.    nan  1020.]
 [   19.    nan   870.]
 [   21.    nan   750.]]