How to iterate through a NumPy array while filtering cells values by index and performing math on it

Question

I am trying to filter/loop and perform math within the same iteration but cant seem to find the right answer. I have a numpy array, that is size 6, 2, and consists of two values that I want to minus together, however I want the values filtered before the minus process commences.

So if the value is greater than in the other column, then the lowest value should be subtracted from the high value, and vice versa . Also this needs to happen in a loop which iterates through the array while performing the filtering and math.

This is my code example:

#minus price
print('minus price trying appending')
minus_p_orgp1 = np.append(dif_p_times1, fp, axis=0)
print(minus_p_orgp1)

for ii, vv in enumerate(minus_p_orgp1):
    print('greater')
    greater_1 = np.all(ii > 0, axis=0)
    greater_0 = np.all(ii <= 0, axis=0)
    if greater_1 < greater_0:
        iit = greater_0 - greater_1
    if greater_1 > greater_0:
        iit = greater_1 - greater_0
        print(iit, ii, vv)

ssss = np.zeros(minus_p_orgp1.size - 1)
for i in range(len(minus_p_orgp1) - 1):
    if minus_p_orgp1[i] < minus_p_orgp1[i]:
        ssss[i] = minus_p_orgp1[i + 1] - minus_p_orgp1[i]
    elif minus_p_orgp1[i + 1] > minus_p_orgp1[i]:
        ssss[i] = minus_p_orgp1[i] - minus_p_orgp1[i + 1]
print(ssss)

This is a print of the array where the upper vector is def_p_time1, and lower vector is fp:

  minus price trying appending
[[79340.33057205 78379.24102508 72188.80527274 76557.26239563
  72857.90423589 71137.7943199 ]
 [43528.22       43705.         43931.07       44571.24
  44330.43       44465.64      ]]

What can I do to achieve my goal?

I have also tried to do the process with just having the array being two separate vectors with size 6, 1. But that also seems very difficult, let me know what you think.

I have also just tried this; however it just prints out zeros when running the code:

trii = np.array([[0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1]])
print(trii)
print(minus_p_orgp1[~(trii >= 1)])
print('it works')
itt = minus_p_orgp1[~(trii >= 1)]
itt1 = minus_p_orgp1[~(trii >= 0)]
sssss = np.zeros(dif_p_times1.size - 1)
ssss = np.zeros(minus_p_orgp1.size - 1)
for i in range(len(dif_p_times1) - 1):
    for ii in range(len(fp) - 1):
        if itt < itt1:
            sssss[i] = itt[i] + itt1[i + 1]
            ssss[i, ii] = fp[ii + 1] - dif_p_times1[i]

        elif itt > itt1:
            sssss[i] = itt[i + 1] + itt1[i]
            ssss[i, ii] = dif_p_times1[i] - fp[ii + 1]
print(sssss)

[[0 0 0 0 0 0]
 [1 1 1 1 1 1]]
[63455.70703442 68744.47486851 77804.44752373 79686.34612013
 69322.78250338 83255.08459329]
it does something
[0. 0. 0. 0. 0.]

Here is a new attempt, however it still doesn't work:

ssss = np.zeros(minus_p_orgp1.size - 1)
x = minus_p_orgp1[::2]
y = minus_p_orgp1[::-2]
z = ssss[::2]

for z, x, y in range(len(minus_p_orgp1) - 1):
    
    if x[i + 1] < y[i]:
        z[i] = y[i + 1] - x[i]
    elif x[i + 1] > y[i]:
        z[i] = x[i + 1] - y[i]
print(z)

Is there a way to iterate through multidimensional lists which has real values not created from the reshape/arrange functions and still being able to perform boolean filtering on individual cells, from where you perform a math operation?

I have looked at nditer, and numba, but all seems to do a version where they transpose the dimensions of a 2d array and reduce the values, which with real values only returned me 1 value, and not an array.

Answer 1

Looking at your first block

minus_p_orgp1 = np.append(dif_p_times1, fp, axis=0)
print(minus_p_orgp1)

looks like minus_p_orgp1 is a (2,6) array, just a row join of the 2 arrays.

for ii, vv in enumerate(minus_p_orgp1):
    print('greater')
    greater_1 = np.all(ii > 0, axis=0)
    greater_0 = np.all(ii <= 0, axis=0)
    if greater_1 < greater_0:
        iit = greater_0 - greater_1
    if greater_1 > greater_0:
        iit = greater_1 - greater_0
        print(iit, ii, vv)

vv is, iteratively, the 2 rows of minus_p_orgp1 . ii is, iteratively, 0 and 1. So this np.all(ii>0...) business is just testing whether ii is 0 or . It sets iit in each loop, but doesn't save the value anywhere. At the end of the loop it has the last value, but so what?

In the following size is 12 (2*6), so ssss in np.zeros(11)

But the iteration is over 2-1, ie it just evaluates for i=0 :

ssss = np.zeros(minus_p_orgp1.size - 1)
for i in range(len(minus_p_orgp1) - 1):
    if minus_p_orgp1[i] < minus_p_orgp1[i]:
        ssss[i] = minus_p_orgp1[i + 1] - minus_p_orgp1[i]
    elif minus_p_orgp1[i + 1] > minus_p_orgp1[i]:
        ssss[i] = minus_p_orgp1[i] - minus_p_orgp1[i + 1]
print(ssss)

minus_p_orgp1[i] < minus_p_orgp1[i] is an array of 6 False. I expect that to raise an ambiguity error when used in the if clause.

minus_p_orgp1[i + 1] > minus_p_orgp1[i] makes more sense, but still can't be used in elif . Are you taking the difference across columns or rows?

The initial word description is a bit vague, but it sure sounds like you just want the positive difference between two arrays:

In [68]: x = np.array([1,3,2,4]); y = np.array([2,1,3,3])
In [69]: x-y
Out[69]: array([-1,  2, -1,  1])
In [70]: y-x
Out[70]: array([ 1, -2,  1, -1])
In [71]: np.abs(x-y)
Out[71]: array([1, 2, 1, 1])

an iterative equivalent (which would work with lists just as well):

In [72]: z = np.zeros(4, int)
In [73]: for i in range(4):
    ...:     if x[i]>y[i]:
    ...:         z[i] = x[i]-y[i]
    ...:     else:
    ...:         z[i] = y[i]-x[i]
    ...: 
In [74]: z
Out[74]: array([1, 2, 1, 1])

or simply:

In [75]: [abs(i-j) for i,j in zip(x,y)]
Out[75]: [1, 2, 1, 1]

Answer 2

As per hpaulj's answer, this worked:

[abs(i-j) for i,j in zip(x,y)]