How to calculate all combinations of difference between array elements in 2d?

Question

Given an array arr = [10, 11, 12] I want to calculate all the ways that one element can be subtracted from another. For a 1xN array the desired output is a NxN array where output[i, j] = arr[i] - arr[j] . My approach was to generate all the possible pairings of two numbers, subtract, and reshape. As follows

opts = np.array(list(product(arr, arr)))
[[10 10] 
 [10 11]
 [10 12]
 [11 10]
 [11 11]
 [11 12]
 [12 10]
 [12 11]
 [12 12]]
 diffs = (opts[:, 0] - opts[:, 1]).reshape(len(arr), -1)
 [[ 0 -1 -2]
  [ 1  0 -1]
  [ 2  1  0]]

This works quite nicely, what I would like to do next is to generalize this to a 2d input. Essentially what I would like to accomplish is given an MxN array to output an MxNxN array, and for each layer (depth-wise) perform the above functionality for each row.

I attempted to reshape the MxN input array to be MxNx1 and then calculate the product as before. My assumption was that it would behave element-wise the same as before, unfortunately not.

My first thought is to initialize an output of the appropriate shape and loop over the rows and set the values "manually" but I was hoping for a vectorized approach. Does anyone know how I can accomplish this in 2 dimensions without looping over thousands of rows?

Answer 1

Here's a generic vectorized way to cover both 1D and 2D cases leveraging broadcasting after reshaping the input array to broadcastable shpaes against each other -

def permute_axes_subtract(arr, axis):
    # Get array shape
    s = arr.shape

    # Get broadcastable shapes by introducing singleton dimensions
    s1 = np.insert(s,axis,1)
    s2 = np.insert(s,axis+1,1)

    # Perform subtraction after reshaping input array to 
    # broadcastable ones against each other
    return arr.reshape(s1) - arr.reshape(s2)

To perform any other elementwise ufunc operation, simply replace the subtraction operation with it.

Sample run -

In [184]: arr = np.random.rand(3)

In [185]: permute_axes_subtract(arr, axis=0).shape
Out[185]: (3, 3)

In [186]: arr = np.random.rand(3,4)

In [187]: permute_axes_subtract(arr, axis=0).shape
Out[187]: (3, 3, 4)

In [188]: permute_axes_subtract(arr, axis=1).shape
Out[188]: (3, 4, 4)

Timings on @ClimbingTheCurve's posted solution func - permute_difference and the one posted in this one on large 2D arrays -

In [189]: arr = np.random.rand(100,100)

In [190]: %timeit permute_difference(arr, axis=0)
     ...: %timeit permute_axes_subtract(arr, axis=0)
1 loop, best of 3: 295 ms per loop
1000 loops, best of 3: 1.17 ms per loop

In [191]: %timeit permute_difference(arr, axis=1)
     ...: %timeit permute_axes_subtract(arr, axis=1)
1 loop, best of 3: 303 ms per loop
1000 loops, best of 3: 1.12 ms per loop

Answer 2

The solution is to write a function for the 1d case and in order to generalize use the function np.apply_along_axis() which takes a function, an axis to apply along, and an input array. This works exactly as intended. The code I used:

from itertools import product

import numpy as np


def permute_difference(arr, axis=1):
    """
    applies the _permute_difference to a 2d array
    along the specified axis

    Parameters
    ----------
    arr numpy.array

    Returns
    -------
    numpy.array
        a 3d array, each 2d array the i^th along the depth
        contains the permuted difference of the i^th row
        in the input array
    """
    def _permute_difference(arr):
        """
        calculates all the differences between all combinations
        terms in the input array. output[i,j] = arr[i] - arr[j]
        for every combination if ij.

        Parameters
        ----------
        arr numpy.array
            a 1d input array

        Returns
        -------
        numpy.array
            a 2d array

        Examples
        --------
        arr = [10, 11, 12]

        diffs = [[ 0 -1 -2]
                [ 1  0 -1]
                [ 2  1  0]]
        """
        opts = np.array(list(product(arr, arr)))
        d = (opts[:, 0] - opts[:, 1]).reshape(len(arr), -1)
        return d

    if arr.ndim == 1:
        diffs = _permute_difference(arr)
    else:
        diffs = np.apply_along_axis(permute_difference, axis, arr)
    return diffs