Python/Numpy：向量化行元素与条件的组合

Question

Is there a way to vectorize the combining of row elements with certain conditions?

Conditions:

1. Empty elements get dropped
2. Rows with more than 1 non-empty element get delimited by '\n'

Note that a) my array has a variable number of rows and columns and will grow quite large hence my interest in vectorization here b) each non-empty string element starts with a '$' character

arr = np.array([       
        ['',  '',  '$c'],
        ['',  '$b', '' ],
        ['',  '$b', '$c'],
        ['$a', '',  '' ],
        ['$a', '',  '$c'],
        ['$a', '$b', '' ],
        ['$a', '$b', '$c']
    ], dtype='U1')

Desired result:

res = [       
        ['$c'],              # <-- reduce to single char element
        ['$b'],              # <-- reduce to single char element
        ['$b\n$c'],           # <-- combine char elements with '\n' delimiter
        ['$a'],              # <-- reduce to single char element
        ['$a\n$c'],           # <-- combine char elements with '\n' delimiter
        ['$a\n$b'],           # <-- combine char elements with '\n' delimiter
        ['$a\n$b\n$c']         # <-- combine char elements with '\n' delimiter
    ]

Any insight into a vectorized approach to achieve the desired end result would be much appreciated. Thank you in advance.

Update:

Due to the differences in requirements, the suggested answer from Reduce multi-dimensional array of strings along axis in Numpy is not the best fit for my use case. See accepted answer below.

Answer 1

Even under your updated circumstances, I would not recommend a numpy-based solution for this, and instead use.

arr = arr.tolist()
empty_removed = [[el for el in row if el != ''] for row in arr]
result = ["\n".join(row) for row in empty_removed]

Even for your small example, you can already see a significant speed difference compared to your solution in the comment:

# array solution
timeit.timeit("['\\n'.join(sub[sub != '']) for sub in arr]", "from __main__ import arr")
# time: 13.177253899999982

# list solution (with initial cast to list)
timeit.timeit("['\\n'.join(row) for row in [[el for el in row if el != ''] for row in arr.tolist()]]", "from __main__ import arr") 
# time: 1.9387359000000117

# list solution (if you can avoid the array in the beginning)
timeit.timeit("['\\n'.join(row) for row in [[el for el in row if el != ''] for row in arr_list]]", "from __main__ import arr_list")
# time 1.4084819999999922

If you want to convert it into a numpy array afterwards to use np.tile and np.repeat , this can certainly be done. However, I would test if that doesn't cause a similar slowdown in your pipeline.

---

Old answer, for reference reasons

I suggest you do not use NumPy arrays and instead switch to plain and simple list comprehension:

arr = arr.tolist() # if you can avoid array creation, even better
result = ['\n'.join(sub) for sub in [''.join(sub) for sub in arr]]
# or if you need the list wrapping the individual elements
result2 = [['\n'.join(sub)] for sub in [''.join(sub) for sub in arr]]

The reason for this is a little more complicated. The gist of it is that numpy can't accelerate array operations on dtype=object in the same way as it can on dtype=np.number . You get the same convenience of fancy indexing (advanced indexing is the name now I think) and tuple-based indexing, but actual performance will not compare. You can get some intuition here: https://stackoverflow.com/a/20942032/6753182