Concatenating 'N' 2D arrays in NumPy with varying dimensions into one 3D array

Question

I have N samples of 2D features with variable dimensions along one axis. For example:

• Sample 1 : (100,20)

• Sample 2 : (150,20)

• Sample 3 : (90,20)

  Is there a way to combine all N samples into a 3D array so that the first dimension (N,?,?) denotes the sample number?

PS: I wish to avoid padding and reshaping, and want to find a way to input the features with their dimensions intact into an LSTM network in Keras. Any other suggestions to achieve the same are welcome.

Answer 1

Keras does allow for variable length input to an LSTM but within a single batch all inputs must have the same length. A way to reduce the padding needed would be to batch your input sequences together based on their length and only pad up to the maximum length within each batch. For example you could have one batch with sequence length 100 and another with sequence length 150. But I'm afraid there is no way to completely avoid padding. During inference you can use any sequence length.