ML.net - error when consuming ONNX in ML.net

Question

Using ML.net in my .NET core app I'm trying to consume a KERAS LSTM model that I exported to an ONNX file. Here is my code:

using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Transforms.Onnx;

public void getmodel(mydata[] data1)
{
    string modelPath = "C:\\MyStuff\\ONNXtest.onnx";
    MLContext mlContext = new MLContext();

    IDataView data = mlContext.Data.LoadFromEnumerable<mydata>(data1);

    OnnxScoringEstimator pipeline = mlContext.Transforms.ApplyOnnxModel(new[] { "output" }, new[] { "input" }, modelPath);

    IEnumerable<mydata> testdata = mlContext.Data.CreateEnumerable<mydata>(data, reuseRowObject: true);
    foreach (mydata row in testdata)
    {
        System.Diagnostics.Debug.WriteLine(row.myval[0]);
        System.Diagnostics.Debug.WriteLine(row.myval[1]);
        System.Diagnostics.Debug.WriteLine(row.myval[2]);
        System.Diagnostics.Debug.WriteLine(row.myval[3]);
        System.Diagnostics.Debug.WriteLine(row.myval[4]);
    }

    OnnxTransformer test = pipeline.Fit(data);

    IDataView transformedValues = test.Transform(data);

    IEnumerable<float[]> results = transformedValues.GetColumn<float[]>("output");

    double result = Convert.ToDouble(results.ElementAtOrDefault(0).GetValue(0));

}

This is how the mydata class looks like:

public class mydata
{
     [VectorType(1,5,1)]
     [ColumnName("input")]   
     public float[] myval { get; set; }
}

I want to feed 5 values in the model and looking at the "System.Diagnostics.Debug.WriteLine" output it seems like everything works and the IDataView data contains 5 values for being fed into the model. However the pipeline.Fit(data) line causes a "System.ArgumentOutOfRangeException" in Microsoft.ML.OnnxTransformer.dll error.

Here is also the code that trains and exports the LSTM in python:

regressor = Sequential()
regressor.add(LSTM(units = 50, return_sequences = True, input_shape = (X_train.shape[1], 1),name ='input'))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units = 50))
regressor.add(Dropout(0.2))

regressor.add(Dense(units = 1,name ='output'))
regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')
regressor.fit(X_train, y_train, epochs = 100, batch_size = 32)

from winmltools import convert_keras
model_onnx = convert_keras(regressor, 7, name='sequential_7')
from winmltools.utils import save_model
save_model(model_onnx, 'C:\\MyStuff\\ONNXtest.onnx')

This is the summary of the network that is exported:

Model: "sequential_7"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input (LSTM)                 (None, 5, 50)             10400     
_________________________________________________________________
dropout_25 (Dropout)         (None, 5, 50)             0         
_________________________________________________________________
lstm_21 (LSTM)               (None, 5, 50)             20200     
_________________________________________________________________
dropout_26 (Dropout)         (None, 5, 50)             0         
_________________________________________________________________
lstm_22 (LSTM)               (None, 5, 50)             20200     
_________________________________________________________________
dropout_27 (Dropout)         (None, 5, 50)             0         
_________________________________________________________________
lstm_23 (LSTM)               (None, 50)                20200     
_________________________________________________________________
dropout_28 (Dropout)         (None, 50)                0         
_________________________________________________________________
output (Dense)               (None, 1)                 51        
=================================================================
Total params: 71,051
Trainable params: 71,051
Non-trainable params: 0

If I just consume the model in Python it works perfectly and a very similar setup (in fact I think it was the same code) a while ago worked perfectly in ML.net. But now I'm not even sure whether my error is on the python or on the c# side. Can anyone help me to figure out how to handle this error?

Answer 1

From what I understand this is an error on the C# side due to ML.NET not allowing for multidimensional input arrays. The documentation of VectorTypeAttribute(dims) is unclear on this point but seems to be an issue that will be fixed soon. See: https://github.com/dotnet/machinelearning/issues/5273