Issue with opening h5 file with Python code in MATLAB environment

Question

I have an issue with calling Python code in MATLAB. My Python code involves predicting the battery state of charge using LSTM with attention ANN based on the inputs sent from MATLAB. The prediction is then sent back to MATLAB. I already have previously trained weights and biases saved in an h5 file, which is loaded and used in the Python code. Below is the Python code:

    import pandas as pd
    import numpy as np
    import tensorflow as tf
    from sklearn.model_selection import train_test_split
    from sklearn.metrics import mean_squared_error
    from tensorflow import keras
    from tensorflow.keras import Sequential
    from tensorflow.keras.layers import LSTM
    from tensorflow.keras.layers import Dense
    from tensorflow.keras import optimizers
    import matplotlib.pyplot as plt
    from tensorflow.keras.layers import *
    from tensorflow.keras.models import *
    from tensorflow.keras import backend as K
    import matplotlib.pyplot as plt
    import matplotlib.ticker as ticker
    from sklearn.model_selection import train_test_split
    from tensorflow.keras.callbacks import EarlyStopping
    from tensorflow.keras.callbacks import ModelCheckpoint
    from tensorflow.keras.models import load_model
    from tensorflow.keras.layers import Dropout, InputLayer 
    import h5py

    #to create sequential data
    def create_inout_sequences(input_data, tw):
inout_seq = []
L = len(input_data)
for i in range(L-tw):
    train_seq = input_data[i:i+tw]
    #train_out = output_data[i:i+tw]
    inout_seq.append(train_seq)
return inout_seq 

def search(inputs):
class attention(Layer):
    def __init__(self, return_sequences=True,**kwargs):
        self.return_sequences = return_sequences
        super(attention,self).__init__()

    def build(self, input_shape):

        self.W=self.add_weight(name="att_weight", shape=(input_shape[-1],1),
                               initializer="normal")
        self.b=self.add_weight(name="att_bias", shape=(input_shape[1],1),
                               initializer="zeros")
        super(attention,self).build(input_shape)

    def call(self, x):
        e=(K.dot(x,self.W)+self.b)
        a = K.softmax(e, axis=1)
        output = x*a
        if self.return_sequences:
            return output
        return K.sum(output, axis=1)

    def get_config(self):
        # For serialization with 'custom_objects'
        config = super().get_config()
        config['return_sequences'] = self.return_sequences     
        return config

#convert test data to sequential form
inputs=np.array(inputs)
inputs=np.tile(inputs, (36, 1))
inputs_new=create_inout_sequences(inputs, 35)
inputs_new=np.array(inputs_new)

model1 = Sequential()
model1.add(InputLayer(input_shape=(35,5)))
model1.add((LSTM(22, return_sequences=True)))
model1.add(attention(return_sequences=False))
model1.add(Dense(104, activation="relu"))
model1.add(Dropout(0.2))
model1.add(Dense(1, activation="sigmoid"))

lr_schedule = keras.optimizers.schedules.ExponentialDecay(
initial_learning_rate=0.01,
decay_steps=10000,
decay_rate=0.99)

model1.compile(optimizer=tf.keras.optimizers.Adam(epsilon=1e-08,learning_rate=lr_schedule),loss='mse')

#call previously trained weights
model1.load_weights('SOC_weights.h5')
x=float(model1.predict(inputs_new, batch_size=100,verbose=0))

return x # send prediction to Matlab

Note: I am using Python 3.6, tensorflow version: 2.5, keras version: 2.4.3, h5py version: 3.1.0, cython version: 0.28

I am able to run this code without any error on Python, but have issues when used in MATLAB 2020a... below is my MATLAB code:

   pyenv('Version','3.6');
   py.importlib.import_module('tensorflow');
   py.importlib.import_module('testingSOC'); % file containing the Python codes
   inputs=[0.555555556,0.435139205,0.68313128,0.499987472,0.241225578];% test inputs
   SOC_output=py.testingSOC.search(inputs)

Below is the error received on Matlab:

    Error using training>load_weights (line 2312)
    Python Error: ImportError: `load_weights` requires h5py when loading weights from HDF5.
    Error in testingSOC>search (line 87)

the error looks like h5py is not identified by MATLAB, so I have tried reinstalling h5py by using the command prompt (I am using Windows 10):

    pip uninstall h5py
    pip install h5py

but no changes...

I have also tried with tensorflow version: 2.2, keras version 2.4.3, h5py version 2.10 and cython version 0.29 but still get the same error.

I would really appreciate if you guys can provide an insight in solving this issue, and if there are any fundamental parts that I have missed. I would be glad to share more details if required.

Thanks!

Answer 1

Thanks to @TimRoberts for pointing out about including 'py.importlib.import_module('h5py')' which helped me in resolving this issue.Below is my solution, for those who would like to refer:

When I included 'py.importlib.import_module('h5py')' in my matlab codes, I received the following error:

    Error using h5>init h5py.h5 (line 1) 
    Python Error: ImportError: DLL load failed: The specified procedure could not be found.

It looks like Python environment seems to use Matlab's h5 library in my case, which does not have the same features as Python's h5 library...I found that there is an option of running Python codes as a separate process which seems to be working for me (as seen in this link): https://www.mathworks.com/help/matlab/matlab_external/out-of-process-execution-of-python-functionality.html?searchHighlight=out%20of%20process%20python&s_tid=srchtitle