How to pass input data to an existing tensorflow 2.x model in Java?

Question

I'm doing my first steps with tensorflow . After having created a simple model for MNIST data in Python, I now want to import this model into Java and use it for classification. However, I don't manage to pass the input data to the model.

Here is the Python code for model creation:

 from tensorflow.keras.datasets import mnist
 from tensorflow.keras.utils import to_categorical.

 (train_images, train_labels), (test_images, test_labels) = mnist.load_data()

 train_images = train_images.reshape((60000, 28, 28, 1))
 train_images = train_images.astype('float32')
 train_images /= 255

 test_images = test_images.reshape((10000, 28, 28, 1))
 test_images = test_images.astype('float32')
 test_images /= 255

 train_labels = to_categorical(train_labels)
 test_labels = to_categorical(test_labels)

 NrTrainimages = train_images.shape[0]
 NrTestimages = test_images.shape[0]

 import os
 import numpy as np

 from tensorflow.keras.callbacks import TensorBoard
 from tensorflow.keras.models import Sequential
 from tensorflow.keras.layers import Dense, Dropout, Flatten
 from tensorflow.keras.layers import Conv2D, MaxPooling2D
 from tensorflow.keras import backend as K

 # Network architecture
 model = Sequential()
 mnist_inputshape = train_images.shape[1:4]

 # Convolutional block 1
 model.add(Conv2D(32, kernel_size=(5,5), 
       activation = 'relu',
       input_shape=mnist_inputshape,
       name = 'Input_Layer'))
 model.add(MaxPooling2D(pool_size=(2,2)))
 # Convolutional block 2
 model.add(Conv2D(64, kernel_size=(5,5),activation= 'relu'))
 model.add(MaxPooling2D(pool_size=(2,2)))
 model.add(Dropout(0.5))

 # Prediction block
 model.add(Flatten())
 model.add(Dense(128, activation='relu', name='features'))
 model.add(Dropout(0.5))
 model.add(Dense(64, activation='relu'))
 model.add(Dense(10, activation='softmax', name = 'Output_Layer'))

 model.compile(loss='categorical_crossentropy',
              optimizer='Adam',
              metrics=['accuracy'])

 LOGDIR = "logs"
 my_tensorboard = TensorBoard(log_dir = LOGDIR,
       histogram_freq=0,
       write_graph=True,
       write_images=True)
 my_batch_size = 128
 my_num_classes = 10
 my_epochs = 5

 history = model.fit(train_images, train_labels,
       batch_size=my_batch_size,
       callbacks=[my_tensorboard],
       epochs=my_epochs,
       use_multiprocessing=False,
       verbose=1,
       validation_data=(test_images, test_labels))

 score = model.evaluate(test_images, test_labels)

 modeldir = 'models'
 model.save(modeldir, save_format = 'tf')

For Java , I am trying to adapt the App.java code published here .

I am struggling with replacing this snippet:

 Tensor result = s.runner()
      .feed("input_tensor", inputTensor)
      .feed("dropout/keep_prob", keep_prob)
      .fetch("output_tensor")
      .run().get(0);

While in this code, a particular input tensor is used to pass the data, in my model, there are only layers and no individual named tensors. Thus, the following doesn't work:

 Tensor<?> result = s.runner()
      .feed("Input_Layer/kernel", inputTensor)
      .fetch("Output_Layer/kernel")
      .run().get(0);

How do I pass the data to and get the output from my model in Java?

Answer 1

I finally managed to find a solution. To get all the tensor names in the graph, I used the following code:

        for (Iterator it = smb.graph().operations(); it.hasNext();) {
            Operation op = (Operation) it.next();
            System.out.println("Operation name: " + op.name());
        }

From this, I figured out that the following works:

        SavedModelBundle smb = SavedModelBundle.load("./model", "serve");
        Session s = smb.session();

        Tensor<Float> inputTensor = Tensor.<Float>create(imagesArray, Float.class);
        Tensor<Float> result = s.runner()
                .feed("serving_default_Input_Layer_input", inputTensor)
                .fetch("StatefulPartitionedCall")
                .run().get(0).expect(Float.class);

Answer 2

With the newest version of TensorFlow Java , you don't need to search for yourself the name of the input/output tensors from the model signature or from the graph. You can simply call the following:

try (SavedModelBundle model = SavedModelBundle.load("./model", "serve");
     Tensor<TFloat32> image = TFloat32.tensorOf(...); // There a many ways to pass you image bytes here
     Tensor<TFloat32> result = model.call(image).expect(TFloat32.DTYPE)) {
    System.out.println("Result is " + result.data().getFloat());
  }
}

TensorFlow Java will automatically take care of mapping your input/output tensors to the right nodes.