Inputs to TensorFlow serving REST API

Question

I have build a seq2seq model by referring to this tutorial. https://keras.io/examples/nlp/lstm_seq2seq/

After training the model, instead of saving the model directly, I saved the encoder_model and decoder_model separately.I'm using TensorFlow serving to deploy these two models.

Encoder model input was a numpy array of length 1, so I converted that to JSON to pass to REST API. This worked and I got the desired result.

But, for decoder this method didn't worked as length of decoder_input is 3 and data_type is numpy array.

Following the block of decode_sequence function where decoder_model is called.

dec_model_url = "http://localhost:8400/v1/models/dec_model:predict"
headers = {
    'content-type': "application/json;charset=UTF-8'",
    'cache-control': "no-cache",
    'Accept':'application/json'
    }
    while not stop_condition:
        decoder_ip = ([target_seq] + states_value)
        target_seq1 = target_seq.tolist()
        target_seq1=[target_seq1]

        states_value1 = states_value
        states_value1[0] = states_value1[0].tolist()
        states_value1[1] = states_value1[1].tolist()


        decoder_ip1 = (target_seq1 + states_value1[0] + states_value1[1])
        start_main = '{"instances":'
        end_main = '}'
        decoder_ip1 = start_main + str(decoder_ip1) +end_main


        output_tokens, h = requests.request("POST", dec_model_url, data=decoder_ip1, headers=headers)

When I run this, I got the following error.

{"error": "instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map'

What is the correct way to pass the decoder_model inputs with REST API?

Answer 1

This issue is exactly similar to Github Issue and I have provided workaround for that issue and it got resolved so could you please look into that issue's workaround here ? I hope it will help you to resolve your issue. Thank you!