Tensorflow服务：如何在输入占位符上进行迭代

Question

I am using Tensorflow to build a RNN (GRU) model, now model is trained and I need to deploy it by Tensorflow Serving for prediction service.
So I create a signature_def function as below, the function need to get ids as input, which length is not fixed (so I set shape as [None]), inside the function, the id in ids is picked one by one to feed into GRU cell. The problem is, with the None shape, I cannot figure out how to iterate over all ids

def signature_def(self):
    ids = tf.placeholder(tf.int32, [None], name='input')

    state = [np.zeros([1, self.rnn_size], dtype=np.float32) for _ in range(self.layers)]

    for i in range(<length_of_ids>):
        id = [ids[i]]
        inputs = tf.nn.embedding_lookup(self.embedding, id)
        output, state = self.stacked_cell(inputs, tuple(state))
    logits = tf.matmul(output, self.softmax_W, transpose_b=True) + self.softmax_b
    outputs = self.final_activation(logits)

    tensor_info_x = tf.saved_model.utils.build_tensor_info(ids)
    tensor_info_y = tf.saved_model.utils.build_tensor_info(outputs)

    return tf.saved_model.signature_def_utils.build_signature_def(
            inputs={'ids': tensor_info_x},
            outputs={'preds': tensor_info_y},
            method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME)

I ever tried tf.map_fn, which report error as ".../dropout/mul is in a while loop" I also tried add another input parameter to pass ids length as below, but seems the length parameter cannot change the for loop, it still as the default value 3:

def signature_def(self):
    ids = tf.placeholder(tf.int32, [None], name='input')
    length = tf.placeholder_with_default([3], [1], name='length')

    state = [np.zeros([1, self.rnn_size], dtype=np.float32) for _ in range(self.layers)]

    for i in range(length.eval()[0]):
        id = [ids[i]]
        inputs = tf.nn.embedding_lookup(self.embedding, id)
        output, state = self.stacked_cell(inputs, tuple(state))
    logits = tf.matmul(output, self.softmax_W, transpose_b=True) + self.softmax_b
    outputs = self.final_activation(logits)

    tensor_info_x = tf.saved_model.utils.build_tensor_info(ids)
    tensor_info_l = tf.saved_model.utils.build_tensor_info(length)
    tensor_info_y = tf.saved_model.utils.build_tensor_info(outputs)

    return tf.saved_model.signature_def_utils.build_signature_def(
            inputs={'ids': tensor_info_x, 'length': tensor_info_l},
            outputs={'preds': tensor_info_y},
            method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME)

Any advise or guidance would be greatly appreciated

Thanks!

Answer 1

Possibly a similar situation which I have to solve. See here: tensorflow serving input function for sliding window over timeseries data . The code in this question does iterate over input timeline (sliding window) using a tf.while_loop. However, it runs into another issue related to the number of examples not matching the serving input. I have not yet found a solution to this subsequent issue.