在 tensorflow 中使用保存的估计器时出现saved_model_cli 问题

Question

I have been working with BERT large uncased for binary text classification. I was using google colab for training model. To save the estimator i have used following serving_input_funtion :

def serving_input_receiver_fn():
  with tf.variable_scope("foo"):
    feature_spec = {
      "input_ids": tf.FixedLenFeature([128], tf.int64),
      "input_mask": tf.FixedLenFeature([128], tf.int64),
      "segment_ids": tf.FixedLenFeature([128], tf.int64),
      "label_ids": tf.FixedLenFeature([], tf.int64),
    }
    serialized_tf_example = tf.placeholder(dtype=tf.string, shape=None,
                                       name='input_example_tensor')

    receiver_tensors = {'examples': serialized_tf_example}
    features = tf.parse_example(serialized_tf_example, feature_spec)
    return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

and saved estimator with following:

estimator._export_to_tpu = False
estimator.export_saved_model(export_dir_base = "/bert_0.3/",serving_input_receiver_fn = serving_input_receiver_fn)

This saved the estimator but now when am trying to use saved_model_cli to test the estimator, its not working. It is throwing me error like:

ValueError: Type <class 'bytes'> for value b'\n\x12\n\x10\n\x08sentence\x12\x04\n\x02\n\x00' is not supported for tf.train.Feature.

Command was :

saved_model_cli run --dir '/bert_0.3/1564572852' --tag_set serve --signature_def serving_default --input_examples '"examples"=[{"input_ids":[b"\n\x12\n\x10\n\x08sentence\x12\x04\n\x02\n\x00"],"input_mask":[b"\n-\n+\n\x08sentence\x12\x1f\n\x1d\n\x1bThis API is a little tricky"],"segment_ids":[None],"label_ids":["label_1"]}]'

It was not accepting string directly so thats why i manually divided it in dict . After converting it to dict , i realised that this accepts only bytes for bytes_list . This is why i converted strings to bytes format.

Stacktrace

 File "/usr/local/bin/saved_model_cli", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/tools/saved_model_cli.py", line 990, in main
    args.func(args)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/tools/saved_model_cli.py", line 724, in run
    init_tpu=args.init_tpu, tf_debug=args.tf_debug)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/tools/saved_model_cli.py", line 420, in run_saved_model_with_feed_dict
    loader.load(sess, tag_set.split(','), saved_model_dir)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/saved_model/loader_impl.py", line 269, in load
    return loader.load(sess, tags, import_scope, **saver_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/saved_model/loader_impl.py", line 423, in load
    self.restore_variables(sess, saver, import_scope)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/saved_model/loader_impl.py", line 377, in restore_variables
    saver.restore(sess, self._variables_path)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/training/saver.py", line 1290, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/client/session.py", line 956, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/client/session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
    run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: Dst tensor is not initialized.
     [[node save_2/RestoreV2 (defined at /lib/python2.7/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]

Any help would be appreciated if someone could tell me where did this went wrong. problem could be in serving_input_receiver_fn as well i guess.

max sequence length is 128

Thanks.

Edit 1 :

If used tf.placeholder() instead of tf.FixedLenFeature

Answer 1

You can view the saved model using the below command:

saved_model_cli show --all --dir <path to/bert_0.3/1564572852>

This will show the dtype,shape and name of inputs and outputs.

Please try to use tf.placeholder() instead of tf.FixedLenFeature inside serving_input_receiver_fn() as follows:

input_ids = tf.placeholder(tf.int32, [None, 128], name='input_ids')
input_mask = tf.placeholder(tf.int32, [None, 128], name='input_mask')
segment_ids = tf.placeholder(tf.int32, [None, 128], name='segment_ids')

If still getting error while using the saved model, please share the screenshot of the error.

You can refer the following github repo for more details: https://github.com/bigboNed3/bert_serving/tree/44d33920da6888cf91cb72e6c7b27c7b0c7d8815

Hope this helps.Thanks.