Tensorflow服务重新训练开始

Question

I am trying to serve my retrained inception model following this guide (you may also see this guide , which explains how to retrain inception). I've modified retrain.py to export my model as follows:

... # Same as in the original script:
# Set up the pre-trained graph.
maybe_download_and_extract()
graph, bottleneck_tensor, jpeg_data_tensor, resized_image_tensor = (create_inception_graph())
... # Same as in the original script:
# Add the new layer that we'll be training.
(train_step, cross_entropy, bottleneck_input, ground_truth_input, final_tensor) = add_final_training_ops(len(image_lists.keys()),
                                         FLAGS.final_tensor_name,
                                         bottleneck_tensor)
... # Added at the end of the original script:
# Export model
with graph.as_default():
    export_path = sys.argv[-1]
    print('Exporting trained model to', export_path)
    saver = tf.train.Saver(sharded=True)
    model_exporter = exporter.Exporter(saver)
    signature = exporter.classification_signature(input_tensor=jpeg_data_tensor, scores_tensor=final_tensor)
    model_exporter.init(sess.graph.as_graph_def(), default_graph_signature=signature)
    model_exporter.export(export_path, tf.constant(FLAGS.export_version), sess)
    print('Done exporting!')

if __name__ == '__main__':
  tf.app.run()

After exporting my model I start running the server:

/serving/bazel-bin/tensorflow_serving/example/inception_inference --port=9000 EXPORT_DIR &> inception_log &

Server log file (inception_log) contains:

I tensorflow_serving/core/basic_manager.cc:190] Using InlineExecutor for BasicManager.
I tensorflow_serving/example/inception_inference.cc:384] Waiting for models to be loaded...
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:129] Attempting to load a SessionBundle from: /tf_files/scope/export/00000001
I tensorflow_serving/example/inception_inference.cc:384] Waiting for models to be loaded...
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:106] Running restore op for SessionBundle
I external/org_tensorflow/tensorflow/contrib/session_bundle/session_bundle.cc:203] Done loading SessionBundle
I tensorflow_serving/example/inception_inference.cc:350] Running...
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:147] File-system polling found servable version {name: default version: 1} at path /tf_files/scope/export/00000001
... 

Finally, I run the client and I get the following error:

/serving/bazel-bin/tensorflow_serving/example/inception_client --server=localhost:9000 --image=TEST_IMG
D0805 09:10:46.208704633     200 ev_posix.c:101]             Using polling engine: poll
Traceback (most recent call last):
  File "/serving/bazel-bin/tensorflow_serving/example/inception_client.runfiles/tensorflow_serving/example/inception_client.py", line 53, in <module>
    tf.app.run()
  File "/serving/bazel-bin/tensorflow_serving/example/inception_client.runfiles/external/org_tensorflow/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "/serving/bazel-bin/tensorflow_serving/example/inception_client.runfiles/tensorflow_serving/example/inception_client.py", line 48, in main
    result = stub.Classify(request, 10.0)  # 10 secs timeout
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 300, in __call__
    self._request_serializer, self._response_deserializer)
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 198, in _blocking_unary_unary
    raise _abortion_error(rpc_error_call)
    grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.INTERNAL, details="FetchOutputs node : not found")
E0805 09:10:47.129263239     200 chttp2_transport.c:1810]    close_transport: {"created":"@1470388247.129230608","description":"FD shutdown","file":"src/core/lib/iomgr/ev_poll_posix.c","file_line":427}

Any advice or guidance in this matter would be greatly appreciated.

Answer 1

So the link on the tensorflow website is just one way to fully serve the model from my experience. A better way to serve the model would be to serve it up from Flask and Kubernetes instead, since its a lighter weight client than all of the tensorflow serving infrastructure, but this is assuming that the volume you have isn't very large(>100 QPS) although you could serve Inception up with Flask and Kubernetes with that load, but at that rate I would op for the in-line solution.

You could serve it from a remote service, and that would work, but depending on your infrastructure you "could" also serve that model up in a streaming job that pushed your requests through an apache_beam.DoFn then outputting it out back to a MQ that your job was listening to. This is just another solution. Hope this helps.

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import logging 
import tensorflow as tf
import numpy as np
import apache_beam as beam


class InferenceFn(beam.DoFn):

  def __init__(self, model_dict):
    super(InferenceFn, self).__init__()
    self.model_dict = model_dict
    self.graph = None
    self.create_graph()


  def create_graph(self):
    if not tf.gfile.FastGFile(self.model_dict['model_full_path']):
      self.download_model_file()
    with tf.Graph().as_default() as graph:
      with tf.gfile.FastGFile(self.model_dict['model_full_path'], 'rb') as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
        _ = tf.import_graph_def(graph_def, name='')
    self.graph = graph

  def start_bundle(self):
    """Prevents graph object serialization until serving. Required for GCP Serving"""
    self.create_graph()

  def process(self, element):
    """Core Processing Fn for Apache Beam."""
    try:
      with tf.Session(graph=self.graph) as sess:
        if not tf.gfile.Exists(element):
          tf.logging.fatal('File does not exist %s', element)
          raise ReferenceError("Couldnt Find the image {}".format(element))
        data = tf.gfile.FastGFile(element, 'rb').read()
        output_tensor = sess.graph.get_tensor_by_name(self.model_dict['output_tensor_name'])
        predictions = sess.run(softmax_tensor, {self.model_dict['input_tensor_name']: data})
        predictions = np.squeeze(predictions)
        yield str(predictions)
    except Exception:
      logging.error("We hit an error in inference on {}".format(element))