TF Serving Predict API Output Interpretation

Question

Is the TensorFlow Serving (TFS) Predict API output the same as the tf.keras.model.predict method (ie the outputs of the model according to the compiled metrics)?

For example, if we have a tf.keras.model compiled with BinaryAccuracy metric, will the output of the TFS predict API be a list of binary accuracy values for each one of the inputs of the predict request?

Thanks in advance!

Answer 1

I am not able to clearly get your question about compiled metrics and the output prediction of the model. But here's the comparision of outputs from Keras predict method and TF Serving's Predict API .

The output format of prediction for both Keras and TF Serving Predict API is similar, which emits a list of probability values of the data point belonging to each class.

Consider that you have a 10 class classification model and you're sending 4 data points to predict method, The output will be of shape 4x10 , wherein for each data point the predicted result contains the probability of that data point belonging to each classes(0–9).

Here's a sample prediction

predictions =  [
 [8.66183618e-05 1.06925681e-05 1.40683464e-04 4.31487868e-09
  7.31811961e-05 6.07917445e-06 9.99673367e-01 7.10965661e-11
  9.43153464e-06 1.98050812e-10],
 [6.35617238e-04 9.08200348e-10 3.23482091e-05 4.98994159e-05
  7.29685112e-08 4.77315152e-05 4.25152575e-06 4.23201502e-10
  9.98981178e-01 2.48882337e-04],
 [9.99738038e-01 3.85520025e-07 1.05982785e-04 1.47284098e-07
  5.99268958e-07 2.26216093e-06 1.17733900e-04 2.74483864e-05
  3.30203284e-06 4.03360673e-06],
 [3.42538192e-06 2.30619257e-09 1.29460409e-06 7.04832928e-06
  2.71432992e-08 1.95419183e-03 9.96945918e-01 1.80040043e-12
  1.08795590e-03 1.78136176e-07]]

You can take a look at the output of make_prediction() function in this reference to understand about how the Predict API in TF Serving works. Thank you!