带有 keras 多输出模型的 TFX 评估器问题

Question

I am using a very simple keras model in TFX, to solve a regression problem. It seems that TFX wants that you use a keras model with named outputs, so I made:

output = {key: tf.keras.layers.Dense(1, name = key)(x) 
          for key in _transformed_names(_LABEL_KEYS)} 

model(inputs, outputs) 

I don't understand how the evaluator maps the label names of my dataset with the output names of my model.

In my code I set up the label_keys and prediction_keys argument in tfma.ModelSpec with a list of form :

[["model output name", "Label key in my Dataset"]]

It seems that the proto message is created correctly, but when I run the Evaluator I get the following error:

ValueError: unable to prepare labels and predictions because the labels and/or predictions are dicts with unrecognized keys. If a multi-output keras model (or estimator) was used check that an output_name was provided. If an estimator was used check that common prediction keys were provided (e.g. logistic, probabilities, etc)

If I try to provide a single label key and a single prediction key using the label_key and prediction_key arguments I get the following error:

TypeError: update_state() takes from 2 to 3 positional arguments but 4 were given [while running 'ExtractEvaluateAndWriteResults/ExtractAndEvaluate/EvaluateMetricsAndPlots/ComputeMetricsAndPlots()/ComputePerSlice/ComputeUnsampledMetrics/CombinePerSliceKey/WindowIntoDiscarding']

I have tried in all possible ways but nothing. Is there a way to use a model with no named outputs (an Dense output layer with more than one node) ? Or a way to solve this problem ?

PS Is there a Tutorials of a TFX pipeline with a multiple outputs keras model ?

Thanks.

Answer 1

in eval_config,setting

options=Options(include_default_metrics=BoolValue(value=False))

eg:

  eval_config = tfma.EvalConfig(
      model_specs = [...],
      slicing_specs=[tfma.SlicingSpec(),...],
      metrics_specs=[...],
      options=Options(include_default_metrics=BoolValue(value=False))
      )
  evaluator = Evaluator(
      ...
      eval_config=eval_config
  )