When using any of the Object Detection models from TensorFlow's Official Models in the ModelZoo, there is an argument called params_override. Based on the code here ( https://github.com/tensorflow/models/blob/master/official/modeling/hyperparams/params_dict.py ), it seems like after creating a ParamsDict with a given a set of default parameters, it then overrides certain parameters. The example that they give on the README.md for the RetinaNet model ( https://github.com/tensorflow/models/tree/master/official/vision/detection ) is --params_override="{ type: retinanet, train: { checkpoint: { path: ${RESNET_CHECKPOINT?}, prefix: resnet50/ }, train_file_pattern: ${TRAIN_FILE_PATTERN?} }, eval: { val_json_file: ${VAL_JSON_FILE?}, eval_file_pattern: ${EVAL_FILE_PATTERN?} } }"
. These default keys (ie type, train.checkpoint, train.prefix, train.train_file_pattern, eval.val_json_file, eval.eval_file_pattern) works fine. However, when I try to change other parameters, it gives an error, such as: KeyError: 'The key `num_classes:2` does not exist. To extend the existing keys, use `override` with `is_strict` = False.'
KeyError: 'The key `num_classes:2` does not exist. To extend the existing keys, use `override` with `is_strict` = False.'
. This happens for any of the parameters I try to change beyond what's provided. Other examples include: architecture{...,num_classes:2},eval:{...,eval_samples:100}, train:{...,total_steps:100, batch_size:8}
, all of which give the same The key does not exist
error. Here is a sample of the params.yaml
file that is outputted as part of the default settings.
anchor:
anchor_size: 4.0
aspect_ratios: [1.0, 2.0, 0.5]
num_scales: 3
architecture:
backbone: resnet
max_level: 7
min_level: 3
multilevel_features: fpn
num_classes: 91
parser: retinanet_parser
use_bfloat16: false
enable_summary: true
eval:
batch_size: 8
eval_dataset_type: tfrecord
eval_file_pattern: annotations\test.record
eval_samples: 5000
eval_timeout: null
input_sharding: true
min_eval_interval: 180
num_images_to_visualize: 0
num_steps_per_eval: 1000
type: box
use_json_file: true
val_json_file: ''
fpn:
fpn_feat_dims: 256
use_batch_norm: true
use_separable_conv: false
isolate_session_state: false
model_dir: probe_detection_models\v1
norm_activation:
activation: relu
batch_norm_epsilon: 0.0001
batch_norm_momentum: 0.997
batch_norm_trainable: true
use_sync_bn: false
postprocess:
max_total_size: 100
nms_iou_threshold: 0.5
pre_nms_num_boxes: 5000
score_threshold: 0.05
use_batched_nms: false
predict:
batch_size: 8
resnet:
resnet_depth: 50
retinanet_head:
num_convs: 4
num_filters: 256
use_separable_conv: false
retinanet_loss:
box_loss_weight: 50
focal_loss_alpha: 0.25
focal_loss_gamma: 1.5
huber_loss_delta: 0.1
retinanet_parser:
aug_rand_hflip: true
aug_scale_max: 1.0
aug_scale_min: 1.0
autoaugment_policy_name: v0
match_threshold: 0.5
max_num_instances: 100
num_channels: 3
output_size: [640, 640]
skip_crowd_during_training: true
unmatched_threshold: 0.5
use_autoaugment: false
spinenet:
model_id: '49'
strategy_config:
all_reduce_alg: null
distribution_strategy: one_device
num_gpus: 1
num_packs: 1
task_index: -1
tpu: Quadro
worker_hosts: null
strategy_type: one_device
train:
batch_size: 64
checkpoint:
path: resnet50-2018-02-07
prefix: resnet50/
frozen_variable_prefix: ''
gradient_clip_norm: 0.0
input_partition_dims: null
input_sharding: false
iterations_per_loop: 100
l2_weight_decay: 0.0001
learning_rate:
init_learning_rate: 0.08
learning_rate_levels: [0.008, 0.0008]
learning_rate_steps: [15000, 20000]
type: step
warmup_learning_rate: 0.0067
warmup_steps: 500
num_cores_per_replica: null
optimizer:
momentum: 0.9
nesterov: true
type: momentum
regularization_variable_regex: .*(kernel|weight):0$
total_steps: 22500
train_dataset_type: tfrecord
train_file_pattern: annotations\train.record
transpose_input: false
type: retinanet
use_tpu: false
How do I change the parameters other than default ones provided in --params_override
?
Appendix - Full Error Message:
Traceback (most recent call last):
File "models/official/vision/detection/main.py", line 265, in <module>
app.run(main)
File "C:\Users\212765830\AppData\Local\Continuum\anaconda3\envs\tf-official\lib\site-packages\absl\app.py", line 303, in run
_run_main(main, args)
File "C:\Users\212765830\AppData\Local\Continuum\anaconda3\envs\tf-official\lib\site-packages\absl\app.py", line 251, in _run_main
sys.exit(main(argv))
File "models/official/vision/detection/main.py", line 260, in main
run()
File "models/official/vision/detection/main.py", line 185, in run
params = params_dict.override_params_dict(
File "C:\Users\212765830\AppData\Local\Continuum\anaconda3\envs\tf-official\lib\site-packages\official\modeling\hyperparams\params_dict.py", line 443, in override_params_dict
params.override(params_dict, is_strict)
File "C:\Users\212765830\AppData\Local\Continuum\anaconda3\envs\tf-official\lib\site-packages\official\modeling\hyperparams\params_dict.py", line 166, in override
self._override(override_params, is_strict) # pylint: disable=protected-access
File "C:\Users\212765830\AppData\Local\Continuum\anaconda3\envs\tf-official\lib\site-packages\official\modeling\hyperparams\params_dict.py", line 183, in _override
self.__dict__[k]._override(v, is_strict) # pylint: disable=protected-access
File "C:\Users\212765830\AppData\Local\Continuum\anaconda3\envs\tf-official\lib\site-packages\official\modeling\hyperparams\params_dict.py", line 176, in _override
raise KeyError('The key `{}` does not exist. '
KeyError: 'The key `total_steps:100` does not exist. To extend the existing keys, use `override` with `is_strict` = False.'
I saw the additional instructions a little bit further down the README document.
python3 ~/models/official/vision/detection/main.py \
--strategy_type=tpu \
--tpu="${TPU_NAME?}" \
--model_dir="${MODEL_DIR?}" \
--mode=train \
--config_file="my_retinanet.yaml"
With the config file as:
# my_retinanet.yaml
type: 'retinanet'
train:
train_file_pattern: <path to the TFRecord training data>
eval:
eval_file_pattern: <path to the TFRecord validation data>
val_json_file: <path to the validation annotation JSON file>
Or 2) You can use inline configuration (YAML or JSON format):
python3 ~/models/official/vision/detection/main.py \
--model_dir=<model folder> \
--strategy_type=one_device \
--num_gpus=1 \
--mode=train \
--params_override="eval:
eval_file_pattern: <Eval TFRecord file pattern>
batch_size: 8
val_json_file: <COCO format groundtruth JSON file>
predict:
predict_batch_size: 8
architecture:
use_bfloat16: False
train:
total_steps: 1
batch_size: 8
train_file_pattern: <Eval TFRecord file pattern>
use_tpu: False
"
For some reason, the JSON inline format didn't work for me, but the my_retinanet.yaml file did for me.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.