堆栈内存溢出
  简体   繁体   English

在访问 TFRecords 以训练深度学习模型时被拒绝访问

[英]Getting access denied on accessing TFRecords for training a Deep Learning model

 原文  2019-02-27 09:26:19  4   1    python/ tensorflow/ deep-learning

问题描述

I am training a model using the Google QuickDraw dataset.我正在使用 Google QuickDraw 数据集训练模型。 I have stored the data from ndjson to tfrecords as instructed in the tutorial for it.我已按照教程中的说明将数据从 ndjson 存储到 tfrecords。 Then I used the code provided by Google link for training the dataset using those tfrecords.然后我使用 Google 链接提供的代码使用这些 tfrecords 训练数据集。 But I am getting the following error:但我收到以下错误：

Traceback (most recent call last):
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call
    return fn(*args)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.UnknownError: NewRandomAccessFile failed to Create/Open: A:\Code\Machine Learning\Software Engineering project\Quick Draw\TFRecord Train dataset : Access is denied.

; Input/output error
     [[{{node IteratorGetNext}} = IteratorGetNext[output_shapes=[[?,1], [?,?], [?,2], [?,1]], output_types=[DT_INT64, DT_FLOAT, DT_INT64, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"](OneShotIterator)]]
     [[{{node IteratorGetNext/_317}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_20_IteratorGetNext", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "A:\Code\Machine Learning\Software Engineering project\Quick Draw\Train_Model.py", line 303, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
    _sys.exit(main(argv))
  File "A:\Code\Machine Learning\Software Engineering project\Quick Draw\Train_Model.py", line 214, in main
    tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 471, in train_and_evaluate
    return executor.run()
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 610, in run
    return self.run_local()
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 711, in run_local
    saving_listeners=saving_listeners)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 354, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1207, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1241, in _train_model_default
    saving_listeners)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1471, in _train_with_estimator_spec
    _, loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss])
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 671, in run
    run_metadata=run_metadata)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1156, in run
    run_metadata=run_metadata)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1255, in run
    raise six.reraise(*original_exc_info)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\six.py", line 693, in reraise
    raise value
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1240, in run
    return self._sess.run(*args, **kwargs)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1312, in run
    run_metadata=run_metadata)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\training\monitored_session.py", line 1076, in run
    return self._sess.run(*args, **kwargs)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
    run_metadata_ptr)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
    run_metadata)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: NewRandomAccessFile failed to Create/Open: A:\Code\Machine Learning\Software Engineering project\Quick Draw\TFRecord Train dataset : Access is denied.

; Input/output error
     [[node IteratorGetNext (defined at A:\Code\Machine Learning\Software Engineering project\Quick Draw\Train_Model.py:70)  = IteratorGetNext[output_shapes=[[?,1], [?,?], [?,2], [?,1]], output_types=[DT_INT64, DT_FLOAT, DT_INT64, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"](OneShotIterator)]]
     [[{{node IteratorGetNext/_317}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_20_IteratorGetNext", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Caused by op 'IteratorGetNext', defined at:
  File "<string>", line 1, in <module>
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\idlelib\run.py", line 144, in main
    ret = method(*args, **kwargs)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\idlelib\run.py", line 474, in runcode
    exec(code, self.locals)
  File "A:\Code\Machine Learning\Software Engineering project\Quick Draw\Train_Model.py", line 303, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
    _sys.exit(main(argv))
  File "A:\Code\Machine Learning\Software Engineering project\Quick Draw\Train_Model.py", line 214, in main
    tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 471, in train_and_evaluate
    return executor.run()
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 610, in run
    return self.run_local()
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\training.py", line 711, in run_local
    saving_listeners=saving_listeners)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 354, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1207, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1234, in _train_model_default
    input_fn, model_fn_lib.ModeKeys.TRAIN))
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1075, in _get_features_and_labels_from_input_fn
    self._call_input_fn(input_fn, mode))
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\estimator\estimator.py", line 1162, in _call_input_fn
    return input_fn(**kwargs)
  File "A:\Code\Machine Learning\Software Engineering project\Quick Draw\Train_Model.py", line 70, in input_func
    features, labels = dataset.make_one_shot_iterator().get_next()
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 421, in get_next
    name=name)), self._output_types,
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 2108, in iterator_get_next
    output_shapes=output_shapes, name=name)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3274, in create_op
    op_def=op_def)
  File "C:\Users\shind\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in __init__
    self._traceback = tf_stack.extract_stack()

UnknownError (see above for traceback): NewRandomAccessFile failed to Create/Open: A:\Code\Machine Learning\Software Engineering project\Quick Draw\TFRecord Train dataset : Access is denied.

; Input/output error
     [[node IteratorGetNext (defined at A:\Code\Machine Learning\Software Engineering project\Quick Draw\Train_Model.py:70)  = IteratorGetNext[output_shapes=[[?,1], [?,?], [?,2], [?,1]], output_types=[DT_INT64, DT_FLOAT, DT_INT64, DT_INT64], _device="/job:localhost/replica:0/task:0/device:CPU:0"](OneShotIterator)]]
     [[{{node IteratorGetNext/_317}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_20_IteratorGetNext", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

I am not getting exactly which function is giving the error.我没有确切地知道哪个函数给出了错误。 So, I have not posted my code, as I don't know the relevant part of it.所以，我没有发布我的代码，因为我不知道它的相关部分。 But why is the access is denied?但是为什么访问被拒绝？ I have used almost the same code as given in the repository.我使用了与存储库中给出的代码几乎相同的代码。 Also, I am not getting what the error is about, is it just access denied or something else?另外，我不明白错误是关于什么的，它只是访问被拒绝还是其他什么？ Please help me that.请帮助我。

1 个解决方案

解决方案1
 0  2019-10-11 10:30:53

Please make sure the file you are accessing has proper extension ".tfrecords" .请确保您正在访问的文件具有正确的扩展名".tfrecords" 。

This issue occurs when the file it is trying to access with does not have proper extension.当它尝试访问的文件没有正确的扩展名时会出现此问题。

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 创建用于训练深度学习 model 的 docker 图像 - Creating a docker image for training a deep learning model 训练深度学习 model 时出错 - Error while training a deep learning model Pytorch 深度学习 - 类模型（）和训练函数 - Pytorch Deep Learning - Class Model() and training function 在 pytorch 中加速训练深度学习 model - Speed up training deep learning model in pytorch 训练深度学习模型时如何处理大型csv文件？ - How to deal with large csv file when training a deep learning model? 通过训练小数据子集可以验证深度学习模型吗？ - Is it possible to validate a deep learning model by training small data subset? Keras 深度学习模型在训练中总是给出相同的 acc - Keras deep learning model always give same acc in training 使用 Keras 训练深度学习检查点 - Training checkpoints for deep learning with Keras 深度学习 CNN model 不学习 - Deep learning CNN model not learning Model 没有学习/训练 pytorch - Model is not learning/training pytorch
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM