简体   繁体   中英

gsutil copy failing with exception can't start new thread

I am trying copy from local to google cloud storage

String gsutilCommand = String.format("gsutil -m cp -r %s %s", localPath, gcsPath);
ProcessBuilder pb = new ProcessBuilder(new String[] { "bash", "-c", gsutilCommand });
pb.redirectErrorStream(true);
Process process = pb.start();

  Process Process-6:Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in 
  _bootstrap
  error: can't start new thread
  self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
  self._target(*self._args, **self._kwargs)
  File "/root/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1716, in _ApplyThreads
       status_queue=status_queue)
  File "/root/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2006, in __init__
       worker_thread.start()
  File "/usr/lib/python2.7/threading.py", line 745, in start
       _start_new_thread(self.__bootstrap, ())
  error: can't start new thread
       self.run()

What worked for me (python3.8, CentOS 8):

Go into your .boto settings, uncomment and change the "parallel" values to 1 as follows:

[GSUtil]                                                                                                                               
parallel_process_count = 1                                                                                                             
parallel_thread_count = 1

This resolved gsutil complaining about threading even in the absence of using the -m flag.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM