具有特征選擇功能的多標簽分類（scikit-learn）

Question

我正在使用scikit-learn解決具有大量標簽的多標簽分類問題。 我遵循了該項目核心開發人員之一的想法 （larsmans）。 它給了我一個運行時警告，然后是內存錯誤。

>>> clf = Pipeline([('chi2', SelectKBest(chi2, k=1000)),('svm', LinearSVC())])
>>> 
>>> multiclf = OneVsRestClassifier(clf, n_jobs=-1)
>>> 
>>> multiclf.fit(Xtr, ytr)
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157:             
RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp

然后，在多次發出這樣的警告之后，它失敗了：下面的錯誤轉儲

Process PoolWorker-21:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers
    pool._maintain_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool
    self._repopulate_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool
    w.start()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start
    self._popen = Popen(self)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-22:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-23:
Process PoolWorker-30:
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-31:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-28:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-32:
Process PoolWorker-29:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-27:
Process PoolWorker-26:
Traceback (most recent call last):
Process PoolWorker-25:
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self.run()
    self.run()
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
    return recv()
    return recv()
MemoryError
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
    return recv()
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-24:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, 'k'.
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-2:
Traceback (most recent call last):
Process PoolWorker-3:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-4:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-6:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-5:
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-7:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: unpickling stack underflow
Process PoolWorker-8:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, ''.
Process PoolWorker-10:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, '�'.
Process PoolWorker-21:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers
    pool._maintain_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool
    self._repopulate_pool()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool
    w.start()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start
    self._popen = Popen(self)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-22:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-23:
Process PoolWorker-30:
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide
  chisq /= f_exp
Process PoolWorker-31:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-28:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-32:
Process PoolWorker-29:
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-27:
Process PoolWorker-26:
Traceback (most recent call last):
Process PoolWorker-25:
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self.run()
    self.run()
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
    return recv()
    return recv()
MemoryError
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
    return recv()
MemoryError
MemoryError
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-24:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, 'k'.
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-1:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-2:
Traceback (most recent call last):
Process PoolWorker-3:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-4:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
Process PoolWorker-6:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
Process PoolWorker-5:
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
    return recv()
MemoryError
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
MemoryError
Process PoolWorker-7:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: unpickling stack underflow
Process PoolWorker-8:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, ''.
Process PoolWorker-10:
Traceback (most recent call last):
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker
    task = get()
  File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
UnpicklingError: invalid load key, '�'.

Answer 1

查看所有MemoryErrors，您可能已經用完了內存。 嘗試將n_jobs = -1替換為n_jobs = 1。 這控制了產生的進程數量（以及因此使用的內存數量）。

如果可行，只要不耗盡內存就可以增加n_jobs，即n_jobs = 2，n_jobs = 3，依此類推。