[英]“MemoryError” in VSCode but not in Jupyter/AnacondaPrompt
I am running into a MemoryError at around 2 GB on my Python implementation of a split_train_test function on the MNIST data set. 我在MNIST数据集上的split_train_test函数的Python实现中遇到了大约2 GB的MemoryError。
Task Manager barely reaches 50% of max memory, including the other apps that are open on my machine. 任务管理器几乎无法达到最大内存的50%,包括在我的计算机上打开的其他应用程序。 I have 16GB of RAM.
我有16GB的RAM。
I see most people point towards a 64 vs 32 bit or python 2 vs 3 problem; 我看到大多数人都指向64 vs 32位或python 2 vs 3问题。 however, both my VS Code and Windows 10 are 64-bit and view > Command Palette > Python: Select Interpreter show that I am using Python 3.7.1 64-bit from anaconda3/conda.
但是,我的VS Code和Windows 10都是64位的,并查看>命令面板> Python:选择解释器显示我正在使用anaconda3 / conda的Python 3.7.1 64位。
I know the code itself works because I have used the output in Jupyter afterI import the py file. 我知道代码本身可以工作,因为导入py文件后,我已经在Jupyter中使用了输出。
def split_train_val(val_frac=0.3, size=1):
"""Splits training and validation set
param val_frac: fraction of total training set to be used for validation
"""
# Read converted csv
X_raw = pd.read_csv('Data/csv/X_train.csv')
Y_raw = pd.read_csv('Data/csv/y_train.csv')
# Rename Label column, concat to X set
Y_raw.columns = ['Label']
df = pd.concat([Y_raw, X_raw], axis=1).sample(frac=size)
# Split training set into train and val
N = df.shape[0]
n = round(val_frac * N)
train = df.iloc[n:,:]
val = df.iloc[:n,:]
x_train = train.drop(['Label'], axis=1)
x_val = val.drop(['Label'], axis=1)
y_train = train.Label
y_val = val.Label
# Return training and validation set
return(x_train, y_train, x_val, y_val)
x_train, y_train, x_val, y_val = split_train_val()
Error Message: 错误信息:
Traceback (most recent call last):
File "preprocessing.py", line 71, in <module>
x_train, y_train, x_val, y_val = split_train_val()
File "preprocessing.py", line 53, in split_train_val
df = pd.concat([Y_raw, X_raw], axis=1).sample(frac=size)
File "C:\Users\...\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\core\reshape\concat.py", line 229, in concat
return op.get_result()
File "C:\Users\...\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\core\reshape\concat.py", line 426, in get_result
copy=self.copy)
File "C:\Users\...\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\core\internals\managers.py", line 2052, in concatenate_block_managers
values = values.copy()
MemoryError
Lastly, I tried changing the jedi.memoryLimit setting to -1 as suggested by some VS Code documentation. 最后,我尝试按照某些VS Code文档的建议将jedi.memoryLimit设置更改为-1。 This did not help either.
这也没有帮助。
I imported then ran the function in Jupyter. 我导入然后在Jupyter中运行该功能。 I have also run this exact code in my Anaconda Prompt.
我也在Anaconda Prompt中运行了此确切代码。 None of them result in any error.
它们均不会导致任何错误。
VS Code and Windows 10 may be 64-bit for you, but your installation of Python is 32-bit as shown by your path: C:\\Users\\...\\AppData\\Local\\Programs\\Python\\Python37-32\\
. VS Code和Windows 10可能是64位的,但是Python的安装是32位的,如路径所示:
C:\\Users\\...\\AppData\\Local\\Programs\\Python\\Python37-32\\
。 Try explicitly installing a 64-bit version of Python and make sure to select it in VS Code. 尝试显式安装Python的64位版本,并确保在VS Code中选择它。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.