为什么我可以从 python 文件而不是笔记本运行我的 PyTorch 数据集文件？

Question

I've written a custom dataset class in PyTorch in a file dataset.py and tried to test it by running the following code in my notebook with the following code:

from dataset import MyCustomDataset
from torch.utils.data import DataLoader

ds = MyCustomDataset("/Volumes/GoogleDrive/MyDrive/MyProject/data/train.pkl",target_type="labels")
dl = DataLoader(ds,batch_size = 16, shuffle = False)
X, y = next(iter(dl))
print(f"X: {X}, y: {y}")

After some unsuccessful troubleshooting, I tried running the exact same code in a file test.py, which worked without issues!

Why can't I run this from my notebook?

For me, the problem is usually the pathing somehow, but in this case, all of the files, both.py, .ipynb and "data"-directory are in the same directory "MyProject". I've tried with both absolute paths (as in the example) and with relative paths, but it's the same result in both cases. I'm using vscode if that gives any insight.

Furthermore, the error message in the notebook is "list indices must be integers or slices, not str", unfortunately, the prompt tells me the wrong lines (there's a comment on the line where the error's supposed to be). But if this is really an error, then it should not work in a python file either, right?

Any help or suggestions are welcome!

Answer 1

Try to check if there is any problem with the path

import os.path
from os import path
a= path.exists("/Volumes/GoogleDrive/MyDrive/MyProject/data/train.pkl")
print(a)

if it returns true it means path is not the issue and you need to provide more details in your question

Answer 2

Jupyter and Python file has different cwd . You can execute this to get the cwd :

import os
print(os.getcwd())

And you can add this in the settings.json file to modify the cwd of jupyter notebook to let it take the ${workspaceFolder} as the cwd like the python file does:

"jupyter.notebookFileRoot": "${workspaceFolder}",