How to upload folders to Google Colab?

Question

I want to run a notebook that uses many header files defined in the directory. So basically I want to upload the entire directory to Google Colab so that I can run the notebook. But I am unable to find any such options and only able to upload files not complete folders. So can someone tell me how to upload entire directory to google colab?

Answer 1

I suggest you not to upload them just in Colab, since when you're restarting the runtime you will lose them (just need to re-upload them, but it can be an issue with very big datasets). I suggest you to use the google.colab package to manage files and folders in Colab. Just upload everything you need to your google drive, then import:

from google.colab import drive
drive.mount('/content/gdrive')

In this way, you just need to login to your google account through google authentication API, and you can use files/folders as if they were uploaded on Colab.

Answer 2

You can zip them, upload, then unzip it.

!unzip file.zip

Answer 3

you can create a git repository and push the files and folders to it, and then can clone the repository in colaboratory with the command

!git clone https://github.com/ {username}/{projectname}.git

i feel this method is faster. but if the file size is more than 100 mb you will have to zip the file or will have to add extentions to push it to github. for more information refer the link below.

https://help.github.com/en/github/managing-large-files/configuring-git-large-file-storage

Answer 4

The easiest way to do this, if the folder/file is on your local drive:

1. Compress the folder into a ZIP file.
2. Upload the zipped file into colab using the upload button in the File section. Yes, there is a File section, see the left side of the colab screen .
3. Use this line of code to extract the file. Note: The file path is from colab's File section.

from zipfile import ZipFile
file_name = file_path

with ZipFile(file_name, 'r') as zip:
  zip.extractall()
  print('Done')

4. Click Refresh in the colab File section.
5. Access the files in your folder through the file paths

Downside: The files will be deleted after the runtime is over.

You can use some part of these steps if your file is on a Google Drive, just upload the zipped file to colab from Google Drive.

Answer 5

The best way to approach this problem is simple yet tricky sometimes.

You first need to compress the folder into a zipped file and upload the same into your google drive.

While doing so, Make sure that the folder is in the root directory of the drive and not in any other subfolder! . If the compressed folder/data is in other subfolder, you can easily move the same into the root directory.

Compresses folder/data in another subfolder often messes with the unzipping process when you will be specifying the file location.

Once you did the afore mentioned tasks, enter the following commands in the colab to mount your drive:

from google.colab import drive
drive.mount('/content/gdrive')

This will ask for an access token that can be generated by clicking on the url displayed in the output of the same cell

!ls gdrive/MyDrive

Check the contents of the drive by executing the above command and ensure that your folder/data is displayed in the output.

!unzip gdrive/MyDrive/<File_name_without_space>.zip

eg:

!unzip gdrive/MyDrive/data_folder.zip

Executing the same will start unzipping your folder into the memory.

Congrats! You have successfully uploaded your folder/data into the colab.

Answer 6

zip your files zip -r file.zip your_folder and then:

from google.colab import files
from zipfile import ZipFile

with ZipFile(files.upload(), 'r') as zip:
  zip.extractall()
  print('Done')

Answer 7

因此，您可以执行以下操作： - 将所需的数据集文件夹上传到您的驱动器 - 通过 colab，安装驱动器，其中“from google.colab import drive drive.mount('/content/gdrive')”会自动显示，您只需需要运行它 - 然后在左侧的 Files 部分检查您的文件（如果文件夹不可见尝试刷新，旁边应该有一个下拉箭头，您可以在其中检查文件夹下的所有文件) - 左键单击获得 COPY PATH 选项的文件夹 - 将复制的路径粘贴到 colab 中的所需位置