Google AI Platform Notebooks: Pandas can't see my data files

Question

I am using AI Platform Notebooks to utilize a GPU since I don't have one.

The JupyterLab proxy launched successfully, and all of my data is showing in the file browser on the left hand side.

However when I try to get at it with pd.read_csv('path/to/my/file.txt') I get the error File path/to/myfile.txt does not exist

This happens no matter where I put the data, and whether or not I provide an absolute or relative path makes no difference.

I'm an amateur programmer and new to this cloud computing stuff. It has been really heinously difficult to do anything online compared to just doing tasks on my PC. Nothing works quite as advertised and I have to spend hours finding work-arounds to get basic code to run. What's going on here?

Thanks in advance.

Answer 1

I've created a solution taking parts from here (Kaggle) and here (SO) .

As a summary of what will follow, you must have your file inside a bucket, under your Google Cloud Project.

Firstly you'll have to install the appropriate libraries:

!pip install google-cloud-storage
!pip install pandas

Then do the appropriate imports:

import pandas as pd
import google.cloud.storage as storage
from io import BytesIO

Then create the storage client by giving your Project ID:

storage_client = storage.Client(project = "project-ID")

Define your bucket by giving your bucket's name:

bucket = storage_client.get_bucket("bucket-name")

Create a blob with data from the file you want to read:

blob = storage.blob.Blob("your-file.csv",bucket)

Download the content of that blob as a string:

content = blob.download_as_string()

Read the data into pandas:

data = pd.read_csv(BytesIO(content))

Now you can view some of your data, using for example the head() function:

data.head()