努力从 Google Cloud Storage 存储桶中读取 csv 文件

Question

There are some CSV files in a folder of a GCS bucket. I am using these codes to access and read those CSV files as pandas dataframe:

from google.cloud import storage
import gcsfs
import fsspec
storage_client = storage.Client()

blobs = storage_client.list_blobs('pipe_ml-data', prefix='postive_data_file/')
all_files = list(blobs)
all_files = all_files[1::]
for blob in all_files:
    file_path = "gs://pipe_ml-data/{}".format(blob.name)
    content = pd.read_csv(file_path)
    content.head()
    break

But getting the following error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/var/folders/9q/h73jxwsx0f1629kdwpdfmdb80000gn/T/ipykernel_31809/388262887.py in <module>
      1 from google.cloud import storage
----> 2 import gcsfs
      3 import fsspec
      4 storage_client = storage.Client()
      5 

/opt/anaconda3/lib/python3.9/site-packages/gcsfs/__init__.py in <module>
      3 __version__ = get_versions()["version"]
      4 del get_versions
----> 5 from .core import GCSFileSystem
      6 from .mapping import GCSMap
      7 

/opt/anaconda3/lib/python3.9/site-packages/gcsfs/core.py in <module>
     18 from fsspec.utils import stringify_path, setup_logging
     19 from fsspec.callbacks import NoOpCallback
---> 20 from fsspec.implementations.http import get_client
     21 from .retry import retry_request, validate_response
     22 from .checkers import get_consistency_checker

/opt/anaconda3/lib/python3.9/site-packages/fsspec/implementations/http.py in <module>
     16 from fsspec.exceptions import FSTimeoutError
     17 from fsspec.spec import AbstractBufferedFile
---> 18 from fsspec.utils import DEFAULT_BLOCK_SIZE, isfilelike, nullcontext, tokenize
     19 
     20 from ..caching import AllBytes

ImportError: cannot import name 'isfilelike' from 'fsspec.utils' (/opt/anaconda3/lib/python3.9/site-packages/fsspec/utils.py)

I already installed and imported gcsfs as well as fsspec libraries. Nothing helped. Could you please give me a solution?

Answer 1

fsspec.utils.isfilelike looks to be very new , like in the last couple of months (Apr 22), so I'm going to guess you've got some sort of version issue going on (like you're importing fsspec 2022.3.0 or older instead of 2022.5.0 or newer).

Use a command like pip show fsspec to see if you've got 2022.5.0 or later installed.