無法讀取以下類型的對象： <class 'numpy.ndarray'>

Question

我有以下python代碼，我正嘗試根據時間戳將其輸出到目錄。

import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq
import uuid

data = {'date': ['2018-03-04T14:12:15.653Z', '2018-03-03T14:12:15.653Z', '2018-03-02T14:12:15.653Z', '2018-03-05T14:12:15.653Z'],
    'battles': [34, 25, 26, 57],
    'citys': ['london', 'newyork', 'boston', 'boston']}
df = pd.DataFrame(data, columns=['date', 'battles', 'citys'])
df['date'] = df['date'].map(lambda t: pd.to_datetime(t, format="%Y-%m-%dT%H:%M:%S.%fZ"))
df.groupby(by=['citys'])
dst_path = "logs/year=" + df['date'].dt.year.astype('str').unique() + "/month=" + df['date'].dt.month.astype('str').unique() + "/day=" + df['date'].dt.day.astype('str').unique() + "/" + str(uuid.uuid4()) + ".parq"
table = pa.Table.from_pandas(df)
pq.write_table(table, dst_path)

但我看到以下錯誤：

python3 test.py
Traceback (most recent call last):
File "test.py", line 15, in <module>
  pq.write_table(table, dst_path)
File "/usr/local/lib/python3.6/site-packages/pyarrow/parquet.py", line 943, in write_table
**kwargs)
File "/usr/local/lib/python3.6/site-packages/pyarrow/parquet.py", line 286, in __init__
**options)
File "pyarrow/_parquet.pyx", line 837, in pyarrow._parquet.ParquetWriter.__cinit__ (/Users/travis/build/BryanCutler/arrow-dist/arrow/python/build/temp.macosx-10.6-intel-3.6/_parquet.cxx:14606)
File "pyarrow/io.pxi", line 835, in pyarrow.lib.get_writer (/Users/travis/build/BryanCutler/arrow-dist/arrow/python/build/temp.macosx-10.6-intel-3.6/lib.cxx:59078)
TypeError: Unable to read from object of type: <class 'numpy.ndarray'>

如何從熊貓時間戳創建目錄？

Answer 1

您的dst_path是一個numpy數組。

print(type(dst_path))

產量

<class 'numpy.ndarray'>

它應該是一個字符串，因此在dst_path行下面添加了以下內容，它可以正常工作。 它不是很優雅，所以您可以研究一種更好的方法。 這里的重點是您需要一個字符串。

dst_path = str(dst_path[0])

請注意，該目錄必須已經存在，否則您將得到一個錯誤，因此可以在write_table之前編寫以下內容。

import os
dir, file = os.path.split(dst_path)
if not os.path.exists(dir):
    os.makedirs(dir)