如何下载 zip 文件并在 python 中解析 csv 文件

Question

我写了一个脚本，它命中了一个 URL 并下载了一个 zip 文件，解压它。 现在我在解析解压缩后得到的 CSV 文件时遇到问题。

import csv
from requests import get
from io import BytesIO
from zipfile import ZipFile

request = get('https://example.com/some_file.zip')
zip_file = ZipFile(BytesIO(request.content))
files = zip_file.namelist()
with open(files[0], 'r') as csvfile:
    csvreader = csv.reader(csvfile)
    for row in csvreader:
        print(row)

Answer 1

当你执行files = zip_file.namelist() ，你只需列出zip存档中文件的名称; 这些文件尚未从zip中提取出来，您无法将其作为本地文件open ，就像您正在做的那样。

您可以使用ZipFile.open直接从zip文件中读取数据流。

所以这应该工作：

zip_file = ZipFile(BytesIO(request.content))
files = zip_file.namelist()
with zip_file.open(files[0], 'r') as csvfile:
    csvreader = csv.reader(csvfile)
    ...

Answer 2

看起来你还没有导入csv模块。 尝试使用import csv将import csv放在顶部。

Answer 3

response = requests.get(url)
with io.BytesIO(response.content) as zip_file:
    with zipfile.ZipFile() as zip_file:
        # Get first file in the archive
        for zip_info in zip_file.infolist():
            logger.debug(zip_info)
            # Open file
            with zip_file.open(zip_info) as file:
                # Load CSV file, decode binary to text
                with io.TextIOWrapper(file) as text:
                    return csv.DictReader(text)

Answer 4

所以。 经过几个小时的搜索和尝试，我终于得到了一些工作。 这是我的脚本。

所以我的需要是：

• 下载一个 ZIP 文件。
• 在该 zip 文件中找到名称中包含“anystring”的特定文本文件
• 从该文本文件中提取包含字符串“csv”的第一个 URL

#!/bin/env python
from io import BytesIO
from zipfile import ZipFile
import requests
import re
import sys

# define url value
url = "https://whateverurlyouneed"

# Define string to be found in the file name to be extracted
filestr = "anystring"

# Define string to be found in URL
urlstr = "anystring"

# Define regex to extract URL
regularex = r"(?i)\b((?:https?://|www\d{0,3}[.]|[a-z0-9.-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|(([^\s()<>]+|(([^\s()<>]+)))))+(?:(([^\s()<>]+|(([^\s()<>]+))))|[^\s`!()[]{};:'\".,<>?«»“”‘’]))"

# download zip file
content = requests.get(url)

# Open stream
zipfile = ZipFile(BytesIO(content.content))

# Open first file from the ZIP archive containing 
# the filestr string in the name
data = [zipfile.open(file_name) for file_name in zipfile.namelist() if filestr in file_name][0]

# read lines from the file. If csv found, print URL and exit
# This will return the 1st URL containing CSV in the opened file
for line in data.readlines():
    if urlstr in line.decode("latin-1"):
        urls = re.findall(regularex,line.decode("latin-1"))
        print([url[0] for url in urls])
        break
sys.exit(0)