[英]HTTP Error when trying to insert csv file from a url to mongodb
I'm trying to send data from a csv file to mongodb using a python insertion script.我正在尝试使用 python 插入脚本将数据从 csv 文件发送到 mongodb。 I have to get this file directly via url, that is, it is not local.
我得直接通过url获取这个文件,也就是不是本地的。 I'm using pymongo and read_csv from pandas to do the insertion, but I'm getting an " HTTP Error 500: Internal Server Error ".
我正在使用 pandas 中的 pymongo 和 read_csv 进行插入,但我得到了“ HTTP 错误 500:内部服务器错误”。 I imagine it is something related to encoding or header.
我想这与编码或 header 有关。 I have tried several combinations, but none have worked.
我尝试了几种组合,但都没有奏效。 Here is the code:
这是代码:
try:
import pymongo
from pymongo import MongoClient
import pandas as pd
import json
except Exception as e:
print("Some Modules are Missing ")
import requests
import urllib.request
from urllib.error import HTTPError
class MongoDB(object):
def __init__(self, dBName=None, collectionName=None):
self.dBName = dBName
self.collectionName = collectionName
#self.client = MongoClient("localhost", 27017)
self.client = MongoClient("<connection_string>")
self.DB = self.client[self.dBName]
self.collection = self.DB[self.collectionName]
def InsertData(self, path=None):
df = pd.read_csv(path, sep=";", encoding='UTF-8', header="infer")
data = df.to_dict('records')
self.collection.insert_many(data, ordered=False)
print("All the Data has been Exported to Mongo DB Server .... ")
if __name__ == "__main__":
mongodb = MongoDB(dBName = 'vacinacao-covid', collectionName='teste')
mongodb.InsertData(path = "https://www.saopaulo.sp.gov.br/wp-content/uploads/2021/04/20210420_percentual_primeira_dose.csv")
UTF-8 does not work on my end. UTF-8 对我不起作用。 I used latin-1 instead.
我改用 latin-1。
import pandas as pd
import requests
from io import BytesIO
user_agent = {'User-agent': 'Mozilla/5.0'}
path = "https://www.saopaulo.sp.gov.br/wp-content/uploads/2021/04/20210420_percentual_primeira_dose.csv"
r = requests.get(path, headers=user_agent)
f = BytesIO(r.content)
df = pd.read_csv(f, sep=";", encoding='latin-1', header="infer")
print(df)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.