[英]Using urllib3 with python 2.7 to download .txt file from url?
I'm using Python 2.7, and I have urllib3.我正在使用 Python 2.7,并且我有 urllib3。 I'm trying to download each of the.txt files in this link: http://web.mta.info/developers/turnstile.html
我正在尝试下载此链接中的每个 .txt 文件: http://web.mta.info/developers/turnstile.html
Here's my code:这是我的代码:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from bs4 import BeautifulSoup
import requests
import urllib3, shutil
http = urllib3.PoolManager()
MTA_url = requests.get("http://web.mta.info/developers/turnstile.html").text
MTA_soup = BeautifulSoup(MTA_url)
#Find each link to be downloaded
MTA_soup.findAll('a')
#Let's test it with the 36th link
one_a_tag = MTA_soup.findAll("a")[36]
MTA_link = one_a_tag["href"]
download_url = 'http://web.mta.info/developers/'+ MTA_link
print download_url #valid url, will take you to download
This is where I'm stuck.这就是我卡住的地方。 I can't seem to figure out how to download the.txt file at
download_url
, let alone iterate through the list.我似乎无法弄清楚如何在
download_url
下载 .txt 文件,更不用说遍历列表了。 I've tried this:我试过这个:
open('/Users/me/Documents/test_output_download.csv', 'wb').write(download_url.content)
But that gives me the error:但这给了我错误:
AttributeError: 'unicode' object has no attribute 'content'
After reading further, I also tried:进一步阅读后,我还尝试了:
out_file = '/Users/me/Documents/test_output_download.csv'
http.request('GET', download_url, preload_content=False) as res, open(out_file, 'wb') as out_file:
shutil.copyfileobj(res, out_file)
But I get past this syntax error:但我克服了这个语法错误:
http.request('GET', download_url, preload_content=False) as res, open(out_file, 'wb') as out_file:
^
SyntaxError: invalid syntax
How can I just download the.txt file that is located at download_url
and save it to my local drive, using urllib3?如何使用 urllib3 下载位于
download_url
的 .txt 文件并将其保存到我的本地驱动器? Thank you in advance.先感谢您。
The 'as' keyword is used for imports. 'as' 关键字用于导入。 I tested the full segment of code and was able to download after making a small change here.
我测试了完整的代码段,在这里做了一个小改动后就可以下载了。
Try shifting this around to declare the objects to variables instead, like so:尝试将其转换为将对象声明为变量,如下所示:
res = http.request('GET', download_url, preload_content=False)
out_file = open(out_file, 'wb')
shutil.copyfileobj(res, out_file)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.