python：从 URL 读取文件

Question

从互联网读取文本文件的正确方法是什么。 例如此处的文本文件https://gist.githubusercontent.com/deekayen/4148741/raw/01c6252ccc5b5fb307c1bb899c95989a8a284616/1-1000.txt

下面的代码有效，但在每个单词前面产生了额外'b

from urllib.request import urlopen
#url = 'https://raw.githubusercontent.com/first20hours/google-10000-english/master/google-10000-english.txt'
url = 'https://gist.githubusercontent.com/deekayen/4148741/raw/01c6252ccc5b5fb307c1bb899c95989a8a284616/1-1000.txt'
#data = urlopen(url)
#print('H w')

# it's a file like object and works just like a file
l = set()
data = urlopen(url)
for line in data:  # files are iterable
    word = line.strip()
    print(word)
    l.add(word)

print(l)

Answer 1

您必须将每个字节object 解码为unicode 。 为此，您可以使用方法decode('utf-8') 。 这是代码：

from urllib.request import urlopen
url = 'https://gist.githubusercontent.com/deekayen/4148741/raw/01c6252ccc5b5fb307c1bb899c95989a8a284616/1-1000.txt'

l = set()
data = urlopen(url)
for line in data:  # files are iterable
    word = line.strip().decode('utf-8') # decode the line into unicode
    print(word)
    l.add(word)

print(l)

Answer 2

使用 pandas 很简单。 只需执行

import pandas as pd
pd.read_csv('https://gist.githubusercontent.com/deekayen/4148741/raw/01c6252ccc5b5fb307c1bb899c95989a8a284616/1-1000.txt')

你们都准备好了:)