使用Python保存从Facebook收集的评论的最佳方法是什么？

Question

I'm collecting all the comments from some Facebook pages using Python and Facebook-SDK.

Since I want to do Sentiment Analysis on these comments, what's the best way to save these texts, such that it's not needed any changing in the texts?

I'm now saving the comments as a table and then as a CSV file.

table.to_csv('file-name.csv')

But if I want to read this saved file, I get the following error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position ...

By the way, I'm working with the German Texts.

Answer 1

Have you tried this?

Set default encoder at the top of your code

import sys
reload(sys)
sys.setdefaultencoding("ISO-8859-1")

or

pd.read_csv('file-name.csv', encoding = "ISO-8859-1")

Answer 2

If you have knowledge about the encoding of the data then, you can simply use pandas to read your csv as follow:

import pandas as pd
pd.read_csv('filename.csv', encoding='encoding')

Answer 3

I would say it really depends on many different factors such as:

• Size of the data
• What kind of analysis, specifically, are you anticipating that you'll be doing
• What format are you most comfortable working with the data

For most of my data munging in python I like to do it in pandas if possible, but sometimes that's not a feasible option given the size of the data. In that case you'd have to think about using something like pyspark. But here is a link to the pandas docs for reference, they have a lot of functionality for reading in all kinds of data: pandas docs