在pandas DataFrame中加载逗号不均的文本文件

Question

15/09/2017, 10:20 - Jatin: Robin is the meeting on???
15/09/2017, 10:23 - Robin: No
15/09/2017, 10:23 - Robin: Thanks for the update
15/09/2017, 10:23 - Robin: can we expect it soon
15/09/2017, 10:24 - Jatin: it will be this weekend, most likely
15/09/2017, 10:24 - Jatin: kindly be prepared
15/09/2017, 10:24 - Robin: Sure no issues
15/09/2017, 10:26 - Jatin: good luck

I have a data file that looks like this. I intend to load this in a pandas dataframe. Issue is that if I do

pd.read_csv("file.txt") 

It throws an error:

Error tokenizing data. C error: Expected 2 fields in line 695, saw 3

Can someone please suggest the easiest possible way to do this with pandas?

Answer 1

It appears to be a watsapp email chat file you are trying to load. I worked on something similar and here is a code that worked for me.

atempt_load=pd.read_table("WhatsApp Chat with Panda.txt")
atempt_load.columns=["namesake"] # this will load the entire message ina single column and we are just giving it a convenient name, in order to use it later
name=[]
message=[]
for i in range(len(atempt_load)):
#now there are 20 characters in front of each line before a name appears,
# we can use this and use the following coed to separate it

    name.append((atempt_load["namesake"][i])[20:25]) #since both the names are of same length this will take out the string from 20:25 words
    message.append((atempt_load["namesake"][i])[26:len(atempt_load["namesake"][i])])

You can do a similar thing if you want timestamps as well.

Limitations: It will not work if the names are of different lengths, I found a way around it by changing the names of contacts in the chat before importing a file in email.

I am sure someone will have a more dynamic and cleaner fix

Answer 2

Alternatively, specify the separator more explicitly:

pd.read_csv('test.txt', names=['timestamp', 'text'], sep=' - ') 

This is will throw a warning about falling back to the python engine. That is just a warning that performance may be reduced for very large files.