DataScience DataFrame 的 Python 编程错误

Question

I am reading my data from a CSV file using pandas and it works well with range 700. But as soon as I go above 700 and trying to append to a list in python it is showing me list index out of range. But the CSV has around 500K of rows Can anyone help me with that why is it happening? Thanks in advance.

import pandas as pd

df_email = pd.read_csv('emails.csv',nrows=800)
test_email = df_email.iloc[:,-1]


list_of_emails = []

for i in range(len(test_email)):    
    var_email = test_email[i].split("\n") #this code takes one single email splits based on a new line giving a python list of all the strings in the email


    email = {}
    message_body = ''

    for _ in var_email:
        if ":" in _:
            var_sentence = _.split(":") #this part actually uses the ":" to find the elements in the list that have ":" present

            for j in range(len(var_sentence)):           
                if var_sentence[j].lower().strip() == "from":
                    email['from'] = var_sentence[var_sentence.index(var_sentence[j+1])].lower().strip()
                elif  var_sentence[j].lower().strip() == "to":  
                    email['to'] = var_sentence[var_sentence.index(var_sentence[j+1])].lower().strip()
                elif var_sentence[j].lower().strip() == 'subject':
                    if var_sentence[var_sentence.index(var_sentence[j+1])].lower().strip() == 're':
                        email['subject'] = var_sentence[var_sentence.index(var_sentence[j+2])].lower().strip()
                    else:
                        email['subject'] = var_sentence[var_sentence.index(var_sentence[j+1])].lower().strip()

        elif ":" not in _:
            message_body += _.strip()
            email['body'] = message_body

    list_of_emails.append(email)

Answer 1

I am not sure of what you are trying to say here (might as well put example inputs and outputs here), but I came across this problem, which might be of the same nature, sometime weeks ago.

CSV files are comma-separated, which means it always takes note of every comma in a line to separate them in columns. If some dirty input from strings in your CSV file are present, then it will mess up the columns that you are expecting to have.

Best solution here is have some code to cleanup your CSV file, change its delimiter to another character (probably '|', '&', or anything that also doesn't mess up with the data), and revise your code to reflect these changes to the CSV.

Answer 2

use the pandas library to read the file.

it is very efficient and saves you time in writing the code yourself.

eg :

import pandas as pd
training_data = pd.read_csv( "train.csv", sep = ",", header = None )