How to split huge CSV file?

Question

I have a csv file with 40k rows and 1 column because all the data is separated by semicolons.

insert_date;currency_from;currency_to;currency_value
0   2017-01-02 00:00:00.000;EUR;TL;3.7073
1   2017-01-02 00:00:00.000;USD;TL;3.5445
2   2017-01-02 00:00:00.000;GBP;TL;4.3510
3   2017-01-02 00:00:00.000;BTC;USD;0.0000
4   2017-01-02 00:00:00.000;EUR;USD;1.0459

This is what my data looks like as a pandas dataframe. I want to split on the semicolon to make separate columns.

Answer 1

In pandas, this is done with the sep parameter per the docs :

import pandas as pd

df = pd.read_csv('/path/to/file.csv', sep=';')

Answer 2

Do you mean you would like to have a list of dicts?

import csv

with open('your/path/to/your/file.csv') as f:
    data = [i for i in csv.DictReader(f, delimiter=';')

process_your_data()

Answer 3

Try using str.spilt()

Syntax: Series.str.split(pat=None, n=-1, expand=False) Parameters:

pat: String value, separator or delimiter to separate string at. n: Numbers of max separations to make in a single string, default is -1 which means all. expand: Boolean value, returns a data frame with different value in different columns if True. Else it returns a series with list of strings.