简体   繁体   English

如何在Python中读取大型文本文件?

[英]How to read a large text file in Python?

I am using Enthought Canopy (a set of many different Python Library packages eg NumPy, Pandas,etc) for data analysis. 我正在使用Enthought Canopy(一组许多不同的Python库软件包,例如NumPy,Pandas等)进行数据分析。 I am trying to read a text file and create a dataframe out of it. 我正在尝试读取文本文件并从中创建数据框。 The text file has 1180598 rows and 18 columns. 文本文件具有1180598行和18列。 All columns have numbers in them. 所有列中都有数字。 I wrote following code for reading and naming data columns: 我编写了以下代码来读取和命名数据列:

from pandas import DataFrame, read_csv
import matplotlib.pyplot as plt

import pandas as pd

print 'Pandas Version ' + pd.__version__
Pandas Version 0.12.0

location=r'C:\UMAIR\Directed Studies\US-101 Data\Main Data\US-101-Main-Data\vehicle-trajectory-data\0750am-0805am\tra.txt'

df=read_csv(location, names=['Vehicle ID','Frame ID','Total Frames','Global Time','Local X','Local Y','Global X','Global Y','Vehicle Length','Vehicle Width','Vehicle Class','Vehicle Velocity','Vehicle Acceleration','Lane Identification','Preceding Vehicle','Following Vehicle','Spacing','Headway'])

df
Out[41]: 
<class 'pandas.core.frame.DataFrame'>
Int64Index: 1180598 entries, 0 to 1180597
Data columns (total 18 columns):
Vehicle ID              1180598  non-null values
Frame ID                0  non-null values
Total Frames            0  non-null values
Global Time             0  non-null values
Local X                 0  non-null values
Local Y                 0  non-null values
Global X                0  non-null values
Global Y                0  non-null values
Vehicle Length          0  non-null values
Vehicle Width           0  non-null values
Vehicle Class           0  non-null values
Vehicle Velocity        0  non-null values
Vehicle Acceleration    0  non-null values
Lane Identification     0  non-null values
Preceding Vehicle       0  non-null values
Following Vehicle       0  non-null values
Spacing                 0  non-null values
Headway                 0  non-null values
dtypes: float64(17), object(1) 

As you can see from Out[41], the file was read to have 1 column only. 从Out [41]中可以看到,文件被读取为只有1列。 What should I do to let Python know that my file has 18 columns so that it is read the way it is meant to be? 我应该怎么做才能让Python知道我的文件有18列,以便按照原意读取文件?

这将正确导入您的数据集:

df = pd.read_csv(location, names=names, header=None, delim_whitespace=True)

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM