从 Python 中的日志文件读取数据

Question

Im trying to read data from a log file I have in Python. Suppose the file is called data.log. The content of the file looks as follows:

# Performance log
# time, ff, T vector, dist, windnorth, windeast
0.00000000,0.00000000,0.00000000,0.00000000,0.00000000,0.00000000
1.00000000,3.02502604,343260.68655952,384.26845401,-7.70828175,-0.45288215
2.00000000,3.01495320,342124.21684440,767.95286901,-7.71506536,-0.45123853
3.00000000,3.00489957,340989.57100678,1151.05303883,-7.72185550,-0.44959182

I would like to obtain the last two columns and put them into two separate lists, such that I get an output like:

list1 = [-7.70828175, -7.71506536, -7.71506536]

list2 = [-0.45288215, -0.45123853, -0.44959182]

I have tried reading the data with the following code as shown below, but instead of separate columns and rows I just get one whole column with three rows in return.

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

file = open('data.log', 'r')

df = pd.read_csv('data.log', sep='\\s+')

df = list(df)

print (df[0])

Could someone indicate what I have to adjust in my code to obtain the required output as indicated above?

Thanks in advance!

Answer 1

The error comes in the sep attribute. If you remove it, it will use the default (the comma) which is the one you need:

eg

>>> import pandas as pd
>>> import numpy as np
>>> file = open('data.log', 'r')
>>> df = pd.read_csv('data.log')  # or use sep=','
>>> df = list(df)
>>> df[0]
'1.00000000'
>>> df[5]
'-0.45288215'

Plus use skiprows to get out the headers.

Answer 2

import pandas as pd 
df = pd.read_csv('sample.txt', skiprows=3, header=None, 
                 names=['time', 'ff', 'T vector', 'dist', 'windnorth', 'windeast'])
spam = list(df['windeast'])
print(spam)
# store a specific column in a list
df['wind_diff'] = df.windnorth - df['windeast'] # two different ways to access columsn
print(df)
print(df['wind_diff'])

output

[-0.45288215, -0.45123853, -0.44959182]
   time        ff       T vector         dist  windnorth  windeast  wind_diff
0   1.0  3.025026  343260.686560   384.268454  -7.708282 -0.452882  -7.255400
1   2.0  3.014953  342124.216844   767.952869  -7.715065 -0.451239  -7.263827
2   3.0  3.004900  340989.571007  1151.053039  -7.721856 -0.449592  -7.272264
0   -7.255400
1   -7.263827
2   -7.272264
Name: wind_diff, dtype: float64

Note, for creating plot in matplotlib you can work with pandas.Series directly, no need to store it in a list.