使用 python 2.7 和 Pandas 将多个空格分隔的文件转换为 CSV？

Question

I have multiple problems that I'm trying to solve with this single file, but but my immediate concern is trying to convert this file which has fields which are delimited by variable numbers of white spaces between the fields to a standard CSV file without 1000 lines of genius level code. I know of one way to do it as I have done it in a previous project a couple of years ago by setting up functions similar to the old left$, mid$ and right$ functions in VB to select out particular characters from the row that I am interested in, because the data is very well defined and neatly parsed, ie, all the way down each column is the same size so I can grab the header row by using those functions to select out the field names of the columns, then go row by row using the same functions to pull the numeric data as strings with mid$(), write that to another file by adding in a "," between each written string, convert the strings back to floats and then I've got my CSV file with headers. But wow is that cumbersome and ugly - I want to use Pandas to make it more elegant, concise and sharp.

Here is a snippet of the first few lines of a data file - I have hundreds of them to process. I the actual files there are dozens more columns, this is just a sample that demonstrates the variable spaces between fields as delimiters.

DATE......................TIME.....................CH4.......................H2O
2021-04-01................01:47:45.407..............2.0063472018E+00..........1.2005321188E+00...
2021-04-01................01:47:46.336..............2.0063472018E+00..........1.2005321188E+00...
2021-04-01................01:47:47.244..............2.0063472018E+00..........1.2025918742E+00...
2021-04-01................01:47:49.049..............2.0059096902E+00..........1.2025918742E+00...

I also need tp parse the DATE and TIME columns as a timestamp object, which I've been trying to do from panda read_csv(parse_dates[[0,1]]), which almost works. I need the dates for plotting the x-axis labels for each series...but this is another problem for another post haha.

Thanks in advance for any assistance!!

john rainh2o

Answer 1

Using Pandas, specify the delimiter as a space (assuming your example has replaced spaces with dots). Next specify skipinitialspace=True . The date and time columns can be converted into a single datetime64 type:

import pandas as pd

df = pd.read_csv('input.txt', delimiter=' ', skipinitialspace=True, parse_dates=[['DATE', 'TIME']])

print(df)
print(df.dtypes)

This would give you:

                DATE_TIME       CH4       H2O
0 2021-04-01 01:47:45.407  2.006347  1.200532
1 2021-04-01 01:47:46.336  2.006347  1.200532
2 2021-04-01 01:47:47.244  2.006347  1.202592
3 2021-04-01 01:47:49.049  2.005910  1.202592

DATE_TIME    datetime64[ns]
CH4                 float64
H2O                 float64