用换行将文本表读取到DataFrame中

Question

I want to read in a text file table as a DataFrame.

I have text files which contain representations of tables but there is some line-wrapping eg

clock_name         total_pwr     leakage_pwr
NA*                3.0675e-05    3.0675e-05
CLK1 (1.3333e+02)  6.8333e-02    6.0083e-03
LONGCLKNAME (3.3333e+02)
                   2.5707e-03    2.0459e-04     
LONGCLKNAME2 (3.3333e+02)
                   1.8777e-03    1.4462e-04     
CLK2 (3.3333e+02)   1.4190e-03    1.1886e-04    
CLK3 (3.3333e+02)
                   1.1038e-03    9.3498e-05  

Currently I read the table into a string line by line and try to convert it directly to a DataFrame using to_csv. The string will be:

string = "clock_name         total_pwr     leakage_pwr    \n\
NA*                3.0675e-05    3.0675e-05\n\
CLK1 (1.3333e+02)  6.8333e-02    6.0083e-03\n\
LONGCLKNAME\n\
 (3.3333e+02)  2.5707e-03    2.0459e-04\n\
LONGCLKNAME2\n\
 (3.3333e+02)  1.8777e-03    1.4462e-04\n\
CLK2 (3.3333e+02)   1.4190e-03    1.1886e-04\n\
CLK3 (3.3333e+02)  1.1038e-03    9.3498e-05"

So I've tried:

df = pd.read_csv(StringIO(string), sep='\t')

and I want the following:

   clock_name         total_pwr     leakage_pwr
0        NA*                3.0675e-05    3.0675e-05
1        CLK1 (1.3333e+02)  6.8333e-02    6.0083e-03
2  LONGCLKNAME (3.3333e+02)  2.5707e-03    2.0459...
3  LONGCLKNAME2 (3.3333e+02)  1.8777e-03    1.446...
4       CLK2 (3.3333e+02)   1.4190e-03    1.1886e-04
5        CLK3 (3.3333e+02)  1.1038e-03    9.3498e-05

but get:

  clock_name         total_pwr     leakage_pwr
0      NA*                3.0675e-05    3.0675e-05
1      CLK1 (1.3333e+02)  6.8333e-02    6.0083e-03
2                                      LONGCLKNAME
3           (3.3333e+02)  2.5707e-03    2.0459e-04
4                                     LONGCLKNAME2
5           (3.3333e+02)  1.8777e-03    1.4462e-04
6     CLK2 (3.3333e+02)   1.4190e-03    1.1886e-04
7      CLK3 (3.3333e+02)  1.1038e-03    9.3498e-05

How do I work around the line-wrap?

Answer 1

Ok, I am going to present you a very ugly code and if your statement that provided example is representative it will work. I can refactor the code, but later if so request.

import re
import pandas as pd

with open("data.txt", "r") as file:
    data = file.read()


data = data.split("\n")


result = []
ind = 0

while ind < len(data):

    if re.match(r"^[a-zA-Z].+\)$", data[ind]):
        result.append(data[ind].strip() + data[ind + 1])

        ind += 2

    else:
        result.append(data[ind])
        ind += 1


dict_result = {}
for i, x in enumerate(result):
    tmp = x.split()
    if len(tmp) == 3:
        dict_result[i] = tmp
    if len(tmp) == 4:
        dict_result[i] = [tmp[0] + tmp[1], tmp[2], tmp[3]]


df_final = pd.DataFrame(dict_result).T


col_names = df_final.iloc[0, :]
df_final.drop(0, axis=0, inplace=True)
df_final.columns = col_names

Here is the output generated by the above code:

Provided there are no surprises in your data (which is beyond what is posted) this should do the trick. The code, ugly as it is, hope it helps :)