简体   繁体   中英

Read portion of file text into pandas dataframe with python

I have data text in following format for example:

HEADER NODE DATA, AIR
            -10000,    15.,    -1.0
HEADER CONDUCTOR DATA, AIR
            1,    AIR.10000,    S25D.1,    56.84441         $ AIR CONV
            2,    AIR.10000,    S25D.2,    56.45712         $ AIR CONV
            3,    AIR.10000,    S25D.3,    53.35623         $ AIR CONV
            4,    AIR.10000,    S25D.4,    45.09633         $ AIR CONV
            5,    AIR.10000,    S25D.9,    77.00067         $ AIR CONV
            6,    AIR.10000,    S25D.10,    80.35013         $ AIR CONV
            7,    AIR.10000,    S25D.11,    50.4933         $ AIR CONV
            8,    AIR.10000,    S25D.12,    91.61026         $ AIR CONV
            9,    AIR.10000,    S25D.13,    53.75025         $ AIR CONV
            10,    AIR.10000,    S25D.14,    75.68577         $ AIR CONV
            11,    AIR.10000,    S25D.15,    110.0111         $ AIR CONV
            12,    AIR.10000,    S25D.16,    114.7913         $ AIR CONV
            13,    AIR.10000,    S25D.17,    81.12207         $ AIR CONV
            14,    AIR.10000,    S25D.18,    72.80061         $ AIR CONV
            15,    AIR.10000,    S25D.19,    72.21327         $ AIR CONV
            16,    AIR.10000,    S25D.20,    90.99183         $ AIR CONV
            17,    AIR.10000,    S25D.21,    66.35648         $ AIR CONV
            18,    AIR.10000,    S25D.22,    76.9787         $ AIR CONV
            19,    AIR.10000,    S25D.23,    52.46601         $ AIR CONV
            20,    AIR.10000,    S25D.24,    68.30105         $ AIR CONV
            21,    AIR.10000,    S25D.25,    114.0903         $ AIR CONV
            22,    AIR.10000,    S25D.26,    70.51425         $ AIR CONV
            23,    AIR.10000,    S25D.27,    36.39104         $ AIR CONV
            24,    AIR.10000,    S25D.28,    78.88668         $ AIR CONV
            25,    AIR.10000,    S25D.29,    70.77477         $ AIR CONV
            26,    AIR.10000,    S25D.30,    87.04871         $ AIR CONV
            27,    AIR.10000,    S25D.31,    64.50918         $ AIR CONV
            28,    AIR.10000,    S25D.32,    76.20373         $ AIR CONV
            29,    AIR.10000,    S25D.33,    90.69899         $ AIR CONV
            30,    AIR.10000,    S25D.34,    83.89708         $ AIR CONV
            31,    AIR.10000,    S25D.35,    88.59924         $ AIR CONV
            32,    AIR.10000,    S25D.36,    72.06826         $ AIR CONV
            33,    AIR.10000,    S25D.65,    157.6095         $ AIR CONV
            34,    AIR.10000,    S25D.66,    140.3513         $ AIR CONV
            35,    AIR.10000,    S25D.67,    215.1395         $ AIR CONV
            36,    AIR.10000,    S25D.68,    131.7191         $ AIR CONV
            37,    AIR.10000,    S25D.69,    173.7129         $ AIR CONV
            38,    AIR.10000,    S25D.70,    106.8654         $ AIR CONV
            39,    AIR.10000,    S25D.71,    153.2247         $ AIR CONV
            40,    AIR.10000,    S25D.72,    151.2406         $ AIR CONV
            41,    AIR.10000,    S25D.73,    136.0113         $ AIR CONV
            42,    AIR.10000,    S25D.74,    217.2005         $ AIR CONV
            43,    AIR.10000,    S25D.75,    121.0298         $ AIR CONV
            44,    AIR.10000,    S25D.76,    178.8008         $ AIR CONV
            45,    AIR.10000,    S25D.77,    160.7517         $ AIR CONV
            46,    AIR.10000,    S25D.78,    132.528         $ AIR CONV
            47,    AIR.10000,    S25D.79,    126.503         $ AIR CONV
            48,    AIR.10000,    S25D.80,    223.1756         $ AIR CONV
            49,    AIR.10000,    S25D.81,    151.6868         $ AIR CONV
            50,    AIR.10000,    S25D.82,    176.8964         $ AIR CONV
            51,    AIR.10000,    S25D.83,    155.6967         $ AIR CONV
            52,    AIR.10000,    S25D.84,    153.6341         $ AIR CONV
            53,    AIR.10000,    S25D.85,    110.9477         $ AIR CONV
            54,    AIR.10000,    S25D.86,    165.114         $ AIR CONV
            55,    AIR.10000,    S25D.87,    139.9079         $ AIR CONV
            56,    AIR.10000,    S25D.88,    137.9431         $ AIR CONV
            57,    AIR.10000,    S25D.89,    138.1562         $ AIR CONV
            58,    AIR.10000,    S25D.90,    204.7094         $ AIR CONV
            59,    AIR.10000,    S25D.91,    159.8541         $ AIR CONV
            60,    AIR.10000,    S25D.92,    174.5026         $ AIR CONV
            61,    AIR.10000,    S25D.93,    160.5398         $ AIR CONV
            62,    AIR.10000,    S25D.94,    143.0478         $ AIR CONV
            63,    AIR.10000,    S25D.95,    175.8055         $ AIR CONV
            64,    AIR.10000,    S25D.96,    99.97353         $ AIR CONV
            65,    AIR.10000,    S25D.97,    119.0528         $ AIR CONV
            66,    AIR.10000,    S25D.98,    147.533         $ AIR CONV
            67,    AIR.10000,    S25D.99,    122.7583         $ AIR CONV
            68,    AIR.10000,    S25D.100,    191.2955         $ AIR CONV
            69,    AIR.10000,    S25D.101,    197.3013         $ AIR CONV
            70,    AIR.10000,    S25D.102,    173.0556         $ AIR CONV
            71,    AIR.10000,    S25D.103,    193.2726         $ AIR CONV
            72,    AIR.10000,    S25D.104,    128.1502         $ AIR CONV
            73,    AIR.10000,    S25D.105,    102.519         $ AIR CONV
            74,    AIR.10000,    S25D.106,    180.8979         $ AIR CONV
            75,    AIR.10000,    S25D.107,    160.9948         $ AIR CONV
            76,    AIR.10000,    S25D.108,    197.6314         $ AIR CONV
            77,    AIR.10000,    S25D.109,    162.5766         $ AIR CONV
            78,    AIR.10000,    S25D.110,    143.3984         $ AIR CONV
            79,    AIR.10000,    S25D.111,    265.7273         $ AIR CONV
            80,    AIR.10000,    S25D.112,    90.36677         $ AIR CONV
            81,    AIR.10000,    S25D.113,    146.033         $ AIR CONV
            82,    AIR.10000,    S25D.114,    182.0126         $ AIR CONV
            83,    AIR.10000,    S25D.115,    142.1242         $ AIR CONV
            84,    AIR.10000,    S25D.116,    156.4711         $ AIR CONV
            85,    AIR.10000,    S25D.117,    263.3278         $ AIR CONV
            86,    AIR.10000,    S25D.118,    133.7923         $ AIR CONV
            87,    AIR.10000,    S25D.119,    134.4939         $ AIR CONV
            88,    AIR.10000,    S25D.120,    87.09295         $ AIR CONV
            89,    AIR.10000,    S25D.121,    168.8858         $ AIR CONV
            90,    AIR.10000,    S25D.122,    133.792         $ AIR CONV
            91,    AIR.10000,    S25D.123,    194.2227         $ AIR CONV
            92,    AIR.10000,    S25D.124,    116.8372         $ AIR CONV
            93,    AIR.10000,    S25D.125,    125.5161         $ AIR CONV
            94,    AIR.10000,    S25D.126,    54.22872         $ AIR CONV

And I want to read it into pandas data frame for further analysis. As you can see the header is different for each subset and also the format of the data. I thought to use re module and pandas for reading the data into frame but don't know how to do it in most efficent way. Also as you can see the AIR string is common to both headers. In the full file there would be more such strings so I would like to create for each string like that a class which attributes (node, conductor) will be a pd dataframe with corresponding data.

Any help will be appreciated:)

edit: Following Alexandre suggestion I've altered a code a little bit but still didn't find a way to put the data into class

# import modules
import pandas as pd
from io import StringIO

class TableData:
    def __init__(self,name):
        self.name = name
        self.nodes_table = self.add_nodes()
        self.conductor_table = self.add_conductors()
    def add_nodes(self):
        pass
    def add_conductors(self):
        pass

out = []
submodels = set()
with open("case1.cc", "r") as f:
    sub = ""
    for line in f.readlines():
        if ("VARIABLES" in line) or ("OPTIONS" in line): # skip non relevant
                                                         # blocks
            continue
        if line.startswith('C'):  # skip comments line
            continue
        if ("HEADER" in line) and ("DATA" in line):
            # Get submodel name and add to set if not present
            submodels.add(line.split(',')[1])
            # Get type of data
            type = line.split()[1]
            # Skip first insertion
            if sub != "":
                out.append(sub)
                sub = ""
        else: sub += line
    # Append last sub (outside for loop)
    out.append(sub)

# "out" is a list of pandas dataframes
out = [pd.read_csv(StringIO(o), sep=r",*\s{2,}", header=None, engine="python") for o in out]

also more complete example file:

HEADER OPTIONS
C    SINDA Data generated with Thermal Desktop 6.0 Patch 2
C    Generated on Fri Sep 01 14:39:30 2017
C    From file: thermal.dwg
C    Case Set: Case Set 1
C    TDUNITS,   Energy   =    J
C    TDUNITS,   Time     =    sec
C    TDUNITS,   Temp     =    C
C    TDUNITS,   Mass     =    kg
C    TDUNITS,   Length   =    m
C    TDUNITS,   Orbit    =    km
C    TDUNITS,   Pressure =    Pa
C    TDUNITS,   Force    =    N
C    TDUNITS,   Angle    =    Degrees
C    TDUNITS,   Volt     =    volt
C    TDUNITS,   Current  =    amp
C    TDUNITS,   Resistance =   ohm
HEADER NODE DATA, AIR
            -10000,    15.,    -1.0
HEADER CONDUCTOR DATA, AIR
            1,    AIR.10000,    S25D.1,    56.84441         $ AIR CONV
            2,    AIR.10000,    S25D.2,    56.45712         $ AIR CONV
            3,    AIR.10000,    S25D.3,    53.35623         $ AIR CONV
            4,    AIR.10000,    S25D.4,    45.09633         $ AIR CONV
            5,    AIR.10000,    S25D.9,    77.00067         $ AIR CONV
            6,    AIR.10000,    S25D.10,    80.35013         $ AIR CONV
            7,    AIR.10000,    S25D.11,    50.4933         $ AIR CONV
            8,    AIR.10000,    S25D.12,    91.61026         $ AIR CONV
            9,    AIR.10000,    S25D.13,    53.75025         $ AIR CONV
            10,    AIR.10000,    S25D.14,    75.68577         $ AIR CONV
            11,    AIR.10000,    S25D.15,    110.0111         $ AIR CONV
            12,    AIR.10000,    S25D.16,    114.7913         $ AIR CONV
            13,    AIR.10000,    S25D.17,    81.12207         $ AIR CONV
            14,    AIR.10000,    S25D.18,    72.80061         $ AIR CONV
            15,    AIR.10000,    S25D.19,    72.21327         $ AIR CONV
            16,    AIR.10000,    S25D.20,    90.99183         $ AIR CONV
            17,    AIR.10000,    S25D.21,    66.35648         $ AIR CONV
            18,    AIR.10000,    S25D.22,    76.9787         $ AIR CONV
            19,    AIR.10000,    S25D.23,    52.46601         $ AIR CONV
            20,    AIR.10000,    S25D.24,    68.30105         $ AIR CONV
            21,    AIR.10000,    S25D.25,    114.0903         $ AIR CONV
            22,    AIR.10000,    S25D.26,    70.51425         $ AIR CONV
            23,    AIR.10000,    S25D.27,    36.39104         $ AIR CONV
            24,    AIR.10000,    S25D.28,    78.88668         $ AIR CONV
            25,    AIR.10000,    S25D.29,    70.77477         $ AIR CONV
            26,    AIR.10000,    S25D.30,    87.04871         $ AIR CONV
            27,    AIR.10000,    S25D.31,    64.50918         $ AIR CONV
            28,    AIR.10000,    S25D.32,    76.20373         $ AIR CONV
            29,    AIR.10000,    S25D.33,    90.69899         $ AIR CONV
            30,    AIR.10000,    S25D.34,    83.89708         $ AIR CONV
            31,    AIR.10000,    S25D.35,    88.59924         $ AIR CONV
            32,    AIR.10000,    S25D.36,    72.06826         $ AIR CONV
            33,    AIR.10000,    S25D.65,    157.6095         $ AIR CONV
            34,    AIR.10000,    S25D.66,    140.3513         $ AIR CONV
            35,    AIR.10000,    S25D.67,    215.1395         $ AIR CONV
            36,    AIR.10000,    S25D.68,    131.7191         $ AIR CONV
            37,    AIR.10000,    S25D.69,    173.7129         $ AIR CONV
            38,    AIR.10000,    S25D.70,    106.8654         $ AIR CONV
            39,    AIR.10000,    S25D.71,    153.2247         $ AIR CONV
            40,    AIR.10000,    S25D.72,    151.2406         $ AIR CONV
            41,    AIR.10000,    S25D.73,    136.0113         $ AIR CONV
            42,    AIR.10000,    S25D.74,    217.2005         $ AIR CONV
            43,    AIR.10000,    S25D.75,    121.0298         $ AIR CONV
            44,    AIR.10000,    S25D.76,    178.8008         $ AIR CONV
            45,    AIR.10000,    S25D.77,    160.7517         $ AIR CONV
            46,    AIR.10000,    S25D.78,    132.528         $ AIR CONV
            47,    AIR.10000,    S25D.79,    126.503         $ AIR CONV
            48,    AIR.10000,    S25D.80,    223.1756         $ AIR CONV
            49,    AIR.10000,    S25D.81,    151.6868         $ AIR CONV
            50,    AIR.10000,    S25D.82,    176.8964         $ AIR CONV
            51,    AIR.10000,    S25D.83,    155.6967         $ AIR CONV
            52,    AIR.10000,    S25D.84,    153.6341         $ AIR CONV
            53,    AIR.10000,    S25D.85,    110.9477         $ AIR CONV
            54,    AIR.10000,    S25D.86,    165.114         $ AIR CONV
            55,    AIR.10000,    S25D.87,    139.9079         $ AIR CONV
            56,    AIR.10000,    S25D.88,    137.9431         $ AIR CONV
            57,    AIR.10000,    S25D.89,    138.1562         $ AIR CONV
            58,    AIR.10000,    S25D.90,    204.7094         $ AIR CONV
            59,    AIR.10000,    S25D.91,    159.8541         $ AIR CONV
            60,    AIR.10000,    S25D.92,    174.5026         $ AIR CONV
            61,    AIR.10000,    S25D.93,    160.5398         $ AIR CONV
            62,    AIR.10000,    S25D.94,    143.0478         $ AIR CONV
            63,    AIR.10000,    S25D.95,    175.8055         $ AIR CONV
            64,    AIR.10000,    S25D.96,    99.97353         $ AIR CONV
            65,    AIR.10000,    S25D.97,    119.0528         $ AIR CONV
            66,    AIR.10000,    S25D.98,    147.533         $ AIR CONV
            67,    AIR.10000,    S25D.99,    122.7583         $ AIR CONV
            68,    AIR.10000,    S25D.100,    191.2955         $ AIR CONV
            69,    AIR.10000,    S25D.101,    197.3013         $ AIR CONV
            70,    AIR.10000,    S25D.102,    173.0556         $ AIR CONV
            71,    AIR.10000,    S25D.103,    193.2726         $ AIR CONV
            72,    AIR.10000,    S25D.104,    128.1502         $ AIR CONV
            73,    AIR.10000,    S25D.105,    102.519         $ AIR CONV
            74,    AIR.10000,    S25D.106,    180.8979         $ AIR CONV
            75,    AIR.10000,    S25D.107,    160.9948         $ AIR CONV
            76,    AIR.10000,    S25D.108,    197.6314         $ AIR CONV
            77,    AIR.10000,    S25D.109,    162.5766         $ AIR CONV
            78,    AIR.10000,    S25D.110,    143.3984         $ AIR CONV
            79,    AIR.10000,    S25D.111,    265.7273         $ AIR CONV
            80,    AIR.10000,    S25D.112,    90.36677         $ AIR CONV
            81,    AIR.10000,    S25D.113,    146.033         $ AIR CONV
            82,    AIR.10000,    S25D.114,    182.0126         $ AIR CONV
            83,    AIR.10000,    S25D.115,    142.1242         $ AIR CONV
            84,    AIR.10000,    S25D.116,    156.4711         $ AIR CONV
            85,    AIR.10000,    S25D.117,    263.3278         $ AIR CONV
            86,    AIR.10000,    S25D.118,    133.7923         $ AIR CONV
            87,    AIR.10000,    S25D.119,    134.4939         $ AIR CONV
            88,    AIR.10000,    S25D.120,    87.09295         $ AIR CONV
            89,    AIR.10000,    S25D.121,    168.8858         $ AIR CONV
            90,    AIR.10000,    S25D.122,    133.792         $ AIR CONV
            91,    AIR.10000,    S25D.123,    194.2227         $ AIR CONV
            92,    AIR.10000,    S25D.124,    116.8372         $ AIR CONV
            93,    AIR.10000,    S25D.125,    125.5161         $ AIR CONV
            94,    AIR.10000,    S25D.126,    54.22872         $ AIR CONV

HEADER VARIABLES 0, U1
C Heat Load-U1 POWER[U1]::DEA
      S67C.Q1 = S67C.Q1 + 27.77842
      S67C.Q2 = S67C.Q2 + 83.60381
      S67C.Q3 = S67C.Q3 + 54.82939
      S67C.Q4 = S67C.Q4 + 38.54834
      S67C.Q5 = S67C.Q5 + 27.77842
      S67C.Q6 = S67C.Q6 + 27.86794
      S67C.Q7 = S67C.Q7 + 54.82939
      S67C.Q8 = S67C.Q8 + 63.72669
      S67C.Q9 = S67C.Q9 + 112.6502
      S67C.Q10 = S67C.Q10 + 44.72019
      S67C.Q11 = S67C.Q11 + 56.17543
      S67C.Q12 = S67C.Q12 + 37.76753
      S67C.Q13 = S67C.Q13 + 41.53661
      S67C.Q14 = S67C.Q14 + 38.89015
      S67C.Q15 = S67C.Q15 + 56.3575
      S67C.Q16 = S67C.Q16 + 134.1606
      S67C.Q17 = S67C.Q17 + 112.4682
      S67C.Q18 = S67C.Q18 + 62.94588
      S67C.Q19 = S67C.Q19 + 124.6098
      S67C.Q20 = S67C.Q20 + 64.84931
      S67C.Q21 = S67C.Q21 + 173.5864
      S67C.Q22 = S67C.Q22 + 289.9135
      S67C.Q23 = S67C.Q23 + 173.7685
      S67C.Q24 = S67C.Q24 + 96.63783
HEADER NODE DATA, U2
HEADER CONDUCTOR DATA, U2
C Contact - Face Contactor-u2 contact[U2][1]::D4B
            1,    S25D.101,    SB82.2,    0.222749
            2,    S25D.101,    SB82.4,    4.481803
            3,    S25D.101,    SB82.11,    0.3112232
            4,    S25D.101,    SB82.12,    3.869354
            5,    S25D.101,    SB82.18,    0.1023536
            6,    S25D.101,    SB82.19,    1.919002
            7,    S25D.116,    SB82.2,    2.166497
            8,    S25D.116,    SB82.9,    0.1857799
            9,    S25D.116,    SB82.12,    0.2024991
            10,    S25D.116,    SB82.18,    0.2656026
            11,    S25D.117,    SB82.4,    0.3480346
            12,    S25D.117,    SB82.11,    0.4082713
            13,    S25D.117,    SB82.19,    0.04685081
            14,    S25D.123,    SB82.1,    6.771217
            15,    S25D.123,    SB82.2,    5.789176
            16,    S25D.123,    SB82.3,    7.999367
            17,    S25D.123,    SB82.4,    1.898946
            18,    S25D.123,    SB82.9,    9.955903
            19,    S25D.123,    SB82.10,    9.947601
            20,    S25D.123,    SB82.11,    9.160776
            21,    S25D.123,    SB82.12,    5.900825
            22,    S25D.123,    SB82.17,    14.49436
            23,    S25D.123,    SB82.18,    18.01836
            24,    S25D.123,    SB82.19,    12.362
            25,    S25D.123,    SB82.20,    18.17145
HEADER VARIABLES 0, U2
C Heat Load-u2 power[U2]::D3F
      SB82.Q1 = SB82.Q1 + 13.14304
      SB82.Q2 = SB82.Q2 + 24.73277
      SB82.Q3 = SB82.Q3 + 23.54379
      SB82.Q4 = SB82.Q4 + 13.10696
      SB82.Q5 = SB82.Q5 + 23.59044
      SB82.Q6 = SB82.Q6 + 13.11356
      SB82.Q7 = SB82.Q7 + 13.13522
      SB82.Q8 = SB82.Q8 + 23.62498
      SB82.Q9 = SB82.Q9 + 17.55859
      SB82.Q10 = SB82.Q10 + 25.07847
      SB82.Q11 = SB82.Q11 + 25.0472
      SB82.Q12 = SB82.Q12 + 25.0852
      SB82.Q13 = SB82.Q13 + 35.30753
      SB82.Q14 = SB82.Q14 + 25.07439
      SB82.Q15 = SB82.Q15 + 25.34528
      SB82.Q16 = SB82.Q16 + 25.05068
      SB82.Q17 = SB82.Q17 + 49.6953
      SB82.Q18 = SB82.Q18 + 45.39197
      SB82.Q19 = SB82.Q19 + 53.80839
      SB82.Q20 = SB82.Q20 + 33.80833
      SB82.Q21 = SB82.Q21 + 41.50079
      SB82.Q22 = SB82.Q22 + 33.93809
      SB82.Q23 = SB82.Q23 + 53.82713
      SB82.Q24 = SB82.Q24 + 36.49191
HEADER NODE DATA, U3
HEADER CONDUCTOR DATA, U3
C Contact - Face Contactor-u3 contact[U3][1]::D53
            1,    S25D.69,    SC04.6,    35.
            2,    S25D.69,    SC04.13,    67.97
            3,    S25D.69,    SC04.14,    66.71
            4,    S25D.69,    SC04.16,    8.82
            5,    S25D.80,    SC04.8,    34.72
            6,    S25D.80,    SC04.14,    0.49
            7,    S25D.80,    SC04.15,    38.78
            8,    S25D.80,    SC04.16,    37.31
            9,    S25D.100,    SC04.7,    35.
            10,    S25D.100,    SC04.13,    2.66
            11,    S25D.100,    SC04.14,    121.38
            12,    S25D.100,    SC04.15,    97.72
            13,    S25D.100,    SC04.16,    51.94
            14,    S25D.111,    SC04.5,    35.
            15,    S25D.111,    SC04.8,    0.28
            16,    S25D.111,    SC04.13,    69.37
            17,    S25D.111,    SC04.14,    21.42
            18,    S25D.111,    SC04.15,    3.5
            19,    S25D.111,    SC04.16,    111.93
HEADER VARIABLES 0, U3
C Heat Load-u3 power[U3]::D40
      SC04.Q1 = SC04.Q1 + 23.4375
      SC04.Q2 = SC04.Q2 + 31.25
      SC04.Q3 = SC04.Q3 + 15.625
      SC04.Q4 = SC04.Q4 + 31.25
      SC04.Q5 = SC04.Q5 + 23.4375
      SC04.Q6 = SC04.Q6 + 31.25
      SC04.Q7 = SC04.Q7 + 46.875
      SC04.Q8 = SC04.Q8 + 31.25
      SC04.Q9 = SC04.Q9 + 164.0625
      SC04.Q10 = SC04.Q10 + 218.75
      SC04.Q11 = SC04.Q11 + 140.625
      SC04.Q12 = SC04.Q12 + 101.5625
      SC04.Q13 = SC04.Q13 + 70.3125
      SC04.Q14 = SC04.Q14 + 156.25
      SC04.Q15 = SC04.Q15 + 109.375
      SC04.Q16 = SC04.Q16 + 257.8125
      SC04.Q17 = SC04.Q17 + 46.875
HEADER SUBROUTINE
      SUBROUTINE TDHTR
C     This routine is to hold heaters at their midpoint temps
C     for steady state solutions.
      RETURN
      END
      SUBROUTINE TDREL
C     This routine is to release heaters for transient runs
C     for steady state solutions.
F     RETURN
F     END
F     SUBROUTINE TDHTOT
C     This routine prints out heater ontime and cycle summaries
F     RETURN
F     END
F     SUBROUTINE TDHTRST
C     This routine resets heater statistics
F     RETURN
F     END
F     SUBROUTINE TDPREBL
C     This routine executes logic before the build statement
M     CALL COMMON
F     RETURN
F     END
F     SUBROUTINE TDPOSTBL
C     This routine executes logic after the build statement
M     CALL COMMON
F     RETURN
F     END
F     SUBROUTINE TDPOSTSL
C     This routine executes logic after the solution
M     CALL COMMON
F     RETURN
F     END
F     SUBROUTINE BASEPLOT
      CALL COMMON
F     RETURN
F     END

'''

You can try the following approach:

  1. Read the file line by line using f.readlines()
  2. For each line, check if the line contains HEADER .
    • If it's the case, that means this a new sub df. Add it to the output
    • Else: append the current line to the current sub data
  3. Convert the list of sub df in text format to pandas dataframe. This discussion explains how. Note:
    • The regex used is ,*\s{2,} that means:
      • match any " , " with " ,* "
      • match any all 2 and more space using "\s{2,}
    • The argument header=None is required either the first line will be used add columns name.

Full code

# import modules
import pandas as pd
from io import StringIO

out = []
with open("data.txt", "r") as f:
    sub = ""
    for line in f.readlines():
        if "HEADER" in line:
            # Skip first insertion
            if sub != "":
                out.append(sub)
                sub = ""
        else: sub += line
    # Append last sub (outside for loop)
    out.append(sub)

# "out" is a list of pandas dataframes
out = [pd.read_csv(StringIO(o), sep=r",*\s{2,}", header=None, engine="python") for o in out]
print(out[0])
#        0     1    2
# 0 -10000  15.0 -1.0

print(out[1])
#      0          1         2          3           4
# 0    1  AIR.10000    S25D.1   56.84441  $ AIR CONV
# 1    2  AIR.10000    S25D.2   56.45712  $ AIR CONV
# 2    3  AIR.10000    S25D.3   53.35623  $ AIR CONV
# 3    4  AIR.10000    S25D.4   45.09633  $ AIR CONV
# 4    5  AIR.10000    S25D.9   77.00067  $ AIR CONV
# ..  ..        ...       ...        ...         ...
# 89  90  AIR.10000  S25D.122  133.79200  $ AIR CONV
# 90  91  AIR.10000  S25D.123  194.22270  $ AIR CONV
# 91  92  AIR.10000  S25D.124  116.83720  $ AIR CONV
# 92  93  AIR.10000  S25D.125  125.51610  $ AIR CONV
# 93  94  AIR.10000  S25D.126   54.22872  $ AIR CONV

# [94 rows x 5 columns]
print(out[1].columns)
# Int64Index([0, 1, 2, 3, 4], dtype='int64')

Hope that helps!

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM