带有正则表达式头定义的熊猫read_table

Question

For the data file formated like this:

("Time Step" "courantnumber_max" "courantnumber_avg" "flow-time")
0 0.55432343242 0.34323443432242 0.00001

I can use pd.read_table(filename, sep=' ', header=0) and it will get everything correct except for the very first header, "Time Step" .

Is there a way to specify a regex string for read_table() to use to parse out the header names?

I know a way to solve the issue is to just use regex to create a list of names for the read_table() function to use, but I figured there might/should be a way to directly express that in the import itself.

Edit: Here's what it returns as headers:

['("Time', 'Step"', 'courantnumber_max', 'courantnumber_avg', 'flow-time']

Answer 1

So it doesn't appear to be actually possible to do this inside the pandas.read_table() function. Below is posted the actual solution I ended up using to fix the problem:

import re

def get_headers(file, headerline, regexstring, exclude):
    # Get string of selected headerline
    with file.open() as f:
        for i, line in enumerate(f):
            if i == headerline-1:
                headerstring = line
            elif i > headerline-1:
                break

    # Parse headerstring
    reglist = re.split(regexstring, headerstring)

    # Filter entries in reglist
        #filter out blank strs
    filteredlist = list(filter(None, reglist)) 

        #filter out items in exclude list
    headerslist = []
    if exclude:
        for entry in filteredlist:
            if not entry in exclude:
                headerslist.append(entry)
    return headerslist

get_headers(filename, 3, r'(?:" ")|["\)\(]', ['\n'])

Code explanation:

get_headers() :

---

Arguments, file is a file object that contains the header. headerline is the line number (starting at 1) that the header names exist. regexstring is the pattern that will be fed into re.split() . Highly recommended that you prepend a r to the regex pattern. exclude is a list of miscellaneous strings that you want to be removed from the headerlist.

The regex pattern I used:

---

First up we have the pipe ( | ) symbol. This was done to separate both the "normal" split method (which is the " " ) and the other stuff that needs to be rid of (namely the parenthesis).

Starting with the first group: (?:" ") . We have the (...) since we want to match those characters in order. The " " is what we want to match as the stuff to split around. The ?: basically says to not capture the contents of the group. This is important/useful as otherwise re.split() will keep any groups as a separate item. See re.split() in documentation.

The second group is simply the other characters. Without them, the first and last items would be '("Time Step' and 'flow-time)\\n' . Note that this causes \\n to be treated as a separate entry to the list. This is why we use the exclude argument to fix that up after the fact.