从字符串（或 DataFrame 对象）中提取时间数据的更有效方法

Question

I'm learning Python on my own and this is my first question here. Always was able to find everything needed already answered. Finally got something I believe it's worth to ask. It's just more specific task, which I don't even know what to search for.

One of our machines is generating a log file, which requires a lot of cleaning after loading to a DataFrame and before being able to use. Without going into too much details, a log file contains time record in a very weird format. It's build of minutes, seconds and miliseconds. I was able to decode it to seconds with use of a function shown below (and further convert it into time format with another one). It works fine, but this is a very basic function with a lot of if statemets.

My goal is to rewrite it into more less amateur looking, however the log time format puts some challenging limitations at least for me. And it's not helping that even the units are a combination of the same two letters.

Here are samples of all possible time record combinations:

test1 = 'T#3853m10s575ms'   # 231190.575 [seconds]
test2 = 'T#10s575ms'        # 10.575
test3 = 'T#3853m575ms'      # 231180.575
test4 = 'T#575ms'           # 0.575
test5 = 'T#3853m10s'        # 231190
test6 = 'T#10s'             # 10
test7 = 'T#3853m'           # 231180
test8 = 'T#0ms'             # 0

I've tried to write it in regular expression format as: T#[0-9]*m?[0-9]*s?[0-9]*ms? however there would always be at least one digit present and at least one unit.

Here is the logic I'm using inside the function: function diagram

And here is the function I apply to a raw time column in a DataFrame:

def convert_time(string):
    if string == 'T#0ms':
        return 0
    else:
        ms_ = False if string.find('ms') == -1 else True
        string = string[2:-2] if ms_ else string[2:]
        s_ = False if string.find('s') == -1 else True
        m_ = False if string.find('m') == -1 else True
        if m_ and s_ and ms_:
            m, temp = string.split('m')
            s, ms = temp.split('s')
            return int(m)*60 + int(s) + int(ms)*0.001
        elif not m_ and s_ and ms_:
            s, ms = string.split('s')
            return int(s) + 0.001 * int(ms)
        elif m_ and not s_ and ms_:
            m, ms = string.split('m')
            return 60*int(m) + 0.001 * int(ms)
        elif not m_ and not s_ and ms_:
            return int(string) * 0.001
        elif m_ and s_ and not ms_:
            m, s = string.split('m')
            return 60*int(m) + int(s[:-1])
        elif not m_ and s_ and not ms_:
            return int(string[:-1])
        elif m_ and not s_ and not ms_:
            return int(string[:-1]) * 60
        elif not m_ and not s_ and not ms_:
            return -1

Like mentioned above a lack of experience doesn't allow me to write a better function to result in similar output (or better, eg directly in time format). Hope that would be interesting enough to get some improvement hints. Thanks.

Answer 1

def str_to_sec(time_str):
    return_int = 0
    cur_int = 0

    # remove start characters and replace 'ms' with a single character as unit
    time_str = time_str.replace('T#','').replace('ms', 'p')

    # build multiplier matrix
    split_order = ['m', 's', 'p']
    multiplier = [60, 1, 0.001]
    calc_multiplier_dic = dict(zip(split_order, multiplier))

    # loop through string and update the cumulative time
    for ch in time_str:
        if ch.isnumeric():
            cur_int = cur_int * 10 + int(ch)
            continue
        if ch.isalpha():
            return_int += cur_int * calc_multiplier_dic[ch]
            cur_int = 0

    return return_int

Answer 2

Using regex:

import re

def f(x):
    x = x[2:]
    time = re.findall(r'\d+', x)
    timeType = re.findall(r'[a-zA-Z]+',x)
    #print(time,timeType)
    total = 0
    for i,j in zip(time,timeType):
        if j == 'm':
            total += 60*float(i) 
        elif j =='s':
            total+=float(i) 
        elif j == 'ms':
            total += float(i)/1000
    return total 

test1 = 'T#3853m10s575ms'   # 231190.575 [seconds]
test2 = 'T#10s575ms'        # 10.575
test3 = 'T#3853m575ms'      # 231180.575
test4 = 'T#575ms'           # 0.575
test5 = 'T#3853m10s'        # 231190
test6 = 'T#10s'             # 10
test7 = 'T#3853m'           # 231180
test8 = 'T#0ms'             # 0

arr = [test1,test2,test3,test4,test5,test6,test7,test8]

for t in arr:
    print(f(t))

Output:

231190.575
10.575
231180.575
0.575
231190.0
10.0
231180.0
0.0
[Finished in 0.7s]

Or you can make look code smaller if you have more time type like an hour, day etc..
Use map for it

import re
def symbol(j):
    if j == 'm':
        return 60 
    elif j =='s':
        return 1  
    elif j == 'ms':
        return .001

def f(x):
    x = x[2:]
    time = list(map(float,re.findall(r'\d+', x)))
    timeType = list(map(symbol,re.findall(r'[a-zA-Z]+',x)))
    #print(time,timeType)
    return sum([a*b for a,b in zip(timeType,time)]) 

test1 = 'T#3853m10s575ms'   # 231190.575 [seconds]
test2 = 'T#10s575ms'        # 10.575
test3 = 'T#3853m575ms'      # 231180.575
test4 = 'T#575ms'           # 0.575
test5 = 'T#3853m10s'        # 231190
test6 = 'T#10s'             # 10
test7 = 'T#3853m'           # 231180
test8 = 'T#0ms'             # 0

arr = [test1,test2,test3,test4,test5,test6,test7,test8]

for t in arr:
    print(f(t))