Converting string time with milliseconds, in a python column, to timestamp

Question

I have a column("arrival_time") in a dataframe ("df") that contains string values of time in the format "H:M:S:f" (f --> milliseconds). Some only have "H:M:S", so the format is not consistent throughout the column.

I've tried converting to timestamp to get numerical representation of the string times.

Sample Data:

0          20:43:09:01
1          06:00:16
2          06:30:21
3          07:00:03
4          06:32:43
5          07:33:31
6          07:37:39:09
7          07:49:01
8          08:52:05
9          08:29:44:10

import time
import datetime

def conv_date(myDate):
    try:
        if str(myDate).count(":") == 3:
            dt = datetime.datetime.strptime(myDate,'%H:%M:%S,%f').timestamp()
        else:
            dt = datetime.datetime.strptime(myDate,'%H:%M:%S').timestamp()
    except:
        return float('NaN')
    return dt

# some values are data type 'float' so converted everything to string
df["arrival_time"] = df["arrival_time"].astype(str).apply(conv_date)

Output:
0         -2.208885e+09
1         -2.208938e+09
2         -2.208937e+09
3         -2.208935e+09
4         -2.208936e+09
5         -2.208933e+09

I get a negative timestamp when I expected a positive value.

Answer 1

Try to append current day with the data and use this:

p = pd.to_datetime("2019-04-22 05:03:35",format='%Y-%m-%d %H:%M:%S.%f')
p.timestamp()
1555909415.0

p = pd.to_datetime("2019-04-22 05:03:35.74",format='%Y-%m-%d %H:%M:%S.%f')
p.timestamp()
1555909415.74

You can append the current date like this:

df.date = df.date.apply(lambda x: datetime.now().date().strftime("%Y-%m-%d") + " " + x)

and the use this to apply to whole dataframe use this:

df["event_timestamp"] = pd.to_datetime(df["event_timestamp"], format='%Y-%m-%d %H:%M:%S.%f')