Deserialize DateTime field in json file using Python (Pandas)

Question

I am parsing a json file using Pandas in Python. There is a field called DateTime with the following string in it: 1581251737000 . Does anybody know the format of this DateTime field so that I can parse it using the pandas.to_datetime() function?

Answer 1

The quickest way to do this is by using pandas:

import pandas as pd
from datetime import datetime

x = "1581251737000"
pd.to_datetime(x, unit="ms")

#Output
Timestamp('2020-02-09 12:35:37')

You can use strftime to convert this to your desired format:

pd.to_datetime(x, unit="ms").strftime("%Y/%m/%d")

#Output
'2020/02/09'

Answer 2

this is called epoch timestamp in milliseconds format, you can remove trailing there zeros and convert it to readable datetime like this:

time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(1581251737))

Answer 3

While using pandas, you can define your custom format to aid the pd.to_datetime function like this:

pd.to_datetime('13000101', format='%Y%m%d', errors='ignore')

In your case, you will first have to convert this timestamp into a date-time stamp by providing the units argument as ms which I'm guessing is true in your case. I'd recommend you define your own format to avoid any trailing zeros

By default, pandas uses YYYY MM DD HH MM SS MSMSMS

Check out the documentation here