tz_localize: KeyError: ('Asia/Singapore', u'occurred at index 0')

Question

Reference to: Python pandas convert unix timestamp with timezone into datetime

Did a search on this topic but still can't find the answer.

I have a dataframe whichh is the following format:

df timestamp
1  1549914000 
2  1549913400  
3  1549935000 
3  1549936800     
5  1549936200  

I use the following to convert epoch to date:

df['date'] = pd.to_datetime(df['timestamp'], unit='s')

This line will produce a date that is always 8 hours behind my local time.

So I followed the example in the link to use apply + tz.localize to Asia/Singapore, I tried the following code on the next line after the above code.

df['date'] = df.apply(lambda x: x['date'].tz_localize(x['Asia/Singapore']), axis=1)

but python return an error as below:

Traceback (most recent call last):
  File "/home/test/script.py", line 479, in <module>
    schedule.every(10).minutes.do(main).run()
  File "/opt/cloudera/parcels/Anaconda-4.0.0/lib/python2.7/site-packages/schedule/__init__.py", line 411, in run
    ret = self.job_func()
  File "/home/test/script.py", line 361, in main
    df['date'] = df.apply(localize_ts, axis = 1)
  File "/opt/cloudera/parcels/Anaconda-4.0.0/lib/python2.7/site-packages/pandas/core/frame.py", line 4877, in apply
    ignore_failures=ignore_failures)
  File "/opt/cloudera/parcels/Anaconda-4.0.0/lib/python2.7/site-packages/pandas/core/frame.py", line 4973, in _apply_standard
    results[i] = func(v)
  File "/home/test/script.py", line 359, in localize_ts
    return pd.to_datetime(row['date']).tz_localize(row['Asia/Singapore'])
  File "/opt/cloudera/parcels/Anaconda-4.0.0/lib/python2.7/site-packages/pandas/core/series.py", line 623, in __getitem__
    result = self.index.get_value(self, key)
  File "/opt/cloudera/parcels/Anaconda-4.0.0/lib/python2.7/site-packages/pandas/core/indexes/base.py", line 2574, in get_value
    raise e1
KeyError: ('Asia/Singapore', u'occurred at index 0')

Did I replace .tz_localize(x['tz']) in correctly?

Answer 1

As written, your code is looking for a column named Asia/Singapore . Try this instead:

df['date'] = df['date'].dt.tz_localize('Asia/Singapore')

Answer 2

you can try

import numpy as np
import pandas as pd
df = pd.DataFrame({'timestamp': [1549952400, 1549953600]},index=['1', '2'])
df['timestamp2'] =  df['timestamp'] + 28800
df['date'] = pd.to_datetime(df['timestamp2'], unit='s')
df = df.drop('timestamp2', 1)