Converting datetime to unix timestamp

Question

Today me and some coworkers were trying to correctly convert a datetime into a timestamp that was in a non-local timezone. After much discussion, we still weren't sure about how to, in a general sense, correctly convert a datetime into a timestamp, and were only partially sure of how to do it for our use case: a datetime instance in the America/New_York timezone from pytz.

So, I figured I'd run through some permutations and try to empirically figure it out, since reading the docs and trying to logically reason it out wasn't getting me anywhere.

Here's in the input data:

pst = pytz.timezone('America/Los_Angeles')
est = pytz.timezone('America/New_York')
utc = pytz.timezone('UTC')

epoch = int(time.time())
# local time for me is Pacific
local_naive = datetime.datetime.fromtimestamp(epoch)
local_aware = datetime.datetime.fromtimestamp(epoch, pst)
est_aware = datetime.datetime.fromtimestamp(epoch, est)
utc_naive = datetime.datetime.utcfromtimestamp(epoch)
utc_aware = datetime.datetime.fromtimestamp(epoch, utc)

And the result was computed in four ways:

1. time.mktime(dt.timetuple())
2. time.mktime(dt.utctimetuple())
3. calendar.timegm(dt.timetuple())
4. calendar.timegm(dt.utctimetuple())

I realize some of those are inherently nonsensical, but I was trying to be complete :).

And here's the output. The diff= part is to help show the conversions that aren't correct.

now epoch                       : 1342671099
est   aware mktime(timetuple)   : 1342681899 diff=-10800
local aware mktime(timetuple)   : 1342671099 diff=0
local naive mktime(timetuple)   : 1342671099 diff=0
utc   aware mktime(timetuple)   : 1342699899 diff=-28800
utc   naive mktime(timetuple)   : 1342696299 diff=-25200
est   aware mktime(utctimetuple): 1342699899 diff=-28800
local aware mktime(utctimetuple): 1342699899 diff=-28800
local naive mktime(utctimetuple): 1342674699 diff=-3600
utc   aware mktime(utctimetuple): 1342699899 diff=-28800
utc   naive mktime(utctimetuple): 1342699899 diff=-28800
est   aware timegm(timetuple)   : 1342656699 diff=14400
local aware timegm(timetuple)   : 1342645899 diff=25200
local naive timegm(timetuple)   : 1342645899 diff=25200
utc   aware timegm(timetuple)   : 1342671099 diff=0
utc   naive timegm(timetuple)   : 1342671099 diff=0
est   aware timegm(utctimetuple): 1342671099 diff=0
local aware timegm(utctimetuple): 1342671099 diff=0
local naive timegm(utctimetuple): 1342645899 diff=25200
utc   aware timegm(utctimetuple): 1342671099 diff=0
utc   naive timegm(utctimetuple): 1342671099 diff=0

From what I can tell from this output, in general, calling calendar.timegm(dt.utctimetuple()) is usually the correct way -- unless its a naive local time. Ok, I can deal with that...except I don't see how you can disambiguate a local naive time from any other naive time, which makes it impossible to correctly convert an arbitrary datetime object into a timestamp? Is that really the case, or am I missing something?

Answer 1

I don't see how you can disambiguate a local naive time from any other naive time

Exactly. As you already know, naive means "no time zone information attached," so timegm is forced to infer the missing information, and your results show that it infers naive datetimes to be UTC.

As was said in the comments, times with TZ attached should be treated like a hot potato - you should convert to UTC as soon as you get input, and convert to local just before you output to the user. This will save you a lot of heartache in the long run.