从 numpy datetime64 获取年、月或日

Question

I have an array of datetime64 type:

dates = np.datetime64(['2010-10-17', '2011-05-13', "2012-01-15"])

Is there a better way than looping through each element just to get np.array of years:

years = f(dates)
#output:
array([2010, 2011, 2012], dtype=int8) #or dtype = string

I'm using stable numpy version 1.6.2.

Answer 1

I find the following tricks give between 2x and 4x speed increase versus the pandas method described in this answer (ie pd.DatetimeIndex(dates).year etc.). The speed of [dt.year for dt in dates.astype(object)] I find to be similar to the pandas method. Also these tricks can be applied directly to ndarrays of any shape (2D, 3D etc.)

dates = np.arange(np.datetime64('2000-01-01'), np.datetime64('2010-01-01'))
years = dates.astype('datetime64[Y]').astype(int) + 1970
months = dates.astype('datetime64[M]').astype(int) % 12 + 1
days = dates - dates.astype('datetime64[M]') + 1

Answer 2

As datetime is not stable in numpy I would use pandas for this:

In [52]: import pandas as pd

In [53]: dates = pd.DatetimeIndex(['2010-10-17', '2011-05-13', "2012-01-15"])

In [54]: dates.year
Out[54]: array([2010, 2011, 2012], dtype=int32)

Pandas uses numpy datetime internally, but seems to avoid the shortages, that numpy has up to now.

Answer 3

There should be an easier way to do this, but, depending on what you're trying to do, the best route might be to convert to a regular Python datetime object :

datetime64Obj = np.datetime64('2002-07-04T02:55:41-0700')
print datetime64Obj.astype(object).year
# 2002
print datetime64Obj.astype(object).day
# 4

Based on comments below, this seems to only work in Python 2.7.x and Python 3.6+

Answer 4

This is how I do it.

import numpy as np

def dt2cal(dt):
    """
    Convert array of datetime64 to a calendar array of year, month, day, hour,
    minute, seconds, microsecond with these quantites indexed on the last axis.

    Parameters
    ----------
    dt : datetime64 array (...)
        numpy.ndarray of datetimes of arbitrary shape

    Returns
    -------
    cal : uint32 array (..., 7)
        calendar array with last axis representing year, month, day, hour,
        minute, second, microsecond
    """

    # allocate output 
    out = np.empty(dt.shape + (7,), dtype="u4")
    # decompose calendar floors
    Y, M, D, h, m, s = [dt.astype(f"M8[{x}]") for x in "YMDhms"]
    out[..., 0] = Y + 1970 # Gregorian Year
    out[..., 1] = (M - Y) + 1 # month
    out[..., 2] = (D - M) + 1 # dat
    out[..., 3] = (dt - D).astype("m8[h]") # hour
    out[..., 4] = (dt - h).astype("m8[m]") # minute
    out[..., 5] = (dt - m).astype("m8[s]") # second
    out[..., 6] = (dt - s).astype("m8[us]") # microsecond
    return out

It's vectorized across arbitrary input dimensions, it's fast, its intuitive, it works on numpy v1.15.4, it doesn't use pandas.

I really wish numpy supported this functionality, it's required all the time in application development. I always get super nervous when I have to roll my own stuff like this, I always feel like I'm missing an edge case.

Answer 5

Using numpy version 1.10.4 and pandas version 0.17.1,

dates = np.array(['2010-10-17', '2011-05-13', '2012-01-15'], dtype=np.datetime64)
pd.to_datetime(dates).year

I get what you're looking for:

array([2010, 2011, 2012], dtype=int32)

Answer 6

Use dates.tolist() to convert to native datetime objects, then simply access year . Example:

>>> dates = np.array(['2010-10-17', '2011-05-13', '2012-01-15'], dtype='datetime64')
>>> [x.year for x in dates.tolist()]
[2010, 2011, 2012]

This is basically the same idea exposed in https://stackoverflow.com/a/35281829/2192272 , but using simpler syntax.

Tested with python 3.6 / numpy 1.18.

Answer 7

Another possibility is:

np.datetime64(dates,'Y') - returns - numpy.datetime64('2010')

or

np.datetime64(dates,'Y').astype(int)+1970 - returns - 2010

but works only on scalar values, won't take array

Answer 8

如果您升级到 numpy 1.7（其中 datetime 仍被标记为实验性），则以下内容应该可以工作。

dates/np.timedelta64(1,'Y')

Answer 9

Anon's answer works great for me, but I just need to modify the statement for days

from:

days =  - dates.astype('datetime64[M]') + 1

to:

days =  - dates.astype('datetime64[M]') + 1

Answer 10

There's no direct way to do it yet, unfortunately, but there are a couple indirect ways:

[dt.year for dt in dates.astype(object)]

or

[datetime.datetime.strptime(repr(d), "%Y-%m-%d %H:%M:%S").year for d in dates]

both inspired by the examples here .

Both of these work for me on Numpy 1.6.1. You may need to be a bit more careful with the second one, since the repr() for the datetime64 might have a fraction part after a decimal point.

Answer 11

convert np.datetime64 to float-year

In this solution, you can see, step-by-step, how to process np.datetime64 datatypes.

In the following dt64 is of type np.datetime64 (or even a numpy.ndarray of that type):

• year = dt64.astype('M8[Y]') contains just the year. If you want a float array of that, do 1970 + year.astype(float) .
• the days (since January 1st) you can access by days = (dt64 - year).astype('timedelta64[D]')
• You can also deduce if a year is a leap year or not (compare days_of_year )

See also the numpy tutorial

import numpy as np
import pandas as pd

def dt64_to_float(dt64):
    """Converts numpy.datetime64 to year as float.

    Rounded to days

    Parameters
    ----------
    dt64 : np.datetime64 or np.ndarray(dtype='datetime64[X]')
        date data

    Returns
    -------
    float or np.ndarray(dtype=float)
        Year in floating point representation
    """

    year = dt64.astype('M8[Y]')
    # print('year:', year)
    days = (dt64 - year).astype('timedelta64[D]')
    # print('days:', days)
    year_next = year + np.timedelta64(1, 'Y')
    # print('year_next:', year_next)
    days_of_year = (year_next.astype('M8[D]') - year.astype('M8[D]')
                    ).astype('timedelta64[D]')
    # print('days_of_year:', days_of_year)
    dt_float = 1970 + year.astype(float) + days / (days_of_year)
    # print('dt_float:', dt_float)
    return dt_float

if __name__ == "__main__":

    dt_str = '2011-11-11'
    dt64 = np.datetime64(dt_str)
    print(dt_str, 'as float:', dt64_to_float(dt64))
    print()

    dates = np.array([
        '1970-01-01', '2014-01-01', '2020-12-31', '2019-12-31', '2010-04-28'],
        dtype='datetime64[D]')
    float_dates = dt64_to_float(dates)


    print('dates:      ', dates)
    print('float_dates:', float_dates)

output

2011-11-11 as float: 2011.8602739726027

dates:       ['1970-01-01' '2014-01-01' '2020-12-31' '2019-12-31' '2010-04-28']
float_dates: [1970.         2014.         2020.99726776 2019.99726027 2010.32054795]

Answer 12

This is obviously quite late, but I benefitted from one of the answers, so sharing my bit here.

The answer by Anon 🤔 is quite right- the speed is incredibly higher using numpy method instead of first casting them as pandas datetime series and then getting dates. Albeit the offsetting and conversion of results after numpy transformations are bit shabby, a cleaner helper for this can be written, like so:-

def from_numpy_datetime_extract(date: np.datetime64, extract_attribute: str = None):
    _YEAR_OFFSET = 1970
    _MONTH_OFFSET = 1
    _MONTH_FACTOR = 12
    _DAY_FACTOR = 24*60*60*1e9
    _DAY_OFFSET = 1

    if extract_attribute == 'year':
        return date.astype('datetime64[Y]').astype(int) + _YEAR_OFFSET
    elif extract_attribute == 'month':
        return date.astype('datetime64[M]').astype(int)%_MONTH_FACTOR + _MONTH_OFFSET
    elif extract_attribute == 'day':
        return ((date - date.astype('datetime64[M]'))/_DAY_FACTOR).astype(int) + _DAY_OFFSET
    else:
        raise ValueError("extract_attribute should be either of 'year', 'month' or 'day'")

Solving the ask dates = np.array(['2010-10-17', '2011-05-13', "2012-01-15"], dtype = 'datetime64') :-

• Numpy method (using the helper above)

%timeit -r10 -n1000 [from_numpy_datetime_extract(x, "year") for x in dates]
# 14.3 µs ± 4.03 µs per loop (mean ± std. dev. of 10 runs, 1000 loops each)

• Pandas method

%timeit -r10 -n1000 pd.to_datetime(dates).year.tolist()
# 304 µs ± 32.2 µs per loop (mean ± std. dev. of 10 runs, 1000 loops each)

Answer 13

How about simply converting to string?

Probably the easiest way:

import numpy as np

date = np.datetime64("2000-01-01")
date_strings = date.astype(str).split('-'). 
# >> ['2000', '01', '01']

year_int = int(date_strings[0])