[英]Filter numpy array of tuples
Scikit-learn
library have a brilliant example of data clustering - stock market structure . Scikit-learn
库是数据聚类- 股票市场结构的杰出典范。 It works fine within US stocks. 在美国股票中它运作良好。 But when one adds tickers from other markets,
numpy
's error appear that arrays shoud have the same size - this is true, for example, german stocks have different trading calendar. 但是,当人们添加来自其他市场的
numpy
, numpy
出现的错误似乎是数组应该具有相同的大小-的确如此,例如,德国股票的交易日历不同。
Ok, after quotes download I add preparation of shared dates: 好的,在报价下载后,我要准备共享日期:
quotes = [quotes_historical_yahoo_ochl(symbol, d1, d2, asobject=True)
for symbol in symbols]
def intersect(list_1, list_2):
return list(set(list_1) & set(list_2))
dates_all = quotes[0].date
for q in quotes:
dates_symbol = q.date
dates_all = intersect(dates_all, dates_symbol)
Then I'm stuck with filtering numpy array of tuples. 然后,我被困在过滤元组的numpy数组。 Here's some tries:
这里有一些尝试:
# for index, q in enumerate(quotes):
# filtered = [i for i in q if i.date in dates_all]
# quotes[index] = np.rec.array(filtered, dtype=q.dtype)
# quotes[index] = np.asanyarray(filtered, dtype=q.dtype)
#
# quotes[index] = np.where(a.date in dates_all for a in q)
#
# quotes[index] = np.where(q[0].date in dates_all)
How to apply filter to numpy array or how to truly convert list of records (after filter) back to numpy
's recarray
? 如何将过滤器应用于numpy数组或如何真正地将记录列表(在过滤器之后)转换回
numpy
的recarray
?
quotes[0].dtype: 引号[0] .dtype:
'(numpy.record, [('date', 'O'), ('year', '<i2'), ('month', 'i1'), ('day', 'i1'), ('d', '<f8'), ('open', '<f8'), ('close', '<f8'), ('high', '<f8'), ('low', '<f8'), ('volume', '<f8'), ('aclose', '<f8')])'
quotes[0].shape: 引号[0] .shape:
<class 'tuple'>: (261,)
So quotes
is a list of recarrays, and in date_all
you collect the intersection of all values in the date
field. 因此,
quotes
是一个recarray列表,在date_all
您将在date
字段中收集所有值的交集。
I can recreate one such array with: 我可以使用以下方法重新创建一个这样的数组:
In [286]: dt=np.dtype([('date', 'O'), ('year', '<i2'), ('month', 'i1'), ('day',
...:
...: ), ('low', '<f8'), ('volume', '<f8'), ('aclose', '<f8')])
In [287]:
In [287]: arr=np.ones((2,), dtype=dt) # 2 element structured array
In [288]: arr
Out[288]:
array([(1, 1, 1, 1, 1., 1., 1., 1., 1., 1., 1.),
(1, 1, 1, 1, 1., 1., 1., 1., 1., 1., 1.)],
dtype=[('date', 'O'), ('year', '<i2'), ('month', 'i1'), ('day', 'i1'), ... ('aclose', '<f8')])
In [289]: type(arr[0])
Out[289]: numpy.void
turn that into a recarray (I dont' use those as much as plain structured arrays): 把它变成一个Recarray(我不像普通结构化数组那样使用它们):
In [291]: np.rec.array(arr)
Out[291]:
rec.array([(1, 1, 1, 1, 1., 1., 1., 1., 1., 1., 1.),
(1, 1, 1, 1, 1., 1., 1., 1., 1., 1., 1.)],
dtype=[('date', 'O'), ('year', '<i2'), ('month', 'i1'), ('day', 'i1'), .... ('aclose', '<f8')])
dtype
of the recarray displays slightly different: dtype
的recarray显示稍微不同的:
In [292]: _.dtype
Out[292]: dtype((numpy.record, [('date', 'O'), ('year', '<i2'), ('month', 'i1'), ....('aclose', '<f8')]))
In [293]: __.date
Out[293]: array([1, 1], dtype=object)
In any case the date
field is an array of objects, possibly of datetime
? 无论如何,
date
字段都是对象的数组,可能是datetime
?
q
is one of these arrays; q
是这些数组之一; i
is an element, and i.date
is the date field. i
是元素,而i.date
是日期字段。
[i for i in q if i.date in dates_all]
So filtered
is list of recarray elements. filtered
是recarray元素列表。 np.stack
does a better job of reassembling them into an array (that works with the recarray too). np.stack
可以更好地将它们重新组装成一个数组(也可以与recarray一起使用)。
np.stack([i for i in arr if i['date'] in alist])
Or you could collect the indices of the matching records, and index the quote array 或者,您可以收集匹配记录的索引,然后对报价数组进行索引
In [319]: [i for i,v in enumerate(arr) if v['date'] in alist]
Out[319]: [0, 1]
In [320]: arr[_]
or pull out the date field first: 或先拉出日期字段:
In [321]: [i for i,v in enumerate(arr['date']) if v in alist]
Out[321]: [0, 1]
in1d
might also work to search in1d
可能也可以搜索
In [322]: np.in1d(arr['date'],alist)
Out[322]: array([ True, True], dtype=bool)
In [323]: np.where(np.in1d(arr['date'],alist))
Out[323]: (array([0, 1], dtype=int32),)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.