[英]How to merge or join a stacked dataframe in pandas?
I cannot find this question answered elsewhere;我在其他地方找不到这个问题的答案; I would like to do a SQL-like join in pandas but with the slight twist that one dataframe is stacked.
我想在 pandas 中做一个类似 SQL 的连接,但稍微有点扭曲的是,一个 dataframe 是堆叠的。 I have created a dataframe A with a stacked column index from a csv file in pandas that looks as follows:
我从 pandas 中的 csv 文件创建了一个带有堆叠列索引的 dataframe A,如下所示:
| | | 2013-01-04 | 2013-01-07 |
|----------:|-----:|-----------:|-----------:|
| Adj Close | OWW | NaN | NaN |
| Close | OXLC | 4.155157 | 4.147217 |
| | OXM | 40.318089 | 42.988800 |
| | OXY | 50.416079 | 62.934800 |
The original csv had repeated what is in the 1st column for every entry like so:原始 csv 重复了每个条目的第一列中的内容,如下所示:
| | | 2013-01-04 | 2013-01-07 |
|----------:|-----:|-----------:|-----------:|
| Adj Close | OWW | NaN | NaN |
| Close | OXLC | 4.155157 | 4.147217 |
| Close | OXM | 40.318089 | 42.988800 |
| Close | OXY | 50.416079 | 62.934800 |
The original csv was the transposed version of this.原来的 csv 是这个的转置版本。 Pandas chose to stack that when converting to dataframe.
Pandas 在转换为 dataframe 时选择了堆叠。 (I used this code: pd.read_csv(file, header = [0,1], index_col=0).T)
(我使用了这个代码:pd.read_csv(file, header = [0,1], index_col=0).T)
In another csv/dataframe BI have for all of those so-called ticker symbols another ID that I would rather like to use: CIK.在另一个 csv/dataframe 中,对于所有那些所谓的股票代码,BI 有另一个我更愿意使用的 ID:CIK。
| CIK | Ticker | Name |
|---------|--------|------------------------------------------------|
| 1090872 | A | Agilent Technologies Inc |
| 4281 | AA | Alcoa Inc |
| 1332552 | AAACU | Asia Automotive Acquisition Corp |
| 1287145 | AABB | Asia Broadband Inc |
| 1024015 | AABC | Access Anytime Bancorp Inc |
| 1099290 | AAC | Sinocoking Coal & Coke Chemical Industries Inc |
| 1264707 | AACC | Asset Acceptance Capital Corp |
| 849116 | AACE | Ace Cash Express Inc |
| 1409430 | AAGC | All American Gold Corp |
| 948846 | AAI | Airtran Holdings Inc |
Desired output: I would like to have the CIK instead of the ticker in a new dataframe otherwise identical to A.所需的 output:我想在新的 dataframe 中使用 CIK 而不是代码,否则与 A 相同。
Now in SQL I could easily join on A.name_of_2nd_column = b.Ticker since the table would just have the entry in the 1st column repeated in every line (like the original csv) and the column would have a name but in pandas I cannot.现在在 SQL 中,我可以轻松地加入 A.name_of_2nd_column = b.Ticker,因为该表只会在每一行中重复第一列中的条目(如原始 csv),并且该列将有一个名称,但在 pandas 中我不能。 I tried this code:
我试过这段代码:
result = pd.merge(data, tix, how='left', left_on=[1] right_on=['Ticker'])
How do I tell pandas to use the 2nd column as the key and/or interpret the first column just as repeated values?如何告诉 pandas 使用第二列作为键和/或将第一列解释为重复值?
What you want is to transcode from one set of identifiers (tickers) to another (CIKs used in the SEC Edgar database I presume).您想要的是将一组标识符(代码)转码为另一组(我认为是 SEC Edgar 数据库中使用的 CIK)。
I would我会
A.index.names=('Data','Ticker')
A = A.reset_index()
transco = B.set_index('Ticker').CIK
A['CIK'] = A.Ticker.map(transco)
A = A.drop('Ticker', axis=1).set_index(['Data','CIK'])
As a step 2.5, you might want to remove the entries for which you don't have any CIK, eg by doing:作为步骤 2.5,您可能希望删除没有任何 CIK 的条目,例如通过执行以下操作:
A = A[A.CIK.notnull()]
A.CIK = A.CIK.astype(int)
You could also merge after doing a reset_index()
, but I would avoid that, as you might end up with uselessly large dataframes, as the result of the merge will have a names column.您也可以在执行
reset_index()
之后合并,但我会避免这种情况,因为您最终可能会得到无用的大数据帧,因为合并的结果将有一个 names 列。 This can grow if you have many different types of data (Adj Close, Close, etc.).如果您有许多不同类型的数据(Adj Close、Close 等),这可能会增加。
I was eventually able to do it the following way:我最终能够通过以下方式做到这一点:
df = A
tix = B
ticker_2_CIK = dict(zip(tix.Ticker,tix.CIK)) # create a dict
tmp = df.reset_index().assign(CIK=lambda x: x['ticker'].map(ticker_2_CIK)) # use dict to find the correct value for colum
# data was unclean, some ticker symbols were created after the period my data is from
# and data was incomplete with some tickers missing
solution = tmp.dropna(subset=['CIK']).astype({'CIK':int})
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.