Pandas: Find first True value in a list within a cell
Question
tl;dr- I have a dataframe where one column is a list of boolean values. I need to create an another column which displays the index of the list where the first True value is found.
Long Explanation:
I have a Pandas dataframe with timelines of economic data (each column is a quarter).
I'm attempting to derive some particular metrics, and the one I'm stuck on is "recovery". I'm trying to determine how many quarters go by before the timeline surpasses its previous peak.
Below is some sample data with all of the work I've done so far. Sorry for its size.
Important columns for this particular problem:
nadir: Lowest point in the timeline nadir_qtr: quarter at which nadir happens
pre_peak: highest point before the nadir pre_peak_qtr: quarter at which the pre-peak happens
To find the recovery quarter, I made a column with a list of boolean values- whether the number is greater than the pre-peak:
df2['recovery_list'] = df2.apply(lambda x: (x['new'][x['nadir_qtr']:] > x['pre_peak']), axis=1)
I'm able to get the number of quarters before you get a True value with the below code:
next((i for i, j in enumerate(list(df2['recovery_list'][0])) if j), None)
Output: 7
But when I use the code to generate a new column:
df2['recovery_qtr'] = next((i for i, j in enumerate(list(df2['recovery_list'])) if j), None)
I get the following error:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Any idea what I'm doing wrong here?
I'm open to alternate approaches to solving this problem, but this is as close as I've come.
| qtrid | index | area_fips | area_title | 2000.25 | 2000.5 | 2000.75 | 2001.0 | 2001.25 | 2001.5 | 2001.75 | 2002.0 | 2002.25 | 2002.5 | 2002.75 | 2003.0 | 2003.25 | 2003.5 | 2003.75 | 2004.0 | 2004.25 | 2004.5 | 2004.75 | 2005.0 | 2005.25 | 2005.5 | 2005.75 | 2006.0 | 2006.25 | 2006.5 | 2006.75 | 2007.0 | 2007.25 | 2007.5 | 2007.75 | 2008.0 | nadir | nadir_qtr | pre_peak | pre_peak_qtr | recovery_list | post_peak | post_peak_qtr | new | recovery | decline | delta |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1374 | 27075 | Lake County, Minnesota | 7518.0 | 8346.0 | 8642.0 | 7714.0 | 8154.0 | 9012.0 | 8818.0 | 8174.0 | 7642.0 | 8288.0 | 8840.0 | 8022.0 | 7854.0 | 8876.0 | 8642.0 | 7926.0 | 7838.0 | 9020.0 | 8644.0 | 8130.0 | 8060.0 | 8968.0 | 8694.0 | 8314.0 | 8182.0 | 9344.0 | 8810.0 | 8202.0 | 8214.0 | 9352.0 | 8898.0 | 8346.0 | 7642.0 | 10 | 9012.0 | 5 | [False, False, False, False, False, False, Fal... | 9352.0 | 29 | [7518.0, 8346.0, 8642.0, 7714.0, 8154.0, 9012.... | 1 | 5 | 340.0 |
| 1 | 3073 | 54063 | Monroe County, West Virginia | 3812.0 | 4126.0 | 4072.0 | 4040.0 | 3894.0 | 4062.0 | 4086.0 | 4060.0 | 3950.0 | 4122.0 | 4014.0 | 3906.0 | 3772.0 | 3912.0 | 4032.0 | 3996.0 | 4112.0 | 4224.0 | 4242.0 | 4232.0 | 3962.0 | 4138.0 | 4090.0 | 4086.0 | 3978.0 | 4096.0 | 4070.0 | 4196.0 | 4138.0 | 4134.0 | 4210.0 | 4158.0 | 3772.0 | 14 | 4126.0 | 1 | [False, False, False, True, True, True, False,... | 4242.0 | 18 | [3812.0, 4126.0, 4072.0, 4040.0, 3894.0, 4062.... | 1 | 13 | 116.0 |
| 2 | 1919 | 36119 | Westchester County, New York | 794524.0 | 820670.0 | 813970.0 | 837170.0 | 807890.0 | 829618.0 | 805440.0 | 823416.0 | 799696.0 | 822272.0 | 807492.0 | 829566.0 | 800922.0 | 822528.0 | 805160.0 | 826352.0 | 808770.0 | 832592.0 | 819536.0 | 842588.0 | 813082.0 | 838078.0 | 823780.0 | 841502.0 | 815112.0 | 842546.0 | 827984.0 | 851376.0 | 830014.0 | 862302.0 | 840974.0 | 865742.0 | 799696.0 | 10 | 837170.0 | 3 | [False, False, False, False, False, False, Fal... | 865742.0 | 31 | [794524.0, 820670.0, 813970.0, 837170.0, 80789... | 1 | 7 | 28572.0 |
| 3 | 3103 | 55011 | Buffalo County, Wisconsin | 8747.0 | 9320.0 | 9322.0 | 9298.0 | 9824.0 | 9445.0 | 9488.0 | 9910.0 | 9768.0 | 9142.0 | 10028.0 | 9060.0 | 9944.0 | 9583.0 | 10390.0 | 9256.0 | 9702.0 | 9462.0 | 9957.0 | 10740.0 | 10502.0 | 11230.0 | 11106.0 | 11158.0 | 11662.0 | 12302.0 | 12298.0 | 11267.0 | 10292.0 | 10938.0 | 10726.0 | 10450.0 | 9060.0 | 13 | 10028.0 | 10 | [False, True, False, False, False, False, True... | 12302.0 | 25 | [8747.0, 9320.0, 9322.0, 9298.0, 9824.0, 9445.... | 1 | 3 | 2274.0 |
| 4 | 3683 | C2706 | Ithaca, NY MSA | 92042.0 | 95082.0 | 93852.0 | 94730.0 | 92978.0 | 96624.0 | 94556.0 | 95156.0 | 92922.0 | 95148.0 | 95666.0 | 96214.0 | 95192.0 | 97904.0 | 97330.0 | 97772.0 | 96510.0 | 98820.0 | 98532.0 | 99160.0 | 100396.0 | 100062.0 | 99060.0 | 99840.0 | 97912.0 | 100914.0 | 99772.0 | 101050.0 | 98418.0 | 101806.0 | 101344.0 | 102708.0 | 92922.0 | 10 | 96624.0 | 5 | [False, False, False, True, True, True, False,... | 102708.0 | 31 | [92042.0, 95082.0, 93852.0, 94730.0, 92978.0, ... | 1 | 5 | 6084.0 |
| 5 | 2365 | 45019 | Charleston County, South Carolina | 362554.0 | 378358.0 | 362860.0 | 369826.0 | 362656.0 | 364742.0 | 360836.0 | 359780.0 | 358752.0 | 370100.0 | 368878.0 | 369464.0 | 369964.0 | 377104.0 | 376108.0 | 378814.0 | 379144.0 | 389946.0 | 389044.0 | 391862.0 | 388852.0 | 402586.0 | 398644.0 | 398042.0 | 396874.0 | 406484.0 | 410372.0 | 416576.0 | 417812.0 | 427552.0 | 424776.0 | 424316.0 | 358752.0 | 10 | 378358.0 | 1 | [False, False, False, False, False, True, True... | 427552.0 | 29 | [362554.0, 378358.0, 362860.0, 369826.0, 36265... | 1 | 9 | 49194.0 |
| 6 | 3448 | C1722 | Clarksburg, WV MicroSA | 35242.0 | 36634.0 | 36225.0 | 36026.0 | 35447.0 | 36476.0 | 35968.0 | 36227.0 | 35571.0 | 36785.0 | 36377.0 | 36863.0 | 35954.0 | 37505.0 | 37048.0 | 37325.0 | 36400.0 | 37701.0 | 37407.0 | 37503.0 | 36407.0 | 37653.0 | 37616.0 | 37383.0 | 36977.0 | 37625.0 | 37037.0 | 37227.0 | 36265.0 | 37617.0 | 37227.0 | 37570.0 | 35571.0 | 10 | 36785.0 | 9 | [False, True, False, True, True, True, False, ... | 37701.0 | 17 | [35242.0, 36634.0, 36225.0, 36026.0, 35447.0, ... | 1 | 1 | 916.0 |
| 7 | 3297 | C1086 | Alice, TX MicroSA | 13001.0 | 13245.0 | 13460.0 | 13800.0 | 13642.0 | 14243.0 | 14145.0 | 14031.0 | 13873.0 | 14086.0 | 14208.0 | 14106.0 | 14522.0 | 14855.0 | 14959.0 | 15120.0 | 15105.0 | 15492.0 | 15554.0 | 15695.0 | 15517.0 | 15713.0 | 15970.0 | 15874.0 | 16127.0 | 16666.0 | 16778.0 | 16876.0 | 17380.0 | 18144.0 | 18178.0 | 17937.0 | 13873.0 | 10 | 14243.0 | 5 | [False, False, True, True, True, True, True, T... | 18178.0 | 30 | [13001.0, 13245.0, 13460.0, 13800.0, 13642.0, ... | 1 | 5 | 3935.0 |
| 8 | 742 | 18063 | Hendricks County, Indiana | 58656.0 | 60586.0 | 60618.0 | 60272.0 | 59964.0 | 63238.0 | 65434.0 | 65000.0 | 65534.0 | 66824.0 | 68710.0 | 68680.0 | 67982.0 | 70936.0 | 72554.0 | 74030.0 | 74836.0 | 77138.0 | 78370.0 | 78876.0 | 77654.0 | 79878.0 | 81936.0 | 82752.0 | 83174.0 | 86144.0 | 88166.0 | 88914.0 | 88606.0 | 91556.0 | 93514.0 | 93744.0 | 65000.0 | 9 | 65534.0 | 8 | [True, True, True, True, True, True, True, Tru... | 93744.0 | 31 | [58656.0, 60586.0, 60618.0, 60272.0, 59964.0, ... | 1 | 1 | 28210.0 |
| 9 | 3618 | C2446 | Great Bend, KS MicroSA | 12532.0 | 13213.0 | 13015.0 | 13207.0 | 12852.0 | 13368.0 | 13077.0 | 13216.0 | 13144.0 | 13396.0 | 13348.0 | 12961.0 | 12487.0 | 12791.0 | 12690.0 | 12638.0 | 12520.0 | 12953.0 | 12679.0 | 12804.0 | 12600.0 | 13152.0 | 12685.0 | 12886.0 | 12946.0 | 13264.0 | 12943.0 | 13339.0 | 13160.0 | 13433.0 | 13111.0 | 13546.0 | 12487.0 | 14 | 13396.0 | 9 | [False, False, False, False, False, False, Fal... | 13546.0 | 31 | [12532.0, 13213.0, 13015.0, 13207.0, 12852.0, ... | 1 | 5 | 150.0 |
1 answers
solution1
2 ACCPTED 2020-12-16 05:42:41
Consider below df :
In [952]: df = pd.DataFrame({'A':[[True, False, True], [False, True, False], [False, False, False, True]]})
In [953]: df
Out[953]:
A
0 [True, False, True]
1 [False, True, False]
2 [False, False, False, True]
You can do this using df.apply andList.index :
In [955]: df['True_index'] = df['A'].apply(lambda x: x.index(True))
In [956]: df
Out[956]:
A True_index
0 [True, False, True] 0
1 [False, True, False] 1
2 [False, False, False, True] 3
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.