Python Pandas-read_csv数据框正在从列中删除值

Question

首先，我知道下面的示例没有缩进，如果有人可以指出正确的方向正确发布DataFrame，我将不胜感激。

现在，我有一个数据框，如下所示：

 Ticker_x Date Close_x Ticker_y Close_y Lead_Lag Ticker 15M_Trend Bollinger_1SD Bollinger_2SD Bollinger_and_Trend_1SD Bollinger_and_Trend_2SD ES M7 6/2/2017 3:29 2433.75 VX M7-CF 11.6 N ES M7 6/2/2017 4:29 2434.75 VX M7-CF 11.6 Short ES M7 6/2/2017 5:29 2435 VX M7-CF 11.55 N ES M7 6/2/2017 6:29 2434.25 VX M7-CF 11.57 N ES M7 6/2/2017 7:29 2430.25 VX M7-CF 11.7 Short ES M7 6/2/2017 8:29 2430.75 VX M7-CF 11.58 Short ES M7 6/2/2017 9:29 2433.25 VX M7-CF 11.63 Short ES M7 6/2/2017 10:29 2436.75 VX M7-CF 11.61 Short ES M7 6/2/2017 11:29 2436.75 VX M7-CF 11.57 Short ES M7 6/2/2017 12:29 2435.75 VX M7-CF 11.67 Short ES M7 6/2/2017 13:29 2437.75 VX M7-CF 11.64 Short ES M7 6/2/2017 14:29 2437.75 VX M7-CF 11.63 Short 

我正在从另一个文件的read_csv函数创建此数据框，并且仅包括以下概述的列。 请注意，我正在从具有“ glob”的文件夹中获取最新文件：

filenameA = max(glob.iglob(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Pairs Trading\VX_ES\CSV\15M\Lead_Lag\*.csv"))

AggregatedA = pd.read_csv(filenameA, usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'])
p2 = r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Individual Trading\VX\CSV\Aggregated\VX_ES_15M\blah2.csv"
AggregatedA.to_csv(path_or_buf = p2)

因此，新文件“ blah2”就是上面显示的csv文件。 您会注意到，最右边的7或8列是空白的。 这些内容不能为空。 在我从“ filenameA”中选取的原始文件中，它们不是空白。

也就是说，它们具有“ Ticker”，“ 15M_Trend”和其他列的一堆值。 出于某种原因，熊猫没有接受这些价值，我不确定为什么。 非常感谢帮助！

编辑：这是数据框的原始行：

 Ticker_x Date Open_x High_x Low_x Close_x Volume_x Open Interest_x Ticker_y Open_y High_y Low_y Close_y Volume_y Open Interest_y ES_returns VX_returns Beta Pairs_Spread zscore Pairs_Spread_Mean Pairs_Spread_sdev ES_percent_change ES_difference VX_percent_change ES_CC VX_CC pairs_spread pairs_zscore ES_Inverse_price ES__Inverse_percent_change Inverse_ES_CC Inverse_pairs_spread Inverse_pairs_zscore Lead_Lag Ticker Open High Low Close Volume Open Interest Index_Num Rolling_OLS_Coefficient 15M_Long Upper_Sdev_Value Lower_Sdev_Value Intercept Middle Sdev Lower_Sdev_value 15M_Trend Rolling_mean Rolling_std Upper_Band Lower_Band Upper_Band_2 Lower_Band_2 Bollinger_1SD Bollinger_2SD Bollinger_and_Trend_1SD Bollinger_and_Trend_2SD Trend_and_LL ES M7 6/2/2017 3:29 2433.25 2433.75 2433 2433.75 3419 0 VX M7-CF 11.55 11.6 11.53 11.6 253 0 0 0.001727116 -6.946994692 0 0 0.001727116 -0.27013895 -0.27013895 2514.335138 2.733208694 2138.25 0 0.270990395 2218.835138 -1.144384967 N VX M7-CF 11.55 11.6 11.53 11.6 253 0 1868 -0.006269114 27.09349175 23.18140592 23.10868419 3.984807553 19.12387664 Short 11.7912 0.1573199 11.9485199 11.6338801 12.1058398 11.4765602 Long N N N N ES M7 6/2/2017 4:29 2435 2435.5 2434.75 2434.75 2847 0 VX M7-CF 11.59 11.6 11.55 11.6 118 0 0.000410889 0 -6.94464418 0.000410889 1 0 -0.434541317 -0.434541317 2515.307872 2.759879939 2137.25 -0.000467672 0.43443517 2217.807872 -1.197621083 Short VX M7-CF 11.59 11.6 11.55 11.6 118 0 1869 -0.006344733 27.26010073 23.31610124 23.24250234 4.017598384 19.22490396 Short 11.7868 0.159558575 11.94635857 11.62724143 12.10591715 11.46768285 Long N N N Short ES M7 6/2/2017 5:29 2436.25 2436.5 2435 2435 5979 0 VX M7-CF 11.55 11.6 11.5 11.55 716 0 0.00010268 -0.004310345 -6.945304375 0.00010268 0.25 -0.004310345 -0.364155097 -0.364155097 2515.218266 2.75742302 2137 -0.000116973 0.364203518 2217.218266 -1.228176348 N VX M7-CF 11.55 11.6 11.5 11.55 716 0 1870 -0.006422122 27.43061644 23.45376171 23.37958619 4.051030244 19.32855595 Short 11.7814 0.162944964 11.94434496 11.61845504 12.10728993 11.45551007 Long N N N N ES M7 6/2/2017 6:29 2434.5 2435 2433.75 2434.25 11821 0 VX M7-CF 11.5 11.65 11.5 11.57 1919 0 -0.000308008 0.001731602 -6.943255051 -0.000308008 -0.75 0.001731602 -0.391364705 -0.391364705 2514.583461 2.74001741 2137.75 0.000350959 0.391371805 2218.083461 -1.183339232 N VX M7-CF 11.5 11.65 11.5 11.57 1919 0 1871 -0.006492588 27.58935246 23.57945638 23.50433714 4.08501532 19.41932182 Short 11.7764 0.165551151 11.94195115 11.61084885 12.1075023 11.4452977 Long N N N N ES M7 6/2/2017 7:29 2431.25 2431.75 2430 2430.25 16785 0 VX M7-CF 11.6 11.75 11.58 11.7 3737 0 -0.001643217 0.011235955 -6.941089619 -0.001643217 -4 0.011235955 -0.795190709 -0.795190709 2511.460749 2.654396241 2141.75 0.001871126 0.79514286 2222.960749 -0.930583022 Short VX M7-CF 11.6 11.75 11.58 11.7 3737 0 1872 -0.006552806 27.73075716 23.68800858 23.61134075 4.119416411 19.49192434 Short 11.776 0.165714286 11.94171429 11.61028571 12.10742857 11.44457143 N N N N Short ES M7 6/2/2017 8:29 2430.25 2431 2429 2430.75 37511 0 VX M7-CF 11.7 11.8 11.53 11.58 15635 0 0.00020574 -0.01025641 -6.944443722 0.00020574 0.5 -0.01025641 -0.801967629 -0.801967629 2511.166658 2.646332626 2141.25 -0.000233454 0.801808413 2221.666658 -0.997646802 Short VX M7-CF 11.7 11.8 11.53 11.58 15635 0 1873 -0.006629471 27.90272657 23.82506069 23.74829141 4.154435156 19.59385626 Short 11.773 0.167907287 11.94090729 11.60509271 12.10881457 11.43718543 Long N N N Short ES M7 6/2/2017 9:29 2432.25 2433.75 2432.25 2433.25 26047 0 VX M7-CF 11.55 11.7 11.55 11.63 7148 0 0.001028489 0.004317789 -6.902663206 0.001028489 2.5 0.004317789 -0.50559849 -0.50559849 2513.527973 2.711077152 2138.75 -0.001167542 0.506119457 2219.027973 -1.134391675 Short VX M7-CF 11.55 11.7 11.55 11.63 7148 0 1874 -0.006703266 28.06957526 23.95751485 23.87955587 4.190019391 19.68953648 Short 11.7706 0.169096157 11.93969616 11.60150384 12.10879231 11.43240769 N N N N Short ES M7 6/2/2017 10:29 2435.75 2437 2435.5 2436.75 31594 0 VX M7-CF 11.65 11.65 11.55 11.61 5090 0 0.001438405 -0.00171969 -6.862573345 0.001438405 3.5 -0.00171969 -0.494655304 -0.494655304 2516.424477 2.790495933 2135.25 -0.00163647 0.495088995 2214.924477 -1.347047627 Short VX M7-CF 11.65 11.65 11.55 11.61 5090 0 1875 -0.006773515 28.23118528 24.08370058 24.00506007 4.226125209 19.77893486 Short 11.7608 0.163729992 11.92452999 11.59707001 12.08825998 11.43334002 N N N N Short ES M7 6/2/2017 11:29 2436 2437 2435.5 2436.75 25538 0 VX M7-CF 11.65 11.65 11.55 11.57 2278 0 0 -0.003445306 -6.86522263 0 0 -0.003445306 -0.46654779 -0.46654779 2516.180626 2.783809828 2135.25 0 0.466999626 2214.680626 -1.359684729 Short VX M7-CF 11.65 11.65 11.55 11.57 2278 0 1876 -0.00683482 28.37745101 24.19388406 24.11480519 4.262645818 19.85215938 Short 11.7512 0.160466666 11.91166667 11.59073333 12.07213333 11.43026667 Long N N N Short ES M7 6/2/2017 12:29 2437.75 2438.75 2435.5 2435.75 37037 0 VX M7-CF 11.58 11.69 11.55 11.67 4309 0 -0.000410383 0.008643042 -6.890566028 -0.000410383 -1 0.008643042 -0.503091577 -0.503091577 2516.162906 2.783323958 2136.25 0.000468329 0.503595438 2216.662906 -1.25695683 Short VX M7-CF 11.58 11.69 11.55 11.67 4309 0 1877 -0.006883805 28.50205329 24.28297379 24.20263978 4.299413516 19.90322626 Short 11.7436 0.154929345 11.89852935 11.58867065 12.05345869 11.43374131 N N N N Short ES M7 6/2/2017 13:29 2437.75 2438.5 2437 2437.75 26228 0 VX M7-CF 11.68 11.7 11.55 11.64 8570 0 0.000821102 -0.002570694 -6.88244494 0.000821102 2 -0.002570694 -0.520801285 -0.520801285 2517.861659 2.82990182 2134.25 -0.00093622 0.521308056 2214.361659 -1.376214577 Short VX M7-CF 11.68 11.7 11.55 11.64 8570 0 1878 -0.006929786 28.62236572 24.3666349 24.2859722 4.336393523 19.94957867 Short 11.734 0.145728291 11.87972829 11.58827171 12.02545658 11.44254342 N N N N Short ES M7 6/2/2017 14:29 2437.75 2437.75 2437.75 2437.75 1 0 VX M7-CF 11.6 11.7 11.55 11.63 28177 0 0 -0.000859107 -6.904693941 0 0 -0.000859107 -0.516701901 -0.516701901 2518.051591 2.835109521 2134.25 0 0.517232222 2214.551591 -1.36637174 Short VX M7-CF 11.6 11.7 11.55 11.63 28177 0 1879 -0.006969382 28.73142917 24.4389793 24.35792539 4.37350378 19.98442161 Short 11.724 0.13474087 11.85874087 11.58925913 11.99348174 11.45451826 N N N N Short 

编辑：这是基于以下建议的新代码：

AggregatedA = pd.read_csv(max(glob.iglob(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Pairs Trading\VX_ES\CSV\15M\Lead_Lag\*.csv")), usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'], delimiter='\t')

AggregatedA.to_csv(r"C:\Users\cost9\OneDrive\Documents\PYTHON\Daily Tasks\Individual Trading\VX\CSV\Aggregated\VX_ES_15M\blah2.csv")

给出错误：

ValueError: Usecols do not match names.

编辑：我也尝试使用delimiter ='，'而不是delimiter ='\\ t'，它消除了错误。 但是，右列保留为空白。

Answer 1

假设输入是给定的，那么pandas解析数据帧是令人惊讶的，因为应该明确地给出定界符。 这样做，代码可以正常工作。 如果in.txt包含您提供的行，则运行

In [26]: AggregatedA = pd.read_csv('in.txt', usecols=['Ticker_x', 'Date', 'Close_x', 'Ticker_y', 'Close_y', 'Lead_Lag', 'Ticker', '15M_Trend', 'Bollinger_1SD', 'Bollinger_2SD', 'Bollinger_and_Trend_1SD', 'Bollinger_and_Trend_2SD'], delimiter='\t')

In [25]: AggregatedA.to_csv('out.txt')

将给出具有以下内容的out.txt ：

$ cat out.txt
,Ticker_x,Date,Close_x,Ticker_y,Close_y,Lead_Lag,Ticker,15M_Trend,Bollinger_1SD,Bollinger_2SD,Bollinger_and_Trend_1SD,Bollinger_and_Trend_2SD
0,ES M7,6/2/2017 3:29,2433.75,VX M7-CF,11.6,N,VX M7-CF,Short,Long,N,N,N
1,ES M7,6/2/2017 4:29,2434.75,VX M7-CF,11.6,Short,VX M7-CF,Short,Long,N,N,N
2,ES M7,6/2/2017 5:29,2435.0,VX M7-CF,11.55,N,VX M7-CF,Short,Long,N,N,N
3,ES M7,6/2/2017 6:29,2434.25,VX M7-CF,11.57,N,VX M7-CF,Short,Long,N,N,N
4,ES M7,6/2/2017 7:29,2430.25,VX M7-CF,11.7,Short,VX M7-CF,Short,N,N,N,N
5,ES M7,6/2/2017 8:29,2430.75,VX M7-CF,11.58,Short,VX M7-CF,Short,Long,N,N,N
6,ES M7,6/2/2017 9:29,2433.25,VX M7-CF,11.63,Short,VX M7-CF,Short,N,N,N,N
7,ES M7,6/2/2017 10:29,2436.75,VX M7-CF,11.61,Short,VX M7-CF,Short,N,N,N,N
8,ES M7,6/2/2017 11:29,2436.75,VX M7-CF,11.57,Short,VX M7-CF,Short,Long,N,N,N
9,ES M7,6/2/2017 12:29,2435.75,VX M7-CF,11.67,Short,VX M7-CF,Short,N,N,N,N
10,ES M7,6/2/2017 13:29,2437.75,VX M7-CF,11.64,Short,VX M7-CF,Short,N,N,N,N
11,ES M7,6/2/2017 14:29,2437.75,VX M7-CF,11.63,Short,VX M7-CF,Short,N,N,N,N