[英]Import of large CSV file using Pandas - Avoid truncated output
Is there any way to import a large CSV data file into Pycharm using Pandas Import?有什么方法可以使用 Pandas Import 将一个大的 CSV 数据文件导入到 Pycharm 中吗? Because no matter what I do, the output seen in the run terminal is severely truncated which is not good for any selection or cleaning data operations.
因为无论我做什么,在运行终端中看到的 output 都被严重截断,这对任何选择或清理数据操作都不利。
Any suggestions would be appreciated.任何建议,将不胜感激。
Pandas provides options for displaying DataFrame. Pandas 提供显示 DataFrame 的选项。
pd.options.display.width
pd.options.display.max_columns
pd.options.display.max_rows
By default, pandas will display a truncated table if the DataFrame has more rows/columns than max_rows
/ max_columns
.默认情况下,如果 DataFrame 的行数/列数多于
max_rows
/ max_columns
,则 pandas 将显示截断表。 You can adjust this if you want.如果你愿意,你可以调整这个。 Here's some sample code.
这是一些示例代码。
>>> import pandas as pd
>>> from random import random
>>> df = pd.DataFrame({
... f'c{col_no}': [random() for _ in range(100)]
... for col_no in range(15)
... })
>>> pd.options.display.max_columns, pd.options.display.max_rows
(0, 60)
>>> df
c0 c1 c2 ... c12 c13 c14
0 0.871826 0.415696 0.962756 ... 0.036385 0.405643 0.807471
1 0.531463 0.516149 0.811182 ... 0.588035 0.015000 0.447855
2 0.703785 0.793341 0.019570 ... 0.374489 0.057472 0.590761
3 0.762984 0.171603 0.127855 ... 0.357097 0.013220 0.132322
4 0.991035 0.113433 0.840822 ... 0.113895 0.707505 0.457993
.. ... ... ... ... ... ... ...
95 0.438203 0.465847 0.287558 ... 0.236885 0.495121 0.115823
96 0.612054 0.709875 0.217789 ... 0.569730 0.779009 0.429083
97 0.396499 0.017465 0.075139 ... 0.032245 0.955732 0.708767
98 0.096672 0.227434 0.347087 ... 0.841708 0.031055 0.689640
99 0.123338 0.199680 0.284335 ... 0.328187 0.362656 0.379024
>>> pd.options.display.width = 200
>>> pd.options.display.max_columns = 15
>>> pd.options.display.max_rows = 100
>>> df
c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 c14
0 0.871826 0.415696 0.962756 0.337541 0.798125 0.641710 0.060606 0.268195 0.033646 0.713952 0.999305 0.266091 0.036385 0.405643 0.807471
1 0.531463 0.516149 0.811182 0.517024 0.907563 0.098621 0.486572 0.105661 0.233740 0.442899 0.882617 0.491250 0.588035 0.015000 0.447855
2 0.703785 0.793341 0.019570 0.656947 0.771691 0.163144 0.739283 0.775620 0.454568 0.739937 0.376440 0.783414 0.374489 0.057472 0.590761
3 0.762984 0.171603 0.127855 0.347233 0.681083 0.469366 0.074852 0.327360 0.583786 0.570660 0.918842 0.140252 0.357097 0.013220 0.132322
4 0.991035 0.113433 0.840822 0.198988 0.117649 0.148605 0.173794 0.126979 0.322275 0.766880 0.011601 0.918334 0.113895 0.707505 0.457993
5 0.027492 0.441665 0.015462 0.425986 0.876837 0.041831 0.385929 0.622585 0.893251 0.207410 0.126994 0.540103 0.132818 0.320651 0.135680
6 0.364498 0.777506 0.571290 0.463168 0.372986 0.727358 0.286281 0.060411 0.091997 0.599882 0.914836 0.713235 0.769993 0.912143 0.973625
7 0.021097 0.271388 0.903971 0.347351 0.255841 0.020190 0.307909 0.189683 0.635788 0.932846 0.740916 0.657532 0.347275 0.677888 0.027598
8 0.594859 0.905407 0.767936 0.929833 0.048191 0.084725 0.967413 0.183815 0.758094 0.686023 0.087515 0.512909 0.942502 0.858353 0.855532
9 0.899373 0.681138 0.546424 0.809373 0.174588 0.691135 0.755386 0.590502 0.161688 0.711284 0.918817 0.579863 0.599287 0.280585 0.691854
10 0.471923 0.523145 0.918165 0.406063 0.095486 0.972089 0.724117 0.231671 0.200418 0.733166 0.019452 0.128490 0.524909 0.895029 0.584772
... print all rows
Reference: Options and settings - pandas参考: 选项和设置 - pandas
In PyCharm you can use SciView to explore DataFrame.在 PyCharm 中可以使用SciView探索 DataFrame。
Click 'View as DataFrame' in 'Variables View' (right panel)在“变量视图”(右侧面板)中单击“以数据帧形式查看”
The DataFrame will opened in 'SciView' panel. DataFrame 将在“SciView”面板中打开。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.