无法处理 python 中的 EmptyDataError

Question

I am trying to read in a huge csv file chunk by chunk using pandas read_csv function in combination with skiprows and nrows. I do this using a while loop.

When I reach the end of the file (skipping more rows than there are in the file) I get an EmptyDataError.

This is no problem per se, I try handling this with a try - except. However, the way I do it does not work...does anybody have an idea why? I am a beginner when it comes to error handling. This is my code:

chunksize = 1000
inputfile = "testfile_with_five_rows.txt"
chunknumber = 0
while True:
    try: 
        data = pd.read_csv(inputfile, skiprows=chunksize*chunknumber, nrows=chunksize)
        print(data)
    except EmptyDataError:
        break

    chunknumber+=1

I get the following NameError, instead of handling the FileNotFound exception:

    ---------------------------------------------------------------------------
EmptyDataError                            Traceback (most recent call last)
<ipython-input-22-e3736135fc2f> in <module>()
      5     try:
----> 6         data = pd.read_csv(inputfile, skiprows=chunksize*chunknumber, nrows=chunksize)
      7         print(data)

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, doublequote, delim_whitespace, low_memory, memory_map, float_precision)
    677 
--> 678         return _read(filepath_or_buffer, kwds)
    679 

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
    439     # Create the parser.
--> 440     parser = TextFileReader(filepath_or_buffer, **kwds)
    441 

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in __init__(self, f, engine, **kwds)
    786 
--> 787         self._make_engine(self.engine)
    788 

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in _make_engine(self, engine)
   1013         if engine == 'c':
-> 1014             self._engine = CParserWrapper(self.f, **self.options)
   1015         else:

~/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py in __init__(self, src, **kwds)
   1707 
-> 1708         self._reader = parsers.TextReader(src, **kwds)
   1709 

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__()

EmptyDataError: No columns to parse from file

During handling of the above exception, another exception occurred:

NameError                                 Traceback (most recent call last)
<ipython-input-22-e3736135fc2f> in <module>()
      6         data = pd.read_csv(inputfile, skiprows=chunksize*chunknumber, nrows=chunksize)
      7         print(data)
----> 8     except EmptyDataError:
      9         break
     10 

NameError: name 'EmptyDataError' is not defined

Answer 1

From comments:

Have you imported EmptyDataError from pandas.errors? – heena bawa

from pandas.errors import EmptyDataError