简体   繁体   中英

pandas read_csv end of section flag

Is there a smart/easy way to tell read_csv in pandas not to load data after a certain "end of section" flag? Or for it to stop if it gets to an empty row?

data = pd.read_csv(path, **params)
eos_line = (data['id'] == eos_string).idxmax()
data = data.drop(range(eos_line-2, data.shape[0]))

I feel like their ought to be a better way. Unfortunately I don't know the number of rows or length of the footer I want to skip before calling read_csv. The data looks roughly something like

1,2,3
4,5,6


dont want any data after this line
7,8,9
10,11,12

(Note: the -2 is b/c there are actually 2 empty rows before the end of section string, but if read_csv only read until this point I guess dropna() would remove these two rows pretty painlesslly)

Wes did think of everything!

In [40]: data = """A,B,C
   ....: 1,2,3
   ....: 4,5,6
   ....: 7,8,9
   ....: want to skip this
   ....: also also skip this
   ....: """

In [41]: read_csv(StringIO(data), skip_footer=2)
Out[41]: 
   A  B  C
0  1  2  3
1  4  5  6
2  7  8  9

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM