Is there a smart/easy way to tell read_csv in pandas not to load data after a certain "end of section" flag? Or for it to stop if it gets to an empty row?
data = pd.read_csv(path, **params)
eos_line = (data['id'] == eos_string).idxmax()
data = data.drop(range(eos_line-2, data.shape[0]))
I feel like their ought to be a better way. Unfortunately I don't know the number of rows or length of the footer I want to skip before calling read_csv. The data looks roughly something like
1,2,3
4,5,6
dont want any data after this line
7,8,9
10,11,12
(Note: the -2 is b/c there are actually 2 empty rows before the end of section string, but if read_csv only read until this point I guess dropna() would remove these two rows pretty painlesslly)
Wes did think of everything!
In [40]: data = """A,B,C
....: 1,2,3
....: 4,5,6
....: 7,8,9
....: want to skip this
....: also also skip this
....: """
In [41]: read_csv(StringIO(data), skip_footer=2)
Out[41]:
A B C
0 1 2 3
1 4 5 6
2 7 8 9
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.