如何删除喜欢python的行之间的更多行？

Question

I have a weird file format

###########################################################
# Name of file#
# stuff[hh:mm:ss:ms] stuff[num] stuff[num] stuff[] stuff[]#
###########################################################
00:00:00.000 -1000 -1000 0.000001 20
00:00:00.001 -1000 -1000 0.000001 20
00:00:00.002 -1000 -1000 0.000001 20
00:00:00.003 -1000 -1000 0.000001 20
00:00:00.004 -1000 -1000 0.000001 20
00:00:00.005 -1000 -1000 0.000001 20
00:00:00.006 -1000 -1000 0.000001 20
00:00:00.007 -1000 -1000 0.000001 20

the problem is I need only info every 2 sec. Which means i need to edit out 1999 lines in between.(the space is actually /t) What is the best way of doing that. I would also like to have the numbers saved as numbers not strings.

df = pd.read_csv('file.txt', sep="\t",
names=("time", "num1", "num2", "num3", "num4"), skiprows=4)
df["abs_time"] = df.index * 1e-3

I had to define time differently i already have the code for that i just need to save it properly.

def get_sec(time_str):
m, s, ss = time_str.split(':')
return int(m) * 60 + int(s) + 0.01*int(ss)

Any help well appreciated.

Answer 1

As you need data for every 2 seconds, it will indicate you need to have second which is even and ending with "000"(you could choose odd seconds as well) assuming you have no missing data

def is_select(time_str):
    return str.endswith(time_str, ".000") and int(time_str[6:8])%2
df['even_seconds'] = pd.apply(lambda x: is_select(x["time"]), axis=1)
select_data = df[df.even_seconds==True]

x["time"][6:8] will give you seconds information (you could adjust the index yourself).

Of course, you could modify lambda function for other data selections.

Answer 2

You can use skiprows parameter to get odd rows (or even). From the documentation:

If callable, the callable function will be evaluated against the row indices, returning True if the row should be skipped and False otherwise. An example of a valid callable argument would be lambda x: x in [0, 2].

Here you have an example csv:

#
#
#
#
A,B
1,1
2,2
3,3
4,4

Then you can:

pd.read_csv('test.csv', skiprows=lambda x: True if x < 4 or x%2 == 1 else False)

Output:

   A  B
0  2  2
1  4  4

As you can see, you can read odd or even lines and thus getting only rows every 2 seconds. Notice though, this assumes:

1. You are using latest pandas version 0.20.2
2. Your data is consecutive, ie one row per second

Answer 3

You cumsum the milisecond and check if they are modulo 2000, assuming you have strings in your first column.

vector_bool = df[df.columns[0]].apply(lambda x: x.split(".")[-1]).astype(int).cumsum().apply( lambda x: x%2000 == 0 )

Then take only the row wich are true.

df_clean = df[vector_bool]