How to split a pandas dataframe using multiple delimiters?

Question

I am new to Pandas, and have a dataframe ("temp") that looks like this:

                           ts;"val"
0       2019-12-02T19:59:32.735;75.2
1       2019-12-02T20:00:53.276;75.2
2       2019-12-02T20:02:01.170;75.2
3      2019-12-02T20:03:09.159;75.02
4       2019-12-02T20:04:17.145;75.2
5       2019-12-02T20:05:25.131;75.2
6      2019-12-02T20:06:33.116;75.02
7      2019-12-02T20:07:40.100;75.02
8      2019-12-02T20:08:48.087;74.84
9      2019-12-02T20:09:56.071;74.66
10     2019-12-02T20:11:04.063;74.66
11     2019-12-02T20:12:12.055;74.48
12     2019-12-02T20:13:20.041;74.48
13      2019-12-02T20:14:28.028;74.3
14     2019-12-02T20:15:36.012;74.12
15     2019-12-02T20:16:42.997;74.12
16     2019-12-02T20:17:50.983;74.12
17     2019-12-02T20:18:58.969;74.12
18     2019-12-02T20:20:06.955;74.12
19     2019-12-02T20:21:14.938;74.12

I want to split this into 3 columns : "Date", "Time" and " Value".

I am currently using temp_d1 = temp['ts;"val"'].apply(lambda x: pd.Series(x.split('T'))) and then repeating this on temp_d1 and then concatenate temp_d1 and temp_d2 (the new dataframe).

Is there a better/easier way to do this?

Answer 1

looks like your dataframe has a preset delimiter set to ;

change your pd.read_csv to handle it ie pd.read_csv(file,sep=';')

then apply a pd.to_datetime

if that doesn't work, then you can do something like the following:

df2 = df['ts;"val"'].str.split(';',expand=True)
df2['time'] = df2[0].apply(pd.to_datetime,format='%Y-%m-%dT%H:%M:%S').dt.floor('s').dt.time

df2[0] = df2[0].apply(pd.to_datetime,format='%Y-%m-%dT%H:%M:%S').dt.normalize()
df2.columns = ['date', 'value','time']
print(df2[['date','time','value']])

---

         date      time  value
0  2019-12-02  19:59:32   75.2
1  2019-12-02  20:00:53   75.2
2  2019-12-02  20:02:01   75.2
3  2019-12-02  20:03:09  75.02
4  2019-12-02  20:04:17   75.2
5  2019-12-02  20:05:25   75.2
6  2019-12-02  20:06:33  75.02
7  2019-12-02  20:07:40  75.02
8  2019-12-02  20:08:48  74.84
9  2019-12-02  20:09:56  74.66
10 2019-12-02  20:11:04  74.66
11 2019-12-02  20:12:12  74.48
12 2019-12-02  20:13:20  74.48
13 2019-12-02  20:14:28   74.3
14 2019-12-02  20:15:36  74.12
15 2019-12-02  20:16:42  74.12
16 2019-12-02  20:17:50  74.12
17 2019-12-02  20:18:58  74.12
18 2019-12-02  20:20:06  74.12
19 2019-12-02  20:21:14  74.12

Answer 2

Here is how you could do it using list comprehension:

temp['Date'] = [x.split('T')[0] for x in temp['ts;"val"']]
temp['Time'] = [x.split('T')[1].split(';')[0] for x in temp['ts;"val"']]
temp['Value'] = [x.split(';')[1] for x in temp['ts;"val"']]

---

Output:

                        ts;"val"        Date          Time  Value
0   2019-12-02T19:59:32.735;75.2  2019-12-02  19:59:32.735   75.2
1   2019-12-02T20:00:53.276;75.2  2019-12-02  20:00:53.276   75.2
2   2019-12-02T20:02:01.170;75.2  2019-12-02  20:02:01.170   75.2
3  2019-12-02T20:03:09.159;75.02  2019-12-02  20:03:09.159  75.02
4   2019-12-02T20:04:17.145;75.2  2019-12-02  20:04:17.145   75.2

Answer 3

You can do it with:

1. First 10 digit is Date

2. 11 digit onward is time

   df['Date'] = df['ts'].str[:10] df['Time'] = df['ts'].str[11:]