我的正则表达式为什么不能正确解析此http请求？

Question

I have the following http request payload.

X-gmsv=9879480&X-subtype=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&X-X-subscription=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&X-gcm.topic=%2Ftopics%2Fphenotype_com.google.android.gms.icing%25servingVersion&X-X-subtype=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&X-app_ver=9879480&X-kid=%7CID%7C7%7C&X-osv=25&X-sig=vWteecmhHl5q2AsrHaGcOOgaF956SpVk6KAdjijNyeX1uADvPgpgvMkNH-Nu-N8IHc-1Z1ujTytjkQDPZot4zjf_FLSjR0ucPIkFXkZhrRi5RU6uFq-ZlQCEBSPpYuHsx27lC5H3xv-TNe_zC0PaX8h8bTqrImtArVSZjMY6-RFG9TUEj2VkCvs1ixAK21vHxE4ladiXALZO-lhZIvbDIGkY4c-fUMaMBN8EhMr1zH31N41S6cUItkPRe0lTOB4YddkrS2FNRI_LZGfW-cc9h9om-80MskZD0IyJtM4AFsumHxVIQQJwSScASSoFd7e7tANTp5ZPJi2hwr6wQqpveQ&X-cliv=iid-9879000&X-gmsv=9879480&X-pub2=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA9VWXXfudfnoAAl-u_RbBClmI6uvaOH8AFEMvjrtOpL1FuLrUYQzdntRwlMyiL4Nba7WUGeb6CrkEAbwTFcR689QYQ87ytkyY65rD2InSUD3eMLWpiaTciFj-n5sUK6hyci5Je5T8Svgsb-VHSy6vWVKQZ4vGsiGqmkj8sDhCa1UbltWOyhywfG95ENiGKuO_ec55Rmvrew9tFNGIit7FzcNiEAmfSrkEifK6dydjnpahu3lAx4U_MTw5Yo0ou5EGrsByXY2P_tkWg78hq1E_SQORk7q7droAY_wupXHlqSwGCAfbGtRs2gXM-64MSZ1iQX7N7pPojkT4akomcyP2JQIDAQAB&X-X-kid=%7CID%7C7%7C&X-appid=cIIP-1V_bTg&X-scope=%2Ftopics%2Fphenotype_com.google.android.gms.icing%25servingVersion&X-subscription=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&X-app_ver_name=9.8.79+%28480-137224771%29&app=com.google.android.gms&sender=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&device=4374365252386389758&cert=58e1c4133f7441ec3d2c270270a14802da47ba0e&app_ver=9879480&info=AoehgPKryS4XzDpwLBRWN3IuplGtswI&gcm_ver=987948

I want to pull out all the pairs of <key>=<value> . For example the first key value pair is X-gmsv=9879480 .

The best regex I have come up with is .*?\\=.*?& but it get's everything but the last key value pair because there is no ampersand at the end of the last variable. So I tried .*?\\=.*?[&|$] which in theory should either match a key value pair ending with an ampersand or the end of the string.

It doesn't get the last string. I've played around with a couple other regexes and can't figure out what's going on.

Any ideas?

Answer 1

If you insist regex... here it is.

.*?\=.*?(?:&|$)

It captures 24 groups. And

len(input.split('&')) 

is 24 too.

Answer 2

I would highly recommend not to use a regex for that. Use the stdlib urlparse.parse_qs() function. It will also take care of URL-encoding and such for you:

>>> import urlparse
>>> urlparse.parse_qs('X-gmsv=9879480&X-subtype=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&X-X-subscription=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&X-gcm.topic=%2Ftopics%2Fphenotype_com.google.android.gms.icing%25servingVersion&X-X-subtype=cIIP-1V_bTg%3AAPA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW&X-app_ver=9879480&X-kid=%7CID%7C7%7C&X-osv=25')
{'X-subtype': ['cIIP-1V_bTg:APA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW'], 'X-osv': ['25'], 'X-X-subscription': ['cIIP-1V_bTg:APA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW'], 'X-kid': ['|ID|7|'], 'X-app_ver': ['9879480'], 'X-gmsv': ['9879480'], 'X-X-subtype': ['cIIP-1V_bTg:APA91bG-C3lFgSEzXCnuaLgpa4oJ0mI3NRk8Yv03NBOTfARjfBMWGhwy9J3d2dKUtGZHt6IKFmt7BBWRrQBqbvPoMobfZ2DP1Za0EyDzBqtfTLz9j-EYUHU1PWVjM2kMnOtIuA1s4EHW'], 'X-gcm.topic': ['/topics/phenotype_com.google.android.gms.icing%servingVersion']}

Note: In Python 3 this is urllib.parse.parse_qs()