Python request.post Not Accepting Dictionary

Question

here is the data that I want to post,

this is the printed version of the dictionary answers:

{'Emails': set([u'vernon.fadel@denesik.com', u'helyn67@cruickshankmckenzie.com', u'tryan@fishercremin.com', u'cecilia.kerluke@gmail.com', u'sonia.ruecker@schmidt.info', u'reichel.tamatha@gmail.com', u'johntyree69@greendale.edu', u'gail34@reichel.com', u'birdman@quitzon.net ', u'onolan@gmail.com', u'lindgren.merry@howe.com', u'wilderman.nyree@heidenreich.com', u'yschneider@jacobson.biz', u'nakia.larkin@gmail.com', u'vblanda@gmail.com', u'pkoch@yahoo.com', u'ygoyette@yahoo.com', u'wiegand.lynnette@beer.com', u'ubradtke@hotmail.com', u'troy67@yahoo.com', u'eileen.gusikowski@oreilly.info', u'glubowitz@yahoo.com', u'inolan@hotmail.com', u'lfranecki@hotmail.com', u'miller.hana@gmail.com', u'morissette.lalla@kuhicmuller.com', u'harber.gisele@gmail.com', u'odalys.dubuque@ebertrunte.net', u'jones.kyree@auer.net', u'adams77@yahoo.com', u'parisian.linda@hotmail.com', u'channing.cremin@toy.org', u'chad.armstrong@yahoo.com', u'isaac.nolan@hotmail.com', u'kozey.luciana@collins.biz', u'cstehr@keebler.com', u'jquitzon@gmail.com', u'hhirthe@yahoo.com', u'emilia16@block.com', u'arice@greenholtconsidine.com', u'tobin.schuppe@gmail.com', u'wyman.junious@gmail.com', u'mrath@ruecker.com ', u' johns.sonji@lakinwalker.info', u'thiel.cassie@hodkiewicz.info', u'fjaskolski@hotmail.com', u'mosciski.destany@hotmail.com', u'vrippin@gmail.com', u'zschuppe@hotmail.com', u'clotilda26@goodwinlind.info', u'lizette.walker@stanton.com', u'hettinger.kamari@hotmail.com', u'emelia54@yahoo.com', u'terry.ezekiel@hotmail.com', u'douglyass.dickinson@hotmail.com', u'zulauf.olympia@gmail.com', u'maxxx.waters@sauer.biz', u'abshire.charmaine@schusterlabadie.com', u'kuvalis.dewitt@wolfschaefer.com', u"elberta.o'connell@yahoo.com", u'henery76@yahoo.com', u'britny60@gmail.com', u'ed.hilpert@hansen.com', u'burnice.feest@yahoo.com', u' ybartoletti@littel.biz', u'sharde.murazik@hotmail.com', u'angelica.klocko@ankunding', u'brandi.klein@hotmail.com', u'dollie49@hotmail.com', u'mrath@ruecker.com', u'hayley27@koelpin.com', u'georgiann.mertz@moen.net', u'floretta39@volkmanrosenbaum.net', u'vicy.predovic@gmail.com', u'raven53@yahoo.com', u'fmann@yahoo.com', u'sdaugherty@gmail.com', u'gerlach.villa@koelpinfeeney.net', u'alysha.gibson@hotmail.com', u'hoy.kozey@gmail.com', u'cheyenne84@lehneroconner.com', u'uoberbrunner@hotmail.com', u'hhaag@swaniawskiterry.info', u'infant80@yahoo.com', u'manilla.west@hotmail.com', u'hconsidine@bernhardhills.info', u'dean94@hotmail.com', u'bahringer.karl@yahoo.com', u'crona.meredith@gmail.com', u'letta.larkin@quitzon.net', u'owindler@strosin.info', u'shanahan.anders@yahoo.com', u'mclaughlin.desi@gmail.com', u'erdman.icy@hotmail.com', u'salvatore21@hintz.com', u'jkoch@hotmail.com', u'red.beatty@yahoo.com', u'helma91@hotmail.com', u'arland.koepp@hotmail.com', u'ymoore@cristhalvorson.org', u'woodie.crooks@kozey.com', None, u'jchamplin@hotmail.com ', u'walter.elwyn@yahoo.com']), '# of Emails per Domain': {u'@hotmail.com': 22, u'@yahoo.com': 18, u'@gmail.com': 18}, 'April Users': 70}

when I run:

 r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data = answers)

I get this error:

{"message": "Could not parse request body into json: Unrecognized token \'Emails\': was expecting (\'true\', \'false\' or \'null\')\n at [Source: [B@311a7237; line: 1, column: 8]"}

Not sure what this means at all, I've tried to parse the data into json, why doesn't it like the 'Email' string?

If I add the json.dump code I get:

TypeError

TypeError: set([u'vernon.fadel@denesik.com', u'helyn67@cruickshankmckenzie.com', u'tryan@fishercremin.com', u'cecilia.kerluke@gmail.com', u'sonia.ruecker@schmidt.info', u'reichel.tamatha@gmail.com', u'johntyree69@greendale.edu', u'gail34@reichel.com', u'birdman@quitzon.net ', u'onolan@gmail.com', u'lindgren.merry@howe.com', u'wilderman.nyree@heidenreich.com', u'yschneider@jacobson.biz', u'nakia.larkin@gmail.com', u'vblanda@gmail.com', u'pkoch@yahoo.com', u'ygoyette@yahoo.com', u'wiegand.lynnette@beer.com', u'ubradtke@hotmail.com', u'troy67@yahoo.com', u'eileen.gusikowski@oreilly.info', u'glubowitz@yahoo.com', u'inolan@hotmail.com', u'lfranecki@hotmail.com', u'miller.hana@gmail.com', u'morissette.lalla@kuhicmuller.com', u'harber.gisele@gmail.com', u'odalys.dubuque@ebertrunte.net', u'jones.kyree@auer.net', u'adams77@yahoo.com', u'parisian.linda@hotmail.com', u'channing.cremin@toy.org', u'chad.armstrong@yahoo.com', u'isaac.nolan@hotmail.com', u'kozey.luciana@collins.biz', u'cstehr@keebler.com', u'jquitzon@gmail.com', u'hhirthe@yahoo.com', u'emilia16@block.com', u'arice@greenholtconsidine.com', u'tobin.schuppe@gmail.com', u'wyman.junious@gmail.com', u'mrath@ruecker.com ', u' johns.sonji@lakinwalker.info', u'thiel.cassie@hodkiewicz.info', u'fjaskolski@hotmail.com', u'mosciski.destany@hotmail.com', u'vrippin@gmail.com', u'zschuppe@hotmail.com', u'clotilda26@goodwinlind.info', u'lizette.walker@stanton.com', u'hettinger.kamari@hotmail.com', u'emelia54@yahoo.com', u'terry.ezekiel@hotmail.com', u'douglyass.dickinson@hotmail.com', u'zulauf.olympia@gmail.com', u'maxxx.waters@sauer.biz', u'abshire.charmaine@schusterlabadie.com', u'kuvalis.dewitt@wolfschaefer.com', u"elberta.o'connell@yahoo.com", u'henery76@yahoo.com', u'britny60@gmail.com', u'ed.hilpert@hansen.com', u'burnice.feest@yahoo.com', u' ybartoletti@littel.biz', u'sharde.murazik@hotmail.com', u'angelica.klocko@ankunding', u'brandi.klein@hotmail.com', u'dollie49@hotmail.com', u'mrath@ruecker.com', u'hayley27@koelpin.com', u'georgiann.mertz@moen.net', u'floretta39@volkmanrosenbaum.net', u'vicy.predovic@gmail.com', u'raven53@yahoo.com', u'fmann@yahoo.com', u'sdaugherty@gmail.com', u'gerlach.villa@koelpinfeeney.net', u'alysha.gibson@hotmail.com', u'hoy.kozey@gmail.com', u'cheyenne84@lehneroconner.com', u'uoberbrunner@hotmail.com', u'hhaag@swaniawskiterry.info', u'infant80@yahoo.com', u'manilla.west@hotmail.com', u'hconsidine@bernhardhills.info', u'dean94@hotmail.com', u'bahringer.karl@yahoo.com', u'crona.meredith@gmail.com', u'letta.larkin@quitzon.net', u'owindler@strosin.info', u'shanahan.anders@yahoo.com', u'mclaughlin.desi@gmail.com', u'erdman.icy@hotmail.com', u'salvatore21@hintz.com', u'jkoch@hotmail.com', u'red.beatty@yahoo.com', u'helma91@hotmail.com', u'arland.koepp@hotmail.com', u'ymoore@cristhalvorson.org', u'woodie.crooks@kozey.com', None, u'jchamplin@hotmail.com ', u'walter.elwyn@yahoo.com']) is not JSON serializable

Answer 1

TypeError: set([ ... ]) is not JSON serializable

Sets are not JSON serializable . You can either convert all of the sets to lists in the dictionary:

answers = {key: list(value) if isinstance(value, set) else value
           for key, value in answers.items()}

^{(Note that this would not work for nested dictionaries as is).}

Or, use a custom JSON encoder that would handle the set serialization.

---

Dumping and loading it back with json usually helps in debugging:

import json

answers = json.loads(json.dumps(answers))
r = requests.post('https://9g9xhayrh5.execute-api.us-west-2.amazonaws.com/test/data', data=answers)

Answer 2

This works:

import requests
import json
answers = {} #...
url = "..."
answers['Emails'] = list(answers['Emails'])
r = requests.post(url, data=json.dumps(answers))

The reason the other way doesn't work is that by default requests sends data with the Content-Type application/x-www-form-urlencoded instead of JSON.

To change that, you can also use json instead of the data parameter:

r = requests.post(url, json=answers)

---

To debug code using the requests library, you can look at a prepared request before sending it.