How to extract data from incoming HTTP POST using python
Question
I have a Ubuntu LAMP webserver and data is being sent to the webserver continuously through HTTP POST method. I need to extract the data from HTTP POST and insert them in a Database. I don't know how to do that. There is a lot of example available on how to handle outgoing HTTP POST request but the incoming HTTP POST request. I want to write a python3 script which will extract the data from incoming HTTP POST request and save them as varible which I will use to insert the data into the database and also return a response to the client.Can anybody help me in this regard?
4 answers
solution1
0 2018-11-17 06:28:03
UPDATE
According to the code you posted below, here is a working answer.
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
Calling it with Postman
and the command line output is
C:\\Development\\Python\\test\\venv\\Scripts\\python.exe C:/Development/Python/test/webserver_old.py 1001 jyoti0 127.0.0.1 - - [19/Nov/2018 21:53:45] "POST /do_something HTTP/1.1" 200 - jyoti1 101
I combined answers from these here: Reference one , two and third And this is also very important to read: https://docs.python.org/3/library/http.server.html
http.server is not recommended for production. It only implements basic security checks.
I believe is ok for a small implementation and some testing around or proof of concept but eventually you'll need to manage this better, maybe I can suggest you to spend some time and use Flask , is actually an excellent & very light framework for Python API building and prototyping.
-
Previous answer (deprecated & updated above)
-
As per a very light and simple reference to this one:
def do_POST(self):
# Doesn't do anything with posted data
content_length = int(self.headers['Content-Length']) # <--- Gets the size of data
post_data = self.rfile.read(content_length) # <--- Gets the data itself
self._set_headers()
self.wfile.write("<html><body><h1>POST!</h1></body></html>")
Update (without and API):
Assuming you are running on or machine at a custom port with a custom trailing part at the URL, then "pure" python would look like this:
import SocketServer
from BaseHTTPServer import BaseHTTPRequestHandler
def doSomething():
print "i did"
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path == '/doSomething':
mail = self.request.POST.get('email')
something = self.request.POST.get('something')
doSomething()
self.send_response(200)
httpd = SocketServer.TCPServer(("", 8080), MyHandler)
httpd.serve_forever()
I assume this way you can reuse the variables freely. Check also this reference here , Brenda's answer.
solution2
0 2018-11-21 10:42:35
@oetoni, I am getting time out error while using:
#!/usr/bin/python3
import socketserver
from http.server import BaseHTTPRequestHandler
import time
import threading
def do_something(site_id, first, last, pass1):
print(site_id)
print(first)
print(last)
print(pass1)
#just to illustrate the point and print the variables
class MyHandler(BaseHTTPRequestHandler):
def do_POST(self): # !important to use 'do_POST' with Capital POST
global site_id, first, last, pass1 #those are still undefined at the module level ;) remember this for later
if self.path == '/do_something':
request_headers = self.headers
site_id = request_headers["m_site_name"]
first = request_headers["m_first_name"]
last = request_headers["m_last_name"]
pass1 = request_headers["m_device_name"]
do_something(site_id, first, last, pass1)
self.send_response(200)
self.end_headers() #as of P3.3 this is required
try:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.serve_forever()
finally:
httpd = socketserver.TCPServer(("localhost", 9001), MyHandler)
httpd.server_close()
But I am getting the correct response while using this code:
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/html')
print('')
arguments = cgi.FieldStorage()
for i in arguments.keys():
print(arguments[i].value)
and it prints the received data on the web browser. I am using this script as a cgi script on the apache web server which can be accessed through a web browser. I am not running this script as a service or application.
solution3
0 ACCPTED 2018-11-22 07:26:10
#!/usr/bin/python3
# -*- coding: UTF-8 -*-
import cgi
import cgitb
cgitb.enable()
print('Content-Type: text/html\n')
arguments = cgi.FieldStorage()
print(arguments["m_site_name"].value)
print("<br />\n")
print(arguments["m_first_name"].value)
print("<br />\n")
print(arguments["m_last_name"].value)
print("<br />\n")
print(arguments["m_device_name"].value)
print("<br />\n")
site = arguments["m_site_name"].value
first = arguments["m_first_name"].value
last = arguments["m_last_name"].value
device = arguments["m_device_name"].value
-----do_other_things_with_the_variables(site,first,last,device)-----
This code solved my problem. Now I can store HTTP POST data into variables with this python cgi script.
my HTTP POST Request: http://your_server_url_or_IP/cgi-bin/python_script.py?m_site_name=MySite&m_first_name=anyname&m_last_name=anylastanme&m_device_name=anydeviceidorname
solution4
0 2021-10-18 23:32:21
With python3, inside do_POST() of your handler class based on http.server.*Handler :
import cgi
enctype, attrs = cgi.parse_header(self.headers['Content-Type'])
if enctype == 'multipart/form-data':
boundary = {'boundary': attrs['boundary'].encode() }
form_data = cgi.parse_multipart(self.rfile, boundary)
file_content = form_data.get('myfile')
fname = 'data/uploads/' + str(time.time()) + '.json'
with open(fname, 'wb') as fp:
for part in file_content:
fp.write(part)
Don't forget to insert Content-Length check to limit the max file size. Presumably cgi.FieldStorage stops reading when limit bytes reached (if specified) and also deals better with large files in general. This is not part of official doc . I read it in the source file cgi.py .
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.