How to send a pygame image over websockets?
Question
I'm currently trying to code something that will let websites view my webcam. I'm roughly following the tutorial linked on this website , except using Python and pygame instead of Processing.
At the moment, my code is grabbing a pygame image (which was originally a SimpleCV image), attempting to convert it into jpg format, and send it over websockets to the client where it will display it inside an img tag. However, I can't seem to figure out how to convert a pygame image into jpg and get it to display properly on the web browser.
This is my code for the server, which uses Flask and gevent:
#!/usr/bin/env python
import base64
import cStringIO
import time
from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
from flask import Flask, request, render_template
import pygame
pygame.init()
import SimpleCV as scv
app = Flask(__name__)
cam = scv.Camera(0)
@app.route('/')
def index():
return render_template('index.html')
@app.route('/camera')
def camera():
if request.environ.get('wsgi.websocket'):
ws = request.environ['wsgi.websocket']
while True:
image = cam.getImage().flipHorizontal().getPGSurface()
data = cStringIO.StringIO()
pygame.image.save(image, data)
ws.send(base64.b64encode(data.getvalue()))
time.sleep(0.5)
if __name__ == '__main__':
http_server = WSGIServer(('',5000), app, handler_class=WebSocketHandler)
http_server.serve_forever()
This is my HTML file:
<!DOCTYPE HTML>
<html>
<head>
<title>Flask/Gevent WebSocket Test</title>
<script src="http://code.jquery.com/jquery-1.10.1.min.js"></script>
<script type="text/javascript" charset="utf-8">
$(document).ready(function(){
if ("WebSocket" in window) {
cam = new WebSocket("ws://" + document.domain + ":5000/camera");
cam.onmessage = function (msg) {
$("#cam").attr('src', 'data:image/jpg;base64,' + msg.data);
};
cam.onerror = function(e) {
console.log(e);
}
} else {
alert("WebSocket not supported");
}
});
</script>
</head>
<body>
<img id="cam" src="" width="640" height="480" />
</body>
</html>
These are the specific lines that I think I'm having trouble with:
while True:
image = cam.getImage().flipHorizontal().getPGSurface()
data = cStringIO.StringIO()
pygame.image.save(image, data)
ws.send(base64.b64encode(data.getvalue()))
time.sleep(0.5)
Currently, if I try and run my code, going to localhost:5000 will display an invalid jpg image. It also becomes really laggy if I try running it on Firefox, but that may be an unrelated issue that I can debug later.
I've checked and made sure that the pygame image is a valid one, since I'm converting it from another library, and also checked that I was using websockets correctly by sending text data back and forth.
I've also tried calling pygame.image.to_string to try and convert the pygame surface into RGB format, but that also doesn't work.
What am I doing wrong?
1 answers
solution1
6 ACCPTED 2013-08-24 03:53:50
Using the underlying PIL image, we can write to a file-like object, read back and base-64 encode it:
from geventwebsocket.handler import WebSocketHandler
from gevent.pywsgi import WSGIServer
from flask import Flask, request
from time import sleep
from cStringIO import StringIO
import pygame
pygame.init()
import SimpleCV as scv
app = Flask(__name__)
cam = scv.Camera(0)
@app.route('/camera')
def camera():
if request.environ.get('wsgi.websocket'):
ws = request.environ['wsgi.websocket']
while True:
fp = StringIO()
image = cam.getImage().flipHorizontal().getPIL()
image.save(fp, 'JPEG')
ws.send(fp.getvalue().encode("base64"))
#fp.close() << benchmark and memory tests needed
sleep(0.5)
if __name__ == '__main__':
http_server = WSGIServer(('',5000), app, handler_class=WebSocketHandler)
http_server.serve_forever()
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.