Python message to other applications

Question

Status Quo:

I have two python apps (frontend-server and data-collector, a database is 'between' them).

Currently using redis as db and its publish/subscribe protocol to notify the frontend when new data is available.

But may I want to use a different database (and don't want to keep redis on the system just for the pub/sub).

Are there any simple alternatives to notify my frontend if the data-collector has transacted new data to the database (without using an external message queue like beanstalkd or redis)?

Answer 1

ZeroMQ is a good option. It has good Python bindings, and it makes communicating between processes on the same machine and processes on different machines look almost identical.

Start by reading the guide: http://zguide.zeromq.org/page:all

Answer 2

As I mentioned in my comment, if you want something that is going across a network then other than setting up a web service (flask app?), or writing your own INET socket server there is nothing built in to the operating system to communicate between machines. Beanstalk has a very simple API in Python and I've used it for this kind of thing very successfully.

try:
    beanstalk = beanstalkc.Connection(host="my.host.com")
    beanstalk.watch("update_queue")
except:
    print "Error connecting to beanstalk"

while True:
    job = beanstalk.reserve()
    do_something_with_job(job)

If you are only going to be working on the same machine, then read up on linux IPC. A socket connection between processes is very fast and has practically zero overhead. They can also be a part of an asynchronous program when you take advantage of epoll call backs.