Where Are the Python/Ruby/etc Messaging Systems?

Posted September 16, 2007

Scaling requires the ability to decompose apps into pieces which can be spread across multiple boxes. An application composed of multiple pieces needs an efficient, reliable way to communicate between the pieces.

With multiple cores now standard, even in a single box scenario we need pieces that can run independently and communicate efficiently.

Java has this via JMS, with multiple good implementations, both open and closed source.

So where are the high quality, standard communication pipes for scripting languages? Where is the equivalent of JMS for Python?

I realize I can use ActiveMQ with HJB and PyHJB, but I’m loathe to introduce three pieces of new software along with a new VM (the JVM required to run ActiveMQ) into the picture.

What I’m looking for is a standard interface for messaging that allows implementations that optimize both intra and inter box scenarios. It should allow work-queue scenarios as well as pub/sub. It should support persistence / fail-safe modes. It should enable message or event driven apps - ie. have a event loop that’s driven by message availability, ala MDBs in the Java world.

Is there such a thing and I’ve missed it? Or is it that no-one wants it?

I suppose there’s always Gearman and TheSchwartz…

3 Comments so far

  1. Mehdi on September 17th, 2007

    Tell me about it. We have a very similar problem and we’ve been putting off solving it.

    Have you seen STOMP?

    http://stomp.codehaus.org/

    It also has client libs for Ruby/Python/etc…

    http://stomp.codehaus.org/Clients

    Of course, we’re loading the entire thing on to the VM (ala JRuby) and once you’re in that world, JMS like solutions get a bit more plentiful.

  2. Parand Darugar on September 17th, 2007

    I did take a look at STOMP and almost went with it, but ended up not using for reasons I don’t recall. Let me know if you get experience with it.

  3. Starling in Python? - Standard Deviations on February 26th, 2008

    [...] answers half of my request for a Python or Ruby messaging server (it does the work-queue half, doesn’t do the pub/sub half). I think I’m going to give [...]

Leave a Reply