Integrate Tornado in Django

Tornado is a nice python WSGI-compliant web server developed by guys at FriendFeed. It’s primarily thought as application server for python web frameworks and according to FriendFeed benchmarks, it’s blazing fast thanks to its non-blocking connections.

UPDATE: For more performance info, James Abley pointed me to a very complete benchmark of available Python asynchronous webservers. It looks like Tornado is a real monster of concurrency.

There are already some how-to’s on the web on plugging Django web framework into Tornado webserver. A quick recap:

  1. A tutorial on Tornado, Django and nginx by Jeremy Bowers.
  2. How to import django framework inside a Tornado project by Lincoln Loop.
  3. A snippet by lawgon.

My approach is slightly different as I wanted to run Tornado using Django management command-line interface.

The 3 easy steps are:

  1. Add Tornado module to your django setup. If you use buildout, add Tornado git checkout to buildout.cfg using minitage.recipe.fetch recipe, like this:
    [buildout]
    ...
    parts =
    ...
        tornado
        django
    ...
    
    [tornado]
    recipe = minitage.recipe.fetch
    urls = git://github.com/facebook/tornado.git | git | | ${buildout:parts-directory}/tornado
    
    [django]
    recipe = minitage.recipe.scripts
    initialization =
        import os
        os.environ['DJANGO_SETTINGS_MODULE']='project.settings.development'
    scripts =
        django
    eggs =
        Django
        ...
    entry-points=
        django=django.core.management:execute_from_command_line
    extra-paths =
        ${buildout:directory}
        ${tornado:location}
    ...
  2. Next, create a command-line extension hierarchy in your project’s main app:
    $ mkdir project/myapp/management
    $ touch project/myapp/management/__init__.py
    $ mkdir project/myapp/management/commands
    $ touch project/myapp/management/commands/__init__.py
  3. Last, add a runtornado.py script in project/myapp/management/commands/ folder with the following content:
    from django.core.management.base import BaseCommand, CommandError
    from optparse import make_option
    import os
    import sys
    
    class Command(BaseCommand):
        option_list = BaseCommand.option_list + ()
        help = "Starts a Tornado Web."
        args = '[optional port number, or ipaddr:port]'
    
        def handle(self, addrport='', *args, **options):
            import django
            from django.core.handlers.wsgi import WSGIHandler
            from tornado import httpserver, wsgi, ioloop
    
    	sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
    	sys.stderr = os.fdopen(sys.stderr.fileno(), 'w', 0)
    
            if args:
                raise CommandError('Usage is runserver %s' % self.args)
            if not addrport:
                addr = ''
                port = '8000'
            else:
                try:
                    addr, port = addrport.split(':')
                except ValueError:
                    addr, port = '', addrport
            if not addr:
                addr = '127.0.0.1'
    
            if not port.isdigit():
                raise CommandError("%r is not a valid port number." % port)
    
            quit_command = (sys.platform == 'win32') and 'CTRL-BREAK' or 'CONTROL-C'
    
            def inner_run():
                from django.conf import settings
                print "Validating models..."
                self.validate(display_num_errors=True)
                print "\nDjango version %s, using settings %r" % (django.get_version(), settings.SETTINGS_MODULE)
                print "Server is running at http://%s:%s/" % (addr, port)
                print "Quit the server with %s." % quit_command
                application = WSGIHandler()
                container = wsgi.WSGIContainer(application)
                http_server = httpserver.HTTPServer(container)
                http_server.listen(int(port), address=addr)
                ioloop.IOLoop.instance().start()
    
            inner_run()

To run your tornado webserver, you just need to call your usual management program like manage.py with runtornado command, with the same syntax as runserver. In my case, I just run production server using supervisord, with a command like this:

$ ./bin/django runtornado --settings=project.settings.production 8000

If you found this quick how-to useful, remember to follow me on Twitter or subscribe to my feed for more django tips.

This entry was posted in Coding, How-tos and tagged , , , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

9 Comments

  1. Posted February 8, 2010 at 11:49 AM | Permalink

    If you haven't checked it out yet, I have found that djangorecipe is a superb way to manage Django with zc.buildout.

  2. Posted February 8, 2010 at 11:55 AM | Permalink

    Hi Adomas,
    I used Djangorecipe successfully before moving to minitage. I actually use minitage.recipe.scripts because I needed some custom extensions to my setup. For instance, I can pass a “Django-patches” property to apply custom patches to Django code.

  3. Posted February 8, 2010 at 12:45 PM | Permalink

    Okay, need of patching makes sense — thanks for explanation.

  4. Posted February 8, 2010 at 2:55 PM | Permalink

    How does this end up being in terms of performance? Is this something you'd probably only want to do to tinker with, or do you feel it's suitable for a small to mid-sized site?

  5. Posted February 8, 2010 at 3:50 PM | Permalink

    @Greg: FriendFeed benchmarks look encouraging, but I haven't found any independent performance comparison yet and I haven't run any serious benchmarking. It's certainly ready for production usage and I currently use it for django projects that my company runs in production, using nginx as frontend. It feels responsive and memory/cpu load is certainly lower than running a lighttpd+fastcgi+django+flup setup.
    It does miss some features like built-in auto-reloading (which is something I don't want/need anyway).
    During this month I will be running more benchmarking to see what kind of setup and optimizations I need to handle a django deployment with 50k users and peaks of more than 100 concurrent connections and I will report some stats on Tornado then.

  6. Posted February 8, 2010 at 9:38 PM | Permalink

    Very neat. I'll have to tinker with this sometime. Thanks for the article!

  7. Posted February 11, 2010 at 12:37 AM | Permalink

    This might be a more independent benchmark of tornado, among others.

    http://nichol.as/asynchronous-servers-in-python

  8. Posted February 11, 2010 at 1:37 AM | Permalink

    Thanks James, I've updated my post to include your link :-)

  9. Eazyigz
    Posted July 30, 2010 at 3:21 PM | Permalink

    This is neat, but I think its too much code just to get a Tornado server running. On tornadoweb's site they show a couple-line example how to launch their server and listen on a particular port. Now, if you want your Tornado server to do ajax push (comet-style) to the client, you will have to run Tornado on a different port than Django. And that's when the real fun begins (headache of making it work with jQuery)!

6 Trackbacks

  1. By uberVU - social comments on February 8, 2010 at 12:41 PM

    Social comments and analytics for this post…

    This post was mentioned on Twitter by geekscrap: Integrate Tornado in Django http://goo.gl/fb/8XuP #coding #howtos #buildout #django #python #supervisord #tornado…

  2. [...] Integrate Tornado in Django — geek scrapgeekscrap.com [...]

  3. By Integrate Tornado in Django on February 8, 2010 at 8:10 PM

    [...] tutorial to integrate tornado in django, versus the alternative ways to integrate django in [...]

  4. By Weekly Digest for February 13th | William Stearns on February 14, 2010 at 5:26 AM

    [...] Shared Geek Scrap: Integrate Tornado in Django. [...]

  5. By Integrate Tornado in Django « WebGlide - Django on February 18, 2010 at 3:00 PM

    [...] Integrate Tornado in Django. A handy ./manage.py runtornado management command for firing up a Tornado server that serves your Django application. [...]

  6. [...] is the original:  Integrate Tornado in Django — geek scrap Tags: buildout, coding, django, exploit, facebook, integrate-tornado, open-source, Python, tools, [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>