Multistage Django deployments: Part 1
At Europython 2008 I gave a presentation of how we were using Capistrano for deploying the Pylons based widgets.opera.com site. When rewriting this blog in Django I decided to look again at more pythonic alternatives. This is the first part of a two part blog post, in this part you will learn the tools I use to create a sane, reproducible deployment. The second part will tie the tools together with version control to perform one click deployment.
Through a development iteration my website will pass though each of the following stages;
New features and functionality begin life on my own machine. The code runs via the Django development server and sqlite is used as a database to avoid the need for running any extra servers.
Once I'm satisfied that something works fine on my local machine I will then push it to a testing installation on my Dreamhost account. Here the application runs as an fcgi application, with all the caching enabled that would normally exist in production and runs against a backup of the production database so that I can test the functionality still works with the existing data.
Once I have a version in testing that I'm sure works well and isn't going to destroy my data I then promote my code to a staging installation of the site. This staging version of the application is configured against the real production database so I can perform a final test that everything is going to work fine before I flick the switch to make it live.
Once satisfied that everything looks good in staging the site is then promoted to production in one smooth operation, no users will notice any downtime or broken functionality during the upgrade.
When developing python applications or deploying several versions to the same machine, it is considered bad practice to install the application in the global site-packages. This means that you need to install your python packages elsewhere and modify the PYTHONPATH before running the application. There are two tools that try and automate this process, these are Virtualenv and zc.buildout.
Although both tools produce a similar result, with Virtualenv you need to use setuptools to install your application and its dependencies into the environment, whereas zc.buildout performs everything in a single step along with adding the possibility to perform other tasks, such as generating configuration files.
For my blog I have the following buildout config
[buildout] parts = django [django] recipe = djangorecipe version = 1.0.2 project = sharebear settings = settings eggs = docutils staticgenerator flup south fcgi = true
This file is all I need to get started with my Django project. After running buildout I have all of my dependencies downloaded (defined in the eggs argument) and there is a bin/ folder created which contains two files. The first of these files is an executable script named django that I can use to run all of the normal django management commands (runserver, syncdb, createapp, etc.) with a correctly configured python path. The other file is django.fcgi an fcgi file suitable for running my Django app using an fcgi server.
In my opinion the most important part of a database oriented project is the database schema and the data within it. Unfortunately it seems that many projects still run without any safe method of testing and performing changes to the database schema. Most projects still seem to rely on a human being running the SQL scripts, that have received limited testing, against the databases as necessary.
The solution to this problem is to make use of a database migration tool. Migration tools allow you to define the steps necessary to upgrade the database to the state it needs to be in and also keeps track of which migrations have already been run against a particular database instance so you don't run the risk of running the same script twice against the same database. Migration tools based on scripting languages can also allow you to do advanced manipulation of data in the database so that you fix corrupt data caused by software bugs as well as doing simple schema changes such as adding columns and indexes.
There are several database migration tools targeted at Django projects but for my blog I have chosen to try out South, which seems to have the feature-set I desire and is under active development. I'm not going to go into detail about how to use South but here is an example migration from my project.
from south.db import db from django.db import models class Migration: def forwards(self): # Adding field 'Entry.content_html' db.add_column('blog_entry', 'content_html', models.TextField(editable=False, blank=True)) def backwards(self): # Deleting field 'Entry.content_html' db.delete_column('blog_entry', 'content_html')
One additional feature of migration tools as you can see here is that, not only can you migrate a database forwards but you can also rollback a migration if you need to back out of a software deployment. This is a situation that many manual migration strategies fail to consider.
Now you've had a brief introduction to the tools that help me organise my website deployment. In my next post I'll share with you details of how I use Fabric to tie everything together into a single action for deployment.