Friday 27 July 2012

Django, Meet Nose

At Glasses Direct, we use nose to run our tests, as it gives us all sorts of nice things like test functions and XUnit-compatible output (which Jenkins loves).  As the majority of our projects use Django, we use django-nose to integrate nose into our Django projects.  This gives us all of that nose loveliness when using manage.py test.
The process for setting this up is simple:
  1. Install django-nose and nose however you normally do (I would use pip install django_nose nose).
  2. Add 'django_nose' to INSTALLED_APPS.
  3. Add TEST_RUNNER = 'django_nose.NoseTestSuiteRunner' to your settings.
  4. Run manage.py test -s to check that nose is being used (the standard manage.py test doesn't have this option).
  5. Read manage.py test -h and nose docs to learn about the exciting things you can do with nose.
I've added a few more nose-related things to my blogging backlog, so I'll get to those eventually.

Wednesday 18 July 2012

Deploying Sentry on Heroku

For a recent personal project, I've been using Sentry to monitor errors and suchlike. I was hosting this on my VPS, but the app it is monitoring is hosted on Heroku, so I thought that (a) it would make more sense to have the monitoring In The Cloud (TM), and (b) it would be an interesting exercise with Heroku, which I have been enjoying using a great deal.

It turns out to be pretty simple, and I've made a git repo available to make it even easier.  Just follow these steps:
  1. Clone my GitHub repository:
    git clone https://github.com/OddBloke/heroku-sentry.git
  2. Generate a secret key and set it as SENTRY_KEY in sentry.conf.py. See point 11 here for a good example of how to generate a good key.
  3. Install the Heroku Toolbelt.
  4. Sign up for a Heroku account.
  5. Log in to your Heroku account from the CLI:
    heroku login
  6. Navigate to your clone of the git repo and create a new Heroku app:
    heroku apps:create
    This command creates a heroku remote in the git repo, which we will use in a minute.
  7. Take the URL it spits out (which will be something like http://floating-earth-1234.herokuapp.com/) and set that as SENTRY_URL_PREFIX in sentry.conf.py.
  8. Commit your changes:
    git commit --all -m "Personalise."
  9. Push your repository up to Heroku:
    git push heroku master
    If all goes according to plan, this will cause Heroku to deploy, and you will see the output as it installs Sentry and all of its requirements. Finally, it will tell you that your app is deployed to Heroku, but we're not quite done.
  10. We need to configure a database for Sentry to use. We're going to use the new Heroku Postgres offering, which is currently in public beta: 
    heroku addons:add heroku-postgresql:dev
  11. The previous command will have given you a database name, something like HEROKU_POSTGRESQL_CHARCOAL. Tell Heroku to use this database: heroku pg:promote HEROKU_POSTGRESQL_CHARCOAL
  12. Finally, we need to run the Sentry setup script, create a default user and tell Sentry to use it: 
    heroku run sentry --config=sentry.conf.py upgrade
    heroku run sentry --config=sentry.conf.py createsuperuser
    heroku run sentry --config=sentry.conf.py repair --owner=JustCreatedSuperuser
    You should use the name of the super-user you create with the second command as the --owner argument to the third command.
Voila!  You should now be able to see Sentry by pointing your browser at the URL you used for SENTRY_URL_PREFIX earlier.

If you have any problems, you can check out your logs using heroku logs.

One final note: Heroku doesn't provide SMTP for you, so you'll also have to modify the SMTP settings to point at your own mail server (or play around with the Heroku add-ons that do provide it).

Friday 13 July 2012

User Fixtures in Django

At Glasses Direct, we are setting up an internal system which needs a simple web UI.  As we use Django for all of our HTTP needs, the admin interface was the obvious, quick solution.  All we need to do is write the requisite model, tie it in to the admin interface and we're done!  Right?  Wrong.

The above description is missing an important part of the puzzle: authentication.  Django's admin interface (sensibly) requires authentication by default.  However, this piece of our system will only be exposed internally, and we don't want to have to manage credentials for all of our internal users (as we are sadly lacking when it come to internal single sign-on).

The obvious course of action is to remove the authentication.  However, this seems to be easier said than done.  Firstly, there is no simple switch to disable authentication. Secondly, even were there, we wouldn't want everyone to have access to the full admin interface, largely because it would be confusing for the target users.  So we can't get rid of authentication.  What we really want is a default user.

A default user is easy enough, you can add it using a User fixture that looks something like this (the easiest way to do this is to create a User object and use the dumpdata management command):
[{"pk": 2,
  "model": "auth.user",
  "fields": {
    "username": "default",
    "is_staff": true,
    ...
  }}]
Voila! After running a syncdb, you'll have a user who can access the admin interface.  Unfortunately, they won't be able to do anything, because they don't have any permissions.  Let's fix that by adding some (again, easiest to do this using dumpdata):
[{"pk": 2,
  "model": "auth.user",
  "fields": {
    "username": "default",
    "is_staff": true,
    "user_permissions": [12, 13, 14],
    ...
  }}]
You can see here that we've granted this user three permissions.  The relevant entries will show up in the admin interface.  We're done!  Right?  Wrong.

Everything will seem to be proceeding happily, possibly for quite some time.  Then, in a few weeks or months, you'll add another model, or app or something and suddenly your default user will have permission to do really weird things.  The problem here is that Django will occasionally regenerate the primary keys of permissions (and other internal objects).  So what are we to do?  After a fair amount of swearing this afternoon, my colleague Ondrej pointed me in the direction of natural keys. With these, you can future-proof yourself against primary key oddities:
[{"pk": 2,
  "model": "auth.user",
  "fields": {
    "username": "default",
    "is_staff": true,
    "user_permissions": [
      ["add_mymodel", "myapp", "mymodel"],
      ["change_mymodel", "myapp", "mymodel"],
      ["delete_mymodel", "myapp", "mymodel"]
    ],
    ...
  }}]
As with the above examples, you should generate this output with dumpdata, passing in the --natural flag on the command line.

To conclude, we've looked at how we can use Django fixtures to give us a default user with a known username and password, with reliable, known permissions. Perfect!


N.B. One option for "auto-authentication" would be to use a middleware class that sets the user on the request to our default user, something along the lines of this.  This component is only meant to be a quick procedure fix, so we haven't taken the time to do that.