Fixing nose.tools pylint errors

What is nose.tools?

If you've engaged in any test-driven Python development, you've probably considered using nose as your test runner. One of my favourite parts of nose (among its many excellent parts) is the nose.tools module. This not only contains a number of helpful tools (documented here), but also all of the assertion methods on unittest.TestCase converted to PEP-8 compatible functions (e.g. self.assertEqual becomes assert_equal).

These assertion functions are useful for a few reasons: readability, line length, PEP-8 compliance. Most importantly, though, is the fact that nose allows you to write test functions and test classes which don't subclass unittest.TestCase. The only way to access these assertions, therefore, is through nose.tools.

The assertion functions in nose.tools are generated at runtime from unittest.TestCase and put in to the nose.tools namespace. This has a couple of consequences: firstly, any new assertions in the standard library will immediately appear in nose.tools (a word of warning: as nose doesn't implement the assertions, you need to pay attention to stdlib unittest assertion changes). Secondly, this means that they don't appear anywhere in the source code if you go looking for them. This will become relevant shortly.

What is pylint?

If you've engaged in any team Python development, you've probably considered pylint to help you maintain code quality. pylint uses static analysis to find problems in your code. Static analysis just means that it looks at your code as text, rather than executing it (which would be dynamic analysis). The problems it finds range from bad syntax to poor style; it can even work out when you've tried to import a non-existent symbol from another module.

What's the problem?

This last feature is really useful most of the time; it has saved me from pushing broken code innumerable times. However, it does cause one major headache: because the nose.tools source code doesn't contain the PEP-8 assertion functions (remember, they're dynamically generated!), pylint's static analysis can't find them; every single time you import one of them you get a pylint error (E0611, to be precise). If you're using nose as part of your continuous integration to check your pylint errors, this can be a major source of pain. Luckily, pylint has a solution: disabling checks.

Fixing with disabled checks

pylint allows you to disable checks in a couple of ways: you can disable them globally (i.e. none of these warnings should ever be displayed), or based on scope (i.e. this module, class, function or line should never have warnings displayed). So, for example, an import line could look like:

from nose.tools import assert_equal  # pylint: disable=E0611

Problem solved! Your pylint run will no longer flag this line up as problematic. There are a few issues with disabling checks, though. Firstly, you have to remember to do it. Secondly, it crufts up the screen when you're trying to read, and gives you more information that you need to process. Finally, and often most crucially, you can accidentally change the scope of your disabling. Consider the following lines at the top of a module:

# pylint: disable=E0611
from nose.tools import assert_equal, assert_items_equal, assert_greater

This does solves the warning (and avoids violating PEP-8's line limit), but it has an unintentional consequence: E0611 is now disabled for the entire file. pylint will no longer catch this obvious error!

# pylint: disable=E0611
from nose.tools import assert_equal, assert_items_equal, assert_greater
from mcok import Mock

Fixing with a pylint plugin

Luckily, there is a better way of solving this problem than disabling individual checks everywhere. pylint provides a plugin interface so that people can implement their own checks and plugins. Using this, we can also modify the way that the static analysis looks at the nose.tools module. In this pylint plugin, I have implemented a function which transforms the abstract syntax tree which pylint uses when looking at nose.tools to inject all of the functions which are generated at runtime:

from nose import tools

function_template = """
def {}(*args, **kwargs):
    pass
"""

def nose_transform(module):
    funcs = ''.join(function_template.format(func_name)
                    for func_name in tools.__all__)
    fake = ASTNGBuilder(MANAGER).string_build(funcs)

    for func_name in tools.__all__:
        if func_name not in module.locals:
            module.locals[func_name] = fake[func_name]

Just follow the installation instructions in the gist, and all of your pylint nose.tools problems will disappear!

Using inotifywait To Run Your Tests When Your Code Changes

Recently, I tweeted about my desire for a change-aware test runner:

I would like a test runner that detects when I have changed tests/code, and runs the relevant tests.
— Daniel Watkins (@Odd_Bloke) January 31, 2013

I haven't quite come up with that (though I do now have a half-finished blog post prognosticating on it), but I do have a solution which covers some of the bases: I determine what tests I want to run, and the test harness runs them whenever my code changes.

This "test harness" is actually a bash snippet which relies on inotifywait, a command-line program which blocks until it detects an inotify event on the files it is watching, and is incredibly simple:

while inotifywait -e close_write -r $CODE_DIRS --exclude=".*sw[px]"; do
    $TEST_COMMAND
done

This while loop is simple. When the inotifywait command stops blocking (and with exit code 0, which it will unless something unusual has happened to the files you're watching), we run $TEST_COMMAND. You can put whatever you want there, so you could limit the number of tests you run that way.

Now let's break that inotifywait call down. $CODE_DIRS can be any number of directories or files that you would like inotifywait to watch. -e narrows down the events that we should unblock on; we don't want to run the tests every time we open a new file. close_write triggers on (to quote the man page):

file or directory closed, after being opened in writable mode

vim triggers this when I save, so it works for me. You might also want to listen for modify (file or directory contents were written), move (file or directory moved to or from watched directory) and create (file or directory created within watched directory). Multiple -e options are comma-separated.

-r, as with many commands, tells inotifywait to watch directories recursively. This will mean that all directories under those that you specify will trigger your tests. I normally just run this command pointing at the top-level directory of my project. It's worth noting that -r applies to all directories that you pass.

Finally, --exclude takes a regular expression of files to exclude from your watch. As a vim user who hasn't configured swap files to be stored out of my tree, I want to ignore them (otherwise my tests run every time I open a file because vim writes out a swap file).

A quick disclaimer: as this uses inotify, this will only work on platforms that support it (which, I think, is only Linux). Mac users might want to examine this StackOverflow question. Windows users might find this answer useful.

UPDATE: +Murali Suriar has helpfully pointed me to kqwait on IRC, which will help out any Mac/BSD users.

Django, Meet Nose

At Glasses Direct, we use nose to run our tests, as it gives us all sorts of nice things like test functions and XUnit-compatible output (which Jenkins loves).  As the majority of our projects use Django, we use django-nose to integrate nose into our Django projects.  This gives us all of that nose loveliness when using manage.py test.
The process for setting this up is simple:

1. Install django-nose and nose however you normally do (I would use pip install django_nose nose).
2. Add 'django_nose' to INSTALLED_APPS.
3. Add TEST_RUNNER = 'django_nose.NoseTestSuiteRunner' to your settings.
4. Run manage.py test -s to check that nose is being used (the standard manage.py test doesn't have this option).
5. Read manage.py test -h and nose docs to learn about the exciting things you can do with nose.

I've added a few more nose-related things to my blogging backlog, so I'll get to those eventually.