Wednesday, 23 January 2013

Git Tip: Remove .pyc Files Automatically

N.B. I've published a follow-up to this here, which includes a way to completely avoid this problem in the first place.

Recently, I've found myself increasingly caught out by stale .pyc files in our project. When I change from our mainline branch to a story branch (or vice-versa), I often find myself with inexplicable test failures because Python is using the .pyc files for no-longer-current code.

Luckily, it's pretty easy to fix this in git, using hooks, specifically the post-checkout hook. To do that, add the following to .git/hooks/post-checkout, and make the file executable:

#!/bin/bash
find $(git rev-parse --show-cdup) -name "*.pyc" -delete

Now, every time you checkout a new branch, all the .pyc files will be cleared out of your git branch.

9 comments:

  1. That's a bit of a big hammer if you have a large project, isn't it?

    Something like the following would target only the files changed:
    #!/bin/bash
    OLD_HEAD="$1"
    NEW_HEAD="$2"
    FLAG="$3"

    olddir="$PWD"
    d=$(git rev-parse --show-cdup)
    if [ -n "$d" ]; then
    cd $d
    fi

    IFS='
    '

    for file in $(git log "$OLD_HEAD".."$NEW_HEAD" --pretty="format:" --name-only|grep '.py$'|sort -u); do
    rm -f "${file}c"
    done

    cd "$olddir"

    ReplyDelete
    Replies
    1. That really depends on your machine; I haven't noticed a problem with ~1000 Python files in our project.

      And are you sure that the extra git calls and shell time won't out-weigh the benefits to be gained with your smaller hammer? ;)

      Delete
    2. Some projects (like ours) have directories with LOTS of images/directories. The find takes ages and eats all the diskio.

      Delete
  2. Why not just adding a .gitignore file in the root of your repo ? Compiled python files should not even get into your repo in first place.

    https://help.github.com/articles/ignoring-files

    Just saying.

    ReplyDelete
    Replies
    1. You completely miss the point. This is about removing stale .pyc files from the filesystem, where Python picks them up, and not about removing them from version control. I'm not enough of a sadist to not have .pyc files in my .gitignore.

      Delete
  3. I recommend setting $PYTHONDONTWRITEBYTECODE to 1 (See http://docs.python.org/2/using/cmdline.html#envvar-PYTHONDONTWRITEBYTECODE).

    ReplyDelete
  4. This is exactly the kind of thing that git clean is for.

    ReplyDelete
    Replies
    1. Any sane person will have pyc files ignored, meaning that you would have to clean all your ignored files. I definitely wouldn't want to do this automatically, nor recommend that people do it automatically.

      Delete
    2. Exactly. If you run git clean then also your local config files will be purged. And you don't want to do that I presume. It is a pretty big mistake to store configuration files in the repository, so it is usually ignored as well so git clean would delete that for you. you don't want to reset your configuration every time you switch branches. Thanks for the author btw. This hook is exactly what I needed.

      Delete