RSS
The Testing Goat

Obey the Testing Goat!

TDD for the Web, with Python, Selenium, Django, JavaScript and pals...

Testing Django Class-Based (Generic) Views

Fri 27 September 2013
By Harry

This blog post is a first rough draft of a planned appendix to my book. It follows on from Chapter 9, which is all about forms and validation. You can take a look at it here

If you want to check out the code to have a play with the examples, you’ll find them on GitHub under the chapter_09 branch and the appendix_II branch

As you’ll see, the content starts out sounding a lot like a "proper" chapter for a book, and turns into more of a blog post and request for comments. Please do let me know what you think!

Update 2013-10-05

There’s been some interesting discussion with minds much greater than my own, such as those of Messrs Russell Keith-MaGee and Trey Hunner, which, for one reason or another, has taken place as line comments on github. Do check it out:

https://github.com/hjwp/www.obeythetestinggoat.com/commit/c5857b416e404c9af5a9205fbf1b10dccaf8161d

Update 2013-10-16

My basic conclusion for how to test CBGVs is now: make sure you have lots of short, single-assertion tests for your views, and it will be easy to adjust to using class-based views from function-based ones, and vice-versa. Cf the re-cap at the end of chapter 11:

http://chimera.labs.oreilly.com/books/1234000000754/ch11.html#_using_the_existing_lists_item_form_in_the_list_view (scroll right to the end)

And the updated version of this post / appendix:

http://chimera.labs.oreilly.com/books/1234000000754/apb.html


Appendix II: Django Class-based views

This appendix follows on from Chapter 9, in which we implemented Django forms for validation, and refactored our views. By the end of that chapter, our views were still using functions.

The new shiny in the Django world, however, is class-based views. In this chapter, we’ll refactor our application to use them instead of view functions. More specifically, we’ll have a go at using class-based generic views.

Warning

this appendix is currently more of a blog post / request for comments than a final appendix. Use at your own peril.

Class-based generic views

It’s worth making a distinction at this point, between class-based views and class-based generic views. Class-based views are just another way of defining view functions. They make few assumptions about what your views will do, and they offer one major benefit over view functions, which is that they can be subclassed. This comes, arguably, at the expense of being less readable than traditional function-based views. The main use case for plain class-based views is when you have several views that re-use the same logic. We want to obey the DRY principle. With function-based views, you would use helper functions or decorators. The theory is that using a class structure may give you a more elegant solution.

Class-based generic views are class-based views that attempt to provide ready-made solutions to common use cases: fetching an object from the database and passing it to a template, fetching a list of objects, saving user input from a POST request using a ModelForm, and so on. These sound very much like our use cases, but as we’ll soon see, the devil is in the detail.

I should say at this point that I’ve not used either kind of class-based views much. I can definitely see the sense in them, and there are potentially many use cases in Django apps where CBGVs would fit in perfectly. However, as soon as your use case is slightly outside the basics — as soon as you have more than one model you want to use, for example, I’ve found that using class-based views becomes much more complicated, and you end up with code that’s harder to read than a classic view function.

Still, because we’re forced to use a lot of the customisation options for class-based views, implementing them in this case can teach us a lot about how they work, and how we can unit tests them.

My hope is that the same unit tests we use for function-based views should work just as well for class-based views. Let’s see how we get on.

The home page as a FormView

Our home page just displays a form on a template:

def home_page(request):
    return render(request, 'home.html', {'form': ItemForm()})

Looking through the options, Django has a generic view called FormView — let’s see how that goes:

lists/views.py (ch21l001)

from django.views.generic import FormView
[...]

class HomePageView(FormView):
    template_name = 'home.html'
    form_class = ItemForm

We tell it what template we want to use, and which form. Then, we just need to update urls.py, replacing the line that used to say lists.views.home_page:

superlists/urls.py (ch21l002)

    url(r'^$', HomePageView.as_view(), name='home'),

And the tests all check out! That was easy..

$ python3 manage.py test lists
Creating test database for alias 'default'...
......................
 ---------------------------------------------------------------------
Ran 22 tests in 0.134s

OK
Destroying test database for alias 'default'...

$ python3 manage.py test functional_tests
Creating test database for alias 'default'...
....
 ---------------------------------------------------------------------
Ran 4 tests in 15.160s

OK
Destroying test database for alias 'default'...

So far so good. We’ve replaced a 1-line view function with a 2-line class, but it’s still very readable. This would be a good time for a commit…

Using form_valid to customise a CreateView

Next we have a crack at the view we use to create a brand new list, currently the new_list function. Looking through the possible CBGVs, we probably want a CreateView, and we know we’re using the ItemForm class, so let’s see how we get on with them, and whether the tests will help us:

lists/views.py

class NewListView(CreateView):
    form_class = ItemForm

def new_list(request):
    form = ItemForm(data=request.POST)
    if form.is_valid():
        list = List.objects.create()
        Item.objects.create(text=request.POST['text'], list=list)
        return redirect(list)
    else:
        return render(request, 'home.html', {"form": form})

I’m going to leave the old view function in views.py, so that we can copy code across from it. We can delete it once everything is working. It’s harmless as soon as we switch over the URL mappings, this time in:

lists/urls.py

    url(r'^new$', NewListView.as_view(), name='new_list'),

Now running the tests gives 3 errors:

$ python3 manage.py test lists
Creating test database for alias 'default'...
...................EEE
======================================================================
ERROR: test_redirects_after_POST (lists.tests.test_views.NewListTest)
 ---------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/harry/Dropbox/book/source/appendix_II/superlists/lists/tests/test_views.py", line 33, in test_redirects_after_POST
    data={'text': 'A new list item'}
    [...]
  File "/usr/local/lib/python3.3/dist-packages/django/forms/models.py", line 370, in save
    fail_message, commit, construct=False)
  File "/usr/local/lib/python3.3/dist-packages/django/forms/models.py", line 87, in save_instance
    instance.save()
  File "/home/harry/Dropbox/book/source/appendix_II/superlists/lists/models.py", line 26, in save
    self.full_clean()
  File "/usr/local/lib/python3.3/dist-packages/django/db/models/base.py", line 926, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'list': ['This field cannot be null.']}

======================================================================
ERROR: test_saving_a_POST_request (lists.tests.test_views.NewListTest)
 ---------------------------------------------------------------------
[...]
django.core.exceptions.ValidationError: {'list': ['This field cannot be null.']}

======================================================================
ERROR: test_validation_errors_sent_back_to_home_page_template (lists.tests.test_views.NewListTest)
 ---------------------------------------------------------------------
[...]
django.template.base.TemplateDoesNotExist: No template names provided

 ---------------------------------------------------------------------
Ran 22 tests in 0.114s

FAILED (errors=3)
Destroying test database for alias 'default'...

TODO: talk through decoding traceback.

Let’s start with the third — maybe we can just add the template?

lists/views.py

class NewListView(CreateView):
    form_class = ItemForm
    template_name = 'home.html'

That gets us down to just two failures. They’re both to do with dealing with valid POST requests. CBGVs that deal with forms want you to put any custom code for valid forms in a method called form_valid. We can just copy across some of the code from the old view function:

lists/views.py

class NewListView(CreateView):
    template_name = 'home.html'
    form_class = ItemForm

    def form_valid(self, form):
        list = List.objects.create()
        Item.objects.create(text=form.cleaned_data['text'], list=list)
        return redirect(list)

That gets us a pass!

$ python3 manage.py test lists
Ran 22 tests in 0.117s
OK
$ python3 manage.py test functional_tests
Ran 4 tests in 15.157s
OK

And we can even save two lines (DRY) by taking advantage of the real point of CBVs: inheritance!

lists/views.py

class NewListView(CreateView, HomePageView):

    def form_valid(self, form):
        list = List.objects.create()
        Item.objects.create(text=form.cleaned_data['text'], list=list)
        return redirect('/lists/%d/' % (list.id,))

And all the tests still pass.

How does it compare to the old version? I’d say that’s not bad. We save some boilerplate code, and the view is still fairly legible. So far, I’d say we’ve got one point for CBGVs, and one draw.

A more complex view to handle both viewing and adding to a list

This took me several attempts. And I have to say that, although the tests told me when I got it right, they didn’t really help me to figure out the steps to get there… Mostly it was just trial and error, hacking about in functions like get_context_data, get_form_kwargs and so on.

One thing I did do which improved my codebase was to add a new unit test:

lists/tests/test_views.py

class ListViewTest(TestCase):
    [...]

    def test_list_view_displays_form_for_existing_lists(self):
        correct_list = List.objects.create()
        response = self.client.get('/lists/%d/' % (correct_list.id,))
        self.assertIsInstance(response.context['form'], ExistingListItemForm)

It’s another good example of the "each test should test one thing" heuristic: that check on the form class could very easily have been tacked onto the end of a different test, but having it separate means I’m immediately told exactly what’s wrong, rather than potentially having the error masked by an earlier failure.

TODO: consider moving this test into ch. 9?

Anyway, after much hacking and swearing, this is the solution I eventually got to work:

lists/views.py

class ViewAndAddToList(CreateView, SingleObjectMixin):
    template_name = 'list.html'
    model = List
    form_class = ExistingListItemForm

    def get_form(self, form_class):
        self.object = self.get_object()
        if self.request.method == 'POST':
            data={
                'text': self.request.POST['text'],
                'list': self.object.id
            }
        else:
            data = None
        return form_class(data=data)

I also had to add a get_absolute_url on the Item class:

(I did try to use get_form_kwargs instead of get_form, but it didn’t want to work for me. Perhaps some CBGV expert out there has a neater solution??)

lists/models.py

class Item(models.Model):
    [...]

    def get_absolute_url(self):
        return self.list.get_absolute_url()
Compare old and new

Let’s see the old version for comparison?

def view_list(request, list_id):
    list = List.objects.get(id=list_id)

    if request.method == 'POST':
        form = ExistingListItemForm(data={
            'text': request.POST['text'],
            'list': list.id
        })
        if form.is_valid():
            form.save()
            return redirect(list)
    else:
        form = ExistingListItemForm()

    return render(request, 'list.html', {'list': list, "form": form})

Not a great improvement. Same number of lines of code, 15. If anything, the function version is better because it has one more line of whitespace. And it’s definitely more readable.

Best practices for unit testing CBGVs?

As I was working through this, I felt like my "unit" tests were sometimes a little too high-level. They told me whether I was getting things right or wrong, but they didn’t offer many clues on exactly how to fix things.

I occasionally wondered whether there might be some mileage in a test that was closer to the implementation — something like this:

def test_as_cbv(self):
    our_list = List.objects.create()
    view = ViewAndAddToList()
    view.kwargs = dict(pk=our_list.id)
    self.assertEqual(view.get_object(), our_list)

But the problem is that it requires a lot of knowledge of the internals of Django CBVs to be able to do the right test setup for these kinds of tests. And you still end up getting very confused by the complex inheritance hierarchy.

I’d be interested to hear how other people out there are testing their CBVs?

Comments

comments powered by Disqus
Read the book

I'm writing a book all about TDD and Web programming. Read the draft and let me know what you think!

Reviews & Testimonials

"Hands down the best teaching book I've ever read""Even the first 4 chapters were worth the money""Oh my gosh! This book is outstanding""The testing goat is my new friend"Read more...

Resources

A selection of links and videos about TDD, not necessarily all mine, eg this tutorial at PyCon 2013, how to motivate coworkers to write unit tests, thoughts on Django's test tools, London-style TDD and more.

Old TDD / Django Tutorial

This is my old TDD tutorial, which follows along with the official Django tutorial, but with full TDD. It badly needs updating. Read the book instead!

Save the Testing Goat Campaign

The campaign page, preserved for history, which led to the glorious presence of the Testing Goat on the front of the book.