Automated tests: examples within the shifts app

Here we'll have a look at some existing tests and try to explain them, hoping to make it easier to get started writing tests for people who are not used to it.

We'll start with tests relative to the FrozenStatusService. They are in tapir.shifts.tests.test_FrozenStatusService.TestFrozenStatusService. Since this file may change after this page is written, here are the links to the service file and to the test file as they are at the time of writing. Concrete examples will be copied from that version.

Let's go through the test file from top to bottom and stop at interesting lines.

Base class

class TestFrozenStatusService(TapirFactoryTestBase):

Most of our test classes inherit from tapir.utils.tests_utils.TapirFactoryTestBase, which itself inherits from django.test.testcases.TestCase. TapirFactoryTestBase adds a few helper function to login with as a Vorstand or Member office user. It also sets the randomization seed for all Factories, so that tests generate the exact same test objects on every run.

Mocking time

    NOW = datetime.datetime(year=2020, month=1, day=30, hour=16, minute=37)

    def setUp(self) -> None:
        mock_timezone_now(self, self.NOW)

setUp() is run automatically before each test. Since the FrozenStatusService depends on the current time, we "mock" the current time using mock_timezone_now. See time-independance. That means that we replace the time function that Django would otherwise use by a temporary one that always returns the same value. This makes our tests more stable.

First simple test

def test_shouldFreezeMember_memberAlreadyFrozen_returnsFalse(self):
        shift_user_data = ShiftUserData()
        shift_user_data.attendance_mode = ShiftAttendanceMode.FROZEN
        self.assertFalse(FrozenStatusService.should_freeze_member(shift_user_data))

We start by looking at FrozenStatusService.should_freeze_member, that tells us if a member should, well, get frozen. If a member is already frozen, they should not get re-frozen: that would create a log entry, send them an email...

Test method name

The function name follows our naming convention:

• test every test function starts with "test
• shouldFreezeMember then comes the name of the element that we are testing. Here we put the method name, the name of the service is already specified in the test file name and the test class name.
• memberAlreadyFrozen this is the "case" that we are testing, that is the context in which we are testing
• returnsFalse this is the expected result.

Test content

This test follows the usual pattern of "given, when, then":

• Given a ShiftUserData that is already frozen, this is the "state of the world" that the test runs in
• When we call the should_freeze_member function, this is the action that we perform
• Then the result should be False, this is the expected result

The "given" part is usually done by mocking functions and calling factories: we create the objects that let us define the starting situation.

The "when" part is usually a function call or a client request (there is no client request in this test file, we'll look at a separate one after).

The "then" part is usually a sequence of self.assert.... Since we inherit from TestCase we have access to a many variants of self.assert. Using the appropriate one produces more readable messages when the test fails.

A slightly more complex test

    @patch.object(ShiftExpectationService, "is_member_expected_to_do_shifts")
    def test_shouldFreezeMember_memberIsNotExpectedToDoShifts_returnsFalse(
        self, mock_is_member_expected_to_do_shifts: Mock
    ):
        shift_user_data = ShiftUserData()
        shift_user_data.attendance_mode = ShiftAttendanceMode.REGULAR
        mock_is_member_expected_to_do_shifts.return_value = False
        self.assertFalse(FrozenStatusService.should_freeze_member(shift_user_data))
        mock_is_member_expected_to_do_shifts.assert_called_once_with(
            shift_user_data, self.NOW.date()
        )

This time we have a test we actual mocking. That's because we now rely on another service that is outside FrozenStatusService:

class FrozenStatusService:
    @classmethod
    def should_freeze_member(cls, shift_user_data: ShiftUserData) -> bool:
        if shift_user_data.attendance_mode == ShiftAttendanceMode.FROZEN:
            return False

        if not ShiftExpectationService.is_member_expected_to_do_shifts(
            shift_user_data, timezone.now().date()
        ):
            return False
        # More checks
        return True

In our first test, we had should_freeze_member return on the first check, so the setup was minimal. Now we want to go up to the second check, so we setup our ShiftUserData object to pass the first check with shift_user_data.attendance_mode = ShiftAttendanceMode.REGULAR.

Why use mocks?

Then is the more interesting part. Members who are not expected to do shifts (for example because they are exempted) should not be frozen. This is determined by ShiftExpectationService.is_member_expected_to_do_shifts. Instead of mocking this function, we could define our shift_user_data so that is_member_expected_to_do_shifts returns True, but that would have two disadvantages:

• If the requirements inside is_member_expected_to_do_shifts, our test could fail even the part that we are currently testing didn't change. We want as few tests as possible to fail, so that it is easier to pinpoint which part of the code is failing.
• In the "given" part of our test, we would need to define more parameters of our shift_user_data. This would make it less clears which parts are actually relevant to the test. Instead, we mock is_member_expected_to_do_shifts: we replace it with a mock object. We define inside the test how we want that mock object to behave.

How to mock

We declare with the @patch.object decorator that we want to mock the function. This adds a parameter to our test function (in this case called mock_is_member_expected_to_do_shifts but only the position is important). Then, with mock_is_member_expected_to_do_shifts.return_value = False, we say that is_member_expected_to_do_shifts will always return False, thus making this test independant of the internal working of is_member_expected_to_do_shifts.

The assert is then done as usual.

The last line may not be very intuitive but it is still important: mock.assert_called_once_with(...). Even though we told our mock objects to return False regardless of what parameters we call it with, it still keeps a history of how many times it gets called, and with which parameter. Thus, with assert_called_once_with(...), we can make sure that the function is called as expected. For example, if we replaced the first check with ... == ShiftAttendanceMode.FLYING and didn't call assert_called_once_with, the test would still pass, but we would not have tested the lines that we actually want to test.

There are many variants of assert_called_... that let you check the number of calls, the parameters of each call, the order...

Mock parameter order

Let's peek at the following test:

    @patch.object(ShiftExpectationService, "is_member_expected_to_do_shifts")
    @patch.object(FrozenStatusService, "_is_member_below_threshold_since_long_enough")
    def test_shouldFreezeMember_memberNotBelowThreshold_returnsFalse(
        self,
        mock_is_member_below_threshold_since_long_enough: Mock,
        mock_is_member_expected_to_do_shifts: Mock,
    ):

You'll notice that the parameter order is reversed relative to the decorators: @patch("is_member_expected_to_do_shifts") comes first, but the corresponding mock object is the second parameter. Naming the parameters in full helps avoid errors.

Testing views

Testing views is similar to testing services, it just has have few specificities. Let's look at tapir.shifts.tests.test_slot_template_create.TestSlotTemplateCreate.test_creating_a_slot_template_also_creates_the_slot_in_future_shifts.

Logging in

def test_creating_a_slot_template_also_creates_the_slot_in_future_shifts(self):
        self.login_as_member_office_user()

When testing views, we reproduce as much as possible the behaviour we would have when visiting the view from a browser. In most cases, we need to login to visit a view. TapirFactoryTestBase gives us a few shortcuts to login as a member with specific access rights, in this example self.login_as_member_office_user().

The test client

A test client is created for each test in tapir.utils.tests_utils.TapirFactoryTestBase.setUp. This client is like the browser we'll be testing with: it can visit URLs and present us with the result.

We can do POST or GET requests. In our example it's a POST:

response = self.client.post(
            reverse(self.SLOT_TEMPLATE_CREATE_VIEW, args=[shift_template.id]),
            {
                "name": self.SLOT_TEMPLATE_NAME,
                "required_capabilities": required_capabilities,
                "warnings": warnings,
                "check_update_future_shifts": True,
            },
            follow=True,
        )

Let's go through it step by step.

reverse gives us the URL corresponding to a view name, as defined in urls.py files.

The second parameter to self.client.post is the post data. That's the data that the user would have filled in the corresponding form. For GET requests, there is no post data.

Checking the response

After visiting a view, we almost always check for the status code first. In our example, we expect the call to succeed, so we expect 200: self.assertEqual(response.status_code, 200).

If the call should have modified database objects, you can fetch them or, if you already had a reference to them like in our example, use refresh_from_db(). Then do some self.assertEqual as usual.

If you need to check the content of HTML return by the visit, you can use response.content.decode() to get the page content as a string. Sometimes though, check for HTML can be tedious. It is also very brittle: a change in the page layout could lead the test to fail, even though the logic is still fine. It can be more convenient to check the context data, as returned by the views get_context_data(), with response.context["..."].