Testing

Last updated on 2026-10-07 | Edit this page

Overview

Questions

  • Why have unit tests?

Objectives

  • Self documenting Makefile
  • Use pytest for unit tests
  • Use coverage to check test coverage

Use the following commands if you want to save your work from the previous episode

BASH 

$ make clean
$ git status
$ git add .
$ git status
$ git commit -m "my work"

Now use the following to get files for this episode

BASH 

$ git switch -c my-04-branch remotes/origin/04-testing
$ git ls-files

which should yield

OUTPUT 

Makefile
books/abyss.txt
books/isles.txt
src/TeX/local.bib
src/TeX/report.tex
src/analysis/countwords.py
src/analysis/testzipf.py
src/plotscripts/plotcounts.py

This episode is primarily about testing, but first we will make the Makefile self documenting. Try

BASH 

$ make help

OUTPUT 

 help                 : Holy mackerel!  What is this?

BASH 

$ tail Makefile

OUTPUT 

	head -n 10 $< |awk '{print $$1, $$2}' > $@

.PHONY : clean
clean :
	rm -rf build

## help                 : Holy mackerel!  What is this?
.PHONY : help
help : Makefile
	@sed -n 's/^## / /p' $<
Challenge

Self Documenting Makefiles

Explain how the help block at the end of the Makefile works and how you could use it to document the targets build/report.pdf, build/results.tex, build/abyss.head, clean, and help.

The sed command simply prints each line in the Makefile that starts with “##”. Now we will edit the file to document each of those targets.

So far we have tested each incremental change of our code with something like

BASH 

$ make clean
$ make

While that takes less than three seconds to redo all the calculations and build the final document, our toy research project is a surrogate for projects with calculations and other steps that take hours if not days. For such a project one can use unit tests to test each chunk of code quickly in isolation.

Callout

Unit Tests

Google AI says:

Unit tests are automated pieces of code written to verify that a small, isolated part of a software application, usually a single function or method, behaves as expected.

Pytest

We will use the utility pytest to run unit tests, and we will put the testing code in the directory tests/. Here is the block required in the Makefile:

## test                : Run pytest on tests/
.PHONY : test
test :
	pytest tests/

And here is the code for tests/test_countwords.py

PYTHON 

"""test_countwords.py

debug with
$py.test --pdb tests/test_countwords.py
"""
import pytest
import pathlib

import analysis.countwords


@pytest.fixture(scope="function")
def shared_count_file(tmp_path_factory):
    # 1. Create a temporary directory using the factory
    temp_dir = tmp_path_factory.mktemp("shared_data")

    # 2. Create the file inside that directory
    count_file = temp_dir / "word_counts"
    count_file.write_text('{"status": "ready"}')

    return count_file


def test_word_count(shared_count_file):
    """Check reading books/abyss.txt

    """
    in_path = 'books/abyss.txt'
    analysis.countwords.word_count(in_path, shared_count_file)


def test_main(shared_count_file: pathlib.PosixPath):
    in_path = 'books/abyss.txt'
    analysis.countwords.main([in_path, f'{shared_count_file}'])

After making those changes to the project we can test testing with

BASH 

$ make test
pytest tests/
================================ test session starts ================================
platform linux -- Python 3.12.12, pytest-8.3.5, pluggy-1.5.0
rootdir: /mnt/precious/home/andy_nix/projects/SIAMDS27/learner_setup
plugins: anyio-4.9.0, cov-6.1.0, dependency-0.6.0
collected 2 items

tests/test_countwords.py ..                                                   [100%]

================================= 2 passed in 0.10s =================================

Next we introduce the utility coverage that reports on the sections of code that are covered by a collection of unit tests.

Callout

coverage.py

Google AI says:

Coverage.py is the standard, authoritative code coverage measurement tool for Python. […] it monitors your program during execution to track which lines of code have been run and which have not. It is primarily used to gauge the effectiveness and completeness of your test suites.

Here is a block for the Makefile

## coverage            : make test coverage report in build/htmlcov/
.PHONY : coverage
coverage :
	rm -rf .coverage build/htmlcov
	coverage run --source tests,src/analysis,src/plotscripts -m pytest tests
	coverage html -d build/htmlcov
	@echo To view: "firefox build/htmlcov/index.html & "

Let’s test that with

BASH 

$ make coverage

OUTPUT 

rm -rf .coverage build/htmlcovcoverage run --source tests,src/analysis,src/plotscripts -m pytest tests
================================ test session starts ================================
platform linux -- Python 3.12.12, pytest-8.3.5, pluggy-1.5.0
rootdir: /mnt/precious/home/andy_nix/projects/SIAMDS27/learner_setup
plugins: anyio-4.9.0, cov-6.1.0, dependency-0.6.0
collected 2 items

tests/test_countwords.py ..                                                   [100%]

================================= 2 passed in 0.16s =================================
coverage html -d build/htmlcov
Wrote HTML report to build/htmlcov/index.html
To view: firefox build/htmlcov/index.html &

After looking at the result in a browser, we see that we should write tests for testzipf.py and plotcounts.py. We have written those and included them in the git branch 05-standards for the next episode.

Key Points
  • Use unit tests for finding bugs quickly, and checking changes.
  • For Python code pytest works
  • Use a coverage tool for finding code that the testing suit missed
  • For Python code coverage works