chat485

pytest Tutorial

pytest is a unit test utility that makes it easy to run unit tests. It is compatible with the Python unittest framework.

Quick start

TL;DR: for debugging, you probably want maximum verbose output (-vv), application stdout to pass through to pytest stdout (-s), and stop after the first error (-x).

$ pytest -vvsx

Step-by-step

This tutorial uses Project 1 as an example. It assumes you have some code and a directory called tests/.

$ pwd
/Users/awdeorio/src/eecs485/p1-chat485-static
$ ls
bin                        env             pyproject.toml
chat485                    handcoded_html  requirements.txt
chat485generator           hello           tests
chat485generator.egg-info  hello_css

You have installed and activated your Python virtual environment (instructions) and installed pytest (pip install pytest if needed). Your version might be different.

$ pytest --version
pytest 9.0.3

By default, pytest will search for tests and run them all.

$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.12.0, pytest-9.0.3, pluggy-1.6.0
rootdir: /Users/awdeorio/src/eecs485/p1-chat485-static
collected 8 items

tests/test_chat485generator_public.py::test_help PASSED                 [ 12%]
tests/test_chat485generator_public.py::test_output PASSED               [ 25%]
tests/test_chat485generator_public.py::test_hello PASSED                [ 37%]
tests/test_chat485generator_public.py::test_hello_css PASSED            [ 50%]
tests/test_handcoded_html_public.py::test_html PASSED                   [ 62%]
tests/test_style_public.py::test_ruff_check PASSED                      [ 75%]
tests/test_style_public.py::test_ruff_format PASSED                     [ 87%]
tests/test_template_html_public.py::test_html PASSED                    [100%]

============================== 8 passed in 2.34s ==============================

Running pytest with the additional -s flag passes output from the program (stdout) to the terminal, e.g., print() statements. This is handy when a test shells out to a tool like ruff and you want to see its output.

$ pytest -vs tests/test_style_public.py::test_ruff_check

Stop after the first failure.

$ pytest -x

Debugging options

Rerun only the tests that failed last time

$ pytest --last-failed

Automatically start a debugger when a failure occurs. The -s is required for PDB+. Don’t forget to add import pdbp.

$ pytest --pdb -s

Start a debugger on the first line of the test.

$ pytest --trace -s

Start PDB+ from a breakpoint().

$ pytest -s

Debugging with VSCode

If you’re using VSCode, you can leverage its visual debugger to debug your pytest tests. Follow these steps to set it up:

Open your project in Visual Studio Code.
Ensure that you have the Python extension installed and enabled.
Set breakpoints in your test files by clicking in the gutter area (next to the line numbers) where you want the debugger to pause.
Open the Command Palette (Ctrl+Shift+P or Cmd+Shift+P) and search for "Python: Configure Tests".
Select the option to configure your tests and choose "pytest" as the testing framework.
Select the "test" directory where your python tests live.
Once configured, a "Run Test" button will appear next to your test functions and classes in the editor.
Click the "Run Test" button next to the test you want to debug.
The debugger will pause execution at the breakpoints you’ve set, allowing you to inspect variables, evaluate expressions, and step through your code. 
Use the debug toolbar at the top of the editor to control the session (e.g., step into, step over, step out, continue).

By using these tools, you can streamline debugging and gain better insights into your how your code is performing on spefic tests.

Acknowledgments

Original document written by Andrew DeOrio awdeorio@umich.edu.

This document is licensed under a Creative Commons Attribution-NonCommercial 4.0 License. You’re free to copy and share this document, but not to sell it. You may not share source code provided with this document.