p1-insta485-static
pytest Tutorial
pytest is a unit test utility that makes it easy to run unit tests. It is compatible with the Python unittest framework.
Quick start
TL;DR: for debugging, you probably want maximum verbose output (-vv), application stdout to pass through to pytest stdout (-s), and stop after the first error (-x).
$ pytest -vvsx
Step-by-step
This tutorial uses Project 1 as an example. It assumes you have some code and a directory called tests/.
$ pwd
/Users/awdeorio/src/eecs485/p1-insta485-static
$ ls
bin hello html insta485generator pyproject.toml
env hello_css insta485 insta485generator.egg-info tests
You have installed and activated your Python virtual environment (instructions) and installed pytest (pip install pytest if needed). Your version might be different.
$ pytest --version
This is pytest version 4.1.0, imported from /Users/awdeorio/src/eecs485/p1-insta485-static/env/lib/python3.7/site-packages/pytest.py
By default, pytest will search for tests and run them all.
$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /Users/awdeorio/src/eecs485/p1-insta485-static
collected 35 items
tests/test_handcoded_html.py . [ 2%]
tests/test_handcoded_index.py ...... [ 20%]
tests/test_handcoded_user.py ...... [ 37%]
tests/test_insta485generator_hello.py . [ 40%]
tests/test_insta485generator_hello_css.py . [ 42%]
tests/test_style.py ... [ 51%]
tests/test_template_followers.py ..... [ 65%]
tests/test_template_following.py ..... [ 80%]
tests/test_template_html.py . [ 82%]
tests/test_template_index.py ...... [100%]
============================= 35 passed in 19.06s ==============================
Run all unit tests in one file.
$ pytest tests/test_template_followers.py
Run one unit test. You can find the exact name of a testcase by running them all with the verbose flag -v.
tests/test_template_followers.pyis a filetest_awdeorio_followersis one unit test, which is one function.$ pytest tests/test_template_followers.py::test_awdeorio_followers
Common options
Print the name of each individual unit test with -v or --verbose. Adding -vv increases the output.
$ pytest -vv
============================= test session starts ==============================
platform darwin -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /Users/awdeorio/src/eecs485/p1-insta485-static/env/bin/python3
cachedir: .pytest_cache
rootdir: /Users/awdeorio/src/eecs485/p1-insta485-static
collected 35 items
tests/test_handcoded_html.py::test_html PASSED [ 2%]
tests/test_handcoded_index.py::test_files PASSED [ 5%]
tests/test_handcoded_index.py::test_images PASSED [ 8%]
tests/test_handcoded_index.py::test_links PASSED [ 11%]
tests/test_handcoded_index.py::test_likes PASSED [ 14%]
tests/test_handcoded_index.py::test_comments PASSED [ 17%]
tests/test_handcoded_index.py::test_timestamps PASSED [ 20%]
tests/test_handcoded_user.py::test_images PASSED [ 22%]
tests/test_handcoded_user.py::test_links PASSED [ 25%]
tests/test_handcoded_user.py::test_likes PASSED [ 28%]
tests/test_handcoded_user.py::test_comments PASSED [ 31%]
tests/test_handcoded_user.py::test_timestamps PASSED [ 34%]
tests/test_handcoded_user.py::test_user_info PASSED [ 37%]
tests/test_insta485generator_hello.py::test_hello PASSED [ 40%]
tests/test_insta485generator_hello_css.py::test_hello_css PASSED [ 42%]
tests/test_style.py::test_pycodestyle PASSED [ 45%]
tests/test_style.py::test_pydocstyle PASSED [ 48%]
tests/test_style.py::test_pylint PASSED [ 51%]
tests/test_template_followers.py::test_files PASSED [ 54%]
tests/test_template_followers.py::test_awdeorio_followers PASSED [ 57%]
tests/test_template_followers.py::test_michjc_followers PASSED [ 60%]
tests/test_template_followers.py::test_jag_followers PASSED [ 62%]
tests/test_template_followers.py::test_jflinn_followers PASSED [ 65%]
tests/test_template_following.py::test_files PASSED [ 68%]
tests/test_template_following.py::test_awdeorio_following PASSED [ 71%]
tests/test_template_following.py::test_michjc_following PASSED [ 74%]
tests/test_template_following.py::test_jag_following PASSED [ 77%]
tests/test_template_following.py::test_jflinn_following PASSED [ 80%]
tests/test_template_html.py::test_html PASSED [ 82%]
tests/test_template_index.py::test_files PASSED [ 85%]
tests/test_template_index.py::test_images PASSED [ 88%]
tests/test_template_index.py::test_links PASSED [ 91%]
tests/test_template_index.py::test_likes PASSED [ 94%]
tests/test_template_index.py::test_timestamps PASSED [ 97%]
tests/test_template_index.py::test_comments PASSED [100%]
============================= 35 passed in 18.78s ==============================
Running pytest with the additional -s flag passes output from the program (stdout) to the terminal, e.g., print() statements. Notice when we add -s, the output from Pylint (“Your code has been rated …”) appears.
$ pytest -vs tests/test_style.py::test_pylint
::test_pylint
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
PASSED
Stop after the first failure.
$ pytest -x
Debugging options
Rerun only the tests that failed last time
$ pytest --last-failed
Automatically start a debugger when a failure occurs. The -s is required for PDB+. Don’t forget to add import pdbp.
$ pytest --pdb -s
Start a debugger on the first line of the test.
$ pytest --trace -s
Start PDB+ from a breakpoint().
$ pytest -s
Acknowledgments
Original document written by Andrew DeOrio awdeorio@umich.edu.
This document is licensed under a Creative Commons Attribution-NonCommercial 4.0 License. You’re free to copy and share this document, but not to sell it. You may not share source code provided with this document.