p1-insta485-static

pytest Tutorial

pytest is a unit test utility that makes it easy to run unit tests. It is compatible with the Python unittest framework.

This tutorial uses Project 1 as an example. It assumes you have some code and a directory called tests/.

$ pwd
/Users/awdeorio/src/eecs485/p1-insta485-static
$ ls
bin  hello      html      insta485generator           setup.py
env  hello_css  insta485  insta485generator.egg-info  tests

You have installed and activated your Python virtual environment (instructions) and installed pytest (pip install pytest if needed). Your version might be different.

$ pytest --version
This is pytest version 4.1.0, imported from /Users/awdeorio/src/eecs485/p1-insta485-static/env/lib/python3.7/site-packages/pytest.py

By default, pytest will search for tests and run them all.

$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.7.2, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/awdeorio/src/eecs485/p1-insta485-static, inifile:
collected 38 items                                                             

tests/test_followers.py .....                                            [ 13%]
tests/test_following.py .....                                            [ 26%]
tests/test_generated_html.py ..                                          [ 31%]
tests/test_handcoded_html.py .                                           [ 34%]
tests/test_handcoded_index.py ......                                     [ 50%]
tests/test_handcoded_user.py ......                                      [ 65%]
tests/test_hello.py ..                                                   [ 71%]
tests/test_hello_css.py ...                                              [ 78%]
tests/test_index.py .....                                                [ 92%]
tests/test_python_style.py ...                                           [100%]

========================== 38 passed in 11.46 seconds ==========================

Print the name of each individual unit test with -v or --verbose. Pro-tip: -vv increases the output.

$ pytest -v
============================= test session starts ==============================
platform darwin -- Python 3.7.2, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 -- /Users/awdeorio/src/eecs485/p1-insta485-static/env/bin/python3
cachedir: .pytest_cache
rootdir: /Users/awdeorio/src/eecs485/p1-insta485-static, inifile:
collected 38 items                                                             

tests/test_followers.py::TestFollowers::test_awdeorio_followers PASSED   [  2%]
tests/test_followers.py::TestFollowers::test_files PASSED                [  5%]
tests/test_followers.py::TestFollowers::test_jag_followers PASSED        [  7%]

...
========================== 38 passed in 11.85 seconds ==========================

Running pytest with the additional -s flag passes output from the program (stdout) to the terminal, e.g., print() statements.

Rerun only the tests that failed last time

$ pytest --last-failed

Stop after the first failure.

$ pytest -x

Run all unit tests in one file.

$ pytest tests/test_followers.py

Run one unit test. You can find the exact name of a testcase by running them all with the verbose flag -v.

Automatically start a debugger when a failure occurs.

$ pytest --pdb

Deprecation warnings

Sometimes, third party libraries trigger warnings that you have no control over. Here’s an example from Project 3. Certain library and Python versions cause this error.

$ pwd
/Users/awdeorio/src/eecs485/p3-insta485-clientside
$ python --version
Python 3.7.2
$ python -c 'import jinja2; print(jinja2.__version__)'
2.10
$ pytest tests/test_index.py::TestIndex::test_feed_load
============================= test session starts ==============================
tests/test_index.py::TestIndex::test_feed_load                     PASSED [100%]
=============================== warnings summary ===============================
...
/Users/awdeorio/src/eecs485/p3-insta485-clientside/env/lib/python3.7/site-packages/jinja2/runtime.py:318: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import Mapping
...
===================== 1 passed, 5 warnings in 2.99 seconds =====================

Configure pytest with a pytest.ini file. We’ll configure it to ignore this warning. Here is a sample pytest.ini.

[pytest]
filterwarnings = ignore::DeprecationWarning

Run pytest again.

$ pwd
/Users/awdeorio/src/eecs485/p3-insta485-clientside
$ cat pytest.ini
[pytest]
filterwarnings = ignore:Using or importing the ABCs.*:DeprecationWarning
$ pytest tests/test_index.py::TestIndex::test_feed_load
...
=========================== 1 passed in 3.06 seconds ===========================

Further reading about pytest warnings can be found at https://docs.pytest.org/en/latest/warnings.html.