p3-insta485-clientside
End-to-End Testing
To test an entire website that uses client-side dynamic pages, we need mimic a web browser. We will use Google Chrome in headless mode, control it using Chrome Driver and the Selenium library.
If you’re running WSL, first take a look at the work arounds for WSL and Chromedriver.
Headless browser
Google Chrome will run our JavaScript in the background while a unit test runs. Chrome supports “headless” mode, where everything is hidden from the screen.
Make sure that you have Chrome version >59 by pasting chrome://settings/help
into the search bar.
Google Chrome on Windows Subsystem for Linux
On WSL, even if you have the Chrome browser installed through Windows, you still need to install Chrome via apt-get
.
$ wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
$ echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list
$ sudo apt-get update
$ sudo apt-get install google-chrome-stable
To open Linux GUI applications invoked through WSL, you’ll also need a Windows X server. Install and run Xming. Once it’s running, you will see in your system tray.
Next, Edit your WSL .bashrc
file and add the line export DISPLAY=:0
. This environment variable contains the address of the X server display.
$ echo "export DISPLAY=:0" >> ~/.bashrc
Restart WSL. If you reboot your computer later, don’t forget that you need make sure Xming is running.
Selenium
Selenium allows us to automate finding elements in the DOM and navigating the web page under test (clicking on buttons, scrolling, etc.).
Make sure your virtual environment is activated. Activate it if necessary with source env/bin/activate
.
$ echo $VIRTUAL_ENV
/Users/awdeorio/src/eecs485/p3-insta485-clientside/env
$ which python
/Users/awdeorio/src/eecs485/p3-insta485-clientside/env/bin/python
Make sure Selenium is installed. In EECS 485, this is just a sanity check, Selenium is included as a dependency in the provided setup.py
. If you’re working on a side project, you might need to do pip install selenium
.
$ pip show selenium
Name: selenium
Version: 3.141.0
...
Chromedriver
Chromedriver is a separate executable from the Google Chrome browser. Chromedriver is used by Selenium to control the Chrome browser.
Install the latest chromedriver
using npm
:
$ npm install chromedriver --detect_chromedriver_version --no-save
Double check that Chrome driver works. Your version may be different.
$ npx chromedriver --version
ChromeDriver 87.0.4280.20 (c99e81631faa0b2a448e658c0dbd8311fb04ddbd-refs/branch-heads/4280@{#355})
Run a test
Run a simple test which performs a Google search for “hello world”.
$ wget https://eecs485staff.github.io/p3-insta485-clientside/test_selenium_hello.py
$ python3 test_selenium_hello.py
"Hello, World!" program - Wikipedia
HelloWorld - Digital promotions & loyalty programs for the ...
Hello, World! - Learn Python - Free Interactive Python Tutorial
The Hello World Collection
Hello World · GitHub Guides
The History of 'Hello, World' - HackerRank Blog
Hello World - Raspberry Pi
Computer Programming/Hello world - Wikibooks, open books ...
Next, run sanity test loads your HTML and JavaScript.
$ pytest -v --log-cli-level=INFO tests/test_index.py::test_anything
...
INFO autograder:conftest.py:77 Setup test fixture 'app'
INFO autograder:conftest.py:130 Setup test fixture 'base_driver'
INFO autograder:conftest.py:160 IMPLICIT_WAIT_TIME=10
INFO autograder:conftest.py:192 Setup test fixture 'driver'
INFO werkzeug:_internal.py:122 * Running on http://localhost:50504/ (Press CTRL+C to quit)
INFO werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET / HTTP/1.1" 302 -
INFO werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /accounts/login/ HTTP/1.1" 200 -
INFO werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET / HTTP/1.1" 302 -
INFO werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /accounts/login/ HTTP/1.1" 200 -
INFO werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /static/css/style.css HTTP/1.1" 200 -
INFO werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /static/images/logo.png HTTP/1.1" 200 -
PASSED
WSL and Chromedriver
On WSL 1, frontend tests will sometimes fail with an error related to connecting to Chromedriver. This will be fixed in the forthcoming WSL 2.
selenium.common.exceptions.WebDriverException: Message: chrome not reachable
(Session info: headless chrome=77.0.3865.90)
Workaround 1: CAEN Linux
The first work around is to run the tests that use Selenium on CAEN Linux. Installing your code on another machine is easy once you’ve created an install script.
$ ssh <uniquename>@login.engin.umich.edu
$ git clone <your git url>
$ cd p3-insta485-clientside
$ ./bin/insta485install
$ source env/bin/activate
$ pytest -v --log-cli-level=INFO tests/test_index.py::test_anything
If you run into a permission error when running ./bin/insta485install
script, make sure you have made it executable using chmod +x ./bin/insta485install
. Then delete the env/
, node_modules/
, insta485.egg-info
(if it exists), insta485/statics/js/bundle.js
and reinstall your solution ./bin/insta485install
, reactivate the environment and run the test again.
Workaround 2: WSL 2
Selenium and headless Chrome work correctly on WSL 2. At the time of this writing, WSL 2 is stable and can be installed using the following WSL 2 Install Instructions.
Understanding Selenium tests
Let’s take a deeper look at the previous example, test_selenium_hello.py
.
First, comment out the line about headless
chrome.
options = selenium.webdriver.chrome.options.Options()
# options.add_argument("--headless") # Comment out this line
Next, add a breakpoint to the test_selenium_hello()
function in test_selenium_hello.py
right after the line driver.get("https://www.google.com")
.
def test_selenium_hello():
"""Perform a Google search using Selenium and a headless Chrome browser."""
...
driver.get("https://www.google.com")
import pdb; pdb.set_trace()
...
Pro-tip: Make sure you have pdbpp
installed first! If you do not, run pip install pdbpp
.
Pro-tip: you can use breakpoint()
instead of import pdb; pdb.set_trace()
in Python 3.7+.
Run the test again and step through it one line at a time (n
for next
at the PDB/PDB++ prompt).
$ python3 test_selenium_hello.py
(Pdb++) n
(Pdb++) n
...
A Google Chrome window opens with a message about being controlled by automated test software.
The Selenium driver enters text into the search box.
Search results appear on the page.
Summary
Selenium automatically “drives” the browser, usually without showing a window (“headless” mode). For debugging, it’s useful to step through the test without the headless option, watching the results in the browser window.