p3-insta485-clientside

End-to-End Testing

To test an entire website that uses client-side dynamic pages, we need mimic a web browser. We will use Google Chrome in headless mode, control it using Chrome Driver and the Selenium library.

If you’re running WSL, first take a look at the work arounds for WSL and Chromedriver.

Headless browser

Google Chrome will run our JavaScript in the background while a unit test runs. Chrome supports “headless” mode, where everything is hidden from the screen.

Make sure that you have Chrome version >59 by pasting chrome://settings/help into the search bar.

Google Chrome on Windows Subsystem for Linux

On WSL, even if you have the Chrome browser installed through Windows, you still need to install Chrome via apt-get.

$ wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
$ echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list
$ sudo apt-get update
$ sudo apt-get install google-chrome-stable

To open Linux GUI applications invoked through WSL, you’ll also need a Windows X server. Install VcXsrv. Once it’s installed, you can launch it from the Start Menu > VcXsrv > XLaunch.

VcXsrv XLaunch Start Menu

When launching VcXsrv, make sure to select “Disable Access Control”

VcXsrv Disable Access Control

After VcXsrv is launched successfully, the icon should appear in the system tray

VcXsrv System Tray

Next, Edit your WSL .bashrc file and add an environment variable containing the address of the X server display.

$ echo 'export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk "{print \$2; exit;}"):0.0' >> ~/.bashrc

Restart WSL. If you reboot your computer later, don’t forget that you need make sure VcXsrv is running.

Selenium

Selenium allows us to automate finding elements in the DOM and navigating the web page under test (clicking on buttons, scrolling, etc.).

Make sure your virtual environment is activated. Activate it if necessary with source env/bin/activate.

$ echo $VIRTUAL_ENV
/Users/awdeorio/src/eecs485/p3-insta485-clientside/env
$ which python
/Users/awdeorio/src/eecs485/p3-insta485-clientside/env/bin/python

Make sure Selenium is installed. In EECS 485, this is just a sanity check, Selenium is included as a dependency in the provided setup.py. If you’re working on a side project, you might need to do pip install selenium.

$ pip show selenium
Name: selenium
Version: 3.141.0
...

Chromedriver

Chromedriver is a separate executable from the Google Chrome browser. Chromedriver is used by Selenium to control the Chrome browser.

Install the latest chromedriver using npm:

$ npm install chromedriver --detect_chromedriver_version --no-save

Double check that Chrome driver works. Your version may be different.

$ npx chromedriver --version
ChromeDriver 87.0.4280.20 (c99e81631faa0b2a448e658c0dbd8311fb04ddbd-refs/branch-heads/4280@{#355})

Run a test

Run a simple test which performs a Google search for “hello world”.

$ wget https://eecs485staff.github.io/p3-insta485-clientside/test_selenium_hello.py
$ python3 test_selenium_hello.py
"Hello, World!" program - Wikipedia
HelloWorld - Digital promotions & loyalty programs for the ...
Hello, World! - Learn Python - Free Interactive Python Tutorial
The Hello World Collection
Hello World · GitHub Guides
The History of 'Hello, World' - HackerRank Blog
Hello World - Raspberry Pi
Computer Programming/Hello world - Wikibooks, open books ...

Next, run sanity test loads your HTML and JavaScript.

$ pytest -v --log-cli-level=INFO tests/test_index.py::test_anything
...
INFO     autograder:conftest.py:77 Setup test fixture 'app'
INFO     autograder:conftest.py:130 Setup test fixture 'base_driver'
INFO     autograder:conftest.py:160 IMPLICIT_WAIT_TIME=10
INFO     autograder:conftest.py:192 Setup test fixture 'driver'
INFO     werkzeug:_internal.py:122  * Running on http://localhost:50504/ (Press CTRL+C to quit)
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET / HTTP/1.1" 302 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /accounts/login/ HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET / HTTP/1.1" 302 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /accounts/login/ HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /static/css/style.css HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /static/images/logo.png HTTP/1.1" 200 -
PASSED

WSL and Chromedriver

On WSL 1, frontend tests will sometimes fail with an error related to connecting to Chromedriver. This will be fixed in the forthcoming WSL 2.

selenium.common.exceptions.WebDriverException: Message: chrome not reachable
         (Session info: headless chrome=77.0.3865.90)

Workaround 1: CAEN Linux

The first work around is to run the tests that use Selenium on CAEN Linux. Installing your code on another machine is easy once you’ve created an install script.

$ ssh <uniquename>@login.engin.umich.edu
$ git clone <your git url>
$ cd p3-insta485-clientside
$ ./bin/insta485install
$ source env/bin/activate
$ pytest -v --log-cli-level=INFO tests/test_index.py::test_anything

If you run into a permission error when running ./bin/insta485install script, make sure you have made it executable using chmod +x ./bin/insta485install. Then delete the env/, node_modules/, insta485.egg-info(if it exists), insta485/statics/js/bundle.js and reinstall your solution ./bin/insta485install, reactivate the environment and run the test again.

Workaround 2: WSL 2

Selenium and headless Chrome work correctly on WSL 2. At the time of this writing, WSL 2 is stable and can be installed using the following WSL 2 Install Instructions.

Understanding Selenium tests

Let’s take a deeper look at the previous example, test_selenium_hello.py.

First, comment out the line about headless chrome.

    options = selenium.webdriver.chrome.options.Options()
    # options.add_argument("--headless") # Comment out this line

Next, add a breakpoint to the test_selenium_hello() function in test_selenium_hello.py right after the line driver.get("https://www.google.com").

def test_selenium_hello():
    """Perform a Google search using Selenium and a headless Chrome browser."""
    ...
    driver.get("https://www.google.com")
    breakpoint()
    ...

Pro-tip: First, make sure that your virtual environment is activated by running source env/bin/activate, and that you have pdbpp installed! If you do not, run pip install pdbpp.

Run the test again and step through it one line at a time (n for next at the PDB/PDB++ prompt).

$ python3 test_selenium_hello.py
(Pdb++) n
(Pdb++) n
...

A Google Chrome window opens with a message about being controlled by automated test software.

The Selenium driver enters text into the search box.

Search results appear on the page.

Summary

Selenium automatically “drives” the browser, usually without showing a window (“headless” mode). For debugging, it’s useful to step through the test without the headless option, watching the results in the browser window.