p3-insta485-clientside

End-to-End Testing

To test a website that uses client-side dynamic pages, we need mimic a web browser. We will use Google Chrome in headless mode, controlling it with Chromedriver and the Selenium library.

If you’re running WSL, first take a look at the work arounds for WSL and Chromedriver.

Path

Make sure your entire project path contains no spaces. Your path may be different. See the instructions to Create a folder.

$ pwd
/Users/awdeorio/src/eecs485/p3-insta485-clientside

WSL

If you’re running the Windows Subsystem for Linux (WSL), check your version. Selenium and headless Chrome work correctly on WSL 2, but not WSL 1.

Start a Windows PowerShell. Verify that you are using WSL 2. Follow the WSL Tutorial if you need to install or upgrade. Your Ubuntu version may be different.

$ wsl -l -v
  NAME                   STATE           VERSION
*  Ubuntu-20.04           Running         2

Chrome

Google Chrome will run our JavaScript in the background while a unit test runs. Chrome supports “headless” mode, where everything is hidden from the screen.

Make sure that you have Chrome version >59. Your version may be different.

$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version
$ google-chrome --version  # Linux/WSL
Google Chrome 94.0.4606.71

macOS

Install Chrome on macOS if you don’t already have it.

$ brew install --cask google-chrome
$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version  # macOS
Google Chrome 94.0.4606.71

WSL

On WSL, even if you have the Chrome browser installed through Windows, you still need to install Chrome to Linux via apt-get.

$ wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
$ echo 'deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main' | sudo tee /etc/apt/sources.list.d/google-chrome.list
$ sudo apt-get update
$ sudo apt-get install google-chrome-stable
$ google-chrome --version
Google Chrome 94.0.4606.71

To open Linux GUI applications invoked through WSL, you’ll also need a Windows X server. Install VcXsrv. After installing, you might have to adjust your firewall settings to allow VcXsrv to work properly. To do so, click Start and type “Windows Defender Firewall” and hit enter. Select “Allow an app or feature through Windows Defender Firewall”. Now, click “Change settings”, look for “VcXsrv windows xserver”, and make sure it is checked to the left, and checked under “Private” and “Public” like in the screenshot below:

VcXsrv Windows Defender Firewall

Now that VcXsrv is installed and your firewall settings are correct, you can launch it from the Start Menu > VcXsrv > XLaunch.

VcXsrv XLaunch Start Menu

When launching VcXsrv, make sure to select “Disable Access Control”.

VcXsrv Disable Access Control

After VcXsrv is launched successfully, the icon should appear in the system tray.

VcXsrv System Tray

Next, Edit your WSL .bashrc file and add an environment variable containing the address of the X server display.

$ echo 'export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk "{print \$2; exit;}"):0.0' >> ~/.bashrc

Restart WSL. If you reboot your computer later, don’t forget that you need make sure VcXsrv is running.

Start Chrome from Linux and verify that a Chrome window pops up.

$ google-chrome

Selenium

Selenium allows us to automate finding elements in the DOM and navigating the web page under test (clicking on buttons, scrolling, etc.).

Make sure your virtual environment is activated. Activate it if necessary with source env/bin/activate.

$ echo $VIRTUAL_ENV
/Users/awdeorio/src/eecs485/p3-insta485-clientside/env
$ which python
/Users/awdeorio/src/eecs485/p3-insta485-clientside/env/bin/python

Make sure Selenium is installed. In EECS 485, this is just a sanity check, Selenium is included as a dependency in the provided setup.py. If you’re working on a side project, you might need to do pip install selenium. Your version may be different.

$ pip show selenium
Name: selenium
Version: 3.141.0
...

Chromedriver

Chromedriver is a separate executable from the Google Chrome browser. Chromedriver is used by Selenium to control the Chrome browser.

Install the latest chromedriver using npm:

$ npm install chromedriver --detect_chromedriver_version --no-save

Double check that Chrome driver works. Your version may be different.

$ npx chromedriver --version
ChromeDriver 94.0.4606.61 (418b78f5838ed0b1c69bb4e51ea0252171854915-refs/branch-heads/4606@{#1204})

Make sure that the major version of Chromedriver matches the major version of Google Chrome. The major version in this example is 94, your version may be different.

$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version  # macOS
$ google-chrome --version  # Linux/WSL
Google Chrome 94.0.4606.71

Run a test

Run a simple test which performs a Google search for “hello world”.

$ wget https://eecs485staff.github.io/p3-insta485-clientside/test_selenium_hello.py
$ python3 test_selenium_hello.py
"Hello, World!" program - Wikipedia
HelloWorld - Digital promotions & loyalty programs for the ...
Hello, World! - Learn Python - Free Interactive Python Tutorial
The Hello World Collection
Hello World · GitHub Guides
The History of 'Hello, World' - HackerRank Blog
Hello World - Raspberry Pi
Computer Programming/Hello world - Wikibooks, open books ...

Next, run sanity test loads your HTML and JavaScript.

$ pytest -v --log-cli-level=INFO tests/test_index.py::test_anything
...
INFO     autograder:conftest.py:77 Setup test fixture 'app'
INFO     autograder:conftest.py:130 Setup test fixture 'base_driver'
INFO     autograder:conftest.py:160 IMPLICIT_WAIT_TIME=10
INFO     autograder:conftest.py:192 Setup test fixture 'driver'
INFO     werkzeug:_internal.py:122  * Running on http://localhost:50504/ (Press CTRL+C to quit)
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET / HTTP/1.1" 302 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /accounts/login/ HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET / HTTP/1.1" 302 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /accounts/login/ HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /static/css/style.css HTTP/1.1" 200 -
INFO     werkzeug:_internal.py:122 127.0.0.1 - - [22/Jan/2020 08:25:52] "GET /static/images/logo.png HTTP/1.1" 200 -
PASSED

Understanding Selenium tests

Let’s take a deeper look at the previous example, test_selenium_hello.py.

First, comment out the line about headless chrome.

    options = selenium.webdriver.chrome.options.Options()
    # options.add_argument("--headless") # Comment out this line

Next, add a breakpoint to the test_selenium_hello() function in test_selenium_hello.py right after the line driver.get("https://www.google.com").

def test_selenium_hello():
    """Perform a Google search using Selenium and a headless Chrome browser."""
    ...
    driver.get("https://www.google.com")
    breakpoint()
    ...

Pro-tip: First, make sure that your virtual environment is activated by running source env/bin/activate, and that you have pdbpp installed! If you do not, run pip install pdbpp.

Run the test again and step through it one line at a time (n for next at the PDB/PDB++ prompt).

$ python3 test_selenium_hello.py
(Pdb++) n
(Pdb++) n
...

A Google Chrome window opens with a message about being controlled by automated test software.

The Selenium driver enters text into the search box.

Search results appear on the page.

Summary

Selenium automatically “drives” the browser, usually without showing a window (“headless” mode). For debugging, it’s useful to step through the test without the headless option, watching the results in the browser window.

Acknowledgments

Original document written by Andrew DeOrio awdeorio@umich.edu.

This document is licensed under a Creative Commons Attribution-NonCommercial 4.0 License. You’re free to copy and share this document, but not to sell it. You may not share source code provided with this document.