EECS 485 Lab

Lab 8: Project 4 Setup

Goals

• Basic use of processes, threads, and sockets in Python
• How to identify busy waiting, a common pitfall
• Project 4 skeleton code

Project 4 Setup

Follow the Project 4 Setup Tutorial. You’ll need to complete all of the setup tutorial without errors before you can start Running the MapReduce Server.

Python Processes, Threads, and Sockets

Follow the Python processes, threads and sockets tutorial.

Running the MapReduce Server

Once you have completed the Project 4 Setup section and tutorial, continue with running the mapreduce server.

Completion Criteria

• Have a clean remote and local git repo with a .gitignore, starter files, and scripts
  Check

    $ pwd
    /Users/awdeorio/src/eecs485/p4-mapreduce
    $ git status
    On branch main
    Your branch is up to date with 'origin/main'.
  
    nothing to commit, working tree clean
    $ tree --matchdirs -I 'env|__pycache__|*.egg-info'
    .
    ├── mapreduce
    │   ├── __init__.py
    │   ├── main
    │   │   ├── __init__.py
    │   │   └── __main__.py
    │   ├── submit.py
    │   ├── utils.py
    │   └── worker
    │       ├── __init__.py
    │       └── __main__.py
    ├── requirements.txt
    ├── setup.py
    └── tests
        ├── correct
        │   ├── grep_correct.txt
        │   └── word_count_correct.txt
        ├── exec
        │   ├── grep_map.py
        │   ├── grep_reduce.py
        │   ├── wc_map.sh
        │   ├── wc_map_slow.sh
        │   ├── wc_reduce.sh
        │   └── wc_reduce_slow.sh
        ├── input
        │   ├── file01
                ...
        │   └── file08
        ├── input_small
        │   ├── file01
        │   └── file02
        ├── testdata
            ...
        ├── test_worker_8.py
        └── utils.py
  

• Have an activated virtual python environment (source env/bin/activate to activate)
  Check

    $ pwd
    /Users/awdeorio/src/eecs485/p4-mapreduce
    $ source env/bin/activate
    $ which python
    /Users/awdeorio/src/eecs485/p4-mapreduce/env/bin/python
  

• Be able to run tests locally on your project
  Show

  Don’t worry if the tests fail at this point.

    $ pwd
    /Users/awdeorio/src/eecs485/p4-mapreduce
    $ pytest -v
    ============================= test session starts ==============================
    ...
  
  

• Register your group on the project 4 autograder.

Lab Quiz

Complete the lab quiz by the due date.