Homework 3: Unit testing with JUnit and Eclemma

To Deliver: 

updated svn repository with maze and test code in a tag release 2.0, 

upload your test classes in a single archive file to the web-cat server

To Download: /home/scratch/kemper/project2.tar with existing Maze codebase

Due date for submission: September 27, at noon (12.00 PM)

Drop out date: October 1, at 5 AM (EARLY MORNING)

Motivation

We discussed test driven development and junit a tool for unit testing. The goal of this homework is to get more experience in developing a test suite that covers main stream intended usage as well as corner cases that a robust implementation should handle as well. 

In the first project assignment you worked on a stand alone problem that connected this class with the CS 241 class. For the rest of the class and further homework and project assignments, we will work on a single different problem which is about a Maze game.  

We will use an existing implementation by Paul Falstad (www.falstad.com) whose code is readily available over the internet. Note that the code is copyrighted. I contacted Paul Falstad and he gave us the permission to use and modify his code for teaching purposes in CS 301. While the original code is available and you may want to read it for your information, we will work on a refactored version that I produced (so blame errors on me). I reorganized the code into a few more classes and added comments which I believe will make it easier to understand. 

Important: Recognize and check if the code works. It should! The documentation came after the fact and need not be correct. If the documentation and code do not match, improve the documentation!

Run the MazeApplication and see how the Maze program works. 

• If it is called with no parameters it runs as like the original code by Falstad (see also: http://www.falstad.com/maze/ for documentation).

• If it is called with parameter "Prim", it will use a randomized version of Prim's algorithm to generate the maze.  

• If it is called with a parameter that provides an input file in a particular XML format, then the code loads that maze from a file and operates on it. The class MazeFileWriter produces the particular XML format. 

To see how you can further operate the application, read the Maze.keyDown method to see what keyboard input the application reads and what it does with that.

Obviously, a rigid set of test cases will be useful to clarify particular functionality of individual classes as well as it will help you to familiarize yourself with the new codebase. 

Note that the intention for creating good test cases is a destructive one: the goal of a tester is to find errors and weaknesses in an implementation. A tester must imagine possible ways to break an implementation and think of input combinations that will lead to exceptions that the code cannot handle or will lead to wrong results. 

However, in the current situation, we have a (hopefully) running piece of code and we will develop test cases to characterize the existing behavior.

This is helpful for regression testing when you will add features to the existing code and may want to check that you did not break anything. 

This homework will set the state for future project assignments!

Requirements

For this homework, we need to develop a set of tests  for two classes: Cells.java and MazeBuilder.java.

This will result in two Junit test classes: CellsTest.java and MazeBuilderTest.java.

 The goal for both test classes is to achieve an overall coverage for Cells.java and MazeBuilder.java towards 100%.

 The grading will basically correspond to the coverage percentage you achieve.

To do list:

• Create a new project called maze in Eclipse
• Get the maze code from /home/scratch/kemper/project2.tar to it. 
• Note: a file with suffix .tar is an archive that you need to expand to access its contents. 
• As you should expand the archive in your own directory, please copy it first. 
• The command to expand (extract) the archive in a Linux terminal is: "tar -xvf project2.tar ". 
• Hint: to find such information you can use the manual command: "man tar" in a Linux terminal (yes, there is a manual page for the manual itself, type "man man" or "man -k manual") or do an internet search for "tar file tutorial" which may lead you here.
• Import the maze code into your new Eclipse project.
• You should get two source code directories src and test. 
• The source directory will contain a package called falstad. 
• The test directory will also contain a simple directory named data. 
• You may want to configure the Eclipse BuildPath to include two source folders (src and test) and to include the Junit 4 library before you can run the code.
• Run the MazeApplication class and see how the Maze program works. 
• See also: http://www.falstad.com/maze/ for documentation. 
• Create three Run Configurations: 
• 1) no parameters, 
• 2) parameter "Prim", 
• 3) parameter "test/data/input.xml"
• Read the Maze.keyDown method to see what keyboard input the application reads and what it does with that.
• At this point, please upload your project into your svn repository (Team->Share Project), such that you can always can get back to this initial version easily.
• Once you feel comfortable with the application, check the Cells.java class and its documentation. We want to test this class in a rigid manner, so you need to understand what it does. DO NOT TRUST THE DOCUMENTATION TOO MUCH!
• Add a new package falstad to your test directory.
• Testing Cells.java
• Add a new junit test case CellsTest.java to that new falstad package in the test directory. 
• (Eclipse helps you doing this by right-click on Cells.java -> New -> Juni test case, then switch the source folder to test/falstad, select "generate comments" in addition to the default settings. Next will give you a selection of methods to pick, choose all methods from Cells.java that you want to have test cases for.)
• Implement as many test cases as you consider reasonable to test the Cells.java class for individual non-trivial methods. 
• Use the tests to explore what the code is doing, improve its documentation, improve code where necessary. 
• NOTE: the comments were added after the fact for a working piece of code. So if comments and code do not match consider making the comments/documentation more precise first, e.g. by adding preconditions if code does not check for a legal range of values for its input parameter settings.
• Do not forget to provide a comment on top of each test case method that clearly describes the purpose of that test and what the expected outcome is.
• A test that has no assertion is not a valid test!
• Check the coverage you achieve with Eclemma, a brief user guide is here.
• Update your subversion repository whenever you made progress and got a particular feature or method going. We will run and evaluate your tests.
• Testing MazeBuilder.java
• Add a new junit test case MazeBuilderTest.java to the falstad package in the test directory. Develop a set of test case for the MazeBuilder class. 
• Be WARNED: this is more challenging than the tests for the Knightstour and the Cells class.
• MazeBuilder operates its calculations in a separate thread.  
• The MazeBuilder operates on behalf of a Maze object and closely interacts with it. Figure out how the communication between MazeBuilder and Maze method works. 
• You will need a mock maze object to operate a MazeBuilder and to be able to check the properties of a generated maze.
• Figure out how to give the MazeBuilder thread sufficient time to perform its calculations (or terminate) before a test proceeds in your junit test cases. HINT: check Java API for Thread class.
• You need to think about properties that are necessary for a maze to be correct. For instance, a correct maze needs to have an exit that can be reached from anywhere in the maze. (Hint: you need not write a maze exploration algorithm yourself, check what the distance matrix in Maze.mazedists contains and consider necessary and sufficient conditions for the existence of a path to the exit.)
• Note that a MazeBuilderTest class can inherit from some other class in Junit 4. This is a valuable asset you can use if you want the MazeBuilder talk to a mock object of a Maze class. 
• What to submit: 
• By subversion: upload your test code and any other changes you made to the code. Create a separate release 2.0 directory in your SVN repository.

Further reading:

There is a brief junit cookbook for junit 4 right here.

Grading:

The grading is based on the release 2.0 of your code in your subversion repository.

You will receive points for:

• An existing and accessible SVN repository that includes a release 2.0.
• CellsTest.java 
• An existing Junit Test class CellsTest that contains test code for the Cells.java class that can be executed.
• The class CellsTest contains serious test code for methods Cells.initialize(), Cells.areaOverlapsWithRoom(), Cells.markArea()
• Running Javadoc on the CellsTest results in reasonable documentation on tests. (You need not extensively document trivialities but provide at least one sentence per test.)
• All tests have at least one assertion.
• Running the CellsTest.java class works without errors and without failures.
• Running the CellsTest.java class with Eclemma is possible and gives coverage for Cells.java. The total coverage for that class in percent gives one point per percent with a max of 85 points. In other words, a coverage of 85 percent or better pays off, the last few percent towards 100 are not necessary to achieve.
• MazeBuilderTest:
• An existing Junit Test class MazeBuilderTest that contains test code for the MazeBuilder.java class that can be executed.
• The class MazeBuilderTest contains serious test code to check if a computed Maze is correct.
• Running Javadoc on the MazeBuilderTest results in reasonable documentation on tests. (You need not extensively document trivialities but provide at least one sentence per test.)
• All tests have at least one assertion.
• Running the MazeBuilderTest.java class works without errors and without failures.
• Running the MazeBuilderTest.java class with Eclemma is possible and gives coverage for MazeBuilder.java. The total coverage for that class in percent gives one point per percent with a max of 85 points. In other words, a coverage of 85 percent or better pays off, the last few percent towards 100 are not necessary to achieve.
• Note: Web-cat will be used to see if your test code executes, all tests are passed and calculation of code coverage. The evaluation will be done with the code that you uploaded to web-cat.