Using Jazz Code Coverage


Laurie Williams, Ben Smith, Sarah Heckman, and Lauren Hayward [Contact Authors]
CSC 326 - Software Engineering
Department of Computer Science
North Carolina State University

Back to Software Engineering Tutorials


0.0 Contents
1.0 Introduction to Jazz Code Coverage
2.0 Running Jazz Code Coverage
3.0 Jazz Code Coverage Coverage Report
4.0 Exercise
5.0 Resources

1.0 Introduction to Jazz Code Coverage

Jazz Code Coverage is a component that measures the test coverage as your projects evolve.

Coverage is defined as a measure of the completeness of the set of test cases. This definition means that the more source code that is executed by your test cases the better. There are four types of coverage:

  • Method coverage: what percentage of your methods have been called by your test cases?
  • Statement coverage: what percentage of your statements have been run by your test cases?
  • Decision/Branch coverage: what percentage of your decision points (if statements) have evaluated both true and false?
  • Condition coverage: what percentage of your Boolean sub-expressions have been evaluated both true and false at a decision point?

For small programs, coverage is relatively easy to calculate by hand; however, for larger programs, the task is much more difficult. Coverage tools, like Jazz Code Coverage, help by automating the coverage process and providing readable reports.

Jazz Code Coverage may be used to calculate coverage for any launched application. We will be using it to calculate the coverage of our JUnit test cases (view the JUnit tutorial). You can run your JUnit test cases, and Jazz Code Coverage will generate the coverage report from the execution. Jazz Code Coverage provides overall, package, and file level information on statement coverage.

Jazz Code Coverage can be downloaded as an incubator from jazz.net. For more information on installing eclipse plugins, consult the tutorial on the subject.

Top | Contents

2.0 Running Jazz Code Coverage
  1. For this tutorial, we will be using the CoffeeMaker project. Download the CoffeeMaker project from here or import the CoffeeMaker project that you used in the JUnit tutorial. Unzip the CoffeeMaker project to your home directory and import the project into Eclipse. Please see the Rational XDE and Eclipse Import/Export Guide for instructions on how to import a project into Eclipse.
  2. In order to begin gathering code coverage information, you must first enable static code instrumentation.
  3. In the Package Explorer view, right-click on your project and select Properties as shown in Figure 1.


    Figure 1. Project Properties

  4. Click the Java Coverage option in the left pane. Check the box Enable static code instrumentation. Static code instrumentation involves inserting probes throughout the software to monitor the software during run time (reference).

    Note: For users of web application containers (e.g. Tomcat), you will need to turn OFF static code instrumentation when you run your server. Also, to run your unit test coverage, you will need to shut down your server. The reason for this is that two separate virtual machines cannot be instrumented and running simultaneously; and Tomcat and JUnit run in separate Java VMs.
  5. The settings automatically set 80% block coverage. In our class, we will try to achieve 80% line coverage as shown in Figure 2. (To learn more about the different types of coverage, view the Emma FAQ). Click OK.

  6. Figure 2. Java Coverage

  7. A dialog box will appear asking you if you would like to rebuild the project now. Click Yes.
  8. Run your unit tests by right-clicking CoffeeMakerTest.java (located in CoffeeMaker > Src > unitttests > edu.csc.ncu.csc326.coffeemaker) and selecting Run As > JUnit Test. The Jazz Coverage tooling not only works with JUnit tests, it works with all Java-based launches supported by Eclipse! Section 3 will show you how to view your results!
Top | Contents

3.0 Jazz Code Coverage Coverage Report

There are two ways to obtain coverage results from a launched application: the label decorations in Java-based views such as the Package Explorer and the Outline view, or a customized coverage report based on an analysis over several Java projects and application launches.

  1. Run your unit tests by right-clicking CoffeeMakerTest.java (located in CoffeeMaker > Src > unitttests > edu.csc.ncu.csc326.coffeemaker) and selecting Run As > JUnit Test.
  2. One way to obtain coverage results is to observe the coverage decorations which are shown in the Package Explorer or the Outline view after every application launch as shown in Figure 3.

  3. Figure 3. Coverage Decorations

  4. When you open a class, you can easily see which code is covered as shown in Figure 4. Lines with a green indicator have been covered while lines with a red indicator have not been covered. Lines without an indicator are non-executable.

  5. Figure 4. Code Coverage Indicators

  6. Another way to obtain coverage results it to view a report. Choose Run > Coverage > Run Analysis. Select the project and application launch you would like to create the report and choose whether you would like to store the report on disk. See Figure 5. Select Run.

  7. Figure 5. Creating Coverage Report

  8. The Coverage Report appears as shown in Figure 6. You can expand the packages to view the coverage on specific classes.

  9. Figure 6. Coverage Report - Report Tab

  10. Click on the Statistics tab of the report to view overall project statistics as shown in Figure 7.

  11. Figure 7. Coverage Report - Statistics Tab

Top | Contents


4.0 Exercise

For this exercise we will be using the CoffeeMaker project. Download the CoffeeMaker project from here or import the CoffeeMaker project that you used in the JUnit tutorial. For more information on CoffeeMaker, consult the CoffeeMaker User Stories and Black Box Test Cases. Unzip the CoffeeMaker project to your home directory and import the project into Eclipse. Please see the Rational XDE and Eclipse Import/Export Guide for instructions on how to import a project into Eclipse.

We all know that most computer scientists love caffeine, so the Computer Science department is looking to put a coffee kiosk in the new building. The coffee kiosk must be able to make coffee for students to purchase.

The CoffeeMaker code is complete; however, we need you to create and run unit tests until you have 90% coverage and a green bar on the CoffeeMaker, Inventory, and Recipe classes. Run your JUnit tests using Jazz Code Coverage and generate a coverage report. (Hint: Run Jazz Code Coverage on the initial project to see what coverage you already have, and what lines you need to test.)

Deliverables to the TA

  • Run all of your JUnit unit tests, which give a green bar.
  • A coverage report with 90% statement coverage or higher for CoffeeMaker, Inventory, and Recipe classes.
Top | Contents

5.0 Resources
Top | Contents

Back to Software Engineering Tutorials
Test Coverage with Jazz Code Coverage ©2003-2008. North Carolina State University, Laurie Williams, Ben Smith, Sarah Heckman, and Lauren Hayward
Questions? Comments? Contact our mailing list

Last Updated: Monday, March 30, 2009 3:02 PM