Reliability Statistics with GERT


North Carolina State University
CSC 326 - Software Engineering
Laurie Williams, Nachi Nagappan, and Sarah Heckman. [Contact Authors]

Back to Software Engineering Tutorials


0.0 Outline
1.0 Intro to GERT
2.0 Extending Eclipse with the GERT Plug-in
3.0 GERT Preferences
4.0 Reliability Files View
5.0 Adding Source and Test Files to GERT
6.0 Reliabilty Metrics View
7.0 Reliability Ratios View
8.0 Resources

1.0 Intro to GERT

GERT (Good Enough Reliability Tool) is an Eclipse plug-in that is used to provide reliability estimations and feedback on the quality of test coverage in an application. GERT provides test coverage infomation by analysing source and test code to determine how well your tests cover the source code. Reliability is based on the STREW metric suite that uses a set of easy-to-measure internal metrics that provide feedback on the thoroughness of a testing effort and can be used as early indication of the external measure of system reliability

Top | Contents

2.0 Extending Eclipse with the GERT Plug-in

Download the GERT plug-in. Extract GERT to your Eclipse install directory (on the machines in the lab it is C:\Program Files\eclipse\). Once you have extracted Eclipse go to your Eclipse directory and verify there is an edu.ncsu.csc.gert_3.0.0/ directory in the plugins/ directory. You may also use the download site for GERT, located at: http://gert.sf.net/.

Start (or restart) Eclipse. This will add the plug-in the the Eclipse platform. To make sure the GERT plug-in was added, select Window > Open Perspective > Other... Choose .

The GERT perspective offers three new views:

  • Reliability Files: There are folders that will contain a list of source and test files for the project you wish to get coverage and reliability statistics from.
  • Reliability Metrics: A page that has a graphical representation of the STREW metric.
  • Reliability Ratios: Contains metrics and coverage statistics for the source and test files selected in Reliability Files.
Top | Contents

3.0 GERT Preferences

To get to the GERT preferences page select Window > Preferences. Select GERT from the tree on the left of the preferences page.

3.1 The general GERT preferences allow the user to enter the number of requirements on the project being measured, and change the colors that highlight the range of the statistic. There are also two check boxes that allow for the STREW model and a custom model to be displayed.

3.2 The coverage preferences allow for the selection of a source and output folder for the project. This was the source and class files can be found and metrics can be computed on them.

3.3 The custom metric preference pages allows for the creation of a new metric. Several variables are listed that the GERT plug-in already calculates for you and can be used in the custom metric.

3.4 The STREW preferences allows for information about the STREW model, like variable coefficients and valid ranges, to be specified.


3.1 GERT Preferences

3.2 Coverage

3.3 Custom Metric

3.4 STREW Values
Top | Contents

4.0 Reliability Files View

The Reliability Files view contains a listing of source and test files that will have reliability and coverage sweep run on them. This view contains two folders: Source Files and Test Files. To add a file to these folders, right click on the file in the Package Explorer and select Add File(s) for Reliability Testing. The file will be added to the appropriate folder under the Reliability Files view. (All test files must have the proper JUnit naming scheme of ClassBeingTestedTest).

The tool bar in the Reliability Files view allows you to remove files, run a coverage sweep, and refresh the coverage report. The first icon removes a selected file. The second icon removes all of the files. The down arrow contains options to remove just source and test files, and run/refresh the coverage report.

Top | Contents

5.0 Adding Source and Test Files to GERT

To add a source or test file to these folders, right click on the file in the Package Explorer and select Add File(s) for Reliability Testing. The file will be added to the appropriate folder under the Reliability Files view. (All test files must have the proper JUnit naming scheme of ClassBeingTestedTest

Top | Contents

6.0 Reliability Metrics View
The Reliabilty Metrics view contains graphical output describing the STREW statistic and any custom metric that you create. In order to view a custom metric's results, make sure that Use custom expression model is checked in the GERT preferences.
Top | Contents

7.0 Reliability Ratios View

The Reliabity Ratios view list coverage metrics and statistics about each class and test class that has been selected to be run in GERT.

The Metrics Total box contains the following metrics (LOC represents lines of code):

  • Test Cases / Source LOC (M1): is an indication of whether there are too few test cases written to test the body of source code;
  • Source LOC / Test LOC (M2): is an indication of whether the ratio M1 was misleading as there might have been fewer, but comprehensive, test cases;
  • Assertions / Source LOC (M3): serves as a crosscheck for both M1 and M2 because there are few test cases but each have a large number of assertions;
  • (Test LOC / Test Classes) / (Source LOC / Source Classes) (M4): is used as an indicator for estimating defect density when prior releases of the software are available so that any change in the relative class size can be analyzed to determine if the increase in the size of the class has be accompanied by a corresponding increase in the testing effort;
  • Source LOC / Requirements (M5): is an indication of the program size with respect to the requirements;
  • Weighting of Conditionals for Source / Weighting of Conditionals for Test (M6):is an indication of the extent to which conditionals are used. The use of conditional statements increases the amount of testing required because there are more logic and data flow paths to be verified ;
  • Overall Statement Coverage: A percentage of all the LOC in the source file that are run by a test. measures test coverage and serves as a cross checks to M1, M2, and M3 to make sure that the same part of the code does not get tested repeatedly
  • Overall Branch Coverage: The percentage of branches (paths through conditional code) in the source course tested.

The colors of the text in the Metrics Totals box describes if the metric is within, above, or below required bounds. The color can be changed in the GERT preference.

The Source Classes area contains statistics about an individual selected source files including LOC, classes (there is more that one class listed if there are inner classes), conditionals, statement coverage percentage, and branch coverage percentage.

The Test Classes area contains statistcs about an individual selected test file including LOC, classes (there is more than one class listed if there are inner classes), test methods, and asserts.

Note: In order to switch between GERT and running JUnit tests, make sure that jcoverage.jar (located in the GERT plug-in directory) is on your project's build path. If there are any other errors, change your code and resave, thus recompiling your code. You should be able to run JUnit then. If you have any other problems, select Project > Clear....


Reliability Ratios View
Top | Contents

8.0 Resources
GERT - Good Enough Reliability Tool
Top | Contents

Back to Software Engineering Tutorials
Reliability Statistics with GERT Tutorial ©2003-2009 North Carolina State University, Laurie Williams, Nachi Nagappan, and Sarah Heckman
Email the authors with any questions or comments about this tutorial.
Last Updated: August 18, 2004