What is the relationship between number of software defects and the number of robustness failures?
The number of software defects (i.e., source code “bugs”) will in general be far smaller than the number of robustness failures reported by Ballista. This is because Ballista generates exhaustive combinations of parameter values when doing testing in order to elicit robustness failures in cases that only occur with certain combinations of parameter values. (Note that in some cases Ballista only samples the testing space in the interest of execution time, but still usually there are fewer software defects than robustness failures reported.) Ballista is predicated on the principle that source code will not be available for off-the-shelf software components. (This assumption may not always be true, but is often true and is useful from a research point of view for seeing how far the technology can be taken with that simplifying assumption.) Given this assumption, and the fact that Ballista is more about testing (in a quality assurance sense) than debugging, the only metric available for re
