‘Combinatorial’ approach helps catch software bugs faster

December 13th, 2007 - 4:31 pm ICT by admin  

Washington, Dec 13 (ANI): A group of computer scientists and mathematicians from the National Institute of Standards and Technology (NIST), in association with University of Texas researchers, are developing an open-source tool which they believe will enable software developers to catch software errors more quickly.

The approach is called combinatorial testing, and promises to squash the bugs faster, and with fewer errors.

Studying software crashes in a variety of applications from medical devices to Web browsers, NIST researchers obtained hard evidence to support long-held conventional wisdom: most software failures result from simple events rather than complex ones.

Even for Web browsers containing hundreds of different variables, most failures were caused by interactions between just two variables. Nonetheless, in the applications that the researchers studied, additional failures could result from interactions of up to six variables.

Based on that insight, the NIST-Texas team went beyond the popular practice of pairwise testing, or exploring interactions between only two variables at a time, and designed a method for efficiently testing different combinations of settings in up to at least six interacting variables at a time. Their technique resembles combinatorial chemistry in which scientists screen multiple chemical compounds simultaneously rather than one at a time.

According to the release, “Their technique resembles combinatorial chemistry in which scientists screen multiple chemical compounds simultaneously rather than one at a time.”

The example that’s given relates to word processing. Suppose there are 10 different settings, superscript, bold, italics, etc. If a program has a bug when three specific ones are set, then doing a hard-code test of all possible three-combinations would require 1,024 tests. However, by looking at the fact that you’re testing groups of three in the set of 10, only 13 tests are required.

This new software program examines the software variables and, based on the number of variables, generates sets of tests which can be run along those same lines using up to six variables. It generates fewer tests because it’s testing the combination of things which have been statistically shown to be the most common cause of software errors.

Rather than testing every single possible condition exhaustively, the team believes that this interaction between a few common variables can result in a better capture rate of errors before the software is released, specifically in web-commerce software.

The researchers plan to release the tool early next year as open-source code. They currently are inviting developers to participate in beta testing of the tool before release.

According to the authors, the new approach for finding bugs to squash may be particularly useful for increasing the reliability of e-commerce Web sites, which often contain many interacting variables, as well as industrial process controls, such as for robotic assembly lines of high-definition televisions, which contain many interacting software-controlled elements that regularly turn on and off. (ANI)

Tags: , , , , , , , , , , , , , , , , , , ,

Posted in Health Science |