Testcover.com Tutorial - Need for the Service

Home
Brochure
Tutorial with Examples
->Need for the Service
->Submitting Requests
->Configuration Example
->Constraints Example
->Calendar Example
->Database Table Example
->Equivalence Partitioning
->UML State Machines
->Definitions of Terms
Performance
WSDL Interface
Background
Partners
Registrations
Contact Information

To see the need for the Testcover.com service, consider the test configurations for a networked computer system, with five test factors and their values as follows.
Test FactorNumber of ValuesTest Factor Values
1. Operating system4XP MacOS Linux Vista
2. Display resolution3low medium high
3. Connection4wi-fi dsl cable lan
4. Browser4IE Firefox Opera Safari
5. Application3App1 App2 App3
The question is how to select test cases to cover the test factor values.

A simple approach is to pick one most common value for each test factor. Then the tester varies each factor through all of its values while holding the others at their most common values. This example would lead to

4 + 3 + 4 + 4 + 3 = 18
test cases because there are 18 individual values to be covered. (The tester may note that the test case with all factors at their most common values is included five times and may remove four of these test cases.) The number of test cases here is relatively small, but their ability to find faults is limited. While all of the values have been covered, all of the value pairs have not. Thus, problems related to interactions between two factors may not be found.

A more ambitious approach is to test all combinations of factor values. In this example there would be

4 • 3 • 4 • 4 • 3 = 576
test cases. This method should find all the faults from interactions among these factors because all combinations are tested. Generally however, the method is not practical because it leads to too many test cases.

The need then is to find a small set of test cases which will include all factor value pairs. Testers have used orthogonal arrays and covering arrays to select test cases when such arrays are known to fit their test experiments. These arrays are mathematical templates which can have the desired coverage properties for the factor value combinations. Their use can improve a test program's efficiency and ability to find faults compared to the methods described above. But when real-world requirements are applied, the test design can become more complicated.

Suppose one of the planned test cases in this example is

 Linux high dsl IE App1
Also suppose that browser IE is not supported on operating system Linux. That is, the test cases must be constrained not to include IE with Linux. What is to be done with the
 Linux high dsl IE App1
test case? It may fail if executed, or it may not be possible to run at all. Removing the test case from the plan may reduce coverage of other factor value pairs which are allowed (IE with high, for example). The tester might decide to replace one of the values in the test case to make it possible. Firefox could replace IE if Firefox is compatible with Linux. Then the test case
 Linux high dsl Firefox App1
would appear in the plan instead. But such a change also could reduce coverage for other pairs: IE is not associated with high in this test case anymore.

The Testcover.com service is designed to handle these types of test constraints easily. The eight examples below illustrate its use in common test situations.
 • The configuration example shows how to generate test cases for a similar test system with five test factors. No constraints are applied in this simple example. • The constraints example shows how to generate test cases for the test system described on this page. The constraints among the browsers and operating systems are easily handled. • The calendar example illustrates how to generate input data for error test cases along with those for normal operation. Different partitions are used to account for different expected results. • The database table example shows how to use the generator to create records for testing a reservation system. Multiple blocks are used to accommodate dependencies (business rules) among the factor values. • The HTML form example illustrates how to partition test inputs when there are different classes of normal expected results. • The temperature conversion example introduces the concept of functional dependence to simplify the constraint analysis in a test design. • The shopping cart example shows two approaches for managing the state of the test system: The sequence unit replay model applies pairwise automated testing to selected equivalence partitions. Alternatively, when the shopping cart is described by a UML state machine, the transition model and the target state model apply pairwise testing to the shopping cart state transitions systematically. • Lastly, the thermostat example illustrates how to generate test cases for a system described by a UML state machine diagram with multiple regions. This example compares designs using four different test models.