Home
Existing User Login
Brochure
Sign up for Risk-Free Trial
About Testcover.com
Frequently Asked Questions
Examples
->Configuration
->Constraints
->Calendar
->Database Table
->Finite State Machine
->UML State Machine
Tutorial
Performance
WSDL Interface
Background
Partners
Registrations
Contact Information
|
How does your test case generator compare?
| Test model |
Test cases |
HTML response time |
WSDL response time |
|---|
| (valuesfactors)a |
(number)b | (key)c |
(seconds)d |
(seconds)d |
|---|
| 210 | 6 | • | 1 | 1 |
| 230 | 8 | • | 1 | 1 |
| 2120 | 10 | • | 2 | 1 |
| 35 | 11 | • | 1 | 1 |
| 36224 | 15 | | 2 | 1 |
| 320 | 15 | = | 1 | 1 |
| 330 | 18 | = | 2 | 1 |
| 390 | 23 | = | 2 | 2 |
| 413326 | 12 | • | 1 | 1 |
| 45 | 16 | • | 1 | 1 |
| 41339235 | 21 | = | 2 | 1 |
| 45220 | 22 | | 1 | 1 |
| 48 | 23 | | 1 | 1 |
| 424 | 28 | = | 2 | 1 |
| 45320 | 29 | | 2 | 1 |
| 415317229 | 29 | = | 2 | 1 |
| 480 | 40 | = | 4 | 2 |
| 5144212 | 20 | • | 1 | 1 |
| 5239220 | 25 | • | 2 | 1 |
| 524333 | 25 | • | 1 | 1 |
| 53327 | 27 | | 2 | 1 |
| 58 | 33 | = | 1 | 1 |
| 511 | 38 | = | 1 | 1 |
| 530 | 45 | = | 2 | 1 |
| 6155224 | 30 | • | 2 | 1 |
| 6251312 | 36 | • | 1 | 1 |
| 69 | 46 | = | 1 | 1 |
| 612 | 57 | | 2 | 1 |
| 624 | 72 | | 3 | 2 |
| 7162312 | 42 | • | 1 | 1 |
| 73420 | 49 | • | 2 | 1 |
| 74412315 | 49 | • | 2 | 1 |
| 710 | 61 | = | 2 | 1 |
| 714 | 81 | | 2 | 1 |
| 730 | 91 | = | 4 | 3 |
| 817261453428 e | 59 | | 3 | 1 |
| 83528 | 64 | • | 3 | 2 |
| 86510415 | 64 | • | 3 | 2 |
| 811 | 78 | = | 2 | 1 |
| 830 | 120 | = | 6 | 5 |
| 95525 | 81 | • | 3 | 3 |
| 910 | 81 | • | 2 | 1 |
| 92616 | 84 | | 2 | 1 |
| 930 | 153 | = | 10 | 8 |
| 103428 | 100 | • | 4 | 3 |
| 1020 | 181 | | 8 | 7 |
| 119721 | 121 | • | 6 | 5 |
| 1114 | 167 | | 5 | 4 |
| 123728 | 144 | • | 8 | 7 |
| 1219 | 210 | = | 11 | 10 |
| 1311820 | 169 | • | 13 | 12 |
| 1327 | 325 | = | 42 | 40 |
| 142729 | 196 | • | 11 | 10 |
| 152927 | 225 | • | 18 | 17 |
| 1641026 | 256 | • | 30 | 28 |
| 2211511012 | 330 | • | 13 | 11 |
| Notes: |
|---|
| a |
Product notation describing test model.
Each exponent gives the number of test factors having the number of test values given by the corresponding base.
For example, model 36224 has 6 test factors with 3 values each and 24 factors with 2 values each.
The indicated product gives the total number of test cases: 36224 = 12,230,590,464.
|
|---|
| b |
Number of test cases generated to cover all pairs of test factor values for this model.
The number can vary if there are constraints excluding some test cases.
No constraints were applied, except for the model described in Note e below.
| c |
Key to bounds on covering array size.
| • | Array size is at lower bound (optimal) |
| = | Array size equals best published upper boundf |
|
|---|
| d |
Elapsed time in seconds from request submission until results display.
Response time can vary with service load, access speed and other factors.
These response times were measured with a light service load and broadband internet access.
Firefox and soapUI were used for HTML and WSDL measurements respectively.
|
|---|
| e |
Test model for the Advanced Web Search Example, demonstrated at the STAREAST 2005 EXPO.
The 21 test factors led to 2715 factor value pairs; all but 6 were covered to comply with required constraints.
See the EXPO Technical Presentation on our
Background
page for more detail.
These results (59 test cases currently vs. 63 at STAREAST) illustrate an improvement in the service
since the conference.
|
|---|
| f |
References for bounds on covering array size.
On the state of strength-three covering arrays.
M. Chateauneuf, D. L. Kreher.
Journal of Combinatorial Designs, 10(4):217-238, 2002.
Products of mixed covering arrays of strength two.
Charles J. Colbourn, Sosina S. Martirosyan, Gary L. Mullen, Dennis Shasha, George B. Sherwood, Joseph L. Yucas.
Journal of Combinatorial Designs, 14(2):124-138, 2006.
Covering arrays on graphs: connections to design theory, extremal combinatorics and algebraic combinatorics.
Karen Meagher.
Presentation for LaCIM, February 2004.
Group construction of covering arrays.
Karen Meagher, Brett Stevens.
Journal of Combinatorial Designs, 13(1):70-77, 2005.
Group construction of covering arrays - part 2.
Karen Meagher.
Technical Report, January 2005.
Upper bounds for covering arrays by tabu search.
Kari J. Nurmela.
Discrete Applied Mathematics, 138(1-2):143-152, 2004.
Strength two covering arrays: existence tables and projection.
Charles J. Colbourn.
Discrete Mathematics, 308(5-6):772-786, 2008.
See also
Comparison of Efficiency
at pairwise.org tools page.
|
|---|
|
|---|
These results were updated May 9, 2012. They are subject to change, particularly improvement, without notice.
|
