Testcover.com
Performance for Comparison

Home
Existing User Login
Brochure
Sign up for Risk-Free Trial
About Testcover.com
Frequently Asked Questions
Tutorial
Performance
Background
Partners
Subscription Prices
Promotion Activations
Student Subscriptions
Privacy Policy
Terms & Conditions
Contact Information



How does your test case generator compare?
Test Model Test cases Response time
(valuesfactors)a (number)b(key)c (seconds)d
21062
23082
2120102
35112
36224152
32015=2
33018=2
39023=2
413326122
45162
4133923521=2
45220222
48232
42428=2
45320292
41531722929=2
48040=2
5144212202
5239220252
524333252
53327272
5833=2
51138=2
53045=2
6155224302
6251312362
6946=2
612572
62472=2
7162312422
73420492
74412315492
71061=2
71481=2
73091=4
817261453428 e613
83528643
86510415643
81178=2
830120=5
95525813
910812
92616842
930153=9
1034281003
10201817
1197211216
11141674
1237281448
1219210=10
131182016912
1327325=55
14272919614
15292722522
164102625636
221151101233015
Notes:
a Product notation describing test model. Each exponent gives the number of test factors having the number of test values given by the corresponding base. For example, model 36224 has 6 test factors with 3 values each and 24 factors with 2 values each. The indicated product gives the total number of test cases: 36224 = 12,230,590,464.
b Number of test cases generated to cover all pairs of test factor values for this model. The number can vary if there are constraints excluding some test cases. No constraints were applied, except for the model described in Note e below.
c Key to bounds on covering array size.
Array size is at lower bound (optimal)
=Array size equals best published upper boundf
d Elapsed time in seconds from request submission until results display. Response time can vary with service load, access speed and other factors. These response times were measured with a light service load and broadband internet access.
e Test model for the Advanced Web Search Example, demonstrated at the STAREAST 2005 EXPO. The 21 test factors led to 2715 factor value pairs; all but 6 were covered to comply with required constraints. See the EXPO Technical Presentation on our Background page for more detail. These results (61 test cases currently vs. 63 at STAREAST) illustrate an improvement in the service since the conference.
f References for bounds on covering array size.
On the state of strength-three covering arrays. M. Chateauneuf, D. L. Kreher. Journal of Combinatorial Designs, 10(4):217-238, 2002.
Products of mixed covering arrays of strength two. Charles J. Colbourn, Sosina S. Martirosyan, Gary L. Mullen, Dennis Shasha, George B. Sherwood, Joseph L. Yucas. Journal of Combinatorial Designs, 14(2):124-138, 2006.
Covering arrays on graphs: connections to design theory, extremal combinatorics and algebraic combinatorics. Karen Meagher. Presentation for LaCIM, February 2004.
Group construction of covering arrays. Karen Meagher, Brett Stevens. Journal of Combinatorial Designs, 13(1):70-77, 2005.
Group construction of covering arrays - part 2. Karen Meagher. Technical Report, January 2005.
Upper bounds for covering arrays by tabu search. Kari J. Nurmela. Discrete Applied Mathematics, 138(1-2):143-152, 2004.
Strength two covering arrays: existence tables and projection. Charles J. Colbourn. Discrete Mathematics, 308(5-6):772-786, 2008.
See also Comparison of Efficiency at pairwise.org tools page.
These results were updated May 1, 2008. They are subject to change, particularly improvement, without notice.
Copyright © 2003-2008 Testcover.com, LLC. All rights reserved.