How Big a Test?
We use a widely accepted statistical formula to calculate sample size.
This calculator can be used for traditional A/B (Split Run) testing as well as more statistically complex scientific testing
Generally, the smaller the change you want to see and/or the lower your response rate, the larger the sample size you need.
Sample Size Calculator
How big a sample does your test require?
Test Type
What type of test do you intend to perform?
Split Run (A/B, traditional test)
Scientific (regression analysis, multi-variate, Taguchi, etc.)
Split Run Test Set-Up
Parameter
Value
Explanation
Number Variables
1
2
3
4
5
A value of 1 means you have data from a previous mailing and you want to compare this test with the prior one.
Values Per Variable
2
3
4
5
6
7
8
9
10
The control value should be counted as one
Typical Response Rate
0.1%
0.2%
0.3%
0.4%
0.5%
0.6%
0.7%
0.8%
0.9%
1%
1.1%
1.2%
1.3%
1.4%
1.5%
1.6%
1.7%
1.8%
1.9%
2%
2.2%
2.4%
2.6%
2.8%
3%
3.2%
3.4%
3.6%
3.8%
4%
4.2%
4.4%
4.6%
4.8%
5%
5.5%
6%
6.5%
7%
7.5%
8%
8.5%
9%
9.5%
10%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
If you weren't doing a test, what would be the reponse rate you would expect to achieve?
Significance Threshold
0.1%
0.2%
0.3%
0.4%
0.5%
0.6%
0.7%
0.8%
0.9%
1%
1.1%
1.2%
1.3%
1.4%
1.5%
1.6%
1.7%
1.8%
1.9%
2%
2.2%
2.4%
2.6%
2.8%
3%
3.2%
3.4%
3.6%
3.8%
4%
4.2%
4.4%
4.6%
4.8%
5%
5.5%
6%
6.5%
7%
7.5%
8%
8.5%
9%
9.5%
10%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
What is the smallest percentage change in response rate you want to be able to detect? The lower the percentage you input, the bigger the required list.
Scientific Test Set-Up
Parameter
Value
Explanation
Typical Response Rate
0.1%
0.2%
0.3%
0.4%
0.5%
0.6%
0.7%
0.8%
0.9%
1%
1.1%
1.2%
1.3%
1.4%
1.5%
1.6%
1.7%
1.8%
1.9%
2%
2.2%
2.4%
2.6%
2.8%
3%
3.2%
3.4%
3.6%
3.8%
4%
4.2%
4.4%
4.6%
4.8%
5%
5.5%
6%
6.5%
7%
7.5%
8%
8.5%
9%
9.5%
10%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
If you weren't doing a test, what would be the reponse rate you would expect to achieve?
Significance Threshold
0.1%
0.2%
0.3%
0.4%
0.5%
0.6%
0.7%
0.8%
0.9%
1%
1.1%
1.2%
1.3%
1.4%
1.5%
1.6%
1.7%
1.8%
1.9%
2%
2.2%
2.4%
2.6%
2.8%
3%
3.2%
3.4%
3.6%
3.8%
4%
4.2%
4.4%
4.6%
4.8%
5%
5.5%
6%
6.5%
7%
7.5%
8%
8.5%
9%
9.5%
10%
10%
11%
12%
13%
14%
15%
16%
17%
18%
19%
20%
21%
22%
23%
24%
25%
26%
27%
28%
29%
30%
31%
32%
33%
34%
35%
36%
37%
38%
39%
40%
41%
42%
43%
44%
45%
46%
47%
48%
49%
50%
51%
52%
53%
54%
55%
56%
57%
58%
59%
60%
61%
62%
63%
64%
65%
66%
67%
68%
69%
70%
71%
72%
73%
74%
75%
76%
77%
78%
79%
80%
81%
82%
83%
84%
85%
86%
87%
88%
89%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
What is the smallest percentage change in response rate you want to be able to detect? The lower the percentage you input, the bigger the required list.
