b2KIT

Multivariate Test Planner

Plan multivariate tests by defining variables and levels. Calculate total combinations, required traffic, and test duration estimates.

Tested tool guide Tested browser tools Checked August 16, 2026

What Multivariate Test Planner does, with a checked example

Multivariate Test Planner maps an experiment before launch: define each page or campaign variable, list its candidate levels, and see how many distinct combinations the full test contains. It then uses the planning values you enter to estimate the traffic requirement and likely test duration. The common surprise is combinatorial growth. Levels are multiplied across variables, so three variables with 2, 3, and 4 levels produce 24 combinations, not 9. Every added combination divides the available test traffic more finely and can extend the proposed experiment.

Worked example

A concrete input and expected output from the current implementation.

Input

Headline: Control, Benefit-led
CTA: Start free, See pricing, Contact sales

Expected output

Total combinations: 6

The first variable has two levels and the second has three, so the full crossing is 2 x 3 = 6. Required traffic and duration depend on the separate planning assumptions entered for those estimates.

How the result is produced

1

Combination count

Each named item is a variable, and each candidate value is one level. The planner treats the design as a full set of cross-variable combinations: every headline level is paired with every image level, button level, and so on. It obtains the total by multiplying the level counts, not adding them. A control must be listed as a level if it should appear in the design.

2

Traffic and duration

After the combination count is known, the traffic and duration sections turn the planner's entered assumptions into operational estimates for the complete multivariate design. They do not, by themselves, prove that each combination has enough observations to detect an effect. If the inputs do not capture baseline performance, the smallest effect of interest, variability, or allocation, validate the traffic target with a separate power analysis.

Good uses

  • Checking how many test cells result from combining headline, image, offer, and call-to-action alternatives on a landing page.
  • Comparing a broad multivariate plan with a smaller design before committing limited campaign traffic to the experiment.
  • Estimating whether eligible daily traffic can support all planned combinations within the available testing window.

Limits and checks

  • The multiplication represents a full factorial design. If certain levels cannot validly appear together, the raw combination count includes test cells that may need to be removed or redesigned.
  • Do not confuse all visits with eligible test traffic. Audience rules, exclusions, repeat visits, allocation, and normal traffic variation can make the observed run time differ from the estimate.
  • A required-traffic figure is not a guarantee of statistical significance. Adequate sample size also depends on the outcome rate, effect size, variability, power, and analysis plan.

Common questions

Why did the combination count grow so quickly?

A multivariate design crosses levels rather than testing each variable in isolation. Two headline levels and three CTA levels make six cells; adding two image levels doubles that to twelve. Removing one level can therefore eliminate several combinations at once. Check that every proposed level is genuinely needed before accepting the resulting traffic demand.

Can I treat the duration estimate as a firm launch schedule?

No. It is a planning estimate tied to the traffic value and test structure you entered. Testing can take longer when only part of site traffic is eligible, allocation is uneven, traffic varies by day, or quality checks remove observations. Use the estimate to compare designs, then monitor actual eligible exposure after the multivariate test starts.

References and verification

The example and behavioral notes were checked against the browser implementation. Standards and primary references below define the relevant format, formula, or platform behavior.

Related Tools