Multivariate testing splits traffic and presents variations of the interface to evaluate how each variation affects behavior. Generally one or more key metrics is used to evaluate success, such as conversion rates. Multivariate testing can evaluate a combination of variations on a product to determine the most effective design.
NOTE: We don't support this method on our platform just yet, but stay tuned!
When the method should be used:
Multivariate testing has to be conducted on a live product such as a website, app or piece of software. When you’re considering making changes and have multiple aspects of change and several variations, multivariate testing will enable you to evaluate how each performs against your desired metrics.
Which combination of features achieves the best results?
Which layout leads to highest conversions?
What difference does it make if we use a different layout?
Evaluate several design variations at once
Clearly link design changes to key metrics of interest
Provide understanding of how elements of the design work together
Large volume of traffic required to evaluate different variations
Users may get confused if they return to your product and have a drastically different experience to their last use
Where several variations are being tested, analyzing results can become complicated
