This guide provides an explanation of the process used to select creative test results. For further information on creative testing, please click here.
How does a winner get selected?
After launching the creative test, all permutations within the assigned dynamic will be live and eligible to receive impressions; this will continue until a decision is made or the impression cap is reached.
Please note, we make sure that all permutations receive over 100 impressions before making a decision.
If a permutation yields data with statistical significance, it will be considered the "winner" of the creative test.
At this point, the test will be paused and the winning permutation will be selected. This successful permutation will then be set to receive all potential traffic, and a "crown" icon will be displayed next to its permutation ID.
When is a test considered inconclusive?
If the impression cap is reached without any statistical significance, the test is considered inconclusive so it gets paused and the default permutation is set to receive all potential traffic.
Important!
The Impression Cap does not include the impressions generated by the default permutation.
The default permutation is displayed when the playable fails to connect to our servers due to issues such as limited internet connectivity.
Inconclusive segments in a test
The creative test can still be completed if one of the segments is inconclusive.
For example, if we have a test with four segments, the test can still find the winner permutation for three of the segments and result inconclusive for the fourth one as having no clear winner.
Important!
Every segment is considered as an individual test, and each of them will have their own set of winning candidates.