Creative Testing enables you to optimize your Dynamic Playable by gathering data about which permutation performs the best with a determined metric.
What happens during a test?
During the test, all possible permutations of the Dynamic Playable are run simultaneously, and the resulting data is evaluated to identify any significant changes in the targeted metric.
If a noteworthy change is detected, the test is paused and the winning permutation is declared. This successful permutation is then designated to receive all potential traffic.
Regardless of statistical significance, we make sure that all permutations receive over 100 impressions before making a decision.
If the impression cap is reached without any statistical significance, the test is paused and the default permutation is set to receive all potential traffic.
How to set up a test?
💡 TIP
Before creating a test, we suggest following this process:
Upload the playable to your chosen ad network
After your creative has been approved by the ad network, move forward with creative testing
Step 1: Create a new test
On the Creative Testing page, click on the "+ New Test" button located in the top-right corner. A form will pop up, prompting you to fill in the necessary details and settings for your new creative test.
Step 2: Choose your settings
In the form, you will need to provide the following information:
Test Name: Give your test a name that will allow you to easily identify it in the future.
Dynamic Playable: Assign the dynamic playable you want to test. This will appear in the "Playable Preview" container.
Permutation (read-only): The number of unique permutations of your dynamic playable.
Impression Cap: The number of impressions the test can reach before it stops.
The impression cap refers to the total number of impressions permitted for the entire test. This cap will be divided equally among the segments, so it's important to ensure that your cap accommodates all possible combinations and segments.
Optimization Target: Select the optimization target that you want to focus on.
You can choose from several options:
Target Option | Description |
CTR (Click-Through Rate) | This option optimizes towards the number of app store clicks your creative receives, and is calculated as the number of clicks divided by the number of impressions. |
IPM (Installs per Mile) | This option optimizes towards the number of app installs you receive per thousand impressions.
Note: This target is only available on ironSource |
Engagement Rate | This option optimizes towards the engagement rate your audience has with your creative. |
Completion | This option optimizes towards how many times your creative has been completed. |
Model Segment: Choose the model segment that you want to target.
Every segment is considered as an individual test, and each of them can have their own set of winning candidates. You can select from the following options:
Segment | Description |
None | This option means that you are not targeting any specific model segment and all impressions are treated the same |
Operating System (OS) | With this segment, you can target permutations based on the operating system (iOS or Android) |
Ad Network | With this option, you can target permutation based on a specific ad network. |
Ad Network+OS | With this option, you can combine the OS and Ad Network segment values. |
Step 3: Activate the test
After you have filled in all the necessary information, click on the "Save" button to save your creative testing. This will not start your test automatically. You will have to navigate to the test and click on the " ▶ Start" button to set your test Live.
Be careful!
When you create a new creative testing with a dynamic playable that is already in another live test, the new test will go live and all other tests with this playable will be paused.
Here is a list of the possible statuses of your creative tests:
Test Status | Description |
Not Started | The test has not been started. |
Live | The test is live on one or more segments. |
Paused | The test is paused. |
Finished | The test has concluded and a winner has been determined |
Inconclusive | The impression cap was reached before a winner was determined. |
Next Steps
After initiating your creative test, you need to wait for the results to generate. Please click here to gain a comprehensive understanding of your results.