App Store screenshot A/B testing: what the data actually shows

Apple lets you test three screenshot variants against your current page, Google lets you test two, and most visitors never scroll past the first one anyway. Here is what each platform actually allows, and what real published tests moved.

What each platform actually lets you test

Apple’s Product Page Optimization, run from App Store Connect, tests up to three alternate treatments against your current app icon, screenshots or preview video, shown only to people on iOS or iPadOS 15 and later. Only one test can run per app at a time, it can run for up to 90 days, and Apple recommends waiting for high confidence before applying a winner: its own published case studies cite results at 98% and 100% confidence, not the raw percentage alone.

Google Play’s Store Listing Experiments work differently. You get up to two variants against your current listing, and you can run one default experiment across icon, feature graphic and screenshots, or up to five localized experiments at once that can also test description text. Google splits your chosen audience percentage evenly across the versions and auto-stops the test after six months if you have not ended it already.

Where the attention actually goes

The reason the first screenshot matters more than any other element on the page comes from a SplitMetrics test written up by Sensor Tower: across the experiments it measured, the average screenshot scroll rate was only 17%, and roughly 1% of visitors read the full app description. The vast majority of people who land on a product page only ever see the first thing on it.

What real tests have moved

Apple publishes a handful of its own developer results. SplitMetrics, an A/B testing vendor, publishes fuller case studies with the specific variants tested. Lined up together, the lifts are real but modest, single digits to the high teens, not the multiples some marketing copy implies.

The Prisma sequence is the more instructive one because it ran three tests in a row rather than one. The first, a straight rewrite (moving text to the top, raising contrast, adding social proof), lifted conversion 12.3%. The second, four new imagery and filter variations, produced no significant change over the existing page. The third, which simply put a proven top image back in place and added a callout for a new video-filtering feature in the second screenshot, produced the 19.7% result. The losing middle test was not wasted; it is what told them the imagery itself was not the lever, the message was.

The test worth running before any other

Given how little of the page most visitors actually see, the highest-leverage single change on either platform is usually the first screenshot, not the fourth or fifth. It is also the cheapest to test: you are not shooting five new images, you are reordering or rewriting one.

Blog · Free ASO tools · Products · AsoTheory