A/B Creative Testing for Alexa Smart Home on amazon.com

Background: Teams across Amazon Devices, including Amazon Smart Home, tended to feature devices on plain backgrounds when merchandising on Amazon’s homepage. Customers are often confused by smart home devices, however, and require additional context in advertisements to increase relevance to them.

Insights: Quantitative and qualitative research indicates customers need 1) more appealing imagery in advertisements/recommendations 2) more visual context to help them understand how a device may fit in their homes.

Hypothesis: If we use contextual lifestyle imagery, rather than plain backgrounds, then customers will be more likely to click and purchase Amazon Smart Home devices from homepage.

Work:

Echo Glow A/B test

Amazon Smart Plug Promotion A/B/C test

Amazon Smart Thermostat A/B/C test

Amazon Smart Air Quality Monitor A/B/C test

Amazon Smart Soap Dispenser A/B/C test

Results and impact: Lifestyle imagery drove statistically significant positive results with a XX% win-rate, average +XX% increase in sales, and +XXXK incremental sales for Amazon Smart Home devices. These results influenced creative and tech team roadmaps to create templates to automate the process of creating lifestyle imagery across Amazon Devices. Usage of lifestyle imagery is expected to drive XXXK incremental sales in 2024.