CASE STUDY: A/B Test – Products AI proves its power with a 66% increase in clicks and a +14% CTR
About Aaiko
- A Dutch fashion label that aims to combine creativity with commercialism uniquely. Specialized in women’s fashion.
- Based in Amsterdam, Netherlands.
- Founded in 2005
- https://aaiko.nl/
About Elevate Digital
- Dutch digital marketing agency that specialises in branding and performance marketing. Elevate Digital has a strong focus on the fashion and lifestyle sectors, tailoring their marketing strategies to meet the unique needs of these industries
- Founded in 2011
- Based in Utrecht, Netherlands
- https://elevatedigital.nl/
About Producthero
- Software vendor for Shopping Ad tools
- Comparison Shopping Service (CSS Google Partner)
- Headquarters in Amsterdam (The Netherlands)
- Producthero.com
The Challenge: improve CTR and drive better performance.
Elevate’s main objective was to chat by aligning product titles more closely with user search terms, ultimately driving better performance.
Approach: A/B test
To evaluate the effectiveness of Products AI optimisations, Elevate conducted an A/B test. The primary objective was to measure the impact of AI-optimized product titles on fashion items, a category where customer preferences are heavily influenced by search relevance and trends.
To ensure the accuracy of this test, they selected products that were comparable in type, price range, and historical performance. The products were divided into two groups:
- A test group (50% of the fashion products)
- A control group (the remaining 50%)
Both groups were balanced based on key performance metrics, including clicks, impressions, and CTR. Additionally, Elevate considered product pricing and the number of products going on sale during peak seasons.
The test was structured to isolate the impact of AI-optimized attributes on the test group while keeping the control group untouched for comparison, ensuring a clear and reliable comparison.
Implementation
Once they selected all the products they wanted to optimize, they adjusted the different optimisation settings, including:
- Template: Apparel
- Language: Dutch
- AI Creativity: Creativity control was set to a medium level to balance originality with relevance.
- Customized Prompt: The AI was configured to incorporate as many trending keywords as possible in the product titles, aligning with current search behaviour.
The setup included clear categorization, creativity parameters, and the use of custom labels to differentiate the test and control groups.
A custom label was applied to the test group to ensure only those products were optimized, allowing precise tracking of the results.
Results
The A/B test took place from the 1st of December until the 10th of January. During this period, the following results were achieved:
| Metric | Insights |
| Clicks | The test group generated 66% more clicks than the control group. |
| Impressions | The test group achieved 45% more impressions than the control group. |
| Click-Through Rate (CTR) | The optimized group had a 14% higher CTR, indicating better relevance and engagement. |
| Distribution of products | Balanced product distribution ensures reliable comparison. |
The A/B test compared the performance of two groups of products in Google Shopping:
- Test Group (Optimized attributes): Products with improved attributes.
- Control Group (Default attributes): Products with standard attributes.
Performance Insights:
- Engagement:
- The test group’s significant increase in clicks (+66%) highlights the positive impact of optimized product attributes on customer engagement.
- Visibility:
- The test group achieved 45% more impressions, suggesting that optimized attributes improved the products’ competitiveness in auctions, possibly due to higher relevance scores.
- Product Distribution:
- The near-even split of products (49.4% test vs. 50.6% control) ensures the test was conducted fairly, without skewing results toward one group.
Next Steps
After reviewing the results, Elevate outlined the next steps for further enhancing Aaiko’s campaigns. Elevate developed a three-step plan to drive continued growth:
- New tests: after this first round of optimisations, Elevate wants to expand the testing to additional product groups to assess the scalability of the optimization process.
- Zombie revival: Focus on underperforming (zombie) products by applying similar AI-driven optimization techniques to increase their visibility and sales.
- Hero Optimization: Maximize the potential of high-performing products by further refining their titles and keywords to maintain their competitive edge.
In the next phase, Elevate is also planning on optimising the control group using AI to broaden Aaiko’s overall SERP visibility. For new product launches, the same prompts and settings can be reused to ensure immediate optimization, perfectly aligning with Aaiko’s brand as a fashion-forward company.
Client testimonial
“Elevate Digital leveraged Products AI to optimize Aaiko’s product titles, resulting in a 66% increase in clicks and a 14% higher CTR compared to the control group. This A/B test demonstrated the significant impact of AI-driven optimization on fashion product visibility and engagement. We highly recommend Producthero’s Products AI for any e-commerce business aiming to enhance their Shopping Ads performance.”
