A/B tested: Smart shopping versus regular shopping campaigns

Producthero had the opportunity to split test Smart Shopping campaigns together with a group of advertisers. We enabled several advertisers to run a campaign experiment to answer the question: do Smart Shopping campaigns outperform regular Shopping campaigns? The results differ between advertisers, but in almost all cases the smart shopping campaigns outperformed the regular shopping campaigns.

This article is created with the help of Producthero Labs colleague Lize Braspenning

Since the start of Smart Shopping Campaigns (SSC) it has been the center of many discussions between PPC specialists. Some are very pleased with the results and the fact that it saves a lot of time. Others are skeptical and doubt the incremental value of Smart Shopping Campaigns over a well setup regular shopping campaign, especially because one has to give up almost all control and insights.

The test cases

We enabled a group of large advertisers in several industries to test Smart Shopping against their regular shopping campaign setup. It has been tested in a 50 / 50 traffic split between their regular Shopping campaigns and a new Smart Shopping campaign. For the test we used a Smart Shopping Campaign Experiment which is (at the point we did the tests) only available at request.

Looking at the setup of the regular shopping campaigns, we can say that the original campaigns were set up by experienced agencies or inhouse teams and were well built and optimized. All regular shopping campaigns had a setup with a branded / non-branded search term split, using the waterfall principle. None of the campaigns were limited by budget. At least all non-branded campaigns were optimized by Google with an automated bid strategy (target ROAS) and some branded campaigns used manual CPC (to push visibility of even more on branded search terms).

The average test period was 5 weeks (excl. learning period and conversion lag). The test duration also depended on the volume of the advertiser.

The results

Revenue:

In 7 out of 8 cases (88%) we saw that the Revenue was higher with Smart Shopping campaigns. The revenue increase (for all campaigns that also delivered a better ROAS) varied from +7% to +22%.

ROAS:

In 7 out of 8 (88%) cases we can say that the ROAS was significantly better or could potentially be better if the target would be adjusted. In only one case we saw the ROAS was significantly lower with Smart Shopping.

In 5 out of 8 cases (63%) the ROAS of the Smart Shopping Campaigns was better. The ROAS improvement varied from 7% to 85%.
In 2 cases (Case 7 and 8) the ROAS was a bit lower (3% and 4%). But the increase in revenue was much higher than than the ROAS drop. So we expect in this case the merchant can still achieve higher revenue when aiming for a higher ROAS with their Smart Shopping Campaigns.
In 1 case (Case 2) the ROAS of the Smart Shopping Campaign was much lower (10%) than the regular shopping campaigns.

The role of display

An interesting question is where the extra volume comes from. In contrast to regular shopping campaigns, Smart Shopping Campaigns also shows display ads in the Google Display Network, Youtube and Gmail. Can the uplift be explained by the addition of this extra channel?

In none of the cases the uplift fully came from display. It differs very much. In 3 from the 7 cases with significant revenue uplift, around half of the uplift came from display. In the other cases display did not add much extra revenue. So the revenue uplift came from shopping ads on search result pages.

Overall conclusion

In this test we let a group of large and savvy agencies/advertisers test regular versus Smart Shopping campaigns. We can say this test has shown that in most cases Smart Shopping campaigns perform better. How much better differs very much per advertiser. In some cases the addition of display partly explains the uplift, but in all cases most of the uplift can be explained by the Smart Shopping algorithm that seems to advertise the products more aggressively and more effectively. Even compared to using an automated bid strategy (target ROAS) with regular shopping campaigns.

The cases

Is smart shopping the holy grail?

Not for all, but for most advertisers Smart Shopping campaigns work better than regular shopping campaigns. But the loss of control is a downside that holds back several agencies and advertisers to switch.

Besides that we have found that:

50% of the costs go to products that are underperforming: they cost the advertiser money
80% of sales come from less than 10% of products
More than 60% of the products are ‘sleeping’: they hardly get any impressions and clicks

This year we have done several tests with approaches to get more control and further improve the performance of Smart Shopping campaigns. This ended up in a solution that gives the advertiser more control and reduces the inefficiencies. Read this article to find out how it works: Producthero Labelizer: forcing Google Smart Shopping algorithm to be more efficient.