Today’s shoppers prefer seeing ads that are so relevant they think, “Wow, I do want that product! How did they know?” To provide such an experience, marketers are constantly testing their ads and targeting strategies to identify improvement potential and gain actionable insights. But it doesn't stop there. Trying out new variants of the product data behind your listings is another stellar way to find out what resonates with your target customers.
It can be difficult to know where or how to optimize your product data in an effort to give customers a better ad experience. However, what we can do is measure the impact of the changes we make to product data. This is done through A/B testing.
What is A/B testing and why does it matter in feed management?
Does the placement of the brand name in a product title make a difference in CTR? Does a shorter description result in a higher conversion rate? Is one image more click-worthy than another? Even a change that seems minor or unnecessary could have a big impact on channel performance. This is precisely where A/B testing can help.
“We’ve found A/B testing for titles can result in up to 60% better performance in key metrics like return on ad spend and cost per click.”
A/B testing allows you to compare two different versions of your product content to identify which performs better. By making a change to a single attribute and sending two separate versions of your feed to the same channel, you can analyze the impact of that specific change. The test results can then be used to optimize your product feed based on real consumer response, driving better performance like a higher click-through rate, ROAS, or CPC.
Which changes should be tested?
The first step to A/B testing is deciding what to test. You’re not limited here so try to think outside of the box and get creative. Almost any change to your product feed can be measured. You will, however, want to focus on changes that could impact the user experience. While it may be difficult to see the value of change upfront, even small tweaks could lead to a boost in performance.
Try out some of these changes to get started.
Product title
- Add or remove product information like color, size, or brand name
- Change the placement of an attribute (e.g. move the brand name from the beginning to the end)
- Replace terms with alternative synonyms (e.g. high heels or pumps)
- Use more or less specific color names (e.g. yellow vs. sunflower)
Categories
- Add subcategories (Outdoors vs. Outdoors > Camping)
- Change the parent category (Home & Garden vs. Kitchen & Bath)
Images
- Show products from an alternative angle
- Increase the size of the product within the image
URLs
- Change link to a newly designed landing page
Custom labels
- Add new or change existing custom labels
The do’s and don’ts of A/B testing product feeds
Results from A/B testing can be used to achieve some incredible results - as long as the testing is done right. If the test isn’t set up properly, the results could be misleading or invalid. To avoid this, here are some best practices and things to avoid when preparing, running, and analyzing your test.
DO
- Get the duration right: Let your test run long enough to yield accurate results. The bigger your feed and the longer you should run the test. For example, this could be 1 to 2 weeks or until you’ve hit 1000 conversions.__
- Define a clear goal: To accurately measure success, you’ll need to set a benchmark. If you’re aiming for a higher click-through rate, define the value that you’d consider a success ahead of time.
- Be open-minded: The impact of certain changes may surprise you! A/B testing is all about trying out new things, so don’t be afraid to get creative.
- Test both variants simultaneously: Consistency is key in A/B testing. Each variant should be tested in the exact same environment to limit the influence of external factors.
- Use a tool: Find a tool that is capable of automating most of the workaround A/B testing. This will make it easier to make a simple change, export your feed variants, and measure results.
DON’T
- Test multiple changes at once: Testing multiple variables at a time will make it difficult to assess which change impacted performance.
- Give up after the first test: A/B testing is all about trial and error. The first change had no impact on performance? Learn from it and try something different.
- Forget about external factors: Product seasonality or existing weak product performance should not be overlooked when A/B testing. Consider all factors that may influence the test results.
- Spend time testing the obvious: Naturally, a product feed containing product images will perform better than one without.
A/B testing a product feed vs. split testing on Facebook
A/B testing and split testing are often used interchangeably. While they do share commonalities, they are not the same thing. As defined earlier, A/B testing is a method of testing the impact of a single change. Split testing, however, allows you to compare versions of ads, webpages, and landing pages that are drastically different from one another.
An example of a split test would be running two different versions of the same ad on Facebook to compare which performed better. Changes could be in design, CTA message, or even audience targeting.
A/B testing a product feed is different, as the testing is single changes to the content itself, rather than the ad structure, format, or design.
Here’s a quick sum-up of the difference between Facebook split testing and A/B testing product feeds.
How does A/B testing work on Productsup?
Productsup is a feed management solution with a built-in A/B testing functionality. Our platform allows you to tailor, optimize, and test your feed changes all in one place. Make a change in seconds, compare it with an unchanged version, use your findings to further enhance your feed’s performance - easy peasy! Here’s what the A/B testing process looks like for a Google Shopping feed.
Step 1: Setup (5 minutes)
To get started, you’ll need two separate, yet identical, product feed versions. Simply duplicate your existing Shopping feed with the click of a button and give it a new name. Next, you’ll need to define which of the feeds will be your main version or “variant,” which can be done, again, with the click of a button.
Next, you’ll apply a single change to the alternate version of your feed. This can be done in seconds using one of our 250+ built-in rules. Here's an example of what this looks like on the Productsup platform.
And just like that, you’ll be ready to run the A/B test!
Step 2: Run A/B test by exporting product feed
It’s now time to export your feed to Google Merchant Center. Based on the number of variants or feed versions that you set up, our algorithm will evenly divide your product feed into groups. If you’ve created and activated two variant feeds, 50% of your products will be sent in the original format and 50% will be sent to Google Shopping with the applied change.
Note: All products will be merged again before export and only one single feed will be submitted to Google Shopping.
Step 3. Analyze
Track clicks and conversions from each feed variant using Productsup’s tracking or your own. If you choose the latter, simply add tracking parameters to the product URL links in both variants. For example, add ?ab=1 to product URLs in the main variant and ?ab=2 to those in the alternative variant. This will allow you to easily compare performance between variants. From there, you can use the insight to further optimize your product feed.
A/B testing your product data doesn't have to be a complex task and can be the difference between a high and low performing product feed. When in doubt, test it out!
This article was originally published on August 27, 2019.
About the author
