Why your TOFU deserves more attention in experimentation strategy

TOFU – important for vegans and for web experiments

It's easy to make the mistake of focusing on macro-conversions only, as linking them directly to revenue seems easy. Let's look at digital sales: did you know that the highest impact comes from experiments that target goals early in the shopping funnel?

Think about all the steps your visitor must make to reach the checkout (a page that businesses tend to focus on optimizing) or cart page:

• Landing on a page with relevant content
• Search
• Adding items to the cart
• Register (or not)

And many others.

They may not be directly preceding the actual payment, but they may have a much higher impact on the conversion rate than you think. Examples?

Undervalued experiment goals

A recent Optimizely analysis of over 127,000 experiments conducted between 2018 and 2023 shows that Search rate increase is the most undervalued experiment goal. Even though it is used in less than 1% of experiments, it has the highest expected impact at 2.3%. It is important to note that users who search typically convert at 2x-3x the conversion rate of all other users.

Over 90% of experiments target 5 common metrics, image source: optimizely.com/the-evolution-of-experimentation

Sounds simple: we all know that the search functionality is now the most common navigation method for websites and apps. Yet still, due to its position at the top of the conversion funnel, its importance often gets overlooked.

Another good example could be the Add to Cart rate. Used only as a metric in 4.2% of experiments, the ones conducted with this one as a primary metric proved to have 4x higher impact than the ones aimed directly at revenue increase.

Experimentation across different industries

We may also look at examples beyond e-commerce.

Spotify, an avid user of Optimizely Experimentation, recently introduced a new feature aimed at increasing user engagement. The company's goal was to measure the impact of this feature on various engagement metrics, such as the number of plays, time spent on the platform, and the number of tracks added to libraries.

They ran a series of tests using Optimizely Feature Experimentation. Users were randomly divided into groups, each seeing a different version of the new feature. Spotify tracked various engagement metrics (e.g., adds to playlist and engagement time) to compare the impact of varying feature variants. The one that proved to be the most effective led to a 20% increase in the number of plays, a 15% increase in time spent on the platform, and a 10% increase in the number of tracks added to libraries. All these metrics, despite not being at the end of the conversion path, contribute to Spotify's end goal: revenue.

Remember, shifting focus from end goal to steps (and metrics!) happens before conversion pays off.

Could you use an extra brain?

Working with our clients, I saw that the tool is only a fraction of a successful experimentation plan. The key to success lies in a well-designed team that can communicate and understand the purpose of each task. The dream team combines analytical insight, strategic vision, and technical implementation expertise.

This precise setup allowed the experiments to be very successful. Yet, companies rarely have all this talent permanently onboard. Often, they rely on a single person or a small team with other tasks to perform – and as a result, the Experimentation tools are left underutilized and underperforming.

Is this something you have been experiencing in your organization, too? If so, we are here to help. Myself with my team of Optimizely certified developers and consultants would love to talk to you about your experimentation strategies and challenges and see if we can be the extra brain-for-hire you may sometimes need to outsource the burden of experiment planning or executing to.

Schedule a call with us here.