Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. You’ve read the Lean Startup. You’ve drank the Kool-Aid. You’re ready to start experimenting.
There’s only one problem. You don’t have any web traffic.
If the only tool in your toolbox is A/B testing, then this might be a problem. Fortunately, there are dozens of other tools that you can and should use.
How To Know How Much Traffic Is Enough Traffic
Start by formulating a good hypothesis.
Suppose I’m trying to optimize my about page here on Product Talk. I read recently that adding testimonials will increase email subscription rates and I want to try it out. So my hypothesis might be:
- Adding testimonials to my about page will increase email subscription rates by 25%.
Using a duration calculator, I find it’s going to take more than 100 days to run this test. My about page doesn’t get enough traffic to run this test in a timely fashion.
When You Don’t Have Enough Traffic
Fortunately, I have other options. Relying on the four levels of product analysis, I have to ask myself, “what am I trying to learn?”
Finding Problems Worth Solving: Testimonials add social proof. I might be trying to learn if using social proof is valuable to my readership. I can do this by interviewing product managers. I might ask them:
- Tell me about a blog that you recently subscribed to.
- How did you learn about that blog?
- Why did you decide to subscribe?
The answers to these questions might provide some clues as to how valuable social proof might be.
For example, if people are getting referrals to blogs from people that they trust or if they mention that testimonials on other sites influenced their decision to subscribe, then I might be encouraged to move forward.
Identifying the Right Feature: Testimonials are only one way of adding social proof. As an alternative, I might add a Facebook widget that shows people which of their friends have liked my blog. This is another form of social proof.
Or I might include how many blog subscribers I have to let people know that my content is popular with people like them.
How do I know which is the best option?
Since I don’t have enough traffic to A/B test, I might create some paper prototypes (mixing and matching the different options) and head over to the next product meet up to get some live user feedback. I could ask people to think aloud as they read my about page.
Optimizing for the Best Design: Suppose I learn that testimonials do have the biggest impact. Now I need to know what the optimal design is.
Should I include the testimonials at the top, at the bottom, on the side throughout? How many should I include? Should they be specific? Should they be general? There are dozens of design decisions and some may convert better than others.
This is where being able to A/B test would come in handy. Since I don’t have enough traffic, I might want to invest in some paid search ads to temporarily boost my traffic to this page.
Or I might learn from those who came before me and gain inspiration from the about pages of other successful blogs. If I take this approach, I want to focus on blogs that have a similar readership and that have a high email subscription rate.
Or I could go back to paper prototyping. I won’t get statistically significant results like I might from an A/B test, but I’ll still learn about which testimonials get noticed, which ones resonate, and which motivate action.
Staying in the Realm of Possibility: Finally, I might need to assess whether or not the best solution and design are feasible. If I learned that a large number of specific testimonials perform best, I might need to focus on collecting testimonials before I can implement this change.
As the saying goes, if all you have in your toolbox is a hammer, suddenly everything starts to look like a nail. A/B testing is only one tool in your toolbox. You can and should be using many more.
Here are the key takeaways to remember:
- Start by formulating a good hypothesis.
- Use a duration calculator with your current traffic to determine for how long to run the test.
- You can get answers with interviews, paper-prototyping, and competitive research.
- If needed, you can boost your traffic with paid search ads.
Don’t let a lack of traffic keep you from learning what you need to learn.
Want more tips on how to develop products with an experimental mindset? Subscribe to the Product Talk mailing list.
