Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. When it comes to continuous discovery, there’s no such thing as “the perfect tool.” Instead of seeking the best tool out there, it’s much more effective to look for the best tool for your team. Teresa often says, “The best tool is the one that your team will use.”
When it comes to continuous discovery, there’s no such thing as “the perfect tool.” Instead of seeking the best tool out there, it’s much more effective to look for the best tool for your team. – Tweet This
That’s why we started the Tools of the Trade series. Our goal is not to give you a blueprint for how to do discovery, but rather offer some inspiration for how different teams are handling common discovery tasks.
In previous posts in this series, we’ve looked across the tech stacks of individual product teams to get an overview of how they use tools to support all of their continuous discovery habits.
Today, we’re taking a slightly different approach. We’re zooming in on one aspect of discovery: automating the recruiting process for customer interviews. We’ve spoken with several continuous discovery champions to understand how they streamline and simplify recruiting and which tools they use for these tasks.
Step 1: Decide Who You Need to Recruit and Where to Recruit Them
The first step is to understand who you’re hoping to interview. Each team has its own needs when it comes to selecting interviewees. You may want to speak with existing customers or users. You might want to speak with former or prospective customers. Or your goal might be to connect with people who fit a particular demographic or behave in a specific way when using your product. Once you’ve defined who you’re hoping to speak with, it will be much easier to choose the tool that will help you to reach those particular people.
Once you’ve defined who you’re hoping to speak with, it will be much easier to choose the tool that will help you to reach those particular people. – Tweet This
Here’s a brief introduction to the continuous discovery champions we spoke with and how they defined their ideal interview participants.
- Mohamed Aboshihata is the Head of Product at MediaZilla, a startup that enables studios to package, distribute, and sell event videos. Mohamed’s team runs continuous discovery with different customer segments.
- Benedikte Holm is currently the CPO at Findable, a startup working to make facility management more sustainable by digitizing building documentation. Since she’s new to this role, she shared her experience based on her previous role as VP of Product at OwnersRoom, an investor and funding platform, where her team focused on interviewing customers.
- Andy Hugelier is the Senior Director of Product Design at Opal. Andy recruits both from within his customer base and external participants.
- Dan Clem is the Senior Product Manager at Zonar Systems, a B2B company that provides hardware and software solutions for commercial vehicle fleet operators. His team does research with existing users to help identify opportunities for new business directions.
- Janis Billepp is a User Researcher at Isha Foundation, a non-profit that has initiated a range of social and environmental projects. Janis says that before starting continuous discovery interviews, his team needed to clarify who their target user was. They decided to differentiate between meditators (who have learned Isha’s core meditation practice) and non-meditators (who have not). While they interview both groups, they chose to focus on non-meditators since they make up the majority of their users.
- Matt Richmond is a Senior Product Manager at 99designs by Vistaprint, a two-sided marketplace and a global creative platform for professional designers to find and do work online. Matt and his team were trying to recruit both Clients and Creators on both sides of their marketplace.
Step 2: Choose the Tools That Work Best for Your Target Audience
The tools you can use to identify and screen interview applicants tend to fall into two main categories: tools that are primarily designed for communicating with your customers and tools that work for a broad audience (not necessarily your existing customers). Tools that fit into the first category (customer communication) are often already being used by your customer-facing teams while the second set of tools (reaching a broader audience) allow you to tap into a larger existing community that’s aligned around a shared interest such as on Reddit or YouTube. Let’s look at each in a bit more detail.
The tools to identify and screen interview applicants tend to fall into two main categories: tools primarily designed for communicating with your customers and tools that work for a broad audience. – Tweet This
When You Need to Recruit Your Own Customers
If you’ve already determined that you want to speak to your existing customers, you might want to use a tool that’s specifically designed for customer communication.
Mohamed at MediaZilla wanted to conduct continuous discovery interviews with a particular segment of users. He chose customer communication tool Intercom because it allows him to target users and display pop-up notifications based on where they are in the app or what they are doing. For example, if a user visits a specific page or takes an action like uploading a file, they will see a pop-up notification inviting them to participate in an interview. There’s a lot of flexibility in Intercom, so it’s possible to use both email and popups or be on a chat window. You can also set a Series to automate a sequence of communication.
He says a secondary benefit of Intercom is that it’s easy to tag people who have participated in an interview in case you’d like to reach out in the future to follow up with them. It’s also easy for anyone on the platform to share the product team’s interview calendar.
This diagram illustrates how MediaZilla uses different tools to set up their continuous discovery interview recruiting process. If no one responds to the first email, they send out a second email five days later.
Mohamed also cautions that while it’s easy to set up Intercom, you have to be careful about interrupting the user experience with your app or other email sequences for onboarding. He also adds that email may not be effective for your users and may lead some users to unsubscribe.
Benedikte also used Intercom at OwnersRoom. “Intercom is a nice tool to start a conversation with customers from inside your app, which means you can catch them when they are engaged with your service and therefore more interested in replying to you,” says Benedikte. “You can choose between displaying banners, large pop-ups, or small chat windows and you set rules for when and where these should be displayed.”
Benedikte would set a rule for a message to pop up in the bottom corner when a customer navigated into a specific page in the app, so that she could target her question to something they were actively looking at. She would also set rules for who she targeted, such as excluding users with mail addresses from her company’s own domain.
Depending on specific page visits users made in the app, she would launch different surveys through integrations with Typeform and Survicate. While both of these tools are very similar to each other, they each have their pros and cons, says Benedikte. For example, at the time when Benedikte was using these tools, Typeform limited the number of questions and monthly replies, while Survicate allowed unlimited responses, but you could only run a survey for seven days. “I particularly like that Survicate has a few components that let the user reply directly via buttons in Intercom without having to launch the survey separately. Less clicks rocks,” says Benedikte.
For booking interviews, Benedikte first ran a Sean Ellis survey from Intercom using the Survicate integration. She launched this survey to every paying customer that had signed up within a certain timeframe and who had visited the app recently. Within a few days, Benedikte had lots of replies. She then looked at the top results of that survey and emailed those people directly to ask for an interview. In her email, she would suggest a slot and also include the option to pick a slot that worked better for them via Calendly.
When he needs to recruit from within his customer base, Andy from Opal uses Great Question, an all-in-one customer research platform. “Great Question has been a fantastic tool for outreach and recruiting because it leverages a suite of sign-up options (landing pages, email templates, links) combined with a built-in scheduling tool to make the whole process frictionless for our users,” says Andy.
He adds that Great Question has quickly become their tool of choice for most research studies, given the fantastic combination of recruiting, testing, and data repository capabilities. “It even helps manage the logistics for studies where we need to leverage other tools like Maze or Typeform for specific needs, without having to port user data into multiple systems. We used to stitch together a variety of specific-use tools to achieve these flows, and this has helped us reduce the complexity in our research tool stack.”
Targeting Smaller Segments of Customers
If you know that you’re targeting a very specific type of customer or user, you can choose a tool that layers on top of your app or website. While this category is similar to the tools we looked at earlier, these tools tend to be designed specifically for user research rather than customer communication.
At Zonar Systems, Dan’s team wanted to hear from as many users as possible. They were already planning to send out a survey to users using Pendo, a product adoption platform that provides analytics, feedback, and in-app guides to users. Dan says, “I proposed we create a similar Pendo pop-up in-app that’s linked to a Microsoft Bookings page that lets users schedule their own interviews based on the team’s availability.”
Dan chose this particular workflow because Microsoft Bookings allows users to schedule their own appointments and it was included standard and integrated into their calendars, so they could ensure there was availability when a user scheduled an appointment. “If this worked,” says Dan, “we could repeat it again and again.” You can read all about how Dan and his team set up Pendo to automate recruiting (in about an hour!) here.
Pendo allows you to create in-app pop-ups like this one that match the look and feel of your brand.
Dan says there are many pros to using Pendo:
- Pendo segments are robust and give you tons of flexibility to recruit users who are relevant to your user research goal. This could be a product behavior like activity across your platform, a feature, or internal targeting data that could describe your customers like a vertical.
- Guides—what you’ll use to build a recruitment pop-up—are highly configurable so it’s easy to match your company’s look and feel, but more importantly, to quickly create variants to maximize conversion.
- Based on your targeting criteria, you’ll see how many users will see the recruitment pop-up, which helps you understand how long it may take to recruit the number of users you’re looking for.
However, Dan cautions that there are a few cons to using Pendo as well:
- There isn’t an out-of-the-box way to see how many different Guides a user may see, which can be distracting or just annoying. “We’ve seen as more teams adopt this process to get interviews, we need to coordinate to not overwhelm our users with Guides not only for interviews but for onboarding, surveys, etc.,” says Dan. “With Pendo, there isn’t an easy way to throttle the max number per user so you’ll need some manual work for this.”
- Some users may want to opt out of interviews, especially in B2B, and there isn’t an easy way to keep this list. “Yes, there is a way to exclude users from seeing Guides, but this can impact every Guide a user sees, which is heavy-handed when they just don’t want to see interviews.”
- Guide actions aren’t well instrumented or integrated into segments. Dan explains, “If you want to do intelligent targeting like ‘Remind me in seven days’ or exclude all users who have selected ‘dismiss’ from a recent Guide, it can be challenging.”
When Matt and his team at 99designs began their continuous discovery journey, they were trying to recruit both Clients and Creators on both sides of their marketplace, so they chose to use SurveyMonkey, a free customizable survey tool. Matt says this was an easy option because, “We have a persistent link built into our product at the end of completing a design project. This is our source of NPS data in addition to qualitative data, and we were just using it as a recruitment source for users who had opted in. These surveys were already live and had a UX Research opt-in question at the end, which allowed us to easily email only those respondents who were interested.” You can read more about how Matt set up the MVP version of his continuous discovery tech stack here.
While Matt at 99designs started out using SurveyMonkey, his team later switched to user research and customer feedback tool Qualaroo. They’re now using a Qualaroo poll at the end of a project to get either side of their marketplace to opt in via a survey and book a time for an interview in Calendly. Matt points out that Qualaroo is just one solution and you can present users with an in-app message through many different products (as you’ve seen in this blog post!).
The product team at 99designs started using Qualaroo to send quick polls to both Clients and Creators in their two-sided marketplace, depending on their research focus at any given time.
According to Matt, the pros of Qualaroo include:
- No more manually emailing users to book a meeting—they can schedule it themselves in Calendly from the link in the Qualaroo poll.
- You can present the poll to users in the context of the product where they are more likely to engage.
- You can turn polls off and on with the click of a button if you’re getting too much or too little interview booking volume.
These stories showcase several of the tools that you can use to communicate with your existing customers. Again, there’s no single best option here—you’ll want to consider which tools your company is already using, what works with your budget, and what will be easiest to fold into your existing processes.
When You Need to Reach a Broad Audience
When you’re trying to reach a broad audience, you might opt to use a tool that will help you tap into an existing community and find interview participants from outside your customer base.
When you’re trying to reach a broad audience, you might opt to use a tool that will help you tap into an existing community and find interview participants from outside your customer base. – Tweet This
Janis from Isha Foundation was focusing his research on one particular app, Sadhguru, which offers yoga and meditation guidance as well as a self-guided course focused on personal growth called “Inner Engineering.” As part of his goal to identify target users for the app, Janis was trying to reach the largest and most diverse audience possible. He created a YouTube community post (a social media post that went out to everyone who follows Sadhguru on YouTube). It included a simple blurb: “Do you have a few minutes to tell us about your journey with Sadhguru, so we can create what matters to you?” and a link to a Google Form, a customizable survey which they set up to include screening questions to identify potential interviewees. Janis says the advantages of using Google Forms are that it’s easy to set up and you’re allowed to collect unlimited responses.
A YouTube Community post catches users’ attention and funnels them to a Google form where they can fill out a survey.
Once someone completes the survey, they’ll receive an email response sent from email platform MailChimp confirming their survey has been submitted. If respondents qualify to participate in an interview, they’ll send another email via MailChimp to invite them to schedule a call.
When recruiting external participants for interviews at Opal, Andy relies on UserInterviews, a service that connects researchers with participants. Andy says, “UserInterviews has been immensely helpful for its wide pool of professional characteristics we can build a target on, and their recruiting service has always managed to find us plenty of candidates to vet.”
In both of these stories, the team is choosing a tool that can help them get in front of their target customer. If you are looking to reach folks outside of your current customer base, you have a lot of options. You can run ads, you can post on Facebook, LinkedIn, YouTube, etc., or you can use one of the many recruiting platforms like UserInterviews. The key to making this work is to get clear about who you want to recruit and then go to where they spend their time online.
Step 3: Continuously Evaluate What’s Working and Iterate Over Time
Like any other aspect of continuous discovery, it’s helpful to take an iterative approach. If you have to choose between getting started sooner with a less-than-perfect process or waiting until you’ve got everything set up perfectly, choose the option that gets you started sooner. You should expect to learn from your early experiments and make improvements to your process based on what you’ve learned.
If you have to choose between getting started sooner with a less-than-perfect process or waiting until you’ve got everything set up perfectly, choose the option that gets you started sooner. – Tweet This
Matt from 99designs is a great example of this. He shares how his team came up with an MVP version of continuous interviewing while already anticipating future iterations and improvements. You can read about Matt’s story in detail here.
Here are a few of the lessons our continuous discovery champions shared from their experiments with automating interviewing.
- Weigh the pros and cons of free tools
Working at a non-profit and trying to reach a large audience means Janis often relies on free tools like Google Forms and MailChimp. Janis says MailChimp allows them to send 2,500 free emails per month and they’re able to personalize the “to” field. However, the new editor is still in beta and it can be a little cumbersome to create aesthetic designs. They’re also only allowed to have 500 subscribers total, so this is somewhat limiting.
- Be thoughtful about how you’re communicating with your customers or users
“Emails aren’t very effective for many folks,” says Mohamed, “so we really optimized this by mixing emails and popups. Luckily Intercom has a Series feature that, based on some rules, will trigger emails and popups.” Mohamed has found the following approach to work well at MediaZilla: First they segment users who are post-onboarding flows and engaged. Next, they create a Series in Intercom that triggers an email. If the email isn’t opened, they follow up with another. If they still don’t get a response, they display a pop-up in app. They make sure all users in the sequence get tagged in Intercom so they can later target or exclude. Finally, based on the results from this sequence, they improve or introduce incentives.
- Learn from the data you have access to
Depending on the tool you’re using, you’ll get access to different types of data, such as how many people see or click on your pop-up or how many survey responses you’re getting. “This makes it easy to tweak copy, layout, and even where/how the guide appears in your product to increase your conversion,” says Dan.
- Schedule interviews as soon as possible
Matt finds that if there’s too long of a window for users to book into interviews (i.e. more than two weeks in advance), they often forget about the interview by the time it comes around. This is why he recommends scheduling interviews as soon as possible.
Final Thoughts: Getting Started is More Important Than Selecting the “Perfect” Tool
We hope this post has shown you that there’s no such thing as a perfect tool. There are pros, cons, and limitations to every tool out there, and these may change depending on the size of your team and the maturity of your continuous discovery practice.
What matters more than trying to select the “perfect” tool is to choose the tool that will allow you to get started with your continuous discovery habits sooner rather than later.
Hoping to hear how other teams are taking small but significant steps to implement continuous discovery habits? We regularly cover topics like this in our Continuous Discovery Habits community. You should join us!
