Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. Continuous discovery is about changing your behavior. That’s why Teresa named her book Continuous Discovery Habits.
You don’t just think your way into continuous discovery. Sure, it’s important to understand the underlying concepts. But the concepts don’t mean much if you’re not making real changes to the way you work—whether it’s how often you talk to customers, what you discuss in your conversations with them, or what you do with those findings.
You don’t just think your way into continuous discovery. Sure, it’s important to understand the underlying concepts. But the concepts don’t mean much if you’re not making real changes to the way you work. – Tweet This
And this is why it can be helpful to consider the tools that will best support you in building your continuous discovery practice.
Let’s be clear: The tools alone won’t automatically make you better at discovery. Tools are no substitute for the hard work of setting up your product trio, speaking to customers weekly, setting a clear outcome, or mapping the opportunity space. But they can help make your life easier as you adopt these continuous discovery habits.
Like many aspects of continuous discovery, there’s no single right way to build your tech stack. Since there are countless ways of approaching this, we’re kicking off a new series, Tools of the Trade. In each edition of Tools of the Trade, we’ll speak with one continuous discovery practitioner to learn about their experience. What tools are they using? How did they choose them? Do they have any insights or lessons they can share with the rest of us?
For this first post, we spoke with Rebecca Klee, a UX Researcher at Optimal Workshop.
Please note that this post is intended to be educational and does not constitute an official Product Talk endorsement of any of the tools that are mentioned.
Meet the Continuous Discovery Champion, UX Researcher Rebecca Klee
Rebecca’s company, Optimal Workshop, makes a suite of UX research tools primarily focused on helping researchers, designers, and other product folks with a few key tasks: evaluating and improving how things are labeled and organized, running quick surveys, and collaboratively capturing and analyzing notes from interviews or usability tests.
“As a UX Researcher, I have the rather meta role of supporting our product and marketing teams to better understand the way research is carried out in other organizations and the needs, challenges, and desires of people who do research in these environments,” says Rebecca. Her work helps Optimal Workshop understand how to improve their existing tools and identify opportunities that they might tackle in the future.
Whether she’s embedded in a product team or positioned as part of an “internal agency,” Rebecca says, “I always aim to get the other squad members involved in as much of the research process as possible.”
When it comes to continuous discovery, Rebecca says she imagines her scenario is pretty common: “There’s aspects of continuous discovery where we have pretty mature practices and other areas where we know there’s room for improvement!”
Here’s a quick overview of how Optimal Workshop is approaching continuous discovery:
- They’re prioritizing frequent touch points with customers across the product development process, both from the perspective of discovery research to understand potential opportunities and in the form of evaluative activities to understand if their proposed solutions work as intended.
- Because their customer base is made up of people who already understand and advocate for the value of research themselves, they are largely engaged and open to participating in Optimal Workshop’s research activities. “Although I wouldn’t claim that every team interacts with customers on a weekly basis, over the course of a year, each product squad would generally be involved in at least a couple dozen customer sessions,” says Rebecca.
- The area where Rebecca sees an opportunity to strengthen their practices is in consistently and confidently prioritizing opportunities based on desired product and business outcomes. “Something that has hindered us in the past is not necessarily having our product data instrumented in the best way to inform and support these decisions,” she explains.
- Another area where Rebecca sees room for improvement is they have previously fallen into the trap of jumping from a desired outcome to potential solutions and narrowing their focus on a particular solution too early in the evaluative phase. A way they are now addressing this is by honing their experimentation approach so they’re focusing on their most critical assumptions and are able to evaluate potential solutions more effectively.
Rebecca says their continuous discovery activities tend to fall into the following categories: intercepting and recruiting interviewees, screening for and setting up discovery interviews, conducting customer interviews and evaluative activities like prototype tests and surveys, and synthesis and mapping. In the rest of this article, we’ll look at the tools they use to support each type of activity.
Intercepting and Recruiting Interviewees
For recruiting customers to interview, Optimal Workshop’s most common approach is to use Pendo guides or Intercom posts to intercept customers (or potential customers) as they use their app or visit their website.
“The benefit of using these tools for recruitment is that they make it quick and easy to set up intercepts without any technical involvement, we can target users based on the actions they take or their location within our app, and we can also set up priority and scheduling rules to ensure that recruitment intercepts don’t clash with any marketing messages or overwhelm the people using our app,” says Rebecca. “Another advantage of tools like Intercom and Pendo is that you can set up A/B tests of your recruitment intercepts to better understand what kind of phrasing or types of incentives best engage your customers.”
The benefit of using tools like Pendo and Intercom is that they make it quick and easy to set up intercepts without any technical involvement. – Tweet This
This is an example of a Pendo intercept an Optimal Workshop customer might see while using the app.
What’s the key to success with in-app intercepts? Rebecca says, “When crafting copy, it’s important to keep it friendly and concise, but aim to include all the key information that potential participants might want to know before signing up.” She recommends including the following points:
- What’s the purpose of the research? You don’t need to give away all the details, but provide an overview so participants know whether this research is relevant to them.
- What kind of commitment would they need to make? When are sessions running and how long will they last?
- What’s in it for them? Will you provide an incentive? And if so, what will this be? Or can you describe another potential benefit for participants?
When crafting copy, it’s important to keep it friendly and concise, but aim to include all the key information that potential participants might want to know before signing up. – Tweet This
Depending on the current research question, they can then direct respondents to an unmoderated research activity like a quick survey in Optimal Workshop’s Questions tool, a first-click test in another one of their tools, Chalkmark, or to Calendly to book a moderated session.
The product teams at Optimal Workshop use their tool, Chalkmark, to run simple tests like the one shown here.
Screening for and Setting up Customer Interviews
For discovery interviews, Rebecca says they will often set up a short screener in Questions to confirm that potential participants have recent experience in the area they’re looking to explore before inviting them to take part in a session.
A short screener survey like the one shown here can help you find interview participants that meet your specific criteria.
When creating your screener, you’ll want to include precise closed questions to help you identify participants who share the particular behaviors, contexts, motivations, or attitudes that are relevant to your research needs. But Rebecca suggests that it can also be a good idea to include at least one open-ended question. “Reviewing how screener respondents answer a slightly more in-depth open question can potentially provide insight on how clear and expressive they’ll be during an in-person interview.”
Reviewing how screener respondents answer a slightly more in-depth open question can potentially provide insight on how clear and expressive they’ll be during an in-person interview. – Tweet This
In the cases where they’re sourcing participants who aren’t already customers, Optimal Workshop teams have used approaches like recruiting via third-party panels like Askable, posting in industry forums or Slack groups (with moderator approval), or running social ads.
Scheduling is a huge part of setting up interviews. While the Optimal Workshop HQ is located in New Zealand, their customers are based all over the globe. “Using Calendly when scheduling interviews takes away the hassle of calculating time differences and back and forth emailing to select a session time or reschedule to another time as needed,” says Rebecca.
Using an automated scheduling tool like Calendly removes the need for lots of back and forth to find a convenient time.
Rebecca recommends including the Google Calendar and Zoom integrations for each event type in Calendly so she doesn’t have to set these up or add them manually. However, Rebecca prefers to manage the emails manually. This way she can refer to a participant’s specific project in the scheduling or reminder emails or a specific learning when sending a thank you note.
Finally, Rebecca notes that getting consent from participants is a critical part of the interview process. At least a day or two in advance of each session, Optimal Workshop asks participants to review and complete their custom online consent form. This provides participants with more details on the kind of information that they might collect during the session and allows them to specify exactly which data they’re happy for them to collect and use for different purposes. It also describes the process to follow if they have questions or wish to withdraw their consent.
Running Customer Interviews
For running customer interviews, Zoom is their tool of choice. Prior to the pandemic, Optimal Workshop conducted the majority of their interviews over video calls, but they previously used a range of different platforms like Skype and Google Hangouts. “The explosion in popularity of Zoom over the past year and a half means that we now use this almost exclusively for remote sessions. We chose it for the platform’s reliability, range of features, and integration capabilities,” explains Rebecca.
Product squad team members take turns observing and note-taking during these sessions.
They use Optimal Workshop’s collaborative note-taking and analysis tool Reframer to ensure they collate notes from each team member in the same place and can easily tag and group observations following the session.
Taking notes during an interview is critical, but it’s also important to store notes and tag them so they’re easily accessible later. This made-up example shows how Reframer could be used in a usability test session.
Taking good notes is a huge topic in and of itself, but Rebecca offers a few tips to keep in mind:
- Focus on capturing things you see or hear during a session rather than your own interpretations.
- You can include things people do (e.g. processes, tasks, tools, challenges) and how they think and feel (e.g. goals, motivations, reasoning, frustrations).
- Try to save your ideas and solutions for later.
For discovery interviews, Rebecca generally auto transcribes the session audio so the product squad can more easily pull out key quotes and check their observations against a participant’s verbatim description if needed. They’ve experimented with transcription tools like Descript and Otter. “I haven’t yet tried out Zoom’s audio transcription or recording highlights, but I’m looking forward to checking out how these features could further streamline our research process,” says Rebecca.
Synthesizing and Mapping What They’ve Learned in Discovery
Once an interview is complete, the product squad needs a way to synthesize and map what they’ve learned. Rebecca has typically used online whiteboarding tool Mural for a wide range of collaboration and visualization exercises like affinity mapping findings, developing experience maps, story mapping solutions, and prioritizing assumptions.
Rebecca notes that Mural doesn’t perfectly fit all their needs. “While Mural supports the collaborative synthesis and mapping process pretty well, something that’s long been a bit of a bugbear is having more seamless integrations between the original sources of data used to inform different mapping exercises and then feeding the outcomes of these exercises into discovery or delivery backlogs.” But, she notes that more recently Mural and competitors like Miro have been building out features and integrations that are helping to bridge that gap, so she’s keeping an eye on this.
Inspired by Continuous Discovery Habits, Rebecca has started collating interview snapshots into database tool Airtable using the page designer app. This way they can be organized and filtered by things like customer segments and opportunities and potentially even repurposed across teams. “It’s currently a bit of an experiment to understand how this approach might fit in a team’s workflow and if it would provide value further down the track,” explains Rebecca.
Prototyping and Running Experiments
The tools that Optimal Workshop uses for running experiments vary depending on what they’re looking to learn. They use their own task-based and simple survey tools (OptimalSort, Treejack, Chalkmark, and Questions) to quickly answer their own research questions. They either run surveys with unmoderated participants or pair them with moderated sessions depending on their needs.
For prototyping, they’ve experimented with several tools over the years such as Sketch and InVision. The team’s current preference is Figma due to its integrated prototyping functionality. “Figma’s powerful design management allows for a much faster prototyping workflow that doesn’t require additional syncing across tools,” says Rebecca.
Finally, it’s important to have a place to keep track of all the experiments they’re running. Since they use Jira for delivery backlog management and cycle planning, this is also where they document discovery experiments. Rebecca says, “This ensures that discovery work is balanced alongside delivery work, not kept in a separate backlog.”
Using Jira to document discovery experiments ensures that discovery work is balanced alongside delivery work, not kept in a separate backlog. – Tweet This
Adding New Tools and Advice for Other Teams
How do product squads at Optimal Workshop make the case for adding new tools to their stack? “We’re fortunate that we’re able to be relatively flexible around the tools that we use at Optimal Workshop. If a team member believes that a particular tool would bring benefits to their work and/or that of the wider organization, then they can submit a request via our tool approval portal and receive a response, generally in a matter of days,” says Rebecca. She notes that all tools must meet strict security and privacy requirements, but otherwise it’s mostly just a matter of describing the gap that this tool would fill or how it would perform the job better than an existing solution.
When considering adopting a new tool for their continuous discovery practice, Rebecca says she keeps the following points in mind. The tool should:
- Align with their security, privacy, and financial obligations
- Provide significant time-saving features
- Integrate effectively with their existing tools
- Have a positive overall user experience
Reflecting on the way they’ve built their current tech stack, Rebecca says she wouldn’t do anything differently. However, as a true continuous discovery champion, she adds, “I’m always on the lookout for ways to better integrate data and decisions across the discovery process.”
