Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. Continuous discovery is not a linear journey—as much as we might want it to be. Like a lot of learning, it often feels messy and chaotic.
Continuous discovery is not a linear journey—as much as we might want it to be. Like a lot of learning, it often feels messy and chaotic. – Tweet This
But if you stick with it, you may eventually find you can look back and see how much you’ve progressed.
That’s certainly the case for Kelsey Terry, who’s sharing her story in today’s Product in Practice. In her former role as Director of Product at Going (formerly known as Scott’s Cheap Flights), Kelsey was tasked with running a pilot product trio. The idea was that her trio would test out the concepts from Continuous Discovery Habits and then guide the broader product organization at Going to adopt the habits.
Did Kelsey’s trio instantly absorb continuous discovery and then seamlessly roll it out to everyone else at her company? This would be a much shorter blog post if that were the case!
In fact, Kelsey and her team faced many challenges along the way. They didn’t have enough time and felt rushed. They had too much time and struggled to make progress. They weren’t sure how to show their work and bring their stakeholders along with them.
But Kelsey kept iterating. She kept looking at what was working and what wasn’t until eventually she found a path forward.
We hope Kelsey’s story will remind you that your continuous discovery journey is likely to experience some twists and turns along the way, but it’s still worth sticking with it through those tough spots.
Do you have a Product in Practice story you’d like to share? You can submit yours here.
Meet the Continuous Discovery Champion, Kelsey Terry, Former Director of Product at Going
Meet the continuous discovery champion, Kelsey Terry, former Director of Product at Going.
Kelsey Terry is the former Director of Product at Going, a B2C SaaS company that helps members save between 40–90% off airfare. The product team at Going includes product management, product design, and a flight experts team. Kelsey oversaw three product squads and reported to the VP of Product, David Krell.
Kelsey’s career began in QA and she’s spent the past ten years working in product. “I really just love focusing on keeping the customer at the forefront of building great, impactful solutions,” says Kelsey.
When Kelsey first joined Going, there wasn’t a clearly defined process in place for talking to customers. “We would have organic conversations about bringing products and features online and we would talk to people when we needed to,” explains Kelsey.
Kelsey’s Introduction to Continuous Discovery
In her time at Going, Kelsey says that their approach to building the roadmap was radically different each year. When Kelsey first joined over three years ago, they were looking at customer trends in the inbox from the customer success team to identify opportunity areas and cross-referencing these ideas with their executive team. “I’d say our roadmap was informed by customer requests married with executive desires in terms of what they thought were great ideas and what they thought would move those metrics forward in terms of retention or our desired growth rate at the time,” she explains.
Kelsey was involved in the process of hiring VP of Product David Krell, and during his interview he talked about how Continuous Discovery Habits had helped highlight ideas for how to keep customers at the forefront of product work.
When David ended up joining Going, he made Continuous Discovery Habits required reading for the product team. “That essentially became a roadmap for how we were going to do product with him as our leader,” says Kelsey.
And Kelsey saw a lot of potential with continuous discovery as well: “Continuous Discovery Habits was the answer to the question of how to keep customers at the forefront of building great, impactful solutions,” she says.
Continuous Discovery Habits was the answer to the question of how to keep customers at the forefront of building great, impactful solutions. – Tweet This
The First Attempt at Continuous Discovery: A Tight Deadline Serves as a Circuit Breaker
Once David joined Going, his goal was to prototype continuous discovery with one team, and that was the team that Kelsey ended up leading.
They looked for an opportunity to apply the concepts from Continuous Discovery Habits and found a use case that had some built-in constraints: One of their subscription tiers, Elite, had its anniversary coming up. Because it was an anniversary for a subscription product, the team knew their outcome would be to reduce churn. And the anniversary itself was coming up fast—they only had about 20 business days.
They also had a steady stream of customer feedback that would get tagged and piped into Slack, so they could specifically look at why customers were contacting customer success about Elite memberships and which themes were recurring.
The team already had a solution in mind, which was to come up with more customer-friendly options than simply canceling, which could include things like having customers pause, ride out their time, or switch plans.
From this feedback, Kelsey’s team quickly zeroed in on a potential solution. They story mapped their customer flow at the time and identified some of the off-ramps they could create for customers that would better serve them if they needed to pause their membership, cancel it, or ride out the benefit until their renewal date.
The product trio also went through assumption mapping for the first time. Kelsey explains, “I asked, ‘What has to be true at every step of this process in order for us to stick the landing?’”
To go through assumption mapping, I asked, ‘What has to be true at every step of this process in order for us to stick the landing?’ – Tweet This
They went through this process in the shared sandbox of FigJam and everyone was encouraged to point out any potential issues they observed. An engineer could call out an assumption they didn’t believe was true or a designer could question whether the choices they were making (e.g. the placement of text or the use of dialogues) were optimal. Kelsey explains, “The way I try to frame assumptions is: We are betting on a lot of things to be right. And how do we visualize that thinking in that shared sandbox so that we can help eradicate our individual blind spots?”
Kelsey believes this was a critical moment for product trio collaboration: “It was a process of bringing all those people together to generate assumptions and then ultimately shape the solution,” Kelsey explains.
The solution they came up with was related to the account management page, where customers have the ability to see their plan type, how much their subscription costs, when they’ll next be charged, and where they have the ability to upgrade their subscription or adjust their email settings.
The concept was a “manage my account” button. If a customer clicked on it, they would see a dialogue with questions like “Would you like to pause your subscription?” “Would you like to keep the benefit until your subscription runs out?” or “Would you like to cancel your subscription right now?” Kelsey explains, “The thinking was if we improved the account management page, it would be less of a surprise for customers when their plan is renewing.”
Next, their tech lead created some lo-fi mockups in FigJam. Kelsey says these were simple wireframes with boxes and sketches. Team members would physically draw ideas at their own desks and then upload them to FigJam so they could riff off of each other. After running through a few iterations this way, they got into mid-fidelity, which is when it made sense to have UX build out some real UIs.
At this stage, they did a “design spot check,” which involved pulling in customer success and marketing partners to get their feedback. They might weigh in on things like whether the copy was confusing or if there were areas where customers were likely to get tripped up. “In this stage, we could refine it while the paint was still wet,” Kelsey explains.
Once the spot checks were complete, they locked in the design, had the designer make their hi-fi version, and asked the engineers to start figuring out how to pull it off.
They were able to make the changes they’d planned and they got the feature out on time. While they weren’t able to prevent all churn, they were able to establish a baseline. They now had a benchmark for both the Elite product and the Premium product, which was something they didn’t have before.
While Kelsey’s team wasn’t able to adopt all of the discovery habits in this first round, they still managed to accomplish a lot—they were able to leverage customer feedback to find a compelling opportunity, story map solutions, iterate on designs by talking through their assumptions, and test prototypes with customers. And most importantly, they did it as a collaborative team. It’s especially impressive that they accomplished all this in only 20 days.
The Second Attempt at Continuous Discovery: Lack of Strict Time Limit Leads to Perfectionism
For the second attempt at continuous discovery, Kelsey’s team no longer had such tight time constraints. Their primary goal was to do everything “right.” If you’ve been reading Product Talk or following Teresa for a while, you might recognize this as a potential problem. And you’d be spot on, as you’ll see shortly.
Having an open-ended timeframe proved to be both an advantage and a disadvantage, according to Kelsey: “We spent a lot of time rereading and highlighting and referring to specific sections of Continuous Discovery Habits. It was very meticulous.”
Because they were the pilot team, there was also pressure to get everything “right” because they would be expected to teach others how to adopt continuous discovery. “It felt very exhausting because we would challenge one another’s beliefs about how we would interpret the book, but none of us were experts.”
In this round, the outcome was focused on increasing web engagement. Research from the data team had shown that people who engage with more of Going’s content and deals online were more likely to be retained. “We wanted to increase engagement of web content via guide consumption, opening up our travel deals, possibly hitting that booking icon. And so we wanted to basically layer on value within the product experience on the web,” explains Kelsey.
For this iteration, they began talking to customers to learn more about how people travel. Through these conversations, they began to identify that many people lack flexibility with their travel while the Going product really demands flexibility. “Deals happen at any time and sometimes you have to book them very fast. But people with children, with school schedules, teachers, they are beholden by budget, calendar, airline preference, or seating preference.”
Based on what they were hearing from customers about flexibility, they began to build out their opportunity solution tree. The team worked with Product Talk instructor Hope Gurion to frame the opportunity as: “I value a particular criteria for my flights when I travel. I have some rigidity.”
Then they had child opportunities listed underneath that opportunity associated with different criteria, like “I want to see all the legendary deals,” “I want to see all weekend based deals,” and “I have Delta status, so I just need to see Delta deals.”
Based on what they were seeing from the customer success Slack integration and what they were hearing in customer interviews, Kelsey’s trio discussed and voted on which opportunities they wanted to pursue.
They ended up narrowing in on “I value a particular criteria for my flights.” The child opportunity within that was “I have limitations around when I can travel.” Keeping that customer quote at the forefront, they wanted to ensure that they built something for specific dates.
Next, they started building out multiple solutions, story mapping different ways they could solve that opportunity. “We ultimately landed on the creation of something called Save Filters. You can set your preferences of Delta under $500 in the month of June and then name that container of filters and we would always be looking for that on the website,” explains Kelsey.
One of their primary concerns with this solution, however, was whether or not they could match customers’ expectations for how it might work. Kelsey explains, “There was this concern that while we were building filters for our website, where you could set the attributes, there was a technical limitation at the time where we were not going to filter the emails we sent you.”
Kelsey’s team wanted to understand if a customer set up a filter on the website, if they would also expect that filter to apply to the alerts they received by email. They used customer interviews and prototype testing to understand this risk. They showed a lo-fi mockup of the product to customers and asked questions to learn how they were interpreting it or what they might expect from certain features.
“This took a very long time,” says Kelsey, describing the second iteration of continuous discovery, which ran for about two and a half months. They were meeting Monday through Thursday as a trio for 60 to 90 minutes each time. They’d come up with their own solutions on their own, share them with the team, and debate.
Going back to the theme of perfectionism, Kelsey says, “There was a lot of debate because I think there was that fear of wanting to get the foundation right. We were the prototype team. And if we’re going to be training other people, we have to get this right. So I think that caused this desire for perfection versus progress.”
Summing up this second round, Kelsey says, “Because we were novices to the process, I think we over-engineered what we needed to do.”
A Brief Pause to Make Adjustments
After the second attempt at continuous discovery, it was clear that something needed to change. There was a lot of pressure and perfectionism, which was frustrating and prevented the team from moving forward at a reasonable pace.
They ran a retro to take a closer look at their process. They wanted to identify what was working and what wasn’t.
At this stage, they created some circuit breakers for each step of the process and started timeboxing different activities. They decided they wanted to go through the process every 20 days.
The team felt an immediate sense of relief. “This created some comfort and encouragement. I felt like this was not going to be like an ultra marathon for the rest of my life,” says Kelsey.
But looking back, Kelsey sees some limitations to this approach. “I think a shortcoming of our perspective back then was that we viewed the process in very much a cardinal order. You must hit every box versus a toolbox where you can pick up a hammer or a screwdriver at the right time.”
I think a shortcoming of our perspective early in continuous discovery was thinking you must hit every box vs. a toolbox where you can pick up a hammer or a screwdriver at the right time. – Tweet This
The Third Attempt at Continuous Discovery
With the goal of focusing on faster iterations, Kelsey’s team was ready to tackle their next outcome.
For this round, they focused on onboarding. Their outcome was to increase the number of customers who got through the onboarding process into the product. When they started this cycle, their onboarding process was 20+ steps. They saw drop-off at every step. So naturally, they thought if they could reduce the number of steps, they could increase the number of customers who completed the process.
The team wanted to be bold in their approach. They story mapped the existing onboarding process and worked to cut everything that wasn’t essential. They whittled it down to a 3-step onboarding process, while sticking to their new 20-day time limit. They were excited about the results.
But when they presented their proposed solution to their executive team, they were surprised when the executives weren’t as excited. The executives deemed the solution too risky. Kelsey says, “We were very off track in terms of what we produced, and as a result we were frustrated and discouraged that we’d put in all this effort and thought, and it was off base.”
Once again, Kelsey felt the need to take a step back and reevaluate.
Applying Continuous Discovery to the Organizational Change of Using Continuous Discovery Habits
At this stage, Kelsey’s team ran another retro with the goal of figuring out how they could make continuous discovery work for the business.
Kelsey started by talking with her “customers”—her executives, members of the product trio, and members of the product team who hadn’t participated in the product trio—to learn what they saw as the shortcomings of continuous discovery.
From these interviews, Kelsey identified three key themes: continuous discovery felt too slow, too exclusive, and very unclear. “It was very hard for people to follow along because this is a highly visual process, and not everyone is versed in visual tools like FigJam, double diamond frameworks, assumption maps, or storyboards,” says Kelsey. She saw her challenge as coming up with a new solution that was true to the spirit of Continuous Discovery Habits but more accessible for her coworkers.
Kelsey mapped out the opportunity space and designed rituals and artifacts that would help her team better show their work throughout discovery. “One of the core things that our executives wanted was insight,” says Kelsey. “They didn’t just want to be handed a FigJam link. We needed to make it more business friendly, so we created a series of meetings and artifacts to support them.”
One of the core things that our executives wanted with discovery was insight. They didn’t just want to be handed a FigJam link. We needed to make it more business friendly. – Tweet This
They came up with a regular cadence for their work:
- Monday: The product trio meets to discuss their current opportunity and whether they have any assumption tests to define or results to review
- Tuesday: The product trio meets with the broader team to refine solutions, which reduces feelings of exclusivity
- Wednesday: The product trio reviews progress on their outcome, opportunities, and solutions they’re looking to de-risk with key executive stakeholders
- Thursday: The engineers demo whatever they’re working on, both in terms of discovery and delivery, sharing work early and often to gather feedback from their peers
Establishing a regular cadence had a forcing function: Both the product trio and engineering teams felt they had to make progress every week since they’d be on the spot at the next weekly meeting. “There’s the internal forcing mechanism of the trio meeting on Monday as well as the larger organizational accountability for progress every Wednesday,” says Kelsey.
These rituals helped Kelsey’s team drive faster discovery cycles, while keeping key stakeholders up to date on their findings. It wasn’t easy, but Kelsey’s team was finally able to find a cadence and process that worked for everyone.
Key Learnings and Takeaways
As she looks back on this experience, Kelsey has a few key learnings and takeaways for other teams.
First, Kelsey says she’d avoid using the label of “continuous discovery.” Here’s her reasoning: “PMs can sometimes get a bad reputation for wanting to adopt the latest new framework.” Instead of using any specific label, Kelsey recommends just introducing the concepts to your team. “Say something like, ‘Hey, this is a way I think we could talk about what we’re seeing with our customers.’”
Another key lesson: Give yourself a lot of grace. Kelsey’s story illustrates that perfectionism can prevent your team from moving forward if you’re not careful. Try not to be too hard on yourself or your team to get everything “right” and treat this as a learning experience.
“If continuous discovery doesn’t work for your org, it doesn’t work at all,” cautions Kelsey. What does this mean for you? “You have to modify it. You have to take the principles of Continuous Discovery Habits and apply them to the business that you find yourself in. You have to understand your business and then apply the spirit of discovery to your business in a way that makes sense.”
If continuous discovery doesn’t work for your org, it doesn’t work at all. – Tweet This
Finally, be aware that not shipping is also a risk that you need to be aware of. “My VP, David always says that by not giving value to our customers, one, it’s a risk to the business that someone else is inching up against us, and two, if we’re just spinning our wheels and we’re just postulating all day long, we’re not even serving our customers. And we are hired to support the business and our customers.”
Kelsey first shared this story in the Continuous Discovery Habits Slack community. If you’re looking for a safe space where you can connect with like-minded product peers and share your own learnings and questions, come join us there!
