Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. What do your customers care about? This seems like a simple enough question, but many product teams struggle to answer it honestly.
Often the first instinct is to frame an answer in terms of what your business cares about. But remember: Your customers don’t care about your business outcomes. They care about having their own problems solved or having an enjoyable experience when they use your product.
Remember: Your customers don’t care about your business outcomes. They care about having their own problems solved or having an enjoyable experience when they use your product. – Tweet This
Today’s Product in Practice is a great reminder of this. One of the product teams at Grailed, a peer-to-peer marketplace for fashion, put a ton of time and resources into building a feed. But instead of launching with great fanfare and adoption, the response was underwhelming.
When the product team took a step back to evaluate what could be better, they realized they could benefit from introducing continuous discovery habits.
Talking to customers regularly allowed them to better frame opportunities in terms their customers actually used. With a better handle on opportunities, it was easier to identify and test assumptions. And the whole process of visualizing their thinking made their work much more enjoyable.
But that’s not all—introducing continuous discovery habits completely transformed their customers’ behavior for the better. And this ultimately had the impact on the business outcomes they cared about most.
Read on to learn about how opportunity mapping led to a major product transformation at Grailed.
Do you have your own Product in Practice story you’d like to share with us? You can submit your story here.
Meet the Continuous Discovery Champions, Rajiv Chopra and DJ Mitchell
Rajiv Chopra is the Group Product Manager and DJ Mitchell is the Staff iOS Engineer on the Discovery team at Grailed. As you’ll see in this blog post, Rajiv and DJ are committed to the values of continuous discovery. But in the context of their company, which is a peer-to-peer marketplace for fashion, “Discovery” refers to helping people discover their style and clothes that could help them express themselves.
Rajiv says, “The real user problem we’re aimed at solving is: ‘I don’t really know what I’m looking for and I need some help—can you help me get inspired and figure it out?’” A small percentage of users—around 10%—arrive at Grailed because they’re looking for a very specific item and it popped up in a search result. But Rajiv says his team is focused on helping the 90% of people who don’t know what they’re looking for to figure it out.
Describing his role, DJ explains that being a staff engineer gives him a lot of latitude to have an impact (and a lot of leeway to define exactly what that means). He sees his role as bringing engineering closer to product, modeling certain types of behavior, and mentoring engineers to be more product minded and outcome oriented.
Before: Claiming to Do Discovery But Struggling to Make an Impact
Rajiv and DJ say that before they introduced the opportunity solution tree and committed to the practice of continuous discovery, their team was struggling with a few common challenges. Rajiv explains, “The PM and leadership would define an outcome based on what we ‘felt’ was important, but it wasn’t really motivated by what we knew or any learnings. And then immediately we’d go into ideating solutions, prioritizing solutions, and then delivering them at full fidelity.”
The Discovery team’s initial approach to building the feed helps illustrate some of these issues. Grailed has both a mobile app and a website, but the mobile app is where they get the majority of their traffic and engagement, so that’s where the team was focused. The original concept of the feed allowed users to curate their experience. Similar to other types of social platforms, users could follow designers or sellers and then browse through content in their feed.
The original premise of this feed, according to Rajiv was: “You set it up, you manage it, curate it, and you allow your feed to represent your style.” The way the Discovery team originally delivered this project was an eight-month investment of building the infrastructure, doing the design work, and building the supporting foundational elements. But when the team got to the end of the eight months, the response was underwhelming. While they didn’t harm the business, they essentially broke even.
This was not the impact they’d been hoping to have, so it prompted Rajiv and DJ to reconsider their approach to product discovery. Around this time, they read Continuous Discovery Habits, which got them thinking about how to apply these concepts at Grailed.
Introducing the Opportunity Solution Tree Leads to Rethinking the Feed
One of the first steps the Discovery team took was to visualize everything that was in their heads through a first draft of an opportunity solution tree. DJ explains, “Our first crack was to do an ideation with the team, just to get everything we think we know. That generated the original version of the opportunity solution tree.” From there, they started talking to users to verify that things were actually true.
Trying to describe this messy process, Rajiv says, “It was basically get everybody together, just dump it out, go through a structuring/affinity mapping/clustering activity, and use that as opportunity solution tree number one.” In that first version, they just had “increase relevance” in the tree.
Since then, their opportunity solution tree has evolved quite a bit. Now the place where the feed lives in the tree is: “I want to browse without any specific idea of what I’m looking for.” Rajiv says, “We want the feed to help accomplish this. That’s the problem this is ultimately trying to solve, and the way that it’s going to do that is helping you browse things that you like that are in your style.”
Describing the evolution of their thinking and the way they framed opportunities, Rajiv explains, “The challenges we were having with the feed were primarily of content quality, which is, ‘I just wish the feed would show me more stuff I like and less stuff that I don’t.’”
And while this might seem obvious, they realized that for several months they were actually solving for the problem, “I wish I could better control what my feed showed me.” At the time, the Discovery team assumed users would curate and manage their own feeds.
But, says Rajiv, “We learned when we talked to users that nobody’s saying this. They don’t say these words. Most people are saying, ‘Show me more stuff I like and less stuff I don’t.’ They’re used to feeds being algorithmic. They’re not thinking about them as, ‘Oh, I need to go and press all these buttons for it to work.’”
We were trying to solve, ‘I wish I could better control what my feed showed me.’ But when we talked to users we learned that nobody’s saying this. They don’t say these words. – Tweet This
DJ compares this evolution to chronological feed debate that’s been happening on every social media platform for the past ten years. “A few people think you want exact perfect control to see everything, but actually the quality of chronological feeds at scale is kind of bad and that’s why no one has them. That’s the same challenge we ran into.”
Looking back on this process, Rajiv says, “We’ve taken a chronological snapshot of our opportunity solution tree over time and you can see how much depth and how certain things get removed and added.” Once they started talking to users about their feeds, they began to pull out opportunities like: “Show me things I interact with” or “Show me stuff that’s popular,” or “There should always be new stuff when I come,” or “I want to get inspired when I come—show me some stuff I don’t already know about.”
As the Discovery team at Grailed has become more comfortable with continuous discovery, their opportunity solution tree has evolved to be much more complex. Click the image to see a larger version.
The Discovery team started taking these statements and identifying very low-lift ways they could start testing. If users claimed they wanted stuff that was popular, for example, they’d figure out an easy way to introduce that into their feed. “Over the course of six months, the feed went from this very simple, reverse chron feed of everything to a feed that has different parameters, different recommendations, weaves things in an intelligent way, and manages stuff you’ve seen before to make sure that it maintains freshness,” says Rajiv.
A Closer Look at the Transformation of the Opportunity Solution Tree
How did the opportunity solution tree evolve to better guide the transformation of the feed? Rajiv and DJ note that the process of mapping opportunities on the opportunity solution tree involved multiple steps and iterations. It takes time to get it right. “Your understanding of the opportunity space is a slow-moving but constantly growing activity,” says Rajiv. Here are some of their other major observations about the evolution of their opportunity solution tree.
Your understanding of the opportunity space is a slow-moving but constantly growing activity. – Tweet This
“Your understanding of the opportunity space grows and evolves over time,” says Rajiv. “We ended up finding that our initial statements were business statements; they weren’t user opportunities. The most valuable thing is ensuring that our opportunities are representative of words that our customers use. There’s nothing more than just talking to customers to find those opportunities.”
In addition to learning to frame opportunities in the language their customers actually use, Rajiv says they’ve gotten much more specific in their opportunity statements. Before, their opportunities would be framed as, “I want to get inspired and discover without any specific idea of what I’m looking for,” which was such a huge solution space that they had a lot of trouble ideating productively and actually knowing what they were solving for. “Now we’re ideating around a really specific problem, like ‘there should always be new stuff when I come’ or ‘show me stuff that’s popular,’ which has allowed us to get really targeted solutions that directly address the specific problems that people have.”
One other observation Rajiv makes is that the lower you are on the tree, the faster you can be and the higher you are, the more evidence you should be building over time. “And one of the things the opportunity solution tree allows you to do—both by going deep and laddering it back up—is to show that you’re slowly building this lattice of understanding of this problem space.”
One of the things the opportunity solution tree allows you to do—both by going deep and laddering it back up—is to show that you’re slowly building this lattice of understanding of the problem space. – Tweet This
You won’t be able to tell whether a high-level opportunity is important to the business in a week. But within 3 months, you can test your way through 20 opportunities and be in a much better position to say yes, there is strong evidence for it, no, there’s strong evidence against it, or the evidence is mixed and you may need to reevaluate your structure of the problem space. “The way we build this evidence is cumulative—it’s not a zero to one in a week,” says Rajiv.
A Total 180 in Impact: Driving Positive Reception from Users and Real Business Results
There’s no doubt that adopting continuous discovery habits—specifically conducting regular customer interviews and mapping opportunities on the opportunity solution tree—has led to impressive results at Grailed. “It’s had a total 180 in impact, both in terms of business impact and the reception from our users,” says Rajiv.
The two key metrics they cared about were how many items were being discovered from the feed they’ve built and how many dollars of gross merchandise volume (GMV) that was driving for their business.
“Discovery and GMV have grown four to five times from the feed specifically (from January to August),” says Rajiv. “In terms of the impact that it’s having on the business, it’s clearly having a big impact, both in engagement and revenue.”
The new feed has had a major impact on both how many new items users are discovering and the dollars of gross merchandising volume—two metrics that matter a lot at Grailed.
The Discovery team also wanted to measure the impact of the new feed on lifetime value (LTV) and whether users were changing their behavior in a permanent way. They compared users who made their first purchase through the home feed vs. users who made their first purchase through search. “The feed shows clear evidence that it drives higher LTV, and specifically the way that it does that is it drives higher levels of retention and engagement,” says Rajiv. “When we look at these people who activate through the feed, they come back and open the app about 30% more than the average user, they discover about 40% more than the average user. And the impact that it’s having is to the order of 20% on LTV and specifically by driving higher levels of retention, which is a good proxy that we use for real, true user value.”
The new feed has also had a significant impact on user retention and cumulative lifetime value (LTV). Click the image to see a larger version.
The Discovery team is also getting qualitative evidence that the users’ entire perception of the product is changing. Rajiv says, “There were a lot of instances where we wouldn’t even bring up the feed and users would unprompted come to us and say, ‘I gotta tell you. This feed is awesome.’”
This was a major change, Rajiv adds, “I can’t remember the last time at Grailed that we released a feature and without even bringing it up users would come to us and be like, ‘Wow, I’m so stoked that you built that.’ Normally we have to remind them, like, ‘Hey we just released this thing. Do you like it? Did you even notice?’” While the LTV and engagement data provide strong evidence, “It was cool to hear users articulate that same story,” says Rajiv.
Learnings and Key Takeaways
Looking back on their continuous discovery journey so far, DJ and Rajiv have a few key learnings and takeaways to share with other product teams.
- Read Continuous Discovery Habits!
Reading the Continuous Discovery Habits book is what kick-started this entire journey for the Discovery team at Grailed. It made such a big impression that they started a book club to encourage others to read it and discuss how to apply the concepts on their teams.
- Map your assumptions and avoid the temptation to jump straight into solutions
“Assumption mapping is also a really valuable tool,” says DJ. “It’s easy to lull yourself into the idea that you don’t need to do any of this stuff and it might seem like extra work when you get it right, but very often you don’t get it right the first time.” DJ emphasizes the importance of following the steps of the opportunity solution tree and not skipping straight to solutions.
It’s easy to lull yourself into the idea that you don’t need to do any of this stuff and it might seem like extra work when you get it right, but very often you don’t get it right the first time. – Tweet This
- Don’t let the perfect be the enemy of the good.
“You see that in the artifacts of the early opportunity solution trees and how they’ve evolved,” says DJ. It’s really easy to feel confused and feel stuck. Set a time limit for yourself and just keep making progress. But when that fails, move on to the next tip, which is…
- When in doubt, do it together
“We used to err more on the side of documentation and now we do things much more in person,” explains DJ. “We’ve found it really effective to just do things in person, talk about it, and figure it out together rather than going into a hole and trying to do it all by yourself.”
- Draw more, talk more, write less
Similarly, drawing (or at least creating a visual representation of ideas) has become the go-to way of communicating at Grailed. “It’s way easier to have a little bit of text and a diagram, a structured visual thing that will explain your thinking. It’s a huge time-saver, but it’s also so much better for clearly expressing what your intention is. It’s a very underrated hack that I’m a diehard believer in now,” says DJ. To facilitate this, they have a section of their InVision board called “scratchpad” where everyone gets a dedicated “visualization corner.”
Drawing is so much better for clearly expressing what your intention is. It’s a very underrated hack that I’m a diehard believer in now. – Tweet This
The members of the Grailed Discovery team are now die-hard believers in visualizing their thinking, as evidenced by the scratchpads everyone keeps on their InVision board. Click the image to see a larger version.
- Your job is to facilitate; not give orders
“I feel like PMs have this existential dread that we’re constantly responsible and accountable for our roadmap,” says Rajiv. “The more you don’t involve your team, the more that gets reinforced. The better I’m doing my job, the less I’m responsible for coming up with ideas and the more I’m responsible for helping the team figure out and discern which ideas to bubble up or dig further into.” By emphasizing facilitation over giving orders, “The quality and quantity of our ideas has gotten much higher. And my emotional wellbeing is so much better than it used to be. Aside from facilitation delivering better outcomes, it makes your job fun and awesome and exciting as opposed to dreadful and oppressive.”
Looking to connect with others who are in the process of introducing continuous discovery habits on their teams? Come join us in the Continuous Discovery Habits community!
