Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. For many product teams, continuous discovery is a brand-new way of working. It involves strengthening your understanding of business outcomes, fostering collaboration in product trios, and cultivating many other habits like conducting regular story-based interviews, mapping opportunities on the opportunity solution tree, and identifying and testing assumptions.
While you might be eager to dive in, it can be a little overwhelming to know how to get started. How do you make the most impact without confusing your team or disrupting the projects they’re already working on?
To learn more about how real product leaders are approaching this challenge, I recently sat down with fellow Product Talk instructor Ellen Juhlin (who’s also a product coach, consultant, and Senior Director of Product Management at Orion Labs). We discussed how Ellen first learned about continuous discovery and the steps she took to roll it out, specifically with the engineers at Orion Labs.
You can watch a video of our conversation or read a lightly edited version of the transcript below.
Full Transcript
Teresa Torres: Welcome, everybody. I’m Teresa Torres.
Ellen Juhlin: And I’m Ellen Juhlin.
Teresa: We are here today to discuss how Ellen introduced her engineers to discovery and how she got them involved in a more continuous cadence to their discovery work. Ellen, do you want to tell us a little bit about the company and your role, and who you’re working with that you introduced continuous discovery to?
Ellen: Sure. My main work and where I’ve been working for the past nine years is Orion. We do voice communication, like a walkie-talkie, but on your phone or on the web. We have web engineers, mobile app engineers, backend engineers, and a lot of our solutions that we’re building cross over this cross-functional thing. An admin might create a user on the web console and then that user logs in on the mobile app and then they send a message through to other members of their group. There’s a lot of thinking about the different components of the product as we’re building new solutions.
Teresa: Who are your typical customers? Who tends to use this product?
Ellen: This is a product mostly for frontline workers. It could be bus drivers, it could be people working in a retail store or a hotel that are using this to communicate with each other throughout their day as they’re working on their shift, as well as the managers or supervisors that are involved in that day-to-day communication as well.
Teresa: It’s like logistics or operations: “How do we keep our team talking with each other, coordinating as they’re doing their jobs?”
Ellen: Yes. Responding to things that come up in the moment, like, “This guest showed up and they are ready to get their car from valet,” or “They need more towels in this room,” or “This person is looking for this same shirt but in a different size,” or “I’m waiting to pick up this passenger, but they haven’t shown up at their address yet,” kind of thing. A lot of different things, but mostly in-the-moment things that are live communication that they’re responding to.
Ellen’s Introduction to Continuous Discovery Habits
Teresa: Great. That’s helpful. I know you got introduced to the continuous discovery habits through our Master Class. Tell me a little bit about how you heard about that. Why did you want to take it? What was your beginning with the discovery habits?
Ellen: I was talking with a good friend of mine who had recently taken the Master Class, and she highly recommended it, saying, “It’s great content, and you get to meet some other interesting people working in small groups.” I already knew you, Teresa, through other social groups so I was like, “Oh, right, that’s cool.” I was able to sign up for that. That was my first introduction to the continuous discovery framework.
Teresa: Were you familiar with even the distinction between discovery and delivery at that point? Were you aware of this broader conversation or was that really your first foray into it?
Ellen: That was my first foray into that nomenclature. There were some things that I’d been doing in my role or at past jobs that as I was introduced to it, I was like, “Oh, this is like that thing I was doing, but it’s got a name and more of a process to it than what I was already doing.” Super helpful.
Teresa: That’s pretty common. A lot of the ideas are things that have been part of the practice for a long time, but maybe we didn’t know what they were called or it adds more structure to them or things like that.
A lot of the continuous discovery habits are things that have been part of our practice for a long time, but maybe we didn’t know what they were called or it adds more structure to them. – Tweet This
Ellen: Or you now had a way of describing to your coworkers why it’s important and different from delivery.
Teresa: Definitely. You heard about it from a friend. What was going on at your company at that time? How was your team working? What did that look like?
The Before State: How Product Worked at Orion Labs Prior to Continuous Discovery
Ellen: I guess when I took the Master Class, it was near the beginning of the pandemic. We’d all transitioned to working from home instead of being in the office and figuring that out. In the time that I’d been with the company over the previous number of years, we had iterated on how to do product roadmap discussions as our company grew from 5 when I started to 40, 50, 60 people and growing the engineering team and trying out different ways to have the “What do we work on next?” conversation.
There’s changing business strategies and changing talking with investors and changing market landscapes and the pandemic. All of these things are like figuring out, “Okay, based on all this other information, how do we tell our engineers what to work on next?” One of my product managers on my team proposed this idea of six-week planning cycles that we introduced at some point to at least constrain what priority level we were looking at. It was like, “Okay, just for the next six weeks we’re going to focus on these things.”
I think we were also still missing tying that back to the business strategy piece. There were fire drills like, “This investor had this idea. Let’s make something that looks like that,” or “This competitor made this thing, can we build something that’s like that?” It was very output focused for sure, like, “Let’s build this thing” or “Let’s build this thing to be able to tell a story or market a thing,” but it wasn’t driven necessarily by that strategy upfront.
Teresa: Was that the way it was? You said you were there from employee 5 to 45 employees. I know for a lot of startups, it’s really output driven and by the founder, right? The founders come in with, “We want to build this thing,” and then you’re going from zero to one. There’s a lot to build to go from zero to one. Then as the company grows, there’s this transition to: “How do we put some more practice around product and how we make decisions about product?”
It sounds like you were well on your way in that transition of, “How do we communicate with engineers about what to build, and where do those decisions come from?” It sounds like this idea of “Let’s just focus on six weeks at a time”—at that time, you said it was really output driven. Who was deciding on that output?
Ellen: It was me as part of the executive team, so head of marketing and sales, and customer success, and whoever else was part of our executive team, so CTO, CEO, etc. We’ve had some changing roles over time as the company grew, of course. We also had a sprint cycle, two-week sprints as part of our scrum process. That was already in place on the delivery side, but it would often get interrupted. It was like, “We planned this thing, but this other customer showed up that we want to build this thing for, can we just stop and make something else?”
The six-week process helped mitigate the sprint being interrupted, but then we still had the, “Okay, what customers do we see on the horizon? How are they similar? How are they different? What do we know about them?” I think especially in the B2B context, when you’re not the end user, you’re not building it for yourself, you really have to do a lot more getting into the customers and figuring out what they really need since it’s not you using it.
Teresa: Then do you remember before you took the Master Class? How were you learning about that? Were you engaging with your customers? Who else at the company was engaging with your customers? Where did that knowledge come from?
Ellen: We had done a lot of consumer research upfront because our company started as a B2C. We did focus groups, folks did taxi ridealongs, we did some UX prototype testing, and so forth. We had a lot of, “How do people interact with a walkie-talkie app?” knowledge, but then when we transitioned to B2B, then it was like, “How do security teams use walkie-talkies, and how do retail stores use walkie-talkies?” We had some of that, but it also had a big influx of different types of customers that we were working with.
We’d certainly get some information from the customer success team, but we had a protective layer at that point between sales and CS. They were nervous about other people talking to customers directly. It was like trying to feel out what these customers needed and gleaning information based on, “Can you build this thing?” It’s like, “Well, what is that for? Tell me what they need. What’s their context? What are they doing?” Even then, it was through a veil of secrecy.
Teresa: That’s so common. Your sales and customer success team were hunting for customers and trying to figure out who needs this B2B app, what do they need for us to build for them to use it, bringing it back to you, and maybe the rest of the executive team. What role were the engineers playing at that point?
Ellen: We’d have a head of engineering as part of our prioritization discussions, too. I was in regular contact with the engineering managers and part of the scrum process as well for most of that, too. I was bridging the day-to-day by sharing things like, “Hey, we just signed a contract to build this thing in six weeks.” Trying to minimize those surprises while also just building up our core platform.
We had a huge list of things that we wanted to do. We had this gigantic spreadsheet that was probably 50 columns long, broken up by component or customer. Before we started doing the six-week cycle, we had meetings every two weeks that were priority meetings, but it was meetings that got bigger and bigger, like, “I just want this person to observe the process,” or, “I think this person should be included.”
Then they just got added on and so it was half the company in these priority meetings every two weeks. That was just completely draining and very unproductive. We had way too many people in it and way too many opinions.
The six-week process was one step in the right direction and then really clamping down—I think when we had a change of CEO, he introduced this product strategy board: Here’s the set of five people that get to have an opinion about the priorities. That made that process a bit more sane as well. Smaller group, deciding on the six-week cycles based on customer needs or changes in the marketplace that we saw coming.
Teresa: I can imagine this meeting because I feel like I’ve been in them. I’ve even been at a startup where there was this one spreadsheet that ruled everything of, “Here’s all the things we could build,” and then everybody got in a room and argued about which ones moved their way to the top, and then the ones at the top are what we built.
Ellen: Yes.
Teresa: Yes, oh yuck.
Ellen: There’s never just one thing at the top. There’s ten things at the top and then you haven’t actually decided anything.
Teresa: Then you’re like, “Okay, we’re going to do these ten things in the next two weeks and you get two of them.”
Ellen: Right. Good luck.
Teresa: Then you have the same fight in two more weeks.
Ellen: Yes.
The First Step: Introducing Continuous Discovery with Story-Based Interviewing
The first continuous discovery habit Ellen introduced was the idea of conducting story-based interviews with customers to learn more about them and get more context on their needs.
Teresa: Then you took the Master Class, you learned about the continuous discovery framework. What happened next?
Ellen: It was interesting that the timing happened—I took the class right when we started a pilot with a pretty big customer that had security teams in different campuses at sites around the world. They were trying out our product, and one of the first things I was able to do—because one of my product managers was also the project manager for this pilot—it was like, “Hey, great, get us talking directly to the folks that are trying out the product to get their feedback.”
And why don’t we start using the story-based interviewing technique of: “Tell me about the last time that you had a security incident,” or “Tell me about the last time you walked a security detail,” and really learning a lot about when they used their walkie-talkies and how they used them. Then how did they use our product during the pilot process?
We were able to learn a lot about what they were doing in their work that was more helpful context than just, “They need this button” or “They need this feature,” which is what we’d been hearing from the sales and customer success team.
By talking to customers, we were able to learn a lot about what they were doing in their work that was more helpful context than just, ‘They need this button’ or ‘They need this feature.’ – Tweet This
With those learnings, then the rest of the folks on our team at the company saw, “Oh, this was really helpful to learn these additional behaviors and patterns,” and asked, “Can you do that with this customer?” That was step one of just changing the way that we talk to customers and getting the buy-in of seeing how useful that feedback was across the rest of the team as well.
Teresa: Let’s get into this a little bit. You said you started with a product manager and they started interviewing end users for a specific customer that was evaluating their product. You had mentioned that there was this culture where sales and CS owned the relationship with a customer. How did that product manager get access to those end users?
Ellen: Basically, nobody else wanted to be project manager. The product manager on my team ended up in that role of managing the Gantt chart and the spreadsheets and the dates and due dates and so forth and then had direct access to the customer that way, and so was able to bridge the gap there of like, “Hey, it’s not scary when a product manager talks to a customer because we’re learning and we’re not promising.” And then that helps everybody else. That was helpful. With the interviewing technique, too—I only had brief exposure in the Master Class, but I was like, “Oh, this is super helpful.”
I had done things like this without knowing what it was called. Then having that additional momentum, I was like, “Hey team, let’s start doing this. This is how we plan our interviews based on what story-based questions we want to ask, and we can practice asking each other questions, and we can take turns interviewing and taking notes, and then giving each other feedback on how we did.” I was able to quickly iterate with my team on this new technique and get that in place, and seeing how helpful that was helped the momentum of that process too.
Teresa: It sounds like at this point you got your product managers interviewing and excited about story-based interviewing. Were your engineers involved at all at this point?
Ellen: Not for this particular customer, but we had another CS person or a salesperson in the process, or watching the interview, or just helping introduce us or whatever as part of it. It wasn’t until later that we had engineers involved in the interviews themselves, but that came along for us with another project later down the line.
Teresa: Then what about designers? Are there designers on your team?
Ellen: One of our product managers was also a designer for the mobile team. She actually started as a designer and then grew into a product role. She was acting as both in our interviews.
The Next Step: Thinking in Terms of Outcomes
Teresa: Your first foray into bringing the continuous discovery habits to your company, you focused on product managers, which makes sense. That was your team. You focused on interviewing, which is a great habit to start with. People get excited about story-based interviewing. It sounds like the organization saw the benefit of that and you got invited to talk to more customers. Where did it go from there? How did you introduce more of the framework to your company?
Ellen: Then we also started introducing outcomes to our priority roadmap. We had the to-do items, the outputs, and then went through and added what is the actual outcome that this would drive. It might be like, win this big customer that we were just working with, or increase engagement, or increase retention, or things like that. We would just link those together in our more sane spreadsheet, but it was still a spreadsheet of the items.
Then we could start introducing that into our priority conversations, like, “Does this outcome look right? Do we think that this feature would actually drive this outcome?” Whether they had stated the outcome in as many words. It’s like, “Should we build this, and then can we see that this will help us win this big customer or maybe some other customers?” Introducing those business strategy pieces into that conversation about priority.
Teresa: At this point, your priority meeting is now your five-person product strategy council, is that right?
Ellen: Yes. Product strategy board.
Teresa: You say you still have your spreadsheet, but now you’re looking at only the next six weeks. You’ve reduced the scope, you’ve reduced the number of people, and now you’re layering in for this row, that’s an output, that’s some solution that somebody’s proposing, you’re adding to it what metric do we think?
Ellen: Yes.
Teresa: What impact did that have on the conversations in that meeting?
Ellen: Also, I should say that within the product team, we had also built an outcome map that was our perspective of what we assumed about the business strategy based on conversations that we had. We were using that, and we’d introduce it here and there to some conversations like, “Does this look right?” but it wasn’t yet to the point of, “Let’s work on this together as a product strategy board or as an executive team to really nail this down.” We’d been thinking in terms of outcomes. One of my product managers took the Defining Outcomes class actually and was able to level up her business thinking around that.
That was an activity that we did within the product team and then used that to also fill in the gaps on the priority spreadsheet, the output spreadsheet. I think it helped us have a different conversation within the product strategy board of which of these outcomes that are listed on this spreadsheet are more important. It was hard to get people out of the mindset of like, “We’ll just build this thing. This is most important.” It was a step forward, but not a complete evolution yet.
Teresa: Just introducing the idea of outcomes in the context of the outputs you were already discussing. What was the next big piece?
Introducing Another Critical Discovery Habit: Story Mapping
Ellen: Then as we were working on engineering projects, feature projects, etc., we also started doing the story mapping. We’d had piecemeal interviews with customers here and there as an issue came up or they were requesting something. We’d always use that opportunity to ask, “Oh, tell me about the last time you needed to export metrics or make some decisions about staffing,” etc., so we got some information that way. We had piecemeal things here and there and so we had an idea about a feature and we had some relevant customer interviews. Within our product team, we were like, “Okay, here’s a couple of the opportunities that we’ve identified, here are the customer needs.”
We just had a really small export of our opportunity tree to then start some brainstorming solutions with the engineers because we knew that there was a need but didn’t have a clear sense of what the solution was. This was specifically for bus drivers. If they step out of their vehicle to help a passenger get in, maybe they’ve missed a message from dispatch, but they don’t have a way to see that on their screen, especially if they’re looking at their navigation map or something like that, because it just played the audio and they might not have heard it. We had a couple of those needs highlighted, and then we met with our mobile team.
This was me and the product manager who was also a designer—who was working with the mobile team—and a couple of the mobile engineers. We started with, “Okay, let’s just brainstorm some solutions here.” I’d also talked them through the process a little, we’re going to brainstorm, we’re going to story map, we’re going to identify assumptions, and then we’ll be able to test out which solution is best.
We did some quiet brainstorming, put some sticky notes on a digital whiteboard, and there’s some ideas in there that I wouldn’t have thought of that our engineers came up with, so that was great. It was like, “Great, it’s working and we’re getting more solutions than we would have otherwise.”
Then we picked a couple of solutions to story map in detail. Even as we were story mapping, including the one that the engineers came up with: “Use the sensors on the tablet to see if the bus has stopped and maybe we can use that in some way to surface a notification or something.” We started story mapping that, and it was like, “Oh, yes, what does this actually mean? They could stop for other reasons, like at a stoplight. We don’t want to pause notifications at a stoplight so let’s go back to our solutions and detail another one and maybe we can use this later on.”
It was really jumping around to some different solutions as well as like, “Oh, and we learned this thing when we story mapped this thing. Maybe we can add in this other piece of this other solution that we worked on.” It was cool to see even though we started with four or five ideas that we liked, we mixed and matched pieces of it as we detailed out the story map for each of those things. Even though it was pretty messy—it wasn’t a linear process—I felt like that got us a lot more clarity about the space of solutions that we could work within.
Even though story mapping was pretty messy—it wasn’t a linear process—I felt like that got us a lot more clarity about the space of solutions that we could work within. – Tweet This
Teresa: That’s great.
Ellen: That was within one hour.
Teresa: That’s amazing. Let me make sure I understand: Up until this point your product managers were interviewing, but your engineers weren’t involved. You were at the product strategy board level. You were starting to talk about outcomes, and with your product managers, you were starting to talk about outcomes, but this meeting was the first time your engineers had been exposed to any of this. Is that true?
Ellen: Yes. They had seen the interview notes and things that we had taken and understood that as we were talking through other things, but this was their first working session on that.
Teresa: Had they been involved in helping generate solutions prior to this?
Ellen: Yes. We had a pretty healthy back and forth when we were working on new features. A designer would come up with some multiple ideas and then be like, “What if we did this or this or which ones are technically feasible?” Then they might be like, “This one’s easier. This one is a little harder. Have you considered this option?” I’d say we’d had a relatively good working relationship between design and engineering on that front, but in a less structured way.
Tackling the Next Challenge: Getting Engineers Involved with Customer Problems
Ellen began sharing customer problems with engineers in much more detail, which also involved coaching them to help them make compare and contrast decisions and identify their assumptions.
Teresa: In the earlier days, your leadership team was saying, “This is the feature we need for this customer.” Your designer might do the design work and work with the engineers on the iteration. Was this the first time your engineers had been brought a customer problem?
Ellen: I think at this level of detail, yes. In terms of early involvement from: “Here’s a customer need, what do we build?” As opposed to: “Here’s a feature that we want to build, what’s the easiest way to do it?”
Maybe the exception would be very technical things like, “We want to take our cloud product and make it run not in the cloud.” Engineers could figure out how to do that, but that’s different from, “What’s the product solution that we’re building? Do you have any ideas of what it could look like or how it might work?”
Teresa: Two things. How did you set the context for them? How did you communicate the need that you wanted them to brainstorm around? Then, how did they respond to that ask? Were they just like, “Yes, let’s do this”?
Ellen: Yes, we had some regular weekly mobile design meetings, so it was easy to say, “Today we’re going to do this exercise, and here’s something a little new. This is some feedback that we heard from our customer of something that they’re having problems with, and we want to figure out some ways that we might approach this.” They’re really open to like, “Okay, great. Let’s give this a try.” In that first meeting, there was certainly some facilitation that I was doing, like, “Oh, that sounds like a new solution, or that sounds like this other solution that we did when we were talking about feasibility.” That’s the thing that engineers are really focused on.
They’re thinking ahead to: “Oh, well, what about this other thing?” Or “How will people know how to turn this feature on?” It’s like, “That sounds like an assumption. Let’s label that as an assumption on our story map here—by the time they get to this point, they will have noticed this.”
We’re assuming that they will be able to learn what it is or find the setting, and then we can put that on a sticky note and label it and then work on the other parts of it and come back to it. That involved redirecting their ideas or thoughts or concerns that came up in the meeting. I’d say: “Oh, great, let’s label that over here, or label it over here,” so I was able to help them through that process.
Teresa: It sounds a little bit like you used that working session to both teach, but also the activities gave you language to give structure to that conversation, right?
Ellen: Yes.
Teresa: If you had a lot of engineers going in a lot of different directions, it gave you a way to say, “Okay, well, we’re defining the solution at the story map level. That’s an assumption that we can capture over here. Let’s jump back over to this solution.” Right? It gave a sense that there’s a place for everything.
Ellen: Totally, yes, because the last thing that I want to do is be like, “No, stop talking. That’s not what we’re working on right now.” That just kills the energy. Even though sometimes they might get fixated on, like, “Oh, but this is a problem that we might have down the road when we put it in the App Store or something.” It’s like, “Okay, but before we get there, let’s figure out some other pieces of it.” Being able to redirect that energy without shutting it down was something that was important to me as part of those conversations.
The last thing that I want to do is be like, ‘No, stop talking. That’s not what we’re working on right now.’ That just kills the energy. – Tweet This
Teresa: That’s great. Okay, so you did all this in an hour. You started with an opportunity, you brainstormed solutions. It sounds like you story mapped several and started to identify assumptions. You did all of that in a single hour?
Ellen: Yes. We had talked a little bit about this idea or this need prior, so there was some context around it—it had come up a couple of times—but this was our first working session and our first session doing this process, too.
One of the engineers was like, “I don’t know how my input is working here, maybe you all can just do this without me next time.” He was less interested in participating or just didn’t know how best to participate, but the other one was like, “Oh, yes, this is great,” and could join in.
It’s interesting to see when people learn a new process, they feel like they’re doing it wrong and that makes them feel like their ideas are wrong. That was the tricky line to walk: “Your idea is not wrong, but let’s label it a little differently.”
It’s interesting to see when people are learning a new process, they feel like they’re doing it wrong and that makes them feel like their ideas are wrong. – Tweet This
Teresa: It sounds like you covered a lot of ground in that hour, but there were maybe some mixed feelings about how it went from the engineering perspective. What happened after this meeting?
Ellen: We had a follow-up meeting where we did more of the assumptions specifically. This was following up on the solutions that we thought about. One thing that I also like to do when we come back together is ask, “Has anybody had any other new ideas, new solution ideas? Probably you’ve been thinking about this in the back of your head. Are there any other things that we should make sure to incorporate?” Just setting that as a pattern that new ideas are always welcome as maybe there’s something better that we haven’t thought of yet and seeing if there’s some additional thoughts or modifications to the story maps that we detailed out and then getting into the assumptions of like, “If we’re using the sensor on the tablet, that assumes that there’s some convenient software development kit (SDK) for tying into that and detecting these things that we have access to or that we have permission for.”
Or that, “If we’re pausing notifications for some reason, that we make that really clear.” Or that, “If there’s something visual that we want to show about notifications, that it can be visible even when the app isn’t in the foreground or that it can be persistent in some way around that.” Identifying those and then putting the assumptions on the known versus unknown and important versus unimportant scale too, and copying those over.
Teresa: It sounds like your second session got more into like, “Let’s go deeper on assumptions, and then let’s prioritize with assumption mapping.” Did the engineer who expressed some reluctance after the first meeting participate in that second meeting?
Ellen: I think he did, but it was more like observing as opposed to jumping in on things. We ended up having to put this particular project on hold, but we picked up the process again with a different feature idea later. When he’d gone through that process once before, then he was more excited about participating the second time, which was thinking about how to quick reply to people. When you hear a message and then you just want to respond to just that person without having to open up the app and do the things.
We followed a similar process for that idea of saying, “Okay, we’ve got a need of being able to quickly send a message back to somebody without having to do a lot of steps. Is there a way that this can work?” We went through a similar process of saying, “Here’s story mapping some ideas and identifying some assumptions.”
One of the assumptions was, “People would know that they’re just replying to one person when the default is sending it to everybody.” That was the thing that the engineers kept coming back to: “Well, how do they know?” It’s like, “Well, this is one really big assumption that we have. Let’s put that over here. We also have a big assumption that it’s helpful or that this is useful for people.”
These are the ones that as we went through the process and they ended up on the chart, in the top right of like, “Okay, these two big assumptions, is there a way that we could test one of these assumptions?” I was like, “We could really quickly build a prototype of the app that’s when you get a direct message, your reply button is a direct message button for the next five or ten seconds or so, and we can just test that out.” Even after talking for 20 minutes about how people find the setting and turn it on and know what it does, it’s like, “Well, let’s just put that aside for the moment and just make sure that what we’re building would be useful for people.”
Then they were able to put together a quick prototype. They also participated in watching the user test. We just tested it internally with some of our employees and so the engineers watched that. Even the one who was reluctant to participate in the process at first. Then it was like, “Oh, well, people really like this, but we have to make this one key change to the sound effect or whatever so that people know what’s happening.” Then it was like, “Oh, great, this is unlocking more ideas and more possibilities” and some tweaks with a relatively small amount of work that went into a thing that could have been pretty complicated or we could have spent many hours opinionating about it over time.
Ellen’s Key to Success: Avoiding the Idea of Perfection
Teresa: You know what I really love about this story is that you really did work on one habit at a time and you didn’t worry about whether everybody was involved in every habit. You looked at where the organization was at. Where can I introduce different pieces for the people who are receptive? Your product managers are pretty receptive to interviewing. Your engineers are solutioning, so they were a little bit more ready for brainstorming and story mapping, and assumption mapping.
One thing that I noticed is some people get really stuck on: “I need a perfect trio. I need everybody doing everything and we need to adopt the whole framework wholesale.” You did not get stuck on that at all. You just looked at, “Where do I match different pieces of this framework to how my organization works today?” What do you think gave you the ability to do that?
One thing I’ve noticed is some people get really stuck on: ‘I need a perfect trio. I need everybody doing everything and we need to adopt the whole continuous discovery framework wholesale.’ – Tweet This
Ellen: I would say experience in trying to stop everything to introduce new process at other points earlier in my career and seeing that that was very unsuccessful. Ultimately, the goal of frameworks and processes like this is to help the business move faster and help you make better decisions and less risky decisions. If you’re telling everybody to stop what they’re doing and learn something, then you’re slowing everything down. Even the perception of slowing things down can get in the way of like, “Well, I just need to get this thing out. We just need to build this thing.”
There have been times in the past when I had come back from a UX conference where I learned all these things and said, “Hey, everybody, let’s try these new techniques. Stop what you’re doing and do this other thing instead.” That was just met with, “No, I don’t have time for that. We have things to do.”
When introducing the new discovery habits, I needed to meet people where they were at and do the least resistance merging of things. I didn’t want to have to try to teach 40 people how to do continuous discovery all in one go.
When introducing the new discovery habits, I needed to meet people where they were at and do the least resistance merging of things. – Tweet This
I was like, “Okay, well, I can talk with this manager about interviewing and I can talk with this manager about brainstorming and solution mapping. I can talk with the product strategy board about outcomes over time gently.”
I’ve noticed especially in introducing outcomes versus outputs—when you change the way you do product strategy, you’re also asking people to change the way they do business strategy. That can be really challenging, especially coming from not the CEO, right? I’m not the CEO, I’m not the head of sales, I’m not the CFO. I’m the head of product and so asking everybody else to change the way they do business strategy is a big conversation and can be really challenging. Still in process today is having that outcome discussion. We’re still iterating on that.
Teresa: Let’s talk about the timing of this. If I remember right, you took our Master Class in the fall of 2020.
Ellen: Yes.
Teresa: We’re four, five, six months into COVID working from home, you took the Master Class, it’s now two and a half years later. You’ve been slowly introducing different habits to different folks as you felt like people were receptive. It’s two and a half years later. How would you describe the progress that you’ve made, how your teams are working today, and how satisfied you are where you are and what’s next?
Ellen: Let’s see. From the top down, I now start all of our product strategy board meetings with the outcome map. Even though there’s still some work to be done there on like, “Hey, everybody else helped contribute to this,” but we start the conversation with the outcome map and like, “Does this still look right? What are we tweaking? What have we learned recently that we should adjust here?” I posted an example of this in Slack, too. The next step of the outcomes across the top with some of the feature solution ideas under that. I was like, “Okay, we had decided that these outcomes were most important, so this is the kind of work stream that’s related to that outcome.”
Whether it’s focusing on a new industry or other partnerships or engagement kind of things, we can see these projects are related to this outcome and that’s where we’re going to focus, but it’s pretty high-level definition of the project as opposed to “add this one button to this webpage to build this thing.” It’s more outcome focused on the priority level whenever I talk to customers.
We’ve had some different folks in our sales and CS positions too, and so I’m much more brought into customer discussions and try to do as much of the story-based interviewing as I can whenever I’m in those conversations. It’s like, “Oh, this customer wants this integration with this other product. Can you add that?”
It was like, “Okay, well, let’s talk about their workflow and what they’re doing today. Tell me about your last day at work.” Being able to successfully integrate that into customer conversations and much to the benefit of everybody that’s been able to hear those or see the notes on this too, they can get more context on the customers.
I’d say while it’s not a regular customer interviewing weekly, they’re still frequent, but they’re like, “They’re talking to somebody in transportation this day and retail the next day.” It’s regular, but maybe not as frequent as I would like and not as much with the actual end user. We’re often talking to managers or supervisors or the buyers or even IT people.
It’s still through a bit of a layer of abstraction. Like, “Oh, this person that has this role and they do this job and they need this information,” as opposed to “Build this feature.” I’d say that’s been more successful.
Then introducing the other brainstorming like we did actually for a recent project that we’re working on that will be probably launching in a month or two, we really started with, “Hey, we have all this feedback from customers that were using an early version of this new feature and now we want to build the real version. Let’s take this from the beginning.”
I built an opportunity solution tree based on the interviews that we had and the feedback and met with the engineering folks that would be working on it. This was more the backend folks that would be primarily working on this voice workflow, and so we started with the opportunity solution tree, brainstormed some solutions. A couple of those rose to the top that we detailed out in story maps.
This was again a lot of facilitation through the process, on my part, of like, “Oh, you had a new solution idea when we were working on the story map for this other solution. Let me put that up at the top here and we’ll come back to that.” Trying to redirect the energy into the process as opposed to like, “Stop, don’t talk, do this process.” That’s not fun.
It was definitely messy, but to generate and talk through in detail, there ended up being three different major solutions that we’re working on, identifying the assumptions. That was something that, in the past, we would’ve had a possibly contentious debate about which solution was better, but being able to have the detail of what the solution is, what are the assumptions that we’re making about it, helps facilitate if we do a thing on the backend or on the client end. Because the client had a lot more risky assumptions if we went that way. Then there were much smaller and easier assumptions on the backend solution way.
Having that all in sticky notes on a whiteboard—it’s not big mockups or UI prototypes or anything—literally sticky notes on a digital whiteboard helped everybody take a look at these things and see, “This one makes a lot more sense. Let’s go that way,” with testing and things along the way too. I think that was a big success story for the opportunity, solution, assumption testing part of the process.
Teresa: One thing I love about story mapping and generating assumptions is that you learn so much even before you run an assumption test. It’s this act of externalizing our thinking and getting everybody to align around like, “How might this work? What are we assuming along the way?” Even if teams just do that—they never run an assumption test—I feel like their solutions are going to be way better. Then obviously if we compare it with assumption testing, you get even more value.
It sounds like you are still in the middle of this. You’ve introduced habits in a lot of different places. You’re iterating on them. You’re getting people excited and comfortable with different pieces of it, but it seems to also already be having an impact. Is that a fair statement?
Ellen: Yes, definitely. It’s taken the opinionating out of the process a lot for these, especially for the big meaty ideas that people have a lot of feelings about or a lot of attention on and directing those feelings and attention into a process of like, “Oh, yes, we’ve done all this work and here are the things that you can look at where we’re at in the process and what we’ve worked on and what we’ve thought about.” It isn’t just like, “I came up with an idea on my own and everybody has to do what Ellen says.” It’s like, “We did this together. We’ve worked on this together and here’s the work that we did.” Having that artifact is helpful for just shortcutting some of those concerns that come up.
Teresa: That’s really great. Well, Ellen, this has been a really great conversation. Thank you so much for taking the time to tell us a little bit about your org transformation.
Ellen: Thanks for having me.
Want to learn more from Ellen? Ellen is teaching Continuous Interviewing starting on June 15th and Opportunity Mapping starting on July 27th.
