Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. 
I recently sat down with fellow Product Talk coach and instructor Hope Gurion to discuss some of the common mistakes we see teams make when it comes to defining outcomes.
You can watch the video of our conversation or check out a lightly edited version of the transcript below.
Full Transcript
Teresa Torres: Hi, I’m Teresa Torres.
Hope Gurion: And I’m Hope Gurion.
Teresa Torres: We’re product discovery coaches.
Hope Gurion: And today we want to talk to you about the most common mistakes that teams and leaders make when they’re trying to set outcomes for their product teams to work towards.
Mistake #1: Disguising Outputs as Outcomes
Teresa Torres: The first one is we see a ton of outcomes being set where really it’s just an output, right? So we say our outcome is to deliver an Android app, where really the answer is: Did you do it—yes or no? It’s not really an outcome. That’s just taking an output and putting it in outcome language.
We see a ton of outcomes being set where really it’s just an output. If your outcome is ‘deliver an Android app,’ that’s just taking an output and putting it in outcome language. – Tweet This
Mistake #2: Not Connecting Outcomes to Business Value
Hope Gurion: The second most common mistake that we see is when we define outcomes, not tying them to our company’s strategy or relating them to our company’s revenue model or business model.
The challenge for teams in this situation is that they might be working towards a goal. It might be something they can measure. It might even be something they think is meaningful, but they can’t effectively understand or communicate the degree to which they’re having an impact on the business because they can’t link it or relate it to a company’s strategy or the way the company makes money. Or if they’re in a cost-saving mode, the way they save money.
That makes it very challenging for teams when they’re negotiating. And this is frankly a product leader’s responsibility to give them that strategic context and to understand or diagnose when your teams either don’t have sufficient understanding of the strategy or don’t understand the company’s business model. It’s very awkward to find out when you are watching the teams try to make progress towards something that they don’t really understand why it matters.
As a leader, it’s very awkward to find out when you are watching your teams try to make progress towards something that they don’t really understand why it matters. – Tweet This
Mistake #3: Giving Teams Outcomes That Are Outside Their Span of Control
Teresa Torres: Definitely. And this is tied to our third common mistake, which is when teams are given outcomes that are outside their span of control. We see this a lot when executives think in terms of business outcomes and they just give those business outcomes to a product team—things like “grow revenue” or “increase market share.” Very few product teams directly impact those outcomes. Instead, what they can impact are product outcomes, behaviors that occur in the product, and they often need help translating from business outcomes to product outcomes.
Mistake #4: Hyper-focusing on a Traction Metric
Hope Gurion: Another thing that sometimes happens—which is almost at the other end of the spectrum that you just described—is they actually set what we call a traction metric as a team’s outcome. This may be within the team’s realm of influence, within their span of control, and happens in the product, but it is usually too narrow for a team to focus on.
An example of something like this would be to increase the percentage of users who viewed the performance report, something that’s maybe behavior-based, maybe within the team’s realm of influence, but it is too narrowly defined and doesn’t give them enough latitude to explore the opportunity space or enough connection between that and the company’s strategy and business outcome. And oftentimes it can be something that maybe the customers don’t even care about. It’s almost like a vanity traction metric for the company.
Teresa Torres: Yeah, the way I like to test whether a traction metric is a good discovery outcome or not is to ask if it is possible to have a happy customer who never uses that feature. If the answer to that question is yes, then why are you trying to convince a happy customer to do something they don’t want to do? We prefer to see outcomes that span features, so focusing more on the value you get from using those features. And we’re looking at sets so that your customers can choose the way that they want to get there, and we’re not forcing them down a very specific path.
Mistake #5: Creating Too Many Dependencies Across Teams
Hope Gurion: Another challenge that we see is when companies set outcomes for teams, and it’s again beyond their realm of influence. Sometimes it is what you said—which is that it’s a business outcome, and it usually requires multiple teams to be able to make progress on a business outcome. It’s also a lagging indicator.
But sometimes we are setting outcomes and then recognizing that we have inter-team dependencies. Maybe it’s other product teams, maybe it’s the marketing team or the customer support team. And that can be very frustrating and demoralizing for a product team when they’re tasked with an outcome that they really have to coordinate with other teams who may not share that same outcome or may have a different outcome that they’re working towards. So again, we want to make sure that we’re finding that sweet spot of an outcome that’s within the team’s realm of influence, or if it truly needs to be a multi-team outcome, to make sure multiple teams who have influence on that outcome are working together to achieve it.
Mistake #6: Measuring Actions Instead of the Value of Those Actions
Teresa Torres: Definitely. And a little bit related to this, we want to measure the value to the customer. Oftentimes that’s not very measurable, so we often see teams shy away from setting that as their outcome.
I’ll give an example from my own personal experience. I worked at a company where we tried to help new college grads find their very first job out of college, and our company was really tempted to measure the value as applying for a job. You were successful when you found a job you wanted to apply to. It’s very measurable. It happens on our site, no problem. But I argued, I don’t get value from applying for a job. I only get value when I get the job. And I really wanted to push our teams to measure all the way through to the hire. There were real challenges with that. Employers don’t want to tell you when they hired somebody, and students certainly don’t want to follow up. They’re too excited, they got a job. They’re not coming back to the job board.
But we found creative ways to do it, and it really led to a much better product because we focused on when the customer gets value. So on day one, it doesn’t have to be measurable. It just has to be directional. And then knowing with time, you’ll figure out how to measure it as you learn more about how to impact it.
On day one, your product outcome doesn’t have to be measurable. It just has to be directional. – Tweet This
Mistake #7: Setting Sentiment Outcomes Without Any Further Direction
Hope Gurion: Another challenge that impacts teams when they have outcomes that are challenging to work on is when they’re just focusing on a sentiment metric like customer satisfaction or NPS—which is even worse, because it’s a lagging customer sentiment metric.
Customer satisfaction and customer sentiment is important—and I want to make sure you recognize that—but when we have such a broad sentiment-based metric, you can make it very challenging for teams to please everybody all the time, which is kind of where you end up when you just have a sentiment metric. So what do you think we should do in those situations?
When you have a broad sentiment-based metric like NPS, you can make it very challenging for teams to please everybody all the time. – Tweet This
Teresa Torres: The other challenge with just a sentiment is it’s not directional. If I’m trying to increase customer satisfaction, I can do anything and everything. So you haven’t actually given the team any guardrails. What we want to look for is: What’s the behavior that a successful and a satisfied customer would do and how do we measure that behavior? A really common one that we see, especially on consumer products, is engagement. Are they engaging with our product? We think a satisfied customer would engage.
In the industry right now, we kind of have an obsession with engagement, and we can take it too far. We see this with Facebook. Facebook was very good at getting us to engage. Most of us engaged with Facebook multiple times a day, definitely daily. And what’s happened is we now have some hostility towards Facebook. We don’t like how addicted we are.
That’s what happens when we focus on engagement at the cost of satisfaction. I like to think about pairing a sentiment metric with a behavior and thinking about it as two sides of the same coin. How do we drive engagement without negatively impacting satisfaction? How do we increase satisfaction while making sure that we’re also encouraging specific behavior in the product?
I like to think about pairing a sentiment metric with a behavior and thinking about it as two sides of the same coin. How do we drive engagement without negatively impacting satisfaction? – Tweet This
Mistake #8: Setting Outcomes Without Considering Accountability
Hope Gurion: Another mistake that I see companies and leaders making is they rush to develop outcomes for their teams, ideally with their teams. And they haven’t really looked at what their model of accountability looks like. And that is also very challenging because it will directly impact not just the outcomes that are set and negotiated, but also the progress that your teams will make, not only towards those outcomes, but to the things that matter in terms of your company strategy and company objectives.
If you are seeking to have more outcome-oriented teams, you want to take a look at the accountability models. How did we make progress towards it? What did we learn? What are we going to do differently next time? Those more reflective accountability models as opposed to individually penalizing performance review associated accountability models, which tend to lead to people being much more risk averse and much more driving towards output-type outcomes, outputs disguised as outcomes, because they want to make sure that they have certainty and they don’t want to take those risks on. So that’s another mistake that I see teams and leaders making.
If you are seeking to have more outcome-oriented teams, take a look at the accountability models. How did we make progress towards it? What did we learn? What are we going to do differently next time? – Tweet This
Teresa Torres: On the accountability side, a way to think about it is: Are you encouraging a learning mindset or are you encouraging a performing mindset? And it’s really hard to get both at the exact same time. So we like to see outcomes used to encourage learning. Obviously we also want to see our outcomes go up, so we want to see some performance. When we optimize for performing, we start to see things like teams sandbagging on their outcomes, individuals competing with each other instead of collaborating. So a good thing to keep in mind is when setting outcomes, think about learning rather than performing.
How to Avoid These Common Pitfalls
One thing we’ll share is we love that more and more teams are talking about defining outcomes. We do see a lot of common mistakes, which is why we designed a class called Defining Outcomes that teaches you how to start with your business model—how your company makes money—to make sure that your outcomes support business value, and then Hope has designed some really great activities to help you map out different ways you can influence those outcomes and really look at what’s the most important one to prioritize right now.
If you want help learning how to set better outcomes, take a look at that course. You can find it at learn.producttalk.org.
We’d love to see you in a future cohort of our Defining Outcomes course. Explore all the Product Talk Academy courses here.
