Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery. Last month, I spoke at the Business of Software (BoS) conference in Boston. BoS has been on my short list for a very long time and it did not disappoint.
Mark Littlewood, the organizer of the event, does a fantastic job and it shows up in every little detail of the event. The audience is made up of entrepreneurs. Most seemed early in their entrepreneurial journey, but there were plenty of experienced folks as well.
What stood out to me was how friendly and eager to learn all the attendees were. There was a distinct lack of Silicon Valley swagger and posturing. As a result, they felt like my kind of people.
The following is a rough summary of my talk. Unlike most events, I didn’t script out my talk ahead of time. But I’ve been thinking about this topic for a long time and it was fun to have a venue to explore it further.
A big thank you to Rich Mironov who referred me to BoS organizer Mark Littlewood, and to Jeff Merrell, my co-instructor at Northwestern, who helped me develop many of these ideas.
Talk Video
Rough Talk Script
Managing by outcomes has been a popular topic for quite some time.
Most recently, it’s been re-popularized by Google advocating for the use of Objectives and Key Results (OKRs). But OKRs pre-date Google. Andy Grove wrote about their use at Intel in his book, High Output Management.
Long before Google or Intel, Peter Drucker famously quipped, “If you can’t measure it, you can’t improve it.”
And yet, we still struggle with managing by outcomes. Today, I want to explore why.
Let’s start by getting some clarity on terms.
When we manage by outputs, we are asking our teams to build specific features, implement specific projects, or deliver on specific initiatives. We are dictating the actions of our teams.
When we manage by outputs, we are asking our teams to build specific features, implement specific projects, or deliver on specific initiatives. We are dictating the actions of our teams. – Tweet This
We may assume that these actions will lead to specific outcomes. We may even share those assumptions with our teams. But, ultimately, when we ask our teams to deliver specific actions or outputs, we are managing by outputs.
The clearest indicators that you are managing (or being managed) by outputs are roadmaps that list a fixed set of features with release dates, Gantt charts that sequence the year’s projects, and annual budgeting cycles where you commit to funding specific projects.
When we manage by outcomes, on the other hand, we are stepping back from action. We aren’t defining features, projects, or initiatives that must be delivered.
Instead, we are defining the outcome we expect a team to achieve. An outcome is typically defined by a quantitative metric—a number that is both meaningful to the business and measurable by the team. It creates autonomy and accountability.
An outcome is typically defined by a quantitative metric—a number that is both meaningful to the business and measurable by the team. It creates autonomy and accountability. – Tweet This
This is often easier said than done. Defining the right metric is challenging in and of itself, let alone setting up the infrastructure to measure that metric reliably and accurately.
This is one of the reasons why OKRs have become so popular. By defining both a qualitative objective and one or more quantitative key results, OKRs allow us to take multiple iterations to define the correct metric without changing the overarching objective.
Even in a world where leaders manage by outcomes, most leaders still need more insight into how their teams will reach those outcomes.
If done well, the leader may communicate strategic initiatives that might influence how a team reaches those outcomes. Or a team might communicate key opportunities they intend to address to reach their outcomes.
In neither case should this inch toward defining specific actions—features, projects, or initiatives—that must be done to reach those outcomes.
The goal is to give the team as much flexibility and latitude as possible to explore the best path to reach those outcomes.
Unfortunately, as managers, even though we’ve been talking about managing by outcomes for decades, most of us don’t have experience managing by outcomes. And, therefore, we aren’t very good at it.
We start off well. We define outcomes for each of our teams, but then we continue to dictate the actions the team must take to reach those outcomes. We end up managing by outcomes and outputs.
We set annual OKRs and commit to a year’s worth of projects during the annual budgeting session.
We assign quarterly OKRs and then we either dictate a fixed roadmap or we ask the team to commit to a fixed roadmap.
We spend all of our time discussing actions and only talk about outcomes during the one or two weeks we spend defining what outcomes to set next.
We are still managing by outputs with some outcome language sprinkled around the edges.
Why does this happen?
We have a trust issue.
Managers don’t trust that teams can reach outcomes on their own. Therefore, managers micromanage their teams’ outputs. As a result, teams don’t communicate their progress toward their outcomes. Which leads to managers continuing to mistrust their teams’ ability to reach those outcomes.
Managers don’t want to be this involved in the details. Teams don’t want to be micromanaged. But until we address the trust issue, managers and teams will find themselves going around and around this vicious cycle.
Managers don’t want to be involved in the details. Teams don’t want to be micromanaged. But until we address the trust issue, managers and teams will find themselves going around and around in a vicious cycle. – Tweet This
To build trust, we need to do two things.
- We need to teach teams how to communicate their progress toward an outcome.
- We need to teach managers how to give better feedback.
We’ll look at both today.
Most teams communicate progress by communicating outputs. We talk about what’s in the next sprint. We share 12-month roadmaps. We talk through project plans.
This sets managers up to give feedback on those outputs.
When teams only communicate their conclusions—the features they’ll implement, the projects they’ll execute, and the initiatives they’ll deliver—managers focus their feedback on those conclusions.
This keeps us solidly in the world of micromanaging outputs.
Too often, when we comment on outputs, we are simply stating our uninformed opinion or personal preference.
Too often, when we comment on outputs, we are simply stating our uninformed opinion or personal preference. – Tweet This
We simply don’t know enough about the context in which the decision was made to give better feedback.
Let’s use a silly example to illustrate this point.
Imagine you were trying to make a kid smile. Your team comes to you and says, “We want to give the kid a red balloon.” You think about it for a minute and respond, “I like blue balloons. Let’s give the kid a blue balloon.”
What happened here?
We never bothered to ask why our team wanted a red balloon. We didn’t communicate why we like blue better. We simply expressed our opinion.
And we probably forgot to ask ourselves, “Is there a meaningful difference between a red balloon and a blue balloon? Or am I simply expressing a personal preference?”
It’s easy, as managers, to assume that our opinion or preference is based on our prior knowledge and expertise. And sometimes it is. But if we don’t take the time to reflect and ask what we are basing our opinion on, we risk outranking our team’s opinion on a whim.
This is why we have the dreaded Hippo acronym.
If we want to break the mistrust cycle, we need to learn to give better feedback.
Giving better feedback requires that our teams share more of their thinking—show more of their work.
Giving better feedback requires that our teams share more of their thinking—show more of their work. – Tweet This
I encourage teams to use an opportunity solution tree to expose more of their thinking. If you are new to this visual, you may want to start by reviewing this article: Why This Opportunity Solution Tree is Changing the Way Product Teams Work.
At Business of Software, I talked through how teams can use opportunity solution trees to show their work. This was the subject of my recent Mind the Product San Francisco talk. Rather than repeating it here, I’ve omitted that section of this talk. If you need to catch up, check out Show Your Work: How to Justify Your Decisions & Get Better Stakeholder Buy-In.
If you had to jump off to reach those last two links, welcome back.
Now that we know how teams can use opportunity solution trees to show their work, let’s talk through how leaders can avoid the red balloon/blue balloon problem.
And put the dreaded Hippo acronym to bed for good.
The most important feedback a leader can give their team is what outcomes they should be focused on. This should be a two-way negotiation between the leader and the team.
The leader should bring the across-the-business view of what the business needs at that moment in time. The team, if they are a true continuous discovery team, should be closest to the customer with a deep understanding of the technology, and should communicate how much progress can be made on what timeline.
For example, a leader may suggest that reducing churn is the most important problem in the business to address. If the business knows that engaged customers churn less, the leader may ask the team to focus on improving engagement.
The team then needs to do the work to understand what’s the right way to measure engagement, what’s the current baseline, what they have tried to do to increase engagement in the past, and how successful those efforts were. They then need to communicate how much they think they can increase engagement in the next quarter.
And this is where the negotiation starts. If the business needs more than the team thinks they can deliver, we need to look at alternative approaches. The team may need more resources. The team may need to take bigger risks with their experiments to find more upside coupled with the leader fully understanding and accepting those risks.
The key is to be on the same page about what is possible and what it will take to get there from the beginning.
Once an outcome is set, the team should work to explore the opportunity space. How we frame the opportunity space impacts what solutions we might explore. See “Show Your Work: How to Justify Your Decisions & Get Stakeholder Buy-In” for more.
Managers should focus their feedback not just on the outputs—the solutions on the tree—but also on how the team is framing the opportunity space.
Leaders can ask, “Is there a clear understanding of the opportunity space?”
Too often, teams forget to explore the opportunity space at all. They jump straight from an outcome to a list of ideas.
Too often, teams forget to explore the opportunity space at all. They jump straight from an outcome to a list of ideas. – Tweet This
Leaders can ask, “Is the opportunity space too shallow?”
It’s easy, after conducting one or two interviews, to jump to solving the first few problems you heard. But we want our teams conducting an exhaustive search. We want them to develop a deep understanding of our customer’s context and needs.
Leaders can ask, “Are these opportunities real?”
Has the team framed the opportunities well? Do they look like real customer needs? Can they support each with evidence?
Leaders can ask, “What opportunities are missing?”
Often times, leaders have knowledge or expertise that teams don’t have. They may have access to a different set of customers. They may have knowledge about business constraints that the team doesn’t have. They may have encountered similar opportunities in their previous roles and can help flesh out more detail around specific opportunities.
Leaders can look for opportunities that are not framed from the customer’s point of view.
I encourage teams to frame opportunities from the customer’s point of view as this helps us to remain human-centered. But this is easier said than done.
The team is close to the product (or project or initiative). It’s hard to maintain the customer’s perspective.
Leaders can help by testing each opportunity. Does this sound like something our customer might say? Or is it too business-centric?
Leaders can look for opportunities that are solutions in disguise.
The best test for whether an opportunity is a solution and not an opportunity is to ask, “Is there more than one way to address this opportunity?” If the answer is no, it’s not an opportunity, it’s a solution in disguise.
As you reframe solutions as opportunities, the leader can help the team reflect on the real need behind the feature request. Often nuances behind these needs can make or break a solution. It’s very hard to deliver a good solution if you don’t first fully understand the underlying need.
Leaders can ask, “Are opportunities well structured?”
Did the team group similar opportunities in a way that logically makes sense? Is it easy for an outsider to quickly grasp the opportunity space?
Leaders can ask, “Are opportunities distinct from each other?”
One of the biggest challenges in structuring the opportunity space is making sure that each opportunity is framed as a distinct need.
We want to rewrite vague needs to be specific. We want to reframe overlapping needs to be truly distinct needs. This is what is going to allow us to focus on solving specific problems iteratively down the road.
Leaders can help identify where opportunities need further work.
The teams should be able to walk a leader through how they are using the tree structure to prioritize where to focus. For more on this idea, see “Prioritize Opportunities, Not Solutions.”
Leaders can ask, “Are they evaluating all opportunities or are they using the tree structure to help them simplify this decision?”
Finally, leaders can ask, “Can the team justify their target opportunity decision?”
Did they make data-informed decisions (without getting bogged down with analysis paralysis) at each decision point?
When we help our teams frame the opportunity space, we share our knowledge and expertise and we learn how our teams are thinking about the problem space.
When we help our teams frame the opportunity space, we share our knowledge and expertise and we learn how our teams are thinking about the problem space. – Tweet This
Once teams have chosen a target opportunity, I encourage them to explore multiple solutions that address that same target opportunity. This helps them set up good “compare and contrast” questions and avoid faulty “whether or not” questions. See this video/article for more on why: How Compare and Contrast Decisions Lead to Better Product Outcomes.
As leaders, instead of falling into the blue balloon/red balloon trap, we can ask questions and give feedback that help our teams better explore these solutions.
First, we can catch when teams are ideating across their whole tree rather than focusing on their target opportunity.
Most teams are drowning in ideas, but they are drowning in a lot of first ideas that each address different problems. This doesn’t help us focus and it also misses the point of ideating.
If your teams present multiple solutions, the first thing you should ask is, “Do all of these solutions address the need?” If the answer is no, you can encourage them to refocus on their target opportunity.
Usually when we refocus, we realize we don’t have that many ideas and we need to keep ideating. I like to encourage teams to generate 15–20 ideas. This helps us get past our first obvious ideas and helps us generate more innovative solutions.
Academic research supports this idea as well. The more ideas we generate, the more diverse and novel those ideas tend to be. If you want to get away from “me too” ideas, focus on generating more ideas.
The more ideas we generate, the more diverse and novel those ideas tend to be. If you want to get away from “me too” ideas, focus on generating more ideas. – Tweet This
Finally, our teams need to be prototyping and experimenting to evaluate their different solutions. They need to be collecting data that helps them evaluate which idea looks strongest.
You can help by giving feedback on their experiment design. Have they identified their riskiest assumptions? Are they missing key assumptions? Can you identify assumptions they may have missed?
Does their experiment plan look sound? Are they experimenting with the right audience? Are they collecting the right data?
And finally, we want to help our teams when they fall prey to the “scurvy problem.”
This is another silly example from my article, “Why This Opportunity Solution Tree Is Changing the Way Product Teams Work.”
This challenge is so important—and leaders play such a critical role in addressing it—that I’m going to include the whole excerpt here.
Imagine you are the captain of a ship crossing the Atlantic in the 1500s and your crew comes to you and tells you that several crew members have come down with scurvy. You now have an opportunity to cure scurvy amongst your crew. So you start to generate ideas.
You suggest, “Citrus is supposed to cure scurvy. Let’s give them oranges.”
A crew member responds, “Oranges could work. But I prefer grapefruit. Let’s give them grapefruit.”
To which another crew member responds, “I don’t want to clean up all those peels. Let’s give them apples.”
And the crew concludes, “Let’s give them apples.”
We end up with a solution that doesn’t deliver on the opportunity, which means we don’t deliver on our desired outcome—crossing the Atlantic safely.
Now each objection that took us from oranges to grapefruit to apples was reasonable. They were important constraints that should be considered. But they led to an outcome where we land on a solution that doesn’t solve the target problem.
This happens all the time. I see it every week. Taking the time to externalize this visual structure will help you to catch these errors.
Now I’ll be the first to admit that this really is a silly example. But I do see it every week. Over and over again, teams fall in love with shiny ideas. Unfortunately, they evolve over time in a way that no longer serves their target opportunity.
As a leader, you want to keep an eye out for this and help your teams save time by not solving the wrong problem.
So where does that leave us?
Our teams have learned how to show their work and we’ve learned how to ask questions and provide guidance that reinforces and improves that work.
Rather than arguing about red balloons vs. blue balloons, we are working together to build better solutions. Isn’t that exactly what we want?
So remember, if you want to break the vicious cycle of mistrust, first teach your teams how to show their work, then fix how you give feedback. Ask more questions and don’t fixate on the conclusions.
The one on the left outlines what your team should be doing. The one on the right outlines how you can give guidance and support the process.
>Long before Google or Intel, Peter Drucker famously quipped, “If you can’t measure it, you can’t improve it.”
Would you be so kind as to provide your source for this Drucker quote? I’m trying to track it down, and am having a hard time finding where Drucker actually said this. Thanks!
Hi,
To my knowledge, that quote belongs to Edwards Deming.
i was an intern at intel/Aloha facility when Andy Grove was there. I was taught “management by objectives” – which is kinda what your talk is about. Lord Kelvin famously quotes about measuring to improve search for the quote. – that is often used in quantitative ( as well as qualitative measure)
Great article ! So many product organisations suffer from the ‘hippo’ disease.
Love the way you put it into perspective here.
Adam Nash’s quote is a great fit here – “always keep in mind: what game are we playing and how do we keep score”
How could we manage bu outcomes when developing new product with a 6-month roadmap?
Lean startup?
I’m not really sure what you are asking.
Another post that resonates so well. This is even more pronounced in the services industry. There are very real project constraints like scope and budget, which gives the perception that there isn’t time to get nuanced around outcomes. And, everyone wants to hastily speed past this part by choosing revenue or CVR, then get to talking about their initiatives (outputs).
Thanks as always for sharing your knowledge, Teresa!
Micromanaging creates a negative cycle where taking initiative is punished because how the task was completed, or the particulars of the result are criticized. It teaches employees that they should seek guidance and check-in often to ensure they are on the right track.
While I cannot but agree with Teresa on that we need to Manage to Outcome rather than Output. Fundamental problem that author fails to address is “why do we do, what we do”. It is for the efficacy of measurement. It is easier to measure outputs (lead) than Outcomes (Lag)
If you step back and try to understand the Outputs we typically get in dev organizations as requirements from Upstream teams are basically derived from Outcomes by the upstream business and demand process teams. This is done through a process we call as the Impact Mapping.
To address this problem, what we need to do? Change organizations need to get better at collaborating with their Business teams to derive lead indicators which are close match to the Outcome and can be tracked by Ops and Business team to corroborate the hypothesis for the Project.
Agree this may be not be a big deal in smaller organization as you can have a 1-1 match to the change and then impact post the implementation. But in larger organization this is not so, as you have multiple things running at any time and finishing at any time and it is difficult to say the outcome you see is directly relatable to which project and / or campaign. Unless we come up with measures that are close to outcome and relatable to the change, and who and how the measurement will happen and at what frequency – this continues to be theoretical pursuit than practical especially in large organization