Setting Product Goals

Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery.

Each quarter, I set goals for my team. I start by asking, what’s the most important thing we can accomplish this quarter that would create the most value for the business.

Q3 will be my fourth quarter with my current company and once again our goals this coming quarter will be different from our previous quarter’s goals.

To give some concrete example, here were our product goals for the past 3 quarters.

Q4 2012

• Double Monthly Active Users (MAU)
• Put a system in place to track at least 5% of job seeker outcomes.

Q1 2013

• Double MAU who view a job

Q2 2013

• Move to Sign Up Cohort Analytics

You can see there is a lot of variety in these goals. They’ve shifted from quarter to quarter as my understanding of our needs have changed.

When I first started, there were some obvious things I knew I could put into place to quickly grow MAU. For example, we weren’t emailing our registered users on a regular basis.

When I started at the company, there was a big emphasis on growing our registered users. While this is important, I also wanted us to grow the percentage of registered users who were active. This, ultimately, is what drives business value. So this is where I wanted to focus.

The second goal that first quarter was around tracking outcomes. I wanted to know how many of the job seekers on our site got a job. This isn’t an easy problem to solve. Employers don’t want to tell you when they hire someone (for too many reasons to get into here) and job seekers have no reason to tell you, as they no longer need your site.

But from my view, if we can’t track outcomes, we have no idea whether or not our product is working. This wasn’t optional. We had to figure it out. So we set out to solve this problem. And we did. And we blew past 5%, which was awesome.

With a good foundation in place, I set a Q1 goal based around trying to improve those outcomes. We had a lot of people visiting our site for other reasons than viewing jobs. My thinking was, if we could get more of them to view jobs, we’d get more of them to apply jobs, and then more of them would get jobs.

We didn’t even come close to hitting this goal. I made a number of incorrect assumptions. First, I was still getting up to speed on our products and I grossly underestimated the number of entry-poitns to our service, the reasons why people came to our site, and how I could impact their behavior.

Basically, I spent most of the quarter optimizing the wrong things. It was unfortunate. But it happens.

So last quarter, when it came time to set goals, I was pretty frustrated. I don’t like to miss my goals. In fact, I prefer to blow them out of the water. So I spent a lot of time (weeks) working to understand why we missed our goal.

Embarassingly, I came to a conclusion I should have already known. Setting goals baed on changing behavior in your monthly active users is silly.

It’s way too large of a group. It takes time for changes to propagate through your active users. And if your service is like ours, where we have lots of entry-points and many reasons why people visit us, it can be a nightmare to affect the overall number.

So for Q2, I took a step back, and said, we need to get better visibility into what our visitors are actually doing. We need to understand how they are finding us, what their intent is, and how well we are delivering on that intent.

So we set a goal around improving our product analytics. Our goal was to improve the cause and effect relationship between the product changes we were making and the outcomes we were measuring.

We actually switched to looking at the performance of each weekly sign up cohort. From this perspective, we actually didn’t do too bad back in Q1. We made a 2.5x improvement in the percentage of new sign ups who were viewing jobs. Not bad.

In fact, weeks after the quarter ended, we started to see this change propagate through our active users. As they returned to the site, they benefited from the changes we made and we influenced their behavior. That made me feel better about the product changes we made. But it didn’t make me feel better about how we decided to measure the impact of those changes. We still got that part wrong. Live and learn.

So what’s the point of all of this?

It’s hard to set goals. Even when you know what you want to improve, it’s hard to know how to measure the impact. You may not get it right the first time. Or the second time. Or the third. Or the fourth.

I’ve been setting goals for a long time and I still get it wrong pretty often. But as long as you keep trying, you’ll keep learning. And each iteration, you’ll take one step closer to where you want to be.

How do you set product goals? Are they consistent from quarter to quarter?