Writing Acceptable Acceptance Criteria

Have you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery.

You’ve written your user stories, engineers have accepted them, development is in progress. Now what? More often than not, odds are you left off one of the most important parts of a user story – the acceptance criteria.

Acceptance criteria tells you when a user story is ready for release. Writing good acceptance criteria can be a challenge. But it’s worth the effort as the quality of your product will go way up.

Let’s take a look at a few things that should be included in your acceptance criteria.

First, criteria should be written from the perspective of the end-user or customer. That seems obvious. After all, user stories  are written from the perspective of the end-user or customer. But even for the best written stories, the acceptance criteria is where the engineering perspective seems to creep in.

The easiest way to do this is to write the acceptance criteria before the story is implemented. Many teams leave the acceptance criteria to engineering or QA. But this will often lead to verifying that the functionality that engineering built works rather than verifying that the intended user behavior exists. This is a fine subtlety, but often where most miscommunication between product and engineering happens. If you write and review the acceptance criteria before implementation begins, it is more likely to capture the user intent rather than the engineering reality.

Be sure to include measures of usability in your acceptance criteria. In other words, you need to identify how to answer the question, is it easy to use? Don’t’ just write, “must be easy to use.” How are you going to measure this?

For any given feature, we may measure ease-of-use by looking at the rate of successful use, setting an acceptable threshold. Or we might set a threshold for time to completion. The key is to identify the right measure and to make sure it is quantifiable.

Your acceptance criteria should also include how you will handle error cases. What data are you expecting that might not be there? How will you handle each instance of missing data? What happens if someone does something out of order? Or they stop half way through and then come back later? What happens if someone doesn’t have the data you are asking for? Can they move on or are they stuck? Enumerating the error cases is a great opportunity to surface all of your assumptions, particularly around what data you need, and how to resolve them if they are wrong.

And finally, your acceptance criteria should include performance and stress test thresholds. What’s the difference? Performance testing is from the perspective of an individual user. Does the page load quickly? Is the UI responsive? Do I have to wait for ads to load before I can view the content. Stress tests measure what happens when you are inundated with lots of users, transactions, or queries. What happens when the system is under stress? Acceptance criteria should define acceptable thresholds for each of these areas.

In this post, I argued user stories are conversation starters. This is no more true than when looking at acceptance criteria. No matter how technical you are as a product manager, you will likely need help from your engineering team to define adequate acceptance criteria. However, it’s your job to hold the perspective of the end-user or customer. Don’t get bogged down in the technical detail. Remember why you are building each story, and make sure that the acceptance criteria validates that benefit, not just the technical integrity of the story.

What are your acceptance criteria gotchas? Please share in the comments.