Writing acceptance criteria that settle arguments early

A Sprint ends. The Developers demonstrate a story that says "a parent can cancel a booking". The Product Owner asks where the refund is. The Developers say nobody mentioned a refund. Both are right, and the argument that follows could have been settled in four sentences two weeks earlier.

What acceptance criteria are

Acceptance criteria are the conditions that a specific backlog item has to meet before it is accepted as complete. They belong to the story, not to the project. The SBOK® Guide describes them as the objective components by which a user story's functionality is judged, and makes the reason explicit: stories are subjective, and the criteria supply the objectivity needed to call a story done or not done.

That is the whole job. A story says what someone needs. Criteria say how you will tell when they have it.

The Scrum Guide does not use the term. It talks about Product Backlog items becoming ready through refinement, and about the Definition of Done, which is a different thing. Acceptance criteria are a practice that sits comfortably inside Scrum, and many teams use them, but Scrum does not require them.

Criteria, Definition of Done and Definition of Ready

These three get confused constantly, and the difference matters.

The Scrum Guide's Definition of Done is the quality bar for the whole Increment: the moment an item meets it, an Increment is born, and an item that does not cannot be released or presented at the Sprint Review. Acceptance criteria are specific to the story: for the refund story they might say the refund is full when the cancellation is made more than 24 hours ahead. A story needs both. It can satisfy its criteria and still fail the Definition of Done if, for example, it has no automated tests. Our post on the Definition of Done covers the team-wide bar in more detail.

The Definition of Ready comes from the SBOK. Its Guide describes it as a set of criteria a story must meet before it is considered for estimation and inclusion in a Sprint, one of which is that stories have well-defined acceptance criteria. The 2020 Scrum Guide does not include a Definition of Ready, so using one is a team choice. The risk to watch for is that it hardens into a gate that holds up work for the sake of paperwork.

Two ways to write them

As a checklist

The simplest form is a short list of things that must be true. Here is an illustrative set for a cancellation story.

• A parent can cancel a lesson from their booking list.
• A cancellation more than 24 hours before the start time gives a full refund.
• A cancellation inside 24 hours gives no refund and says so before the parent confirms.
• The tutor is notified, and the slot becomes available again.

Each line is checkable. Someone can sit down with the finished software and tick them off. Mountain Goat Software's examples have the same shape, with four or so criteria per story, each of which reads as a plain statement of behaviour.

As scenarios (Given, When, Then)

Some teams prefer to write criteria as examples. The Gherkin format, used by Cucumber, structures each as three kinds of step: Given puts the system in a known state, When describes an action or event, and Then describes the outcome.

Given a parent has a booking that starts in 48 hours When they cancel it Then the lesson is removed from their list and a full refund is issued

This works well when behaviour depends on conditions, because each condition gets its own example. It can also be turned into an automated test later. The cost is length: five or six scenarios take more space than a short checklist. Use whichever form your team will actually keep up to date.

How many, and how detailed

Few. A story with twelve criteria is probably two or three stories. A useful range is three to six. If a criterion keeps being argued about, it deserves its own example. If you find yourself writing validation rules for every field, you have started writing a specification. Keep the detail for the conversation and the tests.

Criteria should also be written before the Sprint starts, ideally during refinement. SBOK is firm on one related point: the Scrum Master should make sure the Product Owner does not change a committed story's acceptance criteria in the middle of a Sprint. A moving target is the surest way to turn "done" into a matter of opinion.

Fixing a weak criterion

Most criteria start out weak, and improving them is quick if you ask the same two questions each time: how would someone check this, and what would a different reading of it produce?

Take "the booking list is easy to use". Nobody can check "easy", and two Developers would build different things. Ask what ease means here, and the team might settle on something like: a parent can reach their next lesson from the home screen in two taps, and the list shows tutor, date and time without opening anything. That version can be tested, and it tells the Developers what the Product Owner actually cares about.

A good moment to do this is refinement, with the Developers in the room, because they will spot the readings the Product Owner did not think of. A criterion that two people read differently is a defect in the story, and finding it before the Sprint costs minutes where finding it at the review costs a Sprint.

Partial credit does not exist

Under SBOK, a story that meets most but not all of its criteria cannot be accepted as done. The Guide's reasoning is that the last piece is often the hardest, and carrying half-done stories into the next Sprint disrupts that Sprint, so done is black and white. The Scrum Guide reaches the same place by another route: work that does not meet the Definition of Done is not part of the Increment, and it goes back to the Product Backlog. In both cases the team does not take credit for 90 per cent of a story.

Common mistakes

• Criteria that restate the story. "The user can cancel" adds nothing. Say what happens next.
• Criteria that cannot be checked. "The page loads quickly" is a wish. "The booking list loads in under two seconds with 100 bookings" can be tested.
• Implementation in disguise. "A cancel button is in the top right" fixes the design before anyone has discussed it.
• No negative cases. What happens when something is not allowed? Error and boundary cases are where most disputes begin.
• Writing them alone. Criteria drafted by the Product Owner in isolation miss what the Developers know. Write them together.
• Using them as a contract. They are a shared understanding. If new information turns up, change them before the work is done, not after.

How to practise it

The way to get criteria right is to see what happens when they are vague. In the simulator, the Product Owner accepts or rejects each story against the Definition of Done at the Sprint Review, and rejected work returns to the backlog.