Toward Scrum, part 6: Acceptance Criteria

Russell Bateman
December 2017

This is a Scrum concept, the idea that what's finished is what's asked for. The product fulfills requirements, those requirements were known before starting development, everyone is on the same page.

Acceptance criteria are (yes, criteria is the plural of criterion) a formal list enumerating requirements imposed by a user, a product owner or a manager. All the product scenarios are part of these criteria. All the conditions under which the product will be tested, distributed and operated are described. Anything not in the criteria isn't a criterion (requirement).

What distinguishes acceptance criteria from a user story is generalized detail. This means that the story is a summary or description in general of a requirement for the purpose of a development team working toward its completion, however, the acceptance criteria are a separate, formal list stating what must be in order for the product to be accepted.

In physical terms, acceptance criteria could be kept in the same document as the story, but interactions between the products or components described by two or more stories are usually part of the acceptance criteria, so inclusion of criteria with the story details is usually insufficient.

At the end of the sprint, the product of finished stories (or, sometimes, tasks) is demonstrated—this is a feature of Scrum Agile—and such a demonstration should clarify points of acceptance criteria.

Benefits of acceptance criteria

Identifies user stories associated with a product. This means it's a reverse index to stories and this introduces a difficult, Agile requirement in that, when stories are updated, multiplied or deleted, acceptance criteria must be maintained.
Acceptance criteria establish a common understanding between the end user, product owner, etc. and the development team. The development team go to the acceptance criteria to learn requirements and confirm that they're doing or have done what is required.
As development (stories) is broken down into tasks, upon completion, a task should be confirmed as meeting (or not) a requirement. Adjustment of story tasks is always a possibility when it becomes clear that requirements are not being met.
Acceptance criteria are a crucial aid in the estimation of resources including time in the accomplishment of the work to be done.
Acceptance criteria will help the development team to progress faster and more fluidly because the project scope and end product are well documented.

Composing the list of acceptance criteria

Typically, this is a collaborative effort by pigs.

A developer could write acceptance criteria, but that would suppose he or she has super-knowledge of the finished product. He or she would have to be in close and frequent contact with the product owner. More likely, the user or product owner (product manager) would write acceptance criteria, but if he or she doesn't have sufficient knowledge of technical details, a developer will have to be brought in to expose those details sufficiently to be enumerated.

When to write acceptance criteria?

It's inconceivable to write these after development has begun since development should not start until after the break-down of stories into tasks. Break-down cannot be done without requirements. In essence, requirements represent the smallest amount of work that must be done and still meet what the client needs to declare a successful product.

Broken arrow

Though feared, it's not inconceivable that the development team discover that they're failing to develop the product hoped for. This discovery helps them stop (early and often) and correct their process. Such discovery will not come if there are no acceptance criteria.

It's also a possibility that, once written, acceptance criteria will reveal to users, product owners and managers that the effort, as defined, fails to meet the requirements of a useful product.

Sample criteria

These are given in the presence of user stories.

User story		Acceptance criteria
As a student, I can see tuition cost and what		- the tuition is displayed
I've paid so that I know how much I owe		- the balance paid is displayed
Before semester's end.		- the balance left to pay is displayed
		- nothing is displayed when student identification is not made

A suggested template for writing acceptance criteria might be to divide statements into given..., when..., then... clauses. For example:

Given		User clicks to put something into his or her shopping cart.
When		User's order is verified/validated/fulfills the system requirements,
Then		User's order is accepted (and passed on to payment) or denied, and this is displayed.

However, the above can be a somewhat awkward imposition in some cases. It's just one way of many by which acceptance criteria could be expressed.

Acceptance Criteria: an Earlier Take

I expressed some of the above in a different way a long time ago...

The concept of acceptance criteria is one with close ties to Agile, however, like much of Agile, it's of use to us. The point of this document is to answer the question, "How do I compose acceptance criteria for my ticket?"

Background and discussion

Some background may be useful to set the stage. Be patient, this is actually relevant and helpful in this discussion. A user story (in Agile) is first a back-log item then a current-iteration item defining a feature or change to a feature of a system. It is most often couched in the form

"As a ___________ I want ___________ so that ___________ ."

Example

"As an administrator, I want to create user accounts so that new users can be added to the system."

Acceptance criteria are most meaningful at the story level. In Agile, each story is decomposed into a series of small tasks realizable in just a few hours each. (Most schools of Agile hold that any task that takes longer than a few hours should be broken down into deeper constituent parts that can and failure to do so is an indication that development does not yet fully understand how to complete the task.) These tasks taken together complete the story, but may not individually, as in the case of a research task preliminary to accomplishing the rest of what will conclude the story, imply anything concrete in terms of "acceptance."

Indeed, depending on your point of view, tickets do not always rise to the level of a story (i.e., a complete feature), but are rarely so granular as a mere task (except when reporting a bug). Still, the ticket is the only concrete manifestation of accountability and so, acceptance criteria can not be easily established at any other level.

While the language of a story's expression is a silly thing, even in Agile, it does achieve making the story's purpose unmistakable. We could do a much better job naming tickets (if not going quite so far as to use the sentence format). Doing a better job naming tickets leads to doing a better job at

Passing tickets between ourselves, especially to newer staff members.
Writing acceptance criteria, which in turn...
Alleviates the burden on QA along the same lines as #1 above.
Writing unit tests.

Remember, our Assembla tickets are nothing so granular as required by Agile schools. They are stories without tasks or "mega-tasks." However, the implications of this are part of a different discussion. Our tickets should easily translate into good acceptance criteria nevertheless.

Acceptance

Acceptance criteria fall (English: one "acceptance criterion", two or more "acceptance criteria") in one or more of the following areas, as appropriate in describing what demonstrates the completion of the story (actually: ticket). Most tickets will not an example of each and every one of these.

functional ("When the administrator clicks on the Create button, a dialog appears.")
nonfunctional ("The form Create button will be blue.")
defects (Take as a given: no outstanding tickets!)
performance ("The Create dialog appears immediately.", "User creation demonstrates no delay.", etc. or perhaps none at all)

Formal example of acceptance criteria...

...for a pretty big story or ticket. Here are some non-relevant examples that sort of continue the administrator user-creation story already used to illustrate. These are concrete examples of what might be expressed in an Assembla ticket. Many tickets would not need so many, but the more the criteria describe the outcome of the tickets' implementation the less the developer and QA have to interpolate and guess at to write unit tests and to verify for the final ticket state:

If an administrator, I can create a new user account.
I can create a new user account by entering the following information about the user:
1. name
2. e-mail
3. address
4. phone number
5. account status (Active/Inactive)
6. reports to (from a list of other users)
I cannot assign a new user to report to an "Inactive" user.
I cannot assign a new user to report to another if it creates a cyclical relationship (e.g.: user 1 reports to user 2 reports to user 1).
The system notifies me that it sent an e-mail to the new user's address, containing a system-generated initial password and other instructions.
I am able to verify with the intended recipient of the e-mail that it was received.

← →