Author: Dennis de Booij ● dennisdebooij@gmail.com
Review: Kimberly Snoyl, Kaspar van Dam
The irony of Behaviour Driven Development (BDD) is that it is a software development process to reduce confusion about what software is supposed to do but the term itself is continuously confused with another software development activity: testing. At some point, the confusion was so widespread that the originator of the term BDD, Dan North, gave a conference talk titled: ‘BDD is not about testing‘. That should have sorted the problem, right? Not quite.
When I go to LinkedIn and look for jobs with BDD as keyword, the the top results for my country are Test roles (Image 1).
Try this for any other country and you will get similar results. So, LinkedIn’s job market search algorithm seems to disagree: BDD is primarily about testing. Even more ironic is the reason that I am writing this article. I participate in the Testnet work group for BDD. Testnet is the Dutch association for software testers, and it aims to exchange knowledge related to the testing of IT products. So, why does this BDD work group exist within Testnet when the founder of BDD tells us that it is not about testing?
What’s in the name?
To explore this question, we must go back to the reason why the term BDD was formulated. Behaviour Driven Development was initially coined as an alternative name for Test Driven Development (TDD). TDD is a style of programming where you:
- First write a single unit test;
- Then run the test so you can see it fail (as the new functionality has not yet been built);
- Write just enough code to make the test pass;
- Refactor (rewrite) the code to make it simpler, cleaner and more understandable.
Dan North noticed that a lot of developers that he worked with believed that the idea of TDD was to test your code before writing the actual program[1]: ‘It has resulted in a generation of programmers who think they’ve somehow “got testing covered” because they used TDD.’[2] However, the purpose of TDD is to compel you to design your code well: to write your unit of code in such a way that it does just one thing well so it is easy to understand. Testing, on the other hand, is more than just confirming what you already know. If you believe that testing is limited to writing unit tests, you will be thoroughly disappointed when your code runs in a production environment[3] (See example in image 2).
To get away from the notion of ‘test’ in TDD, Dan North used the term Behaviour Driven Development, and he applied it on a larger scale. He was no longer focussing on a unit of code but on what a software program enabled the user to do: on the feature level. You define examples in the form of a scenario that describes what behaviour an end user exhibits and how a program should react to that. These scenarios are structured into three parts:
Given some initial context (the pre-conditions),
When an event occurs,
Then some outcomes should happen.
Here is a concrete example of this structure:
Scenario:A professional successfully signs up for a training course
Givena training course 'BDD for beginners’ has been planned for ‘12 June 2023’
Andthe training course 'BDD for beginners’ is not yet fully booked
Whenprofessional ‘Jane’ registers for the 'BDD for beginners’ training course on ‘10 May 2023’
Then'Jane’ is registered as participant for the training course 'BDD for beginners’
And'Jane’ has received the option to save the date ‘12 June 2023’in her calendar
This Given-When-Then structure has been formalized in the Gherkin language which is used in for instance the Cucumber tool for automating BDD scenarios. Collect enough examples and you have specified what behaviour you want to see in the software solution you are about to build. When you can automatically run your scenarios, you have executable specifications. Once the software solution is built, you will be able to check if the program behaves the way you have described it on the feature level. As the entire BDD process is set up as a team effort, the idea is that the behaviour that the team has specified guides the development process as everyone knows what the intended outcomes are.
However, the software development community has not universally adopted the idea that BDD is about something else than testing. People define the unknown in terms of what they already know. So, people still associate BDD with TDD as that is where the idea came from. Case in point: the Agile Alliance website defines BDD as ‘a synthesis and refinement of practices stemming from Test Driven Development (TDD) and Acceptance Test Driven Development (ATDD).’ Naming things is one of the hard challenges in software development so in hindsight a different name would have been better to avoid the confusion with testing[4]: Example-Guided Design (Image 3).
Defining BDD
Although the name may be problematic, the main reason for the ‘BDD equals testing’ misunderstanding is that there is no authoritative source on what BDD is and what it is not. Although Dan North is a wonderful storyteller, he has not written the definitive guide on BDD. His motivation for not writing the BDD bible is that he does not want BDD to be bounded to one rigid description but instead welcomes anything that encourages collaboration between all the stakeholders involved in delivering customer value through software[5].
Furthermore, as a friend of Dan has pointed out[6], attempts to define BDD can be problematic as well:
‘BDD is a second-generation, outside-in, pull-based, multiple-stakeholder, multiple-scale, high-automation, agile methodology. It describes a cycle of interactions with well-defined outputs, resulting in the delivery of working, tested software that matters.’ Dan North, 2009[7]
Although every phrase in this definition is meaningful and can make sense if you are familiar with the context, it is not helpful for someone new to the field that is trying to learn what BDD is about. So, if there is no definitive and accessible source of truth about BDD, is the term Behaviour Driven Development open for anyone’s interpretation?
Let us dive a little deeper to see if we can find a more helpful description of BDD. In an interview at a conference in 2007, Dan North emphasises that BDD is about behaviour[8]: what I want the system to do to help me solve problems. Behaviour is what drives the software development process for writing and delivering a computer-based solution that solves the problem for you. You define how such a system would look like and how it would behave. In short, BDD is about analysis. Specifically, about defining ‘a ubiquitous language for the analysis process itself!’[9]: a way to define requirements using a consistent vocabulary for analysts, testers, developers, and business people to eliminate a lot of the ambiguity and miscommunication that can occur when technical people talk to business people. In the Testnet workgroup, we have currently expressed these ideas in the following definition:
Behaviour Driven Development is:
What: A collaborative method to achieve a shared understanding among stakeholders (business & IT) about the anticipated behaviour of the user and the desired behaviour of the solution.
How: By using practices which can include impact mapping, event storming, example mapping and specification by example.
Why: To enable people to document, test and develop the right ‘thing’ (the solution).
BDD practices
So, if BDD is a team effort to get everyone on the same page, why is it that my LinkedIn search result for BDD primarily lists job openings for testers? Where are the information analysts or business analysts in the top results? Even better, why aren’t the top results a mixture of job openings for product owners, analysts, domain experts, programmers, testers, UX designers, scrum masters or anyone else that is needed in a team? Is BDD something that only testers like?
To address these questions, we need to look at the practices within BDD. These practices are commonly sliced into the following three steps:
- Discovery;
- Formulation;
- Automation.
Many people focus on the third step because automation is the part where they think BDD can make a difference, especially if you know how to automate like programmers do. It is easy to explain these steps using testing terminology. For example, on the website of Specflow, a BDD-framework, these three steps are explained as follows[10]:
- Discover – specify collaboratively using examples;
- Formulate – write Gherkin test cases in natural language;
- Automate – Avoid regression by automating test cases.
The problem with using this type of testing vocabulary to explain BDD is that it represents a shallow notion of testing. It limits testing to confirmation: is what we currently know about the system still true? But, to quote Dan North, ‘the fact you got an example working says nothing about whether it is secure, compliant, resilient, observable, compatible with anything else, unless your design journey included examples of these. Which it probably didn’t. It probably had examples of behaviour for the most part.’ [11]. ‘In no reasonable sense should you consider an application “tested” just because you designed it that way‘[12]. So, there lies a real danger in equating BDD to testing. You will limit your test coverage to functional coverage only and ignore all the other types of risk in your software product that can cause damage. To quote Rob Meaney: ‘Deep testing is focused on uncovering the important ways in which a system won’t behave as desired, not on showing it can work once.’[13]
As a counterpoint, BDD focuses on uncovering the important ways that a system should behave before we start building it. The most important step in BDD is the first step: discovery. Dan North calls this step ‘Deliberate Discovery’ to emphasize ‘that the biggest impediment to your throughput was what you didn’t know.’[14] Gáspár Nagy and Seb Rose, who wrote ‘The BDD books’, have described the three steps more accurately and without references to testing[15]:
- Discover – explore behaviour using examples;
- Formulate – express examples using Given/When/Then;
- Automate – create and maintain executable specifications.
Why testers find BDD appealing
So, if BDD is indeed not about testing why should I take an interest in BDD as a tester? Well, it is easy to see that BDD can appeal to testers:
- Collaboratively exploring what the product should do will lower the risk of building the wrong thing;
- Having many different sets of eyes on the ball will increase the chance that requirements are explicit, unambiguous, non-contradictory, correct and complete before we start designing the solution and writing code;
- BDD begins with the end in mind: you start with conversations about acceptance criteria from the perspective of all the stakeholders.
In the past, the primary focus of most testers was to find problems in the first versions of a software solution to prevent negative impact on the user and the business. Testers in an agile context have broadened their focus by getting involved in the early stages of the software development process. They will employ their skills to prevent problems from finding their way into the software solution in the first place. BDD provides a way to strengthen this important role by making this early involvement a collaborative effort that is designed to achieve the highest possible level of quality for all stakeholders involved in the BDD process.
So, while BDD can enable testers to hit the ground running at the start of the development process, testers are certainly not the only stakeholders in the software development process so do not get BDD confused with testing.
BDD for all stakeholders; not just testers
Behaviour Driven Development challenges an idea: the idea that we as business and IT people know enough to start writing code and delivering software – even when we have not done our due diligence. BDD is about proving that we can achieve a deep and shared understanding about our problem space within a reasonably rapid time frame before we start documenting, testing and developing. Ironically, there does not yet appear to be a shared understanding on what BDD encompasses. BDD is often confused with testing primarily because it originated from Test Driven Development (TDD) and there is no clearly defined and generally accepted description of BDD. Many agile testers are drawn to BDD because it helps to prevent problems from finding their way into the software by collaborating early as a team.
Therefore, we propose the following definition of BDD: Behaviour Driven Development is a collaborative method to achieve a shared understanding among stakeholders (business & IT) about the anticipated behaviour of the user and the desired behaviour of the solution by using practices which can include impact mapping, event storming, example mapping and specification by example. The end goal is to enable people to document, test and develop the right ‘thing’ (the solution).
With this article we hope to wake up all stakeholders, not just the testers, but foremost our goal is to get rid of the confusion around BDD. So, grab your customers, business analysts, product owners, software developers, UX designers and anyone else that matters. Ask them: do we really understand how all our users behave (or misbehave)? Do we really know what all our stakeholders need (and actually don’t need)? Have we given enough thought on what problems we are trying to solve? If not, let’s start the deliberate discovery of BDD.
This article is based on the discussions and activities within the workgroup BDD of TestNet. Testnet is the Dutch association for software testers. If you would like to reach out, contact us at bdd@testnet.org
[1] ‘Introducing BDD’, article by Dan North (2006)
[2] Tweet by Dan North on Twitter (26 Dec 2018)
[3] Tweet by Ben Halpern on Twitter (9 Nov 2019)
[4] Tweet by Dan Terhorst-North on Twitter, 26 December 2018
[5] Comment by Daniel Terhorst-North on a LinkedIn by Bilel Loussaief, 6 February 2023
[6] Conference talk by Gojko Adzic ‘BDD: Busting the myths’, 2013
[7] Wikipedia entry on BDD with a citation of Dan North’s definition of BDD in 2009
[8] Interview with Dan North on Behavior-Driven Development, OOPSLA conference, 2007
[9] ‘BDD provides a ‘ubiquitous language’ for analysis’ from ‘Introducing BDD‘, Dan North, 2006
[10] Three Phases of BDD Development, Specflow website
[11] Tweet by Daniel Terhorst-North on Twitter, 26 December 2018
[12] Tweet by Daniel Terhorst-North on Twitter, 26 December 2018
[13] Tweet by Rob Meaney on Twitter, 26 December 2018
[14] Introducting Deliberate Discovery, Daniel Terhorst-North, 30 Augustus 2010
[15] The BDD Books website, Gáspár Nagy and Seb Rose