In recent years software organizations strives for client satisfaction and for that reason testing focus changes more on what customer needs. Acceptance Test- Driven Development is one of the wonderful testing practices where business users will get to test the product, which makes this practice not only be most important for client but is the last major test before delivery. In this article I will present how to create Acceptance Testing Driven Development tests in .NET using SpecFlow. SpecFlow is an open source tool, it integrates with Visual Studio. You can define automated test using Selenium Web Driver for browser interaction, write scenarios in 100 % Gherkin language and automation code in C#.
A successful Continuous Integration pipeline is defined by automated tests that run fast, have good coverage and no erroneous results. The automation tests are usually divided in multiple suites each with their own objective: unit tests, acceptance tests, integration tests, system tests and security tests.
Acceptance tests are a crucial stage in the deployment pipeline. They take delivery teams beyond basic continuous integration. Once you have automated acceptance tests in place, you are testing the business acceptance criteria of your application that is, validating that it provides users with valuable functionality. Acceptance tests are typically run against every version of your software that passes the commit tests.
An individual acceptance test is intended to verify that the acceptance criteria of a story or requirement have been met. Acceptance criteria come in many different varieties: they can be functional or nonfunctional. Functional criteria are related to end-user scenarios, closely connected to the business processes that the application enables, while nonfunctional criteria deal with the operation of a system, rather than specific behavior of functions. Performance, capacity, availability, security, flexibility to change, error handling and usability are good examples of non-functional acceptance criteria. The key point, is that a specific story or requirement is considered complete and working, only when its associated acceptance test are demonstrated to pass.
The acceptance test suite as a whole, not only ensures that the application delivers the business value expected by the end-user, it also helps mitigate potential defect regression or breaking changes to the application's functions. Automated acceptance tests catch serious problems that unit or component test suites, however comprehensive, could never catch. From a development cycle point of view, there is another major benefit to emphasizing acceptance testing: it is one of the few processes that ensures that all parties are involved in the delivery process, the customers, the analysts, the project managers and the development team (developer, testers and operational).
On complex application especially, the overhead to create and maintain a proper acceptance test suite is significantly lower than the cost of performing frequent manual acceptance and regression testing, or that of the alternative of releasing poor-quality software. Effective acceptance testing should be done for each release, as a formal phase once development is complete and a release is approaching.
In .NET world SpecFlow represents one of the most popular plug-in for Acceptance Testing Driven Development.
SpecFlow aims at bridging the communication gap between domain experts and developers by binding business readable behavior specifications to the underlying implementation. It is an open-source .NET tool inspired by the Cucumber framework which allows writing specification in human readable Gherkin format. Gherkin is a Business Readable, Domain Specific Language that lets you describe software behavior without dealing with how that behavior is implemented and required functionality for a given system.
The most frequently reported benefits of using SpecFlow are:
- Encourages collaboration between team members - both technical and business oriented
- Feature description can be written and/or understood by non-technical people
- Implementing changes more efficiently
- Higher product quality
- Less rework
Integrating SpecFlow with Visual Studio is easy and consists in two steps. The first step is to install Integrated Development Environment using the Extensions and Updates option from the menu, as you can see in the screenshot bellow:
The last step is to set up the project from Visual Studio to work with SpecFlow by installing the corresponding NuGet package.
SpecFlow supports multiple popular test execution frameworks like Nunit and MsTest, but also comes with a dedicated test execution engine named SpecFlow+Runner.
Like any other tests from Cucumber family, a SpecFlow acceptance tests requires: a feature file, step definitions and business code.
In order to define the behavior of the system a creation of Feature file is mandatory. The feature file contains an overview of the user story functionality and the scenarios written in Gherkin language. Having cross functional teams working on feature file creation is advised as it leads to improved team integration and communication, spans organizational boundaries and reduces the production cycle time in new product development flows. A feature file usually contains a list of scenarios, which starts with the word Scenario. Using tagging you are able to group features and scenarios together and you can reuse parts from already defined scenarios. Every scenario is defined by a list of steps, which must start with one of the keywords: Given, When, Then, But or And.
Here is an example:
The Given keyword sets up preconditions or context for the scenario. The action, the behavior that we've focus on is marked by When. The scenario validation is performed using the Then statement; it actually verifies that the right things happen in the When stage. And can be used in any of the three sections and it serves as a nice shorthand for repeating the Given, When or Then. There are some situations where a negative step is required and the But keyword can be used.
Translating lines into steps is performed by right-clicking in the feature file and selecting "Generate Step definition". A wizard tool will create the step file skeleton.
Executing acceptance tests at the UI level requires browser interaction, where the Selenium Web Driver allows us to abstract the communication with the web application. Inside of the auto-generated steps we can execute different operations based on the Selenium Web Driver functionality which facilitates Document Object Model interaction.
Like any other automation tests, in order to handle test initialization and test clean-up, SpecFlow provides hooks: BeforeScenario and AfterScenario.
Generally, the hooks can be used to perform additional automation logic on specific events. Even if they are global, they can be restricted to run only for specific features or scenarios. The BeforeScenario and AfterScenario hooks contain automation logic that has to run before/after execution each scenario, for example, browser initialization at the beginning of each scenario and browser disposal after each test execution.
As best practices when we create an acceptance test with SpecFlow you should take into consideration the following:
feature files should actually be features and not entire portions of an application, should use descriptive title or tags and should contain scenarios that are independent and deterministic
steps should be small, simple and concise chunks of code, you should aim to be re-usable, have a single responsibility and a good separation of concerns
write scenarios for the non-happy-path cases as well
Building the product right and building the right product are two different things, but you need both of them to succeed.
Automated acceptance tests represent an important step in Continuous Integration pipeline not only as a quality gate in delivery process but also focusing on creating business- process documentation which help us avoiding the common long-term maintenance problems.
Implementing acceptance tests leads at integrating cross functional teams where testers, analysts and developers work together to build the right specification of the system. If the communication between business stakeholders and developers is clearly the software teams work best and avoiding misunderstandings by defining acceptance tests also will raise the team collaboration level.
Having acceptance tests implemented using SpecFlow tool allows us using the real world example to describe the behavior of the system which makes sense to our stakeholders.
"Specification by Example: How Successful Teams Deliver the Right Software" 1st Edition 2011 by Gojko Adzic
"The Cucumber Book: Behaviour-Driven Development for Testers and Developers (Pragmatic Programmers) 1st Edition" 2012 by Matt Wynne and Aslak Hellesøy
