It seems a new test framework is gaining traction on the internet (especially in the Java area) reviving the concept of BDD. Cucumber is a software tool for running automated acceptance tests in a business-readable domain-specific language. BDD or Behavior-Driven Design, however, has been around for a while, the first articles and projects (JBehave [5] and RSpec [6]) appearing in 2007. As mentioned on the site of the project, Cucumber is a rewriting of the previous two frameworks.
The most important concept to grasp in this article is the concept of BDD. Behaviour-Driven Design is not just about choosing a relevant test framework and start writing tests, it's about really understanding the idea behind this development process (See articles [7] and [8]). Only then can we expect to obtain value from it. BDD was created out of a need for better collaboration between business people, developers and testers. If we want to profit from using this process we need to start with the idea of a joint effort between all stakeholders.
A Cucumber test requires the following 3 items
A feature file is a plain text file written in Gherkin language, a business-readable domain-specific language that allows the description of the application's behavior in free text.
Below is a sample feature file:
Feature: Refund item
Scenario: Jeff returns a faulty microwave
Given Jeff has boght a microwave for $100
And he has a receipt
When he returns the microwave
Then Jeff should be refunded $100
The feature file is composed of at least one scenario (which is essentially a test) which is in turn composed of at least one step. Feature files can also have different supporting constructs like descriptions, comments, tags, a before hook and different ways to parameterize scenario steps.
As we saw above, a scenario is composed of multiple steps like “Jeff has bought a microwave for $100” and “he has a receipt”. All those steps are plain text abstractions of supporting code also called glue code. In order to support the first step, we need to define a Java method like the following
@Given("^Jeff has bought a microwave for \\$(\\d+)$")
public void Jeff_has_bought_a_microwave_for_$(int amountOfMoney) {
business.buyMicrowave(amountOfMoney);
} The Gherkin step is matched to the Java method by using regular expressions. In this case the regex "^Jeff has bought a microwave for \$(\d+)$" will match “Jeff has bought a microwave for $100”. Notice that the microwave price is extracted as a regex group. This is the mechanism that Cucumber uses to extract variables from the free text specification.
The entire Java step definition file can be found below
public class RefundSteps {
private Business business = new Business();
@Given("^Jeff has bought a microwave for \\$(\\d+)$")
public void Jeff_has_bought_a_microwave_for_$(int amountOfMoney) {
business.buyMicrowave(amountOfMoney);
}
@And("^he has a receipt$")
public void he_has_a_receipt() {
business.hasReceipt();
}
@When("^he returns the microwave$")
public void he_returns_the_microwave() {
business.returnMicrowave();
}
@Then("^Jeff should be refunded \\$(\\d+)$")
public void Jeff_should_be_refunded_$(int amountOfMoney) {
business.refund(amountOfMoney);
}
} Step definition Java files can also use @Before and @After hooks similar to JUnit and a custom reporting mechanism by injecting a Scenario object where steps can output text or even image resources.
Last, the step definition code needs to work with the application business code.
From the language point of view, Cucumber is very flexible, supporting all of the major programming languages like Java, .NET, C++, PHP and others. As for the Java side of things, there are 2 artifacts available: one for Java 8 and one for previous versions.
Some of the main supported frameworks are Spring, Selenium and Ruby on Rails.
Even though the framework is not extremely popular, it has been supported by the main Java IDEs for a while now (Intellij IDEA 12 and up; Eclipse starting with January 2014). With features such as syntax highlighting, step autocompletion, step navigation, step usage, step stub generation and test run, Cucumber development is fairly easy.
Maven integration is straightforward for basic usage (provided you use an empty test class annotated to run with Cucumber and place all artifacts in the same package) but more advanced use cases could benefit from a dedicated pure-Java Maven plugin (see [12]). (It may also be the case that your hierarchical feature file structure is more high level and thus different from your step definition package structure)
A Java test class that will run all features in the same package
package com.tsm.bdd;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
@RunWith(Cucumber.class)
public class RefundTest {
} The approaches to running Cucumber tests with a more complex setup using Maven are
use the exec-maven-plugin to call the Cucumber main class and pass the needed arguments
Using the exec-maven-plugin
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>${exec-maven-plugin.version}</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>java</goal>
</goals>
<configuration>
<classpathScope>test</classpathScope>
<mainClass>cucumber.api.cli.Main</mainClass>
<arguments>
<argument>--format</argument>
<argument>${format}</argument>
<argument>--strict</argument>
<argument>--glue</argument>
<argument>target/test-classes</argument>
<argument>target/test-classes/.</argument>
<argument>--tags</argument>
<argument>~@ignore</argument>
<argument>--tags</argument>
<argument>${tagArg}</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>Using the maven-surefire-plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<test>${java.test}</test>
<systemPropertyVariables>
<cucumber.options>
--monochrome
--strict
--plugin pretty
--plugin html:..\logs\report\html-report-java
--plugin json:..\logs\report\json-reports\${json.name.java}
--plugin junit:..\logs\report\junit-java.xml
--tags ~@skip
</cucumber.options>
</systemPropertyVariables>
</configuration>
</plugin> It is worth noting that Cucumber can also be integrated with Jenkins with the help of plugins.
From a BDD perspective, Cucumber doesn’t add any major contribution especially when compared to JBehave (see [11]) which takes a very similar approach.
Overall, Cucumber seems to serve its purpose and provide easy collaboration between the involved parties with the use of the Gherkin language, and as a side-effect enhance the readability of developer written tests. However, from a programmer point of view, the tool feels not quite as mature with small gaps on test specification and writing (see [9]), unfinished documentation (see [10]) and complexity in configuring a unified way to run acceptance tests (Maven vs IDE).
by Ioana Varga
by Peter Lawrey
by Monica Rațiu
by Călin Biriș
