If you want your Business Analyst to provide some acceptance tests for your implementation it would be best to make sure you get it in some form your tests can read. That way you can make sure your code does really do that what the customers wants.
Two tools to achieve that are Concordion and Cucumber (and cuke4duke). Both taking the approach of Fit a step further. You can find very good documentation for the tools on their web pages or as examples along with the code.
Concordion
Concordion let’s you write the specifications in html. You can instrument them with some Concordion code. Since the instrumentation code is in html, it is hidden when you render it.
<html xmlns:concordion="http://www.concordion.org/2007/concordion"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.concordion.org/2007/concordion ">
<body>
<p>
<span concordion:execute="setCurrentTime(#TEXT)">09:00AM</span>
The greeting for user <span concordion:set="#firstName">Bob</span>
will be:
<span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span>
</p>
</body>
</html>
Concordion can instrument free text as well as tables to have some sort of data provider for tests. It integrates directly with JUnit (no TestNG support yet).
Concordion Specification/Fixture/System
The idea is that your specifications is stable. This will not change as much as maybe your code to fulfill the requirements of the customer. Your fixture bridges these two together.
package com.maxheapsize.concordion;
import org.concordion.integration.junit3.ConcordionTestCase;
public class DemoTest extends ConcordionTestCase {
public String greetingFor(String firstName) {
return "Hello " + firstName + "!";
}
public void setCurrentTime(String time) {
System.out.println(time);
}
}
Your java code provides the methods which are called from the specification. At that point you are most likely to use a DSL to build up your business logic. Concordion will assert the expected values.
After you run the unit test you will get a nice looking html page.
Concordion output
Passed tests will be green, failed red, pretty easy.
Pro Concordion:
- Instrumention of free text
- The final report is nice looking
- JUnit support
- Easy setup
- Pure Java
Con Concordion:
- The business analyst needs to know a little bit of html
- Usually the programmer needs to change the specification to instrument it
- You program parts in html, refactoring support is missing
- No TestNG support
Cucumber
Cucumber is a ruby based behaviour driven development tool. Since you can run Ruby code inside the JVM with the help of JRuby it is possible to use it for your Java code. The usually way so far is that you use maven or ant to startup JRuby and the Cucumber ruby code.
In Cucumber you write your specification (or feature) in plain english (or russian, german,lolcode or 36 or so more supported languages). No instrumentation is necessary but your customer has to stick to a certain layout (or better: structure). It follows the well known given-when-then pattern.
Feature: Demo
Scenario: A Car with 4 wheels
Given I have a car
When I add 4 wheels
Then I should be able to drive the car
You can write your step definitions (like the fixture in Concordion) with the help of annotations.
package com.maxheapsize.cucumber;
import cuke4duke.Given;
import cuke4duke.Then;
import cuke4duke.When;
import org.junit.Assert;
public class CarFeature {
private Car car;
@Given("I have a car")
public void iHaveACar() {
car = new Car();
}
@When("I add (\\d+) wheels")
public void iAddWheels(int numberOfWheels) {
car.setWheels(numberOfWheels);
}
@Then("I should be able to drive the car")
public void iShouldBeAbleToDriveTheCar() {
Assert.assertTrue(car.canDrive());
}
}
The regular expressions defined in the annotations must match the feature definitions.
Cucumber results
You can define information in a table like structure as well to provide more data in an easy way. The given when then structure can be used to have multiple when’s or then’s. So it is very flexible that way.
Pro Cucumber
- Feature can be described in plain language
- No extra programmer work on he feature required
- Instrumentation is directly on the Java code via annotations
Con Cucumber
- More complex due to the Ruby/JRuby/Java chain
- Business analyst needs to stick to predefined format
- The only way (without extra work to write a runner) to run it seems to be within ant or maven, for rapid turn around and TDD I would like to have JUnit/TestNG support of some kind
Verdict
It is easy to get started with both tools. I spent about a couple of hours with each of them and was able to get simple and a bit more complex examples running. They get you going with BDD and acceptance based testing.
I like parts of both concepts. I want to be able to write a feature like in Cucumber. The instrumentation should be directly on my code and not in the part of the customer. The easy setup and integration of Concordion with JUnit (need TestNG here) and the output of it is very nice. So where is the tool which has these features 
.
So which tools will I use?
I have yet to see the use case where the business analyst writes the (and updates) the acceptance tests. I don’t mind add some extra code to the specification and I know html. I might get started with Concordion for now.
You might also check my post about Behavior driven development with EasyB (and vs. TestNG).
Do you use any of these tools? Or a different one? What’s your expierence?
14 Responses to “Concordion vs. Cucumber and Java based Acceptance Testing”
Leave a Reply
Additional comments powered byBackType
New blog post: Cucumber vs. Concordion and Java based Acceptance Testing http://bit.ly/IPT5i
This comment was originally posted on Twitter
Concordion vs. Cucumber and Java based Acceptance Testing … http://bit.ly/wQL1c
This comment was originally posted on Twitter
Guter Vergleich zwischen den beiden guten Testtools #cucumber und #concordion http://bit.ly/13FzlC #bdd #tdd
This comment was originally posted on Twitter
Interesting for #agiletd folks? RT @Sanitz: Vergleich zw. beiden guten Testtools #cucumber und #concordion http://bit.ly/13FzlC #bdd #tdd
This comment was originally posted on Twitter
I use Concordion for Java und Cucumber for Ruby. I don’t think that is that much difference.
Cucumber seems a little bit more smarter but I suspect thats due to Ruby and not due to Cucumber itself.
@Stefan Yes, it is not a big difference. But still for Java I would prefer Concordion due to the better integration.
Concordion vs. Cucumber and Java based Acceptance Testing: http://bit.ly/13FzlC #programming #bdd
This comment was originally posted on Twitter
Thanks for the nice writeup of the two frameworks! FYI, you can have cucumber output JUnit XML test results using the ‘junit’ formatter. (So… ‘cucumber -f junit –out reports -f pretty’ will output the xml in the resports dir and also give you pretty output to the console.) I don’t know if that will help you but it has helped me in the context of build servers.
Cool, thanks.
I will try it out.
[...] Concordion vs. Cucumber and Java based Acceptance Testing – Oliver Wehrens (maxheapsize.com); [...]
Hi,
I’ve been using Concordion for the last 6 months now. Used to use Fitnesse.
I love the simplicity of Concordion and the fact that the author has a very strong grasp on what makes ATDD tests maintainable.
Haven’t used Cucumber but I like the idea of natural language.
Worth mentioning that you can use Groovy rather than Java to write Concordion tests.
I’ve got a full example on my blog here.
http://bit.ly/8FpyzX
great to see someone else writing about Concordion!
Neil
Hi Oliver,
I’ve been using BDD for more than 2 years now in big distributed projects realised that it’s very good in the beginning, however if we make some mistakes, especially no following your recommendatino of having a “stable specification”, we can get in trouble because the “specification” is usually a “non-refactorable” file (txt in JBehave, Html in Concordion, etc). Which means that, any change in these files will be painful. Furthermore, we don’t have autocompletion when writing these files…
Although, we can still have BDD practices without having to use these tools. I’m not saying that the tools are not useful, they are very useful, but bear in mind their problems as well… Only use a tool if you really need to, otherwise, try something like this:
http://fabiopereira.me/blog/2010/05/28/bdd-with-scenario-code-dsl-sometimes-you-dont-need-a-tool
Cheers,
Yes, refactoring is a problem. Your specification needs to be stable and the adapter or DSL (Fixture in the picture above) needs to make sure to adapt.
I would also think that having a DSL for your business problem is the way to go. In my dreams even my business analyst can write a test (with code completion).
[...] * Why I prefer Concordion to FitNesse * Writing Stupid Specifications with Concordion * Concordion vs. Cucumber and Java based Acceptance Testing * ATDD Acceptance Testing using Concordion * Automated Acceptance Tests and Requirements [...]