16 Development

Developing software ain’t easy.

How do you know how you are doing ?

You could start collecting metrics about your code. These can give you some indication how maintainable and  reliable  it is.

The metric which come to mind to the most people is code coverage. Some people say it must be near 100%, other say 80% is a good number. At the end it can’t tell you if you are doing great or good. The only thing you might read from it is that a low number indicates a potential problem.

Duplication is another one you should look out for. A high number of duplicated code might bring up the problem that a bug was fixed in one place but lives on in other. If you are doing code generation (e.g. from WSDL’s) you might have a lot of duplicated code reported to you. A closer inspection is needed to really make sure this is a problem in the code you have written and is not auto generated.

Test success is not a metric you should be worried to much about. It just needs to be 100% at every checking in a shared repository. No discussion.

Cyclomatic complexity let’s you measure the number of linearly independent paths through your code. A high number means a more complex code. A complex code base is hard to maintain and involves a lot of work to adopt it to new requirements.

Dependencies between packages is another important value to check on (see Uncle Bob’s paper on Acyclic Dependency Principle). Basically it revolves around that packages should not have cyclic dependencies between them since this does have an impact on releasing and testing (and therefore coding). A change in one package could trigger a whole system build to make sure everything works together.

Tools like Findbugs let you define rules to which your code should comply to. This is great because it is tailored to your code/company and ensures you control and fine tune the measures. Or so. It requires some time to define these rules. They need to be refined over time and you have to understand them to make use of it. Of course you can leave out some of these and reduce complexity. Do not underestimate the effort to implement these.

This rounds up the metrics you can get from your code base.

But wait, there is more.

A valuable metric is the meantime between reporting a feature or bug and fixing. This gives your team (and management of course) a rough estimate how fast you can react to market changes.

To get an estimate of the mean time before a failure happens you can measure the time between two bugs. The higher the number the better. Current reported bugs per lines of code can also be measured. These metrics will give you some confidence in your code base.

At the end of the day the ultimate measure is the money on the company’s bank account. If this is not working out you are in trouble.

How to get all of these ?

There are some static code analysis tools to get you started. I only can talk about Java related tools but there are others as well.

• Checkstyle
• FindBugs
• PMD
• Cobertura
• Structure 101
• Sonar

To get more information about your bugs should query your bug tracking system like Jira or Bugzilla.

Summary

There are much more numbers out there but I think these are the ones you should always keep an eye on.

Look out for

• Code Coverage
• Duplication
• Cyclomatic complexity
• Cyclic dependencies
• Test success
• Time to market
• Mean time between failure

What metrics do you look out for ? Let me know in in the comments and why.

5 Development

Whenever you create an object you have to find a meaningful name.

While renaming later in modern IDE’s is no problem at all you should not pick the first name which comes to your mind.

Imagine you have an external fraud detection web service and it will return a ‘hit’ or ‘miss’. How do you name this object? HitOrMiss maybe (since this is the representation you get from the webservice) ? It might describe the problem at hand. If you go with this name it will dripple down through all layers. The domain model, the dto’s, services and more will now reference to this object (and create their own derivates for it). Your colleagues which code up other parts of the system will refer to it and create their own variants of the name (like structure for web pages). The wrong name is now all over your code.

So much for refactoring in your IDE. You will never catch all the places automatically where the wrong name was used.

While it seemed lost time discussing the ‘right’ name for 5 minutes in the beginning it saves you much more time afterwards. Of course you might get also confused a couple of weeks later remembering what this thing really does and what it is used for.

Make sure you take time in naming your objects. A wrong decision can do some harm to your code. The later it turns out that the name was not correct the much more expensive it becomes to correct this mistake. If you are not really sure how to name it, talk to your colleagues and discuss the name. A second opinion can not hurt.

Maybe FraudDetectionResult would have been a better choice.

4 Development

In Response to Matt Raible’s question about my preferred development stack.

Source control

In my own development project I switched to Git some time ago. I was using svn and before that cvs for a couple of years. Git just makes it easy to try out small ideas very quickly without reverting the code. We will introduce Git at work pretty soon (I hope).

The Atlassian cloud

I was using Confluence as a Wiki in my open source project and my current employer Hypoport uses it as well. Bug tracking is pretty much a task of Jira. Those two integrate very nicely. You can have ping backs in your Jira Ticket from Confluence, so you see all related files. I would love to use FishEye but I’m not sure how to justify the costs and what the added value is in our case.

The Code

I’m pretty much sold to Jetbrains IntelliJ Idea. I’m using it for 5 years now. With it comes Teamcity. It’s a very nice continuous integration tool where the biggest plus (among other things) is the delayed (and pre tested) checkin. No more late night checkins which break the master. We just rolled out the latest version which has a much nicer Git integration (so we can make the switch).

So it comes down to:

The List

• Source control: Git
• Wiki: Atlassian Confluence
• Continuous Integration: Jetbrains Teamcity
• Bugtracker: Atlassian Jira

We are about to use Sonar for code analysis and development over time, but this is just in the beginning.

What’s yours ?

11 Development

If you want your Business Analyst to provide some acceptance tests for your implementation it would be best to make sure you get it in some form your tests can read. That way you can make sure your code does really do that what the customers wants.

Two tools to achieve that are Concordion and Cucumber (and cuke4duke). Both taking the approach of Fit a step further. You can find very good documentation for the tools on their web pages or as examples along with the code.

Concordion

Concordion let’s you write the specifications in html. You can instrument them with some Concordion code. Since the instrumentation code is in html, it is hidden when you render it.

<html xmlns:concordion="http://www.concordion.org/2007/concordion"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.concordion.org/2007/concordion ">
<body>
<p>
  <span concordion:execute="setCurrentTime(#TEXT)">09:00AM</span>
  The greeting for user <span concordion:set="#firstName">Bob</span>
  will be:
  <span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span>
</p>
</body>
</html>

Concordion can instrument free text as well as tables to have some sort of data provider for tests. It integrates directly with JUnit (no TestNG support yet).

Concordion Specification/Fixture/System

The idea is that your specifications is stable. This will not change as much as maybe your code to fulfill the requirements of the customer. Your fixture bridges these two together.

package com.maxheapsize.concordion;
import org.concordion.integration.junit3.ConcordionTestCase;
public class DemoTest extends ConcordionTestCase {

    public String greetingFor(String firstName) {
        return "Hello " + firstName + "!";
    }

    public void setCurrentTime(String time) {
        System.out.println(time);
    }
}

Your java code provides the methods which are called from the specification. At that point you are most likely to use a DSL to build up your business logic. Concordion will assert the expected values.

After you run the unit test you will get a nice looking html page.

Concordion output

Passed tests will be green, failed red, pretty easy.

Pro Concordion:

• Instrumention of free text
• The final report is nice looking
• JUnit support
• Easy setup
• Pure Java

Con Concordion:

• The business analyst needs to know a little bit of html
• Usually the programmer needs to change the specification to instrument it
• You program parts in html, refactoring support is missing
• No TestNG support

Cucumber

Cucumber is a ruby based behaviour driven development tool. Since you can run Ruby code inside the JVM with the help of JRuby it is possible to use it for your Java code. The usually way so far is that you use maven or ant to startup JRuby and the Cucumber ruby code.

In Cucumber you write your specification (or feature) in plain english (or russian, german,lolcode or 36 or so more supported languages). No instrumentation is necessary but your customer has to stick to a certain layout (or better: structure). It follows the well known given-when-then pattern.

Feature: Demo

  Scenario: A Car with 4 wheels
    Given I have a car
    When I add 4 wheels
    Then I should be able to drive the car

You can write your step definitions (like the fixture in Concordion) with the help of annotations.

package com.maxheapsize.cucumber;

import cuke4duke.Given;
import cuke4duke.Then;
import cuke4duke.When;
import org.junit.Assert;

public class CarFeature {
  private Car car;
  @Given("I have a car")
  public void iHaveACar() {
    car = new Car();
  }

  @When("I add (\\d+) wheels")
  public void iAddWheels(int numberOfWheels) {
    car.setWheels(numberOfWheels);
  }

  @Then("I should be able to drive the car")
  public void iShouldBeAbleToDriveTheCar() {
    Assert.assertTrue(car.canDrive());
  }
}

The regular expressions defined in the annotations must match the feature definitions.

Cucumber results

You can define information in a table like structure as well to provide more data in an easy way. The given when then structure can be used to have multiple when’s or then’s. So it is very flexible that way.

Pro Cucumber

• Feature can be described in plain language
• No extra programmer work on he feature required
• Instrumentation is directly on the Java code via annotations

Con Cucumber

• More complex due to the Ruby/JRuby/Java chain
• Business analyst needs to stick to predefined format
• The only way (without extra work to write a runner) to run it seems to be within ant or maven, for rapid turn around and TDD I would like to have JUnit/TestNG support of some kind

Verdict

It is easy to get started with both tools. I spent about a couple of hours with each of them and was able to get simple and a bit more complex examples running. They get you going with BDD and acceptance based testing.

I like parts of both concepts. I want to be able to write a feature like in Cucumber. The instrumentation should be directly on my code and not in the part of the customer. The easy setup and integration of Concordion with JUnit (need TestNG here) and the output of it is very nice. So where is the tool which has these features .

So which tools will I use?

I have yet to see the use case where the business analyst writes the (and updates) the acceptance tests. I don’t mind add some extra code to the specification and I know html. I might get started with Concordion for now.

You might also check my post about Behavior driven development with EasyB (and vs. TestNG).

Do you use any of these tools? Or a different one? What’s your expierence?

4 Development

We have been using our own flavor of Fit for Rules (which is build on top of fit) for about 1 1/2 years now to test our business logic written in JBoss Rules 5. It’s relatively easy to get the Business Analyst on board since he is using his tool (which is Microsoft Excel) to communicate test cases for the rules. So in theory, he writes the tests in Excel, we do the rules coding and voila, all tests turn green.

Reality is, we have to tweak the Excel sheets. We need to put in imports of our fact model, insert facts and create objects within that not so programmer friendly table environment. A couple of days ago we got the request to tweak some rules and we all had to start doing rules again (and we used to use Eclipse for writing rules because that’s the only IDE having a plugin for that).

After half a day of coding Java syntax in Excel sheets we decided that the ramp up time for the not so knowledgeable rules/fit programmers like me is too much. With debugging, copy and paste we spent easily 5-10 times more time on making the tests work than writing the code itself. Test driven design is not really an option here, since you need to know the imports of the rules file to get the sheet even to compile.

So what did we do ? Well why not try to get things working the way we used to do it ? TestNG anyone ?

There are many pros to use unit testing but also some cons. The biggest issue is that we will loose the direct communication to the business analyst. It’s always better if someone else writes the test and I just have to implement the solution. Maybe we can find another solution involving Active Spec or DSL. For now we stick to unit tests and the task the we have to make sure we convert every Excel test case to java code (but hey, that’s what our code reviews are for).

Checkout our current base class for testing our rules:

public abstract class AbstractRulesTest {
  public abstract String[] getRulesFileNames();
  private final String GET_FINDINGS = "import com.maxheapsize.RulesFinding;" +
                                      "query \"getAllRulesFindings\"\n" +
                                      "  finding : FRulesFinding()\n" +
                                      "end";

  private static Logger LOG = Logger.getLogger(AbstractRulesTest.class);

  public final List<FRulesFinding> fireRules(Set factsForWorkingMemory) {
    KnowledgeBase ruleBase = setUpKnowledgeBase();
    return fireRules(ruleBase, factsForWorkingMemory);
  }

  public KnowledgeBase setUpKnowledgeBase() {
    KnowledgeBaseConfiguration configuration = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
    KnowledgeBase ruleBase = KnowledgeBaseFactory.newKnowledgeBase(configuration);

    KnowledgeBuilder build = KnowledgeBuilderFactory.newKnowledgeBuilder();
    build.add(ResourceFactory.newReaderResource(new StringReader(GET_FINDINGS)), ResourceType.DRL);
    String[] fileNames = getRulesFileNames();
    for (String fileName : fileNames) {
      File userDefinedFile = new File(fileName);
      build.add(ResourceFactory.newFileResource(userDefinedFile), ResourceType.DRL);
    }

    handleBuilderErrors(build);

    ruleBase.addKnowledgePackages(build.getKnowledgePackages());
    return ruleBase;
  }

  private void handleBuilderErrors(KnowledgeBuilder build) {
    if (build.hasErrors()) {
      KnowledgeBuilderErrors knowledgeBuilderErrors = build.getErrors();
      for (KnowledgeBuilderError knowledgeBuilderError : knowledgeBuilderErrors) {
        int[] ints = knowledgeBuilderError.getErrorLines();
        LOG.error("Error at : "+ints[0]+" : "+ints[1]);
        LOG.error(knowledgeBuilderError.getMessage());
      }
    }
  }

  private List<FRulesFinding> fireRules(KnowledgeBase ruleBase, Set facts) {
    List<FRulesFinding> result = new ArrayList<FRulesFinding>();
    StatefulKnowledgeSession statefulSession = ruleBase.newStatefulKnowledgeSession();
    for (Object fact : facts) {
      statefulSession.insert(fact);
    }
    statefulSession.fireAllRules();

    QueryResults results = statefulSession.getQueryResults("getAllRulesFindings");
    try {
      FRulesFinding finding = (FRulesFinding) results.iterator().next().get("finding");
      result.add(finding);
    }
    catch (NoSuchElementException e) {
      result = new ArrayList<FRulesFinding>();
    }
    return result;
  }
}

All my rules insert a RulesFinding (and only one at the moment) into the working memory when triggered. The rest is pretty easy. You subclass it, overwrite getRulesFileNames and call fireRules with a set of objects (your tests) which need be insert into the working memory. To get the finding back you need to execute an already inserted query which needs to have an identifier (line 3, 20, 52, 54). It will contain the result of your rule execution.

Sample code would look like this:

public class RulesTest extends AbstractRulesTest {

  private Set facts;

  @BeforeMethod
  public void setUp() {
    facts = new HashSet();
  }

  @Override
  public String[] getRulesFileNames() {
    return new String[]{
        "src/main/rules/myrules.drl",
        "src/main/rules/generealRules.drl"
      };
  }

  @Test
  public void testDemoRule() {

    FMyFact myFact = new FMyFact();
    myFact.setColor("green");
    facts.add(myFact); // add all your facts here

    List<FRulesFinding> findings = fireRules(facts);
    Assert.assertTrue(findings.size() == 1);
    FRulesFinding finding = findings.get(0);
    Assert.assertTrue(finding.getStatus() == FStatus.OK);
  }
 }

Depending on how you cut your rules you can extract the assertion of the status.

Each test case in Excel now takes about 5-10 lines of java code. Considering we are covering each rule with about 5-15 test cases and boundary conditions this amounts to 75-150 lines of test code. I take that any day over programming in Excel.

6 Development

At my current job management buys into Test driven development. It slowly starts to become our primary development style.

Some advantages of TDD for me are:

• Focus on problem solution
• No unnecessary development for use cases dreamed up by the developer
• Always make sure your code fulfills the requirements (tests)
• If you have tests you can safely refactor your code at any time

Even though we all knew about the advantages of TDD, we figured out that we do not always write tests first. While this might me acceptable in some situations like UI tests, most of the time it is not desired.
After seeing that problem for a couple of weeks (and always wondering how I did not write test first) all of the sudden I realized why sometimes it was easier to develop code with writing tests first and sometimes I just simply did not thought about it and had to write the tests later.

It’s simply because of pair programming.

What certainly happens to me when coding alone, is that sometimes I get into ‘crunch’ mode too fast. With a second person to slow me down, we are discussing the problem and make sure we both have the same understanding of what we are trying to achieve. During this process it is much clearer what needs to be done and what the goals are. This is a very important step. If I’m all by myself, I start writing down some classes, look at methods and I’m already thinking in code and not about the problem. Once you know what the real problem is, it is very easy to start writing your tests first. It helped me a lot.

Where pairing also helps is to focus both programmers to stay in TDD mode. Once the driver of the pair programming couple starts to hack away code, the other will remind him to write the tests first. This is a very valuable.

After a while I got better at test driven development, even programming alone.

So if you recognize that you do not write tests first (but you want to) and you yet again don’t know how that happened, try pair programming. You learn how to start with TDD (and get used to it) and get all the benefits of pair programming as well.

9 Development

Whenever you write some code you better make sure you have it covered by some tests. There are different possibilities on how to achieve this (and you might want to use all of them).

Unit Tests

Unit tests are a classic. You usually write them while you code. Write tests, run failing test, write code and run passing test. Repeat. Keep this small and simple. Test the smallest unit which makes sense. You might end up with hundreds if not thousands of them. Make sure you have no external dependencies in them, like databases or external webservices. Mock them away. They are usually written in TestNG or JUnit.
These tests can run at every check in in the VCS.

Integration Tests

Integrations Test are one step higher. You want to make sure your applications runs with all or some dependencies and things plays well together. These usually have a much higher run time since they might involve some external services. Usually you would use a framework like TestNG, nUnit for them.
These tests are run not so often since they can be very expensive.

UI Tests

Many applications have some sort of graphical user interface. To make sure everything works alright there are several tools like Selenium or HP Quality Center to simulate a user going through the GUI. These tests are written more towards the end of the projects since in the beginning the interface might change fast and often and the overhead to change them might be significant. You usually can not run through all paths of your applications. Choose the ones which are the most critical.
They can be run a couple of times a day depending on the size of the application and the tool (e.g. Quality Center tends to be sort of slow) you use.

Reference System Tests

If you have to implement some business rules or calculations and your client can specify these in Excel you might want to use a framework like Fit (or Fitnesse). You run your system against the values in the sheets and verify that it meets the expectation.

There are many more test possibilities and techniques to make sure the system meets the demands of the customer. These are just the ones I tend to use in my projects and I found them very useful.
Tests (and specifications) with tools like Concordion or the use of a Behavior Driven Design frameworks like EasyB are offering nice
ways to make sure the customers gets what he pays for (and he understands it). I still need to play with them.

2 Development

No matter how good a developer is, with two hardware upgrades you can speed up most machines and cut down development time for everybody.

Your big fat IDE

Depending what you use to crunch out your code, these things are eating your memory. And then you have your application server running, and an internet browser with 20+ sessions, plus Twitter, plus Photoshop, plus, plus, plus.

I used to have 2 GB RAM in my laptop and I thought it is ok. Turns out, having 4 GB, speeds up my overall development significantly. No more swapping of programs and more programs can stay open. So far I never reached the point where my memory was exhausted (well I never really tried).

Swapping in 4 GB of RAM (DDR2) on my laptop took about 10 minutes and 50 USD. Really worth the money.

Your mama’s harddrive

My IDE of choice starts indexing all my files, scans them, creates indices (which will get updated every so often through maven) and so on. My applications server writes files, scans files for changes, deployed them and does thousand other things. This is your hard drive at work. Looking for a better solution I was looking at RAM disks to have the indices written on there but there is a more obvious solution. A Solid State Drive.
I bought the OCZ Vertex 120 GB off Amazon. Putting this baby into a laptop can be a bit tricky (try OWC for MacBook install videos) but is certainly doable.

I ran some benchmark before and after to see the read/write speed up. The HD was a 7200 rpm drive and all values are in MB/sec.

I used xBench to get the read/write numbers. My laptop went up from a 118 to 194 in this benchmark. Especially the random read is where the SSD really shines.

The SSD was about 330 USD and it took me about 40 minutes from shutdown with the old hard drive to boot up with my SSD drive.
Of course reinstalling your OS (or rolling it back from a backup) will take some time too.

Upgrading your processor is also an option but I never used that one on my Desktop machines or Laptops. If this ever was a bottle neck it usually meant that the whole machine was outdated (mainboard, bus, ram speed). But is an option as well.

I you have an older machine or even with a fast machine and you have some money left, I think upgrading these two components really makes your machine fly.

500 USD compared to the time you save on development is about nothing. So every employer should upgrade their developers machines with RAM and SSD’s. It pays out very soon (and you will make your developers happy, which is also important ).

3 Development

I’m pretty busy at work right now but I could find some time to add one more feature to my little property sorter plugin for IntelliJ Idea.

Comments above property entries will now be preserved when you sort the file.

The plugin is available from within Idea or at Google Code.

From unsorted ...

... to sorted with comments.

1 Development

In the last couple of month I was writing some flows using Spring Webflow 2. I knew that I (in theory) I could test them but I never did. The overall documentation of Webflow 2 is not that great at the moment, so I hope to improve that a little bit.

Spring Webflow 2 supports only JUnit tests so far. I guess this will change at some point but until then you have to pull out that junit.jar again.

Sorry for the awkward line breaks. Spring just does love long method and class names (which is not that bad).

Your Testclass needs to extends AbstractXmlFlowExecutionTests and to implement protected FlowDefinitionResource getResource(FlowDefinitionResourceFactory resourceFactory).

public class MyWebflowTest extends AbstractXmlFlowExecutionTests
  protected FlowDefinitionResource getResource(
     FlowDefinitionResourceFactory resourceFactory) {
    FlowDefinitionResource flowDefinitionResource =
      resourceFactory.createResource("flows/administration/skinManagement.flow.xml");
    return flowDefinitionResource;
  }

If your flow now happen to have a parent flow you are in a bit of trouble. You just can include one flow in the FlowDefinitionResource. The parent tag in your flow points to an id of the parentflow which is usually defined in a webflow configuration file. If this parent flow again has parents as well…yes, same game again.

To get a hold of these flows you need to override getModelResources[].

@Override
protected FlowDefinitionResource[]
 getModelResources(FlowDefinitionResourceFactory resourceFactory) {
  FlowDefinitionResource[] flowDefinitionResources =
 new FlowDefinitionResource[3];
  flowDefinitionResources[0] =
    resourceFactory.createResource("flows/common.flow.xml");
  flowDefinitionResources[1] =
    resourceFactory.createResource("flows/common-exceptionHandling.flow.xml");
  flowDefinitionResources[2] =
    resourceFactory.createResource("flows/common-menu.flow.xml");
  return flowDefinitionResources;
}

If one of your flows is using beans you want to mock them in the test. You can do that by registering with the MockFlowBuildContext. The method configureFlowBuilderContext will be called by AbstractXmlFlowExecutionTests

@Override
protected void configureFlowBuilderContext(
    MockFlowBuilderContext builderContext) {
  builderContext.registerBean("validationExceptionHandler",
    new ValidationExceptionHandler());
  builderContext.registerBean("infastructureExceptionHandler",
    new InfastructureExceptionHandler());
  builderContext.registerBean("springSecurityExceptionHandler",
    new SpringSecurityExceptionHandler());
}

With this setup you now can start testing the flow.

...
public void testSkinManagementView() {
  MockExternalContext mockExternalContext = new MockExternalContext();
  startFlow(mockExternalContext);
  assertFlowExecutionActive();
  assertCurrentStateEquals("showSkinManagementOverview");
}
...

I think the code is pretty self explanatory.

You have several possibilities including start and resume flow and assert certain states. You can test action methods doing something (like calling a service) and making sure the correct flow is executed.

Figuring out the initial setup with the parent flow and beans was the most time consuming task. You need to make sure that you only put flow logic in the definition of the flow since that is what webflow is for. I think with some clever usage of set and result you can make things complicated and a bit tricky.