Selenium IDE

Selenium is well know for automatic testing web pages. It does support many browsers, operating systems and languages. A Selenium IDE exists to aid you creating automated tests.

Selenium IDE

It is possible to write extensions in JavaScript to have data driven tests. If you organize your selenium tests in a way that you split the pages and forms in components, you can load up new data for the tests (written in xml) to fill out forms differently for each use case. The Selenium IDE will be of a great help here since you start develop the tests in the IDE and later ‘just’ parametrize the pages. With a couple of more javascript magic you can run these as well in Selenium Remote Control (Selenium RC for short). This even works for running the tests in continuous integration systems like Hudson or Teamcity.

Great Stuff.

The problem.

The drawback coding up your tests with the Selenium IDE is: HTML.

You will end up with some test suites including many test scenarios. It will be one big ball of

<tr>
  <td>type</td>
  <td>accountnumber:Value</td>
  <td>82892892811</td>
</tr>
<tr>
  <td>type</td>
  <td>accountAmount:Value</td>
  <td>2345.33</td>
</tr>

You will build components and you will end up with GotoIf and Label statements. Back to Basic programming from the 80ties. Without more javascript extensions you also can’t reuse combinations of scenarios.

While this works pretty ok: I’m a programmer and do have a problem with the quality of the produced code. It’s easy to mess up. It’s not safe for refactoring. To extend functionality you need to have javascript knowledge. Depending on your company profile (e.g. java shop) , this might be a problem. Having a dedicated tester on the team certainly helps to overcome these obstacles, but if everybody in the team just once in a while looks into the Selenium IDE tests, the quality and the reliability of the code will not improve. Not to mention all the copy and paste. Been there, done that.

Selenium 2

A much nicer approach for me would be something like this:

WebDriver driver  = new FirefoxDriver();
AppWebTest test = new AppWebTest(driver, "http://myapp.com/login");
LoggedInUser user = test.login("user", "password");
Account account = user.createNewAccount("DemoAccount");
account.addPerson(PersonFactory.createPersonWithNoIncome());
...
account.verify();

This would log in a user, creates an account and assigns a person to it. Everybody could read and understand it. Furthermore you can easily reuse components and logic. I would imagine that creating a DSL, matchers and finders is everything one would need to have a very nice test scenario.

All that is already possible with Selenium 1 and the language bindings (more or less).

Just with the release of the first alpha of Selenium 2 mid december 2009 it became just a little bit easier.

Selenium 2 is the combined effort of Google’s Webdriver and Selenium. Webdriver tries to tackle Seleniums problems with Javascript and the security model as well as the complex Selenium RC API.

Selenium is written in JavaScript which causes a significant weakness: browsers impose a pretty strict security model on any JavaScript that they execute in order to protect a user from malicious scripts.
…
Additionally, being a mature product, the API for Selenium RC has grown over time, and as it has done so it has become harder to understand how best to use it. For example, it’s not immediately obvious whether you should be using “type” instead of “typeKeys” to enter text into a form control. Although it’s a question of aesthetics, some find the large API intimidating and difficult to navigate.

Webdriver does not use JavaScript to control the web browser but uses native controls, e.g. an extension for Firefox or the native automation extensions of IE. Furthermore it introduces an object based API instead of following Seleniums directory based approach.

So given the above example the login method would look something like this:

 public LoggedInUser login(String userName, String password) {
    driver.get(url);
    driver.findElement(By.id("user")).sendKeys(userName);
    driver.findElement(By.id("password")).sendKeys(password);
    driver.findElement(By.id("login")).click();

    LoggedInUser user = new LoggedInUser(driver);

    WebElement startPage = driver.findElement(By.id("startPage"));

    assertThat(startPage().getText()).isEqualTo("Start");
    return user;
  }

You can take this to any level. It took me just a few hours to model the above use case with componentized input elements.

With a bit more abstraction addPerson looks something like the following code. Whatever is needed to add a person to an account will be replayed. This is what Selenium IDE would normally do.

public void addPerson(Person person) {
    jumpToAccountOverView();
    clickOnLink("menu:Overview");
    clickOnLink("table_personTable:link_newPerson");

    inputGuiElement(person.getTitle());
    inputGuiElement(person.getLastName());
...
    clickSave();

And now for the best part. You don’t really need a real web browser. Selenium 2 supports 4 WebDriver:

• Firefox
• Chrome
• Internet Explorer (on Windows)
• HtmlUnit

HtmlUnit

With enabled JavaScript on HtmlUnit (it uses Rhino, so not all JavaScript tricks might work, could be a drawback depending on your scenario) I can run my test as TestNG or JUnit test case. No need for a Selenium RC anymore. No browsers running locally or on remote machine listing to ports to execute tests and transmitting results back.

It is very useful to have the Selenium RC take screen shots during it’s run. Selenium 2 can’t do this right now (and with HtmlUnit never can). But by simply integrating some logging in the components and elements and you know right away where things went wrong.
Developing with the Selenium IDE might seem like a must, but during the work on my prototype all I needed was Firebug and a XPath Searcher. I did not miss it at all. To do integration tests you could test all components independently and one at a time. Once these work, chain them together and let them run as unit tests. Did I mention HtmlUnit is fast ?

Conclusion

I barely dug into the webdriver code, I played around with it for a couple of hours and I’m sure there more is stuff (and bugs) to find. Given that Selenium 2 is in alpha 1 stage right now I expect very nice things coming out of it. It certainly looks like the way to go.

Another strong contender is Canoo WebTest. I will have a look at this one in another post.

Update Ajax and HtmlUnit 1/5/2010

It does not seem to work. You need a real browser to get dynamically content re-rendered through Ajax.

A method waiting for an id to be rendered (or timeout after x seconds) could look like this:

  public void waitForElementOrTimeout(String idToWaitFor, int timeout) {
    long end = System.currentTimeMillis() + timeout * 1000;
    RenderedWebElement resultsDiv = null;
    while (System.currentTimeMillis() < end) {
      try {
        resultsDiv = (RenderedWebElement) driver.findElement(By.id(idToWaitFor));
      }
      catch (NoSuchElementException e) {
        //
      }
      if (resultsDiv != null && resultsDiv.isDisplayed()) {
        break;
      }
    }
    assertThat(resultsDiv).isNotNull();
  }

This will either find the element specified or throw an assertion (updated example from the selenium 2 page).