Marc Guillemot’s blog

The stack trace of an exception in a log file is a sign that something went wrong. Even if nobody complains it is wise to take a bit time and investigate it before it really hurts.

Exactly the same applies to JavaScript errors in a web application. I believe that each JavaScript error should be considered as a bug and they should all be fixed before delivering, no matter if they directly impact the user experience or not.

Many great tools (JavaScript Console, Firebug, WebDeveloper Toolbar, …) allow to see the JavaScript errors while developing with Firefox as well as with other browsers and a wisely used error handler can report JavaScript errors to the server in production.

Sadly the support is not good while running automated tests. HtmlUnit provides a great help for that (see for instance this) but often integration tests should run in “real” browsers and not in HtmlUnit (for some good and many bad reasons). I don’t know currently any browser automation tool providing access to the JavaScript errors and to other information available in browsers like Firefox or Chrome that can be helpful for developers conscientious of the quality of their work

WebDriver is here not better than its concurrents and the issue “API for checking for JavaScript errors on the page” is opened since 2 1/2 years without any sign of life from the project’s members. The Selenium 1 issues SEL-522 and SEL-613 are equally asleep. Luckily WebDriver provides the possibility to configure additional extensions when using the FirefoxDriver, what allows to improve the situation.

Tester’s friendly application and injected error handler

The simpliest way to catch JavaScript errors for later retrieval is to add something like that

<script type="text/javascript">
window.jsErrors = [];
window.onerror = function(errorMessage) {
  window.jsErrors[window.jsErrors.length] = errorMessage;
}
</script>

in every page of your application (see for instance this post).

This allows you to retrieve the captured errors with

((JavascriptExecutor) driver).executeScript("return window.jsErrors");

This is a simple solution but it has different drawbacks. First you need to add this code at the beginning of all your HTML pages. If you’re in the ideal situation to be both developer and tester of the application it’s “just” one Ctrl+C and a lot of Ctrl+V. If you need to convince your colleagues or your management that the web pages should be modified for testability, it will be more difficult. From my experience it is not always easy to make such a requirement accepted.
To overcome this problem we could imagine injecting the handler while running the test:

String script = "window.collectedErrors = [];"
  + "window.onerror = function(errorMessage) { "
  + "window.collectedErrors[window.collectedErrors.length] = errorMessage;"
  + "}";
((JavascriptExecutor) driver).executeScript(script);

but besides the fact that it has to be done on every page, we can’t do it early enough: a lot of JavaScript may already have been executed in the page when our executeScript statement is reached.

The second problem of an error handler per window is the difficult access to the captured errors. The JavaScript errors won’t necessarily occur in the currently focused window forcing you to iterate through all (i)frames and windows to check for errors. This can still be seen as a minor issue but the real problem is that captured errors get lost when a new document is loaded. At the end this solution allows to capture some JavaScript errors but not all JavaScript errors.

Add support within the FirefoxDriver

Firefox allows extensions to get notified of JavaScript errors, no matter where they occur.
This mechanism is used for instance by Chris Pederick’s famous Web Developer Toolbar to display a warn/error icon when an error has occurred on a page. Once you’ve succeeded in overcoming the sandbox mechanism separating extensions and loaded pages, it is quite easy to write a new Firefox extension that captures the JavaScript errors and makes them available in each content window.
WebDriver on its side allows to add custom extensions to run in a FirefoxDriver. If you simply add this errors capturing extension to the FirefoxProfile you’ll have access to all JavaScript errors from within your tests. Luckily you don’t need to write it again and you can simply use my JSErrorCollector (Apache 2 Licence). It allows to write following:

...
import net.jsourcerer.webdriver.jserrorcollector.JavaScriptError;
...
final FirefoxProfile profile = new FirefoxProfile();
JavaScriptError.addExtension(profile);
final WebDriver driver = new FirefoxDriver(profile);
// ...
// navigate to some pages
// ...
List jsErrors = JavaScriptError.readErrors(driver);
assertThat(jsErrors, is(empty())

The JavaScriptError class holds the message, source name and line number of a JavaScript error just like what you can see in Firefox’s JavaScript console.

Next steps

Other browsers

It would be nice to have a simple access to JavaScript errors as well in the other browsers supported by WebDriver. For HtmlUnit it is a no-brainer. For Chrome it should be quite simple as an extension can register a global error listener. No idea what concerns IE and Opera.

Integration

Ideally the API should be provided natively by WebDriver, let’s see what will be the interest in my JSErrorCollector (follow issue 148 to stay informed).
Retrieving the JavaScript errors is a first step on which different interesting features could be based. Verifying the absence of JavaScript errors after each test (using for instance JUnit’s @After) works fine but is tedious. One interesting feature would be to let the tests directly fail on the first JavaScript error (I’ll address wrapping drivers in a future post). Alternatively a precise dedicated reporting would have great benefits too. Some filtering features to ignore errors in third party libraries would be a nice feature as well.

JSErrorCollector is open source therefore patches are welcome. If you can’t/don’t want to contribute directly, I’m available to work on it on a contract base.

WebDiver (alias Selenium 2) uses the By class to hold the mechanism used to locate elements within a document. It’s fluent API allows to write nice code like this:

driver.findElement(By.id("someId"));

WebDriver’s API offers currently eight locators allowing to retrieve elements by id, name attribute, tag name, complete or partial link text, XPath, class name, and css selector. These location mechanisms are (always?) backed directly by the driver implementation what ensures that the best execution method is used for each browser.

These locators are very powerful but often too low level. Luckily it is easy to write your own ones.

Locate input fields by associated label

Do you know the HTML tag <label …/>? Even if it is quite an old tag as it was introduced in HTML 4.0 back in 1998, it is not as widely used as it should. Besides its ability to improve accessibility  it can be a wonderful help when writing test code.
The <label …/> tag allows to associate some content (the label) to a form control. When a label element is clicked, it passes the event to the associated control.
This means that you don’t need to precisely target the small box/circle area to check/uncheck a checkbox (or a radio button) but rather the, mostly larger, associated description. For text fields, a click on the label will bring the focus to the field, allowing to enter content. Just nice.

Example:

 <label for='firstName'>First name</label> <input name='firstname' id='firstName'>

This clear association between label text and input field can be used for test automation.
Instead of

driver.findElement(By.id("firstName"))

or

driver.findElement(By.name("firstName"))

with a bit XPath experience, you can write:

driver.findElement(By.xpath("id(//label[text() = 'First name']/@for)"));

but it would be nicer to write

driver.findElement(By.label("First name")); // doesn't compile!

The bad news is that there is no such a label method in WebDriver’s By class. The good news is that it is quite easy to implement your own By subclass and to pack it in a static method:

public static By byLabel(final String label)
{
  return new By() {
    @Override
    public List<WebElement> findElements(final SearchContext context)
    {
      final String xpath =
         "//*[@id = //label[text() = \"" + label + "\"]/@for]";
      return ((FindsByXPath) context).findElementsByXPath(xpath);
    }

    @Override
    public WebElement findElement(final SearchContext context)
    {
      String xpath =
        "id(//label[text() = \"" + label + "\"]/@for)";
      return ((FindsByXPath) context).findElementByXPath(xpath);
    }

    @Override
    public String toString()
    {
      return "ByLabel: " + label;
    }
  };
}

Now we can retrieve the input field in a short and readable way, relying only on information directly visible in the rendered HTML page rather than on (sometime ugly and volatile) name or id attributes:

driver.findElement(byLabel("First name"));

Write custom locators tailored for YOUR application

I encourage my customers to write custom locators. WebDriver is not targetted to any particular web framework or page layout neither does it know anything about
your application domain therefore it can only contain general purpose locators.
When you write tests, you know about the application under test and you can easily leverage WebDriver’s API to write dedicated locators. This allows you to avoid duplication of the location’s complexity by centralizing it in the locators and makes your code easier to read and to maintain.
Candidates for custom locators are for instance menu item or tab by name, spreadsheet cell by column and row index, or product quantity field by product description.