Exactly the same applies to JavaScript errors in a web application. I believe that each JavaScript error should be considered as a bug and they should all be fixed before delivering, no matter if they directly impact the user experience or not.

Many great tools (JavaScript Console, Firebug, WebDeveloper Toolbar, …) allow to see the JavaScript errors while developing with Firefox as well as with other browsers and a wisely used error handler can report JavaScript errors to the server in production.

Sadly the support is not good while running automated tests. HtmlUnit provides a great help for that (see for instance this) but often integration tests should run in “real” browsers and not in HtmlUnit (for some good and many bad reasons). I don’t know currently any browser automation tool providing access to the JavaScript errors and to other information available in browsers like Firefox or Chrome that can be helpful for developers conscientious of the quality of their work

WebDriver is here not better than its concurrents and the issue “API for checking for JavaScript errors on the page” is opened since 2 1/2 years without any sign of life from the project’s members. The Selenium 1 issues SEL-522 and SEL-613 are equally asleep. Luckily WebDriver provides the possibility to configure additional extensions when using the FirefoxDriver, what allows to improve the situation.

Tester’s friendly application and injected error handler

The simpliest way to catch JavaScript errors for later retrieval is to add something like that

<script type="text/javascript">
window.jsErrors = [];
window.onerror = function(errorMessage) {
  window.jsErrors[window.jsErrors.length] = errorMessage;
}
</script>

This allows you to retrieve the captured errors with

((JavascriptExecutor) driver).executeScript("return window.jsErrors");

This is a simple solution but it has different drawbacks. First you need to add this code at the beginning of all your HTML pages. If you’re in the ideal situation to be both developer and tester of the application it’s “just” one Ctrl+C and a lot of Ctrl+V. If you need to convince your colleagues or your management that the web pages should be modified for testability, it will be more difficult. From my experience it is not always easy to make such a requirement accepted.
To overcome this problem we could imagine injecting the handler while running the test:

String script = "window.collectedErrors = [];"
  + "window.onerror = function(errorMessage) { "
  + "window.collectedErrors[window.collectedErrors.length] = errorMessage;"
  + "}";
((JavascriptExecutor) driver).executeScript(script);

but besides the fact that it has to be done on every page, we can’t do it early enough: a lot of JavaScript may already have been executed in the page when our executeScript statement is reached.

The second problem of an error handler per window is the difficult access to the captured errors. The JavaScript errors won’t necessarily occur in the currently focused window forcing you to iterate through all (i)frames and windows to check for errors. This can still be seen as a minor issue but the real problem is that captured errors get lost when a new document is loaded. At the end this solution allows to capture some JavaScript errors but not all JavaScript errors.

Add support within the FirefoxDriver

Firefox allows extensions to get notified of JavaScript errors, no matter where they occur.
This mechanism is used for instance by Chris Pederick’s famous Web Developer Toolbar to display a warn/error icon when an error has occurred on a page. Once you’ve succeeded in overcoming the sandbox mechanism separating extensions and loaded pages, it is quite easy to write a new Firefox extension that captures the JavaScript errors and makes them available in each content window.
WebDriver on its side allows to add custom extensions to run in a FirefoxDriver. If you simply add this errors capturing extension to the FirefoxProfile you’ll have access to all JavaScript errors from within your tests. Luckily you don’t need to write it again and you can simply use my JSErrorCollector (Apache 2 Licence). It allows to write following:

...
import net.jsourcerer.webdriver.jserrorcollector.JavaScriptError;
...
final FirefoxProfile profile = new FirefoxProfile();
JavaScriptError.addExtension(profile);
final WebDriver driver = new FirefoxDriver(profile);
// ...
// navigate to some pages
// ...
List jsErrors = JavaScriptError.readErrors(driver);
assertThat(jsErrors, is(empty())

The JavaScriptError class holds the message, source name and line number of a JavaScript error just like what you can see in Firefox’s JavaScript console.

Next steps

Other browsers

It would be nice to have a simple access to JavaScript errors as well in the other browsers supported by WebDriver. For HtmlUnit it is a no-brainer. For Chrome it should be quite simple as an extension can register a global error listener. No idea what concerns IE and Opera.

Integration

Ideally the API should be provided natively by WebDriver, let’s see what will be the interest in my JSErrorCollector (follow issue 148 to stay informed).
Retrieving the JavaScript errors is a first step on which different interesting features could be based. Verifying the absence of JavaScript errors after each test (using for instance JUnit’s @After) works fine but is tedious. One interesting feature would be to let the tests directly fail on the first JavaScript error (I’ll address wrapping drivers in a future post). Alternatively a precise dedicated reporting would have great benefits too. Some filtering features to ignore errors in third party libraries would be a nice feature as well.

JSErrorCollector is open source therefore patches are welcome. If you can’t/don’t want to contribute directly, I’m available to work on it on a contract base.

driver.findElement(By.id("someId"));

WebDriver’s API offers currently eight locators allowing to retrieve elements by id, name attribute, tag name, complete or partial link text, XPath, class name, and css selector. These location mechanisms are (always?) backed directly by the driver implementation what ensures that the best execution method is used for each browser.

These locators are very powerful but often too low level. Luckily it is easy to write your own ones.

Locate input fields by associated label

Do you know the HTML tag <label …/>? Even if it is quite an old tag as it was introduced in HTML 4.0 back in 1998, it is not as widely used as it should. Besides its ability to improve accessibility  it can be a wonderful help when writing test code.
The <label …/> tag allows to associate some content (the label) to a form control. When a label element is clicked, it passes the event to the associated control.
This means that you don’t need to precisely target the small box/circle area to check/uncheck a checkbox (or a radio button) but rather the, mostly larger, associated description. For text fields, a click on the label will bring the focus to the field, allowing to enter content. Just nice.

Example:

 <label for='firstName'>First name</label> <input name='firstname' id='firstName'>

This clear association between label text and input field can be used for test automation.
Instead of

driver.findElement(By.id("firstName"))

or

driver.findElement(By.name("firstName"))

with a bit XPath experience, you can write:

driver.findElement(By.xpath("id(//label[text() = 'First name']/@for)"));

but it would be nicer to write

driver.findElement(By.label("First name")); // doesn't compile!

The bad news is that there is no such a label method in WebDriver’s By class. The good news is that it is quite easy to implement your own By subclass and to pack it in a static method:

public static By byLabel(final String label)
{
  return new By() {
    @Override
    public List<WebElement> findElements(final SearchContext context)
    {
      final String xpath =
         "//*[@id = //label[text() = \"" + label + "\"]/@for]";
      return ((FindsByXPath) context).findElementsByXPath(xpath);
    }

    @Override
    public WebElement findElement(final SearchContext context)
    {
      String xpath =
        "id(//label[text() = \"" + label + "\"]/@for)";
      return ((FindsByXPath) context).findElementByXPath(xpath);
    }

    @Override
    public String toString()
    {
      return "ByLabel: " + label;
    }
  };
}

Now we can retrieve the input field in a short and readable way, relying only on information directly visible in the rendered HTML page rather than on (sometime ugly and volatile) name or id attributes:

driver.findElement(byLabel("First name"));

Write custom locators tailored for YOUR application

I encourage my customers to write custom locators. WebDriver is not targetted to any particular web framework or page layout neither does it know anything about
your application domain therefore it can only contain general purpose locators.
When you write tests, you know about the application under test and you can easily leverage WebDriver’s API to write dedicated locators. This allows you to avoid duplication of the location’s complexity by centralizing it in the locators and makes your code easier to read and to maintain.
Candidates for custom locators are for instance menu item or tab by name, spreadsheet cell by column and row index, or product quantity field by product description.

I’ll present two sessions at this year’s W-JAX conference in Munich:

• Monitoring von Webanwendungen (together with my friend Peter Roßbach of Apache Tomcat‘s fame)
• JUnit 4 ist nicht nur @Test!

We will hold our session about web application monitoring twice as conference’s organisation expects a high number of attendees.

If you read this post and are present at W-JAX 2010, come to say hello. I offer a beer to the first one who tells me that he has read this post.

This release comes later as originally planned as we wanted to avoid as most as possible any side effect of the large internal changes. I’m now quite confident that we have reached a really good status.

Here is an extract of the change log:

• single thread for background JS execution
• migration to HttpClient 4
• initial support for FF3.6 simulation
• support for SOCKS proxy
• initial support for Google AppEngine
• support for large (binary) download
• …

Download: http://sourceforge.net/project/showfiles.php?group_id=47038

The support for Google AppEngine which is based on contributions from Google’s GWT team is exciting as it will open the door to new usage of HtmlUnit.

The number of HtmlUnit users continuously grows and, as the consequence, so does the number of reported issues and feature wishes. We will try to address it in next release.

Note: I’m not a Maven expert. The proposed code works but it can perhaps be improved.

1. Create a new Maven project
   

   mvn archetype:create -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId=my.domain -DartifactId=myWebTestApp
   

2. Edit the generated pom.xml
   1. Add WebTest as dependency:
      

          <dependencies>
              ...
              <dependency>
                  <groupId>com.canoo.webtest</groupId>
                  <artifactId>webtest</artifactId>
                  <version>3.1-SNAPSHOT</version>
              </dependency>
          </dependencies>
      

   2. Add reference to WebTest’s Maven snapshot repository
      

      	    <repositories>
      	        <repository>
      	            <id>webtest_dependencies_snapshot</id>
      	            <name>WebTest dependencies</name>
      	            <url>http://webtest.canoo.com/webtest/m2-repo-snapshots</url>
      	        </repository>
      	    </repositories>
      

      Note that this is needed only if you want to use the latest WebTest snapshot (which is always the best one). You don’t need it if you use a version that is available on the central repository. As of Apr. 30, WebTest 3.0 has not been uploaded to the central repository. The upload request is available here: MAVENUPLOAD-2439.

      Update: WebTest 3.0 is available in Maven central repository since May 19.

   3. Configure the GMaven plugin
      

      <build>
      	<plugins>
      		<plugin>
      			<groupId>org.codehaus.groovy.maven
      			</groupId>
      			<artifactId>gmaven-plugin</artifactId>
      			<version>1.0-rc-5</version>
      			<executions>
      				<execution>
      					<goals>
      						<goal>compile</goal>
      						<goal>testCompile</goal>
      					</goals>
      				</execution>
      			</executions>
      		</plugin>
      		<plugin>
      			<groupId>org.codehaus.mojo</groupId>
      			<artifactId>build-helper-maven-plugin</artifactId>
      			<executions>
      				<execution>
      					<id>add-test-source</id>
      					<phase>generate-sources</phase>
      					<goals>
      						<goal>add-test-source</goal>
      					</goals>
      					<configuration>
      						<sources>
      							<source>src/test/groovy</source>
      						</sources>
      					</configuration>
      				</execution>
      			</executions>
      		</plugin>
      	</plugins>
      </build>
      

3. Create the Eclipse project
   

   mvn -Declipse.downloadSources=true eclipse:eclipse
   

   I guess that something similar exists for other IDEs.

4. Import the project in your workspace
5. Add Groovy support in the IDE
   The Groovy support in Eclipse is not as good as in other IDEs but it is better than nothing.
6. Write your first WebTest in Groovy src/test/groovy/my/domain/FirstWebTest.groovy
   

   package my.domain
   
   import com.canoo.webtest.WebtestCase
   
   /**
    * My first WebTest in Groovy.
    */
   class FirstWebTest extends WebtestCase {
   
   	void testGoogleSearch() {
   		webtest("My first test") {
   			invoke "http://www.google.com/ncr", description: "Go to Google (in English)"
   			verifyTitle "Google"
   			setInputField name: "q", value: "WebTest"
   			clickButton "I'm Feeling Lucky"
   			verifyTitle "Canoo WebTest"
   		}
   	}
   }
   

7. Run the test
   • As normal unit test from the IDE
     As it is a normal unit test and you can use the IDE support for that. In Eclipse this can be done with right mouse click / Run As... / Junit Test or with the shortcut Alt + Shift + X Z
   • Or from the command line
     

     mvn test
     

8. Enjoy the results!
   Running a WebTest this way produces “normal” JUnit reports as well as the traditional reports of WebTest that contain precise information to quickly find the cause of failed tests.
9. (optional) Run headless
   Per default WebTest shows a console that informs about the running tests and opens the reports in your browser once the tests are finished. This is often useful on a workstation but it may be really annoying on a build server. To avoid that, you just need to set the wt.headless property:
   

   mvn test -Dwt.headless
   

This year I’m involved in the organization of a small part of the event: the Java sub conference. If you have some interesting topic(s) that you want to present around Java and Open Source, don’t wait any longer, go to the Call for Paper site and submit your proposal.

Here is an extract of the change log:

• improved JS support, particularly full support for MooTools
• repackaged Rhino classes to allow to use HtmlUnit and a regular Rhino version in the same project
• experimental WebClient.waitForBackgroundJavaScript() and WebClient.waitForBackgroundJavaScriptStartingBefore() for simple, fast and deterministic AJAX testing
• reworked handling of background JS tasks using JDK5 executors
• …

Download: http://sourceforge.net/project/showfiles.php?group_id=47038

They will present ” HtmlUnit: An Efficient Approach to Testing Web Applications ” at the biggest Java conference in San Francisco (May 31 – June 05).

This is an additional very good sign for the constantly growing recognition of HtmlUnit and I hope that it will be the occasion for them to meet a lot of HtmlUnit users.

Nothing is decided, we’re just thinking loud and are interested to hear what our users think of this idea. I’ve started the discussion in HtmlUnit user mailing list but comments are welcome here too.

I will present “HtmlUnit: An Efficient Approach to Testing Web Applications” together with Daniel Gredler at TheServerSide Java Symposium in Las Vegas (March, 18).

This will be funny because I work together with Daniel in the HtmlUnit project since a few years and I’ve never met him until now.

I think that this will be a great occasion to show HtmlUnit’s potential and I hope that we will meet a lot of HtmlUnit users.

Thanks to Tools & Techniques chair Frank Cohen for inviting us.