Here are the slides of the presentation I’ve made yesterday:
This release contains numerous improvements and bug fixes, particularly in handling of ill formed HTML code and in AJAX support as well as noticeable speed gains making it even faster in different situations.
If you look at the download statistics of HtmlUnit and HttpUnit at Sourceforge (here for HtmlUnit and here for HttpUnit), you’ll see that HttpUnit still has more downloads than HtmlUnit. This is quite surprising when you have some knowledge of both libraries and particularly of the fact that HttpUnit is deeply asleep since so many years (the recent 1.7 release doesn’t really change the deal).
In his blog post “HtmlUnit vs HttpUnit“, Daniel Gredler wrote “HttpUnit is to the web app testing world what Struts is to the web app framework world: there are many “better” options out there, but it just won’t go away!“. I totally agree but it is interesting to examine the reasons why it doesn’t go away. I believe that one element of the answer is that many “experts” have outdated knowledge in this domain and still talk about HttpUnit.
- Rod Johnson
In “System Integration Testing Using Spring” at the The Spring Experience conference, Rod talked about HttpUnit but not about HtmlUnit.
The JBoss project JSFUnit contains HttpUnit not HtmlUnit.
Nevertheless it seems that in this case the mistake has been identified as a feature request “Investigate replacing HttpUnit with HtmlUnit” already exists.
- Recent books
Different recent books mention HttpUnit but not HtmlUnit, like for instance Java Power Tools (by John Ferguson Smart), Test Driven, Practical TDD and Acceptance TDD for Java Developers (by Lasse Koskela), or Ant in Action, 2nd Edition (by Steve Loughran and Erik Hatcher). In this latest case, rather than HtmlUnit, they should probably have written about WebTest as it is probably the tool – web testing or not – that makes the most intensive usage of Ant.
Why did all these famous developers talk / write about HttpUnit? My guess is simply that functional tests of web applications are not their favorite field of expertise and therefore that they didn’t update their knowledge in this area since a – too – long time.
I see two consequences. First it gives a bad impression on the whole presented content: no matter how good it may be, when you detect outdated areas, you start to question the rest. Second, it influences newcomers that have no idea of the domain and this can be a good explaination for HttpUnit surprising high download numbers.
If you search for a load testing tool to test your AJAX application, you will surely find many vendors pretending that their tool support AJAX applications. If you look a bit more carefully in the detailed features, you’ll quickly find that it is done by recording the http traffic.
HTTP traffic recording limits
Capturing should always be considered very carefully for functional testing as well as for load testing. In the case of AJAX applications, it can simply lead you to generate a load testing scenario that may work… but that has nothing to do with the traffic that would really occur for the number of simulated users.
A small example
An example is worth 1000 words, let’s look at the code of this minimal AJAX application
<html><head><title>Welcome</title><body> <script> var xhr = window.XMLHttpRequest ? new XMLHttpRequest() : new ActiveXObject('Microsoft.XMLHTTP'); xhr.open('GET', 'numberOfVisitors.txt', false); xhr.send( "" ) var nbConnectedVisitors = parseInt(xhr.responseText) var nextPage; if (nbConnectedVisitors < 100) nextPage = "fullFeaturedSite.html" else if (nbConnectedVisitors < 1000) nextPage = "limitedSite.html" else nextPage = "temporarilyUnavailable.html" window.location = nextPage </script> </body></html>
fullFeaturedSite.html” if less than 100 users are connected, “
limitedSite.html” for 100 until 1000 connected users and “
temporarilyUnavailable.html” when over 1000 users are connected.
If you use a recorder to create a test for this application, you’ll probably be the single connected user while recording. When you’ll run the recorded tests to simulate the last of a large number of users, you’ll perform charge on your server of course (here requesting “
fullFeaturedSite.html“), but this charge is something that would never occur when over 100 users are connected, what means that your test results have nearly no signification.
Client side JS evaluation needed
It wouldn’t be really complicated here to adapt the load script to parse the content of the response from “
numberOfVisitors.txt” in order to perform the adequate next request. The downside of such a strategy is naturally the duplication of the logic of the application in the load test scripts which cause high maintenance costs and is not necessarily always as easy as in this oversimplified example.
A place for HtmlUnit as load testing tool?
HtmlUnit in .NET
HtmlUnit is a Java library but its usage seems not limited to the JVM. For some time I found the blog post “Some Goodies: Model-View-Presenter Article, Java in .NET, HtmlUnit” where the author describes how he successfully used IKVM to use HtmlUnit from his .NET environment. Quite funny.
HtmlUnit in JRuby: Celerity as faster Watir
The initial setup for the second example was surely simpler as it runs on the JVM. The Norwegian provider FINN.no had performance problems with its Watir test suite completing in 3 hours which is not very agile. To improve this situation, they decided to create the Open Source project Celerity to provide a Watir-compatible API wrapping HtmlUnit to improve execution speed without to rewrite all their tests. It would be interesting to know how fast their suite now runs and if they use new possibilities of this approach like running tests in parallel.
The Celerity project perfectly demonstrates that speed matters. You can frequently find “experts” writing that functional tests are slow per nature. This is fully wrong, functional tests won’t be as fast as unit tests but HtmlUnit and derived tools (like Celerity, WebDriver with HtmlUnit driver, WebTest) demonstrate that functional tests can be quite fast.
.NET, JRuby, and then?
The interest for HtmlUnit continually grows and I’m curious to see what will be the next “exotic” usage of HtmlUnit. Don’t hesitate to communicate your experiences.
Nevertheless we are blocked by some issues in Rhino that don’t get fixed – or not quickly enough – even when we propose patches (for instance bug 412928). As discussed in Rhino mailing list  this seems to come from a lack of resources on Rhino’s side as well as from a slightly different focus: for us the most important is to simulate browser’s behaviour whereas speed and respect of ECMA standard are very important for Rhino.
Releases of htmlunit-core-js will be made available at the same time than future HtmlUnit releases.
My biggest hope for this project is… to be able to declare it dead and useless as soon as possible, once Rhino fills all our need. For this purpose, we will continue to open issues and provide patches to the Rhino project to help improving it.
 Rhino mailing list thread “Rhino project (half) asleep?“
This time HtmlUnit committer Daniel Gredler has been faster to blog about it!
HtmlUnit 2.1 is now available. We’ve put this release less than 2 weeks after version 2.0 to quickly fix some performance problems that occurred with HtmlUnit 2.0 in CSS processing (users have reported that 2.1 runs up to 5 times faster than 2.0). Additionally this release contains a few improvements and bug fixes as documented in the change log.
Four months after release 1.14, I’m glad to announce the availability of HtmlUnit 2.0. Thanks to everyone who has contributed to this release and particularly to the committers Ahmed Ashour and Daniel Gredler.
From the large list of improvements contained in this release, following are particularly important:
- first release to target Java 5: HtmlUnit now makes intensive usage of Java 5 features, particularly of generics
- HtmlUnit DOM now implements org.w3c.dom.* making interaction with 3rd party libraries easier
- improved support for incorrect HTML code
- and as always a very large number of improvements in JS support with a major achievement: the GWT 1.4 tests now pass when simulating Firefox as well as when simulating Internet Explorer!
This major release contains a few incompatible changes and can’t be dropped as a replacement of HtmlUnit-1.14.