HtmlUnit: Re-synchronize AJAX Calls for Simple, Deterministic Test Automation

July 24, 2007 at 1:24 pm (HtmlUnit, Test Automation)

The upcoming release of HtmlUnit will contain a very large number of fixes and improvements (particularly for the JavaScript support of libraries like Prototype or GWT). An interesting new feature is the experimental AjaxManager that allows to easily control the execution of AJAX calls.

As a user you often don’t even notice that the changes performed in an html page have been performed asynchronously. But this is just because you’re too slow and automated tests (which are very fast) will encounter a problem: they will want to perform some action on the page before this one has been updated.

Current approaches
Automated tests tools propose currently 2 approaches (*) to face this problem and (try to) ensure a deterministic test execution.

• add an instruction to sleep for a certain amount of time before executing the next instruction (that’s the case for instance of tools like QTP or Squish)

• add a waitXxxx instruction to wait until some element is available in the page (like for instance in WebAii or Selenium), element that gets created by the AJAX call

The first strategy is very rudimentary: if you don’t wait long enough, you may see your test failing just because the web server is more busy than usual. And if you wait too long, you will end up with test suites taking hours to finish. The second strategy has not this problem but requires the specification of an additional condition (that has to be maintained when the application evolves).

Drawback
In fact none of these strategies has tested that some call has been performed asynchronously (and this test was not required) and both require additional instructions that don’t really belong to the functional test.

Resynchronize (some) AJAX calls to allow simple, deterministic testing
Upcoming HtmlUnit release will allow to control AJAX execution and to turn asynchronous calls back to a synchronous execution mode. The consequence is that the next instruction of the test program is first executed when the AJAX execution is completed. This makes usage of sleep or waitXxx-like instructions needless what simplifies the test conception.

Nevertheless not all AJAX calls need to be turned synchronous. Typically calls that are not triggered by a user interaction like those from Prototype‘s Ajax.PeriodicalUpdater don’t be need to be re-synchronized (**). For this purpose HtmlUnit now contains the experimental NicelyResynchronizingAjaxController class which can be used to resynchronize only the AJAX calls made from the main execution thread which are normally the ones triggered by user actions.

Limitations
This feature is very new in HtmlUnit and the experience is missing to make a definitive evaluation. The re-synchronisation worked just fine for the application I had to test and allowed me to eliminate many lines of code. Nevertheless there are still cases where it wouldn’t probably be enough. If I correctly remember, when Prototype‘s Ajax.Updater for instance is configured to evaluate the scripts it receives, it uses a setTimeout to perform this task after having treated the rest of the response content. The consequence is that the action of these scripts wouldn’t be intercepted to be executed in the main “flow”. At least in this case the time consuming server round-trip would have been catch, allowing a far smaller “sleep” delay.

Could this technique be used by other test tools than HtmlUnit?

WebTest will of course benefit soon from this technique but this would probably make sense for other web app testing tools as well. This wouldn’t be a problem for those who simulate a browser as they are exactly in the same situation as HtmlUnit and have plain control over AJAX execution. For the tools that work within the browser using JavaScript, I guess that it could be possible to redefine the open method on the XMLHttpRequest objects (as well as on MS’s XMLHttpRequest ActiveX variants) to change the value of the asynchronous parameter. It may become a bit more tricky to determine what is triggered by user interaction to perform a selective resynchronisation. Finally for the tools that drive browsers using their API it will depend of the features offered by this API but this is a domain that I don’t know.

——
(*) at least this is what I could find in the documentation of the numerous tools I’ve looked at to write this post (it’s amazing how many – useless? – web testing tools you can find on the net!)
(**) in fact it probably doesn’t matter if resynchronized or not as the execution already runs in a separate thread