HtmlUnit: Re-synchronize AJAX Calls for Simple, Deterministic Test Automation

The upcoming release of HtmlUnit will contain a very large number of fixes and improvements (particularly for the JavaScript support of libraries like Prototype or GWT). An interesting new feature is the experimental AjaxManager that allows to easily control the execution of AJAX calls.

As a user you often don’t even notice that the changes performed in an html page have been performed asynchronously. But this is just because you’re too slow and automated tests (which are very fast) will encounter a problem: they will want to perform some action on the page before this one has been updated.

Current approaches
Automated tests tools propose currently 2 approaches (*) to face this problem and (try to) ensure a deterministic test execution.

  • add an instruction to sleep for a certain amount of time before executing the next instruction (that’s the case for instance of tools like QTP or Squish)
  • add a waitXxxx instruction to wait until some element is available in the page (like for instance in WebAii or Selenium), element that gets created by the AJAX call

The first strategy is very rudimentary: if you don’t wait long enough, you may see your test failing just because the web server is more busy than usual. And if you wait too long, you will end up with test suites taking hours to finish. The second strategy has not this problem but requires the specification of an additional condition (that has to be maintained when the application evolves).

Drawback
In fact none of these strategies has tested that some call has been performed asynchronously (and this test was not required) and both require additional instructions that don’t really belong to the functional test.

Resynchronize (some) AJAX calls to allow simple, deterministic testing
Upcoming HtmlUnit release will allow to control AJAX execution and to turn asynchronous calls back to a synchronous execution mode. The consequence is that the next instruction of the test program is first executed when the AJAX execution is completed. This makes usage of sleep or waitXxx-like instructions needless what simplifies the test conception.

Nevertheless not all AJAX calls need to be turned synchronous. Typically calls that are not triggered by a user interaction like those from Prototype‘s Ajax.PeriodicalUpdater don’t be need to be re-synchronized (**). For this purpose HtmlUnit now contains the experimental NicelyResynchronizingAjaxController class which can be used to resynchronize only the AJAX calls made from the main execution thread which are normally the ones triggered by user actions.

Limitations
This feature is very new in HtmlUnit and the experience is missing to make a definitive evaluation. The re-synchronisation worked just fine for the application I had to test and allowed me to eliminate many lines of code. Nevertheless there are still cases where it wouldn’t probably be enough. If I correctly remember, when Prototype‘s Ajax.Updater for instance is configured to evaluate the scripts it receives, it uses a setTimeout to perform this task after having treated the rest of the response content. The consequence is that the action of these scripts wouldn’t be intercepted to be executed in the main “flow”. At least in this case the time consuming server round-trip would have been catch, allowing a far smaller “sleep” delay.

Could this technique be used by other test tools than HtmlUnit?

WebTest will of course benefit soon from this technique but this would probably make sense for other web app testing tools as well. This wouldn’t be a problem for those who simulate a browser as they are exactly in the same situation as HtmlUnit and have plain control over AJAX execution. For the tools that work within the browser using JavaScript, I guess that it could be possible to redefine the open method on the XMLHttpRequest objects (as well as on MS’s XMLHttpRequest ActiveX variants) to change the value of the asynchronous parameter. It may become a bit more tricky to determine what is triggered by user interaction to perform a selective resynchronisation. Finally for the tools that drive browsers using their API it will depend of the features offered by this API but this is a domain that I don’t know.

——
(*) at least this is what I could find in the documentation of the numerous tools I’ve looked at to write this post (it’s amazing how many – useless? – web testing tools you can find on the net!)
(**) in fact it probably doesn’t matter if resynchronized or not as the execution already runs in a separate thread

About these ads

20 Comments

  1. Elion said,

    July 24, 2007 at 4:21 pm

    I think the override of the open method of the Ajax operations are good in certain cases but you still need the Waitxx for situations like you mention when users use the settimeout to update the DOM. There might be other cases so you can’t really get rid of Waitxxx methods completely.

    There is also another missing point here that people need to be aware of when altering the flow of an application. You are no longer testing the app as your customers are going to use it. So to be as confident as you can in your application, you need to let Asyc ajax calls be async to detect any deadlocks/sync issues your app might have. I’ve seen many issues with apps having these problems and really the only way to catch them is to have your app behave as it is designed. Overriding javascript fundamental classes can be dangerous.

    Thanks for the ideas though…

  2. Marc Guillemot said,

    July 25, 2007 at 8:22 am

    Thanks for the comment.

    Re-synchronized Ajax calls only modify the application flow, when further script execution is performed after the call to the send() method. A typical XMLHttpRequest async use case it to react to a user action with no other processing being done once send() has been called, the “real” work being done in the callback. In this case, the flow just stays the same.

    It’s very optimistic to tell that deadlocks will be detected when the web app is tested the way it is designed. Normal skilled programmers will avoid most of the deadlock situations and therefore such deadlocks will only occur in particular race conditions. Such conditions are only reproducible in automated tests if you can “control the time”. This goes far beyond this post but is an interesting subject.

  3. Harald Hoyer said,

    July 25, 2007 at 7:41 pm

    Very nice! Keep up the good work! Exactly what I need since two hours ago :-)

  4. August 12, 2007 at 12:48 am

    […] HtmlUnit: Re-synchronize AJAX Calls for Simple, Deterministic Test Automation « Marc Guillemot… – The upcoming release of HtmlUnit will contain a very large number of fixes and improvements (particularly for the JavaScript support of libraries like Prototype or GWT). An interesting new feature is the experimental AjaxManager that allows to easily cont […]

  5. August 15, 2007 at 2:47 am

    […] which cut the build time by a third), and a couple of new features like Marc’s experimental AJAX controller. The change log has all the […]

  6. Alex said,

    October 27, 2007 at 2:07 am

    You can also check SWExplorerAuromarion (SWEA) from http://webius.net. SWEA monitors IE network traffic and simulates AJAX complete events making test execution deterministic.

  7. October 29, 2007 at 9:21 am

    […] Contrary to popular belief, you don’t need to run your test as JavaScript inside a browser to test AJAX functionality. HtmlUnit and thus WebTest is just as well up to the task. It can even be considered superior as it allows better control over how to schedule the in-page requests making the unpredictable browser behavior predictable (see for instance my previous post). […]

  8. Marc Guillemot said,

    October 30, 2007 at 12:57 pm

    Hi Alex,

    monitoring alone is not enough to make tests deterministic. I can’t tell more as I couldn’t find any documentation explaining what you indicate and I haven’t time to download a trial version and test by myself.

  9. Vanchinathan said,

    March 24, 2009 at 10:40 pm

    Hi,
    I’m using Html Unit 1.10. I was trying to test sites with ajax calls. The html unit does not get me the updated DOM.

    Please advise me on this.

    Vanchinathan

  10. Marc Guillemot said,

    March 25, 2009 at 7:45 am

    Hi Vanchinathan,

    HtmlUnit 1.10 has been released on… Nov 03, 2006. Are you interested in archaeology? ;-)

  11. Nelson said,

    May 11, 2009 at 12:14 pm

    i am trying to simulate an onChange event in select using htmlunit 2.5
    but it did’t seems to be happenning

    i am accompanying some code snippet which will support in helping me

    HtmlPage page = (HtmlPage) webClient.getPage(“http://loginassistant.iciciprulife.com/admin/InputPage.aspx”);
    HtmlForm form = page.getFormByName(“frmInput”);
    HtmlSelect product = form.getSelectByName(“CmbProductList”);
    product.setSelectedAttribute(“LifeLinkSuper”,true);

    • Marc Guillemot said,

      May 13, 2009 at 7:54 pm

      Nelson, please use HtmlUnit’s mailing list for such questions.

  12. sandrar said,

    September 10, 2009 at 9:33 pm

    Hi! I was surfing and found your blog post… nice! I love your blog. :) Cheers! Sandra. R.

  13. jlkl@fg.klghfg said,

    January 6, 2014 at 1:30 pm

    Hi! I was surfing and found your blog post… nice! I love your blog. :) Cheers! Sandra. R.

  14. jlkl@fg.klghfg said,

    January 6, 2014 at 1:38 pm

    i am trying to simulate an onChange event in select using htmlunit 2.5
    but it did’t seems to be happenning

    i am accompanying some code snippet which will support in helping me

    HtmlPage page = (HtmlPage) webClient.getPage(“http://loginassistant.iciciprulife.com/admin/InputPage.aspx”);
    HtmlForm form = page.getFormByName(“frmInput”);
    HtmlSelect product = form.getSelectByName(“CmbProductList”);
    product.setSelectedAttribute(“LifeLinkSuper”,true);

  15. January 6, 2014 at 1:39 pm

    Leave a Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: