AJAX and load testing: HtmlUnit as a solution?

If you search for a load testing tool to test your AJAX application, you will surely find many vendors pretending that their tool support AJAX applications. If you look a bit more carefully in the detailed features, you’ll quickly find that it is done by recording the http traffic.

HTTP traffic recording limits

Capturing should always be considered very carefully for functional testing as well as for load testing. In the case of AJAX applications, it can simply lead you to generate a load testing scenario that may work… but that has nothing to do with the traffic that would really occur for the number of simulated users.

A small example

An example is worth 1000 words, let’s look at the code of this minimal AJAX application

<html><head><title>Welcome</title><body>
<script>
var xhr = window.XMLHttpRequest ? new XMLHttpRequest() 
  : new ActiveXObject('Microsoft.XMLHTTP');
xhr.open('GET', 'numberOfVisitors.txt', false);
xhr.send( "" )

var nbConnectedVisitors = parseInt(xhr.responseText)
var nextPage;
if (nbConnectedVisitors < 100)
	nextPage = "fullFeaturedSite.html"
else if (nbConnectedVisitors < 1000)
	nextPage = "limitedSite.html"
else
    nextPage = "temporarilyUnavailable.html"
	
window.location = nextPage
</script>
</body></html>

It is extremely simple: the first html page uses an XMLHttpRequest to ask the server for the number of currently connected users. Client side javascript logic is used to determine the page to show next, depending on the current charge on the server: “fullFeaturedSite.html” if less than 100 users are connected, “limitedSite.html” for 100 until 1000 connected users and “temporarilyUnavailable.html” when over 1000 users are connected.
If you use a recorder to create a test for this application, you’ll probably be the single connected user while recording. When you’ll run the recorded tests to simulate the last of a large number of users, you’ll perform charge on your server of course (here requesting “fullFeaturedSite.html“), but this charge is something that would never occur when over 100 users are connected, what means that your test results have nearly no signification.

Client side JS evaluation needed

It wouldn’t be really complicated here to adapt the load script to parse the content of the response from “numberOfVisitors.txt” in order to perform the adequate next request. The downside of such a strategy is naturally the duplication of the logic of the application in the load test scripts which cause high maintenance costs and is not necessarily always as easy as in this oversimplified example.

A place for HtmlUnit as load testing tool?

Load testing tools are able to simulate a large number of simultaneous users because they are low level and perform only minimal processing of the server responses. Visual web browsers on the other side can interpret the JavaScript code on the visited web sites but are too heavy weight to be used to produce load unless you have a very large number of computers available for the exercise. HtmlUnit could play an interesting role between these two extrema: it is lightweight enough to allow to simulate a few 100 users on a single machine and it has already an excellent support for JavaScript, allowing it to behave just like a “normal” browser for numerous AJAX applications. As far as I know, HtmlUnit is unique in this category therefore I believe that its usage for load testing will grow in the future months.

10 Comments

  1. Mark Leiter said,

    May 22, 2008 at 1:42 pm

    Our company would definitely consider using HtmlUnit for load testing. It would be great if we could use the same test scripts for both QA and load testing instead of having two sets of tests in two different environments. A second advantage would be the cost. Many of the existing load testing tools are very expensive.

  2. David Feid said,

    June 16, 2008 at 7:39 pm

    I find that HTMLUnit won’t even run the kitchen sink application in GWT. That makes me wonder just how good a tool it is….

  3. Marc Guillemot said,

    June 17, 2008 at 7:40 am

    Tests with GWT 1.4.60 run in each build of HtmlUnit (https://htmlunit.svn.sourceforge.net/svnroot/htmlunit/trunk/htmlunit/src/test/java/com/gargoylesoftware/htmlunit/libraries/GWT14Test.java). Nevertheless you are right, there is currently an issue with GWT 1.4.61/62 as well as 1.5RC1. I think that we will fixed it in release 2.2.

  4. Gopal Addada said,

    September 17, 2008 at 3:45 pm

    After reading this blog post, I started building load tests for our application using HTMLUnit. (I tried to use JMeter, it works for NON-AJAX Struts part of out application, but not for AJAX applications. Also, Httpunit that I am fammiliar with provides ZERO support for AJAX and Java Script) GOOD NEWS is that Load test is successful and my task is accomplished. I STRONGLY recommend people top use htmlunit for load testing AJAX web apps.

    THANKS MARC for your tool and this useful Blog.

  5. Marc Guillemot said,

    September 17, 2008 at 6:40 pm

    Thanks Gopal for this comment. Can you perhaps give some information on your load tests (number of simulated users, required CPU time, …)?

  6. Marc Guillemot said,

    September 17, 2008 at 6:59 pm

    @Gopal:

    HtmlUnit is not MY tool. Even if my involvement is not insignificant, current state is the result of the work of many committers (and contributors).

  7. Gopal said,

    September 18, 2008 at 4:38 pm

    Hi Marc,
    Sorry for wrong statement. What I mean is your initiative!

    Ours is web portal that involves so many search pages (that search Database as well as some index-based searches). We used new load test application to simulate up to 25 simultaneous users (using Thread Pool) and hit the app and perform searches. I calculated the response time by getting system time before search is performed and after results page is retrieved (using Sytem.currentTimeMillis() ).

    Our application uses “iceFaces”, hence to make that work I used:
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());
    and this does the magic!

    Thanks,
    Gopal.

  8. Jose said,

    September 30, 2008 at 6:19 pm

    Hi Marc,

    I’m trying to do a performance test on a GWT framewotk (use AJax) and I can’t run the test with Jmeter, I dowload HTMLUnit, but I can’t Install it😦😦

    Someone can help me with this? How install HTMLUnit?

    Thanks in advance,

    Jose

  9. Marc Guillemot said,

    October 1, 2008 at 7:23 am

    How to install HTMLUnit? Just place the libs in your classpath and start coding. But please prefer the HTMLUnit user mailing list for such questions.

  10. Jose said,

    October 1, 2008 at 12:42 pm

    thanks!!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: