WJAX 2010: WebApp Monitoring & JUnit 4

I’ll present two sessions at this year’s W-JAX conference in Munich:

We will hold our┬ásession about web application monitoring twice as conference’s organisation expects a high number of attendees.

If you read this post and are present at W-JAX 2010, come to say hello. I offer a beer to the first one who tells me that he has read this post.

Call for Papers: FrOSCon Java Sub-Conference, Sankt-Augustin 22-23.08.09

The 4th edition of the Free Open Source Conference (in short FrOSCon) in Sankt Augustin (near to Bonn, Germany) will take place on 22. and 23. August 2009.

This year I’m involved in the organization of a small part of the event: the Java sub conference. If you have some interesting topic(s) that you want to present around Java and Open Source, don’t wait any longer, go to the Call for Paper site and submit your proposal.

Thoughts on Selenium Grid

I’m impressed by the activity around Selenium. Not because I’m impressed by Selenium but because I think that all this work could be better invested. For instance it would surely be a smaller effort to make for instance HtmlUnit able to work with its recognized speed with numerous browser configurations, simulating perfectly their JavaScript execution, and not only 2 browsers as Selenium really supports. The last Selenium related project I’ve discovered is Selenium Grid.

Run tests in parallel

Selenium Grid’s aims to allow running tests on multiple computer because:

  • Selenium remote control is quite slow at driving the browser“.
  • You can only run a limited number of concurrent tests on the same remote control before seriously impacting its stability. Practically speaking, launching more than 6 browsers on the same Selenium Remote Control is not advisable. The limitations are even more drastic for Internet Explorer.

In other words: slow and not scalable. If I correctly understand the documentation, Selenium Grid will allow to run approximatively 6 x N tests in parallel, where N is the number of computers in the grid. Not really impressive.

An HtmlUnit user reported that he performed load testing with HtmUnit using 350 threads in the same VM and one WebClient instance per thread. For this purpose he allocated 512M of his 1Gig laptop to the VM and it worked very well. WebTest surely adds some overhead to HtmlUnit “pure” but, due to the indications of this user, I could imagine that it would be possible to run 500 threads of WebTest tests on one computer within a single VM. This would save just… 80 computers compared to Selenium Grid!

Parallelisation often limited

Selenium Grid’s documentation says: “Selenium Grid cuts down on the time required to run a Selenium test suite to a fraction of the time that a single instance of Selenium instance would take to run“. Sadly this isn’t that easy. Not all tests can be run in parallel: as soon as you test the functionalities of registered users, you’re often limited to a serial test execution (or a just a few parallel executions) when your application doesn’t accept numerous simultaneous login of the same user and you don’t have so much users available with the right settings for the tests. In this case only pure speed can help.

Grid computing is an exciting large area, perhaps is Selenium Grid only a side product of some more significant activity…

Only Selenium can test it? What? Surely not this simple page

When you look at a conference program, the session abstracts are naturally a kind of publicity to motivate people to attend and should be considered as such. Nervertheless an abstract shouldn’t exagerate too much to stay credible.

“… including applications that only Selenium can test”. Where?

In Neal Ford’s numerous sessions about Selenium, you will see again and again that he will present “testing Ajax applications (including applications that only Selenium can test)” (see for instance in
Advanced Selenium (No Fluff Just Stuff)
). When you write such a provocative statement, you have to prove it. I’ve attended one of his presentations (it was really poor, see previous post here) but I haven’t seen anything that would cause trouble to all existing web testing tools except to Selenium.

But very simple pages can’t be tested with Selenium

On the other side a page as simple as this example

document.getElementById = function(_id)
return {id: _id, nodeType: 3, data: 'he he, you failed!'};
<body id='bodyId'>
this is a very simple page

will currently cause enorm problems to Selenium that won’t even be able to execute correctly a simple
verifyText id=bodyId this is a very simple page

The reason? Selenium uses the DOM functions to work on the html document and therefore relies on them.

Ok, this point isn’t a very big limitation as changing the functionality of the DOM functions is not a common need. In fact this post was just to say that I wait to see a web application that “only Selenium can test” but that even a very small page can cause problems to Selenium whereas it is handled correctly by most of the other web testing tools.

Let’s start

Until recently I thought that a blog was nothing for me… and now I’ve mine too.

The main subjects of my posts will probably be WebTest, HtmlUnit and test automation of web application in general as well as Groovy.

The main language will be English. This means my not-really-perfect-but-hopefully-understandable English with a French accent and a touch of German.

Now let’s see how long it takes before my blog receives the first comments.

PS: thanks to Daniel Gredler who recommended me to use WordPress and whom I stole the style for this blog.