XPath Power (2): Detect Lack of Experience

July 31, 2007 at 11:28 am (Test Automation)

A lot of people feel qualified to talk about test automation of web applications, probably because the principles are really easy: tests have “only” to do what a “normal” user would do. In the practice many of these persons don’t have real test automation experience and you shouldn’t trust their statement directly.

The problem is that it’s difficult to determine if a person has a real experience and therefore if the proposed solutions have proved their worth or not. Declarations from a person that hasn’t written tests receiving at least a few hundreds of responses from the web server are questionable. Exactly the same is valid for people talking from recording tools without to give any advice on what can be expected from the recorded scripts.

An other way to detect lack of experience is to look at the XPath expressions used. XPath support is indispensable in test automation of web application but depending how the xpath expressions are written, they may be far too “strong” and fail on the first unrelated change in the tested html page.

A few “bad” XPath examples
Here are a few examples of “bad” xpath available in blogs, conference slides or even tool documentation:

/html/body/table[2]/tbody/tr/td[2]/table[2]/tbody/tr/td/div/ul[1]/li[4]
from http://agiletesting.blogspot.com/2006/01/useful-tools-for-writing-selenium.html

//body[1]/table[1]/tbody[1]/tr[1]/td[1]/img[1]
from http://www.artoftest.com/Resources/WebAii/Documentation/topicsindex.aspx?topic=elementid

//html[1]/body[1]/div[1]/div[1]/h3[1]
from http://artoftestinc.blogspot.com/2007/01/ajax-testing-and-waiting-for-page.html

//html/body/div[@id='outerDiv']/div[@id='innerDiv']/div
[@id='pinDialog']/table/tbody/tr/td
from http://www.nealford.com/downloads/conferences/canonical/Neal_Ford-Advanced_Selenium-handouts.pdf

/html/body/table/tr[2]/td/table/tr[1]/td/a
from http://wiki.rubyonrails.org/rails/pages/HowtoFunctionalTest

9 Comments

XPath Power

July 25, 2007 at 1:01 pm (Test Automation, WebTest)

XPath support is indispensable for any serious web test automation tool because xpath expressions allow to precisely select nearly everything in an HTML page using a concise syntax.

Last week I had again a case showing the power of XPath. Colleagues had to write a WebTest script that should (among other steps) change the value of a select like this one:

<select name="mySelect">
<option value=""></option>
<option value="1">English</option>
<option value="2">French</option>
<option value="3">German</option>
</select>

(this blog doesn’t allow to “display” this select field it as it would normally appear, but you have surely enough HTML knowledge to understand it)

No assumption could be made on the current value of the select, the test had just to select an other valid option (meaning not the first one).

First approach
The first approach could be to read the index of the selected option, increment it, perform a modulo operation and avoid the first option. More pragmatic would be to take the second one (here “English”) if it is not already selected and the 3rd one (here “French”) if “English” was selected.
These solutions work but require some logic that can only be expressed in a script.

More elegant solution with xpath
In fact you can simply formulate the test specification with an XPath like this one:

//select[@name='mySelect']/option[text() and . != ../option[@selected]]

This XPath just selects all the options of select field mySelect that have a “content” (i.e. not the first, empty one) and that are not currently selected.

——
NB: WebTest’s setSelectField step takes automatically the first option when the xpath evaluation returns more than one result. For other test tools, you may need to select explicitly the first option like this (//select[@name='mySelect']/option[text() and . != ../option[@selected]])[1]

2 Comments

HtmlUnit: Re-synchronize AJAX Calls for Simple, Deterministic Test Automation

July 24, 2007 at 1:24 pm (HtmlUnit, Test Automation)

The upcoming release of HtmlUnit will contain a very large number of fixes and improvements (particularly for the JavaScript support of libraries like Prototype or GWT). An interesting new feature is the experimental AjaxManager that allows to easily control the execution of AJAX calls.

As a user you often don’t even notice that the changes performed in an html page have been performed asynchronously. But this is just because you’re too slow and automated tests (which are very fast) will encounter a problem: they will want to perform some action on the page before this one has been updated.

Current approaches
Automated tests tools propose currently 2 approaches (*) to face this problem and (try to) ensure a deterministic test execution.

  • add an instruction to sleep for a certain amount of time before executing the next instruction (that’s the case for instance of tools like QTP or Squish)
  • add a waitXxxx instruction to wait until some element is available in the page (like for instance in WebAii or Selenium), element that gets created by the AJAX call

The first strategy is very rudimentary: if you don’t wait long enough, you may see your test failing just because the web server is more busy than usual. And if you wait too long, you will end up with test suites taking hours to finish. The second strategy has not this problem but requires the specification of an additional condition (that has to be maintained when the application evolves).

Drawback
In fact none of these strategies has tested that some call has been performed asynchronously (and this test was not required) and both require additional instructions that don’t really belong to the functional test.

Resynchronize (some) AJAX calls to allow simple, deterministic testing
Upcoming HtmlUnit release will allow to control AJAX execution and to turn asynchronous calls back to a synchronous execution mode. The consequence is that the next instruction of the test program is first executed when the AJAX execution is completed. This makes usage of sleep or waitXxx-like instructions needless what simplifies the test conception.

Nevertheless not all AJAX calls need to be turned synchronous. Typically calls that are not triggered by a user interaction like those from Prototype’s Ajax.PeriodicalUpdater don’t be need to be re-synchronized (**). For this purpose HtmlUnit now contains the experimental NicelyResynchronizingAjaxController class which can be used to resynchronize only the AJAX calls made from the main execution thread which are normally the ones triggered by user actions.

Limitations
This feature is very new in HtmlUnit and the experience is missing to make a definitive evaluation. The re-synchronisation worked just fine for the application I had to test and allowed me to eliminate many lines of code. Nevertheless there are still cases where it wouldn’t probably be enough. If I correctly remember, when Prototype’s Ajax.Updater for instance is configured to evaluate the scripts it receives, it uses a setTimeout to perform this task after having treated the rest of the response content. The consequence is that the action of these scripts wouldn’t be intercepted to be executed in the main “flow”. At least in this case the time consuming server round-trip would have been catch, allowing a far smaller “sleep” delay.

Could this technique be used by other test tools than HtmlUnit?

WebTest will of course benefit soon from this technique but this would probably make sense for other web app testing tools as well. This wouldn’t be a problem for those who simulate a browser as they are exactly in the same situation as HtmlUnit and have plain control over AJAX execution. For the tools that work within the browser using JavaScript, I guess that it could be possible to redefine the open method on the XMLHttpRequest objects (as well as on MS’s XMLHttpRequest ActiveX variants) to change the value of the asynchronous parameter. It may become a bit more tricky to determine what is triggered by user interaction to perform a selective resynchronisation. Finally for the tools that drive browsers using their API it will depend of the features offered by this API but this is a domain that I don’t know.

——
(*) at least this is what I could find in the documentation of the numerous tools I’ve looked at to write this post (it’s amazing how many – useless? – web testing tools you can find on the net!)
(**) in fact it probably doesn’t matter if resynchronized or not as the execution already runs in a separate thread

13 Comments

WebTest’s New Motto: The Most Effective Way To Test Your Web App!

July 23, 2007 at 9:47 am (Test Automation, WebTest)

We’ve change WebTest’s motto (previously it was “Free open source tool for automated testing of web applications”) to better express what WebTest’s strengths are.

A lot of tools for automated testing of web app are available and have different features that may appear appealing in a first time. WebTest shines in what I call “professional test automation” because it allows to simply write test scripts that are reliable, fast, easy to run, cheap to maintain, helpful to quickly find errors and it scales fine and is easy to extend. Lisa Crispin, author of Testing Extreme Programming, described this perfectly: “the ROI on WebTest is many orders of magnitude higher than any other tool I’ve used”.

This new motto is only the first step. Keep tuned!

Leave a Comment

iX Bessere Software! Conference 2007: WebTest Tutorial & Session accepted

July 20, 2007 at 8:36 am (Test Automation, WebTest)

I will present a full day tutorial as well as a session about efficient test automation of web applications in general and WebTest in particular together with Dierk König at the next iX Bessere Software! conference, 19-21 November, in Düsseldorf (Germany).

We’ve already presented a WebTest tutorial at the previous edition of this conference last year in Frankfurt. This was really nice and I think that our tutorial was the most visited one. I hope that this year’s edition will be as good as the last one.

Leave a Comment

FrOSCon 2007: Groovy session accepted

July 20, 2007 at 8:09 am (Groovy)

I will present “Dynamische Programmierung mit Groovy” at the next FrOSCon conference in Sankt Augustin (Germany) August 25th. I’m a bit disappointed that my WebTest proposal has not been accepted too but I understand that the available time is limited.

This will be quite interesting to go to a conference nearly “at home”.

2 Comments

Only Selenium can test it? What? Surely not this simple page

July 10, 2007 at 8:47 pm (Misc, Test Automation)

When you look at a conference program, the session abstracts are naturally a kind of publicity to motivate people to attend and should be considered as such. Nervertheless an abstract shouldn’t exagerate too much to stay credible.

“… including applications that only Selenium can test”. Where?

In Neal Ford’s numerous sessions about Selenium, you will see again and again that he will present “testing Ajax applications (including applications that only Selenium can test)” (see for instance in
Advanced Selenium (No Fluff Just Stuff)). When you write such a provocative statement, you have to prove it. I’ve attended one of his presentations (it was really poor, see previous post here) but I haven’t seen anything that would cause trouble to all existing web testing tools except to Selenium.

But very simple pages can’t be tested with Selenium

On the other side a page as simple as this example

<html>
<head>
<script>
document.getElementById = function(_id)
{
return {id: _id, nodeType: 3, data: 'he he, you failed!'};
}
</script>
</head>
<body id='bodyId'>
this is a very simple page
</body>
</html>

will currently cause enorm problems to Selenium that won’t even be able to execute correctly a simple
verifyText id=bodyId this is a very simple page

The reason? Selenium uses the DOM functions to work on the html document and therefore relies on them.

Ok, this point isn’t a very big limitation as changing the functionality of the DOM functions is not a common need. In fact this post was just to say that I wait to see a web application that “only Selenium can test” but that even a very small page can cause problems to Selenium whereas it is handled correctly by most of the other web testing tools.

Leave a Comment