XPath Power (2): Detect Lack of Experience

A lot of people feel qualified to talk about test automation of web applications, probably because the principles are really easy: tests have “only” to do what a “normal” user would do. In the practice many of these persons don’t have real test automation experience and you shouldn’t trust their statement directly.

The problem is that it’s difficult to determine if a person has a real experience and therefore if the proposed solutions have proved their worth or not. Declarations from a person that hasn’t written tests receiving at least a few hundreds of responses from the web server are questionable. Exactly the same is valid for people talking from recording tools without to give any advice on what can be expected from the recorded scripts.

An other way to detect lack of experience is to look at the XPath expressions used. XPath support is indispensable in test automation of web application but depending how the xpath expressions are written, they may be far too “strong” and fail on the first unrelated change in the tested html page.

A few “bad” XPath examples
Here are a few examples of “bad” xpath available in blogs, conference slides or even tool documentation:

/html/body/table[2]/tbody/tr/td[2]/table[2]/tbody/tr/td/div/ul[1]/li[4]
from http://agiletesting.blogspot.com/2006/01/useful-tools-for-writing-selenium.html

//body[1]/table[1]/tbody[1]/tr[1]/td[1]/img[1]
from http://www.artoftest.com/Resources/WebAii/Documentation/topicsindex.aspx?topic=elementid

//html[1]/body[1]/div[1]/div[1]/h3[1]
from http://artoftestinc.blogspot.com/2007/01/ajax-testing-and-waiting-for-page.html

//html/body/div[@id='outerDiv']/div[@id='innerDiv']/div
[@id='pinDialog']/table/tbody/tr/td

from http://www.nealford.com/downloads/conferences/canonical/Neal_Ford-Advanced_Selenium-handouts.pdf

/html/body/table/tr[2]/td/table/tr[1]/td/a
from http://wiki.rubyonrails.org/rails/pages/HowtoFunctionalTest

10 Comments

  1. August 2, 2007 at 5:14 pm

    Interesting way to judge a person’s testing experience….by looking at the XPath expressions they use. FWIW, in the agiletesting post you mention, I was using a tool that came up with that XPath expression. I’m sure your vast XPath experience tells you that it’s notoriously hard to come up with the simplest XPath expression that works for a particular HTML element, so a tool that does 80% of the job helps.

    I’m not going to comment on your practice of bad-mouthing your peers.

    Grig

  2. Neil said,

    August 3, 2007 at 1:47 am

    I don’t understand how you are relating lacking of XPath experience with lack of test experience. I’ve been in software testing for 10 years and worked on many automation projects at Fortune 500 companies and I can tell you there is nothing that relates experience with XPATH and experience with software testing…

    Actually, this blog post simply indicates to me and to anyone that reads it your lack of experience with real software testing and automation projects since we try to build automation infrastructure that are simple and powerful enough to help us avoid XPATHs/REGEXs usage since they are hard to craft and maintain.

    PS: Having a blog about software testing to promote your tools does not make you an “Expert” in software testing. Sorry…

  3. Marc Guillemot said,

    August 3, 2007 at 9:04 am

    Hi Neil,

    if you have such an experience in test automation of web applications, you’ve surely seen that XPath support is indispensable. I agree with you that XPath and regex should be avoided when more stable identification possibilities are available but unless the application is very very carefully designed for testability, you simply can’t avoid them totally (ant there are sometimes better than any other solution).

    Once you use XPath, you have to write your expressions in a way where they don’t fail due to changes unrelated to what you want to test. This is exactly the point of this post: many different xpath expressions allow to identify the same element(s) in the page, some of these expressions are stable and will be very easy to maintain whereas other will fail on the first unrelated change of the page structure (for instance minimal design change). The examples collected here belong to this second category.

  4. Marc Guillemot said,

    August 3, 2007 at 9:15 am

    Grig,

    when you simply use a tool to capture an XPath expression and don’t “refactor” the captured expression, you just follow a record/playback strategy and “Record/Playback is the least cost-effective method of automating test cases” (not from me, but from Keith Zambelich
    Co-author of the WinRunner ATS Toolkit). It’s a strategy, but not the one that I recommend.

  5. Ron said,

    August 3, 2007 at 11:32 am

    > I’m not going to comment on your practice of bad-mouthing your peers.

    ???
    Should he congratulate you on this?

    Grig, close your blog if you don’t accept any review!

  6. Phil Fearon said,

    August 13, 2007 at 11:19 am

    I’m keeping out of the debate of what actually makes a ‘good’ XPath expression. But, if anyone wants to see (and comment on) a new approach in a tool used to generate a number of related/complex XPath expressions and subsequently store and modify these, please have a look at my SketchPath project (freeware) at http://www.sketchpath.com.

    A tool can’t prevent poorly designed XPath expressions but it should make it easier to ‘refactor’ expressions to make these more robust for test purposes.

    Thanks

    Phil Fearon

  7. Dierk König said,

    August 15, 2007 at 8:38 am

    I support Marc’s bold statement in that using absolute XPath expression for functional testing is a sure sign of lacking experience in 99% of all cases (they are only appropriate in a few pathological cases). I even consider it a typical newbee error.

    Disregarding the need for having maintainable XPath expressions may only show a lack of _XPath_ knowledge but neglecting the need for separating guaranteed application features from accidental ones (like you can do with proper XPath expressions) is a sure sign of lacking general knowledge and experience about testing in general and automated functional testing in particular.

    When it comes to testing, everybody seems to have an opinion and is allowed to speak. I personally listen only to those that can prove to have written a few thousand tests themselves and maintained them over multiple releases of the application under test.

    Dierk König

  8. xpath monkey said,

    October 15, 2007 at 9:33 pm

    As far as I can tell, they all look like valid XPath and in the grand scheme of functional testing, they may very well be “good enough”. Instead of just labeling them bad, why don’t you suggest some “better” expressions.

  9. Marc Guillemot said,

    October 16, 2007 at 7:10 am

    Hi xpath monkey,

    it seems that you haven’t correctly read my post. All these xpath expressions are fully valid and they have probably tested what they’ve been written for. The point here is that such XPath expressions test too much things and therefore will fail due to unrelated changes in the page. Just imagine which ones will still work if an additional div or table is added on the top of the page containing for instance a new banner.

    The html source files and the test motivation would be needed to provide “good” alternative XPath expressions here. The test motivation is really important because depending of it, the “best” XPath expression will differ (and this is exactly the reason why a recorder can only produce good drafts but no good test script).

    I agree that explanations on how to write good XPath expressions would be good. We already plan a screencast on this subject.

  10. Aaron said,

    January 3, 2010 at 10:37 am

    Actually web applications CAN be tested without using XPaths. Have a look at Sahi (http://sahi.co.in) which does not use XPaths.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: