WebTest as universal DSL for automated web testing in Groovy thanks to its Ant roots?

What for a long title!

In the discussion in Grails-user mailing list following my previous post “WebTest vs Selenium“, Marc Palmer and James Page requested the creation of a kind of meta DSL in Groovy for automated functional tests of web applications.

The idea was to provide a DSL allowing to write functional tests in a tool agnostic way and to run them with WebTest as often as needed because it is fast and for instance once each night using Selenium because it uses a real browser but is quite slow.

I’m still not fully convinced on the utility of such a feature because WebTest is so good 😉 but I think that there is no need for a new DSL: it already exists! The AntBuilder allows WebTest to have a really nice syntax in Groovy and this could be simply reused for other “target tools” thanks to Ant.

Ant theory

When you write something like this (the examples in this post use Groovy AntBuilder but the same would apply for “pure” Ant in XML format):

ant.webtest(name: "a simple test)
{
  invoke "http://webtest.canoo.com"
  verifyTitle "WebTest website"
  clickLink "Manual"
  verifyXPath(xpath: "count(id('navigation-top')//li)", text: "5",
      description: "check the number of top menu elements")
  ...
}

this looks like WebTest but this is not WebTest as long as you haven’t configured Ant with for instance something like

ant.project.addTaskDefinition("webtest", com.canoo.webtest.ant.WebTestTask)
ant.project.addTaskDefinition("invoke", com.canoo.webtest.steps.request.Invoke)
...

This has 2 consequences:
– it is possible to configure other tasks than WebTest’s ones for “WebTest steps”
– it is possible to inspect the “parsed” tasks tree

This means that it wouldn’t be too difficult to run a “WebTest” test with an other automated test tool like for instance Selenium (as long as the tests don’t use any WebTest feature for which no Selenium equivalent exists). Let’s see how this could be done.

First solution: redefine “WebTest steps”

The first approach consists in redefining the “WebTest steps” to provide an alternative implementation for each WebTest step before executing the test. This could look like following for verifyTitle:

import org.apache.tools.ant.*class SeleniumVerifyTitle extends Task
{
  String text
  void execute()
  {
    if (text != selenium.title)
      throw new BuildException("Wrong title: expected $text, got ${selenium.title}")
  }
  def getSelenium()
  {
    project.references.'selenium' // assuming that test start placed it there
  }
}
ant.project.taskDefinitions["verifyTitle"] = SeleniumVerifyTitle

verifyTitle is quite simple, for other steps it would be more tricky or even impossible (like the pdf or email steps) to write a Selenium equivalent.

Generate script from Ant tree

The second approach consist in the generation of a script from the Ant structure. This has the “advantage” that the
generated script doesn’t necessarily have to be run on the Java Virtual Machine.

class WebTest2SeleniumRubyConverter extends WebTest
{
  File targetFile // additional attribute to Ant task: the file to write in
  private converters = [
    'invoke': { "open \"${it.attributeMap.url}\"" },
    'verifyTitle': { "assert_equal \"${it.attributeMap.text}\", @selenium.get_title" },
    'clickLink': { "@selenium.click \"link=${it.attributeMap.label}\"" },
    'verifyXPath': { "# ?? I haven't found an example" },
    'pdfVerifyTitle': { "# skipped because not supported: pdfVerifyTitle ${it.attributeMap.title}" },
    ...
  ]
  def execute()
  {
    def rubySteps = runtimeConfigurableWrapper.children.collect { converters[it.elementTag](it) }
    def scriptTemplate = """
require 'test/unit'
require 'selenium'
class ExampleTest < Test::Unit::TestCase
include SeleniumHelper

def setup
@selenium = Selenium::SeleniumDriver.new("localhost", 4444, "*firefox", "http://localhost", 10000);
@selenium.start
end

def teardown
@selenium.stop
end

def test_something
<% steps.each { %>
$it
<% }%>
end

end"""

    def engine = new groovy.text.SimpleTemplateEngine()
    def template = engine.createTemplate(scriptTemplate)
    targetFile.withWriter {
      it << template.make([steps: rubySteps])
    }
  }
}
ant.project.taskDefinitions["webTest"] = WebTest2SeleniumRubyConverter

NB: I don’t have any experience in Ruby and have written the example code above only by looking at the samples on Selenium’s website.

Naturally this is only an example and it is a bit more complicated to do it correctly (handle different combinations of task attributes, special characters, Ant macros, …) but surely not so much.

Conclusion: not very complicated but does it makes sense?

These two examples shows that it wouldn’t be complicated to “convert” simple WebTest tests to allow them to be executed with an other tool like Selenium (but other tools like for instance WebDriver could be a target too) as long as they use the subset of WebTest features that the target tool accepts. Of course WebTest features like its particularly rich reporting wouldn’t be available either.

Personally I’m not really convinced of the utility of such a “generic DSL” due to WebTest’s excellent quality and unless a new client (you? ;-)) really wants this feature I don’t plan to work on it but I’m ready to provide technical assistance if someone wants to do the job.