Canoo has released WebTest 2.0, an open source tool for automated testing of web applications.

New features in WebTest 2.0 include:

• Comprehensive JavaScript support due to the integration of HtmlUnit 1.8
• Improved and enlarged set of test steps, featuring core steps, extensions, filters, PDF files, Excel, email, applets, and technical details such as HTTP headers and cookies
• Support of both the traditional specification of test cases using Ant / XML and scripting test cases in Groovy

Other Resources:

• Download
• Installation Guide
• Mailing list
• Webtest wiki

How does it compare with Selenium, specially regarding Javascript / AJAX / Browser compatibility, plus ease of use? I read WebTest uses HTMLUnit in the background for javascript support, but I have had problems with its javascript support in the past, has it improved lately?

TIA

How does it compare with Selenium, specially regarding Javascript / AJAX / Browser compatibility, plus ease of use? I read WebTest uses HTMLUnit in the background for javascript support, but I have had problems with its javascript support in the past, has it improved lately?TIA

I doubt WebTest will ever be as good as Selenium, mainly because it's hard to reproduce a browser environment w/o a browser. That being said, I've used WebTest for a few years and its JavaScript support is much in recent months. I filed a few bugs recently for Prototype and Script.aculo.us and HtmlUnit now works well with both - at least in my applications.

I doubt WebTest will ever be as good as Selenium, mainly because it's hard to reproduce a browser environment w/o a browser.

I've tried using Selenium myself, although not too successful testing it with JUnit based test cases and I couldn't find any good documentation regarding that integration. I've blogged about my experience on this (didn't put it here since its a bit off topic)

We tried out both - HTMLUnit and also WebTest. Because we had complex requirements - especially supporting JavaScript - for us it took to much time to create tests that are usable over the whole iterative developement cycle.
Now we use the capture & replay tool JStudio SiteWalker that provides browser based automation, which supports JavaScript in excellent way.
Because recording is element based the tool also supports recurrent tests on each iteration.

Selenium follows the capture/replay stile of test automation, WebTest follows the test modelling paradigm. Both have their value depending on what you try to achieve.

WebTest shines when you want define your tests before you even have the production code, i.e. in a test-driven scenario. With WebTest you specify what your code is expected to do. It scales even when your test suites raises up to ten-thousands of tests and your (regression) tests run automated and unsupervised with your build process.

Selenium follows the capture/replay stile of test automation, WebTest follows the test modelling paradigm.

If Selenium does work as it shows in the examples, it can also implement a test modelling paradigm. I would like to see Selenium work, but at the moment all I can do is revert back to HttpUnit tests.

but at the moment all I can do is revert back to HttpUnit tests.

When 'reverting' have a look at HtmlUnit as a replacement for HttpUnit. It's provides much more functionality. In fact, it *is* a browser, albeit one that is not visible on the screen.

When 'reverting' have a look at HtmlUnit as a replacement for HttpUnit. It's provides much more functionality. In fact, it *is* a browser, albeit one that is not visible on the screen.

I'll have to try it out thanks! FWIW, HttpUnit already simulates a browser including JavaScript though not visible on screen, although for every new framework I use I have to create a new set of "Finders" since the ones that come with HttpUnit are not generic enough.

Selenium does not work if your website to test comtains javascipt that removes frames (e.g. for phishing prevention)
result: you website to test removes the selenium frames ;-(

I'm currently looking in to picking an automated web testing framework. I have some experience with WebTest in the past, is the general opinion that WebTest and Selenium are the leading frameworks? Any other to consider?

Cheers,

/ Christian

Is the general opinion that WebTest and Selenium are the leading frameworks? Any other to consider?

That's basically my impression. Of course since WebTest uses HtmlUnit under the covers, you can use HtmlUnit directly if you'd rather write Java instead of XML.

A lot of people use HttpUnit, but that's mostly because of inertia. HttpUnit is the grand-daddy of 'em all, but it has quite a few bugs, primitive JS support, and you wouldn't be able to get a patch applied to the official source to save your life.

There's also a spattering of packages that provide interfaces on top of HttpUnit, but they're not very well-maintained either, so you end up tying yourself to two rotting codebases.

So yes, there are a couple dozen frameworks out there to choose from. However, it's my impression that if people spent five minutes looking into the viability of each project (when was the last release? how often do they release? how many patches have been accepted lately? how many bug reports have been closed lately?), that number could extremely quickly be whittled down to just three or four. And yes, Canoo WebTest and Selenium are among them ;-)

you can use HtmlUnit directly if you'd rather write Java instead of XML.

Another option that I really fell in love with is writing WebTests as Groovy scripts. It gives you the full power of Canoo WebTest with the flexibility and expressiveness of a dynamic programming language.
Find howtos under http://webtest.canoo.com/webtest/manual/groovyTesting.html and see how the world-rocking Grails project uses that feature: http://grails.codehaus.org/Functional+Testing .

You may also want to take a look at JSUnit. The issue I had with this tool was in automtating it - such as when a dialog box is popped up and some input needs to be automatically provided.

http://www.edwardh.com/jsunit/

I'm currently looking in to picking an automated web testing framework. I have some experience with WebTest in the past, is the general opinion that WebTest and Selenium are the leading frameworks? Any other to consider?Cheers,/ Christian

I have used Sahi on a couple of projects (one with frames) and it worked well with my continuous integration setup.
(full disclosure: I work for ThoughtWorks and Sahi is developed by Narayan Raman of ThoughtWorks).

From http://sahi.sourceforge.net/

Sahi is an automation and testing tool for web applications, with the facility to record and playback scripts.
Developed in java and javascript, this tool uses simple javascript to execute events on the browser.
Features include, in-browser controls, text based scripts, ant support for playback of suites of tests,
and multi threaded playback. It supports HTTP and HTTPS.

Sahi runs as a proxy server and the browser needs to use the sahi server as its proxy.
Sahi then injects javascript so that it can access elements in the webpage.
This makes the tool independant of the website/ web application.

Take a look at Open Source Testing
The site improved recently, there is even a RSS feed.

Bruno

Selenium does not work if your website to test comtains javascipt that removes frames (e.g. for phishing prevention) result: you website to test removes the selenium frames ;-(

Well, at least you can use Selenium for *one* of your tests ;-)

The javascript support is quite good. Webtest uses htmlunit snapshots to provide fixes for js problems as soon as possible. In the past monthes, the main (only one?) reason for unanswered js problems was insuffisant information provided by users (like Matt ;-)).

An other feature of htmlunit that can be used in conjunction with webtest is its ability to simulate the behavior of a special browser: for instance js expressions like document.all... will work when simulating IE but not when simulating Firefox.

Does Canoo support AJAX? How does it compare with Selenium?

Thanks in advance,

Tracey

We started to use Canoo WebTest for functional web testing, but moved away from it to JWebUnit because: