Java Development News:

Using MCP for Automated Testing of Ajax Applications

By Ed Burns

01 Mar 2007 | TheServerSide.com

Introduction

Screencast: Part One - Click Here
Screencast: Part Two - Click Here

Back in late 2001, at the start of development of JSF 1.0, the Sun implementation team decided to use Test Driven Development for the entire implementation. We surveyed the state of the art of dollar cost free automated testing tools and chose JUnit, Cactus, and HtmlUnit to help us do TDD. We quickly ran up against the limitations of the JavaScript support in HtmlUnit and had to employ some ugly workarounds to get the job done. At the time, we thought, "I wish there was a better way," but since we were really busy with the implementation we just continued with our workaround. We're still using those workarounds today! There still isn't a better way... Until now.

The Mozilla Webclient Project started in 1999 as a part of the Sun/AOL/Netscape alliance. The ambitious misson statement of the project is:

The webclient project aims to provide the premier browser-neutral Java API that enables generic web browsing capability. This capability includes, but is not limited to: web content rendering, navigation, a history mechanism, and progress notification. The actual capabilities implemented depend on the underlying browser implementation. 

At one point, webclient worked on Windows, Mac OS 9, Solaris 8, and GNU/Linux. We even had the ability to wrap the Internet Explorer ActiveX control, or the ICEbrowser in addition to Mozilla. Layoffs happened, Netscape was closed down, and the project went into hibernation. I have not been able to give the project the attention it deserved until Ajax happened and the pain of not having a nice way to do detailed automated testing of Ajax web applications became too great to bear.

Problem Statement

I decided to dust off webclient, on Windows at least, and see if it could be made to allow automated testing of Ajax applications. Specifically I wanted a solution that:

  • Could be used from JUnit or TestNG.

  • Had a very small and simple API.

  • Allowed writing tests that make assertions about the response to Ajax requests.

  • Allowed calling through to the full Webclient API for more detailed page inspection.

A Solution

I'm calling the result the Mozilla Control Program in a reference to one of my favorite movies of all time, Tron. This article accompanies a screencast showing the automated testing of a simple Ajax application. The application is variant of the JSF Cardemo that has been ajaxified using the Sun Web Developer Pack, though it could just as well be any old web application. The sample app is running live at http://webdev1.sun.com/jsf-ajax-cardemo/faces/chooseLocale.jsp.

The remainder of this article takes you through the automated test shown in the screencast. This testcase is completely aware of the structure and design of the application being tested. In that sense, it is a white box test. In particular, values of HTML "id" and "name" attribute values are assumed, and knowledge of which kinds of user interactions will trigger Ajax transactions is also assumed by the testcase. I feel these are valid assumptions for a testcase author to make.

Listing 1: The setUp method
  1.     private MCP mcp = null;   
  2.    
  3.     public CarDemoTest(String testName) {
  4.         super(testName);
  5.     }
  6.  
  7.     public void setUp() {
  8.         super.setUp();
  9.  
  10.         mcp = new MCP();
  11.         try {
  12.             mcp.setAppData(getBrowserBinDir());
  13.         }
  14.         catch (Exception e) {
  15.             fail();
  16.         }
  17.        
  18.     }

The code listings are excerpted from a JUnit test included in the binary distribution of Webclient. Listing 1 shows the constructor, the setUp method, and a private instance variable of the test, the MCP instance. The setUp method is a great place to call mcp.setAppData(). This important method tells Webclient where to find the binary of the native browser (in this case, Xulrunner). We want the testcase to fail if it webclient cannot be properly initialized, as shown on line 15.

Listing 2 shows the actual testCarDemo() unit test.

Listing 2: The testCardDemo method
  1. public void testCardemo() throws Exception {
  2.     mcp.getRealizedVisibleBrowserWindow();
  3.     final BitSet bitSet = new BitSet();
  4.     AjaxListener listener = new AjaxListener() {
  5.       public void endAjax(Map eventMap) {
  6.           bitSet.flip(TestFeature.RECEIVED_END_AJAX_EVENT.ordinal());
  7.           if (null != eventMap) {
  8.               bitSet.flip(TestFeature.HAS_MAP.ordinal());
  9.           }
  10.           // Make some assertions about the response text
  11.           String responseText = (String) eventMap.get("responseText");
  12.           if (null != responseText) {
  13.               if (-1 != responseText.indexOf("<partial-response>") &&
  14.                   -1 != responseText.indexOf("</partial-response>")) {
  15.                 bitSet.flip(TestFeature.HAS_VALID_RESPONSE_TEXT.ordinal());
  16.               }
  17.           }
  18.           Document responseXML = (Document)
  19.               eventMap.get("responseXML");
  20.           Element rootElement = null, element = null;
  21.           Node node = null;
  22.           String tagName = null;
  23.           try {
  24.               rootElement = responseXML.getDocumentElement();
  25.               tagName = rootElement.getTagName();
  26.               if (tagName.equals("partial-response")) {
  27.                   element = (Element) rootElement.getFirstChild();
  28.                   tagName = element.getTagName();
  29.                   if (tagName.equals("components")) {
  30.                       element = (Element) rootElement.getLastChild();
  31.                       tagName = element.getTagName();
  32.                       if (tagName.equals("state")) {
  33.                           bitSet.flip(TestFeature.
  34.                                       HAS_VALID_RESPONSE_XML.ordinal());
  35.                       }
  36.                   }
  37.               }
  38.           }
  39.           catch (Throwable t) {
  40.               
  41.           }
  42.           
  43.           String readyState = (String) eventMap.get("readyState");
  44.           bitSet.set(TestFeature.HAS_VALID_READYSTATE.ordinal(),
  45.                      null != readyState && readyState.equals("4"));
  46.           bitSet.flip(TestFeature.STOP_WAITING.ordinal());
  47.           
  48.       }
  49.     };
  50.     mcp.addAjaxListener(listener);
  51.    
  52.     // Load the main page of the app
  53.     mcp.blockingLoad("http://javaserver.org/jsf-ajax-cardemo/faces/chooseLocale.jsp");
  54.     // Choose the "German" language button
  55.     mcp.blockingClickElement("Germany");
  56.     // Choose the roadster
  57.     mcp.blockingClickElement("roadsterButton");
  58.     // Sample the Basis-Preis and Ihr Preis before the ajax transaction
  59.     Element pricePanel = mcp.findElement("zone1");
  60.     assertNotNull(pricePanel);
  61.     String pricePanelText = pricePanel.getTextContent();
  62.    
  63.     assertNotNull(pricePanelText);
  64.     assertTrue(pricePanelText.matches("(?s).*Basis-Preiss*15700.*"));
  65.     assertTrue(pricePanelText.matches("(?s).*Ihr Preiss*15700.*"));
  66.    
  67.     // Choose the "Tempomat" checkbox
  68.     bitSet.clear();
  69.     mcp.clickElement("cruiseControlCheckbox");
  70.    
  71.     while (!bitSet.get(TestFeature.STOP_WAITING.ordinal())) {
  72.         Thread.currentThread().sleep(5000);
  73.     }
  74.    
  75.     // assert that the ajax transaction succeeded
  76.     assertTrue(bitSet.get(TestFeature.RECEIVED_END_AJAX_EVENT.ordinal()));
  77.     assertTrue(bitSet.get(TestFeature.HAS_MAP.ordinal()));
  78.     assertTrue(bitSet.get(TestFeature.HAS_VALID_RESPONSE_TEXT.ordinal()));
  79.     assertTrue(bitSet.get(TestFeature.HAS_VALID_RESPONSE_XML.ordinal()));
  80.     assertTrue(bitSet.get(TestFeature.HAS_VALID_READYSTATE.ordinal()));
  81.     bitSet.clear();
  82.    
  83.     // Sample the Basis-Preis and Ihr-Preis after the ajax transaction
  84.     pricePanel = mcp.findElement("zone1");
  85.     assertNotNull(pricePanel);
  86.     pricePanelText = pricePanel.getTextContent();
  87.    
  88.     assertNotNull(pricePanelText);
  89.     assertTrue(pricePanelText.matches("(?s).*Basis-Preiss*15700.*"));
  90.     assertTrue(pricePanelText.matches("(?s).*Ihr Preiss*16600.*"));
  91.    
  92.     mcp.deleteBrowserControl();
  93. }

Line 2 causes the actual browser window to be created and shown. Because MCP uses java.awt.Robot to simulate mouse and keyboard input, we do need an actual, visible browser window. This usage model also implies that you cannot use the computer while the automated test is running. Line 3 uses creates a final BitSet instance. This is a useful approach to handle browser interactions that require a callback from the browser to the user application. In this case, we have an inner class that is an instance of AjaxListener, shown on lines 4 - 49 of Listing 2.

AjaxListener has three methods that are intended to be overridden by the test. endAjax, errorAjax, and startAjax. endAjax is called when the browser successfully completes an Ajax transaction (Ajax Response code 4). errorAjax is called when an Ajax transaction terminates abnormally. startAjax is called when code within the browser calls send on an XmlHttpRequest instance, but before the browser actually makes the network connection. All three of the methods are passed a Map that contains information about the request. The contents of the map differ depending on which of the three methods is overridden.

For startAjax

The map will contain the following keys and values:

headers
a java.util.Map of all the request headers.
method
the request method for this event.
readyState
a String of the numerical value of the XMLHTTPRequest readyState
For endAjax and errorAjax

The map will contain the following keys and values:

method
the request method for this event.
responseXML
a org.w3c.dom.Document instance of the response XML.
responseText
a String instance of the response Text.
status
the response status string from the server, such as "200 OK".
headers
a java.util.Map of all the response headers.
method
the request method for this event.
readyState
a String of the numerical value of the XMLHTTPRequest readyState

For all notification methods, the map will contain an entry under the key "URI" without the quotes. This will be the fully qualified URI for the event.

On line 6, we use a testcase specific enum, TestFeature that defines the bit assignments within the BitSet for different assertions we are testing. Line 6 flips the bit that says, "the ajax transaction did indeed end successfully". Line 8 flips the bit that says, "the MCP did give me a Map for this transaction". Line 15 says, "the responseText of the XmlHttpRequest is as expected. Lines 24 - 34 use the W3C DOM API to inspect the response XML and verify it is as expected. Line 44 flips the bit that says, "the readyState value is as expected". Line 46 flips the bit that says, "I'm done inspecting the ajax response". This will be important later.

We are out from the definition of the inner class now, on line 50, and we simply add the ajaxListener instance.

Line 53 causes the mcp to load the given URL, and block until the load completes. Line 55 causes the mcp to click the button with the id or name "Germany", and wait for the load to complete. Note that the HTML spec requires "id" attribute values to be unique within a page, but "name" attribute values need not be. Therefore, MCP uses a heuristic if no element is found with the given id: it simply finds the first element with a matching "name" attribute. Line 57 chooses the "Roadster" car.

On lines 59 - 65, we sample the page to get the price information for the Roadster. We use java regular expressions to assert that the base price and "your price" are both 15700. Line 68 has us clearing the bitset ahead of clicking the checkbox that will cause the ajax transaction. In this case, clicking the checkbox indicates the user wants to dd "cruise control" to their car. Line 69 does the actual click, and note that this time we are not using the blocking click method. Instead, we are doing the blocking ourselves, manually. On lines 71 - 73 we are in an infinite loop that sleeps for 5 seconds between checking if the "I'm done inspecting the ajax response" bit has been flipped.

When it is flipped, we continue and do actual JUnit assertions on the tests conducted in our AjaxListener inner class.

Finally, on lines 84 - 90 we inspect the pricing to verify that the price went up after choosing the cruise control.

Conclusion

The simple API exposed by MCP is sufficient for testing a wide range of ajax applications. More complex tests may use the Webclient API directly.