Sequence Diagram for Retrieving And Displaying Information From the Network

First, the user enters an address. The User Input Form object creates the Wait object, which displays a 'Waiting for Data' message to the user. The Wait object starts a new thread, the Logic object, that is responsible for retrieving and processing the data, and passes itself to the Logic object as a reference. The Wait object is pushed onto the display stack maintained by the Controller object, by the User Input Form, which then causes the Wait object to be drawn on the screen. Meanwhile, in the Logic object, network access is deferred to yet another thread, the Network Worker. The Logic object communicates with the Network Worker through one or more Transfer objects, which are simply beans with synchronized get and set methods to which both the Logic and Network Worker objects have references. The Network Worker connects to the network, and receives an XML input stream containing information regarding the traffic jams around the address the user entered, which it puts in the Transfer object (see Example 2, “Network Connection Code” for code). The Logic object has been waiting for this input stream, and processes it by parsing the XML into domain objects (see Example 3, “XML Parsing Code” for code). These objects are then passed back to the Wait state, which then creates a new Form object, passing the retrieved data to the new Form in its constructor. Then the Wait object passes control to the new Form by pushing it onto the stack. This then causes the Form to be drawn on the device, displaying whatever data has been retrieved from the network and perhaps prompting the user for more data.

Example 2. Network Connection Code

1   hc = (HttpConnection) Connector.open(urlst);
2   if (null != sessionCookie) {
3      hc.setRequestProperty("cookie", sessionCookie);
4   }
5   if (post) {
6      hc.setRequestMethod(HttpConnection.POST);
7      hc.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
8      hc.setRequestProperty("Content-Length", Integer.toString(message.length()));
9      out=hc.openOutputStream();
10     if (null != out){
11        out.write(message.getBytes());
12     }else{
13        Log.logError("couldn't write post data to output stream");
14     }
15  } else {
16     hc.setRequestMethod(HttpConnection.GET);
17  }
18  hc.setRequestProperty("Connection", "close");
19  is = hc.openInputStream();
20  t1.setPayload(is);

This is a section of the Network Worker object's run() method. It opens up a network connection on line 1, attaches a cookie to the request on lines 2-4, sets appropriate properties depending on whether the HTTP request is a POST or a GET on lines 5-18, and retrieves the input stream containing the server's response on line 19. The input stream is then passed to a transfer object on line 20.

Example 3. XML Parsing Code

1   while ((event = parser.read()).getType() != Xml.END_DOCUMENT) {
2      int type = event.getType();
3      if ( (type == Xml.START_TAG) && (event.getName().equals("INTERSECTION"))) {
4          String name = event.getName();
5          IdDescriptionPair intersection = new IdDescriptionPair();
6          intersection.id = event.getValue("code");
7          intersection.desc = event.getValue("description");
8          returnVector.addElement(intersection);
9      }
10  }

XML input is parsed by checking for start tags on line 3. The scenario here is that the traffic service application does not have enough information based on the address entered, so a set of suggested intersections is returned. Intersection data, as represented by the INTERSECTION tag, has two attributes, code and description. The values of those attributes are placed into a domain value object on lines 6 and 7. There will likely be more than one, so a Vector is used on line 8 to hold all of them for iteration and display on the screen.

Networking code has some phone specific peculiarities. In particular, the header handling has been different between phones. On some Motorola phones, when there is more than one cookie, the later ones are not seen. In addition, handling cancel events means communicating back to the network accessing thread; this communication can only happen at certain times (not when the thread is blocking on network access). Also, most phones ask users if an application can access the network, since doing so costs the user money. If the user says no, a security exception is thrown, and must be handled by the application. These issues, as well as the deadlock issue mentioned above, make it clear that testing on as many real phones and emulators as possible helps smoke out non standard behavior that will adversely affect the user experience.

Cell phone applications exhibit decent network performance. The user experience has depended greatly upon the cell phone network--some carriers simply have higher data transfer rates than others. This is not a rigorous analysis, but while demoing applications, I have never seen it take more than ten seconds or so to download ten KB of information from the network.

Security

Mobile phones are lost, stolen or misplaced more easily than laptops. Anything that needs to be secure should not be on a cell phone; if data needs to be secure and must reside on the cell phone, look at encrypting it. In the case of the traffic congestion client, sensitive data, including authentication tokens to remote databases, could be kept on the server.

Permanent storage

Cell phones have static memory which J2ME exposes via the recordstore API, using classes in the javax.microedition.rms package. Recordstores are an array of byte arrays. The store is easier to work with if treated as a persistent hashtable.^[18]

Images that are downloaded from the network and then displayed can be cached in the recordstore for a performance increase. However, doing so requires storing the raw byte[] array before displaying the image. Such a a byte[] array can be easily turned into an javax.microedition.lcdui.Image object, displayable on a screen. However, there is no way to turn an Image object into a byte[] array after it has been created.

The Server Side

The server that the J2ME application will communicate with need not be complex. A typical open source J2EE application will suffice: Apache, fronting a tomcat servlet engine, with struts as the MVC framework, and using mysql as a datastore. I am only going to address a few points where the nature of the J2ME client influences the intermediary server.

The struts framework can be used in two separate scenarios: to build a typical web application, allowing users to sign up and pay for the traffic congestion service as well as other account management activities best done with a browser, and also to generate XML for the cell phone application. In the second case, not all the features of struts need to be used--client side validation and action forms are both superfluous, for example. Since all that is communicated between the the cell phone and the server application are sets of XML messages, the main benefit of the action form, validation and error messages, is not useful. When transmitting binary data such as images, a simple custom servlet streamed the data as a java.io.ByteArrayOutputStream over HTTP.^[19]

JSPs can be used to generate XML. Since WAP supports a subset of XHTML, adding another view layer to the server application is easy. (This took about 6 hours for our application and not a single Java class was touched.) However, limitations of WAP mean advanced features are not available to the WAP browser client--in particular, information collection for a given request can (and should) be spread over multiple screens with the rich client, while it must be gathered on one screen with the WAP browser.

In order to generate and manipulate images, JAI was used. It is poorly documented; the main form of documentation, javadoc, does not provide much help for beginners, and the tutorial is a few versions behind. However, the mailing list is very useful. The JAI library can be used to transform GIFs and JPGs into PNGs.^[20] JAI can be used to flatten images from 24 to 8 bit color. Since images are shown on cell phone displays of such small resolution, trading color depth for download speed can be acceptable.

Development Tools

J2ME development is similar to J2EE and J2SE development in many ways. Developers write classes that fit within frameworks like Swing and struts. However, due to the limitations of the cell phone platform, there are several J2ME specific tools.

Useful J2ME tools

• J2ME emulators mock up the J2ME runtime environment on a desktop PC, allowing development to occur on a device that has more memory, a faster network connection, and a keyboard. Emulators are available from many sources: Sun, IBM, most device manufacturers and even a few carriers. The Sun Wireless Tool Kit (WTK) has an excellent emulator. IBM's WebSphere Device Developer (WSDD) has an emulator integrated with an IDE; this made for a great development environment, but the version I used did not handle the network threading deadlock appropriately and cost money. Both of these emulators support Windows and Linux. However, most cell phone emulators only run on Windows. Other emulators can be downloaded from device manufacturers' or carriers' websites.

  Be aware that when developing using emulators, the advantages gained can also lead to user experience issues: for example, it may make sense to download a logo for the traffic service every time if a fast reliable network connection is available, but this is not the case if the application is using the cell phone network, which is neither fast nor reliable. Be aware of such non functional problems and test on real phones as often as possible.

• Building midlets take more steps than building a normal web or java application, including preverification of classes^[21] and jad file generation. WSDD and WTK can take care of the additional steps, but then development has to take place in that context. antenna, on the other hand, is a set of ant tasks that leverage existing WTK classes to enable command line midlet builds. In addition, antenna also provides the ability to build midlets for other platforms and create automated builds.

• The size of the application jar file can be important, if for no other reason than application download time. Obfuscators are java programs that can be used with any class file; they twiddle with byte code and rename classes and variables to make decompilation more difficult. As a happy side effect, however, they also decrease the size of class files. proguard is one such obfuscater. Easy to use and feature rich, it also integrates with the WTK. Jode has also been recommended, but I found it difficult to use.

Rolling Out The Application

Actually rolling out a J2ME application to cell phones is easy. Buy a J2ME capable phone; there are lists of compatible phones on the web. ^[22] Get a data plan--4MB a month has been ample for me. Place the jad and jar files on a web server. Modify the web server's mime types to assign the correct content types to the .jad and .jar extensions:

Example 4. Addition to web server mime types for Apache

...
application/java-archive                jar
text/vnd.sun.j2me.app-descriptor        jad
...

Every phone capable of running a midlet will have a browser. To install, download the jad file; the midlet support infrastructure asks the user for permission to install the application, deals with versioning issues, and downloads and actually installs the application. In some cases, browsing directly to the jad file triggers the installation process; however, in one or two cases bizarre behavior prevented installation in this manner. The solution is to browse to an XHTML file that has a link to the jad file; such an XHTML document need not be complex (see Example 5, “XHTML File Used to Install a Midlet”).

Example 5. XHTML File Used to Install a Midlet

<a href='file.jad'>Application</a>

For an application such as the traffic congestion service, licensing and piracy are not huge issues. This is the case for most networking midlets, since the value comes from remote data delivered at the appropriate time, rather than the application itself. Delivery takes place after authentication, so if a user is not recognized or has not paid for the service, access can be easily denied.

Some aspects of J2ME deployments are not in the developer's control. For example, one source of frustration for me has been Verizon. This carrier, the largest in the USA, simply does not support J2ME.^[23] Optional package implementation is similar--if a device manufacturer has not implemented an API needed, the application simply cannot run on that device. Contrast this with the J2EE world, where the server is under the application developer's control, or at least influence. In addition, J2EE's typical delivered product, HTML, has a fairly standard feature set; with J2ME, clients differ in their capabilities.

Jar file size

In addition to speedy download as a reason to decrease jar file size, some phones, particularly inexpensive Nokia models, simply do not accept jar files larger than 64KB. Obfuscation tools such as proguard help, but even with obfuscation and elimination of unused classes, jar files might be too large. Other means of decreasing the size include design compromises and removing logging statements. Often, value objects on the client side have, in good javabeans style, get and set methods. To decrease the size of the class files, these can be replaced by public variables (see lines 6 and 7 of Example 3, “XML Parsing Code”). Commenting out logging statements, which generate a large number of String objects, decreases the size of the jar file significantly, as commented out Strings are not included in the class files.

This optimization experience was something new to the development team. After all, in the world of J2SE and J2EE, there are a different set of constraints--programmer time, database access, execution speed. How many developers ever really pay attention to the size of their class files?

Scalability

If an application has not been widely rolled out, scalability is difficult to ascertain. The traffic congestion service does have usage scenarios: a user will login, enter some data, and receive an answer. Perhaps they will check for traffic jams once or twice during the commute, but most users will not be using the service all day. Therefore, the application should be built to handle spikes during rush hours. One tool that can be used to gain an idea of performance is The Grinder. Of course, requests from a PC or set of PCs are not entirely reliable emulations of the real usage scenarios, but it is a starting point.

One concern is, though the MIDP 1.0 profile supports HTTP 1.1,^[24] clients with slow network connections require the server to have a larger number of sockets open for a given number of clients over a specified period of time, when compared to speedier clients. Ten clients that each take ten seconds to read a given amount of data, starting in one second intervals, will require a maximum of ten sockets, whereas ten clients that each take one second to read the same amount of data, starting in one second intervals, will require one socket. I am not sure at what point such behavior will have a negative effect on performance, but it should be considered when thinking of the number of supportable clients.

In Conclusion

In this paper, I have examined the kind of problems that rich client applications residing on cell phones solve. Cell phone platforms were examined, and I explained why J2ME was a good choice for an application like the traffic congestion service, where data was needed quickly while away from the desktop computer and little user data entry was required. I also covered the basics of J2ME and discussed some of the quirks that I have encountered, including device discrepancies. Even in the J2ME world, it is important to realize that, while MIDP 1.0/CLDC 1.0 gives a firm base from which to build, APIs for carriers and device makers will vary. I also looked at the different types of client/server solutions available to solve the problem, and some of the tools that made building midlets easier.

For many problems, J2ME can be a good solution with a low barrier to entry. As more and more phones can support more and more complex functionality, J2ME will continue to grow in promise. The variety of functionality available now reminds me of the web browser market in the mid 1990s; exciting things are in the works, even if many of the bugs have not been shaken out yet.

References

• Enterprise J2ME, by Michael Yuan
• The MIDP Homepage
• The optional package listing
• The MIDP 1.0 Style Guide
• The J2ME JavaRanch Forum members were helpful in solving quandaries.
• The carriers' websites: AT&T Wireless, Sprint, T-Mobile, and Nextel

Thank You

Thanks to Brian Rook, who brought me into his J2ME project.

Much thanks to my reviewers, who caught many mistakes and made some great suggestions. Any errors still present are of course my responsibility.

• Dion Almaer
• Bryan Buus
• Aprille Jordan
• Michael Yuan
• Corey Snipes
• Andy Pai
• Pat Meyers
• Jeff Barry
• Mike Mahon