Java Development News:

A RESTful Core for Web-like Application Flexibility - Part 3 - Logical Level Programming

By Randy Kahle and Tom Hicks

01 Sep 2008 | TheServerSide.com

Introduction

In the first two articles of this series [1] [2], we described an approach to application software architecture that is based on a core set of "RESTful" principles. This Resource Oriented Computing (ROC) approach is motivated by the desire to see if the flexibility found in the World Wide Web can be incorporated into application software.

While the previous articles examined the physical level of such an architecture, this article examines details of the boundary between the physical and logical levels. Our intent is to show how the logical level is supported by the physical level and to provide a solid foundation for discussing the logical level in more detail in a subsequent article. It is at the logical level that we will see how flexible applications can be composed from resources and small, low-overhead, internal services, referred to as micro-services.

URI Schemes

As discussed in the earlier articles, a client asks for information by issuing a request for a resource which is identified by a URI address. A URI address is comprised of a scheme and a scheme specific part [3]:

   {scheme-name} : {scheme-specific-part}

One can see that the syntax of URIs allows new schemes to be easily introduced into the system. Each new scheme specifies a set of logical addresses which are mapped to one or more physical endpoints.

For example, imagine that we want to add a new URI scheme called "var:", where the URIs of the var: scheme will represent resources managed by a temporary storage service. To map these logical addresses to a physical endpoint, we can specify a mapping such as:

   <map>
      <match>var:.*</match>
      <class>com.mycomp.endpoint.VARSchemeEndpoint</class>
   </map>

This mapping uses a regular expression in the match element to map all URI addresses using the new var: scheme to a single Java endpoint class that implements the temporary storage service.

The Java endpoint could be coded to support a full set of request verbs (SOURCE, SINK, NEW, EXISTS and DELETE) and could manage var: scheme resources using any suitable storage mechanism, such as in-memory Java objects. Such an implementation would enable client code to issue requests for temporary storage via a SINK to a var: scheme address, such as "var:tax-rate". Later clients can issue requests to the same address but with the SOURCE verb to retrieve the resource representation.

Another approach to designing logical address URIs is typified by the active: scheme; a versatile, service-oriented URI scheme first proposed by Hewlett-Packard researchers [4]. The active: scheme uses the following syntax to encode a service call, with zero or more named parameters, in the form of a URI address:

   active: {service-name} ['+' {parameter-name} @ {parameter-uri-address}] *

A few examples should make the active: scheme and its use clearer. An example service that does not require any parameters is:

   active:random

A request for this address with the SOURCE verb results in a representation with a numeric value between 0 and 1.

An example of an active: scheme service which does use parameters is the XSLT micro-service. This service requires two parameters: "operand", which references the XML information to be transformed; and "operator", which references the XSLT stylesheet that defines the transformation. Assuming that the active:xslt address space is bound to an instance of the XSLT service through this mapping

   <map>
      <match>active:xslt.*</match>

      <class>com.mycomp.endpoint.XSLTEndpoint</class>
   </map>

then a request for the logical URI address

   active:xslt+operand@resource:/data.xml+operator@resource:/style.xsl

with the SOURCE verb results in a call to the XSLT service via the microkernel. The physical level XSLT service endpoint code will, in turn, issue two sub-requests back into the logical address space; one for the representation of the resource at address "resource:/data.xml" and the other for the representation of the resource at address "resource:/style.xsl". After the XSLT transformation completes, the XSLT service will create a representation of the result and return that representation to the microkernel, which will forward it on to the requesting client.

The text that forms each active: scheme address must be parsed to extract the name of the service and the names and URI addresses of each parameter. While each endpoint could do this parsing, allowing the microkernel to interpret the active: scheme makes endpoints easier to write. With microkernel parsing support, the XSLT service endpoint class can implement the processRequest method with code such as:

   public void processRequest(Context context) throws Exception
     {
     Request req;
     Representation dataRep;
     Representation styleRep;
     String uri;

     XMLDOM domData;
     XMLDOM domStyle;

     // get the resource specified by the operand parameter
     uri = context.getParameter("operand");
     req = context.createRequest(uri);
     req.setVerb(Request.SOURCE);
     req.setType(XMLDOM.class);
     dataRep = (XMLDOM)context.issueRequest(req);

     // get the resource specified by the operator parameter
     uri = context.getParameter("operator");
     req = context.createRequest(uri);
     req.setVerb(Request.SOURCE);
     req.setType(XSMLDOM.class);
     styleRep = (XMLDOM)context.issueRequest(req);

     // do the XSLT transform on the operand resource
     resultRep = ...

     response = context.createResponseFrom(resultRep);
     response.setMimeType("text/xml");
     response.setCacheable();
     context.setResponse(response);
     }

We hope that this section demonstrates that the use of logical URI addressing for software construction is not a limiting factor in the ROC architecture. To the contrary, URI addressing is flexible enough to permit the creation and use of service-specific addresses and customized URI schemes.

Mappings and Address Resolution

Logical address resolution sits squarely at the boundary between the logical and physical levels of the ROC architecture. In order to resolve the logical address of a request, the microkernel searches through a set of mappings to find the first one that "matches". A variety of different address matching schemes are possible, which allows great flexibility in controlling the matching (and therefore mapping) process. When the microkernel finds a matching mapping it uses the mapping information to bind the request to the service endpoint and schedules the processing of the request.

We saw above how single addresses, or sets of addresses, can be mapped to physical endpoints. But the address resolution process can also function at the logical level. As the microkernel attempts to resolve a logical address to a physical endpoint, it may also utilize logical to logical address mappings. If one of these mappings matches the logical address of the current request, the microkernel will replace the URI of the current request with the replacement address dictated by the mapping. This example mapping

   <map>
      <match>resource:/customer/(.*)</match>
      <to>resource:/$1</to>

   </map>

uses regular expression matching and replacement to transform all logical addresses that are anchored at "resource:/customer/" to logical addresses that have the "/customer" part of the address removed. For example, the microkernel would replace the URI address "resource:/customer/som" with the address "resource:/som" and then continue searching for a physical endpoint using the new address.

This capability allows the application designer to relocate whole address spaces. For example, the internal address space for a Wiki application could be anchored at "resource:/" within the Wiki module. When the Wiki is used in a larger application suite, a logical to logical rewrite rule can place the Wiki address space at, for example, "resource:/public/application/wiki/.*".

A logical level address can also map to a micro-service. For example, the following logical to logical address mapping

   <map>
      <match>resource:/customers</match>
      <to>active:sqlQuery+operand@resource:/sql/customersQuery.sql</to>

   </map>

would cause requests for information about customers to be mapped to the active:sqlQuery service. Using the SQL query specified by the "operand" parameter, this service might query a relational database to return a representation of information about all of the customers [5].

Modules and Address Resolution

As mentioned in a previous article [2], logical address spaces in our ROC architecture are defined, implemented, and managed by modules. The export statements within a module declare the addresses that the module will accept and, therefore, the address space for which it claims responsibility [6]. Modules are also the containers for the logical address resolution mappings described above.

Since a module can import other modules which, in turn, can import still other modules, the address resolution process must navigate through the layered address spaces and mappings defined by the interconnected modules [6]. During the resolution process, the microkernel must determine whether to "descend" into an imported module in its search for an endpoint mapping. The microkernel examines the current request's URI address and compares it to the exported address space declaration of each imported module, in the order in which they are imported. If there is a match, the microkernel descends into the module to continue the resolution search. Once a module is entered, however, the resolution process either succeeds or fails. The resolution search does not exit an entered module to search other modules at the upper level (after all, a module is entered if and only if it has declared that it can handle the address space of the request's address).

The effect of the resolution process is that requests are "routed" along a tree structure of modules and, hence, through a set of address spaces. For example, assume module ACC exports the address space "resource:/acounting/.*" and module HR exports the address space "resource:/hr/.*". If module APP imports both modules ACC and HR, then a request issued into the address space of APP for "resource:/accounting/post_to_journal" would be routed to the "accounting" module (ACC) and a request for resource:/hr/number_of_employees would be routed to the "hr" module (HR).

Logical Level Programming

With the ability to specify logical addresses for resources, create service invocations, map logical addresses to logical addresses, and define modular import/export structures we have the foundation needed to structure and build applications at the logical level.

Transports and Root Requests

Fundamentally, applications in a Resource Oriented Computing environment are request-response systems. When an external request arrives it must somehow trigger an internal request for a specific resource. The application will then respond to the external request by returning the internally generated resource representation to the external client. Viewed in this way, the arrival of the external request is an event to which the application responds.

Our ROC architecture includes external event detectors called transports. These straddle the boundary between the world of external events and the internal request processing of an application. A transport is responsible for detecting a specific type of external event and creating an initial, or root request. The root request is then issued to the microkernel for resolution and processing.

There are many possible types of transports, each detecting a different kind of event. Some typical transports detect events such as:

  • Scheduled events: a Cron transport can fire a root request on a configurable schedule,
  • Filesystem changes: an in-tray transport can notice when a file is added to a monitored directory,
  • SMTP events: an SMTP transport can monitor the status of an email inbox and fire an internal request to process email when it arrives,
  • JMS messages: a JMS transport can monitor a JMS message queue for incoming messages,
  • HTTP requests: an HTTP transport can monitor a TCP/IP port for the arrival of a request using the HTTP protocol.

Each transport is responsible for dealing with the particulars of a certain class of external events, including all protocol issues associated with those events. By placing transports on the edge of ROC systems and making them responsible for the details of external events, applications can be built that are decoupled from and independent of transport protocols. For example, a resource such as "resource:/customers/" can be requested via a root request from any of the JMS, Cron, or HTTP transports. With the decoupling provided by transports, applications need not care where requests come from nor how they arrive.

Application Design

Applications architected for an ROC environment can have many different designs but they tend to share some common characteristics:

  • Information is modeled as resources, which are identified by URI addresses,
  • Resource addresses are organized into logical address spaces,
  • Logical address spaces contain endpoints which generate and/or accept resource representations,
  • Channels are created, through which information flows to and from external clients,
  • Logical address spaces are composed into layers which implement the goals of the application.

As a very brief illustration, the design of a forum application might begin with the identification of some sample actions and request URIs corresponding to these actions:

Action

Request URI

Show most recent posting

http://mycomp.com/forum/search/1

Return topic 512 in the first discussion area.

http://mycomp.com/forum/area/1/topic/512

Create a new topic with content of HTTP POST

http://mycomp.com/forum/update/topic

Once one of these requests enters the forum application, the difference between the ROC-based approach and a traditional object-oriented system becomes evident immediately. An ROC application will focus on routing the request to the resource or internal micro-service that can satisfy it. All the routing is done at the logical level; through logical to logical mappings, mapping services, and module imports.

The preparation of the representation to be returned to the external client is also different from a traditional application. It is accomplished by composing resources and, optionally, applying formatting transformations to the result. For example, a request from a browser for the display of a forum web page could result in a "mashup" operation, where information is derived from multiple sources. The mashup service could be driven by a page template which would include logical addresses for the required resources (menus, submenus, subject areas, topic summaries, etc.). Each of these resources would be identified by its URI address. When page assembly occurs, separate logical address space requests are issued for each constituent part. The decoupling of the logical from the physical means that the page assembly process has no idea whether a menu known as "resource:/menus/menu1" is static and stored in a file or is dynamically generated by Java code. And the same is true for all the other resources required by the template. In fact, the template itself is a resource and it could be based on a static representation or be dynamically generated by yet another service.

A Resource Oriented Computing architecture has another tremendous advantage over traditional applications: the ability to cache the returned representation for every request and to manage the cache for optimal performance. This ability is a direct result of the logical level properties of ROC applications: resources are logically addressed, logical addresses may be mapped to other logical addresses, responses are composed from logically addressed resources, and responses are transformed by logically addressed services.

In the forum example, if all resources are dynamically generated the first time they are requested, then subsequent requests run much faster because their representations are obtained from the cache. If a forum resource is updated (such as when a new item is posted) then all cached resources that are impacted by or dependent on the change are automatically and atomically flushed from the cache. The next time a forum page is requested, only those resources which were invalidated need to be recomputed - all other information remains available from cache.

Caching in an ROC architecture is similar to the use of memoization [7], except that it requires no extra work by developers and it is automatically applied across the entire system. Memoization is usually applied (if at all) to just a small corner of a selected application, for one specific data type, and only where someone has put in the extra effort to implement it.

Conclusion

This article continued the discussion about building application software using the Resource Oriented Computing approach; a flexible, Web-like architecture based on RESTful principles. In our discussion, we moved upwards from the physical code level to examine those capabilities which lie at the boundary between the physical and logical levels of the architecture. We saw how ROC features, such as logical URI addressing, logical address spaces, address mappings, transports, and modules support clean and flexible application design. Hopefully, the reader is beginning to see how one might utilize these ROC features to program at the logical level within a resource-oriented computing environment.

A subsequent article will explore design and architectural patterns that become available at the logical level. Just as patterns of object relationships and interactions were important for object-oriented programming and design, logical level patterns allow one to think and reason about system design in a powerful and concise way.

References

[1] A RESTful Core for Web-like Application Flexibility - Part 1
[2] A RESTful Core for Web-like Application Flexibility - Part 2
[3] RFC 2396: Uniform Resource Identifiers (URI): Generic Syntax
[4] The "active" URI scheme
[5] In this example, a subsequent mapping of active:sqlQuery to a physical endpoint would be used to establish the connection to a database, execute the query, and return a result representation. Such a mapping need not be defined in each application module but could be imported from a commonly available relational database query module.
[6] Since modules export only logical addresses, the coupling between modules is at a logical level. Modules are free to change the internal implementation of the services provided at the exported logical addresses.
[7] Wikipedia entry for memoization