From
Java EE to
Google App Engine to
GigaSpaces, the idea of developing against a middleware or "infrastructure" API is well established in the Java world.
But these are
fixed environments. With the (re-)advent of virtualization, it is now becoming feasible to package and rapidly provision your
own environment, custom-designed to meet your application's needs.
As the big middleware vendors are realizing, it is not just possible to create such Virtual Appliances, but
necessary: a production app's setup inevitably includes
more than just a couple of EARs.
Here, we'll look at the current state of cloud and middleware deployment tooling, examine possible future developments and draw parallels between deployment and related processes.
Cloud: The end of the beginning
The initial wave of
IaaS cloud providers resulted in a bunch of infrastructure services with deceptively similar specs but frustratingly different, proprietary APIs, parameters, concepts and formats. The first wave of cloud tooling has aimed to tackle this portability nightmare by attempting to introduce common APIs and terminologies.
At the same time, there have been proposals for
open virtual image and
hypervisor API formats, but there is as yet no sign of widespread adoption in the short term.
One difficulty for the current "meta API" approaches is the fact that generic support, i.e. providing features supported by
all providers, leads to a lowest common denominator that is too small to be of much practical use.
jclouds'
TemplateBuilder is an interesting attempt to tackle this problem, but ultimately only the adoption of an IaaS standard will makes this one go away.
From a developer's perspective, a more fundamental issue is that a "raw" machine such as is represented by a virtual image profile is just too low-level to be useful. IaaS vendors then to come from the provisioning world which deals in servers, racks and cores. But developers are used to developing against "environment services", be that a Java EE cluster, a LAMP stack or a Hadoop installation.
Middleware: Java EE and then some
After the large initial effort of putting together "corporate" Java EE implementations, the large enterprise vendors quickly moved to expanding the feature set of resources available to applications, such as persistent queues or clustered datasources.
As developers discovered and made use of these features, applications became progressively more tied to specific, often proprietary, middleware platforms. As a result, the development "deliverable" quickly grew to include not just the EARs envisaged by Java EE, but a variety of configuration options and resource definitions for the middleware container.
Keeping track of and deploying these
loose packages, whilst remembering to apply appropriate modifications specific to the target environment, has proven to be a difficult, time-consuming and error prone task. Indeed, users regard the automatic management of these packages as a major benefit of XebiaLabs' deployment automation product Deployit.
Slowly, though, the Big Vendors are also realizing that an effective "management unit" for today's Java applications contains goes beyond Java EE artifacts, and also needs to contain information about the middleware configuration: a "Virtual Appliance", you might say. Both IBM's
WebSphere CloudBurst and Oracle's upcoming
Assembly Builder attempt to address this issue by making it possible to capture WebSphere and WebLogic configurations, modify them and apply them to different installation.
Both tools run only in the context of virtualized environments - in fact, they are largely built on top of the vendors' virtualization offerings - and still appear to be limited to the IBM and WebLogic stacks, respectively.
Towards configurable Virtual Appliances
Current virtual image formats are essentially a long list of bits and bytes of memory and disk storage. I might be able to get some low-level infrastructure details such as IP addresses or number of cores from the API, but beyond that the best source of information about what this image actually
contains is a free-text "description" field, intended mainly for human readability. So even if I have configured a Websphere ND 7.1 installation with a database and a cluster, all according to company policy, most of the details of the installation are lost.
Certainly, it is very hard to programmatically check dependencies, enforce policies or even find an image matching certain software requirements (Java installed, version >1.5, latest OS patches etc.).
One interesting possibility is to define an image as a base OS together with a list of
yum,
rpm or
Conary packages. This gives a much more developer-level view of what is in the image and avoids reinventing this particular packaging and installation wheel.
Unfortunately, though, these package managers are all OS-specific and don't leave much room for customization of package installation and configuration.
Moving from clones to templates, and a "feature wishlist" for future Virtual Appliance support tooling, can be found in the
original post.
