Schema Value Object

Author

Olivier Brand (obrand at yahoo dot com)
08/23/2001

Context

Value Objects must include validation code in order to optimize network traffic and XML features for B2B exchanges.

Problems

While implementing a three tier application using J2EE Core Patterns, a usual design will involve at least some Value Objects, some Session Facades, a few
Entity Beans and Data Access Objects for the underlying persistence logic.

A Value Object is the representation of a business data. Therefore it contains attributes and associated business methods.
A Value Object can be initialized by the Enterprise Java Bean layer (in the case of reads) or by the Client (in the case of creations or updates).
Usually the Value Object does not need to be validated when being created in the Enterprise layer since we assume that the data has already been
constrained by the persistent store.

A problem occurs when creating or updating these Value Objects (mutable Value Objects) in the Client layer.
For instance an attribute representing the email address of a customer in the Value Object can be tied to a specific column in a database with additional
constraints such as the length of the column and its format following a specific pattern (regular expression driven, ...).
Now the developer needs to make a decision where to add the validation code on this field.

The most common solution is to rely on the underlying datastore which will reject the insert or update if a field does not conform to a specific constraint.

The problem of relying on the datastore to check the data constraints is expensive in term of network overhead.
Another problem is that the implementation is more complex since the eventual thrown exceptions need to traverse multiple layers and be interpreted by the
client layer.

B2B data exchange must be realized through the HTTP protocol to overcome firewall restrictions. Standards such as SOAP, eBxml, ... are already defined.
Multiple implementations of these standards allow to wrap Enterprise Beans in a Web Service. Since the Enterprise Bean layer plays the role of a Value
Object Factory, the reulting data needs to be transformed at one point as XML to be encapsulated into SOAP or eBxml packets.

Forces

     The validation of a Value Object is done early on, avoiding Network overheads (similar to the role of Javascript for form validation in web
     applications)
     Simplify the three tier layer in term of exception handling
     Can rely on W3C specifications such as XML Schemas (or DTDs) for documenting and generating (see solution) data constraints.
     XML aware Value Objects can be easily included in B2B exchanges

Solution

Use an existing XML framework to express data constraints and automate the process to unmarshal XML documents to Java Objects and to
marshal Java Objects to XML documents.

There are many XML frameworks available. Some are relying on DTDs, others are relying on the newest XML Schema (aka XSD) specifications.
I chose to rely on the XML Schema specification in order to express the Value Object constraints.XML Schemas are defined in XML (as opposed to
DTDs), therefore can be created using any XML aware tools, documentation can also be generated using XSL transforms (output HTML, ....).
XML Schemas are "object oriented friendly" since they allow for complex types creation and extension.

The framework of choice used for implementing this pattern is Castor from Exolab. This framework is on its way to support the final W3C specification, but
it showed in any projects that its Schema support is enough for implementing this pattern. This framework is also Open Source.

The main idea is to write an XML schema corresponding to the structure of the final Value Object (in term of attributes and types), and add the constraints
as defined in the database (length, format, .....). Some constraints such as uniqueness cannot be expressed, but typically, you want the underlying database to
reject the record if such constraint is violated.

Then you can use the Castor source generator class to generate the Java classes associated with the XSD Schema. Each complex type is sually defined as a
Class. Each element, attributes are defined as properties (class variable) and corresponding setter/getter methods are automatically generated.

Relying on a source generator prevents coding complex validation rules in a language that cannot be understand by non Java developers. For instance a
Database modeler and an Architect (or Senior Developer) can be involved early on in order to define those schemas. This will be the starting point for the
DDL and the Schema Value Objects generations. This is a nice approach to bring 2 different teams together on a design and implementation phases.

Then the Schema Value Objects need to be created. At this stage there are 2 possible choices:

     The Schema Value Object extends the generated Java class from the XSD schema.
     The Schema Value Object encapsulates (wraps) the generated Java class from the XSD schema.

I prefer the second solution for 2 reasons:

     The Castor framework (or any other Schema aware framework generators) defines new types which correspond to the richer XSD Schema definition
     (compared to Java types). Therefore, you might want to hide these "proprietary" classes allowing later on (if needed) to replace the Java Schema
     generator with another one (for instance, when Javasoft will provide a Schema module for its JAXB framework).
     You might want to validate field by field (when each setter is being called) and add some dirty flag support in the same methods. Wrapping the
     generated setters with your own setters make then more sense.

When generating the Java classes from the XSD Schema using Castor, special classes called Descriptors are created.
These classes are handling the validation part (the constraints defined in the XML Schema are showing in this classes).

Do we want to validate field by field or the entire document ? Once again this is an implementationd ecision mostly driven by the client's implementation.

In my implementations, we decided to validate field by field (the validation is done in each setters).

To perform a field level validation, the Castor XML API offers a class to perform validation at the field level (as opposed as the entire document):

     FieldValidator.

I highly encourage you to look at castor.exolab.org in order to get familiar with the possibility of the product.

The second part of the pattern is trivial since any Schema (or DTDs) aware code generator, Castor included, include marshaling and unmarshaling facilities.
These methods allow to transform Java to XML and XML to Java.

Castor offers 2 classes to perform such transformations: Marshaller and Unmarshaller.

The Marshaller class can transform a Java object to XML. Some methods allow to marshal the object directly in a W3X DOM Node object.
This method is highly optimized when dealing with SOAP where you want to include a fragment (therefore the generated Node) in a SOAP enveloppe.
This method is also preffered when dealing with XSL (in frameworks like Apache Cocoon, ....) since the DOM tree is already built.
This method also works with the W3C SAX API.

The Unmarshaller class can transform any XML sources to the Castor generated Java object.

One of my projects included this pattern and was successfully implemented using the Apache SOAP framework with a WSDL extension we had to write.
WSDL is an XML document containing the definition of the operations exposed as web services. Any complex types are referred to an XML Schema.
This is one of the reason why this pattern perfectly applies to B2B exchanges by bringing the J2EE and B2B worlds together.

Consequences

     Takes advantage of the Value Object pattern
     Therefore it reduces network traffic.

     Uses W3C standards to express data constraints
     Schemas or DTDs can be understood by non java developper and is the common ground for B2B exchanges (see WSDL, SOAP, UDDI, ...)

     Allows for automated code generation
     Once the XSD Schema written, a Java Binding framework like Castor, can be used to generate the Java classes.

     Includes an XML framework to serialize/deserialize Java object to and from XML.
     This fits perfectly in the Web Services world. This can also be used in Web Applications driven by JSPs and XSLT processing.

Related patterns

     Value Object (Sun)
     Proxy (GoF)

Links

     SUN patterns: http://developer.java.sun.com/developer/restricted/patterns
     GoF Patterns: http://www.hillside.net/patterns/DPBook/DPBook.html
     Castor XML: http://castor.exolab.org

This pattern could (and should) be renamed: Schema Data Object.
This approach applies to a lot of scenarios and schemas are a good complement to add constraints to languages.
The Value Object could be one implementation of this pattern.

At sourceforge.net there is a project called jvalid that was started last year that does validation with XML Schemas.

http://sourceforge.net/projects/jvalid/

Also, there is a XSLT Stylesheet called Schematron that is designed to allow validation of XML Documents to be done with a definition of the validation in XML.

http://www.ascc.net/xml/resource/schematron/schematron.html

I have started a project in sourceforge.net to build some J2EE components that can be re-used that leverage the Schematron Stylesheet.

Needless complexity.

EJB for persistence and selection, value objects for data marshalling, xml schemas for data formatting information. How many other technologies are we going to add for a simple select or insert?

I really think using XML schemas/DTD's as a validation techniche does not gain you very much. The traditional if then else works and can be incorporated in the value objects at very little comparitive cost.

Jonathan

Jonathan,

This pattern does not only apply to value Objects but can be used mostly everywhere you need to have validation.
The generation of java object from a schema or DTD (JAXB), shields the developer from having to recode the format of certain fields.
As Brett (JDOM) clearly stated in his XML book, this kind of technology is the missing piece to any language: validating types. You can do some of it in the database, and now you can do it at the code level.
XML Schemas and DTDs are also an excellent starting point for both Database Modelers and Java Architects in order to express constarints on business objects.
Another point is that you can use this technology to read configuration file. I remember before using this kind of tools, having to deal with SAX or DOM just to parse a configuration file. Now you can automatically generate the parser and the object model !! It saves hours (days) of headaches and testing. And most important, you can focus on the real application and not the parser.

Hi Olivier,

I agree about the code generation (see my lowroad code generator).

However, most people do not have all of the skills that are mentioned in your reply. My argument is that you do not need this. Coding them into value objects is sufficient - its core Java, so you have a wide base of skilled people available. It's architecturally very simple, people understant it. You can provide the validation of fields as static functions if you like - making them available to everyone.

As for XML as data modelling, again I agree. Kind of. Its almost as good as paper :) The advantage is that you can churn through models quickly to see if they work - via simple tweaks to the XML.

OK, so lets change my argument:
If you are just starting a new project, and have no existing schema then an XML code generator is good. But using an instream XML (schema) validation technique is way too generic and slow. Java code can be used to validate value objects and this is fine.

Imagining your counter argument that the adelaide stuff will generate your custom java code and this is quick at xml validation. I agree, if the data source is XML. But most data is sourced from databases, and so validating an XML format is silly. The database is serving base types, not strings embedded in XML.

I do admit that it's been a while since I looked at JAXB, and when I did I thought it was a dead end. Which is why I did my own....(and yes it generates parsers, validating objects and so on, so maybe I agree with you, but not the standard tools).

Jonathan

oops, I meant here for the lowroad.

Jonathan

Jonathan,

In JAXB or Castor, there are 2 distinct parts:

- A validator
- A marshaller/unmarshaller

The main part (first part) of the pattern I put together focuses on the validation part.

I am not using XML at all (which as you pointed out might be heavy and slow, especially when using an already existing datasource such as a database).
This patterm focuses on the XML Schemas language as a tool to constraint java data (String, Integer, ...).

A tool like JAXB or Castor can then be used to avoid to have to code the validation code.
Castor generates from a schema a set of classes where basically each Schema complex type is translated into a type. A validation class comes along too. Then you could write in your DAO the following code around a business method:

public void setPurchaseOrder(PO aPO) throws ValidationException
{
    // Validate the PO with the Castor framework
    POValidator.validate(aPO);

    this.po = aPO;
}

In your schema, you will have a complex type called PO.

This way the code is very generic and flexible, you can consult some XML schemas (or schematron if easier to understand) in order to understand the constraints on the data.

I believe that XML Schemas will be used in a near future by Database vendor in order to put constraints on tables, rows, ...

We have worked on a project in order to define database constraints using XML schemas, therefore creating a special language based on XML schema. This is really helpful in order to bring 2 worlds together.

Hi Olivier,

OK, I misunderstood the pattern. All the Soap stuff made me think you were validating using an XML stream. In fact what you have is exactly what I did:

XML spec of data + constraints
Code generator to churn out EJB, Value Objects, Validators, and XML marshallers.

You seperate the validator from the value object, I tend to incorporate them into the same object.

The question is then one of tool selection. You are praising Castor, I wrote my own. I know that many of the IDE's also have their own EJB generators, and there are some that work by querying a database to extract the schema, and then generate the code.

The issue with these tools is the XML used to specify the structure - presumably vendors will go their own way, adding their own value added hooks. This is definatly an issue, just as SQL has its vendor syntax's then so too will the XML spec. And the entire area is way too young to pick a winner.

I'll shut up now, I agree with what you have done, I misunderstood.

Jonathan

Hi, I am interested by the initial mention of a SOAP/WSDL EJB web service implementation.

I am working on a project where we would like to invoke both Stateful and
Stateless EJBs via a web service with SOAP as the protocol.

We do not control the design of the EJBs and cannot influence such.
Therefore the EJBs may return primitive, simple or complex objects.

From reading various mailing lists, it seems that when people have asked about complex
and deep data structures the advice has been not to do it. It should be a
case of writing (de)serializers for the complex types but again this seems
to be advised against and there is no documentation for doing such.

Another architectural alternative would be to provide our own Provider and
use Apache SOAP for just the SOAP part; again there seems to be no
documentation.

Finally, the client to our system will not be Apache SOAP, nor MS - it will
be a C SOAP implementation running on Solaris. Considering the intial
problems between Apache and MS implementation would I be right to think that
we could hit problems?

From what I have researched and read it would seem that although our project
looks a cert for web services and SOAP, the implementations are not quite
there in terms of being ready and useable in large projects today - is this
fair?

Jonathan, thanks for your response, as you pointed out, we are on the same page. I am sure that the new book of Brett Mc... is going to the same direction since he talks about several generators.
before castor, I started to write my own generator, but you can quickly see the complexity of endorsing 100% the XML Schema spec. Maybe this is the reason Javasoft did not release a Schema version of JAXB as it was previously mentionned in the JSR. I do not think that doing DTD was very exciting, especially with all the WSDL/SOAP paradigm following the XML Schema specification for handling complex types. So wait and see.... Maybe they will use Castor as they did for Apache (Xerces, Xalan).

Darren, I understand your issues with SOAP,WSDL. I had to face the same issues several months ago.
I do not have a straight answer but here is what I have done around SOAP:

- I have wrapped the Apache SOAP implementation and augmented it with a WSDL backend. The engine was dynamically translating the WSDL(s) and registering the services using the proprietary API of Apache SOAP. This worked very good.

- We did not have time to implement UDDI (it was still pretty shaky: v1.0) but we wrote a flexible API for the client in order to discover the services. This API is using SOAP which is VERY important so it can be transalted to any languages: VB, C#, .. Along with this API, I have used the Apache SOAP "sniffer" tool in order to document the XML-SOAP API. Very very useful !! The concepts are very similar to UDDI which sends XML requests. What is missing from UDDI and maybe it is in JAXR (did not have the time to look at it), is the ability to perform some work around the WSDL descriptions (which are very cryptic...). With our API, we were able to query any services, its descriptions, parameters,... as you would do with the Java Reflection API. Very powerful !

- All the complex types were defined as XML Schema in the WSDL file itself (in fact a link to the XML Schema).

- The framework, at startup, is looking in a specific package for Castor generated Java classes and uses a CastorSerializer/Deserializer. This way, any complex types can be included in a SOAP enveloppe.

- The security is handled the same way UDDI does it: a token is generated nad has to be passed along the XML-SOAP enveloppe.

- I wrote a simplistic authorization API (still as a SOAP service) very similar to what the Code Version System: Perforce handles it. Our URIs are coded this way:

provider.service.method

So you could write something like:

user * theserverside.pattern.* allow
user joeblow theserverside.pattern.* deny

Which means: anybody beside joeblow can use all the methods provided by theserverside for the pattern service.

Methods can be: publish(PatternObject), delete(PatternObject), ....

The nest step will be to expose the services as Session Beans (Facade pattern) and using ValueObjects with the pattern I wrote. This business objects (value objects) will be automatically included in a SOAP enveloppe using the Castor Serializer/Deserializer.

Another step coul be using JAAS to handle authentication and authorization. You just have to wrap your own Auth* using the JAAS framework. You could this way, change how the security is handled using the JAAS configuration file where each application entry entry could be a provider, service, ... Here you have the flexibility to write your own configuration engine for JAAS (which I highly recommend).

As a conclusion:

All the server side is J2EE compliant, but other than that, the Client can be anything.

I would be interested to know if anyone managed to make these free/opensource products work on a complex schema (DTD-defined or XSD-defined) such as FIXML or FPML. So far:

- JAXB1.0(beta) looks promising but if you modify an objet, if wont marshall any more (validation is way too strict)
- Castor 0.9.3 (exolab.org) wont work on complex schemas, although the generated code compiles
- microsoft's XSD (VS7.0 beta2) generates classes that compile but don't do much, and can't generate datasets on complex schemas, though the technology looks promising
- zeus 3.1 (enhydra.org) generates source code that doesnt compile on most complex schemas
- I couldn't find anything of the sort for gc++, wonder how the unix geeks will do ?

On the brighter (but commercial) side, breeze (breezefactor.com) worked fine on the complex FIXML and FPML schemas. I'm also looking at BeanStork (zenaptix.com).

Does anyone have an example of genuinely complex schema (imposed by a B2B consortium of some sort) that actually binds with the free/opensource tools ?

Elian

How do you perform filed level validations using Castro, for an example, HTTP Form? Does anyone has any examples?

Thanks

Anyone aware of any tools for automated generation of XML schema from SQL Schema? I'm interested in doing tying Castor in with a complex legacy database and getting Java Value Objects "for free".

You can validate a field using Castor as the following example shows:

public static void validateObjectField(String fieldName, XMLClassDescriptor classDesc, Object objToValidate) throws
FieldValidationException
    {
       try
       {
           XMLFieldDescriptor xmlDesc = classDesc.getFieldDescriptor(fieldName, NodeType.Element);
           FieldValidator fieldValidator = xmlDesc.getValidator();
           fieldValidator.validate(objToValidate);
       }
       catch(org.exolab.castor.xml.ValidationException valEx)
       {
            throw new FieldValidationException(valEx.getMessage(), fieldName);
       }
    }

What do you guys think? Should setters of SVO be overloaded to accept a String as a parameter?

I think there are many advantages of doing it... for instance you'll not need to use java.text.* everywhere on your controller servlets and you can generate the parsing code as well.

Leonardo Bueno.

I really like this piece of info. I can understand using xml schema we can handle field validations. But in case of conditional validations (like say field A should have value 'X' if field B has a value 'Y'). If these can also be handled then it will be great.

Rgds,
Chandra

Hi,


I would like to have method's argument validated against the schema. For eg. If i have a part number that is of a string type and defines following constraint.

<simpleType name="PartNumber">
       <restriction base="string">
           <maxLength value="20"/>
           <enumeration value= "AAA-AAA" />
           <enumeration value = "BBB-BBB"/>
       </restriction>
   </simpleType>

How would I validate it in a java method ? Pl share some code snippet, if possible.

Thanks