Problem
Data Transfer Objects is an effective pattern to help decrease the number of cross-network calls, but the code on these objects can grow quite large if validation is required. How can we validate large amounts of data being transferred between a data object and an EJB without adding a significant amount of code to the Data Transfer Object? How do we eliminate duplicative validation logic across interfaces into our EJB server? Further, how can we automate the validation of the Data Transfer Object?

Forces
At our company we use the Data Transfer Object pattern for moving large amounts of data between the client and the EJB. We use IBM's command framework for taking requests from jsp's and other sources. We were writing long methods for validating inputs, which resulted less than maintainable code. We have a heterogeneous interfaces into our applications, including the web container(jsp's and servlets), MQ Series messages, XML messaging, and file upload programs. Consequently, javascript does not provide the tools for validating data outside of web pages. The XML can be validated via dtd's but only to a limited extent. We wanted to standardize the way our data is validated and eliminate duplication of code. Another requirement for validation is that we did not want to change the InventoryItemDataBean, nor the InventoryEJB.

Solution
 To start, we refractored the existing code by pulling all of the validation calls to a separate class. We found that although this helped to make the code clearer, we were still making lots of method calls to the validation class. The question was "How could we automate call all of the methods in the validation class?" An influence to our design was Junit, which uses the java reflection API to automate method calls in the Testing process. We wanted a similar structure.

To implement this structure, we created a set of methods, with the prefix of "validate", in the validation class based on the fields in the InventoryDataBean object that require validation. For the reflection based method calls to execute, we needed to be careful with the spelling and capitalization of each of the methods. Once we had this structure in place, we needed to implement the logic for the method calls. We placed a method in the InventoryLoaderCmdImpl named "validateInventoryItem" that provided the mechanism for validating the items(see code below). We made a call to this method on each InventoryItemDataBean that was loaded. If it fails validation, the EJB is not created and messages are sent back to the requester. The method below does the following:

1.)Create an array of all methods for InventoryItemDataBean Data Transfer Object.
2.) Filters out the getter methods.
3.)Create an array of all of the methods of the InventoryItemDataBeanValidator that start with "validate".
4.) Manipulate the name of the method validation method to see if the field name suffix matches the field name suffix of the InventoryItemDataBean getter method.
5.) Where method prefixes, or fieldnames match, the field is validated via the reflection API.
6.) In instances where validation fails, it logs a message that is returned back to the requesting interface.

public boolean validateInventoryItem(InventoryItemDataBean argdb) {
InventoryItemDataValidator inv_validator = new InventoryItemDataValidator();
boolean is_ok = true;
try{
      java.lang.Class InventoryItem = InventoryItemDataBean.class;
java.lang.Class inventoryvalidate = InventoryItemDataValidator.class;
java.lang.reflect.Method InventoryItem_methods [] = InventoryItem.getMethods();
java.lang.reflect.Method inventoryvalidate_methods [] = inventoryvalidate.getMethods();

for(int k = 0;k<InventoryItem_methods.length;k++){
Method temp = InventoryItem_methods[k];
               String method_name = temp.getName();
                    if(method_name.startsWith("get")){
for(int l = 0;l<inventoryvalidate_methods.length;l++){
                            Method validator = inventoryvalidate_methods[l];
String validator_name = validator.getName();
if(validator_name.startsWith("validate")){
String validate_reformat = "get"+validator_name.substring(8,validator_name.length());
if(method_name.equalsIgnoreCase(validate_reformat)){
Object [] empty_stuff= new Object[0];
Object [] stuff = new Object[]{temp.invoke(argdb,empty_stuff)};
Boolean b = (Boolean) validator.invoke(div,stuff);
System.out.println(b.booleanValue());
if(!(b.booleanValue())){
is_ok=false;
String results = ("INVENTORY ITEM WITH SPEC NUMBER:"+argdb.getMfPartNumber()+" VALIDIDATION TEST RESULTS: method calling:"+method_name+" validator calling:"+validator_name+" ResultOfTest: "+ b+"<BR>\n");
System.out.println(results);
errorit(results);
return false;
}//IF
}//IF METHOD
}//IF VALIDATOR
}
}
}
}catch(Exception e){
System.out.println(e);

}
return is_ok;

}

One big benefit that I forgot to mention is that I can customize the validator class for specific purposes. Some fields do not need to be validated in certain cases- like when we receive messages from our mainframe. In other cases, like when we receive data via a file upload, validation criteria is much tighter. This pattern provides the flexibility to change the validation criteria. If I don't want to validate a particular field, I just leave it out of the validation class.

Hi,

I have been using almost same kind of system to validate
DTO content. It seems to work fine!

This is a brilliant idea, thanks for posting it here.

Best Regards,

Vesa Varimo
vesa at regex dot fi

Hi
This is good idea.
But i think one problem is that use of Reflection API is very costly. Generally we use Reflection API for heavy process, while 'general Validations ' can not be considered as heavy process.

In one of my case , in EJB while setting values in Value Object, i am directly calling some 'Validating methods' are which are in one common class.


Viren Jasani.
viren_jasani@hotmail.com

You are right- Reflection is a costly API. I'll put some timers on the method calls to see the difference between calling the methods directly and using the API to call them dynamically and post it out here.

Another way to validate your DTO's is to have separate static constructor method for clients, and keep the more flexible constructor package private.

For example:
public class DataObject implements java.io.Serializable
{
  public static final long serialVersionUID = 123456789L;
  DataObject(int a, int b, int c, String d, Map e)
  {
    ...do whatever needs to be done
  }
  public static DataObject create(int c, String d, Map e)
  {
    ...validate c, d, e...
    return new DataObject(value1, value2, c, d, e);
  }
  DataObjectState state = null;
}

Note also the state object which is null until any of the fields are modified (after the object is created).
StateObject can maintain the information about the changed dependant object members (if there are any) so the database updates are considerably slower.

There's more to this approach that I'm happy to post if anyone is interested.

Ups. the database updates are considerably faster. Heh =).

Hi Hezekiel,

Please let us know more about your DataObjectState.

Cheers

Ismael

You could also use AspectJ to add the validation as a cross cutting concern,especially cause you use a certain name format (the validate in the method name) to trigger this code with reflection. read a nice article about it on javaworld:
http://www.javaworld.com/javaworld/jw-04-2002/jw-0412-aspect3.html

Validation API.

I think this is slight improvement to the validation method would be the following pattern.

public void validate<Property>( <Bean> bean, <Object> property, String propertyName ) throws ValidationException.

This allows the validation method to

(1) Cross validate property values using the bean that is passed in. For example triangulation on a Foreign Exchange amounts.

(2) Validate property values against the old property value, using the bean that is passed in. For example only allowing a property to move from one state to another.

(3) Throw a Validation error. This allows a fully response than a boolean, with possible sub classes of ValidationException. Error messages etc could be added.

(4) Throw the property name in the ValidationException - where the property name may not be the same as the individual property name. For example on a form with two properties of the same name. It would be nice to identify the field on the form to highlight it.

(5) Allow the Validation classes to be put inside the property methods to guard against erronous values when the property is set.

I also found that there may be different validation checks for when an object is created, as opposed to when it is updated - particually regarding the expectation of the existence of primary keys.

Nice concept. Your posting mentions an "IBM Command Framework". Can you provide a reference to this? It might solve an issue we are having.

Thanks

http://www.redbooks.ibm.com/pubs/pdfs/redbooks/sg245754.pdf
Its actually Websphere's command framework.


Here is an example of a command framework from scratch.
http://www-106.ibm.com/developerworks/ibm/library/i-extreme13/

This pattern is similar to the old NextStep NSValidation interface, which lives on in WebObjects.

In short, objects that implement the interface must implement a generic validateValueForKey(value,propertyName) method, which returns the validated value (which might have been coerced) or throws a ValidationException. It does this by calling the appropriate, optional, validate[Key] method if it exists. So you write the validateKey methods for each property you want to validate; the rest are considered valid by default.

In WebObjects, this interface is implemented by both WOComponent (the base view-level object, similar to a form-bean or page-bean) and EOEnterpriseObject (similar to a JDO).

The default implementation of the interface does, indeed, use reflection in validateValueForKey, partly because of NextStep's objective-C legacy of cheap reflection, but that certainly isn't necessary.

The JDO-like object also implements a sub-interface, EOValidation, which "validate an entire object to see if it's valid for a particular operation" (insert, update, save, delete), which are invoked automatically when the JDO-like framework initiates the associated operation. These operation-validators, in turn, typically would call the lower-level validation methods defined for NSValidation.

Typically, you'd use this by creating a base class for most of your interesting objects (e.g., a base entity object and a base view object) that provide default implementations (i.e. reflective), but expect to override them if and when performance were an issue.

Typically, these validation methods (validate[Key]) are themselves implemented in the object being validated, but there's no reason they couldn't be moved to a validationDelegate class.

Ray

Having worked through this very issues on ATG two years ago, and then hit it in another discussion here, let me just add the following:

  You are thinking procedurally. Validation is not external to the objects, but integral. Let the objects validate themselves.

    There is generally (at least) two types of validation. Format and existance. Format is things like Social Security Number ,Phone number. Existance is either simple: Address 1 must not be blank, or relational, (if other is checked, the descirption filed must not be blank)

    The first should be done with immutable objects that are coupled with builders ala String and StringBuffer. The secobd should be checked after all setters are called and should be done by your object that represents the form. Ideally, both of the validations would be done by your message objects, and would be wrapped by a form method that collects up the exceptions. Struts provides good support for this.


    Validation is a part of object creation. Treat it as such. Ideally you would use the builder pattern that throws an execption on the completion stage and returns all the problems.

Hi Adam

This sounds closer to what I would think object validation is all about ie. the object itself (in this case, an entity bean) should be reponsible for its own validation. However, I am not quite sure how one achieves this effectively ie. DTO objects are certainly necessary, but what validation does one do the on the web container side, and what validation does one perform on the server side, or does one even have to do validation on both sides. Surely performing validation in the web container enables qicker response to the end-user, and doing it from the bean is better object-oriented programming.

Any thoughts on this would certainly help my confusion in this area??

Regards
Chris

I don't agree. The whole scheme of using DTO's is to encapsulate persistent logic into a local copy data object. Adding business logic(validation)makes the code less maintainable. Also- it goes against the whole idea of encapsulation.

There seems to be some benefit in, and some pattern discussion supporting, the placement of validation code in the value object / data transfer object, and having the entity bean becoming an extension of its VO/DTO, thereby making the validation code accessible on the client and server sides of the application. This implies that the type of validations that can/should be done this way are focused on the internal correctness / completeness of the VO's/DTO's/Entity's state, as opposed to any business logic validation between two entities. Consequently, there is no problem with a VO/DTO validating its own state, and it does not violate encapsulation; what knows better how to validate its state than the object itself?

I agree with you Adam but I'd like to go a bit further.
While I thought that DTO's were an obvious solution to the network traffic problem I've always thought they should be implemented as properly constructed business objects.
If I am going to the trouble to instantiate a data object to hold the entity's data why not construct the actual business object with all of it's validation and business logic in place and leave the Entity Bean to the persistence mechanisms which it serves best.
Entity beans don't have inheritance, are very restrictive from a business object's design requirements and are best left to simply deal with the day to day mundane tasks associated with persistence handling.
My business object can then expose all of it's validation and formatting rules to whatever layer wants it and if complex service type business logic is required (i.e. validation across multiple entities) then that should be handled by the session bean managing the business processes mandated by the use cases.
As designer I want to construct robust object models and use them anywhere without being constrained by the persistence mechanism of the day.

You are right in terms of encapsulation. Adding the validation logic does not violate encapsulation rules. However, modularity of the code is something to consider-especially if the validation logic is quite complex. Wouldn't you agree that concerns over modularity(which was the whole reason we developed this pattern), do carry significant weight?

I most certainly agree that modularity is important, but I believe this should be properly designed in the context of business obeject's responsibility. I generally find that if validation gets too complex then I am probably trying to make my business object responsible for the application of business logic which is outside of it's real world responsibility. This logic should be deferred to a policy manager class which is implementing the business rules found in the use cases. I think that good design and appropriate distribution of responsibility generally leads to a simplification of otherwise overly complex business objects.
I think that your pattern is a quite clever solution to a problem that most often can be designed out.

Hi Adam.

I think you can put validation logic into your DTO (or ValueObject, whatever you may call it) as long as the validation does not depend on the context in which the DTo is used.

Example: You have a Customer object with attributes password, login, firstName, LastName, etc. When registrating a Customer only the attributes password and login must be set. When using other functionality the ohter attributes must be set as well.

You cannot handle this context-sensitive validation inside the DTO (at least I dont know how to do it in a sensible way).

Regards,
    Dirk

This problem can be viewed by different angle. I use refactoring mechanism. I would appreciate your comments on my suggestion

Assumptions
DTO contains
1. Optional Fileds
2. Required Fileds

All required field need validation always.
Optional field validation is not required always
All Required and Optional field need to be validated always

Let Any DTO have 'n' number of Validation methods.

I create a method by name

public boolean validateAll()
public boolean validateRequiedFields()
public boolean ValidateOptionalFields()

All other validations will be private methods. and each such method responsible for validating each field in that DTO.

I also create a public field with only get which will contain the Valdition errors as a String. These errors are concatenated and separated by desired delimeter. and the getValidationErrors will return the validation Errors if falde is returned by any of those methods.

We can use the appropriate method as applicalble, in our code to Validate a DTO

B Harish Kamath
Architect
ZapApp India Pvt Ltd.
Bangalore
hkamath@firstam.com