High Performance and Standard Transformation for Complex Financial Services Messages

Java Development News:

High Performance and Standard Transformation for Complex Financial Services Messages

By John Davies

01 Sep 2006 | TheServerSide.com

Introduction to C24's Integration Objects

C24's Integration Objects (IO) is essentially bound Java code that represents a model. The model can be imported (as in the case of XML Schema, a database and XMI etc.), pre-defined (as in the case of SWIFT and EDIFACT), or user-defined, in typically a comma-delimited or fixed format file.

Any model imported into the IO Editor can be managed as meta-data. The meta-data can be edited, re-used, and re-exported as XML schema, XMI, or even Java (hence Java-binding).

A complex XML schema can be imported into the IO Editor, then Java "deployed" and compiled into a jar. This jar effectively then provides a rich API for both the model and instance documents.

FpML example

Take, for example, FpML (www.fpml.org). We can import the extremely complex schema including idrefs and substitution groups, etc., and deploy an fpml.jar. Ignoring several other advanced features for now, we can use the API in this jar to parse FpML instance documents and validate them. This includes, by the way, the several dozen rules that exist outside of the schema, something that very few tools can currently handle - rules typically defined in plain English or more recently in schematron and OCL. Once parsed the object can be passed from application to application either locally, by reference, or remotely via its own customized serialization. The object can be accessed for reading and updating through a Java-bean API or built-in XPath 2 navigators.

Code examples

The code below demonstrates a simple method to create an instance of the FpML object and then use it to parse a file containing an instance document, returning a generic ComplexDataObject. I've bolded the relevant lines; the rest are not relevant at this point.

 

public ComplexDataObject readFpMLFromFile( String filename ) {

   ComplexDataObject complexDataObject = null;

   try {
      Element fpml = FpMLElement.getInstance();

      // Set logging level for the parser
      fpml.getLog().addAppender(new ConsoleAppender(new SimpleLayout()));
      fpml.getLog().setLevel(Level.ERROR);

      FileReader fileReader = new FileReader(filename);
      XMLSource xmlSource = (XMLSource) fpml.getModel().source();
      xmlSource.setReader(fileReader);

      // Actually parse the file into the new object
      complexDataObject = xmlSource.readObject(fpml);

      // Tidy up logging
      fpml.getLog().removeAllAppenders();

      // Tidy up file
      fileReader.close();

   } catch (IOException e) {
      e.printStackTrace();
   }

   // Return the IO object in its generic form
   return complexDataObject;
}

We can now access the data through a number of ways...

Bean classes

 

DataDocument dataDoc = (DataDocument) readFpMLFromFile( filename );

println("tFound "+dataDoc.getParty().length+" parties");

for (int i=0; i<dataDoc.getParty().length; i++)
{
   println("ttParty name: "+dataDoc.getParty()[i].getPartyName());
   println("ttParty id: "+dataDoc.getParty()[i].getPartyId());
}

Trade[] trades = dataDoc.getTradeThenPortfolioGroupOrEventGroupGroup().getTradeThenPortfolioG
roup().getTrade();

println("tFound "+trades.length+" trades");
for (int i=0; i<trades.length; i++)
{
   println("ttTrade "+(i+1)+" has a product of 
   "+trades[i].getProduct().getName());
}

Notice above how easy it is to understand the business logic (assuming you understand FpML).

Through XPath 2.0

 

List list = IOXPathFactory.getInstance("//party/partyId").getList(obj);

for (Iterator it = list.iterator(); it.hasNext();)
{
   IOContext ioc = (IOContext) it.next();
   println(ioc.getInstance().toString());
}

Since XPath is text it can be stored in a properties file or applied at runtime dynamically.

We can validate the document including the extra non-schema validation rules.

 

ComplexDataObject obj = readFpMLFromFile( filename );

obj.setValidationMechanism(ValidationMechanismEnum.VALIDATION_MECHANISM_EXCE
PTION); // this is the default anyway

try
{
   obj.validate())
}
catch(ValidationException ve)
{
   println("Validation failed: " + ve.getMessage());
}

Of course the validation would be done interactively using a listener - this is better because if several things fail, they're then easier to report back.

 

ComplexDataObject obj = readFpMLFromFile( filename );

obj.setValidationMechanism(ValidationMechanismEnum.VALIDATION_MECHANISM_SYNCHRONOUS_EVENT);

ValidationEventCollector vec = new ValidationEventCollector();
obj.addValidationListener(vec);

try
{
   obj.validate()

   if (vec.hasEvents())
   {
      println("Processing validation events...");

      if (vec.hasFailEvents())
         processValidationEvents(vec.getFailEvents());
      if (vec.hasPassEvents())
         processValidationEvents(vec.getPassEvents());
   }
}
catch (ValidationException ve)
{
    // will never happen
}

We could write the object back out as XML, which will actually be lossless, i.e. binary compatible with whatever we read in.

 

new XMLSink(System.out).writeObject(obj);

Do the same but return a String - this is useful for outputting objects as full XML for integration into other non-Java systems.

 

StringWriter sw = new StringWriter();
new XMLSink(sw).writeObject(obj);
return sw.getBuffer().toString().trim();

Remember now that we can read in any format including SWIFT and CSV, etc. This means that we can output a CSV or SWIFT messages as XML without having to define a transformation. We can even generate a fully valid XML schema for SWIFT 7775 and 15022 messages.

XML is one output form. Another is tag/value pairs.

 

StringWriter sw = new StringWriter();
new TagValuePairSink(sw).writeObject(obj);
return sw.getBuffer().toString().trim();

Transformation

Being given so many ways of getting and setting data through IO opens us up to a number of different ways to transform something, say from XML to FpML, FpML to XML, Database to FpML, FpML to CSV, one version of FpML to another, etc., etc.

Since we produce code we started off by suggesting transformations should be done in code and we still have a number of clients that use this method. It's very fact-based and given the tools (the generated API and XPath) it's very easy to knock up a fast and maintainable transform.

The business people, however, wanted to play with this and didn't want to have to rely on a pool of Java programmers (boo), so we wrote a GUI a few years ago. In keeping with our love of high performance, the GUI generated code and the code can be compiled and jar'd with the binding and rules code.

Finally, as we started to get more and more customers, and we're talking about the vast majority of the investment banks and several clearing houses, the architects started to worry about the GUI being too proprietary; i.e., they wanted a standard for transformations, something that would work on a number of tools, something that used standards.

Earlier this year we introduced XSL and XQuery. We asked the world's leading expert and spec leader in these subjects (Mike Kay) to help us out. What we wanted was the ability to execute a standard XSL transform of XQuery on any of our IO objects without having to convert the data into XML first.

Mike did an excellent job on this and we are now proud to have the world's ONLY native XPath 2.0, XSLT 2.0 and XQuery engine that works on native code without XML. As you might expect, it's pretty fast and very powerful. It doesn't require a large server and can be deployed onto almost any platform that runs Java. In a nutshell, we can read in a message as CSV, XML, fixed length, SWIFT, etc. and execute an XQuery or XSL transform on it without having to convert it to XML.

At this point I'd like to introduce Mike and pass you on to his XQuery paper. We will have a second paper on XSLT shortly, so please watch this space.

http://www.c24.biz/download/c24_white_paper-using_xquery.pdf