Appendix to Lightweight Domain Specific Modeling

This is an appendix to the article Improving Developer Productivity with Lightweight Domain Specific Modeling by Patrik Nordwall.

Sequence Diagram

Normal execution flow from Service, via Repository, to Access Object.

Figure 1.  Normal execution flow from Service, via Repository, to Access Object.

Ecore

Ecore has been chosen as the "format" of the DSL model in the example in this article. The API class diagram is useful when learning Ecore. UML2 was also considered, but Ecore includes the needed features and was easy to understand. UML2 provides a lot more features, which were not of interest for the purpose of this article.

We use ArgoUML to define the DSL model, which we convert to an Ecore model with the argo2ecore plug-in.

Tip: Refresh the .zargo file and remove existing Ecore model before using argo2ecore.

There are several other options to create the Ecore model:

  • Use the built-in primitive Ecore editor.
  • Export from a modeling tool such as Rational Rose or RSA
  • Annotate Java interfaces with model properties
  • Use XML Schema to describe the model

JET Basics

There are several good articles that describe JET, such as JET Tutorial. Since Java-code is generated and the scriptlets in the template are also written in Java it can be a bit difficult to read the templates until you get used to it. Two important scripting elements that you need to understand:

  • <% this is a scriptlet %>

    Scriptlets can contain any valid Java code fragment. Scriptlets are typically used for flow control, such as if and loop statements.

  • <%= this is an expression %>

    An expression element is a Java expression that is evaluated and the result is appended to the generated result. Expressions are typically used to output variables (from the model).

When developing a template you can easily try it out by using the preview pane.

Figure 2.  When developing a template you can easily try it out by using the preview pane.

Tip: When an exception occurs when using the templates you have to look in the .log file in the Eclipse workspace .metadata directory.

Merlin Tools

Follow these steps to generate code with Merlin, by creating a mapping between model elements and JET templates.

  1. Create a JET Template model from the template project.

    You find 'JET Template Model' in the Merlin Tools section of the New > Other Wizard of Eclipse.

    Choose 'Load From Java JET Project' and select the GenTemplates project.

  2. Create a JET Mapping Model

  3. Use the JET Mapping Editor menu to define the input root as the Ecore model and output root as the jettemplate created above.

  4. Define the mapping by drag-n-drop of the model elements to the templates, see Figure 3, “JET Mapping”. A model element is typically a class, operation or package, but it can be anything defined in the model. The model element will be the input argument to the template.

In Merlin's JET Mapping Editor you define model elements as input to templates. In the upper part you drag the model elements to the left onto the templates to the right. In the lower part you can see which model elements that has been mapped to a template.

Figure 3.  In Merlin's JET Mapping Editor you define model elements as input to templates. In the upper part you drag the model elements to the left onto the templates to the right. In the lower part you can see which model elements that has been mapped to a template.

Tip: When you have added more templates to the JET Template Model or changed the Ecore model you have to close and reopen the JET Mapping Model to refresh the changes.

  1. Generate code by right clicking on the model root or individual model elements in the JET Mapping Model.

Tip: We couldn't generate code when we used src/java as source output directory. When we changed to src it worked fine. It must be a defect in Merlin.

Tip: There might be a defect in Merlin that it doesn't always overwrite generated code as expected when model or template is changed. Removing the generated code will help, but it is an annoying defect that must be corrected.

Mixing Generated and Hand Written Code

It is convenient to be able to mix generated code with manually crafted code, but there are drawbacks as described in the pattern Separate generated and non-generated code.

You can modify the generated code. Remove the @generated tag and your changes will be preserved. EMF includes JMerger, which facilitates this merging. In this example the approach with adding hand written code to generated code was used. In this small example it has worked out rather well. However, Merlin doesn't always overwrite generated code as expected when model or template is changed. It is probably better to separate generated code from hand written code in separate files. That means that hooks/plug-ins need to be implemented by hand written code. That is often solved by subclassing (generated code as base class) or delegation. It can be hard to know in advance what extension points are necessary, but that is not a big problem with this approach since you can add more extension points later on, as you need.

Domain Model

DSL Model

Each class that is part of the domain model of the ecore model is mapped to the BasicClass.javajet template.

Generated Code

(Click on the files to see the generated Java code.)

Template: BasicClass.javajet

The first section of the template file declares the compiled template package and class name. It also defines the imports used in the scriptlets.

<%@ jet package="compiledtemplates" 
imports="java.util.* util.EcoreGenerationHelper 
  org.eclipse.emf.ecore.* org.eclipse.emf.codegen.util.*" 
class="BasicClass" %>
                        

The input argument is the Ecore model element that was mapped to the template. The helper class instance is created.

<%EClass eClass = (EClass) argument;
EcoreGenerationHelper h = new EcoreGenerationHelper();
                        

The ImportManager takes care of the import section. Imports can be added explicitly or automatically. In the end of the template the ImportManager is invoked to sort and include the import section at the right place.

h.makeImportManager(eClass.getEPackage());
StringBuffer importStringBuffer = stringBuffer;
int importInsertionPoint = stringBuffer.length();
h.getImportManager().addCompilationUnitImports(
  stringBuffer.toString());

...

<%importStringBuffer.insert(importInsertionPoint, 
  h.getImportManager().computeSortedImports());%>
                        

Each attribute of the eClass generates an instance variable and accessor methods.

<%for (EAttribute attribute : h.getAttributes(eClass)) {%>
    /** @generated */
    private <%=h.getTypeName(attribute)%> <%=h.getName(attribute)%>;
<%}%>

...

<%for (EAttribute attribute : h.getAttributes(eClass)) {%>
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
    <%=h.getVisibility(attribute) %><%=h.getTypeName(attribute)%> <%=
            h.getGetAccessor(attribute)%>() {
        return <%=h.getName(attribute)%>;
    }
    <%if (attribute.isChangeable()) {%>    
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
    <%=h.getVisibility(attribute) %>void set<%=h.capName(h.getName(attribute))
          %>(<%=h.getTypeName(attribute)%> a<%=h.capName(h.getName(attribute))
          %>) {
        this.<%=h.getName(attribute)%> = a<%=h.capName(h.getName(attribute))%>;
    }
    <%}%>    
<%}%>
                        

Associations with multiplicity 1 generate similar code as the attributes. Associations with many-multiplicity generate a collection instance variable and accessor methods. The collection type can be List, Set, or Map and it is defined with the annotation 'collectionType' and it is provided by the helper class in the methods getCollectionInterfaceType and getCollectionImplType.

Note the convention to use typed collections (generics).

<%for (EReference ref : h.getAllManyReferences(eClass)) {%>
<%
     h.getImportManager().addImport("java.util." + h.getCollectionInterfaceType(ref));
     h.getImportManager().addImport("java.util." + h.getCollectionImplType(ref));
%>
    /** @generated */
    private <%=h.getCollectionInterfaceType(ref)%><<%=h.getTypeName(ref)%>> <%=
                    h.getName(ref)%> = new <%=h.getCollectionImplType(ref)%><<%=
                    h.getTypeName(ref)%>>(); 
<%}%>

...

<%for (EReference ref : h.getAllManyReferences(eClass)) {%>
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
    <%=h.getVisibility(ref) %><%=h.getCollectionInterfaceType(ref)%><<%=
          h.getTypeName(ref)%>> <%=h.getGetAccessor(ref)%>() {
        return <%=h.getName(ref)%>;
    }
    <%if (ref.isChangeable()) {%>    
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
    void set<%=h.capName(h.getName(ref))%>(<%=h.getCollectionInterfaceType(ref)
          %><<%=h.getTypeName(ref)%>> a<%=h.capName(h.getName(ref))%>) {
        this.<%=h.getName(ref)%> = a<%=h.capName(h.getName(ref))%>;
    }
    <%}%>
<%}%>
                        

toString is a simple, but useful feature. equals and hashCode requires some thought when used with Hibernate, see the discussion at the Hibernate site. This implementation supports natural keys, if defined with annotations on the attributes that are part of the key. If no natural keys are defined a UUID property is generated. This mechanism is handled in a consistent way in the generation of DDL and Hibernate mapping files.

Notice that these are defined in a separate jetinc file that is included. This mechanism should be used to extract duplicated template fragments. In this article we have chosen not to do it, but in real world duplications in templates should be avoided for the same reasons as in ordinary code.

<%@ include file="equals.jetinc"%>

<%@ include file="hashCode.jetinc"%>
   
<%@ include file="toString.jetinc"%>
                        

Access Objects

DSL Model

The repository classes in the DSL model are mapped to the templates; AbstractFactory.javajet and ConcreteFactory.javajet.

Some of the operations in the repository are mapped to CommandInterface.javajet and CommandImpl.javajet, when a custom Access Object is required, i.e. when the generic Access Objects of the application framework can't be used.

Generated Code

(Click on the files to see the generated Java code.)

Naming conventions are used to realize specific features. The input class must end with "Repository", and the part before that is used as part of the name in several of the classes. Package naming conventions are also defined in the templates.

if (!eClass.getName().endsWith("Repository")) {
    throw new IllegalArgumentException(
      "Expect name of class argument to end with \"Repository\""); 
}
String baseName = eClass.getName().substring(0, eClass.getName().length() 
  - "Repository".length());
String productType = "Access";
                        

The interesting part of the factory is the generation of the create methods. Annotations are used to tag operations for special cases, in this case 'noaccessobject=true' is used to tag that a corresponding Access Object does not exist and factory methods should not be generated.

The template is aware of the generic Access Objects of the application framework and generates slightly different implementations for the different types of Access Objects.

<%for (EOperation op : h.getOperations(eClass)) {
  if (h.getAnnotation(op, "noaccessobject") != null) continue;
  String mappedOpName = h.getMappedOperationName(op);
%>
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
    public <%=h.capName(mappedOpName)%><%=productType%><%=h.getGenericType(op)
        %> create<%=h.capName(mappedOpName)%><%=productType%>() {
      <%if (mappedOpName.equals("findById")) {%>
        return new <%=h.capName(mappedOpName)%><%=productType%>Impl<%=
          h.getGenericType(op)%>(getPersistentClass());
      <%} else if (mappedOpName.equals("findAll") ||
                     mappedOpName.equals("findByExample")) {%>
        return new <%=h.capName(mappedOpName)%><%=productType%>Impl<%=
          h.getGenericType(op)%>(getPersistentClass());
      <%} else if (mappedOpName.equals("findByQuery")) {%>
        return new <%=h.capName(mappedOpName)%><%=productType%>Impl<%=
          h.getGenericType(op)%>();
      <%} else if (mappedOpName.equals("create") ||
                     mappedOpName.equals("update") ||
                     mappedOpName.equals("delete")) {%>
        return new <%=h.capName(mappedOpName)%><%=productType%>Impl<%=
          h.getGenericType(op)%>();
      <%} else {%>
        return new <%=h.capName(mappedOpName)%><%=productType%>Impl<%=
          h.getGenericType(op)%>();
      <%}%>
    }
     
<%}%>
                        

The AbstractFactory.javajet is similar to the above ConcreteFactory.javajet, but it is simpler.

The implementations of the Access Objects is for Hibernate persistence and they extend a common base class, which is not generated. This is a typical example of a nice mix of generated code and frameworks.

For each parameter in the repository operation a setter method in the Command is generated.

<%for (EParameter parameter : h.getParameters(eOperation)) {%>
...
    
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
    public void set<%=h.capName(h.getName(parameter))%>(<%=
          h.getTypeName(parameter)%> a<%=h.capName(h.getName(parameter))%>) {
        this.<%=h.getName(parameter)%> = a<%=h.capName(h.getName(parameter))%>;
    }
<%}%>
                        

The return type of the operation is used in the getResult method.

<%if (!h.getTypeName(eOperation).equals("void")) { %>
    /**
     * The result of the command.
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
    public <%=h.getTypeName(eOperation)%> getResult() {
        return this.result;
    }
<%}%>
                        

The generated Access Object implementation requires that you fill in the implementation details in the performExecute method.

    public void performExecute() throws HibernateException {
        // TODO Auto-generated method stub
    }
                        

The CommandInterface.javajet is similar to the above CommandImpl.javajet, but it is simpler.

One can imagine additional implementation alternatives, such as a dummy stub, or in memory persistence. Abstract factory and separated interfaces make it possible to have several pluggable implementations.

Repository

DSL Model

Generated Code

(Click on the files to see the generated Java code.)

Template: Repository.javajet

The interesting part of Repository.javajet is the section that generates the methods. It generates code that delegates to the Abstract Factory and Access Object, but if the 'noaccessobject' annotation has been defined on the operation an empty method stub is generated, to be filled in by hand written code.

<%for (EOperation op : h.getOperations(eClass)) {
  // a few naming mapping conventions
  String mappedOpName = h.getMappedOperationName(op);
  boolean findById = (mappedOpName.equals("findById"));
%>
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
     <%=h.getVisibility(op) %><%=h.getTypeName(op)%> <%=h.getName(op)%>(<%=
       h.getParameterList(op)%>) <% if (findById) {
       %>throws <%=baseName%>NotFoundException<%}%> {
  <% if (h.getAnnotation(op, "noaccessobject") != null) {%>
        // TODO Auto-generated method stub
        throw new UnsupportedOperationException("<%=mappedOpName
          %> not implemented");
  <%} else {%>
        <%=h.capName(mappedOpName)%><%=productType%><%=h.getGenericType(op)
          %> ao = <%=h.uncapName(baseName)%><%=productType%>Factory.create<%=
          h.capName(mappedOpName)%><%=productType%>();
    <%for (EParameter parameter : h.getParameters(op)) {%>        
        ao.set<%=h.capName(h.getName(parameter))%>(<%=h.getName(parameter)%>);
    <%}%>
        ao.execute();
    <%if (!h.getTypeName(op).equals("void")) {%>
      <%if (findById) {
        EParameter idParam = h.getParameters(op).get(0);
      %>
        if (ao.getResult() == null) {
            throw new <%=baseName%>NotFoundException("No <%=
              baseName%> found with <%=h.getName(idParam)%>: " + <%=
              h.getName(idParam)%>);
        }
      <%}%>
        return ao.getResult();
    <%}%>
  <%}%>       
     }
     
<%}%>
                        
[Note]

Note the convention to use typed Lists (generics). In the model we use array types of the operations, e.g. Media[] that is generated as List<Media>. This mapping is implemented in the method getTypeName in the helper. This is a typical design decision that is implemented in the templates as a convention.

DDL

DSL Model

Database definition file can be generated from the domain model. The domain package is the input for generation of the DDL file.

Generated Code

(Click on the file to see the generated SQL code.)

Template: DDL.sqljet

Create table syntax is often database vendor specific and this code generator is targeted for MySQL. You can easily adopt it to your specific database.

The tables must be created in the order that referenced tables are created first. E.g. MEDIA and PERSON must be created before ENGAGEMENT. Actually it is the foreign key constraints that must be created in that order, but we have chosen the syntax where the foreign key constraints are included in the table definition. An alternative is to add the constraints afterwards with alter table, and that requires the same type of order.

<%for (EClass eClass : h.getClassesInCreateOrder(ePackage)) {
  boolean hasSuperClass = !h.getExtends(eClass).isEmpty();
  boolean hasOneReferences = !h.getAllOneReferences(eClass).isEmpty();
%>

CREATE TABLE <%=h.getDatabaseName(eClass) %> (
  <%if (h.getNaturalKeys(eClass).isEmpty()) {%>
  UUIDSTRING VARCHAR(255) NOT NULL, 
  <%}%>
  <%for (Iterator<EAttribute> iter = h.getAttributes(eClass).iterator();
      iter.hasNext();) {
    EAttribute attribute = iter.next();
  %>
  <%=h.getDatabaseName(attribute)%> <%=h.getDatabaseType(attribute)%><%
    // skip , on last line
    if (iter.hasNext() || hasOneReferences || hasSuperClass) {%>,<%}%>
  <%}%>
  
  <%for (Iterator<EReference> iter = h.getAllOneReferences(eClass).iterator(); 
      iter.hasNext();) {
    EReference ref = iter.next();
  %>
  <%if ("list".equals(h.getCollectionType(ref))) {%>
        <index column="<%=h.getDatabaseName(ref)%>_INDEX" />
  <%}%>
  <%=h.getForeignKeyName(ref)%> <%=h.getForeignKeyType(ref)%>,
  <%=h.getForeignKeyConstraint(ref)%><%
    // skip , on last line
    if (iter.hasNext() || hasSuperClass) {%>,<%}%>
  <%}%>
  
  <%if (hasSuperClass) {
    EClass superClass = h.getExtends(eClass).get(0);
  %>
  <%=h.getForeignKeyName(superClass)%> <%=h.getForeignKeyType(superClass)%>,
  <%=h.getForeignKeyConstraint(superClass)%>
  <%}%>
);
     
<%}%>
                        

Many-to-many relations are realized with an additional relation table in the database. Those tables are resolved in the method resolveManyToManyRelation in the DatabaseGenerationHelper.

<%for (EClass eClass : h.resolveManyToManyRelations(ePackage)) {%>
CREATE TABLE <%=h.getDatabaseName(eClass) %> (
  <%for (Iterator<EReference> iter = h.getAllOneReferences(eClass).iterator(); 
      iter.hasNext();) {
    EReference ref = iter.next();
  %>
  <%=h.getForeignKeyName(ref)%> <%=h.getForeignKeyType(ref)%>,
  <%=h.getForeignKeyConstraint(ref)%><%
    // skip , on last line
    if (iter.hasNext()) {%>,<%}%>
  <%}%>
);

<%}%>
                        

The above two problems are good examples of when you need to implement some advanced logic in the helper class.

The helper class defines a mapping of Ecore types to database types. This mapping is used in getDatabaseType method. The annotation 'databaseType' can be used to override the default type mapping for specific attributes. It can be used to define the length of VARCHAR fields. In the same way the 'nullable' annotation can be used to override the default NOT NULL declaration.

There is a design convention that each class should define an 'id' attribute, which is used in primary and foreign keys. This could have been done more general, if needed, by using annotations to define the keys, or always generate an id property even if it is not defined in the DSL model.

A built in design consideration is that class extension is realized according to the pattern Class Table Inheritance.

Hibernate Mapping

DSL Model

The domain package is the input for generation of the Hibernate mapping file.

Generated Code

(Click on the file to see the generated XML code.)

Template: Hibernate.xmljet

Hibernate has a lot of capabilities and there are many alternatives. Therefore it is not easy to build a code generator for the Hibernate mapping file that supports all alternatives. Fortunately, we don't have to do that. Once again, the Lightweight DSM approach let us implement only the features that we need for the system we develop. The Hibernate generator was developed in 3 hours. It handles many of the ordinary mapping constructions, including inheritance, different collection types, many-to-one and many-to-many relations. That proves that when we have a good toolset and examples to start from it is not a big investment or a difficult task to add another code generator.

The Hibernate generator uses the same helper as for the DDL generator, i.e. the DatabaseGenerationHelper.

  <%for (EReference ref : h.getAllOneReferences(eClass)) {%>
  
    <many-to-one name="<%=h.getName(ref)%>" column="<%=
        h.getForeignKeyName(ref)%>" 
      class="<%=h.getQualifiedName(ref.getEReferenceType())%>" />
  <%}%>
  <%for (EReference ref : h.getManyToOneReferences(eClass)) {%>
  
    <<%=h.getCollectionType(ref)%> name="<%=h.getName(ref)%>" 
      lazy="<%=h.getAnnotation(ref, "lazy", "false")%>" 
      inverse="true"
      <%if ("bag".equals(h.getCollectionType(ref))) {%>
      order-by="<%=h.getAnnotation(ref, "orderBy", "ID")%>"
      <%}%>
      cascade="<%=h.getAnnotation(ref, "cascade", "all")%>">
        <!-- use cascade="cascade-delete-orphan" to delete 
             children when parent is deleted -->
        <key column="<%=h.getForeignKeyName(eClass)%>" />
        <%if ("list".equals(h.getCollectionType(ref))) {%>
        <index column="<%=h.getDatabaseName(ref)%>_INDEX" />
        <%}%>
        <one-to-many class="<%=h.getQualifiedName(
          ref.getEReferenceType())%>" />
    </<%=h.getCollectionType(ref).toLowerCase()%>>
  <%}%>
  <%for (EReference ref : h.getManyToManyReferences(eClass)) {%>
  
    <set name="<%=h.getName(ref)%>"
      <%if (h.isInverse(ref)) {%>
      inverse="true"
      <%}%> 
      table="<%=h.getManyToManyJoinTableName(ref)%>"
      lazy="<%=h.getAnnotation(ref, "lazy", "false")%>" 
      cascade="<%=h.getAnnotation(ref, "cascade", "all")%>">
        <key column="<%=h.getForeignKeyName(eClass)%>" />
        <many-to-many
          column="<%=h.getForeignKeyName(ref.getEReferenceType())%>" 
          class="<%=h.getQualifiedName(ref.getEReferenceType())%>" />
    </set>
  <%}%>
                        

There are several built in design considerations, e.g. class extension is realized according to the pattern Class Table Inheritance, which is known in Hibernate as Table per subclass.

  <%for (EClass subClass : h.getSubClasses(eClass)) {%>
  
    <joined-subclass name="<%=h.getQualifiedName(subClass)%>" 
         table="<%=h.getDatabaseName(subClass)%>">
       <key column="<%=h.getForeignKeyName(eClass)%>" />
       <%for (EAttribute attribute : h.getAttributes(subClass)) {%>
       <property name="<%=h.getName(attribute)%>" />
       <%}%>
       <%// TODO associations also, maybe extract above into jetinc %>
    </joined-subclass>
  <%}%>
                        

Service Layer

DSL Model

A separate class LibraryService in the model defines the operations of the service. Associations to the repositories are used for delegating method implementation.

Generated Code

(Click on the files to see the generated Java code.)

The interesting part of the ServiceImpl.javajet is the generation of the methods.

A simple, but very useful, feature is that an operation in the service class in the DSL model can be tagged with 'delegate' annotation and that will result in a generated method that delegates to the referenced repository. For example the createLibrary operation is tagged with delegate=libraryRepository.create.

In this way ordinary CRUD operations can be implemented easily without any additional manual coding. CRUD operations are automated in Repository and Access Objects also.

<%for (EOperation op : h.getOperations(eClass)) {%>
    /**
     * <!-- begin-user-doc -->
     * <!-- end-user-doc -->
     * @generated
     */
     public <%=h.getTypeName(op)%> <%=h.getName(op)%>(<%=
          h.getParameterList(op)%>)<%=h.getThrows(op)%> {
        try {
  <%if (h.getAnnotation(op, "delegate") != null) {%> 
        <%if (!h.getTypeName(op).equals("void")) {%>
            return <%} else {%>
            
            <%}%><%=h.getAnnotation(op, "delegate")%>(<%
        for (Iterator<EParameter> iter = h.getParameters(op).iterator(); 
            iter.hasNext(); ) {
          EParameter parameter = iter.next(); %><%=
          h.getName(parameter)%><%if (iter.hasNext()) {%>, <%}%>
        <%}%>);
  <%} else {%>     
             // TODO Auto-generated method stub
             throw new UnsupportedOperationException("<%=
               h.getName(op)%> not implemented");
  <%}%>
         } catch (RuntimeException e) {
            LOG.error(e.getMessage(), e);
            throw e;
        } finally {
            HibernateUtil.closeSession();
        }
     }
     
<%}%>
                        

The ServiceInterface.javajet is similar to the above ServiceImpl.javajet, but it is simpler.

Further extension of the Service Layer is possible, if needed. For example:

  • Remote Facade, with EJB and/or Web Services implementations.

  • Client side proxy classes, which encapsulate different remote communication protocols. Maybe implemented as a client side Command API, with one Command for each service method.

  • Usage of DTO, so that domain model classes are not exposed to clients. Assembler for mapping of domain model objects to DTOs.