The first in a series of blog posts on the “Top 10 Principles for Code Generation”. Jack Kennedy summarizes his point of view on successful generative techniques that simplify the delivery of enterprise software, including:
1. Offer Incremental and Additive Approaches
- Developers should be able to realize the benefits of software generation on their own terms using as much or as little of a generation system as they wish
2. Extensibility is Key
- The generation system must be extendible, configurable, and customizable at every level
3. Exploit Every Asset
- The possible sources of inputs to the generation should be as diverse as possible and be expandable by developers
Read the full list here
What principles would you add to this list? What principles do you disagree with and why?
It looks like a pretty good list to me, altough I don't really support point #4:
"Support Every Developer"
IMHO you shouldn't provide too much possible methods for coding/creating new code. This will only make things confusing. I'd rather have ONE way of doing things instead of multiple. This makes it easier for people to communicate about their work.
I think text-based code generation works best, the big advantage is that the text you type can do directly into SVN. This allows you to edit and merge conflics just like you merge code.
How are you going to merge two changes in a graphical model or UML diagram?
Mod4j is a framework based around much of the same principles, it allows you to create applications very fast. Also you can extend every part of the framework. Its only text-based (at the moment) with a graphical viewer on the way. The viewer will allow you to visualize the model(s) but the underlying data is still text.
A very important point code generation frameworks should have is:
- The ability to quit at any moment, leaving understandable and readable code without dependency on the generating framework.
- The ability to mutate all the behaviour while still being able to keep generating. Not just a quick-start.
Very good points...
While the title "Support every Developer" sounds a little far reaching, the intent of this point is to communicate the position that as much as we would like every enterprise to build its software using the same technologies following the same patterns, and integrating with the same packaged applications, the fact is they do not. It would also be nice if everyone processed purchase orders using the exact same PO domain model and the same business rules, but again, the technology and business rules for each engagement are fractured. My point here is that this is not where generation stumbles, but rather, where it shines. If one shop requires UML and another wants to start with a command line interface, the generation engine, and ideally the tooling around that engine should serve the purpose of providing value to both approaches and act as a bridge between those two worlds. Skyway in no way requires the use of UML, but if you are using UML, we do seek to provide additional value to your investment. In essence, we support the idea that no matter where you start, that there is an opportunity to capitalize on the information that is being captured in any one layer by using it to synchronize and generate redundancies in other layers. I definitely support the idea that developers in every shop need a consistent approach with a set of best practices and tooling that meet their specific needs. Groups that "industrialize" their approach see great benefits in creating "one way" for their teams to deliver software. My point is that the generation technology has to support a plurality of "one ways".
This sounds a lot like: If somebody wants to use UML we'll please them are try to do our best to (kind-of) integrate it.
Personally I don't like this behaviour. I'd rather work with a company that says:
"We've thought everything through, we use command-line/textual models because "
"You want to use UML? (which is a mismatch with code generation) We'll try to give it some support..."
And yes, I think UML isn't good for code generation. That claim is based on the discussions I had with Jos Warmer. He was one of the core members that created UML, and now does a lot of work with code-generation/DSL's. He is (as one of the brains behind it) a huge fan of UML, but just not in this case. The models become to monolithic and too general. UML is used to model 'everything'. Code generation works much better if you have small specific pieces/models, no huge monolithic main model.
UML is not good for code generation, yes.
But your code generator must be good enough to take an UML model or a part of an UML model as an input.
With UML and UML profiles you can do a lot of things. The same thinks you can do, for example, with the java annotations (which is the same mechanism than the UML profiles)
Yes, that is the point we must agree to disagree on. I rather have a framework giving me one method instead of being able to (poorly) work with dozens of methods (like UML).
I'm not sure how you derived the statements you did from the content of my message, but allow me to further clarify.
I understand the point of view that states that there is a mismatch between UML and code generation. I also understand your point of view regarding your preference for working with companies that provide you with only one option.
At its core, Skyway offers a DSL and a set of generation capabilities around that DSL. We are appropriately agnostic to how the information needed by the generation engine gets into this DSL. For some developers, they prefer to design their applications visually. Other developers prefer to enter their definitions as text.
The point is that for companies who are working with UML, there are valuable design concepts that are captured in the notation that can be used during the generation process. We do not subscribe to the point of view that for these companies, that they should either abandon their investments and "rekey" these concepts into a command line interface.
Just to add to Jack's post, effective use of Profiles within UML turns standard UML into a DSL. In fact, we do this at Skyway with our UML integration. While usually an EMF ecore model contains our DSL specific data, we also have a profile for UML that captures the the DSL data. This enables us to capture the same degree of richness, regardless of the developer / architect starting with UML, our DSL, or a command line interface.
The topic of matching the input to the code was also mentioned above. EMF and JDT provides many utilities for doing this matching when the developer inevitably modifies the code directly in a way that is not matched in the model. We can reverse engineer from the code straight to our DSL and then to UML to ensure there is no loss.
Within ADP (http://www.accelerateddeliveryplatform.com/ModelDrivenArchitecture.ashx
) we have been using (a subset of) UML succesfully to generate .NET and JavaEE applications, applying smart use cases and several DDD-vocabularity and UML stereotypes, so I completely disagree that UML is not suitable for code generation. I think it a.o. depends on:
- the type of applications you want to generate
- how much of the application you want to generate
For fun and because of #4 we're also looking into a DSL based on the same smart use cases, DDD-stuff and custom UML stereotypes.
Thumbs up for the list of principles, we're currently working on several patterns for #6.
#7 should read "don't bother making your generated code readable'.
Why not publish smarter interface APIs to the generated code and just generate machine code like Java. Better still, drag programming semantics up a level and allow the developer to increasingly express his/her "intent" to the generation tool and not bother with the generated output. Initially keep the generated code to a specific domain, say serialisation, then expand to allow "straight through" processing...
The subject of whether or not the future of enterprise software development will be more "interpreted" or more "generated" probably deserves its own blog but I think it would be good to continue this part of the conversation here.
There are many who make the comparison between a "DSL" generating down to Java code, and Java generating down to byte code. Many infer from the comparison that if the DSL being used is syntactically complete enough to capture the intent of the developer, then the generated Java code is an unnecessary redundancy in the process that adds complexity without offering value.
Skyway has many clients who have built their entire application from start to finish without touching a line of Java code. For these customers and these projects, we would agree that the Java code itself could have been completely unreadable without causing an issue to the running application, and in fact, that if there were a DSL interpreter sitting on the server, that it could have just as easily "deployed" the DSL by creating what ever machine instructions were most optimal for the job.
What we have also found however is that while there are a set of developers who are comfortable with this approach, and in many cases, who love this approach, there are others who abhor it. These developers will accept "generation" as a mechanism for performing the robotic development tasks that they wish to avoid as long as the generator is doing it in the language that they know and love.
Skyway remains impartial on this subject. We certainly can sympathize with those who do not want to lose control over the ability to modify the application using the lowest human readable syntax available, since this syntax ultimately offers the widest range of controls. For this group, we simply aim to provide them with a very complete skeletal generation and an incremental step towards complete application generation. We also offer them a tooling strategy by which they can take the good hand coding that they do repetitively and create their own generations from it.
Our experiences to date have proven to us that for generation of enterprise applications to really take hold, that it will be important to prove to developers that what is being generated is of the same approach and quality that they would build themselves, and that should they wish to walk away from the generation technology all together, that they can walk away with standard Java code in hand.
+1 for readability of generated code.
Sometimes it is a matter of style other times it is more of compliance with the company's programming guidelines but I sometimes like to be able to edit the code
(e.g. if you extracted JPA entities from the database [Teacher has many Students] in Netbeans, the students' collection would be called studentCollection or studentList depending and I like to change it to students)
Losing that ability would be a big deal for me.
+1 here too
Readable code is a huge pre in code generation. In some cases clients want to have the option to stop generating and take matters into their own hands.
And what about debugging? The moment you allow people to change default behaviour they want to be able to debug the generated code.
If you have meta-data when generating, its a small step to include this in making code pretty and/or generating Javadoc. This really shows the quality of code generating frameworks IMHO.
A code generator is a compiler, starting with a (usually new) programming language, and generating code in a different language.
When using a code generator, there should be sufficient tool for working with the input language:
- support for SCM tools (easy when using a text-based format, but it also means the editor should preserve formatting)
- a way to debug based on the input language, so problems can be traced right back to the specification
- test tools
- Post-Generation Changes, #6 from the article
Of course there are exceptions; Rails for example generates code, but most of this is mostly throw-away code to get you started. I don't think the article applies in those cases.
You are absolutely correct that it is imperative that generation approaches work well with existing tools and processes.
In a separate post on this thread there is a discussion regarding the drawbacks and benefits to generating the Java code at all.
You make a good additional point for generating clean readable code in that your existing tools can be used for source control, deployment and debugging.
Our approach is to make it easy to link from the generated code to the originating model and back.
I posted a follow up to this top 10 list that adds additional detail to our point of view.
It sounds like you would support the direction to add the command line interface and the ability to script the model.
... for general principles. In some cases they may be right but not in others. E.g. "Provide tooling options that enable the widest set of development approaches and experience levels" is neither reasonable nor feasible for most code generation projects.
Thanks for the post, and agreed, this list is diverse.
This list wasn't meant to set the requirements for every code generation project as much as it was meant to be a starting point for the discussion regarding the things that are important in the software generation space. You are absolutely correct that providing these tooling options is a significant undertaking. One of our goals at Skyway is to provide a starting point for this type of tooling in an effort to make it both feasible and reasonable for a wider audience.
7 Avoid Lockin : Generated artifacts should be indistinguishable from those that would be written by hand by a developer following best practices...
Seems to be conceptually impossible.
How do you manage this.
Im my UML model, I've a Class A, this class has an attribute xx. I generate the Java code. Then, in my UML model, I rename my attribute into yy. I try to generate the code again.
What the generator will do ?
To be really "intelligent" you've 2 solutions :
- Generate a code with a link to the model using somethink like the UML UID of the element (Rational Software Architect do that)
- Consider the Java code as your UML meta-model (Together control Center do that)
I cannot see other solutions but.... ?