Les A. Hazlewood has written an article for OnJava called "An Ant Modular Build Environment for Enterprise Applications," describing a J2EE build structure many are probably familiar with, and he describes it very simply.

One of his points covers the use of a /lib directory to hold dependent jar files, for various reasons.

While it's not an article likely to inspire massive changes in how projects are organized (since many projects are already doing things similarly), it's written for those to whom such things might not be quite so obvious. It also shows either how Ant is used to build modules (as Maven does) or, if you like, a possible inspiration for Maven itself, since Maven attempts to lighten the burden of creating this kind of organization structure.

What do you think? What advice would you have on organizing project structures? What works and what doesn't?

It looks nice.. It is well explained, but Iam missing alot of stuff I normally have in a project.. like junit and dbunit tasks, db related tasks (like create db, load db etc..)

lib creation for third parties etc.... And of course all related clean, compile comnpile all compile test etc.. tasks that makes your buildfile a living nightmare...

But i found it a nice start....

What I find particularly interesting in this build setup is that there are both explicitly defined dependencies (as <path> elements in the module buildfiles) and also an explicitly defined build order (in the "template" target) in the buildfile of the top-level module (and in modules with sub-modules).

First I thought this was only for pragmatic reasons. Ant does not deliver out-of-the-box modular build functionality. Nor does the build order have to be inferred from the dependencies.

But then the author goes on and explains that the build order is there to prevent modules from declaring "illegal" dependencies. I don't think I've seen this done like this before.

It's a good start for enterprise builds, but I've taken mine further in 2 spots:

1) Using Ant 1.6 you can make your build files modular too. I've got a common-parent-build.xml and common-build.xml which template out the build of a parent module (and the root project build) and an artifact module respectively. My actual module build files are very small (around 5-10 lines) because they just import the common build files.

2) Dependency management gets to be hairy, and although the method listed here will work, it misses many advantages from a documentation and 3rd party reuse point of view. I've moved to using Ivy (http://jayasoft.org/ivy) for all dependency management. My modules declare all of their dependencies, including inter-module dependencies. When an artifact module builds it publishes its artifact to a local integration repository where the other modules can get them when resolving dependencies. 3rd party dependencies are stored in a separate version control repository and published to a web-site where Ivy can download them. This allows you to reuse the same dependencies across projects (not just modules) and keeps your sync times for your project down since it doesn't have to sync libraries.

I haven't use the Maven myself except seen it in XDoclet When I have to modify some XDoclet template.

But I have read the following from Cafe au lait Java news
http://www.cafeaulait.org/

<quote>

Friday, June 17, 2005
The Apache Software Foundation has posted the first beta of Maven 1.1, an open source build tool for Java that's more declarative and less procedural than Ant. Maven is also much more authoritarian and less configurable than Ant.
.....

I've been using Maven lately as part of my work with the Jaxen project, and it has some nice features; but I really can't recommend it. It's too controlling. If your build process doesn't look like what Maven wants it to look like, you're going to be fighting against it. Maven's probably a little easier to set up than Ant for basic tasks like build, test, and deploy. However, as soon as you want to do something a little different than Maven expects, you're S.O.L. It's really Maven's way or the highway.
</quote>

Chester

It's a good start for enterprise builds, but I've taken mine further in 2 spots:1) Using Ant 1.6 you can make your build files modular too. I've got a common-parent-build.xml and common-build.xml

I think this can work, up to a point. I've had some problems trying to create really clean and modular build scripts using the 1.6 features, as even though import and macrodef are nice, they are still pretty limited. Trying to do complex conditional-based handling in Ant build scripts can get messy.

For dependency management, I'm looking at Maven, which I've used before and quite liked. It would be nice to be able to break out the dependency management side of things completely from Maven and utilize them from vanilla Ant (maybe encapsulated as taskdefs). I've seen that the Maven guys have written a few Ant tasks along these lines.

Cool article. I also found out another similar ant configuration, which I think addresses more things. It is from Erik Hatcher, the author of "Java Development with Ant", and it's here: http://www.ehatchersolutions.com/JavaDevWithAnt/

Cool article. I also found out another similar ant configuration, which I think addresses more things. It is from Erik Hatcher, the author of "Java Development with Ant", and it's here: http://www.ehatchersolutions.com/JavaDevWithAnt/

yes, that was how we wrote the stuff for ant1.5, back in 2002.

You should know that the 2nd edition of the book is being written (about 1/3 of the way through, don't hold your breath), and it will improve on big file builds with:

1. use of the maven ant tasks to do library management. that is for importing external stuff, and for publishing stuff locally.

2. exploring the newer ant1.6+ stuff of <subant>, <import> and <macrodef>

<import> is profound, it adds basic OO-like subclassing to a build. So you have a common.xml file that defines all your main targets (compile, test, deploy), then child projects can run with the default, or override them.

<subant> is another useful little beast; it lets you pass a fileset down of directories to run, such as */build.xml.

I will be talking about this at apachecon 2005 in stuttgard next month; expect to see the slides up online at the beginning of august.

steve loughran
co author, Java Development with Ant.

I think this can work, up to a point. I've had some problems trying to create really clean and modular build scripts using the 1.6 features, as even though import and macrodef are nice, they are still pretty limited. Trying to do complex conditional-based handling in Ant build scripts can get messy.

I haven't run into these situations yet, and I can't see where they'd come from. I keep one module = one artifact and keep the module layout the same, so it's all pretty much the same.

For dependency management, I'm looking at Maven, which I've used before and quite liked. It would be nice to be able to break out the dependency management side of things completely from Maven and utilize them from vanilla Ant (maybe encapsulated as taskdefs). I've seen that the Maven guys have written a few Ant tasks along these lines.

Maven's dependency management, even in Maven 2, is still significantly behind the full transitive dependency resolution that Ivy gives. Give it a look and see what you think. I highly recommend it.

It's a good start for enterprise builds, but I've taken mine further in 2 spots:

1) Using Ant 1.6 you can make your build files modular too. I've got a common-parent-build.xml and common-build.xml which template out the build of a parent module (and the root project build) and an artifact module respectively. My actual module build files are very small (around 5-10 lines) because they just import the common build files.

This is one of the great things about Ant 1.6. Between import and macrodef, it becomes very easy to use a standard ant library script across a whole organisation, and just apply a small amount of customisation per project.

2) Dependency management gets to be hairy, and although the method listed here will work, it misses many advantages from a documentation and 3rd party reuse point of view. I've moved to using Ivy (http://jayasoft.org/ivy) for all dependency management. My modules declare all of their dependencies, including inter-module dependencies. When an artifact module builds it publishes its artifact to a local integration repository where the other modules can get them when resolving dependencies. 3rd party dependencies are stored in a separate version control repository and published to a web-site where Ivy can download them. This allows you to reuse the same dependencies across projects (not just modules) and keeps your sync times for your project down since it doesn't have to sync libraries.

Actually, I find pure Ant 1.6 fully able to handle inter-project dependecies very well (including transitive dependencies), without having to use any third party dependency management add-ons. I wrote about this technique in this article: Project Dependencies Using Ant.

Actually, I find pure Ant 1.6 fully able to handle inter-project dependecies very well (including transitive dependencies), without having to use any third party dependency management add-ons. I wrote about this technique in this article: Project Dependencies Using Ant.

I don't think your solution handles transitive dependencies very well. What if 'web' requires log4j-1.2.8.jar but 'model' is using log4j-1.2.9? Which one will be included in the dist? Will it include both? Also, a lot of the time your 3rd party jars will have dependencies of their own that you need to resolve.

This is the sort of thing that makes Ivy a very useful tool.

Kim

Actually, I find pure Ant 1.6 fully able to handle inter-project dependecies very well (including transitive dependencies), without having to use any third party dependency management add-ons. I wrote about this technique in this article: Project Dependencies Using Ant.

I don't think your solution handles transitive dependencies very well. What if 'web' requires log4j-1.2.8.jar but 'model' is using log4j-1.2.9? Which one will be included in the dist? Will it include both? Also, a lot of the time your 3rd party jars will have dependencies of their own that you need to resolve.

In that particular example, the developer/architect should be asking some very hard questions as to why multiple versions of a single library are required within a single software system.

I view management of third-party jar dependencies as an orthogonal issue to management of inter project/module dependencies. The article purposefully doesn't mention the subject to keep a clear focus. However, there are a number of strategies you could use, including using tasks to pull dependencies from external repositories (e.g Ivy or the new repository tasks in Ant 1.7), or manage them as versioned artifacts within the system.

Personally, I prefer the latter option. The linked article uses this technique, and gives a number of very good reasons for this. I'll quote Mr Hazlewood:

The lib/ directory is a little special. It contains all third-party .jars required to either build or run the application. We keep all third-party .jars used by our modules in this directory, instead of in the modules themselves, for three reasons:

1. It's easier to manage third-party dependencies from a single location. Whether or not a module uses one of these libraries is defined in a module-specific Ant <path> entry in the module's build.xml file.

2. It avoids classloading or API version conflicts by eliminating the possibility of duplicate .jars. If more than one module uses Jakarta Commons Logging, which module is responsible for storing the commons-logging.jar file? If each stored their own copy, there is the potential that one module might have one version and another module might have a different version. When the application is being run, only the first .jar file found in the classpath is used to satisfy the dependency, potentially causing a conflict for the other module. We avoid that by managing only one .jar at the root level.

3. Third-party dependencies are versioned with your source code. Often overlooked in many projects, this is the most important reason why you want to store your dependency libraries in a RCS. By doing this, you ensure that no matter what version or branch of your software you check out, you'll always have the proper versions of third-party libraries needed to run that particular version of your software.

Cheers,
Joe

I agree with Joe concerning transitive dependencies as orthogonal to inter project/module dependencies.

The article was written as an overview of an environment that has worked quite well for a long time and could be of use to many development shops out there. It was certainly not saying "this is the best way" (read the sentence right before the Caveats).

It was also written to enable people wo don't want to use anything other than Ant to create an easily managed build environment. There's a whole political issue about which build tool to use, and I definitely won't get in to that. The article is geared towards folks who already know ant, are comfortable with it, and want a clean modular system without having to learn other tools. The intent is to use what you know and get you up and running fast if you have an enterprise project around the corner. I certainly encourage others to evaluate other tools and methods. At the end of the day, its about leveraging what you know vs. spending time to learn other things (which may or may not be marginally better).

The example application accompanying the article does use the newer <import> task to consolidate definitions across modules and submodules. Because of this, module and submodule ant build.xml files are usually very succinct, since most behavior is "import"ed from the root build file.

Best regards to all,

Les

Take a look at the demo from Termiit (http://www.termiit.com/termiit/screenshots.php) - it's a real CASE tool for java builds.

Ant is very good cross-platform scripting language, but not a 'easily managed build environment'.

Great article, but I have to ask...Why not use Maven? It looks to me that the author took several of Maven's core concepts and figured out how to do the same things with Ant. I'll give the benefit of the doubt and assume that this is a case of "all roads lead to Rome"...maybe the project pre-existed Maven and we all eventually come to the same conclusion/best practices.

I am curious though, are there any advantages that I'm not aware of to using Ant over Maven in an enterprise application such as this?

I think Apache's Maven Project does a good job of standardizing your build, unit, and integration test process.

I just started using it about a month ago for several projects, and its fantastic.

You know - another small thing about Maven that actually helps is the documentation generation. I have found it to be EXTREMELY useful, especially the metrics section.

-Michael

I agree with you in that maven does a much better job in the aspects you mentioned. However, Maven 1 is bound to disapear and Maven 2 is still on alpha versions. I have used Maven in a big project and i really liked it. Im planning to use it more, but when Maven 2 is out.

It cannot be denied that Maven is the way forward

I think that Maven is a great evolution of the modular build idea. Ant originally allowed people to create their own custom modular builds but Maven standardizes that modularity by using plugins. I'm glad to see the Maven project picking up steam with the new 2.0 technology release and I think Maven will start to become even more popular as more and more people learn about how much time it can save them. Of course, Maven is built on top of Ant so I'm not saying it's a replacement but rather a really nice complement to Ant.

I think that Maven is a great evolution of the modular build idea. Ant originally allowed people to create their own custom modular builds but Maven standardizes that modularity by using plugins. I'm glad to see the Maven project picking up steam with the new 2.0 technology release and I think Maven will start to become even more popular as more and more people learn about how much time it can save them. Of course, Maven is built on top of Ant so I'm not saying it's a replacement but rather a really nice complement to Ant. I'm not sure if trying to recreate what Maven has already done is the way to go. Rather, why not contribute and continue to evolve the maven project?

Readers of your article may be interested in looking into JAM http://www.javagen.com/jam/. JAM is a mature modular Ant build framework that allows sophisticated test-driven J2EE builds to be created by importing pre-written modules. Highlights include:

  Standard set of build targets: gen, compile, test, dist, deploy, undeploy
  Standard targets are easy to override and extend
  Resources and dependencies are managed in a high-level declarative manner
  Extensive XDoclet support
  Built in unit and integration testing including Cactus support
  Modules for JMX Hibernate EJBs JUnit TestNG WebDAV Axis CVS etc…
  Interchangeable application servers modules

A unique feature of JAM is the use of Maven POM files to handle the project-specific aspects of the build. Once the POM (project.xml) file is specified running ‘maven jam’ generates Ant equivalents that manage classpaths and resources allowing work to proceed in a pure-Ant environment. The dependencies can be stored in the Maven repository or copied to the local ‘lib’ directory. Like Maven, sub-modules are single- artifact-per-build, but it is possible to generate multiple artifacts by overriding the ‘dist’ target, for example for generating client and server ejb jars. A typical JAM build file consists of about 30 lines of Ant code - mostly import statements followed by a few overridden standard targets.

i dont think so ..

A very nice introductory article... Thanks!

Similar approach: EJOSA - article series:
http://www.jaxmagazine.com/itr/news/psecom,id,21908,nodeid,146.html

To be able to standardized the ant build/execution files we generate them with AndroMDA (we designed the modules with UML and generate them with AndroMDA)... Mixing the latest capability of Ant (inheritance, etc.) with generator is a great possibility (think of generating miscellaneous Ant files for different application servers...)

Cheers,
Lofi.
EJOSA + OpenUSS

Its a decent article. The whole "modular", "hierarchical" and "artifact driven" of course not being revelations but it is useful to see an example via ant. Maven is great too, but the point is more that you should get to those goals and this is an example of doing it with ant (which some people are more comfortable with, and with which it can be done), rather than what build tool is better.

And just a hint, even with ant <= 1.5 you can use AntContrib iterators and a props file to decide what modules to call, rather than putting the process and order actually in the build file.

From the article:

A module is a logically aggregated unit of functionality that corresponds to a named feature in a system.
[...]
An artifact-driven build is one where each module or submodule exists for the purpose of generating a single deployable artifact.

With Les Hazelwood's definition of a module, I haven't been in any project, where this assumption is true. Usually we have _many_ artifacts per module. For example a server-side JAR file, a client JAR for accessing server-side functionality, a JAR containing the tests for a module. AFAIK, maven also has this "philosophy" of a single artifact per module. This was one of the reasons I never used maven, since it simply does not fit me needs.
Further, I wouldn't call a "Hazelwood-module" a module. IMHO it's a subsystem or component: to me a module is a physical unit, not a logical.

Just my 2Cents,
     Dirk

From the article:

A module is a logically aggregated unit of functionality that corresponds to a named feature in a system.[...]An artifact-driven build is one where each module or submodule exists for the purpose of generating a single deployable artifact.

With Les Hazelwood's definition of a module, I haven't been in any project, where this assumption is true. Usually we have _many_ artifacts per module. For example a server-side JAR file, a client JAR for accessing server-side functionality, a JAR containing the tests for a module. AFAIK, maven also has this "philosophy" of a single artifact per module. This was one of the reasons I never used maven, since it simply does not fit me needs.Further, I wouldn't call a "Hazelwood-module" a module. IMHO it's a subsystem or component: to me a module is a physical unit, not a logical.

Just my 2Cents,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Dirk

It sounds like you have a module with submodules then. I don't see how that can't be accomplished with the proposed approach.

It sounds like you have a module with submodules then.

I think, that I don't, because I have physical modules, that do not contain disjunct artifact sets (e.g. client JARs contain EJB remote interfaces, as well as the server JARs do).

Having read the article once again, I'm quite confused what Hazelwood actually means by "module". On the one hand he says, that a module is a "logically aggregated unit of functionality", on the other hand he imposes a directory structure, that is, IMHO, clearly related to _physical_ modules (JARs, or whatever).

Regards,
    Dirk

Agree this is a good introduction to build organization.

One thing that I've found useful in the submodule organization is to separate sources into src/java, src/ejb, etc. This allows you to handle categories of source code differently. This is also how many open source projects are working.

If you find Maven too heavy and your Ant build files difficult to maintain, you may want to look at EL4Ant http://el4ant.sourceforge.net/