Les A. Hazlewood has written an article for OnJava called "An Ant Modular Build Environment for Enterprise Applications," describing a J2EE build structure many are probably familiar with, and he describes it very simply.
One of his points covers the use of a /lib directory to hold dependent jar files, for various reasons.
While it's not an article likely to inspire massive changes in how projects are organized (since many projects are already doing things similarly), it's written for those to whom such things might not be quite so obvious. It also shows either how Ant is used to build modules (as Maven does) or, if you like, a possible inspiration for Maven itself, since Maven attempts to lighten the burden of creating this kind of organization structure.
What do you think? What advice would you have on organizing project structures? What works and what doesn't?
-
A Modular Build for Ant (27 messages)
- Posted by: Joseph Ottinger
- Posted on: June 23 2005 08:29 EDT
Threaded Messages (27)
- Not complete enough.... by Marc de Kwant on June 23 2005 09:51 EDT
- dependencies AND build order by Knut Wannheden on June 24 2005 10:56 EDT
- A Modular Build for Ant by Jason Carreira on June 23 2005 10:27 EDT
- Ant vs Maven by Chester Chen on June 23 2005 11:19 EDT
- A Modular Build for Ant by rory Winston on June 23 2005 12:51 EDT
-
A Modular Build for Ant by Guillermo Castro on June 23 2005 12:59 EDT
- 2nd edition will be enhanced for ant1.7 by Steve Loughran on June 23 2005 06:26 EDT
- A Modular Build for Ant by Jason Carreira on June 23 2005 02:37 EDT
-
A Modular Build for Ant by Guillermo Castro on June 23 2005 12:59 EDT
- A Modular Build for Ant by Joe Schmetzer on June 23 2005 15:36 EDT
-
A Modular Build for Ant by kim pepper on June 23 2005 07:15 EDT
-
A Modular Build for Ant by Joe Schmetzer on June 23 2005 08:34 EDT
-
A Modular Build for Ant by Les Hazlewood on June 23 2005 09:01 EDT
- Take a look at Termiit(EAP) by Sergey Vasiljev on June 24 2005 05:36 EDT
-
A Modular Build for Ant by Les Hazlewood on June 23 2005 09:01 EDT
-
A Modular Build for Ant by Joe Schmetzer on June 23 2005 08:34 EDT
-
A Modular Build for Ant by kim pepper on June 23 2005 07:15 EDT
- A Modular Build for Ant by Thad Smith on June 23 2005 10:57 EDT
- A Modular Build for Ant - That would be Maven by Michael Dowling on June 23 2005 11:32 EDT
- But Maven needs a bit of stabilising by Galder Zamarre??o on June 24 2005 04:40 EDT
- A Modular Build for Ant by Eric Roberts on June 23 2005 12:17 EDT
- A Modular Build for Ant by Eric Roberts on June 23 2005 12:18 EDT
- JAM Build Framework by Richard Easterling on June 23 2005 20:47 EDT
- heard about maven by sean decor on June 24 2005 00:13 EDT
- Nice article... by Lofi Dewanto on June 24 2005 00:51 EDT
- A Modular Build for Ant by Charlie Collins on June 27 2005 10:27 EDT
- A Modular Build for Ant by Dirk Ludwig on June 28 2005 06:10 EDT
- A Modular Build for Ant by Donald Diego on June 28 2005 12:11 EDT
- A Modular Build for Ant by Dirk Ludwig on June 29 2005 04:15 EDT
- A Modular Build for Ant by Donald Diego on June 28 2005 12:11 EDT
- A Modular Build for Ant by Donald Diego on June 28 2005 12:23 EDT
- Between Ant and Maven by Yves Martin on November 17 2005 04:36 EST
-
Not complete enough....[ Go to top ]
- Posted by: Marc de Kwant
- Posted on: June 23 2005 09:51 EDT
- in response to Joseph Ottinger
It looks nice.. It is well explained, but Iam missing alot of stuff I normally have in a project.. like junit and dbunit tasks, db related tasks (like create db, load db etc..)
lib creation for third parties etc.... And of course all related clean, compile comnpile all compile test etc.. tasks that makes your buildfile a living nightmare...
But i found it a nice start.... -
dependencies AND build order[ Go to top ]
- Posted by: Knut Wannheden
- Posted on: June 24 2005 10:56 EDT
- in response to Marc de Kwant
What I find particularly interesting in this build setup is that there are both explicitly defined dependencies (as <path> elements in the module buildfiles) and also an explicitly defined build order (in the "template" target) in the buildfile of the top-level module (and in modules with sub-modules).
First I thought this was only for pragmatic reasons. Ant does not deliver out-of-the-box modular build functionality. Nor does the build order have to be inferred from the dependencies.
But then the author goes on and explains that the build order is there to prevent modules from declaring "illegal" dependencies. I don't think I've seen this done like this before. -
A Modular Build for Ant[ Go to top ]
- Posted by: Jason Carreira
- Posted on: June 23 2005 10:27 EDT
- in response to Joseph Ottinger
It's a good start for enterprise builds, but I've taken mine further in 2 spots:
1) Using Ant 1.6 you can make your build files modular too. I've got a common-parent-build.xml and common-build.xml which template out the build of a parent module (and the root project build) and an artifact module respectively. My actual module build files are very small (around 5-10 lines) because they just import the common build files.
2) Dependency management gets to be hairy, and although the method listed here will work, it misses many advantages from a documentation and 3rd party reuse point of view. I've moved to using Ivy (http://jayasoft.org/ivy) for all dependency management. My modules declare all of their dependencies, including inter-module dependencies. When an artifact module builds it publishes its artifact to a local integration repository where the other modules can get them when resolving dependencies. 3rd party dependencies are stored in a separate version control repository and published to a web-site where Ivy can download them. This allows you to reuse the same dependencies across projects (not just modules) and keeps your sync times for your project down since it doesn't have to sync libraries. -
Ant vs Maven[ Go to top ]
- Posted by: Chester Chen
- Posted on: June 23 2005 11:19 EDT
- in response to Jason Carreira
I haven't use the Maven myself except seen it in XDoclet When I have to modify some XDoclet template.
But I have read the following from Cafe au lait Java news
http://www.cafeaulait.org/
<quote>
Friday, June 17, 2005
The Apache Software Foundation has posted the first beta of Maven 1.1, an open source build tool for Java that's more declarative and less procedural than Ant. Maven is also much more authoritarian and less configurable than Ant.
.....
I've been using Maven lately as part of my work with the Jaxen project, and it has some nice features; but I really can't recommend it. It's too controlling. If your build process doesn't look like what Maven wants it to look like, you're going to be fighting against it. Maven's probably a little easier to set up than Ant for basic tasks like build, test, and deploy. However, as soon as you want to do something a little different than Maven expects, you're S.O.L. It's really Maven's way or the highway.
</quote>
Chester -
A Modular Build for Ant[ Go to top ]
- Posted by: rory Winston
- Posted on: June 23 2005 12:51 EDT
- in response to Jason Carreira
It's a good start for enterprise builds, but I've taken mine further in 2 spots:1) Using Ant 1.6 you can make your build files modular too. I've got a common-parent-build.xml and common-build.xml
I think this can work, up to a point. I've had some problems trying to create really clean and modular build scripts using the 1.6 features, as even though import and macrodef are nice, they are still pretty limited. Trying to do complex conditional-based handling in Ant build scripts can get messy.
For dependency management, I'm looking at Maven, which I've used before and quite liked. It would be nice to be able to break out the dependency management side of things completely from Maven and utilize them from vanilla Ant (maybe encapsulated as taskdefs). I've seen that the Maven guys have written a few Ant tasks along these lines. -
A Modular Build for Ant[ Go to top ]
- Posted by: Guillermo Castro
- Posted on: June 23 2005 12:59 EDT
- in response to rory Winston
Cool article. I also found out another similar ant configuration, which I think addresses more things. It is from Erik Hatcher, the author of "Java Development with Ant", and it's here: http://www.ehatchersolutions.com/JavaDevWithAnt/ -
2nd edition will be enhanced for ant1.7[ Go to top ]
- Posted by: Steve Loughran
- Posted on: June 23 2005 18:26 EDT
- in response to Guillermo Castro
Cool article. I also found out another similar ant configuration, which I think addresses more things. It is from Erik Hatcher, the author of "Java Development with Ant", and it's here: http://www.ehatchersolutions.com/JavaDevWithAnt/
yes, that was how we wrote the stuff for ant1.5, back in 2002.
You should know that the 2nd edition of the book is being written (about 1/3 of the way through, don't hold your breath), and it will improve on big file builds with:
1. use of the maven ant tasks to do library management. that is for importing external stuff, and for publishing stuff locally.
2. exploring the newer ant1.6+ stuff of <subant>, <import> and <macrodef>
<import> is profound, it adds basic OO-like subclassing to a build. So you have a common.xml file that defines all your main targets (compile, test, deploy), then child projects can run with the default, or override them.
<subant> is another useful little beast; it lets you pass a fileset down of directories to run, such as */build.xml.
I will be talking about this at apachecon 2005 in stuttgard next month; expect to see the slides up online at the beginning of august.
steve loughran
co author, Java Development with Ant. -
A Modular Build for Ant[ Go to top ]
- Posted by: Jason Carreira
- Posted on: June 23 2005 14:37 EDT
- in response to rory Winston
I think this can work, up to a point. I've had some problems trying to create really clean and modular build scripts using the 1.6 features, as even though import and macrodef are nice, they are still pretty limited. Trying to do complex conditional-based handling in Ant build scripts can get messy.
I haven't run into these situations yet, and I can't see where they'd come from. I keep one module = one artifact and keep the module layout the same, so it's all pretty much the same.For dependency management, I'm looking at Maven, which I've used before and quite liked. It would be nice to be able to break out the dependency management side of things completely from Maven and utilize them from vanilla Ant (maybe encapsulated as taskdefs). I've seen that the Maven guys have written a few Ant tasks along these lines.
Maven's dependency management, even in Maven 2, is still significantly behind the full transitive dependency resolution that Ivy gives. Give it a look and see what you think. I highly recommend it. -
A Modular Build for Ant[ Go to top ]
- Posted by: Joe Schmetzer
- Posted on: June 23 2005 15:36 EDT
- in response to Jason Carreira
It's a good start for enterprise builds, but I've taken mine further in 2 spots:
1) Using Ant 1.6 you can make your build files modular too. I've got a common-parent-build.xml and common-build.xml which template out the build of a parent module (and the root project build) and an artifact module respectively. My actual module build files are very small (around 5-10 lines) because they just import the common build files.
This is one of the great things about Ant 1.6. Between import and macrodef, it becomes very easy to use a standard ant library script across a whole organisation, and just apply a small amount of customisation per project.2) Dependency management gets to be hairy, and although the method listed here will work, it misses many advantages from a documentation and 3rd party reuse point of view. I've moved to using Ivy (http://jayasoft.org/ivy) for all dependency management. My modules declare all of their dependencies, including inter-module dependencies. When an artifact module builds it publishes its artifact to a local integration repository where the other modules can get them when resolving dependencies. 3rd party dependencies are stored in a separate version control repository and published to a web-site where Ivy can download them. This allows you to reuse the same dependencies across projects (not just modules) and keeps your sync times for your project down since it doesn't have to sync libraries.
Actually, I find pure Ant 1.6 fully able to handle inter-project dependecies very well (including transitive dependencies), without having to use any third party dependency management add-ons. I wrote about this technique in this article: Project Dependencies Using Ant. -
A Modular Build for Ant[ Go to top ]
- Posted by: kim pepper
- Posted on: June 23 2005 19:15 EDT
- in response to Joe Schmetzer
Actually, I find pure Ant 1.6 fully able to handle inter-project dependecies very well (including transitive dependencies), without having to use any third party dependency management add-ons. I wrote about this technique in this article: Project Dependencies Using Ant.
I don't think your solution handles transitive dependencies very well. What if 'web' requires log4j-1.2.8.jar but 'model' is using log4j-1.2.9? Which one will be included in the dist? Will it include both? Also, a lot of the time your 3rd party jars will have dependencies of their own that you need to resolve.
This is the sort of thing that makes Ivy a very useful tool.
Kim -
A Modular Build for Ant[ Go to top ]
- Posted by: Joe Schmetzer
- Posted on: June 23 2005 20:34 EDT
- in response to kim pepper
Actually, I find pure Ant 1.6 fully able to handle inter-project dependecies very well (including transitive dependencies), without having to use any third party dependency management add-ons. I wrote about this technique in this article: Project Dependencies Using Ant.
I don't think your solution handles transitive dependencies very well. What if 'web' requires log4j-1.2.8.jar but 'model' is using log4j-1.2.9? Which one will be included in the dist? Will it include both? Also, a lot of the time your 3rd party jars will have dependencies of their own that you need to resolve.
In that particular example, the developer/architect should be asking some very hard questions as to why multiple versions of a single library are required within a single software system.
I view management of third-party jar dependencies as an orthogonal issue to management of inter project/module dependencies. The article purposefully doesn't mention the subject to keep a clear focus. However, there are a number of strategies you could use, including using tasks to pull dependencies from external repositories (e.g Ivy or the new repository tasks in Ant 1.7), or manage them as versioned artifacts within the system.
Personally, I prefer the latter option. The linked article uses this technique, and gives a number of very good reasons for this. I'll quote Mr Hazlewood:The lib/ directory is a little special. It contains all third-party .jars required to either build or run the application. We keep all third-party .jars used by our modules in this directory, instead of in the modules themselves, for three reasons:
1. It's easier to manage third-party dependencies from a single location. Whether or not a module uses one of these libraries is defined in a module-specific Ant <path> entry in the module's build.xml file.
2. It avoids classloading or API version conflicts by eliminating the possibility of duplicate .jars. If more than one module uses Jakarta Commons Logging, which module is responsible for storing the commons-logging.jar file? If each stored their own copy, there is the potential that one module might have one version and another module might have a different version. When the application is being run, only the first .jar file found in the classpath is used to satisfy the dependency, potentially causing a conflict for the other module. We avoid that by managing only one .jar at the root level.
3. Third-party dependencies are versioned with your source code. Often overlooked in many projects, this is the most important reason why you want to store your dependency libraries in a RCS. By doing this, you ensure that no matter what version or branch of your software you check out, you'll always have the proper versions of third-party libraries needed to run that particular version of your software.
Cheers,
Joe -
A Modular Build for Ant[ Go to top ]
- Posted by: Les Hazlewood
- Posted on: June 23 2005 21:01 EDT
- in response to Joe Schmetzer
I agree with Joe concerning transitive dependencies as orthogonal to inter project/module dependencies.
The article was written as an overview of an environment that has worked quite well for a long time and could be of use to many development shops out there. It was certainly not saying "this is the best way" (read the sentence right before the Caveats).
It was also written to enable people wo don't want to use anything other than Ant to create an easily managed build environment. There's a whole political issue about which build tool to use, and I definitely won't get in to that. The article is geared towards folks who already know ant, are comfortable with it, and want a clean modular system without having to learn other tools. The intent is to use what you know and get you up and running fast if you have an enterprise project around the corner. I certainly encourage others to evaluate other tools and methods. At the end of the day, its about leveraging what you know vs. spending time to learn other things (which may or may not be marginally better).
The example application accompanying the article does use the newer <import> task to consolidate definitions across modules and submodules. Because of this, module and submodule ant build.xml files are usually very succinct, since most behavior is "import"ed from the root build file.
Best regards to all,
Les -
Take a look at Termiit(EAP)[ Go to top ]
- Posted by: Sergey Vasiljev
- Posted on: June 24 2005 05:36 EDT
- in response to Les Hazlewood
Take a look at the demo from Termiit (http://www.termiit.com/termiit/screenshots.php) - it's a real CASE tool for java builds.
Ant is very good cross-platform scripting language, but not a 'easily managed build environment'. -
A Modular Build for Ant[ Go to top ]
- Posted by: Thad Smith
- Posted on: June 23 2005 10:57 EDT
- in response to Joseph Ottinger
Great article, but I have to ask...Why not use Maven? It looks to me that the author took several of Maven's core concepts and figured out how to do the same things with Ant. I'll give the benefit of the doubt and assume that this is a case of "all roads lead to Rome"...maybe the project pre-existed Maven and we all eventually come to the same conclusion/best practices.
I am curious though, are there any advantages that I'm not aware of to using Ant over Maven in an enterprise application such as this? -
A Modular Build for Ant - That would be Maven[ Go to top ]
- Posted by: Michael Dowling
- Posted on: June 23 2005 11:32 EDT
- in response to Joseph Ottinger
I think Apache's Maven Project does a good job of standardizing your build, unit, and integration test process.
I just started using it about a month ago for several projects, and its fantastic.
You know - another small thing about Maven that actually helps is the documentation generation. I have found it to be EXTREMELY useful, especially the metrics section.
-Michael -
But Maven needs a bit of stabilising[ Go to top ]
- Posted by: Galder Zamarre??o
- Posted on: June 24 2005 04:40 EDT
- in response to Michael Dowling
I agree with you in that maven does a much better job in the aspects you mentioned. However, Maven 1 is bound to disapear and Maven 2 is still on alpha versions. I have used Maven in a big project and i really liked it. Im planning to use it more, but when Maven 2 is out.
It cannot be denied that Maven is the way forward -
A Modular Build for Ant[ Go to top ]
- Posted by: Eric Roberts
- Posted on: June 23 2005 12:17 EDT
- in response to Joseph Ottinger
I think that Maven is a great evolution of the modular build idea. Ant originally allowed people to create their own custom modular builds but Maven standardizes that modularity by using plugins. I'm glad to see the Maven project picking up steam with the new 2.0 technology release and I think Maven will start to become even more popular as more and more people learn about how much time it can save them. Of course, Maven is built on top of Ant so I'm not saying it's a replacement but rather a really nice complement to Ant. -
A Modular Build for Ant[ Go to top ]
- Posted by: Eric Roberts
- Posted on: June 23 2005 12:18 EDT
- in response to Joseph Ottinger
I think that Maven is a great evolution of the modular build idea. Ant originally allowed people to create their own custom modular builds but Maven standardizes that modularity by using plugins. I'm glad to see the Maven project picking up steam with the new 2.0 technology release and I think Maven will start to become even more popular as more and more people learn about how much time it can save them. Of course, Maven is built on top of Ant so I'm not saying it's a replacement but rather a really nice complement to Ant. I'm not sure if trying to recreate what Maven has already done is the way to go. Rather, why not contribute and continue to evolve the maven project? -
JAM Build Framework[ Go to top ]
- Posted by: Richard Easterling
- Posted on: June 23 2005 20:47 EDT
- in response to Joseph Ottinger
Readers of your article may be interested in looking into JAM http://www.javagen.com/jam/. JAM is a mature modular Ant build framework that allows sophisticated test-driven J2EE builds to be created by importing pre-written modules. Highlights include:
Standard set of build targets: gen, compile, test, dist, deploy, undeploy
Standard targets are easy to override and extend
Resources and dependencies are managed in a high-level declarative manner
Extensive XDoclet support
Built in unit and integration testing including Cactus support
Modules for JMX Hibernate EJBs JUnit TestNG WebDAV Axis CVS etc…
Interchangeable application servers modules
A unique feature of JAM is the use of Maven POM files to handle the project-specific aspects of the build. Once the POM (project.xml) file is specified running ‘maven jam’ generates Ant equivalents that manage classpaths and resources allowing work to proceed in a pure-Ant environment. The dependencies can be stored in the Maven repository or copied to the local ‘lib’ directory. Like Maven, sub-modules are single- artifact-per-build, but it is possible to generate multiple artifacts by overriding the ‘dist’ target, for example for generating client and server ejb jars. A typical JAM build file consists of about 30 lines of Ant code - mostly import statements followed by a few overridden standard targets. -
heard about maven[ Go to top ]
- Posted by: sean decor
- Posted on: June 24 2005 00:13 EDT
- in response to Joseph Ottinger
i dont think so .. -
Nice article...[ Go to top ]
- Posted by: Lofi Dewanto
- Posted on: June 24 2005 00:51 EDT
- in response to Joseph Ottinger
A very nice introductory article... Thanks!
Similar approach: EJOSA - article series:
http://www.jaxmagazine.com/itr/news/psecom,id,21908,nodeid,146.html
To be able to standardized the ant build/execution files we generate them with AndroMDA (we designed the modules with UML and generate them with AndroMDA)... Mixing the latest capability of Ant (inheritance, etc.) with generator is a great possibility (think of generating miscellaneous Ant files for different application servers...)
Cheers,
Lofi.
EJOSA + OpenUSS -
A Modular Build for Ant[ Go to top ]
- Posted by: Charlie Collins
- Posted on: June 27 2005 10:27 EDT
- in response to Joseph Ottinger
Its a decent article. The whole "modular", "hierarchical" and "artifact driven" of course not being revelations but it is useful to see an example via ant. Maven is great too, but the point is more that you should get to those goals and this is an example of doing it with ant (which some people are more comfortable with, and with which it can be done), rather than what build tool is better.
And just a hint, even with ant <= 1.5 you can use AntContrib iterators and a props file to decide what modules to call, rather than putting the process and order actually in the build file. -
A Modular Build for Ant[ Go to top ]
- Posted by: Dirk Ludwig
- Posted on: June 28 2005 06:10 EDT
- in response to Joseph Ottinger
From the article:A module is a logically aggregated unit of functionality that corresponds to a named feature in a system.
[...]
An artifact-driven build is one where each module or submodule exists for the purpose of generating a single deployable artifact.
With Les Hazelwood's definition of a module, I haven't been in any project, where this assumption is true. Usually we have _many_ artifacts per module. For example a server-side JAR file, a client JAR for accessing server-side functionality, a JAR containing the tests for a module. AFAIK, maven also has this "philosophy" of a single artifact per module. This was one of the reasons I never used maven, since it simply does not fit me needs.
Further, I wouldn't call a "Hazelwood-module" a module. IMHO it's a subsystem or component: to me a module is a physical unit, not a logical.
Just my 2Cents,
Dirk -
A Modular Build for Ant[ Go to top ]
- Posted by: Donald Diego
- Posted on: June 28 2005 12:11 EDT
- in response to Dirk Ludwig
From the article:
A module is a logically aggregated unit of functionality that corresponds to a named feature in a system.[...]An artifact-driven build is one where each module or submodule exists for the purpose of generating a single deployable artifact.
With Les Hazelwood's definition of a module, I haven't been in any project, where this assumption is true. Usually we have _many_ artifacts per module. For example a server-side JAR file, a client JAR for accessing server-side functionality, a JAR containing the tests for a module. AFAIK, maven also has this "philosophy" of a single artifact per module. This was one of the reasons I never used maven, since it simply does not fit me needs.Further, I wouldn't call a "Hazelwood-module" a module. IMHO it's a subsystem or component: to me a module is a physical unit, not a logical.Just my 2Cents, Dirk
It sounds like you have a module with submodules then. I don't see how that can't be accomplished with the proposed approach. -
A Modular Build for Ant[ Go to top ]
- Posted by: Dirk Ludwig
- Posted on: June 29 2005 04:15 EDT
- in response to Donald Diego
It sounds like you have a module with submodules then.
I think, that I don't, because I have physical modules, that do not contain disjunct artifact sets (e.g. client JARs contain EJB remote interfaces, as well as the server JARs do).
Having read the article once again, I'm quite confused what Hazelwood actually means by "module". On the one hand he says, that a module is a "logically aggregated unit of functionality", on the other hand he imposes a directory structure, that is, IMHO, clearly related to _physical_ modules (JARs, or whatever).
Regards,
Dirk -
A Modular Build for Ant[ Go to top ]
- Posted by: Donald Diego
- Posted on: June 28 2005 12:23 EDT
- in response to Joseph Ottinger
Agree this is a good introduction to build organization.
One thing that I've found useful in the submodule organization is to separate sources into src/java, src/ejb, etc. This allows you to handle categories of source code differently. This is also how many open source projects are working. -
Between Ant and Maven[ Go to top ]
- Posted by: Yves Martin
- Posted on: November 17 2005 04:36 EST
- in response to Joseph Ottinger
If you find Maven too heavy and your Ant build files difficult to maintain, you may want to look at EL4Ant http://el4ant.sourceforge.net/