Dear Manager, They Need a Build Machine

Discussions

News: Dear Manager, They Need a Build Machine

  1. Dear Manager, They Need a Build Machine (34 messages)

    Mike Clark has started a series of entries of letters that you wish you could write to your boss. It consists of concepts which seem so obvious to us, but which the bosses don't get.

    Intro
    Dear Manager,

    When I talk with the developers on your team, they tell me they need a dedicated build machine. I'm partly to blame. See, I've been showing them a free build scheduler that, after just a few minutes of configuration, will continuously build your software with no human intervention. That's good for them, and it's even better for you. Let me tell you why.

    If you build and test your software once an hour, no problem is more than an hour old.
    This makes it easier to find and fix problems, which saves time and money, and lets your team concentrate on adding new stuff, not fixing old stuff.
    The continuous build process feeds you valuable and timely information, letting you manage the development more tightly.
    Here's how it works: Every hour, for example, and only if new work has done, the build scheduler checks out a fresh copy of your project from version control and attempts to build the project. The build process includes compiling all of your project's source files, running an arbitrary number of tests, and generating any other build artifacts, such as project metrics. If, for any reason, the build process is unsuccessful, then the build scheduler can notify you via email, your cell phone, RSS, or a visual device such as a lava lamp. You can also use your web browser to view the current status of the build, or any prior build.

    Read Dear Manager, They Need a Build Machine

    Threaded Messages (34)

  2. amazed[ Go to top ]

    Isn't it amazing how many projects don't have automated, periodic builds?
  3. amazed[ Go to top ]

    Yes it is. Especially when you are on one of those projects and you can't fix it! Ahhhhhh!
  4. Re: amazed[ Go to top ]

    Yes it is. Especially when you are on one of those projects and you can't fix it! Ahhhhhh!
  5. Yes It has happend to me[ Go to top ]

    Isn't it amazing how many projects don't have automated, periodic builds?
    I have been involved in a Fortune 500 J2EE project where it was a "healthy” practice to compile application in WSAD and "export" it for production deployment.
    I hope they are not on TSS J I was asked to write a paper describing the benefits of automated builds to the management!.
    This brings up a question as to if IDEs like WSAD which have internal build mechanisms can ever replace external build mechanisms (leave alone a build machine!)
  6. Yes It has happend to me[ Go to top ]

    I have been involved in a Fortune 500 J2EE project where it was a "healthy” practice to compile application in WSAD and "export" it for production deployment.I hope they are not on TSS J I was asked to write a paper describing the benefits of automated builds to the management!.This brings up a question as to if IDEs like WSAD which have internal build mechanisms can ever replace external build mechanisms (leave alone a build machine!)

    I think IDE as crutch is common. Most IDE's also have the ability to integrate ANT. The IDE route is great for just throwing something up there quickly in development, but for documentation sake alone, an ANT that tells you where every jar and component came from is priceless...

    Would that IDE's had the ability to express their internal build strategy as an ANT to export. I use both Eclipse and JDeveloper now...Eclipse doesn't really bother to make a deployable for you (I don't really use any J2EE plug-ins in Eclipse) but is clean. Jdeveloper does a respectible job of creating your J2EE cruft, but insists on it's own internal format.

    Also...my real beef with this is that each IDE has it's own idosyncratic or even idiotic way of structuring your projects, so that you have to hack around with ANT and CVS to get someting fairly neutral as a product!

    Any IDE's out there that live and breathe ANT and a good structure for complex apps? I've used tons over the years and they all suck in their own special ways!

    Mike
  7. I've heard that NetBeans 4 uses Ant as its native build tool.
  8. But I'm pretty sure IDEA has the ability to export it build processes as an Ant scripts.

    (I write my own Ant scripts and hope to start using cruisecontrol or anthill or maven soon.)
  9. But I'm pretty sure IDEA has the ability to export it build processes as an Ant scripts.(I write my own Ant scripts and hope to start using <em>cruisecontrol</em> or anthill or <em>maven</em> soon.)
    Well, actually you can use both maven as your base build system and reporting tool (in which you can directly use your already written ant scripts), and cruisecontrol as your continuous build system since there is a cruisecontrol plugin for maven ;)
    http://maven.apache.org/reference/plugins/cruisecontrol/
  10. Netbeans 4 can be your friend[ Go to top ]

    Netbeans 4 projects are centered around ant.

    You can create a project that uses an existing ant script and assign targets to common actions in the IDE such as build, run, test, debug, etc. You can execute any target in the build script as well if it is an uncommon task.

    It's the best non-invasive ant integration in an IDE (well, 3.6 allowed you to do the same in different way).
  11. Yes It has happend to me[ Go to top ]

    Also...my real beef with this is that each IDE has it's own idosyncratic or even idiotic way of structuring your projects, so that you have to hack around with ANT and CVS to get someting fairly neutral as a product!

    If you're using Maven, you could go the other way. Create the Maven project first, then run a goal to create the equivalent IDE project files. Eclipse and Idea supported out of the box. Don't know about others.

    Kit
  12. Yes It has happend to me[ Go to top ]

    Also...my real beef with this is that each IDE has it's own idosyncratic or even idiotic way of structuring your projects, so that you have to hack around with ANT and CVS to get someting fairly neutral as a product!
    If you're using Maven, you could go the other way. Create the Maven project first, then run a goal to create the equivalent IDE project files. Eclipse and Idea supported out of the box. Don't know about others.Kit

    This is really quite an interesting thread. In the past I've been more about the technology versus the process. I'm thinking too about CDS's like Savane or CollabNet. Does anyone feel like they have the process nailed, both in terms of builds/source control, but also things that a CollabNet might cover, such as bug tracking and project mgmnt?

    The thing that gets me about the ANT integration is the fact that lots of 'corporate developers' really need to point and click to say 'build me a web module, add a servlet and generate the mappings', etc. They know accounting systems cold, not J2EE descriptors. An easy way for them to click and get a properly formatted set of descriptors, plus have it all represented as ANT would be gold. I'd like it too to get started, as ANT is the most tedious thing I've done since my JCL days 15 years ago!

    Mike



    Regards,
    Mike Conway
  13. I think all processes are troublesome.

    I've worked with Collab.net, and am now using public sourceforge for my day work. Behind the firewall we have cruise control running, sending hate mail to anyone who breaks the build. We also tie in to apache gump for nightly regression builds against all the other OSS projects; very handly, though you cannot run functional tests there.

    We are encountering some scaling problems with Ant, even with the new stuff in Ant1.6. I'm planning on making some changes to Ant1.7 to fix those. First off will be a task to autoload dependent libraries from the maven repository, and internal repositories that look like it.

    -steve

    Steve loughran
    co-author, Java Development with Ant
  14. Maven can come handy[ Go to top ]

    Also...my real beef with this is that each IDE has it's own idosyncratic or even idiotic way of structuring your projects, so that you have to hack around with ANT and CVS to get someting fairly neutral as a product!
    If you're using Maven, you could go the other way. Create the Maven project first, then run a goal to create the equivalent IDE project files. Eclipse and Idea supported out of the box. Don't know about others.Kit

    Many of us are talking about builds theseday, it would be nice thing to give some attention to MAVEN http://maven.apache.org . Having used maven for almos two projects, I can only say, yes it requires some initial configuration to be made in the project.xml and surely your custom build goals in the maven.xml. But once you have some basic things done, it creats some fantastic arifacts for you.

    I use maven for..

    - Building multi projects (Shared, Business server, Client, Web)
    - Running checkstyles, PMD, Pattenr testing
    - Running Junit tests
    - Generating various reports (Source meterics, Dependencies, checkstyle, clover etc)
    - Generating deployment descriptors for various App servers needed like (Weblogic, Sun Java System, JBoss) using a custom plugin.
    - Packaging required jars needed during runtime (e.g oscache, hibernate etc)
    - Installing EAR to the target App server.

    The artifacts that gets generated are goodies with proper versions and one can easily revert/refer to earlier version if required.Moreover continous builds, assembly builds and release versions are properly managed.
  15. You know whats worse? We bought a new build machine over a year ago to move from a C type script to ant for our builds but management just uses it as a 'backup' build machine.

    -Pete
  16. Wouldn't a few powerpoint slides suite the target group better... ;-)
  17. Depends where you put them. ;-)

    -Pete
  18. Slides for target group[ Go to top ]

    Wouldn't a few powerpoint slides suite the target group better... ;-)

    Hey, nice timing. I've just created some slides for an Introduction to CruiseControl which cover some of Mike's points.
  19. Slides for target group[ Go to top ]

    Whow, how did you created the html slides?
  20. Slides for target group[ Go to top ]

    Whow, how did you created the html slides?

    I was looking for a simple way to do the presentation, when I came across Eric Meyer's Simple Standards-Based Slide Show System (S5 for short). Goodbye powerpoint.
  21. So what is it[ Go to top ]

    what is the "free build scheduler". As a doer not a manager I'd like a URL :-)

    Daniel
  22. anthill has a free version[ Go to top ]

    http://www.urbancode.com/projects/anthill/default.jsp
    I haven't used it myself. So no further comments.
  23. anthill has a free version[ Go to top ]

    http://www.urbancode.com/projects/anthill/default.jspI haven't used it myself. So no further comments.

    Hi,

    we are using the free version of anthill in our current project. Works very fine, and is quite easy to configure (also through a Web-based admin console).

    Another free (even open-source) buld scheduler is CruiseControl: http://cruisecontrol.sourceforge.net/
    We used that one in one of our previous projects. I found that one harder to configure. Nevertheless, you got more options there. For example, we used the email notifcation feature of CC quite extensively, where (in case the build fails) an email is send to all users you have checked in into CVS since the last build. As far as I know anthill cannot be configured that way. It only sends emails to dedicated users.

    Regards,
        Dirk
  24. Use mailing list[ Go to top ]

    If you want to have the anthill build logs or whatever send to a group of people you can just set up a distribution list on the mailserver...
  25. Use mailing list[ Go to top ]

    If you want to have the anthill build logs or whatever send to a group of people you can just set up a distribution list on the mailserver...

    Yes, but that is not what I wanted. We were doing continous builds every hour in order to check consistency of the sources developed by different (sub-)teams, at different locations. I want anthill to send emails only to those users, who checked in since the last sucessfull build. This is because one of these users _must_ have checked in something, that caused the build to fail. I want these users to fix the buld-failure cause as soon as possible, but without bothering the other users (teams might get quite large, > 20 people).

    Maybe anthill supports this, and I just have not found that feature yet.


    Regards,
        Dirk
  26. Cruise Control does[ Go to top ]

    I dont know about anthill; cruise control does extract cvs or svn user names and can send them email. its easiest if the user name maps to an account on some server, or you have to set up a mapping table.

    CC is a git to set up, as there are multiple config files that need to be consistent. Which is a shame, as you should bring it up as early as possible, and leave it running.

    Steve loughran
    co-author, Java Development with Ant
  27. cron[ Go to top ]

    cron. It not only schedules builds, but anything you want. Only available for real OS's
  28. Free scheduler[ Go to top ]

    Check out Maven. It'll do a lot more for you than "just" schedule your builds.
  29. Free scheduler[ Go to top ]

    I think the talk here about Cruise Control (http://cruisecontrol.sourceforge.net)
  30. We have it set up that every minute, if there are new files and a build is not in process, it does a complete build (including unit tests, etc).

    I don't really see the reason for only doing it every hour???

    thanks - dave
  31. It depends on visionary of the manager. It's good to have bronze, silver and gold development environment. Not only build, code, database schema, test data, library or IDE, but the process.

    http://c2.com/cgi/wiki?ExtremeProgrammingChallengeThirteenPointFive
    http://www.extremeprogramming.org/stories/testdb.html

    I hope this would help.

    Surapong K.
    Mfatix Company Limited
    http://www.mfatix.com
  32. dear nitin,[ Go to top ]

    when i read your message i had a strong feeling of
    XP going rampant...

    before i start digging deeper into the issue just a
    clarification: i am not disagreeing in principle. but
    i do disagree when stating such things without context
    (as you obviously did).
    If you build and test your software once an hour, no problem is more than an hour old.
    so what? knowing a problem after an hour might be a
    problem but it also might not be. this depends on your
    development process.
    This makes it easier to find and fix problems, which saves time and money, and lets your team concentrate on adding new stuff, not fixing old stuff.
    sorry, but stated in such a way this is simply wrong.
    utterly nonsense.
    again your chosen development process is key to the
    question if this saves money or if it wastes money.

    just a simple example: assume a subproject SA which is
    used by subproject SB. now the maintainer of SA has done
    some quite extensive api reworking on his module (which
    breaks almost all clients of SA). he checks it in. of
    course the integration build will fail because SB has
    not changed to the new api of SB. AND the B group is
    just reworking a lot of their stuff and immediatly
    incorporation of the changes of A is simply no option
    for the next three days. so what use is it for group B
    to be reminded every hour that they break the build?

    if this happens twice your integration build will not
    be used anymore because people will start to boycott
    this part of your process. belive me they will!

    again i am not saying that an automatic build has no
    value (of course it has), i am not saying that a dedicated
    build machine has no value (i am sure it has), i am not
    saying that continous integration has no value (i am no
    friend of it but it certainly has value).
    what i am saying is: dont use your golden "make me happy"
    xp hammer for every conceivable problem.
    there are issues which can be solved (better) without it.
    please take care of the context.

    ciao robertj
  33. dear nitin,[ Go to top ]

    when i read your message i had a strong feeling ofXP going rampant...before i start digging deeper into the issue just a clarification: i am not disagreeing in principle. buti do disagree when stating such things without context(as you obviously did).
    If you build and test your software once an hour, no problem is more than an hour old.
    so what? knowing a problem after an hour might be aproblem but it also might not be. this depends on yourdevelopment process.
    This makes it easier to find and fix problems, which saves time and money, and lets your team concentrate on adding new stuff, not fixing old stuff.
    sorry, but stated in such a way this is simply wrong.utterly nonsense. again your chosen development process is key to the question if this saves money or if it wastes money.just a simple example: assume a subproject SA which isused by subproject SB. now the maintainer of SA has done some quite extensive api reworking on his module (whichbreaks almost all clients of SA). he checks it in. ofcourse the integration build will fail because SB hasnot changed to the new api of SB. AND the B group isjust reworking a lot of their stuff and immediatlyincorporation of the changes of A is simply no optionfor the next three days. so what use is it for group Bto be reminded every hour that they break the build?if this happens twice your integration build will not be used anymore because people will start to boycottthis part of your process. belive me they will!again i am not saying that an automatic build has novalue (of course it has), i am not saying that a dedicatedbuild machine has no value (i am sure it has), i am notsaying that continous integration has no value (i am nofriend of it but it certainly has value). what i am saying is: dont use your golden "make me happy"xp hammer for every conceivable problem.there are issues which can be solved (better) without it.please take care of the context.ciao robertj

    You're managing your inter-project dependencies incorrectly if this happens. Either both need to be refactored together or you need to check a binary build of A into B and only update it when you're ready to refactor B for the changes to A and check the refactorings in with the new A build as one checkin.
  34. Continous builds[ Go to top ]

    just a simple example: assume a subproject SA which is
    used by subproject SB. now the maintainer of SA has done
    some quite extensive api reworking on his module (which
    breaks almost all clients of SA). he checks it in. of
    course the integration build will fail because SB has
    not changed to the new api of SB. AND the B group is
    just reworking a lot of their stuff and immediatly
    incorporation of the changes of A is simply no option
    for the next three days. so what use is it for group B
    to be reminded every hour that they break the build?

    Hi Robert,

    I see that this scenario is a problem. But it is not a problem of the continous build, rather a problem of the SW-development process. In the scenario you described, no one within the whole project is able to perform a build for whole 3 days!!! In my opinion intensive refactorings of module interfaces (not realizations within modules) must be planned in advance. Especially when we talk about _big_ projects.

    Regards,
        Dirk
  35. In my opinion intensive refactorings of module interfaces (not realizations within modules) must be planned in advance. Especially when we talk about _big_ projects. Regards,&nbsp;&nbsp;&nbsp;&nbsp;Dirk

    I strongly agree here; there is no point having cruise control tell you that things are broken when somebody did a a fundamental change to their bit of the system, and everyone else has to scuttle around to change theirs to match.

    I know you arent meant to 'own' bits of a project, but sometimes there are bits that only one person can work with, and when they make an unannounced change, bad things happen.

    The rule here: change control. I know this goes against the refactor continually rule, but interfaces (and semantics behind interfaces) are special -you dont change them without warning, and you do it in a time and a way that people are happy with.

    dont do that and you create team tensions that take a while to die down...