Continuous Integration paper updated

Discussions

News: Continuous Integration paper updated

  1. Continuous Integration paper updated (6 messages)

    Martin Fowler has updated his "Continuous Integration" paper, completely revising the table of contents and removing the emphasis on ThoughtWorks' CruiseControl package. The updates make the paper far more useful than it already was, being less skewed towards CruiseControl and updating references from CVS to SVN, among other things.

    The section "Introducing Continuous Integration" looks like this:
    So you fancy trying out Continuous Integration - where do you start? The full set of practices I outlined above give you the full benefits - but you don't need to start with all of them.

    There's no fixed recipe here - much depends on the nature of your setup and team. But here are a few things that we've learned to get things going.

    One of the first steps is to get the build automated. Get everything you need into source control get it so that you can build the whole system with a single command. For many projects this is not a minor undertaking - yet it's essential for any of the other things to work. Initially you may only do build occasionally on demand, or just do an automated nightly build. While these aren't continuous integration an automated nightly build is a fine step on the way.

    Introduce some automated testing into you build. Try to identify the major areas where things go wrong and get automated tests to expose those failures. Particularly on an existing project it's hard to get a really good suite of tests going rapidly - it takes time to build tests up. You have to start somewhere though - all those cliches about Rome's build schedule apply.

    Try to speed up the commit build. Continuous Integration on a build of a few hours is better than nothing, but getting down to that magic ten minute number is much better. This usually requires some pretty serious surgery on your code base to do as you break dependencies on slow parts of the system.

    If you are starting a new project, begin with Continuous Integration from the beginning. Keep an eye on build times and take action as soon as you start going slower than the ten minute rule. By acting quickly you'll make the necessary restructurings before the code base gets so big that it becomes a major pain.

    Above all get some help. Find someone who has done Continuous Integration before to help you. Like any new technique it's hard to introduce it when you don't know what the final result looks like. It may cost money to get a mentor, but you'll also pay in lost time and productivity if you don't do it. (Disclaimer / Advert - yes we at ThoughtWorks do do some consultancy in this area. After all we've made most of the mistakes that there are to make.)
  2. interesting read[ Go to top ]

    Overall, I think continuous integration is a combination of several good practices that are also good independently from each other.
    - Using a modern versioning system seems sensible. Having extensive experience with subversion and cvs, I'd advice people still using the latter to upgrade from its severe limitations. Also be sure to not abuse subversion as if it is the same as CVS. Commit coherent changesets, not individual files. Never ever lose version history of anything in the repository needlessly. Unlike CVS you can actually rename, move and copy files without destroying their history. Subversion is easy to set up, even for private projects.
    - Convert the complicated set of actions required to build your system into a build script to ensure that no errors are introduced by making mistakes in executing the steps. A nice side effect is that this makes the process more idiot proof and also more predictable.
    - Build often, and particularly before committing.
    - If you have automated tests, use them. In my experience, creating automated tests is easy for some categories of software and problems but difficult for other categories of software.
    - Use nightly builds or continuous builds for time consuming builds and tests.

    The main problem I have with continuous integration is that it doesn't seem to actually include integration testing. In my experience, developers only test the stuff they are working on. This is called unit testing. What's typically not tested is the effect the new code has on other code. A large set of automated unit tests is helpful but not enough to do proper integration testing. This generally requires a human to run through a complex testplan doing all sorts of things users do that are hard to test automatically. Especially with interactive software this is a necessity. And like unit testing it needs to be done often but not continuously (drives developers and testers crazy).

    Proper integration testing is a lot of work. Preferably it should be done as a separate activity from development. A model we had at a previous company was that of weekly internal releases. Each of these releases was tested for an entire week. Bugs were reported against this release and fixed during the week. As a rule, the tester did not receive upgrades during this week. Also he worked using an actual testplan based on the requirements.
  3. If CruiseControl is a great tools for Continuous Integration at build layer, I have create a project which provide Continuous Integration at deployment layer. The purpose is to have an automatic deployment tools which take the build continuous integration artifact (for exemple tar.gz archives provided by CruiseControl) and deploy applications on target application servers. This tool is named AutoDeploy and is a part of the BuildProcess J2EE administrator toolbox.
    For now, AutoDeploy supports BEA Weblogic 6/7/8, IBM WebSphere 5.1 and JBoss application servers.
    http://buildprocess.sourceforge.net
  4. Good Arcticle but ...[ Go to top ]

    Continuous integration is really a good concept, and we practice this in our company.

    One concept I don't agree with Mr. Fowler is the "Keep your use of branches to a minimum. In particular have a mainline: a single branch of the project currently under development. Pretty much everyone should work off this mainline most of the time.".

    This approach seems to be the defacto standard of all open source project. And it seems to work well in that environment, which can afford product release delay due to feature delay. However, consider a commerical product. Let's say we want to release our project prior to a specific date (e.g. a trade show or something). Let's say we have two main features A and B in development. We're racing against the time to make both features available before the deadline. However, chances are one of the feature won't be able to make it. Using the one mainline approach, there's no easy way to remove one feature and release the other completed feature.

    In our company, each feature will be assigned a separate branch and a separate unit in the CI build server. This approach works fine for us.
  5. Good Arcticle but ...[ Go to top ]

    Yes but if you have seperate branches you're not really integrating until the end. Sooner or later features A and B have to get merged into the mainline. And if B has problems integrating with A, you won't find out until the end. Exactly the problem continous integration is trying to avoid. Separate branches subvert the idea of CI; I think that's why Fowler stressed in the rewrite. Also, Mmjor refactorings with multiple branches can be a nightmare.

    Part of the solution is to break larger tasks into smaller tasks that can be checked regularly in without breaking anything. So for example you can incrementally check in the code for that new spell checker feature to the mainline incrementally, but just not enable the menu item until its totally ready.

    Iterative development helps too -- release in very small chunks. May not work as well for a commercial product; I have no experience in that area. However many commerical open source projects use the single branching approach. (Stuff that's experimental still deserves its own branch.)
  6. Re: interesting read[ Go to top ]

    Jilles, You hit the nail with this list of practices! Viewtier offers articles on more advanced topics, particularly on capacilty planning for build servers, ways to avoid build breakage patterns and build performance. By the way, we've just started an early access program for Parabuild which is a commercial counterpart of CruiseControl and we offer product licenses for every bug you find in the EAP builds. Regards, Slava Imeshev Viewtier Systems, Inc.
  7. "Continuous Integration: The Cornerstone of a Great Shop". http://www.methodsandtools.com/archive/archive.php?id=42