Discussions

News: Preflight builds: There's a time and a place

  1. Preflight builds: There's a time and a place (11 messages)

    Over the past several years, build management and Continuous Integration tool vendors have suggested that these tools should manage not just authoritative builds from source control, but also replace the test builds done on the developer’s desktop. While useful technology in specialized situations, these preflight (precommit) builds have the potential to undermine best practices -- doing more harm than good. Preflight builds are builds done by the continuous integration server of developer code that hasn't yet been committed to source control. They enable developers to test their changes on the authoritative build server before exposing the rest of the team to the changes. For teams that have complex builds that require specialized hardware, this kind of build can be the only kind that accurately predicts the quality of the code prior to commit. Large teams adopting continuous integration but whose projects take too long to build, or have too many developers on a single component can also use preflight builds as a crutch to address the "continuously broken build" problem. For most teams though, the extra time and effort inherent in preflight builds just adds trouble for negligible returns. In the full article, we look at preflight builds in detail and examine how they can help some teams and hinder others. Full Article

    Threaded Messages (11)

  2. What about the use of feature branch ?[ Go to top ]

    I really agree with that article - preflight build cannot replace relevant unit tests and simple integration tests that can be ran on the developer workstation. With a good test partitioning, a developer will only run tests concerned by his change set. From my point of view, preflight builds can be avoided by the use of feature branches in any version control tool with a cheap branch support (like Subversion). A developer or a set of developers work on a branch to develop new features and patches - and manually ask the CI tool to process this branch (instead of trunk). When satisfied by CI results, the branch content can be merged into the trunk and be closed or live for a while according to needs.
  3. Every feature has pros and cons[ Go to top ]

    To me it looks like author has not tried to use this feature in a day to day work. For example, who said that automated commit is a must? TeamCity allows you to start personal build (this is how preflight builds are called in TeamCity) without automated commit. You can commit your changes later manually if build results satisfy you. If someone interested in practical experience with personal builds I would suggest to take a look at this blog post: http://teamcitydev.blogspot.com/2008/03/personal-builds-practical-experience.html There is no hype here, personal builds just work.
  4. Re: Every feature has pros and cons[ Go to top ]

    I think that for the most part, we do agree on the core advantage of personal, or precommit builds, they allow you access to more hardware. It's better for cross-platform builds, builds requiring odd hardware, and perhaps for slow tests that can take advantage of distribution. I tend to think that slow tests don't need to be run before commit, but I understand the arguments for running them. To each their own.
  5. Re: Every feature has pros and cons[ Go to top ]

    Yes, I agree with you on advantages, but I do not share your fears about personal builds :)
  6. It seems to me..[ Go to top ]

    that the problems with PFC are, yet again, down to poor choice of SCM. A DVCS+Build environment can be pulling "uncommitted" changes directly from developers repositories and speculatively running CI tests on them. This is significantly *more* agile rather than "In the rare case of a problem commit, the error should be rectified in less than 30 minutes.". It's not rare, and in the event of breaking everyone else (assuming said developer hasn't cleared off for the weekend) is significant (particularly if your SCM, like SVN, has no good rebasing tools). Even if it is rare, the *overall cost* is the significant factor. You can of course do this today, easily, with a combination like Git+Hudson (or probably Hg+Hudson).
  7. Re: It seems to me..[ Go to top ]

    Brandon, You can accomplish the same sorts of things in stream based source control systems like an Accurev or ClearCase. In any case though, I do think that we have to admit that any delay in integrating our changes with the common code base for our (hopefully small) team is antithetical with CI. We want to run the fastest build that will give us a strong indication that we are unlikely to ruin the day (or hour) of our teammates. For most teams, that's a local, incremental developer build with the core, fast unit tests. If this passes, we may still be introducing a bug that a more rigorous build, and test will catch for us, but it shouldn't be the kind of problem that prevents other people from getting work done.
  8. Ja man![ Go to top ]

    Agreed. DVCS + Hudson. Tiered builds, even on the go! No remote connectivity needed.
  9. I don't quite understand why "build intended to be run in a test environment" should be treated as "uncommitted build". We commit test code all the time around my place. Test code is valuable intellectual property and sometimes takes a lot of time to create. Why would we not put this under source control? Source control tools support working with branches - allowing part of the team to check in code and yet not affect the code other developers use. That requires the team to do the hard work of process control, but it protects the intellectual property they're creating. That's the solution for this we have adopted and it's not that hard. I'm interested to hear why that does not work for others ?
  10. Sometimes branches are too expensive. I would not create a branch every time when I need to add another small feature. Personally I prefer to merge more often but in smaller parts than to merge entire branch after a couple of weeks of active development. Moreover if you are using CI most likely you will need to create new build plan/build configuration/whatever to run tests in the newly created branch. You also need to configure your IDE to work with this new branch. So I would not say that branching is easy, not at all. In case of relatively small features (which does not require several weeks of refactoring) personal (preflight) builds are more convenient than branches. They offer you a way to test that your code can be compiled and is working in different environments. As a result the code which is coming to the source control is more robust and complete. And you do not spend your time trying to remember how to create branches in CVS... At the same time while you are fighting with a hundred of tests failed in your personal build your teammates can create a release build and test it. Otherwise your code would be committed and testers would wait till you fix the tests. Personal builds smooth the development process, and as for me it's a good thing.
  11. Sometimes branches are too expensive. I would not create a branch every time when I need to add another small feature. Personally I prefer to merge more often but in smaller parts than to merge entire branch after a couple of weeks of active development.

    Moreover if you are using CI most likely you will need to create new build plan/build configuration/whatever to run tests in the newly created branch. You also need to configure your IDE to work with this new branch. So I would not say that branching is easy, not at all.

    In case of relatively small features (which does not require several weeks of refactoring) personal (preflight) builds are more convenient than branches. They offer you a way to test that your code can be compiled and is working in different environments. As a result the code which is coming to the source control is more robust and complete. And you do not spend your time trying to remember how to create branches in CVS...

    At the same time while you are fighting with a hundred of tests failed in your personal build your teammates can create a release build and test it. Otherwise your code would be committed and testers would wait till you fix the tests.

    Personal builds smooth the development process, and as for me it's a good thing.
    Well, that all makes sense to me, but these are all the same reasons against using version control at all. Certainly it's more 'convenient' not to have to check things in and out and set up CI and so forth ... and seriously I would not want to minimize the cost of doing that. Things become a lot less 'convenient' for you - and more importantly, for your company - when your hard drive crashes and you lose a week's work you never committed. Obviously, we all know this, and it's always a balance between ease of use and security. It just seems to me that encouraging the practice of not checking in is asking for trouble ... I'm also not certain of the value of testing a feature without incoporating the rest of the team's changes. Obviously, this is why we do CI to begin with : to become aware of possible problems with changes as soon as possible. After all, the feature isn't tested until it's tested in the full build. Eh, don't want to rain on anybody's parade. I just think ... CM work is undervalued in general, and this topic is one aspect of that ... maybe I'm overstating the value CI in general.
  12. It probably depends on implementation of personal / preflight builds. But technically it is possible to store changes included into the personal build on the CI server side and then retrieve them to developer PC in case of disk crash.