Discussions

News: Cedric Beust: Agile people still don't get it

  1. In "Agile people still don't get it," Cedric Beust (pronounced, incidentally, as "bust") says that "Agilists just don't understand the meaning of calculated risk." Along the way he says that Agile advocates need to stop being so arrogant about the strengths of the technique - while strong, it's not magic or perfect.
    Fundamentally, I am disturbed by the Agilists' dishonesty when it comes to presenting their arguments. They offer you all these nice ideas such as Test-Driven Development and Pair Programming but they never -- ever -- disclose the risks and the downsides. To them, Agility is a silver bullet that is applicable in all cases with no compromises. The truth is that these practices come at a price, and for a lot of organizations, the price gets high very quickly. Agile development will never go far if its proponents keep ignoring these organizations and make condescending comments to its members. I like Test-Driven Development. I really do, and I'm fortunate enough to work on a project that lets me use TDD most of the time. But the truth is: at times, I don't do TDD because implementing a feature quickly is more important than a fuzzy feeling. And I'm also aware that TestNG is an open source project with less than five developers, all of them on the bleeding edge and aware of the latest advances in software engineering. And this is my main beef with Agilists: I strongly suspect that most of them are spending their time on open source projects with like-minded fellows, but none of them have any experience what companies whose survival depends on shipping software have to go through to organize huge code bases growing thousands of lines of code every day under the combined push of hundreds of a developers, all with their personal background, education and bias. So here is my advice to Agilists: get real now, or you will become irrelevant soon.

    Threaded Messages (70)

  2. Costs of Agile approaches?[ Go to top ]

    Excellent topic for discussion! Hopefully this doesn't degenerate into a holy war of agile vs. waterfall. I'd love to hear what people have experienced as the cost of using an agile approach. Once cost I've kept running into is that *typically* you need to have the work load for domain experts throttled back so that they have a reasonable amount of time to evaluate your internal releases, give feedback to the dev team, answer day-to-day questions, etc. Often times the domain expert is already fully tasked so other people in that dept have to pick up the extra work. I can think of a few others, but I'd rather not write an overly long post this morning. Don't get me wrong...I am an absolute believer that an agile approach offers much better chances of success over more heavyweight approaches, but working in a conservative town I've seen enough organizations where the culture simply wouldn't be able to support it.
  3. Re: Costs of Agile approaches?[ Go to top ]

    Excellent topic for discussion! Hopefully this doesn't degenerate into a holy war of agile vs. waterfall.
    Yes, please, partly because I don't think that dichotomy is real.
  4. Re: Costs of Agile approaches?[ Go to top ]

    Back when the cold war was coming to a close, I remember seeing a political cartoon with a bear and an eagle on an ice-float drifting away from an iceberg labelled "Cold War". The caption said something like "I don't know where we are going but anywhere but back there." I work at a company that has crafted it's own homegrown development process based on the waterfall model. As I am currently looking for new employment, that political cartoon keeps coming to mind. So I am very interested in Agile methodologies but I am not sold on them either. It's kind of an anywhere that's not where I am now. I really really agree with Cedric's main premise that there are no panaceas and you insult people's intellegence when you claim that there are.
  5. Re: Costs of Agile approaches?[ Go to top ]

    I work at a company that has crafted it's own homegrown development process based on the waterfall model.
    I'm in the same situation. The process is so heavyweight so as to mandate that we track time spent on tasks down to the minute level (specifically being mindful to exclude interuptions such as bathroom breaks, email, etc). To me, situations like these illustrate the far extreme of allowing upper level managers dictate the *how* in development.
    So I am very interested in Agile methodologies but I am not sold on them either.
    I'm not 100% sold on them insofar that I don't believe they can be applied everywhere. However, in organizations where they *can* be applied I'll always bet on agile over waterfall.
  6. Re: Costs of Agile approaches?[ Go to top ]

    The process is so heavyweight so as to mandate that we track time spent on tasks down to the minute level (specifically being mindful to exclude interuptions such as bathroom breaks, email, etc). To me, situations like these illustrate the far extreme of allowing upper level managers dictate the *how* in development.
    Precisely. The only people in our organization that think this process is highly effective are those that do not have to work inside it. I can count on a single hand the number of times I've actually looked at old documents generated through this methodology. I quickly discovered that the provided almost no value and finding the documents took longer than what I saved by looking at them. The biggest flaw that I see it that nothing in the process requires that the full and current state of the architecture and design be documented and updated. All we have a bread0-crumbs showing 'bitwise' modifications. If the documentation were fully accurate and complete (which it is not) you could theoretically pull together thousands of documents lay them out in cronological order and (accouning for parallel development) and deduce the current state of the system. This is obviously not feasible. So yeah, in a way, upper-managment is micro-managing the development teams.
  7. Re: Costs of Agile approaches?[ Go to top ]

    The only people in our organization that think this process is highly effective are those that do not have to work inside it.
    Amen to that! These were almost, verbatim, my exact words when I tried to relay my frustrations to my manager. You must be an incredibly sharp individual :) lol
  8. Re: Costs of Agile approaches?[ Go to top ]

    The only people in our organization that think this process is highly effective are those that do not have to work inside it.


    Amen to that!

    These were almost, verbatim, my exact words when I tried to relay my frustrations to my manager.
    I thought you might work at the same company as I but I can't find you in the directory. I don't know whether I should feel better that I am not alone or depressed that this absurdity is so pervasive. It's kind of like how I feel whne reading Dilbert. I guess I'll have to laugh to keep from crying. Anyway, luckily for me, I'll be moving on fairly soon. I'm considering a smaller organization where I might actually be able to affect a change.
  9. Process slowdown strike[ Go to top ]

    Isn't one of the best ways to point out the flaws in a process is to have all employees follow it to the letter? In union-shops that is how a slow-down strike is organized. Nobody can get fired since they are following the company's rules to the letter but everybody knows that the more experienced people skip over a lot of the B.S. in the process so they can ship stuff out the door. Any process needs to have the ability to change and adapt as a core part or it will quickly degrade into simple red tape. Any manager who picks a process off the shelf (RUP, XP, Waterfall, whatever) and doesn't tailor it to their project, people, management, existing workflow will end up claming that the "process" was a failure. This is not my quote (well paraphrase really) but it sticks with me. All the software processes are the result of something that worked for one team, in one environment, at a certain time. Don't expect the same results without some real tailoring work. ______________ George Coller DevilElephant
  10. Re: Process slowdown strike[ Go to top ]

    Any process needs to have the ability to change and adapt as a core part or it will quickly degrade into simple red tape. Any manager who picks a process off the shelf (RUP, XP, Waterfall, whatever) and doesn't tailor it to their project, people, management, existing workflow will end up claming that the "process" was a failure.
    The flipside to that is when the management had a hand in creating a process from scratch, it will always be proclaimed successful (and crucial) no matter how poor the results. What I see is that following the process get's you marked as a poor performer. If you go around the process you must lie and say you are using it, otherwise you are non-compliant. Too many people have gone around it so management is throwing in more roadblocks to make sure we are following the process. This leads to absurdities like me having to write to business requirements documents for a project that is driven purely by technical requirements and isn't even funded by a business area. Well, there is a business requirement but I can't turn over a two documents that say "Increase transaction capacity." The final chapter to this comedy of errors is that this process has driven the cost of development to ridiculous levels and now half the department is being outsourced. Oh yeah, and many of the first-level managers will agree that there are big problems with all of this in private but will not say anything to their superiors. People who do things like that disappear and their names are never mentioned again.
  11. Pee Tracking[ Go to top ]

    The process is so heavyweight so as to mandate that we track time spent on tasks down to the minute level (specifically being mindful to exclude interruptions such as bathroom breaks, email, etc
    So do you have a special pee bucket in your time tracking software? I can see that coming up in a employee-performance meeting. Well you seem to be taking 10% more time each month going to the bathroom so we'll need to take that out of your bonus. Later that evening the employee is seen at Walgreens buying adult diapers. Thanks for making me feel slightly better about my client's process. ______________ George Coller DevilElephant
  12. Re: Costs of Agile approaches?[ Go to top ]

    >I'm not 100% sold on them insofar that I don't believe they can be applied everywhere. However, in organizations where they *can* be applied I'll always bet on agile over waterfall.
    Hey, you're not supposed to sound reasonable on The Server Side! I'm involved in developing a project that is using agile development, and it's working our very well. One aspect that is not often mentioned is that it works much better for product managers too. The key benefit for product managers is that new customer requirements often arrive after a project has started. PJ Murray, CodeFutures Software Data Access Objects and Service Data Objects
  13. Excellent topic for discussion! Hopefully this doesn't degenerate into a holy war of agile vs. waterfall.
    I don't understand why we compare agile to waterfall. Shouldn't we compare it to heavyweight, document oriented iterative and incremental methodologies?
  14. Excellent topic for discussion! Hopefully this doesn't degenerate into a holy war of agile vs. waterfall.


    I don't understand why we compare agile to waterfall. Shouldn't we compare it to heavyweight, document oriented iterative and incremental methodologies?
    To me this is the more interesting question. I was originally very sceptical that you could deliver a system without a fair amount of detailed analysis and design up front prior to the start of each iteration (or phase as it use to be called). In practice it is interesting to discover just how little UML and BUFD (Big Up Front Design) is actually needed before you can start coding. A previous poster made a good point about Agile being the elimination of waste. By just thinking differently and by opening up the bandwith available for face-to-face communication, be that with end users, or fellow developers, it is surprising just how little ceremony and documentation is actually needed. I don't want to sound like I'm preaching, but it is very difficult to question long held assumptions until you've actually seen a low ceremony process working in practice. Like I said I was a sceptic once, but I am now convinced that much of what I use to do under the guise of analysis and design was ultimately a waste. If the business really does want design documentation (other than the code of course), then they can schedule documentation stories to be implemented in preference to additional features. The issue of documentation then becomes a business decision. Paul.
  15. In "I strongly suspect that most of them are spending their time on open source projects with like-minded fellows, but none of them have any experience what companies whose survival depends on shipping software ...
    I think this statement is true, although only agilists can verify it. I have tried many of the agile methods although not all in my last project. Because it was a small team of 2-3 developers, the development itself worked wonderfully. But we had issues when it came to interacting with other teams, testing, etc. especially when they do not share the same viewpoint. In such cases, as agile does not follow a formal process and depends more on interpersonal faith, it becomes harder to justify our actions or even introduce accountability. I think a better approach would be to promote agile development with a project management component such as Crystal or any other methodology. Most developers see only one portion and fail to take into account the other. For agile development to successfully work, agile management should also be put in place. You cannot make agile development work with a traditional approach to project management, which is what most developers tend to do, as they cannot change the project management component. Message was edited by: joeo at enigmastation dot com (corrected html)
  16. ...as agile does not follow a formal process and depends more on interpersonal faith, it becomes harder to justify our actions or even introduce accountability....
    Not sure what you mean by interpersonal faith...could you clarify? As far as personal accountability, I've found actually found the exact opposite. The lighterweight process implicitly places more emphasis on each individual's skill (IMO heavyweight processes have the byproduct of trying to augment personal skill level w/ process whether intended or not...but that might be a whole 'nother discussion)...as a result each person is more accountable for the work they do. Things like continuous builds, unit tests, etc quickly and immediately identify poor work. Repeatedly identified poor work from the same person identifies a poor developer.
    For agile development to successfully work, agile management should also be put in place.
    I absolutely agree.
  17. I am working for a client, a very big organization who totally buys into agile up from the top level. Developers are told (a very mild word to use here) to follow the full deal of the agile process - TDD, pair, continuous integration, daily standup, story card party, ... you name it,.... the whole deal. And the whole process is managed by an ultmate agile consulting firm, having both developers and project management staff on board working/pairing side by side with employees. I'd be honest. After a while, I gained accetance of agile as (almost) a non-believer. It works especially well if you start a brand new project with it. And it also improved, IMO, the quality of the existing million lines of codebase. On the flip-side, though, the client does run into some of the problems mentioned in the blog. So my experience implies to me, agile does have enormous benefit although not as much as most of the "agilists" claim; but it has a fundamental flaw that goes with it - you have to pratice the whole agile deal in its entirety for your process; it has little compatibilty with othere "traditional" processes. Co-exist is almost impossible; and you run into all kinds of issues. But in real life, you have to have the ability to live with others. Otherwise, you either live on an island with nobody but mother nature, or you die. Ric
  18. ... you have to pratice the whole agile deal in its entirety for your process; it has little compatibilty with othere "traditional" processes. Co-exist is almost impossible...
    Sounds like a combination of grass-roots support combined with top-level buy in worked for you. On the orginal argument, this is a really easy statement for Cedric Beust to make as most companies have a 85-90% failure rate at adopting new methodologies. Check Gartner on this. He likes certain techniques but feels the whole process doesn't work for everyone and has risks. Would he like it if agile processes were sold like a perscription drugd. "Use XP for 'projectial' dysfunction but be aware of these side effects, risks, and please consult your doctor." An agile adopter (company, department, team) is responsible for ensuring a successful integration and continued use of agile practices if they can work for that company. Caveat Emptor
  19. It works for me, as do other more "traditional" processes. As a developer, many things work if you keep an open mind. But I wouldn't go as far as saying that it's working for the organization. At certain places, it worked. For example, automatic build and integration process. But that success is not at all unique to Agile. You bring in a bunch of consulting guys who do this sort of stuff day in and day out, of course it's going to work, Agile or not. On the other hand, issues exist; and at places, things get so Fragile rather than Agile - someone outside of the agile camp checks in some code without even knowing there is some unit test somewhere, the test/build breaks while the code works just fine on the integration box; To get certain percentage of test coverage, the consultants just put in unit tests without even an assert (they might have wanted to, but they have no idea about how the code should behave). So coverage chart looks good, everybody is happy, only people who really care know that it doesn't hold water at all.... That was why I brought up the co-exist and compatibility issue, which is a fundamental flaw agile has to address better in order to survive. Ric
    ... you have to pratice the whole agile deal in its entirety for your process; it has little compatibilty with othere "traditional" processes. Co-exist is almost impossible...


    Sounds like a combination of grass-roots support combined with top-level buy in worked for you.

    On the orginal argument, this is a really easy statement for Cedric Beust to make as most companies have a 85-90% failure rate at adopting new methodologies. Check Gartner on this. He likes certain techniques but feels the whole process doesn't work for everyone and has risks. Would he like it if agile processes were sold like a perscription drugd. "Use XP for 'projectial' dysfunction but be aware of these side effects, risks, and please consult your doctor."

    An agile adopter (company, department, team) is responsible for ensuring a successful integration and continued use of agile practices if they can work for that company.

    Caveat Emptor
  20. Hi,
    That was why I brought up the co-exist and compatibility issue, which is a fundamental flaw agile has to address better in order to survive.
    I don't get this point. No one ever complained about Waterfall having to learn to co-exists with let say ad-hoc development. The way I see it, a companies approach to process is a given and as an employee you either accept it or leave. Agile isn't for everyone, and within a single organisation I can see that some teams may choose to be more or less Agile than others. The real guts of Agility is to do with people and management. IMO at the core a company needs to decide on an approach to quality, business/customer value and managing change and new service/product delivery. If they decide to become an "Agile" or "Lean" Organisation then Agile Software Development is a natural fit. If they aren't Agile or Lean then Agile Software development will have a problem "co-exisitng". So I would turn your point on it's head:
    That was why I brought up the co-exist and compatibility issue, which is a fundamental flaw that business has to address better in order to survive.
    Paul.
  21. I think you don't get the point because you are missing one piece of the puzzle: The existence of competitors in a in a species life cycle. Anything from its get-go to its end exists in a given environement with its peers/competitors. You can't expect anybody that is already there to get out of the way because you are born. Nor can you expect and 3rd party (the "business" or anything like that) to give you a mighty hand. It's tough out there - You either survive by blowing away anything that completes with you, or you do that by co-exist and see who out-lasts who.

    So I would turn your point on it's head:

    That was why I brought up the co-exist and compatibility issue, which is a fundamental flaw that business has to address better in order to survive.


    Paul.
  22. Hi, I understand your point. I understood the first time too. The funny thing is that the people I've found that tend to like and adopt Agile the easiest is the Business. In my experience they love the control and the freedom to try things and see. They also like the visibility over what they are getting for their money and the ability to decide for themselves whether it still represents good value at any point. I guess you're assuming an hostile reception. Again in my experience, the biggest resistence tends to come from developers themselves (and BA's to some degree). I think the resistance from developers comes partly from the fact they feel that they are being told how to do their job, and people tend not to like change (especially if imposed from outside using Consultants). I guess my point is that the organisation needs to provide a strong and consistent lead from the top. If this is done then at least everyone knows where they stand. Paul.
    I think you don't get the point because you are missing one piece of the puzzle: The existence of competitors in a in a species life cycle. Anything from its get-go to its end exists in a given environement with its peers/competitors. You can't expect anybody that is already there to get out of the way because you are born. Nor can you expect and 3rd party (the "business" or anything like that) to give you a mighty hand. It's tough out there - You either survive by blowing away anything that completes with you, or you do that by co-exist and see who out-lasts who.





    So I would turn your point on it's head:

    That was why I brought up the co-exist and compatibility issue, which is a fundamental flaw that business has to address better in order to survive.


    Paul.
  23. Hi Ric,
    I think you don't get the point because you are missing one piece of the puzzle: The existence of competitors in a in a species life cycle. Anything from its get-go to its end exists in a given environement with its peers/competitors. You can't expect anybody that is already there to get out of the way because you are born. Nor can you expect and 3rd party (the "business" or anything like that) to give you a mighty hand. It's tough out there - You either survive by blowing away anything that completes with you, or you do that by co-exist and see who out-lasts who.
    On reflection I agree with what you are saying. Ken Schwaber gives an example of a project he was consulting on where they came up with a novel way to use a gantt chart (MS Project) to track the progress of stories. The management where happy to try something new on this project, which had failed a number of times in the past, but insisted on using the same project tracking process and tools that had failed them in the past. I think this example makes my point too:
    I guess my point is that the organisation needs to provide a strong and consistent lead from the top. If this is done then at least everyone knows where they stand.
    I guess your point is that we need to remain open minded and be willing to be flexible and inventive inorder to instigate change. On reflection I think you're right. Paul.
  24. Hi All,
    For agile development to successfully work, agile management should also be put in place.
    I totally agree. Quality costs, and there is no such thing as a free lunch. If your management are happy not knowing the quality of the software at release and can live with that.. And if they can live with the increased cost of change as technical debt grows and needed refactors cannot occur because of a lack of tests... And they are happy to live with Systems with a short lifespan, because the lack of refactors leads to entropy that can only be resolved with a complete code re-write .. then, sure forget testing, forget peer review (pair programming), forget quality; just write the code and ship it! Paul.
  25. if they can live with the increased cost of change as technical debt grows and needed refactors cannot occur because of a lack of tests...
    The problem w/ this notion is that it assumes the stake holders/management will be willing to both recognize the consequence of the decision (in how you said it which I agree w/) and accept the responsibility for it. In my experience, I've found that managers who dictate your process to you are unable/unwilling to accept that quality needs to be compromised by wholesale shortcuts. You will get into philisophical arguments/debates about how quality doesn't have to be sacrificed. Uninformed and uneducated statements like "we'll mitigate bugs by making sure we clearly understand requirements upfront etc" can't be undeniably refuted by logic alone because *in theory* they're right....but then again...in theory communism works too.
  26. Middle Ground[ Go to top ]

    I've had a short experience in a pilot project using PSP (SEI's Personal Software Process) some years ago, and ended up with good impressions about it. Too bad we hadn't had the chance of using TSP (Team Software Process) further on, but I found it to be a good balance between process formalism and agility. IMO that's what we should aim for in a software development process: a good balance between control and agility.
  27. Re: Middle Ground[ Go to top ]

    I've had a short experience in a pilot project using PSP (SEI's Personal Software Process) some years ago, and ended up with good impressions about it. Too bad we hadn't had the chance of using TSP (Team Software Process) further on, but I found it to be a good balance between process formalism and agility. IMO that's what we should aim for in a software development process: a good balance between control and agility.
    Wow...that's the process I'm following now. Personally I think PSP and TSP is a god aweful process. I like it's intentions (particularly gathering metrics), but HATE it's execution. I feel there's times/places where it should be preferred over an agile approach such as when developing uber-critical applications that have a lot of externally imposed regulations (aviation or medical applications)...or perhaps when the development team is mostly composed of very anal and methodical developers. I think the majority of business applications and teams would do well to steer clear of it though.
  28. Re: Middle Ground[ Go to top ]

    I've had a short experience in a pilot project using PSP (SEI's Personal Software Process) some years ago, and ended up with good impressions about it. Too bad we hadn't had the chance of using TSP (Team Software Process) further on, but I found it to be a good balance between process formalism and agility. IMO that's what we should aim for in a software development process: a good balance between control and agility.


    Wow...that's the process I'm following now. Personally I think PSP and TSP is a god aweful process. I like it's intentions (particularly gathering metrics), but HATE it's execution. I feel there's times/places where it should be preferred over an agile approach such as when developing uber-critical applications that have a lot of externally imposed regulations (aviation or medical applications)...or perhaps when the development team is mostly composed of very anal and methodical developers. I think the majority of business applications and teams would do well to steer clear of it though.
    Well, maybe I should clarify my situation: back then we had no process at all, so I think any process would be seen as a "good thing" anyway... OTOH, we are now locked in a waterfall process, which may put PSP in a good light too by comparison. I agree gathering metrics is a pain, and it will always be, but this is a great part of control. This is an area in which process can be improved: somehow gather good metrics automatically while leaving people free to do their actual jobs. Automatic audit of CVS/SVN LOC commits (my wild guess), or something in this line...
  29. Re: Middle Ground[ Go to top ]

    I've had a short experience in a pilot project using PSP (SEI's Personal Software Process) some years ago, and ended up with good impressions about it. Too bad we hadn't had the chance of using TSP (Team Software Process) further on, but I found it to be a good balance between process formalism and agility. IMO that's what we should aim for in a software development process: a good balance between control and agility.


    Wow...that's the process I'm following now. Personally I think PSP and TSP is a god aweful process. I like it's intentions (particularly gathering metrics), but HATE it's execution. I feel there's times/places where it should be preferred over an agile approach such as when developing uber-critical applications that have a lot of externally imposed regulations (aviation or medical applications)...or perhaps when the development team is mostly composed of very anal and methodical developers. I think the majority of business applications and teams would do well to steer clear of it though.

    Well, maybe I should clarify my situation: back then we had no process at all, so I think any process would be seen as a "good thing" anyway... OTOH, we are now locked in a waterfall process, which may put PSP in a good light too by comparison.
    I agree gathering metrics is a pain, and it will always be, but this is a great part of control. This is an area in which process can be improved: somehow gather good metrics automatically while leaving people free to do their actual jobs. Automatic audit of CVS/SVN LOC commits (my wild guess), or something in this line...
    Hi I've used PSP and TSP and it felt like a time and motion study (having to use a stop watch and everything). It definately didn't suit me and I wouldn't recommend it. As for control, have you heard of imperical process control? As mentioned by a previous poster:
    Inspect and Adapt.
    It works, and it's based in science (closed loop control systems). Imperical process control is central to all Agile methods, but the Scrum book (Schwaber and Beele) gives a pretty good explanation. IMO the dificulty with introducing Agile approaches are that you need to practice them to really get it. I was a sceptic once, but after experiencing it working in practice I became a true believer. Paul.
  30. If you, like me, think software should be developed using a process or methodology tailored for the combination of risks, skills etc, you may find Balancing Agility and Discipline useful. Forewords by Grady Booch, Alistair Cockburn and Arthur Pyster, along with a sample chapter Software Development: Agile vs. Disciplined Methods.
  31. It is a well-known fact that Agile development places many constraints on things that developers do not control, such as how frequently a client is available to developers, and many others. I have not really seen the equation of agile advocate = arrogant, but I can definitely see why he would get frustrated talking to Agile developers who happen to be arrogant. The shortest path to end of frustration is to stop talking to arrogant people. (Somebody please give me the same advice when I might need it next.) Guglielmo Enjoy the Fastest Known Reliable Multicast Protocol with Total Ordering .. or the World's First Pure-Java Terminal Driver
  32. Hi Cedric, No, I think you should say that "People still don't get it". It sounds as though it's time for the Agile backlash. I consider myself an Agile Thinker and I totally agree with your main point, namely that there are no hard and fast rules. Agility IMO is all about values and principles. Values and principles do not lend themselves to hard and fast rules. I think the Poppendieck's put it well in their book "Lean Software Development":
    Principles are universal, but it is not always easy to see how to they apply to particular environments. Practices, on the other and, give specific guidance on what to do, but they need to be adapted to the domain.
    When most developers talk about Agile, what they really mean is XP, which is unfortunate as XP is the only Agile approach that chooses to focus on specific technical practices. The XP practices are consistent with Agile values and principles, but they can by no means be considered universal. I've taken Kent Beck to task about this very point myself, and the fact that the 2nd edition of his book "XP Explained" adds an additional 12 practices would imply that he agrees (in fact I think he makes this very same point quite clearly in both editions of his book, but this fact is seldom acknowleged). The truth is that there needs to be as many processes out there as there are teams and organisations. Each team needs to work out it's own process for itself. Having said this I believe that Agile values and principles can be universally applied, I also believe that XP as a boiler plate process is often a good starting place and I would caution on tinkering with XP untill you've given it a real good go with the help of an experienced XP Coach and have gained the required understanding through personal experience. To get a more values focused view of Agile development I would recommend reading Alistair Cockburn (Crystal Clear), ken Schwaber (Scrum) or as I mentioned earlier the Poppendiecks (Lean Software Development). So the next time you go to a presentation where the focus is on a set of Agile practices, ask them to explain which Agile values and principles the practices support and why. If met with silence then perhaps the best thing to do is leave :^). Paul.
  33. A few months back I went to a talk by Mike Cohn, who's written a few books on Scrum. It was well worth the time & money, mainly because Mike has lots of experience applying Scrum and other agile-like techniques to software development processes in real-world situations. During the first few minutes of the talk, Mike said (I'm paraphrasing here) "The next slide is really the best advice I can give you about managing any empirical process." He flipped the slide, which simply said: Inspect and Adapt. His point is this: any hard-liner that strictly follows the Waterfall development model in a real-world project will run into his share of trouble. Ditto for an Agile junkie, a category that the speaker Cedric is commenting on probably fits. You should constantly (as in daily) inspect your development process and see where it is failing. Then try to resolve the problems with any tools you have at hand: agile, waterfall, or otherwise. The trick is being able to change the process in-flight, and solve issues with the process while you are still delivering software on time.
  34. Cedric worries that agilists will be too risk averse to ship any untested software. I've seen the opposite problem. On the Agile Management group (http://groups.yahoo.com/group/agilemanagement/?yguid=151433109) there's an agilist whom I respect a lot and a traditionalist who develops spacecraft control software using a traditional, proven method. Some time ago they went back and forth about converting to agile. The conversation went something like this: Agilist: You should convert your process to agile. You'll reap big productivity benefits. Traditionalist: Our process works and there are unknown risks to the changes you propose. Agilist: But try it. You stand to gain a lot and what do you have to lose? Traditionalist: The whole spacecraft. It's easy to gamble with somebody else's project. Agilists like Kent Beck claim that agile is a risk management strategy, but I've never seen a comparison of the risks of agile to the risks of other processes. I don't see how you can quantify the risks of various software processes, so claims about risk reduction are subjective at best. Here's a counter example where risks were reduced by making the process more heavyweight: http://www.fastcompany.com/online/06/writestuff.html That's what I would call extreme programming. When they find a bug in the software they don't just fix the bug, they start an investigation into their process to see what in their process allowed the bug in the first place. I wouldn't want to work like that but I'm glad they do. Would you want to fly on an airplane whose software was written by typical XPers? You need the right people and the right process. I'm an agilist, and I've grown tired of the hype from agilists. Many agilists have never thought through the claims made for agile, but they repeat them anyway. Tim Lister was asked what he thought about agile methods once. Part of his response was 'Let's avoid the sophomoric idea that everything new is The Answer'.
  35. Cedric worries that agilists will be too risk averse to ship any untested software. I've seen the opposite problem. On the Agile Management group (http://groups.yahoo.com/group/agilemanagement/?yguid=151433109) there's an agilist whom I respect a lot and a traditionalist who develops spacecraft control software using a traditional, proven method. Some time ago they went back and forth about converting to agile. The conversation went something like this:

    Agilist: You should convert your process to agile. You'll reap big productivity benefits.

    Traditionalist: Our process works and there are unknown risks to the changes you propose.

    Agilist: But try it. You stand to gain a lot and what do you have to lose?

    Traditionalist: The whole spacecraft.


    It's easy to gamble with somebody else's project.

    Agilists like Kent Beck claim that agile is a risk management strategy, but I've never seen a comparison of the risks of agile to the risks of other processes. I don't see how you can quantify the risks of various software processes, so claims about risk reduction are subjective at best.

    Here's a counter example where risks were reduced by making the process more heavyweight:

    http://www.fastcompany.com/online/06/writestuff.html

    That's what I would call extreme programming. When they find a bug in the software they don't just fix the bug, they start an investigation into their process to see what in their process allowed the bug in the first place. I wouldn't want to work like that but I'm glad they do.

    Would you want to fly on an airplane whose software was written by typical XPers? You need the right people and the right process.

    I'm an agilist, and I've grown tired of the hype from agilists. Many agilists have never thought through the claims made for agile, but they repeat them anyway.

    Tim Lister was asked what he thought about agile methods once. Part of his response was 'Let's avoid the sophomoric idea that everything new is The Answer'.
    You should read Craig Larman's book : Agile and Iterative Development: A Manager's Guide. Best book I have found on the subject. There is an entire chapter consacred to statistical evidences of the increased productivity. And there is a chapter that describe how to sucessfully mix some of the best practices of XP, RUP and SCRUM based upon your needs. I have no affiliation with the author but I must say his book was worth every cent I paid.
  36. You jumped from a comment about risk right into productivity. Not recognizing the risks of abandoning a proven process is part of the problem with agile promotion. If your existing non-agile process isn't working, however, then those risks are small. Larman's book has risk profiles for waterfall vs. iterative development, but he doesn't say where he got them from or if the vertical axes are the same (how the risks compare). It's also a false dichotomy to say it's either waterfall or agile. Iterative development pre-dates agile, and it is the iterations that can reduce risk. I say "can" reduce risk because you can have iterations that don't run from front to back or that don't touch all the integration points. Then you are still left with those risks. I've heard the term "steel thread" used to describe functionality that runs all the way through an application. What you really want is iterations with steel threads. Phillip Armour points out in his book "The New Laws of Software Process" that the waterfall has always had feedback loops and iterations. They're just longer than agile feedback loops. One failed project I know of used iterations. The project manager set the iteration close date several weeks in advance based on when the QA team was scheduled to be ready to begin another round of testing. Whatever was ready at the iteration close date is what got sent to QA regardless if there was any new functionality that was working from front to back. Developers were frequently assigned to implement new functionality even if their previous work wasn't working all the way through because it was blocked by something else. As a result most iterations had little new functionality to test. Management still claimed each iteration showed progress because several new features were added that were 60% - 80% complete. The number of 60% - 80% complete features always increased, but the number of 100% complete features changed little. Iterations aren't magic. Shortening the iteration cycle makes sense most of the time, but it has to be done right.
  37. You jumped from a comment about risk right into productivity. Not recognizing the risks of abandoning a proven process is part of the problem with agile promotion. If your existing non-agile process isn't working, however, then those risks are small.

    Larman's book has risk profiles for waterfall vs. iterative development, but he doesn't say where he got them from or if the vertical axes are the same (how the risks compare).

    It's also a false dichotomy to say it's either waterfall or agile. Iterative development pre-dates agile, and it is the iterations that can reduce risk. I say "can" reduce risk because you can have iterations that don't run from front to back or that don't touch all the integration points. Then you are still left with those risks. I've heard the term "steel thread" used to describe functionality that runs all the way through an application. What you really want is iterations with steel threads.

    Phillip Armour points out in his book "The New Laws of Software Process" that the waterfall has always had feedback loops and iterations. They're just longer than agile feedback loops.

    One failed project I know of used iterations. The project manager set the iteration close date several weeks in advance based on when the QA team was scheduled to be ready to begin another round of testing. Whatever was ready at the iteration close date is what got sent to QA regardless if there was any new functionality that was working from front to back. Developers were frequently assigned to implement new functionality even if their previous work wasn't working all the way through because it was blocked by something else. As a result most iterations had little new functionality to test. Management still claimed each iteration showed progress because several new features were added that were 60% - 80% complete. The number of 60% - 80% complete features always increased, but the number of 100% complete features changed little.

    Iterations aren't magic. Shortening the iteration cycle makes sense most of the time, but it has to be done right.
    Agree with you. I am not convinced in every agil practices. I was just recommanding the book because I think it is the best intro on the subject.
  38. Hi,
    One failed project I know of used iterations. The project manager set the iteration close date several weeks in advance based on when the QA team was scheduled to be ready to begin another round of testing. Whatever was ready at the iteration close date is what got sent to QA regardless if there was any new functionality that was working from front to back. Developers were frequently assigned to implement new functionality even if their previous work wasn't working all the way through because it was blocked by something else. As a result most iterations had little new functionality to test. Management still claimed each iteration showed progress because several new features were added that were 60% - 80% complete. The number of 60% - 80% complete features always increased, but the number of 100% complete features changed little. Iterations aren't magic.
    Wow. This little episode breaks virtually every Agile value, principle and practice I know... Like some else pointed out:
    Give a monkey an hammer...
    I would suggest that get yourself a good Agile reading list, that way you'll recognise "pilot error" when you see it. Paul.
  39. Agilists like Kent Beck claim that agile is a risk management strategy, but I've never seen a comparison of the risks of agile to the risks of other processes. I don't see how you can quantify the risks of various software processes, so claims about risk reduction are subjective at best.
    Risk stems from uncertainty. Traditional processes often fall down because requirements are "done" but there's a low confidence that they are "correct," designs are "done" but there's a low confidence that they are correct to the requirements - which of course are low confidence to start with. Consequently, by the time you write code, you're pretty sure that it isn't correct. Consequently, the s/w development team gets blamed for the problem when the root cause is the "customer" didn't really know what it wanted in the first place. So Agile turns the process on it's head. It assumes that: 1. The customer doesn't really know what it wants 2. The customer will only know what it wants when it sees it or more generally implementing working software is the most cost effective way to ellicit correct requirements. 3. The customer should control the order in which functionality is implemented at a fine grained level - which for all practical purposes means the customer determines how development resources are allocated. ----------------- I would say that the primary motivation for these changes are not that it is a more effective way to develop software, but rather that it keeps responsibility for the big picture out of the hands of the development team and in the hands of the customer. So if the project fails, it's the customers fault. If most "customers" are like my "customers," projects can't fail because of them, so placing fault on the "customer" magically transforms a failed project into a glowing success. ----------------- Therefore, I would argue that Agile doesn't mitigate or avoid risk, it simply transfers it from the development team to the customer, and from the customer's "project team" to the organization as whole. Why do I say this? Because none of the "Agile assumptions" are universal truths, but rather simply common conditions. ----------------- So, when the "Agile assumptions" hold, is Agile desirable? Maybe. It depends on how competitive you want your development organization to be. If you want to be truly competitive, you have to fix the root cause, not just (rightly) avoid blame for it.
  40. Hi Erik, Your analysis is more or less spot on IMO. I think your complaint isn't so much with Agile development, but with the (possible) lack of robustness and precision that the business may display when envisioning software systems. Placing the responsibility for envisioning new systems in the hands of the business/workers IMO correctly aligns responsibility with control. In an Agile business decision making is left to the workers on the ground. The assumption is that they know best what they need, and that they will be first to detect the need for change and hence new system requirements. It is a fallacy that centralised planners and analysts are more able than small teams on the ground acting autonomously to address problems in there day to day work. It is also a fallacy that deterministic analysis is always more effective and efficient than emperical adaptation and change through experimentation. There has been research that shows that as systems become more complex, the most effective approach to continuous improvement is to emperically inspect and adapt. It is these ideas that are at the core of Agile thinking, and they stem from ideas that have been used to revolutionise business in Japan at companies like Toyota and Honda. The traditional approach to software is more akin to Taylorism and mass production, where you assume that the workers are dumb and everything can/should be analysed and planned by smart people at the centre. Another good analogy is a planned economy versus a market economy or Just In Time production verses centralised Material Resource Planning. For complex systems a number of autonomous intelligent and independent entities working in concert will always out perform a single planner at the centre who is trying to address every eventuality through detailed analysis and plans. This idea is generally accepted in business, even the Chinese Government have woken up to the benefits :^) If I wanted to know how to improve any given process in a company I would talk to the people who use that process each and every day, and have years of practical experience. I would talk to the workers. I would trust what they had to say. I would not have much faith in a report produced by an Analyst from HQ or a MBA Management Consultant hired yesterday who had never actually done the job. Agile development, places software developers face to face with workers, removing much of the noise in between. It comes down to whether you are comfortable trusting customer teams (i.e. workers or what the Japanes would call quality circles) to quickly home in on the best solution to their own problems themselves. This places a large responsibility on the workers, but IMO this is how it should be. Paul.
  41. Your analysis is more or less spot on IMO. I think your complaint isn't so much with Agile development, but with the (possible) lack of robustness and precision that the business may display when envisioning software systems.
    I wouldn't call it a complaint, more of an observation.
    Placing the responsibility for envisioning new systems in the hands of the business/workers IMO correctly aligns responsibility with control.
    Yup.
    There has been research that shows that as systems become more complex, the most effective approach to continuous improvement is to emperically inspect and adapt.
    I pretty sure research to the contrary with equal support could be found. It depends on the organization and the products/services the organization provides.
    Another good analogy is a planned economy versus a market economy or Just In Time production verses centralised Material Resource Planning. For complex systems a number of autonomous intelligent and independent entities working in concert will always out perform a single planner at the centre who is trying to address every eventuality through detailed analysis and plans.
    The empirachal fact that MRP systems tend to be costly sources of inefficiency rather than the panaceas of cost reduction does not prove that the concept MRP is incorrect, but rather that the current implementation (business process and software) is ineffective. In my opinion, most efficiency gains over the past century have been due to macro optimizations. Artisans can't compete with assembly lines. But now there's so much efficiency at the macro level and fat at the micro level, the opportunities for optimization are at the bottom of the structure. Eventually that will change and the pedulum will swing back towards macro optimization. Ultimately, innovation in business and the software systems that support it will come from freeing businesses from "the tyranny of 'OR'" by enabling them to optimize the forrest and the trees concurrently, rather than swinging back and forth.
    If I wanted to know how to improve any given process in a company I would talk to the people who use that process each and every day, and have years of practical experience. I would talk to the workers. I would trust what they had to say.
    I would also trust that I would receive 3 definitions of the process for every 1 person I asked about it - and none of them would be what's really done. The reality is to people process is contextual. If you put your questions in a certain context, you'll get it in that context. If you don't, you'll get whatever context is at the front of the person's mind. In my opinion, traditional analysis methods should be used until the team arrives at a stable definition of the process (often through fatigue ;-) that is sufficiently general to cover all the uncovered contexts. Usually this means defining a substantially lower level of automation than is desired, because all the context sensitive decisions must be made by people, not software. If this level of automation is insufficient to make the application useful, then you define context sensitive automation for the extremely common cases. Finally, you implement it, and use Agile techniques to gradually add the additional cases.
    I would not have much faith in a report produced by an Analyst from HQ or a MBA Management Consultant hired yesterday who had never actually done the job.
    Me too, but they illustrate a substantial opportunity. Why do these high-price snake oil salesmen continue to rake in money when most people know they're con artists? It's because businesses have a substantial need to get their hands around their processes in an understandable and actionable way. The fact that snake oil doesn't cure cancer doesn't change the fact that you have cancer and the only option being placed on the table is snake oil. Some technology consulting companies try to capitalize on this opportunity by offering other types of consulting as well, but this just leads to more greater snake oil production. The reason for all the snake oil is that such firms (and frequently internal people as well) sell themselves to executives, not the people executing the processes.
    Agile development, places software developers face to face with workers, removing much of the noise in between.
    I don't think this scales in terms of problem complexity and organizational size. That's why companies that successfully implement Lean use the efficiencies they gain to dedicate the best workers to Kaizen rather than to reduce heads. Now, for the rest of us who don't work for Toyota and a few other elite manufacturers, we need a way to accurately define and optimize processes w/o access to people dedicated to the task.
    This places a large responsibility on the workers, but IMO this is how it should be.
    ...and the workers fail, but the projects don't because the concept of success has become so muddled due to increased politics and obscured objectives that it's easy to declare success even while making the business less efficient. ...and Agile gets good stats on successful projects. The long term opportunity here is to provide someone (or be someone) with sufficient expertise in the subject matter, process optimization in general, and in s/w development to fill the gaps. This person could come from the business, from the development team, or from an outside party. Shifting the responsibility from the dev team to the business better aligns it with who has the ability, but it ignores the opportunity to address a substantial need. Implementing requirements that will change tomorrow, and then continuously refactoring to address the daily changes may produce lots of uncontested billable hours, but ultimately it doesn't lead to increased success for the customer. Ultimately your long term success is going to be determined by the value received from the applications you build. Consequently, it's in your best interest to ensure your customer is on the right path.
  42. Hi Erik,
    Implementing requirements that will change tomorrow, and then continuously refactoring to address the daily changes may produce lots of uncontested billable hours, but ultimately it doesn't lead to increased success for the customer. Ultimately your long term success is going to be determined by the value received from the applications you build. Consequently, it's in your best interest to ensure your customer is on the right path.
    It's obvious that you do not trust your customers to define their own solutions. Poor story telling can lead to the requirements churn you describe. But good collaboration, between intelligent customers and skilled developers can lead to remarkable results. It's all down to choice, and any process can be made to work or fail depending on the quality of the people involved in its execution. Paul.
  43. It's obvious that you do not trust your customers to define their own solutions.
    No, I don't. I don't even fully trust them to precisely define their problems and opportunities. But that's not a matter of "trust" in the more personal sense, it's a matter of skillset. S/W built on inconsistent logical constructs is fundamentally flawed. Am I to expect all my customers to predicate logic? Set theory? Relational theory? Differential equations? Let's say you're going to implement a workflow as a state machine. The customer specified flow leads to a state machine that can relatively easily get stuck in a non-final state, but it will only happen on ~5% of the cases. If you implement the customer specified process, you are knowingly implementing software that will fail ~5% of the time, potentially bringing a critical business process to a grinding halt. So, in my opinion, it's my job to work with my customer until they have a sufficiently logically sound definition of their process for it to be implemented in software.
    Poor story telling can lead to the requirements churn you describe. But good collaboration, between intelligent customers and skilled developers can lead to remarkable results.
    I have no doubt of this. In fact, I'd say the problem isn't the interaction between the developers and the customer. It's the interaction between the various customer stakeholders. In my experience, customer stakeholders disagreeing to the point of yelling at each other - even ones from the same department - is more the rule than the exception. And honestly it's usually a sign of progress, because it means they've stopped being intentionally vague about anything where they know there will be disagreement. How do you finish a user story if the users storm out of the meeting in anger (at each other) before it is finished? If you work for a contract development company or an outsourcing company, then I 100% agree that the customer should have these issues worked out before it brings you in to develop software. It's embarassing to show such juvenille behavior to outsiders. But it's also a natural consequence of putting a bunch of intelligent, passionate, pig-headed people in a room together.
  44. Hi Erik,
    How do you finish a user story if the users storm out of the meeting in anger (at each other) before it is finished?
    I was splitting my sides when I read this one! So you've got a team of customers who can't agree. OK, is this team able to self organise and appoint themselves a leader? If helped, are they able to break down what they want into small usage scenarios (user stories)? With help are they able to then prioritise these stories? Try giving them some rules along the lines above, and arbitrating (or even taking a back seat). What are the chances of them coming up with some workable ideas themselves? If the answer is slim, then I would suggest that you escalate to senior management. This team obviously has no (political) incentive to come to a consensus, and the project is unlikely to succeed.
    Am I to expect all my customers to predicate logic? Set theory? Relational theory? Differential equations?
    No, but you should expect them to know their business, and be able to reason about how they intend to use the system using business language. If there are flaws in their reasoning, then it is your job to point them out, naturally. What makes you more qualified than them to spec out what's needed? The only difference appears to be that you’ve got a clear incentive to make the project succeed. They should have a clear stake in the project success too; otherwise what are they doing on the customer team? (I have come across users whose sole aim is to ensure that a project fails). I would suggest that you've got a people problem. There are guerrilla tactics that you could use to help deal with this without taking over the role of customer yourself.
    So, in my opinion, it's my job to work with my customer until they have a sufficiently logically sound definition of their process for it to be implemented in software.
    I totally agree, but that doesn't mean becoming the customer. After all, if it ends up being your system, what buy-in does your customer now have? Also what incentive does your customer have to champion your system and make it succeed in the field? ... I wish we could discuss this one further offline. Have fun :^) Paul.
  45. I wish we could discuss this one further offline.
    erik dot engbrecht at gmail dot com
  46. I would also trust that I would receive 3 definitions of the process for every 1 person I asked about it - and none of them would be what's really done. The reality is to people process is contextual. If you put your questions in a certain context, you'll get it in that context. If you don't, you'll get whatever context is at the front of the person's mind. In my opinion, traditional analysis methods should be used until the team arrives at a stable definition of the process (often through fatigue ;-) that is sufficiently general to cover all the uncovered contexts. Usually this means defining a substantially lower level of automation than is desired, because all the context sensitive decisions must be made by people, not software. If this level of automation is insufficient to make the application useful, then you define context sensitive automation for the extremely common cases. Finally, you implement it, and use Agile techniques to gradually add the additional cases.
    This is precisely the type of "analysis" that occurs during story telling. The only difference, is that the stories are firmed up at the last possible moment (a story is an agreement to discuss a required feature, in the extreme a customer explains the final details just before the programmer codes) rather than up front. The stories become more defined, detailed and concrete as the team come closer to the iteration in which they will be implemented. There are several advantages to leaving decisions open as long as possible. This allows the customers to keep their options open allowing them to make choices based on feedback from stories that have just been implemented. Meaning that they do not get locked in to poor decisions made early on. There is a whole set of skills required to make Agile work. It requires at least one experienced practitioner on the team, sometimes referred to as a Coach or a Scrum master. This person is responsible for coaching the developers and the customers and often also the management too. A sterile discussion on pros and cons of Agile is pointless IMO. All I can say is try it out with a team that know what they are doing and then come to a decision for yourself. I have and I haven't looked back. Paul.
  47. Disagree[ Go to top ]

    I would disagree with Cedric. I'm not a agilist, however I'm working in a huge SW development company and we are using Agile methods (Scrum). Our projects are done in 3 countries in 2 different hour zones (allthoug with only 1 hour difference). Code base is huge, continious integration process spans acccross multiple sites. Allot of Agilist are supporting our company on this way and for me it seems to be quite successfull and most of the people are much more enjoy Scrum that previous waterfall (and for me this is the most important factor that people are enjoyed and like to work with that).
  48. Give a monkey a hammer...[ Go to top ]

    A hammer, screwdriver or even a nail is only as good as the monkey you give it to in combination with the situation you put the monkey and the hammer in. For god's sake, just do what makes sense for the the combination of culture, developers and customer. Any person stuck on one method is only as useful as what they preach anyway. This is obvious to the developers and architects here but not so obvious to management who aren't usually comfortable in loosely defining process. Oh well.
  49. This has been going on for a long time, now, and is becoming really worrying. There is a whole bunch of people who'd do everything to be "recognized" as "thought-leaders" Soon enough we won't have anybody who bothers to think. We will only have three groups: people who try to make buzz of trivial stuff, that other people have been practicing for a very long time; people who were too late to get on that train and hence try to retaliate by criticizing; and finally the largest and most disgusting group of penguins who are too lazy to think for themselves and just blindly join one or the other group. That way, they won't have to justify their poor decisions to management - just mention some buzz words, some "legendary" names and - you are off the hook, if your manager is stupid nough, which of course, they usually are. For God's sake, when are people going to stop throwing around buzz words and and start thinking for a change? PEOPLE, it does not freaking matter whether you do agile or whatever, as far as you do your job well. And if you dont - you could as well get lost in a very agile way. sigh and cheers to all people who still have some common sense.
  50. BS[ Go to top ]

    And this is my main beef with Agilists: I strongly suspect that most of them are spending their time on open source projects with like-minded fellows, but none of them have any experience what companies whose survival depends on shipping software have to go through to organize huge code bases growing thousands of lines of code every day under the combined push of hundreds of a developers, all with their personal background, education and bias.
    This statement, and the blog that are complete crap that demonstrate and ignorance of the origins of agile software development. Agile methodologies were invented by people in corporate settings who were tired been forced to dance the big death march that waterfall projects imposed. The author wrongly thinks agile software is related to the open source phenominon. The relationship is one of choice, and many open source projects are NOT agile (look for big suites of unit tests to exclude the ones that aren't). At its heart, agile software is based on the same principles as JIT and lean manufacturuing. Some agile software methodologies originated in the automotive industry during the time that they were questioning how they did their manufacturing operations. The guiding principle of agile software is a contextual restatement of the "eliminate waste, eliminate inventory" mantra that drives most successful manufacturing companies today. In software, inventory is the stock pile of programming artifacts (docs, code, test scripts, etc...), and waste is any activity that isn't an *inherent* part of shipping working production code. While it isn't possible to eliminate all waste, you can compare two processes and ask which is "better". The graph of shipped, working functionality vs time will move upwards as successful releases occur. The lower the waste of the process, the greater the slope of this line, on average. Agile processes move up in smaller, more frequent increments. His points are banal: - "Tests are not specs". "how do you specify an exponentiation function with a test?" Simple: assert(2^3,8), assert(3^2, 9), assert (4^4,256), assert (5^5,3125). Could someone figure out a mathematical function that also matched the test cases. Yes. Is any real person likely to do so. No. And if they did, I could quickly add a new assertion that totally broke their code. They know this, so they don't provide flippant answers. His fallacy is that he expect tests to exclude all possible nonconforming solutions. That is meaningless if you accept what experience shows -- that requirements change over time. This is true even for something as appearently static as mathematical exponentiation. I challenge anyone who agrees with the author to post their "spec" for the exponentiation function. I will then confer with my on site customer (me) and provide a new requirement it doesn't meet. Didn't handle non-integers? I want that. Did handle them? Then I don't care, but I do want performance for integers beyond what floating point ops can provide? Complex numbers, symbolic functions, etc, etc, etc... If you have the crystal ball that allows you to once and for all predict what businesses need in the future, you should be playing the stock market, not coding. The fact is that picking which "spec" is the correct one is just as much of an ad hoc proclaimation as picking the sample assertions, except it takes A LOT more time and is more error prone, and takes a higher degree of training to avoid saying things which have no information content. I think the author is the one who doesn't understanding "the meaning of calculated risk". - "If it's not testable, it's useless." His entire argument is based on a pathetic ability at reading comprehension -- his arguments are rebuttals to the very different assertion that "If it's not test**ed**, it's useless". - "But as the deadline looms, these choices need to be reconsidered all the time and evaluated while keeping a close eye of the final goal." The root of true waste in software systems is sacrificing quality of implementation to make arbitrary deadlines. Things should ship when they are ready. Not before, not after, no exceptions. Every deviation from this is waste. I encourage agilists to keep being as obstinate as your are. People who defend waterfull DESERVE derision. They are advocating a stupid methodology with proven track record -- of FAILURE after FAILURE. The people who advocated big batch manufacturing in the 70's and 80's were proven wrong, and so will the few remaining advocates of waterfall.
  51. Re: BS[ Go to top ]

    The guiding principle of agile software is a contextual restatement of the "eliminate waste, eliminate inventory" mantra that drives most successful manufacturing companies today.
    Doesn't work. You can minimize inventory, and that level may be surprisingly low. But you can't eliminate it without destroying your business. The change in mentality is that you treat inventory as a cost rather than an asset, so it must be generating value in order to be kept.
    In software, inventory is the stock pile of programming artifacts (docs, code, test scripts, etc...), and waste is any activity that isn't an *inherent* part of shipping working production code.
    So what does "inherent" mean? One project may need hundreds of pages of specification, while another just needs a few powerpoint slides outlining what the system will do. The focus is on "what produces value."
  52. To them, Agility is a silver bullet that is applicable in all cases with no compromises.
    Says who? Give me names. You won't find mine among those who don't compromise. Are there arrogant people in the Agile world? Sure. Are there arrogant people in the non-Agile world. Yes. Are there arrogant people in any industry? Yup. Someone who is sick and tired of watching money thrown away using software development methods that just don't work very well may be perceived as being arrogant when they encounter resistance to methods that they have seen work. I do believe that Agile methods can (and have been) applied to very diverse projects and industries, including real-time and life critical applications. Indeed, Kent Beck first saw TDD being used for the firmware on a pacemaker. Someone already mentioned the article in FastTimes about the Space Shuttle avionics software team. They have always practiced iterative development, and their testing strategies fall very much within the realm of what an Agile team would want. Larman and Basili's article mentions that software on the Mercury Project was test-driven... in 1959!!
    The truth is that these practices come at a price, and for a lot of organizations, the price gets high very quickly.
    Whose truth, and what price? Please be specific. I would argue that the price of not using Agile methods is exorbitantly high. You may cut some corners now to save time, but that will come back to bite you in the back-side at a later date. TDD is an investment that pays off over the lifetime of a project. It's quite similar to a credit card - it you pay it off each month, then your interest charges ar minimal or simply 0. If you carry a balance and only make the minimum payment, then you pay through the nose. If you continue to rack up the charges on the card, you end up with Hired Goons knocking at the door.
    But the truth is: at times, I don't do TDD because implementing a feature quickly is more important than a fuzzy feeling.
    "Hired goons!"
    And this is my main beef with Agilists: I strongly suspect that most of them are spending their time on open source projects with like-minded fellows, but none of them have any experience what companies whose survival depends on shipping software have to go through to organize huge code bases growing thousands of lines of code every day under the combined push of hundreds of a developers, all with their personal background, education and bias.
    That's the biggest load of crap I've heard in a long time. I've been officially using Agile methods for 5 years in a corporate environment. I have contributed a single bug fix to an open source project over that time. I have coached teams to become Agile in a corporate environment. None of them were producing open source software. With the exception of some people I know at what used to be OTI (the people behind Eclipse), every person I know in the Agile community works on software that isn't open source. Get your head out of the sand, Cedric!!
    So here is my advice to Agilists: get real now, or you will become irrelevant soon.
    Funny, I was thinking the same thing in the other direction. "First they ignore you, then they ridicule you, then they fight you, then you win." Mahatma Gandhi That being the case, then things are looking good for Agile development. Dave Rooney Mayford Technologies
  53. I do believe that Agile methods can (and have been) applied to very diverse projects and industries, including real-time and life critical applications. Indeed, Kent Beck first saw TDD being used for the firmware on a pacemaker. Someone already mentioned the article in FastTimes about the Space Shuttle avionics software team. They have always practiced iterative development, and their testing strategies fall very much within the realm of what an Agile team would want. Larman and Basili's article mentions that software on the Mercury Project was test-driven... in 1959!!
    How many agile methods does a team need to follow in order to be agile? How short does an iteration need to be before it is an agile iteration? I think XP may have tainted the concept of agile development, because it quite explicitly says you need to adopt all of its practices in order for it to work. Consequently, people think you're either agile or not. Really, IMHO, agile simply means adopting practices that enable you to cope with change in a cost/time effective manner. Depending on the situation, an "agile practice" may or may not be agile. I would argue that it's more agile to have one analyst/mediator work with the customer stakeholders for six months than to have 5 programmer/analysts spend six months reinventing their codebase. Sometimes iteration leads to oscillation, and oscillation is definately NOT agile.
  54. Someone already mentioned the article in FastTimes about the Space Shuttle avionics software team. They have always practiced iterative development, and their testing strategies fall very much within the realm of what an Agile team would want.
    Claims like this hurt agile's credibility. If you actually read that article their process is the antithesis of agile. It is very heavyweight, very Big Up Front Design: http://www.fastcompany.com/online/06/writestuff.html
  55. Claims like this hurt agile's credibility. If you actually read that article their process is the antithesis of agile. It is very heavyweight, very Big Up Front Design:
    I did read the article. In fact, when I first read it a few years ago, I was struck by how test-infected the group was, and how similar that level of dedication to testing was to XP. While I agree that many aspects of their process are "traditional", they develop iteratively, use rigorous automated testing, and when fixing a bug not only just fix it but look for other places in the code where the same sort of bug could occur. Just because good ideas are used in other development processes doesn't mean they can't be adopted elsewhere. Indeed, there isn't anything new in the XP practices (which Kent Beck readily acknowledges) except for the fact that they are taken to the extreme (pardon the pun). So, please explain how taking good ideas from many different processes hurts the credibility of agile development. I had a conversation the other day in which we talked about how Agile development had changed from 5 years ago, and how it will likely be very different in another 5 years. We're learning every day what it is about agile that works, why it works, and how to amplify that. We're also learning about the shortcomings, why they are shortcomings, and how to fix them. Again, how is that hurting Agile's credibiity? Dave Rooney Mayford Technologies
  56. What hurts agile's credibility is claiming that agile somehow can be expected to produce results like a process that is the polar opposite of agile just because they share some practices. You started your paragraph in which you mentioned the shuttle software with the following: "I do believe that Agile methods can (and have been) applied to very diverse projects and industries, including real-time and life critical applications." I'd like to know what life critical applications have been done with XP, SCRUM, or FDD, and, if there are any, did they really follow those processes or just have some practices in common with them. Just because a Yugo has four wheels, an engine, transmission, etc. doesn't mean it will perform like a Toyota. Yugo may very well be using many of the same manufacturing processes that Toyota uses, but the results are very different.
  57. What hurts agile's credibility is claiming that agile somehow can be expected to produce results like a process that is the polar opposite of agile just because they share some practices. You started your paragraph in which you mentioned the shuttle software with the following:

    "I do believe that Agile methods can (and have been) applied to very diverse projects and industries, including real-time and life critical applications."

    I'd like to know what life critical applications have been done with XP, SCRUM, or FDD, and, if there are any, did they really follow those processes or just have some practices in common with them.

    Just because a Yugo has four wheels, an engine, transmission, etc. doesn't mean it will perform like a Toyota. Yugo may very well be using many of the same manufacturing processes that Toyota uses, but the results are very different.
    Well the Canadian Air Traffic Control system (CAATS) is one example. Feel free to google on the subject.
  58. What software in an air traffic control system is life critical? There are always humans watching those screens. The screens themselves probably have some software in them, but there are always back up screens. It's been some time since I filed a flight plan but when I did they would occasionally have to file it manually because the computer was down. Air traffic control systems have back up procedures at all critical points in case the computers go down. Life critical software is software like that described in the space shuttle article because if it fails the crew will probably die. Flight control software, nuclear power plant control software, and fire control software in combat aircraft and ships would be other examples. This article, which mentions CAATS, says that parts of some large projects are done with agile, though not the entire project: http://www-128.ibm.com/developerworks/rational/library/content/RationalEdge/aug04/5558.html It's a rather tentative endorsement of agile, though. Still looking for an example of life critical software done using agile.
  59. What software in an air traffic control system is life critical? There are always humans watching those screens. The screens themselves probably have some software in them, but there are always back up screens.
    When was the last time you worked on an Air Traffic Control System or something similar? Just about every single part is life critical.
  60. If you answer my question with a question we won't get anywhere. If you think every part of an air traffic control system is life critical who dies when one of their servers goes down? Who dies if the software to file flight plans goes down? No one does. Flights may be delayed but safety is guaranteed by other means. The software for the space shuttle is life critical because a human pilot cannot perform the flight operations needed in the time necessary. If that software fails the crew probably dies.
  61. If you answer my question with a question we won't get anywhere. If you think every part of an air traffic control system is life critical who dies when one of their servers goes down? Who dies if the software to file flight plans goes down?

    No one does. Flights may be delayed but safety is guaranteed by other means.

    The software for the space shuttle is life critical because a human pilot cannot perform the flight operations needed in the time necessary. If that software fails the crew probably dies.
    Ok, fair enough. The flight plan software is only a small part of the overall system. The part where you actually book it probably isn't even really part of the system. It's the software that gets the information from the radars to the air traffic controllers display, talks to the computers on the planes, etc that is life critical. About a third of SLOCS in an ATC system is dedicated to ensuring in the event of a failure it comes back up quickly (30 seconds to my last knowledge, but that's a couple years old). As for agile applying to such programs, it's really a matter of defining agile. If you spend months or more working out the math and physics behind a flight path prediction algorithm, is that big up-front design? Or is it agile because you prototype and test the algorithm in Matlab or another such environment as you go, prior to implementing it in Ada or C++? My philosophy is "do what makes sense."
  62. Hi Erik, Firstly, I would not hold up the space shuttle as an example of quality design (fitness for purpose?). Secoundly, trying to define life critical in some narrow sense is just plain stupid. If I'm on a plane it's criticl to me that the fight control systems work :^)
    As for agile applying to such programs, it's really a matter of defining agile.
    Well there is a pretty good definition in the Agile Manifesto. It repeats what I've said several times: values and principles.
    As for agile applying to such programs, it's really a matter of defining agile. If you spend months or more working out the math and physics behind a flight path prediction algorithm, is that big up-front design?
    Well I think the space shuttle is a good example. Was the space shuttle the simplest thing that could possibly work? Envisioning the system to be occurs prior to product development. I would categorise doing the maths and physics as part of working out the system vision (what is possible). As for development, an agile approach would be to learn through doing. So comming back to the space shuttle, what in the past lead to the final space shuttle design? It seems to me that not a lot of learning went on, and a big up front design decision was made that a "return vehicle" should form part of the system vision. AFAIK no alternative vision was explored once they had locked themselves in to this crucial decision.
    My philosophy is "do what makes sense."
    This is it. Agile thinking is just common sense. If you are not sure what to do when solving a complex problem, then what you do is try something or even several things in parallel and reflect and see. From small experiments you learn and improve. Non agile thinking assumes that we can know everything up front (deterministically), so we make crucial decisons early on and provide ourselves little opportunity for correction and learning. The real issue IMO is that most of us "experts" are unwilling to admit that we are not sure, and really do not know. Why do we try to apply a deterministic process to complex and undeterministic problems? We then try and hide behind mathematical statistics that tell us that our decisions are safe! In this regard IMO the Soviet Soyuz program was a lot more Agile than the Space Shuttle program. The Russians explored reuseable aircraft amongst other options and rightly rejected the idea, choosing to build on their experience with conventional rockets. As far as life safety is concerned, I would rather travel into space on a Soyuz rocket than itch a ride on a Space shuttle. It's just plain common sense. My 2 cents. Paul.
  63. "Well I think the space shuttle is a good example. Was the space shuttle the simplest thing that could possibly work? " We were talking about the shuttle's flight control software, not the entire shuttle. Software is different from other fields of engineering. There are several reasons for this, but one of the main reasons is that aeronautical engineering is constrained by the laws of nature, whereas software is not. Software is knowledge put into executable form. That knowledge can be derived from the real world, or from someone's imagination or some combination. Software probably has more in common with writing a novel than it does with building a bridge.
  64. Hi, My point is to do with new product development. The mistake we have made in the past is to equate software development with a manufaturing processes. When you manufacture a product, the exact steps required are known, they are deterministic so you can predict the outcome. When you create something new, the exact steps to get to an ideal product are unknown, you cannot predict. Agilist such as Jim Highsmith and the Poppendieks have clearly described the analogy between new product development and software development. This is a more appropriate analogy than manufacturing. (I have also seen similarities between software development and fashion design, but that is a whole different subject). The point is that the space shuttle system would have required less software if the system design was simpler. Simplicity is an Agile principle which applies just as much to space craft as it does to software (or motor vehicle design) or any other creative and innovative discipline. I'm trying to move the discussion beyond practices (XP) onto values and principles. I would suggest that you read the Agile Manifesto: http://agilemanifesto.org And you will see my point. Paul.
  65. Simplicity[ Go to top ]

    The point is that the space shuttle system would have required less software if the system design was simpler. Simplicity is an Agile principle which applies just as much to space craft as it does to software (or motor vehicle design) or any other creative and innovative discipline.
    The problem with simplicity is that it is entirely subjective. Most people measure the simplicity of a concept or a thing first relative to their own understanding of it, and second by how complicated it is relative to similar things they understand. If a person doesn't understand something, it is complicated - and therefore not simple. Consequently, simplicity becomes more a measure of the observer than the observed, and as a result the persuit of simplicity becomes the mantra for those who never want to learn anything new. Now, if people took the time to understand something before declaring it "simple" or "complicated," things would be better. Not perfect, because some people just can't understand some concepts, especially really elegant ones. For example, I can't bring myself to understand Lisp.
  66. Re: Simplicity[ Go to top ]

    Hi Erik, All of the Agile values and principles are subjective. They are based on heuristics, experience, feel and judgement rather than objective deterministic measurement. This is one of the reasons why I believe that you need to work on an Agile project with a good Agile coach in order to get a feel of what Agile is all about. At a basic level a single rocket with the human carrying capsule above the fuel tank where it can be easily ejected in an emergency is both a simpler and safer design than two solid rocket boosters and placing the astranuats with their bums inches away from a third massive fuel tank. I guess simplicity is relative. So you can reason that X is simpler than Y, but of course there is no absolute value you can place on simplicity. For example you cant say that X is 2.567 simple. This brings me to another Agile value, people. Skilled people will make good judgements and will be able to say that X is simpler than Y without the need for an objective measure. Deterministic processes try to rule people out of the equation. If this was truly possible then computers would have started writing software long ago. I was once told that there are three levels of learning: knowledge, understanding and skill. Skilled Agile practitioners have a good feel for what is simple. A good discussion on Agile values and simplicity can found on the c2 wiki: http://c2.com/cgi/wiki?DoTheSimplestThingThatCouldPossiblyWork The point I'm making is that Agile does not equal XP or FDD or DSDM or Crystal Clear. Agile is what you want it to be as long as your practices embody Agile values and principles and you have taken the time and engaged in the practice needed to climb the learning curve and become skillful. Paul.
  67. Software is different from other fields of engineering. There are several reasons for this, but one of the main reasons is that aeronautical engineering is constrained by the laws of nature, whereas software is not.
    I think that myth has hampered advancement of software engineering as a discipline every since Brooks called softwre "pure thought stuff." Software is bound by it's execution environment. Unlike the machines computer scientists like to dream about, real computers have finite speed and finite memory. Software optimized for machines with a single CPU is very different from software optimized for machines with 32 CPUs. So software is constrained by the gifts bestowed on it by the electrical engineers (who deal in physical reality), and they're constrained by the heat dissipation provided to them by the mechanical engineers.
  68. Firstly, I would not hold up the space shuttle as an example of quality design (fitness for purpose?).
    Depends on what you think the purpose of the space shuttle is. If you think it is "safely launch astronauts into space and return them to Earth" then I agree with your many critisisms of the space shuttle. I just don't think that that's the purpose. I believe it is the vision of the US that space travel be commercialized much like airline travel is commercialized today. In order for that to happen, we need reusable space vehicles that can safely and comfortably carry a large number of passengers and/or cargo. Trying to go from disposable rocket capsules to a space plane in one leap would be foolish. The space shuttle was an attempt to implement some of the traits required for a commercial spaceliner (it's reusable). It's a small step in a long evolution. So it's true that we probably could have built a suite of platforms that performs the same tasks as the space shuttle more reliably for less money based on disposable transports and more specialized equipment. But then we'd know nothing about building reusable plane-like spacecraft. It's agile development, just on a much longer time scale. The space shuttle fleet is just an incremental delivery of working equipment so we can learn what works and what doesn't. Disclaimer: I'm not privy to any more information about the purpose of the space shuttle than any other person and didn't do any research for this post, so if anything is wrong, please let me know.
  69. Hi
    Still looking for an example of life critical software done using agile.
    This statement alone shows that you haven't got the first idea of what agile is all about. Agile IS NOT XP, DSDM, FDD etc. Agility is a set of values and principles that can lead to more effective software development. So you need safety critical software, Fine. Where has it been proven that low bandwith, out of date and partial communication (by this I mean written documentation) leads to increased safety? Agile focuses on quality, fitness for purpose. In safety critical systems part of the purpose is a low MTBF and fault tolerance. These things are not inconsistent with Agile values and principles (open honest communication, simplicity, elimination of waste, learning through doing etc), but they may require different or additional practices then those found in a method like XP (there are examples of companies that have passed ISO9000 audits using an XP based process. And I have worked on a Drug Safety System, audited by the FDA amongst others using an XP based process). I've worked on three Agile projects now and numerous so called "Waterfall" ones. Interestingly the degree of agility depended not so much on whether the practices were clearly identifiable as "Agile", but more on the values and the principles lay behind the adopted practices and where embodied in the development team (people). If you spent more time understanding the underlying principles that lay behind some of the frequently touted Agile practices, then you would have a clearer idea of how to adapt/replace those same practices for any given situation. IMO their is too much focus on a specific set of practices like those of XP, which are then ill applied by people who do not understand Agile principles. A primary Agile principle is to be open to learning through feedback: Inspect and Adapt. So I turn your question on its head. Why can't you inspect and adapt, or eliminate waste, or build the simplest thing that can possibly work when developing Safety Critical Systems? BTW. I'm currently reading Lean Software Development by Mary Poppendiek. I recommend it. Paul.
  70. Someone already mentioned the article in FastTimes about the Space Shuttle avionics software team. They have always practiced iterative development, and their testing strategies fall very much within the realm of what an Agile team would want.


    Claims like this hurt agile's credibility.

    If you actually read that article their process is the antithesis of agile. It is very heavyweight, very Big Up Front Design:

    http://www.fastcompany.com/online/06/writestuff.html
    Wow! As I was reading that article, I kept asking myself: "Is this 'The Onion' that I'm reading right now?" I mean, the idyllic world portrayed in there hinges on being ridiculously bigoted. I've got a feeling that the author was barely restraining himself from claiming that the process is so phenomenally successful because all the participants are middle class god-fearing Christians, they're all white folks of heterosexual orientation who support the troupes overseas.
  71. What Needs to be Said[ Go to top ]

    All I've read so far is squabling; like it was about my mechanics vs. your mechanics... blah, blah. TDD is a practice that shapes your code. Don't treat it like some lifeless list of things to do. It shapes it in such a way that it is much more durable and higher quality. It makes you focus on the objective 'cuase you are writing the test first. It coaxes you to do good things like using interfaces so that you can mock collaborators. Interfaces reduces coupling and allow dependency injection and deployment flexibility. Code should be refactorable. You can't refactor without fear unless you have a good suite of unit-tests. So the 'cable-guy' arguments I hear about blasting something out 'cause "I gotta get-r-done" and "you guys are living in some acedemic vacuum and I gotta pay the bills" does not fly. In the long run you pay. You pay 'cause code is dynamic and maintenence is what really makes it expen$$$ive not "getting it done."