Discussions

News: Why it is Important that Software Projects Fail

  1. Automation in agriculture has dramatically decreased the proportion of GDP invested in it. Yet bureaucracies actually grow despite the huge investment in computer automation. This paper examines why. http://berglas.org/Articles/ImportantThatSoftwareFails/ImportantThatSoftwareFails.html Abstract This paper challenges the long established misconception that the catastrophic failure of expensive software projects is detrimental to society. It first provides detailed theoretical and empirical evidence for Berglas's corollary to Parkinson's law, namely that software automation does not improve real productivity. It is then shown that not only is it acceptable for software projects fail, but that it is essential that they fail if society is to function effectively. In this way the heavy burden of guilt can be lifted from the shoulders of the numerous project managers that have subconsciously devoted their careers to ensuring that projects rarely, if ever, succeed.

    Threaded Messages (25)

  2. Automation in agriculture has dramatically decreased the proportion of GDP invested in it
    I think the author may be unfamiliar with EU and US farm subsidies.. :D
  3. When engineers set out to build a bridge a bridge gets built. Our water ways are not cluttered with abandoned structures that cannot be made to work; bridge building is almost always successful Engineers don't build bridges they design them. The idea that software development is just like bridge building is a falsehood. Software development is just like bridge design. There are many bridge designs that litter the waste paper baskets of engineer's offices or lie crumbled at the bottom of wind tunnels, just as there are many software development efforts that never make it production and languish in no longer accessed source control repositories. My view on this comes from the Code As Design article. As for the rest of the article, I agree with the sentiment that many software projects should fail even if it appears to contradict itself with regards to productivity in the post computer automated world (early the claim is no productivity gain because the ATO still consumes the same percentage of GDP, later it admits the ATO processes vastly more work while costing the economy that same percentage of GDP). Many projects are kicked off because of a product manager's "brilliant" idea for a new product or a legislator's idea of a "fairer deal" for society. They may well be good ideas, but be enormously complex (and thus expensive)to model in software. Of course such projects should ideally stop long before a line of code is written. If they stop early is this still considered failure?
  4. bridge building is almost always successful ... But when a large team of programers sets out to develop software it is very uncertain whether anything of use will be delivered at all
    Well, humans started to build bridges probably several thousands of years B.C. and needed all those several thousands of years of tentatives before the romans started building solid and reliable bridges. We all started building software just a few tenths of years ago.
  5. bridge building is almost always successful ...
    Aside from pointing out that a large bridge just collapsed recently in the US, killing dozens of people, the reality is that most software is not critical. No one dies when it doesn't work. If we put software under the kind of scrutiny that we put civil engineering projects, you'd see different results. And as others have pointed out, bridges have clear goals and metrics of success and thousands of years of engineering behind them. Of course that's a bit oversimplified. There are serious culture issues in the software industry that IMO prevent it from achieving increased quality. The most important of these (again IMO) is the idea that it would be easy if you just had the right tools and processes in place.
  6. bridge building is almost always successful ... But when a large team of programers sets out to develop software it is very uncertain whether anything of use will be delivered at all

    Well, humans started to build bridges probably several thousands of years B.C. and needed all those several thousands of years of tentatives before the romans started building solid and reliable bridges.
    We all started building software just a few tenths of years ago.
    And I would guess that a large percentage of bridge building projects exceed both estimated time and cost; and they may not be used as much as originally expected.
  7. See a lot of people talking about bridge building being successful compared to software. I'm not so sure. The Channel Tunnel, not a bridge per se, but still, civil engineering, was budgeted to cost £4.9 billion, ended up costing £12 billion. Current London Olympic development, overrunning budgets by several times, and it's still 4 years to go! The new Wembley stadium? Cost several times the projections. ..and so on and so forth. The only reason people make bridge building/civil engineering metaphores for software development is because they know squat s**t about civil engineering for the most part. And besides, the metaphore is really bad in the first place because it is not particularly applicable.
  8. The subtitle of the post suggests this isn't a joke. If that's really true, I think you need to step away from the bong.
  9. bridge building is almost always successful ... But when a large team of programers sets out to develop software it is very uncertain whether anything of use will be delivered at all
    Bridge building process has clearly defined functional and non-functional requirements. General business application development almost by the rule works with no clear requirements, which change in the middle of development process etc. (Users usually have no idea what they want) Also, middleware software projects by definition work with (almost fully) clear requirements (formal specification in that domain), hence middleware companies in general have more stable products. So, imho, requirements specification plays major role(if we assume same quality of developer teams working in bridge team, business app development and middleware development)
  10. bridge building is almost always successful ... But when a large team of programers sets out to develop software it is very uncertain whether anything of use will be delivered at all


    Bridge building process has clearly defined functional and non-functional requirements.

    General business application development almost by the rule works with no clear requirements, which change in the middle of development process etc. (Users usually have no idea what they want)

    Also, middleware software projects by definition work with (almost fully) clear requirements (formal specification in that domain), hence middleware companies in general have more stable products.

    So, imho, requirements specification plays major role(if we assume same quality of developer teams working in bridge team, business app development and middleware development)
    I agree. I think the article has some definite flaws. One thing that grabbed me was the suggestion that we struggle to afford the basics (food and shelter) while our grandfathers had it all, with a stay-at-home wife to boot. Well, though it is sadly true that there are many impoverished people out there, it is well-beyond dispute that we enjoy a much higher standard of living today then we did back then... For most of us today, it's how to afford the basics + big screen T.V. + 4,000 sq. ft. home. Anyway, at the same time, it does make you think... There's probably some truth to the fact that government will continue to bloat and absorb any productivity gains acquired through new technology. Government wants only to get bigger and bigger...
  11. Anyway, at the same time, it does make you think... There's probably some truth to the fact that government will continue to bloat and absorb any productivity gains acquired through new technology. Government wants only to get bigger and bigger...
    Please, let's keep TSS from turning into a political forum. During the presidential campaign season, it's the only internet forum I can turn to, without having to read bipartisan arguments going back and forth.
  12. Also, middleware software projects by definition work with (almost fully) clear requirements (formal specification in that domain), hence middleware companies in general have more stable products.

    So, imho, requirements specification plays major role(if we assume same quality of developer teams working in bridge team, business app development and middleware development)
    Interesting analogy. As another analogy consider a tasks like "building a city", something very much like implementing a full scale E2E ERP solution. Even if you succeed with the task itself it does not mean that the thing you built is actually good or even usable. Consider the French Banlieues or Brasilia. Or consider the so called "Just There" Bridges you come across once in a while. Full scale bridges without a road or track from or to the actual bridge because the motorway/road project has been abandoned just after the bridges were built....
  13. Because...[ Go to top ]

    ... when software projects fail, dumb middle managers get fired. There. I saved you all from reading the academic caca. Roy Russo http://www.loopfuse.com
  14. Interesting ....[ Go to top ]

    Not to nit pick but... "burocracy" is spelled wrong.
  15. Re: Because...[ Go to top ]

    ... when software projects fail, dumb middle managers get fired.

    There. I saved you all from reading the academic caca.

    Roy Russo
    http://www.loopfuse.com
    Really? Where? In the work of IT that I've often read about, middle managers have a way of getting transferred out of projects that are about to implode, and thus get to live another day. Either that, or the sponsor of the project redefines "success", and everyone gets a juicy bonus for working on the "successful" project. Mark
  16. Re: Because...[ Go to top ]

    He said "dumb middle managers get fired". The middle managers who see the train wreck coming and escape are not the dumb ones.
  17. Good feedback. The Bridge Building reference seems to be a huge distraction to many, maybe I should remove it. (It was meant to just be an informal way of introduction, not a vigourous analysis.) The key point which I think is worth discussing, is that bureaucrasies such as Tax Office and Banks used to get essentially the same job done with essentially the same budget without any computerization at all. Isn't that surprising! As to grandfather's wealth, it is true that real economic quality of life has not gone up that much since the 1950s (unless you really value electronic gadgets). Certainly not in line with compounding productivity statistics. One reason may be bureaucrasy, but another not mentioned is the huge concentration of wealth over the decades. Maybe software has facilitated the latter as well as the former? I like the reference to EU and US farm subsidies, and will add that. Anthony
  18. my 2c[ Go to top ]

    In my opinion software automation does improve productivity somewhat, but we cannot do without since we simply do not have the manpower to process all the information in limited amount of time we have to do it in. This is why we tend to silently suffer the buggy software that surrounds us in our daily routine. Because we know that if we would ask for flawless software, we will not get it in time and without software... well you will not get home in time for dinner :) Marc
  19. The key point which I think is worth discussing, is that bureaucrasies such as Tax Office and Banks used to get essentially the same job done with essentially the same budget without any computerization at all. Isn't that surprising.
    No they don't. In fact they get much much more work done with computarization than without, even though we sometimes do not see it that way. As an example, in Germany, when earning more than a certain amount (you don't have to be a millionaire at all) the tax office will query the banks of any task payer for all his or her assets. This would be simply impossible without computarization (whether it is better is of course a different story altogether). Stuff like credit default swaps would not have been constructed without heavy numerical simulation tools. A very big part of today's banks business. And don't get me started on the turnaround times to change production lines in a modern automobile factory.
  20. The key point which I think is worth discussing, is that bureaucrasies such as Tax Office and Banks used to get essentially the same job done with essentially the same budget without any computerization at all. Isn't that surprising.


    No they don't. In fact they get much much more work done with computarization than without, even though we sometimes do not see it that way. As an example, in Germany, when earning more than a certain amount (you don't have to be a millionaire at all) the tax office will query the banks of any task payer for all his or her assets.

    This would be simply impossible without computarization (whether it is better is of course a different story altogether). Stuff like credit default swaps would not have been constructed without heavy numerical simulation tools. A very big part of today's banks business.

    And don't get me started on the turnaround times to change production lines in a modern automobile factory.
    Aside from that, if it's true that 'the tax office' budget is the same (I assume adjusted for inflation) and given that we know that the population of the world is increasing over time, 'the tax office' is clearly more productive. What's surprising about all of this is that someone calling himself a doctor is making specious and false claims and expecting people to take his arguments seriously.
  21. Tax office productivity[ Go to top ]

    What's surprising about all of this is that someone calling himself a doctor is making specious and false claims and expecting people to take his arguments seriously. Um, did you bother to read the article? Tax office size is roughly the same as a proportion of GDP. Ie it has grown substantially in real terms, faster than population. As to tax fraud, I doubt that there is any evidence that there is less now than in the 1950s. Certainly the tax office does much more than they did in the 1950s. That is because they oversea much more complex tax rules. Are those rules actually better for society? Again, the paper addresses that. And it is not just about tax/government. For example, private banks are similar. Of course the article has a satirical tone. But there is also a strong grain of truth in it. My guess is that about 20% of software applications add real value to society. Anthony
  22. Re: Tax office productivity[ Go to top ]

    Um, did you bother to read the article? Tax office size is roughly the same as a proportion of GDP. Ie it has grown substantially in real terms, faster than population.

    As to tax fraud, I doubt that there is any evidence that there is less now than in the 1950s.
    If it is roughly the same size than 1955 is has probably not grown at all in real terms assuming that the growth of the GDP and inflation balance over time. Which is quite consistent with an organization trying not to cut its overall budget. Because its productivity is higher the only way to do that is to perform more and complex tasks. That said, I can hardly think of any large scale building project (anything larger than a small residential buildung or a six storey office building) that hasn't overrun budget by more that 100%, regardsless whether it is a public or a private project. The question would now be: Why is this good for society.
  23. Re: Tax office productivity[ Go to top ]

    What's surprising about all of this is that someone calling himself a doctor is making specious and false claims and expecting people to take his arguments seriously.


    Um, did you bother to read the article? Tax office size is roughly the same as a proportion of GDP. Ie it has grown substantially in real terms, faster than population.
    I misunderstood the point. However, it's still a specious argument. You don't show that the tax office does the same thing now as it did in 1955, you assume it. And the real comparison would be work accomplished per worker. That's how worker productivity is normally measured.
    Certainly the tax office does much more than they did in the 1950s. That is because they oversea much more complex tax rules. Are those rules actually better for society?
    Maybe, maybe not. It's a hard thing to judge objectively.
    My guess is that about 20% of software applications add real value to society.
    If true, that would make the software industry a great contributor of value to society, IMO.
  24. Stuff like credit default swaps would not have been constructed without heavy numerical simulation tools. A very big part of today's banks business.

    Yeah, and look where that got them :-) Jim
  25. Berglas's Corollary[ Go to top ]

    From the article: "No one need ever again be embarrassed by participation in a failed software project. Rather they should be proud to have spared society from yet another burden of complexity." Following this logic, one should try to sabotage all software projects one is involved in, for a better world. Not that I haven´t seen individuals doing just that. I´m just not so sure they were motivated by such...noble reasons.
  26. We see the failure of TCard capping further complication of ticket prices: http://en.wikipedia.org/wiki/Tcard On the other hand we don't have conductors any more. I can't see the problem with dropping a coin into a box and getting a printed piece of paper. Or in paying for public transport via the tax office. The way mutual funds charge fees to their clients is another case study. Complex mobile phone plans are another. So 80% of software (Tax Office included) is a sophisticated way of charging $9.99 instead of $10.