37signals' book "Getting Real" now available for free

Discussions

News: 37signals' book "Getting Real" now available for free

  1. If you're willing to read it in a hypertext, slightly disjointed fashion, 37signals' book "Getting Real" is available for free on the web. The book focuses on development style and process, and unapologetically advocates Ruby on Rails, but even if Ruby isn't your cup of tea it's an interesting and valuable read. The book addresses the project's lifecycle, from conception to execution to production to support. The development processes advocate doing enough to accomplish a purpose, rather than anticipating usage (thus, fitting the "YAGNI" mindset, where "YAGNI" stands for "You aren't going to need it.") The book's tone is addressed early, in the third essay, called "Caveats, disclaimers, and other preemptive strikes":
    "You take too much of a black and white view." If our tone seems too know-it-allish, bear with us. We think it's better to present ideas in bold strokes than to be wishy-washy about it. If that comes off as cocky or arrogant, so be it. We'd rather be provocative than water everything down with "it depends..." Of course there will be times when these rules need to be stretched or broken. And some of these tactics may not apply to your situation. Use your judgement and imagination.
    The book is comprised of essays that can generally be taken independently, although the most benefit will be seen by reading it entirely. It's definitely a change from the typical "do-everything" approach seen in so many Java frameworks, and has a lot to do with the urge to move to lighter, easier, more agile processes, even in Java. It's also an effective piece of evangelism, and shows writers how they can communicate and justify their points of view on a given subject. Worth reading, even if the technology isn't appropriate for your needs (also a point they address in the "caveats" chapter.)

    Threaded Messages (89)

  2. Some points are interesting, some are complete bunch of BS, IMO. First, the notion of keeping feature set to a minimum (at least that's my understanding) is crap. Tell that to the business owner who requests these features and see the response. Second... Interface First design methodology, shows that they are focused on small web apps. Any enterprise application can't start off with an interface first design, as interfaces are pretty abstract at that point. You're creating services that need to service various interfaces, from web services, to web interfaces, to rich clients, etc... Yes, they have nice clean apps there at 37signals, but they haven't gotten to the complex features sets yet, as I truly doubt they have enterprise customers using their applications and that require integration end points, etc... Let's see them design Salesforce.com with that methodology. Ilya
  3. First, the notion of keeping feature set to a minimum (at least that's my understanding) is crap. Tell that to the business owner who requests these features and see the response.
    Remember the business that they are in, and think about it economically. Every feature adds cost, both up-front and maintenance. So if a feature does not enable them to increase their monthly revenue by more than the anticipated meantenance costs, it shouldn't be added. Also one of the big selling points is simplicity. The easiest way to achieve simplicity is to avoid adding features.
    Yes, they have nice clean apps there at 37signals, but they haven't gotten to the complex features sets yet, as I truly doubt they have enterprise customers using their applications and that require integration end points, etc...
    I doubt the ever will get to complex feature sets. They don't want large corporate customers. It would destroy their business model. Look at their payment methods:
    Currently we accept Visa, Mastercard, and American Express. At this time we only accept payments online so we will not be able to accept a P.O., invoice you, or take an order over the phone.
    So the ONLY way for me to purchase their services for work is to put it on my AMEX and pray my expense report goes through.
    Interface First design methodology, shows that they are focused on small web apps. Any enterprise application can't start off with an interface first design, as interfaces are pretty abstract at that point.
    Rephrase this as "Identify the most critical portion of your system, then design and implement it first" and it makes a lot more sense. For their apps, it's the UI. For others it may be intergrations, database schema, architecture to achieve five 9s of availability, scalability, some key algorithms, etc. It depends on the application, but I would say that if you can't narrow it down to one or two aspects, then you have a problem.
  4. Remember the business that they are in, and think about it economically. Every feature adds cost, both up-front and maintenance. So if a feature does not enable them to increase their monthly revenue by more than the anticipated meantenance costs, it shouldn't be added.

    Also one of the big selling points is simplicity. The easiest way to achieve simplicity is to avoid adding features.

    Agree to an extent. Each project has specific requirements and it's best to keep things simple and functional. On the other hand, one can argue that let's not have any features at all for the sake of simplicity. It depends on what you define as features. I'm sure no one just sits there and thinks of random features to implements, most are a product of the requirements specs.
    I doubt the ever will get to complex feature sets. They don't want large corporate customers. It would destroy their business model. Look at their payment methods
    Ok, that's all fine, but my argument was that supposedly the book is about managing software projects. It would be fine if titled "Manage trivial software specs/projects", but my biggest problem with 37sigs is that they try to push their methodology on the rest of the world, as if nothing else exists other than trivial web apps they are out to develop.


    Rephrase this as "Identify the most critical portion of your system, then design and implement it first" and it makes a lot more sense. For their apps, it's the UI. For others it may be intergrations, database schema, architecture to achieve five 9s of availability, scalability, some key algorithms, etc. It depends on the application, but I would say that if you can't narrow it down to one or two aspects, then you have a problem.
    Agree, but I'm always against an interface first design, unless what is meant by the interface is the definition of use cases. If we start talking about GUI design, you'll run into enormous problems later, as your GUI design will then shape your use cases and service interfaces, which is a horrible thing. The concerns of presentation should never reflect upon your core business domain, again, in all cases of non-trivial apps, which as we can see they are not developing. Either way, I have a lot of respect for these folks there, they are headed in the right direction I think. Ruby is a great language, but not yet for enterprise apps. My biggest thing with RoR, is the lack of a stable deployment platform. Most of the ones out there are either no longer actively developed or is considered unstable. Ilya
  5. Ok, that's all fine, but my argument was that supposedly the book is about managing software projects. It would be fine if titled "Manage trivial software specs/projects", but my biggest problem with 37sigs is that they try to push their methodology on the rest of the world, as if nothing else exists other than trivial web apps they are out to develop.
    Read their "Disclaimer" section. When I first read the "Design the interface first" section I got all roiled up. Almost enough to buy the PDF so I could critisize them intelligently. Then I realized that I would be spending money on "wasting" time, which is never a good thing. But then the posted everything for free, I read their disclaimer. It made sense. I think there's a lot that's good about what they're saying. I certainly wouldn't apply all of it, especially literaly. But if you look at the practices in the frame of their products and business model, back it up to the underlying motivations, then reason it forward to your own products and business model - most of it makes sense. It just is no longer what they are most vehemently writing.
  6. Ok, that's all fine, but my argument was that supposedly the book is about managing software projects. It would be fine if titled "Manage trivial software specs/projects", but my biggest problem with 37sigs is that they try to push their methodology on the rest of the world, as if nothing else exists other than trivial web apps they are out to develop.


    Read their "Disclaimer" section. When I first read the "Design the interface first" section I got all roiled up. Almost enough to buy the PDF so I could critisize them intelligently. Then I realized that I would be spending money on "wasting" time, which is never a good thing.

    But then the posted everything for free, I read their disclaimer. It made sense.

    I think there's a lot that's good about what they're saying. I certainly wouldn't apply all of it, especially literaly. But if you look at the practices in the frame of their products and business model, back it up to the underlying motivations, then reason it forward to your own products and business model - most of it makes sense. It just is no longer what they are most vehemently writing.
    Hi Erik, I understand what you are saying here, and I believe that you are being more open minded then most. The thing with "light weight", "agile" ideas like these is that you really need to practice them yourself to get it. Even then you need to be coached by someone who as done it before, learning by example sort of thing. Reading it from a book and trying to put it into practice by yourself just won't do. There are two acronyms amongst the Agle community that I believe are relevant here: YAGNI (You A'nt Gonna Need It) and IKIA (I Know It Already). All big systems can be evolved from small systems, and when you build systems this way you see that much of the infrastructure you felt you needed you didn't: YAGNI. But of course we know this isn't going to work because we've done it before and we Know It Already: IKIA. My advice is to listen and suspend judgement. I was lucky to work on a project where I was coached by a board member of the Agile Alliance. I didn't believe either, until I saw it working before my very eyes. Paul.
  7. Suspend Judgment[ Go to top ]

    My advice is to listen and suspend judgement. I was lucky to work on a project where I was coached by a board member of the Agile Alliance. I didn't believe either, until I saw it working before my very eyes
    I thought I was suspending judgment.
  8. Re: Suspend Judgment[ Go to top ]

    My advice is to listen and suspend judgement. I was lucky to work on a project where I was coached by a board member of the Agile Alliance. I didn't believe either, until I saw it working before my very eyes


    I thought I was suspending judgment.
    h Yes, I think you are! I'm saying that you may find more of what they have to say directly relevant to your situation if you gave it ago and if you had a coach. Knowadays I try to start off with the most lean approach thinkable, and even then I often surprise myself when I realise that things could be even simpler. P.
  9. Sorry Paul, but I couldn't resist..
    There are two acronyms amongst the Agle community that I believe are relevant here: YAGNI (You A'nt Gonna Need It) and IKIA (I Know It Already).
    Perhaps there should be an 'community' for experienced and cynical old programmers like me. Acronyms that are likely to be popular in this community would include ITYYNTBYWLWY (I Told You You Needed That, But You Wouldn't Listen Would You), IAGTBRW (It's All Got To Be Re-Written) and FBUL (Fashionable But Underpowered Language), just to name a few. They sadly cover so many real-life examples of projects I have encountered (and often had to fix) that followed YAGNI and ignored IKIA. (Mind you, I have to admit that this might have something to do with the fact that the only Agile principles they used was to follow YAGNI and ignore IKIA).
  10. Perhaps there should be an 'community' for experienced and cynical old programmers like me. Acronyms that are likely to be popular in this community would include ITYYNTBYWLWY (I Told You You Needed That, But You Wouldn't Listen Would You), IAGTBRW (It's All Got To Be Re-Written) and FBUL (Fashionable But Underpowered Language), just to name a few. They sadly cover so many real-life examples of projects I have encountered (and often had to fix) that followed YAGNI and ignored IKIA. (Mind you, I have to admit that this might have something to do with the fact that the only Agile principles they used was to follow YAGNI and ignore IKIA).
    +1
  11. Perhaps there should be an 'community' for experienced and cynical old programmers like me. Acronyms that are likely to be popular in this community would include ITYYNTBYWLWY (I Told You You Needed That, But You Wouldn't Listen Would You), IAGTBRW (It's All Got To Be Re-Written) and FBUL (Fashionable But Underpowered Language), just to name a few. They sadly cover so many real-life examples of projects I have encountered (and often had to fix) that followed YAGNI and ignored IKIA. (Mind you, I have to admit that this might have something to do with the fact that the only Agile principles they used was to follow YAGNI and ignore IKIA).


    +1
    Hi Guys, I can see the funny side too. Expensive, over specified, buggy and late - I guess that passes as a proven approach. We've all done it and got paid handsomely for years. The thing is now though is that our customers can get cheap, over specified, buggy and late in India, so were does that leave us? Paul.
  12. Hi Guys,

    I can see the funny side too. Expensive, over specified, buggy and late - I guess that passes as a proven approach. We've all done it and got paid handsomely for years
    If that is what you read from my post, you really didn't get it :)
  13. Hi Guys,

    I can see the funny side too. Expensive, over specified, buggy and late - I guess that passes as a proven approach. We've all done it and got paid handsomely for years


    If that is what you read from my post, you really didn't get it :)
    I guess I didn't. Paul.
  14. Experience[ Go to top ]

    I can see the funny side too. Expensive, over specified, buggy and late - I guess that passes as a proven approach. We've all done it and got paid handsomely for years.
    An experienced developer has typically worked on many more applications than a user has experience with, and seen the impacts of decisions made during development on the operational effectiveness of the applications. An experienced developer is has also interacted with far more users, and knows that the collection of users before him is not representative of the average user. But the developer will be judged more on how the average user reacts to the application than this collection of people, consequently he tried to act as the advocate for the average user. A good developer is also detail oriented, and doesn't forget that the user said "I'm going to need this" or identified some "exception" that occurs 40% of the time that the user now insists is completely unimportant because "let's just get the basic flow right now." Finally, users rarely link in terms of systems or even complete applications. They think vary narrowly about one task at a time. It's the developers job to see how all of these tasks are intertwined and ensure they don't end up tangled. I could go on and on. There are times when I wish I as paid for "I told you so"'s or even "didn't you tell me this 6 months ago?"
  15. Re: Experience[ Go to top ]

    I can see the funny side too. Expensive, over specified, buggy and late - I guess that passes as a proven approach. We've all done it and got paid handsomely for years.


    An experienced developer has typically worked on many more applications than a user has experience with, and seen the impacts of decisions made during development on the operational effectiveness of the applications. An experienced developer is has also interacted with far more users, and knows that the collection of users before him is not representative of the average user. But the developer will be judged more on how the average user reacts to the application than this collection of people, consequently he tried to act as the advocate for the average user.

    A good developer is also detail oriented, and doesn't forget that the user said "I'm going to need this" or identified some "exception" that occurs 40% of the time that the user now insists is completely unimportant because "let's just get the basic flow right now."

    Finally, users rarely link in terms of systems or even complete applications. They think vary narrowly about one task at a time. It's the developers job to see how all of these tasks are intertwined and ensure they don't end up tangled.

    I could go on and on. There are times when I wish I as paid for "I told you so"'s or even "didn't you tell me this 6 months ago?"
    Hi Erik, I haven't read all the book, but I know this isn't what they mean. When asked for requirements customers tend to throw in everything they can think of including the kitchen sink just in case, because they know once they've signed off the spec then they can't change their minds and add anything new. If they do, the fact that the projects goes drastically over budget and is late will be their fault, right? The other problem along side over specification is wrongly specified systems. Users and business analysts are just meant to know what is required. The fact that they may only have a inkling for the kind of thing they want, and aren't sure about the details just won't do. Software development is a science and we need a detailed and specific specification before we can start building software, right? Both of these are disastorous and expensive misconceptions strongly held by many. Like I said lot of us fall into IKIA. These misconceptions need to be challenged as there are more creative and productive ways. For me seeing was believing, which is why I advocate using a coach. Paul.
  16. Re: Experience[ Go to top ]

    I haven't read all the book, but I know this isn't what they mean.
    I know. It was a counterpoint. Balance is what is required.
    When asked for requirements customers tend to throw in everything they can think of including the kitchen sink just in case, because they know once they've signed off the spec then they can't change their minds and add anything new.
    Can you send some of those customers who know they can't change the requirements after they've signed off on them my way? ;-)
    If they do, the fact that the projects goes drastically over budget and is late will be their fault, right?
    Actually, I'm of the opinion that it is the responsibility of the development team to ensure requirements a captured with sufficient precision and accuracy, and that the requirements are understood and stable.
    Software development is a science and we need a detailed and specific specification before we can start building software, right?
    Software development is science's less disciplined yet strangely pragmatic cousin - engineering. That being said, a lot of science is about experimentation - disciplined experimentation. There's nothing wrong with building software as an experiment intended to increase your understanding of the problem. But experiments must be disciplined. In other words, you should have a clear idea of what you are going to learn from the experiment from the start. "Let's see if the users like this..." isn't a disciplined experiment.
    These misconceptions need to be challenged as there are more creative and productive ways.
    I agree. I think the single most important part of software development is understanding the problem - meaning the entire team (development, user, sponsor, and champion) achieving the same understanding of the problem. You rarely can achieve that with paper alone. You also rarely can achieve that by simply iterating a solution until people find it acceptable.
  17. Re: Experience[ Go to top ]

    I haven't read all the book, but I know this isn't what they mean.


    I know. It was a counterpoint. Balance is what is required.

    When asked for requirements customers tend to throw in everything they can think of including the kitchen sink just in case, because they know once they've signed off the spec then they can't change their minds and add anything new.


    Can you send some of those customers who know they can't change the requirements after they've signed off on them my way? ;-)

    If they do, the fact that the projects goes drastically over budget and is late will be their fault, right?


    Actually, I'm of the opinion that it is the responsibility of the development team to ensure requirements a captured with sufficient precision and accuracy, and that the requirements are understood and stable.

    Software development is a science and we need a detailed and specific specification before we can start building software, right?


    Software development is science's less disciplined yet strangely pragmatic cousin - engineering. That being said, a lot of science is about experimentation - disciplined experimentation. There's nothing wrong with building software as an experiment intended to increase your understanding of the problem. But experiments must be disciplined. In other words, you should have a clear idea of what you are going to learn from the experiment from the start. "Let's see if the users like this..." isn't a disciplined experiment.

    These misconceptions need to be challenged as there are more creative and productive ways.


    I agree. I think the single most important part of software development is understanding the problem - meaning the entire team (development, user, sponsor, and champion) achieving the same understanding of the problem.

    You rarely can achieve that with paper alone. You also rarely can achieve that by simply iterating a solution until people find it acceptable.
    Hi Erik, Why not just try it? But then again you know it already. As well as measuring concrete progress every two weeks (working code), I also asses my process and decide what's working and what isn't. At the end of this we may decide to try something new, and see how it goes two weeks down the line. For complex, non-deterministic and unpredictable problems, this empirical, closed loop, feedback approach works and within a few iterations you begin to home in on what works for this given team (customers, developers, sponsors etc) and what doesn't. Of course your process can oscillate and never reach a steady state. This usually implies two much open loop gain (your changes are two big) or not enough damping (I think, it as been a while since I've done any electronics). But this can be addressed. Empirical closed loop process control is a well proven science. Get a good book on thermodynamics, or electronic systems or better still find out how the central heating control in your home works. Act-inspect-adapt works, but it does mean letting go of your pre-conceptions. Paul.
  18. Act-inspect-adapt[ Go to top ]

    Act-inspect-adapt works, but it does mean letting go of your pre-conceptions.
    When did I say that it doesn't? I said that it should be done in a disciplined fashion. I said it's important to understand what the problem is that you are trying to solve with software. I said that it's important that the team has a common understanding - and I'll add critical that they don't have conflicting conceptions, as the business stakeholders often do. This probably means more analysis and documentation than "agile" people desire, but in my opinion it's value added. When you make a change, the team should know why they think that change will get them closer to the solution, and have an idea of how they will validate that it achieved the desired effect.
    For complex, non-deterministic and unpredictable problems
    I would argue that applying a piece of deterministic equipment to a problem that is not only non-deterministic, but also unpredictable by means of statistics or other mathematical methods, is a waste of money. But I suspect I'm interpretting your statement too literally...
    this empirical, closed loop, feedback approach works and within a few iterations you begin to home in on what works for this given team (customers, developers, sponsors etc) and what doesn't.
    Yes, you may home in on what works for the team, but does it actually solve the problem?
    Of course your process can oscillate and never reach a steady state. This usually implies two much open loop gain (your changes are two big) or not enough damping (I think, it as been a while since I've done any electronics). But this can be addressed.
    Yes, there's this problem, and there's also the problem of reaching local maxima. Oscillation that lasts too long tends to be killed by budget contraints - which motivates most people to put an end to it. Local maxima are a whole different beast, because it's easy to label them as success - and indeed often times they are. But frequently they also are optimal solutions to only a portion of the problem, and attaining a more holistic solution will require throwing the current solution away.
    Empirical closed loop process control is a well proven science. Get a good book on thermodynamics, or electronic systems or better still find out how the central heating control in your home works.
    So when we develop software, are we executing a well-defined control system to reach an acceptable state; or are we applying something resembling the scientific method to advance our understanding?
    Act-inspect-adapt works, but it does mean letting go of your pre-conceptions.
    Which ones?
  19. Re: Act-inspect-adapt[ Go to top ]

    Act-inspect-adapt works, but it does mean letting go of your pre-conceptions.


    Which ones?
    Hi Erik, Your whole post is full of pre-conceptions. I wouldn't know were to start. Like I said before, why not just try it with the help of a coach and see if it actually works? Paul.
  20. Re: Experience[ Go to top ]

    When asked for requirements customers tend to throw in everything they can think of including the kitchen sink just in case, because they know once they've signed off the spec then they can't change their minds and add anything new. If they do, the fact that the projects goes drastically over budget and is late will be their fault, right?
    Actually, I have considerable experience of a very different scenario - customers under-specify projects and take shortcuts because they think that is the only way to reach a deadline or budget. To do this they proclaim YAGNI, and insist the developers use FBULs (see above). Then the deadline is missed and the budget over-runs because they discover new requirements for the supposedly 'finished' software, and that the FBUL won't do the job, and the short-cuts they took mean more work to change things. Of course, saying 'I told you so' at this stage doesn't help, so I try not to (but usually fail).
  21. Re: Experience[ Go to top ]

    When asked for requirements customers tend to throw in everything they can think of including the kitchen sink just in case, because they know once they've signed off the spec then they can't change their minds and add anything new. If they do, the fact that the projects goes drastically over budget and is late will be their fault, right?


    Actually, I have considerable experience of a very different scenario - customers under-specify projects and take shortcuts because they think that is the only way to reach a deadline or budget. To do this they proclaim YAGNI, and insist the developers use FBULs (see above). Then the deadline is missed and the budget over-runs because they discover new requirements for the supposedly 'finished' software, and that the FBUL won't do the job, and the short-cuts they took mean more work to change things. Of course, saying 'I told you so' at this stage doesn't help, so I try not to (but usually fail).
    Hi Steve, FBUL? I don't know what you are refering to, or the relevance to this discussion. If you mean languages like Ruby, well it's been around since 1993 and can trace its ancestory back to Lisp which goes back to 1956. So nothing, fashionable or unproven there. I don't think there is any "one" way to develop software, so I find these academic debates futile. The point I'm making is very simple. To know what works you need to try it. To know whether you are doing it right you need some one who has done it before to mentor you and show you by example (a coach). Only then are you in a position to judge. You can't make a true judgement on a process you've just read about. What ever situations you have found yourself in in the past, there was more than one solution. The way you chose to solve the problem, was just one way, that doesn't make it the most optimal way. For complex, non-deterministic problems with many inter-related variables the only way to find an optimal solution is to continually act, inspect and adapt. Even then you are in danger of finding a local maxima, the global maxima may require a Kuhnian shift in approach: http://en.wikipedia.org/wiki/Thomas_Samuel_Kuhn This issue is well known, and explains why scientific progress occurs in stuts and starts: http://en.wikipedia.org/wiki/Scientific_method My only advice is to keep an open mind. Paul.
  22. Re: Experience[ Go to top ]

    FBUL? I don't know what you are refering to, or the relevance to this discussion. If you mean languages like Ruby, well it's been around since 1993 and can trace its ancestory back to Lisp which goes back to 1956. So nothing, fashionable or unproven there.
    Ruby not fashionable? I must be out-of-date. No, seems I am not.
    I don't think there is any "one" way to develop software, so I find these academic debates futile.
    You see, they aren't academic (at least not by my definition). You can tell they aren't academic because of the repeated use of the term 'experience'. If only they were academic, but they sadly aren't! The 'I told you so' and 'it all needs to be rewritten' situations are often expensive. Still, helps pay the bills.
    The point I'm making is very simple. To know what works you need to try it.
    Only in software engineering would one find such a comment. Meanwhile, structural engineers don't try to build bridges from cardboard, or if they do they only do it as students, so they learn what does and doesn't work.
    For complex, non-deterministic problems with many inter-related variables the only way to find an optimal solution is to continually act, inspect and adapt. Even then you are in danger of finding a local maxima, the global maxima may require a Kuhnian shift in approach:

    http://en.wikipedia.org/wiki/Thomas_Samuel_Kuhn

    This issue is well known, and explains why scientific progress occurs in stuts and starts:

    http://en.wikipedia.org/wiki/Scientific_method

    My only advice is to keep an open mind.

    Paul.
    Although I have a Ph.D. in Biology, and considerable postgraduate experience in numeric simulation work, I may be a bit rusty about the scientific method, so I have never come across an attempt to apply numerical minimization techniques to project management of software development. Perhaps I should read up a bit more on Psychohistory? :)
  23. Re: Experience[ Go to top ]

    Hi Steve, Debate is only of any use if we choose to listen to each other. I have said very little, other than give it a go before coming to a judgement. Yet your responses seem very defensive.
    The point I'm making is very simple. To know what works you need to try it.
    Only in software engineering would one find such a comment.
    Really. And you say you've got a PhD in Biology? I'm proposing nothing other than the scientific method. Books and accepted scientific theories can be wrong. Einstien spent his life proving this. I guess you chose to ingnore my reference to Thomas Khun. You win, you have all the answers you need. I conceed. Paul.
  24. Re: Experience[ Go to top ]

    Hi Steve,

    Debate is only of any use if we choose to listen to each other.

    I have said very little, other than give it a go before coming to a judgement. Yet your responses seem very defensive.
    No, I think you simply interpret disagreement as being defensive and closed-minded.
    The point I'm making is very simple. To know what works you need to try it.
    Only in software engineering would one find such a comment.


    Really. And you say you've got a PhD in Biology? I'm proposing nothing other than the scientific method.
    You are proposing the precise opposite of the scientific method. "Try whatever works" is not scientific. Science - including Einstein - is based upon searching for similarities and patterns based on existing ideas and similar situations; about 'standing on the shoulders of giants', as Newton put it. Science is almost always iterative, not 'start from scratch each time'. Einstein based his much of his work on that of Minowski and Maxwell.
    Books and accepted scientific theories can be wrong. Einstien spent his life proving this.
    He spent a lot of his life trying and failing to prove many scientific theories wrong - check the long debates with Bohr.
    I guess you chose to ingnore my reference to Thomas Khun.
    You ignored my reference to Azimov! Fair is fair :)
    You win, you have all the answers you need. I conceed.
    If only that were true, but being a good scientist I realise "the great ocean of truth [..] all undiscovered before me." (Newton)
  25. Re: Experience[ Go to top ]

    Only in software engineering would one find such a comment. Meanwhile, structural engineers don't try to build bridges from cardboard, or if they do they only do it as students, so they learn what does and doesn't work.
    Actually you'll find it in electrical, mechanical, structural, and materials, too. Maybe not if you're building a bridge, but these words are often spoken when engineers are asked to do something novel. The two differences are they do a lot of analysis first, and they are doing something novel. Ask them to do yet another variation on something they've done 1000 times, and they won't need to try it first to see if it works. They certainly don't run out a try something to see if it works when they don't even know what "works" means. Well, I'm sure there are exceptions, but you get the point. There's a lot that makes sense in what Paul and other agile advocates say. The problem is the idea that acting (coding) and perceiving (seeing the users' reaction) are substitutes for thinking (analysis and design), and that verbal communication and a "working" application are substitutes for quality documentation (meaning up-to-date documentation that people actually read and use a reference material).
  26. Re: Experience[ Go to top ]

    The two differences are they do a lot of analysis first, and they are doing something novel. Ask them to do yet another variation on something they've done 1000 times, and they won't need to try it first to see if it works.

    They certainly don't run out a try something to see if it works when they don't even know what "works" means. Well, I'm sure there are exceptions, but you get the point.
    I fully agree.
    There's a lot that makes sense in what Paul and other agile advocates say. The problem is the idea that acting (coding) and perceiving (seeing the users' reaction) are substitutes for thinking (analysis and design), and that verbal communication and a "working" application are substitutes for quality documentation (meaning up-to-date documentation that people actually read and use a reference material).
    I also agree that a lot of the agile approach makes sense. There is not one of the Principles behind the Agile Manifesto that I disagree with - It is the way some of us have been working for a very long time, but then I have had the good fortune to work in the 'home ground' of Agile work: a small number of senior, experienced developers, where such principles can arise naturally.
  27. Re: Experience[ Go to top ]

    I also agree that a lot of the agile approach makes sense. There is not one of the Principles behind the Agile Manifesto that I disagree with - It is the way some of us have been working for a very long time, but then I have had the good fortune to work in the 'home ground' of Agile work: a small number of senior, experienced developers, where such principles can arise naturally.
    Agility did rise from experience. It was not a thought of a novice coder. Though, I think agility appeals to novices (in relative terms), because it gets rid of a lot of grudge work - often rightly so. However, agility is not designed novices in mind. The apparent freedom of creation requires an experienced person, or chaos follows. You still need to plan ahead, know what you are doing, ability to design, and just plainly possessing knowledge (schooling, concrete experience). The main attribute of agility is managing change which is difficult, not automatically solved by any methodology.
  28. Re: Experience[ Go to top ]

    I also agree that a lot of the agile approach makes sense. There is not one of the Principles behind the Agile Manifesto that I disagree with - It is the way some of us have been working for a very long time, but then I have had the good fortune to work in the 'home ground' of Agile work: a small number of senior, experienced developers, where such principles can arise naturally.


    Agility did rise from experience. It was not a thought of a novice coder. Though, I think agility appeals to novices (in relative terms), because it gets rid of a lot of grudge work - often rightly so.

    However, agility is not designed novices in mind. The apparent freedom of creation requires an experienced person, or chaos follows. You still need to plan ahead, know what you are doing, ability to design, and just plainly possessing knowledge (schooling, concrete experience). The main attribute of agility is managing change which is difficult, not automatically solved by any methodology.
    Hi Tero, I'm glad somene out there is well informed. Although a lot of the practices of the 37Signals guys may look new to some, they are actually tried and tested. They were discovered through years of experience in OO developement by people not in the mainstream. The roots of the Agile movement today stems from people like Jeff Sutherland, who used Smalltalk for many years at Easel Soft before inventing Scrum, and people like Kent Beck, Ron Jefferies and Ward Cunningham who too used Smalltalk for many years on many projects, before finally deciding to "turn the volume up" on the C2 project which lead to XP. These people realised that through the use of OO and various technical and mangement practices that they could flatten the cost of change curve. Before this the assumption was that the cost of change would grow exponentially throughout the life time of a project. They showed that infact this could be avoided. Avoiding the cost of change growing exponentially enables a more evolutionary approach to development, where small systems (2 weeks worth of analysis, design and development) can be gradually evolved into large fully featured system (months and years worth of work), whilst continuingly delivering production quality releases along the way. Change is viable at all stages. To do this requires a great degree of skill, but it can be taught. Not in the traditional classroom way, but in the same way a master craftsman teaches his apprentices. By learning through example at the hands of an experience practitioner you can learn what Agile is all about and get it. People who have not had this experience, just do not get it, which is why I advocate people trying it for themselves with the help of an experienced coach. BTW. You can do agile with junior people, but you need to maintain a ratio between junior and senior staff. Alistair Cockburn (author of Surviving OO projects, one of the first books to research OO development practices), suggests a ration of 1:3 between senior team lead capable people and others. Also because of the rapid decision making and the high bandwidth in communication needed for continuous change, agile tends to work best with small teams, but if your intention is to build a small system that you will grow through change into a big one, then a small team is not a hinderence, it's a bonus. Delivering business value early on buys you time. So you don't need an army of developers to meet the deadline. Paul.
  29. Re: Experience[ Go to top ]

    I'm glad somene out there is well informed.
    Paul, your post is a good history, but I suspect you are not presenting anything that many of us don't already know. Personally, I am not sure that Agile principles weren't at already widespread in some areas of development. My impression is that there have always been a significant number of small development teams or individual developers, where these principles tend to arise simply through common sense and freedom. They made no assumptions about the costs of change. There have always been developers who talk freely with each other, and who learn from more senior people in real projects. There have always been projects that are loosely specified at the start and grow, first through prototypes and then with regular releases based on feedback. The key development is to get these approaches into larger organisations, where appropriate. My view is that the Agile manifesto is being taken to extremes, and being misinterpreted. It is a mild statement. It suggests one set of priorities. For example, there seems to be a fashion against use of 'tools'. It seems to me that some developers are taking the manifesto statement 'we value [..] Individuals and interactions over processes and tools' as a rejection of IDEs and frameworks! It is nothing of the sort. If anything it would be about project management tools - those used to produce huge Gantt charts and to analyse things to the n'th degree even before coding has started. Other statements are also being taken to extremes. 'we value responding to change over following a plan' does not mean 'have no plan!' It does not mean start off with no ideas about the project and assume nothing and try anything. One has to use common sense. Agile development is great - I have made many choices as a developer to ensure that I could continue using Agile methods (even though they weren't yet named as such). But taking things to extremes, claiming it is a new form of enlightenment - a dramatic Kuhnian paradigm shift - is not just mistaken, it is going to put many developers and teams off, as practices need to be evolved, not abandoned. I shall stop before I contribute to a stack overflow Thanks for the discussion.
  30. Re: Experience[ Go to top ]

    I'm glad somene out there is well informed.


    Paul, your post is a good history, but I suspect you are not presenting anything that many of us don't already know.

    Personally, I am not sure that Agile principles weren't at already widespread in some areas of development. My impression is that there have always been a significant number of small development teams or individual developers, where these principles tend to arise simply through common sense and freedom. They made no assumptions about the costs of change. There have always been developers who talk freely with each other, and who learn from more senior people in real projects. There have always been projects that are loosely specified at the start and grow, first through prototypes and then with regular releases based on feedback. The key development is to get these approaches into larger organisations, where appropriate.

    My view is that the Agile manifesto is being taken to extremes, and being misinterpreted. It is a mild statement. It suggests one set of priorities. For example, there seems to be a fashion against use of 'tools'. It seems to me that some developers are taking the manifesto statement 'we value [..] Individuals and interactions over processes and tools' as a rejection of IDEs and frameworks! It is nothing of the sort. If anything it would be about project management tools - those used to produce huge Gantt charts and to analyse things to the n'th degree even before coding has started. Other statements are also being taken to extremes. 'we value responding to change over following a plan' does not mean 'have no plan!' It does not mean start off with no ideas about the project and assume nothing and try anything. One has to use common sense.

    Agile development is great - I have made many choices as a developer to ensure that I could continue using Agile methods (even though they weren't yet named as such).

    But taking things to extremes, claiming it is a new form of enlightenment - a dramatic Kuhnian paradigm shift - is not just mistaken, it is going to put many developers and teams off, as practices need to be evolved, not abandoned.

    I shall stop before I contribute to a stack overflow

    Thanks for the discussion.
    Hi Steve, Nothing in my posts or in the "Getting Real" book contradicts anything you say here. Infact I find some of what you say contradicts what you have said previously. For example FBUL has no place in this discussion. I believe the problem stems from people wanting to see things in black and white. The "Getting Real" book does not do this, and they offer a disclaimer to clarify their postion. What they are doing IMO is offering a strong lead. And with our industry threatened by ofshore competition, I believe that this lead is timely. Beyond a certain level of competence software development is no longer a science but it becomes an art. Someone once recommended I read the Zen of Motorcycle Maintenance, I haven't, but I guess it makes the same point. Artists are use to grey and learning through example, in my experience many programmers struggle with abstract concepts when things aren't black and white. This is why I said it is important to learn at the knee of an experienced practitioner, through example, so that you can develop your own "feel" for what is possible. Much of what the 37Signal guys say may "feel" counter-intuitive to many but with the right mentoring at the hands of a coach, your intuition can change, and you may find that your whole perspective changes. It is this change in how software development should "feel" that can be a Kuhnian shift for many people. It was for me. Paul.
  31. FBUL[ Go to top ]

    FBUL? I don't know what you are refering to, or the relevance to this discussion. If you mean languages like Ruby, well it's been around since 1993 and can trace its ancestory back to Lisp which goes back to 1956. So nothing, fashionable or unproven there.
    Common Lisp is an ANSI standard, with multiple commercial and open source implementations on a variety of platforms offering performance competitive with C and highly suited for numerical computations. Some implementations even provide significant static code analysis (checking) and most offer both native code compilation and a bytecode based VM. In other words, Common Lisp is everything Steve looks for in a language, except that almost no one knows how to program in it. Although I'm beginning to think that could be a good thing, because I suspect that very few bad programmers ever take the time to learn it. Ruby is simply inspired by Lisp...and unfortunately Perl.
  32. Re: FBUL[ Go to top ]

    In other words, Common Lisp is everything Steve looks for in a language, except that almost no one knows how to program in it.
    Exactly.
  33. We at Cynergy are huge supporters of a Front To Back development approach. We use it across the board in our projects and it has proven itself time and time again on teeny projects to massive ones. Now let me take one quick second to make a point since I can predict the knee jerk reaction because we are a RIA company who has been doing and growing a lot around Flex. *YES* we come from a Java/J2EE enterprise development background. I was the principal architect for one of the first J2EE branded app servers ever. I was also a three time speaker at JavaOne, SDI, and did one of this years Keynotes at AJAXWorld. Most every app we build has a Java/J2EE backend that uses Tomcat, Hibernate, Spring, etc etc etc. Ok, now that I got that outta the way, lets get down to brass tacks.
    Some points are interesting, some are complete bunch of BS, IMO. First, the notion of keeping feature set to a minimum (at least that's my understanding) is crap. Tell that to the business owner who requests these features and see the response.
    Ok, let's look at the alternative then. We do a traditional back to front API centric approach to development and the same business owner writes checks for months and months and months and can "see" ZERO progress. So that business owner starts wondering where the heck all his money is going. Now as a very good very smart architect you espouse about how elegant your designs are and how this framework will make his application so maintainable in the future. Now the alternative? Take that huge feature set, slash it down. Implement it very quickly, get it up online, let them see it, touch it, and tell you what features they need next, what features the next customer has to have first. They see true progress, and you end up developing the features real users need and not the ones the developers guessed they wanted.

    Second...

    Interface First design methodology, shows that they are focused on small web apps. Any enterprise application can't start off with an interface first design, as interfaces are pretty abstract at that point. You're creating services that need to service various interfaces, from web services, to web interfaces, to rich clients, etc... Yes, they have nice clean apps there at 37signals, but they haven't gotten to the complex features sets yet, as I truly doubt they have enterprise customers using their applications and that require integration end points, etc...

    Let's see them design Salesforce.com with that methodology.

    Ilya
    I have to again tell you from real-world experience I think you are simply dead wrong there. We develop software for some of the biggest companies in the world. People like one of the top 3 largest retail banks in the nation. Or one of the top 10 largest software companies in the world and we do it using Front To Back (Interface First) development. Let's take just one application we are building. It is a line of business application to run a medical business. It is essentially not just a CRM system like Salesforce.com but so much more. It includes scheduling, resource allocation, inventory management, invoicing, accounting, third party integration to third party services and things like EDI. This is a major ERP system to run major medical facilities. I actually contend that those massive applications are actually the PERFECT for this approach (which we call LookFirst). Here are a few reasons why.  This application has 200+ tables and at least that many screens alone. The odds that in some requirements gathering session you can even pretend to come up with all those fields by doing requirements analysis and writing a spec.  Secondly now when you do make a mistake and miss a field it is usually found how? By a user looking at a screen and noticing its missing.  Now when you fix that, you have to fix it in 3 or more tiers. Add it to the DB, the persistence layer, the controller, and the UI. When you do the UI first, you fix it in one place. And now for the biggie. If you develop the API before you develop the UI you are stuck with an API that is pretty much guaranteed to have zero fidelity with the UI. What I mean is an API that requires UI’s to do things like make multiple network calls, or to take data in one format and structure and tear that structure apart to display it the way the users need it displayed. That isn’t what UI’s should do and they lead to the slow and primitive user experiences we are delivering today in traditional back to front development. Look, I know I am throwing a lot of opinions out there, but I will back them up. Email me. Call me. Lets hop online and do demos of the apps. Major apps written by 3 to 4 guys in months. Apps like Over-C (video on our site) which is a major enterprise app that integrates with third party services, account systems, RFID mobile devices, SMS servers, IVR systems and on and on. These are not toys and these are developed front to back. My keynote from AjaxWorld is actually posted up on the AjaxWorld magazine websites. I go into a ton more detail about why this approach works. -- Dave Wolf Cynergy Systems http://www.cynergysystems.com/blogs Email: dave dot wolf at cynergysystems dot com Office: 866-CYNERGY
  34. I used to think it always made better sense to start building an application, especially a complex one, from the back-end (database, API design, UML Diagrams, frameworks, etc). However, recently one of our clients asked us to turn their Client/Server-based stockbroking application into a Web-based app. We were looking at our first full AJAX application. Where did we start our design from? The FRONT of course! The customer needed to see the screens and be sure we could give them an application that resembled and behaved like the one they had been using. So if you ask me, the choice of whether to start from the GUI or the back-end actually depends on what you're building and it simply won't matter if you're knowledgeable. What customers are mostly afraid of is a pretty interface with little or no functionality to back it up! P.Kofon
  35. We at Cynergy are huge supporters of a Front To Back development approach.
    Dave - good to hear from you again. I spent a lot of time in the trenches of that "first J2EE branded app server" starting in '99, and you were a big help back in those early days! I agree completely with the importance of interface first development. On a few projects now, we have split stories such that building a visible but non-working version of the UI is a separate, high-priority story. Having the UI in front of the Customer, whether it actually works or not, is so much better for generating more questions and ideas than a low-fidelity mockup. Simple things such as a grouping of fields can expose differences in how the developers and Customer see the business domain, and prevent a pile of work that was unnecessary or just simply wrong. These "Hollywood set" UI's are also very easy to change at the Customer's request, so they can play around with the layout to see what works. This also works with the interaction designers, testers, etc. I don't always implement all of the UI first, though. The Customer does stipulate the priority of what's built and when, but I do highly encourage them to put the basic UI stories as high up the priority list as possible. BTW, we (IL) use BaseCamp. It's a great application that serves us very well! Dave Rooney I n d u s t r i a l L o g i c I n c. http://industriallogic.com http://industrialxp.org
  36. Ha ha ha ...[ Go to top ]

    Dave Wolf Cynergy Systems http://www.cynergysystems.com/blogs Email: dave dot wolf at cynergysystems dot com Office: 866-CYNERGY Get a Life and a CUSTOMER. Give the customer what they want not what you think is cool. You had better go back to school to learn computing ethics. Do not make a complete fool of yourself trying to sell a what was it? Serverside wont give you business for TALKING bull, show em something, which you obviously CANNOT. Clearly, ONCE and DONE! is your philosophy!
  37. 74 Signals[ Go to top ]

    Eh, WTF! Move on..
  38. 74 Signals[ Go to top ]

    Eh, WTF! Move on..`
  39. Oh, so many impressive references, Kuhn, Newton, and Einstein, in the same thread. Brooks is missing, even though "No silver bullet" should be whispered by the small guy on your agile shoulder. But all of us are not building OS/370, and programming is not an art, but a craft, and a craftsman can grasp all but the biggest system in his mind. Which is why a good craftsman can build, and guide others in building, all but the largest systems, either back-to-front och front-to-back. What matters is how well the customers picture is communicated from him to the craftsman. Since the craftsman is a professional in seing others visions, but the customer is not a professional in showing his visions to others, this communication should use the model most suited to the customer. So, if the customer thinks front-to-back, use front-to-back, if the customer thinks back-to-front, use front-to-back. Unless you´re actually building OS/370...
  40. Hi John,
    But all of us are not building OS/370, and programming is not an art, but a craft, and a craftsman can grasp all but the biggest system in his mind.
    This is interesting. Art is not craft? Please explain. Also I was of the impression that our cognitvie abilities generally limited us to holding 7 +/- 2 things in our minds at once. How does a craftsman transcend this? Please expand. Paul.
  41. Hi John,

    But all of us are not building OS/370, and programming is not an art, but a craft, and a craftsman can grasp all but the biggest system in his mind.


    This is interesting. Art is not craft? Please explain. Also I was of the impression that our cognitvie abilities generally limited us to holding 7 +/- 2 things in our minds at once. How does a craftsman transcend this? Please expand.

    Paul.
    Hello Paul, Of course art *can* be craft, and craft *can* be art. But they are not synonyms, or are they? A Craftsman is of course human, and has the same limitations as other humans. A good craftsman is however adept at using "divide-and-conquer" to divide each of the seven things in his head into seven smaller things, which he can then ponder on, and so on. I didnt prohibite a craftsman from taking notes, did I? Anyhow, its nice that you take your time to raise objections to my thoughts, but I think that you need to take a more serious stab at it if you want to prove me wrong rather than mock me. Brrrr - J
  42. Hi John,

    But all of us are not building OS/370, and programming is not an art, but a craft, and a craftsman can grasp all but the biggest system in his mind.


    This is interesting. Art is not craft? Please explain. Also I was of the impression that our cognitvie abilities generally limited us to holding 7 +/- 2 things in our minds at once. How does a craftsman transcend this? Please expand.

    Paul.


    Hello Paul,

    Of course art *can* be craft, and craft *can* be art. But they are not synonyms, or are they?

    A Craftsman is of course human, and has the same limitations as other humans. A good craftsman is however adept at using "divide-and-conquer" to divide each of the seven things in his head into seven smaller things, which he can then ponder on, and so on.

    I didnt prohibite a craftsman from taking notes, did I?

    Anyhow, its nice that you take your time to raise objections to my thoughts, but I think that you need to take a more serious stab at it if you want to prove me wrong rather than mock me.

    Brrrr - J
    Hi John, You've been on TSS too long. No I wasn't raising objections, I was merely seeking clarifications. BTW I agree with you, I didn't want to assume that your understanding was the same as mine that's all. BTW. (in the same vien) Why refer to OS/370? And give us an example of a customer who knows "the system to be" from back-to-front (versus front-to-back). Again, I believe I know what you mean here, but you may be suggesting something new. Paul.
  43. Hi John,

    You've been on TSS too long. No I wasn't raising objections, I was merely seeking clarifications. BTW I agree with you, I didn't want to assume that your understanding was the same as mine that's all.

    BTW. (in the same vien) Why refer to OS/370? And give us an example of a customer who knows "the system to be" from back-to-front (versus front-to-back).

    Again, I believe I know what you mean here, but you may be suggesting something new.

    Paul.
    Paul, Well, the reference to OS/370 comes from Fred P. Brooks, since it is the construction of this system which is the main source of experience for "The Mythical Man Month". I just used it as an example of a system so complex that you cant really design it front-to-back and expect to succeed. I have worked with quite a number of customers who knows the system to be back-to-front, but this usually comes from bad experiences in the past, where they havent spent enough attention on that when specifying. But, then again, there are usually one group of people from the customers side who thinks front-to-back, and one who thinks back-to-front, and it is often a challenge to satisy both groups. Guess, you will recognize that as well. Br - J
  44. Hi John,

    You've been on TSS too long. No I wasn't raising objections, I was merely seeking clarifications. BTW I agree with you, I didn't want to assume that your understanding was the same as mine that's all.

    BTW. (in the same vien) Why refer to OS/370? And give us an example of a customer who knows "the system to be" from back-to-front (versus front-to-back).

    Again, I believe I know what you mean here, but you may be suggesting something new.

    Paul.


    Paul,

    Well, the reference to OS/370 comes from Fred P. Brooks, since it is the construction of this system which is the main source of experience for "The Mythical Man Month". I just used it as an example of a system so complex that you cant really design it front-to-back and expect to succeed.

    I have worked with quite a number of customers who knows the system to be back-to-front, but this usually comes from bad experiences in the past, where they havent spent enough attention on that when specifying.

    But, then again, there are usually one group of people from the customers side who thinks front-to-back, and one who thinks back-to-front, and it is often a challenge to satisy both groups.

    Guess, you will recognize that as well.

    Br - J
    Yep, I recognise both. This is why keeping things simple, keeping things small, and open communication and feedback with customers in any form that makes them comfortable is so important. I've now read through most of "Getting Real", and I just can't see what there is for people to object to. To me it is mostly (not so) common sense :^). Paul.
  45. Yep, I recognise both. This is why keeping things simple, keeping things small, and open communication and feedback with customers in any form that makes them comfortable is so important.
    Yep, I agree. But, I did forget one group: The Printouts Specification Team. Now, they dont think front-to-back, and they dont think back-to-front, and to them a pixel is not a pixel, but a mile. Perhaps this is where the craftsman must call in the artist? Br - J
  46. Yep, I recognise both. This is why keeping things simple, keeping things small, and open communication and feedback with customers in any form that makes them comfortable is so important.


    Yep, I agree.

    But, I did forget one group: The Printouts Specification Team. Now, they dont think front-to-back, and they dont think back-to-front, and to them a pixel is not a pixel, but a mile. Perhaps this is where the craftsman must call in the artist?

    Br - J
    Yep, "The Printouts Specification Team" are a whole industry within themselves. I've always expected oposition from them (Business Analysts, Spec writers, lead architects, the strategy team, etc), but what as alway surprised me is that some of the strongest opposition to cutting out the bull actually comes from developers. Many companies tend not to have a culture where you are allowed to make mistakes and not know things. So developers who where once "the experts" in the old world of specs and absolute certainty suddenly find themselves thumbling learners in the new touchy feely world of uncertainty, face-to-face communication and change. I'm speculating now, but I believe that the new "Agile" world calls for people who are comfortable using both sides of their brain (left sided and right sided thinkers) accessing their creative and collaborative skills aswell as their analytical abilities. Hopefully things will change and creative types will become more attracted to software, and who knows perhaps more women too. Then maybe the culture will change, and people will see creativity and flair as far more important then being able to follow a spec. Paul.
  47. Then maybe the culture will change, and people will see creativity and flair as far more important then being able to follow a spec.
    (emphasis mine) I'm disturbed by this statement on so many levels. I hope you just picked the wrong word...or that it has different connotations in the UK than in the US so I'm not really grasping what you're saying.
  48. Then maybe the culture will change, and people will see creativity and flair as far more important then being able to follow a spec.


    (emphasis mine)

    I'm disturbed by this statement on so many levels.

    I hope you just picked the wrong word...or that it has different connotations in the UK than in the US so I'm not really grasping what you're saying.
    Hi Erik, This is speculation, but I have come across evidence that backs it up. Think about other professions, that rely on highly skilled individuals: Fashion Designers, Play writes, Journalists etc. In these professions the finished product is uncertain, and a direct result of the creativity and flair of the professionals involved. Having said this the staff members need a great deal of technical skill to do the job. They need years of academic training, followed by years of mentoring at the knee of a master, building and honing their own skills. Finally they become recognised as master crafts people in their own right, and their work is valued and appreciated by many. In all of these professions they have deadlines, yet what is required by the market can be very difficult to know for sure. So they use time-boxing: fixing budget and time, and leaving scope as a variable. They often work iteratively, with several drafts or prototypes of their work created over time, and soliciting feedback where possible. Finally when the deadline approaches, they tidy up their best work so far, and go with that. The final product is judged by their customers, so in all these professions they need to know their customers well. They all rely on customer feedback to know whether they are delivering what is wanted by the market. Finally, all these professions support multi-million dollars industries. They may be seen as artists and creative people, but in the final analysis they are business people, and if they do not deliver value to their customers they will be replaced by organisations and individuals that do. I could speculate more, but it may be better holding this conversation offline. Paul.
  49. Then maybe the culture will change, and people will see creativity and flair as far more important then being able to follow a spec.
    I would say that being able to follow a specification is required, but not enough. True excellence needs also creativity and flair. But, this depends a bit. I have some experience in guiding teams working from remote locations, and this usually requires much more detailed specifications than when you are co-located. The more detailed the specification, the less room for creativity and flair on the implementers side. The creativity must instead flow from the specification writer. However, once specifications get detailed enough, one might as well formalize them and write a code generator. Brooks wrote about the master coder as "the surgeon". A mighty target, but surely worth aspring to.
  50. Then maybe the culture will change, and people will see creativity and flair as far more important then being able to follow a spec.


    I would say that being able to follow a specification is required, but not enough. True excellence needs also creativity and flair.

    But, this depends a bit. I have some experience in guiding teams working from remote locations, and this usually requires much more detailed specifications than when you are co-located.
    Yes. And this is why there is no golden rules. A spec. is a low bandwith form of communication. When bandwith is short then you may need to go with a spec, but when bandwith is plentiful, then of course face-to-face communication is best.
    The more detailed the specification, the less room for creativity and flair on the implementers side. The creativity must instead flow from the specification writer.

    However, once specifications get detailed enough, one might as well formalize them and write a code generator.
    Yes. The best, most detailed specs do use a code generator. It's called a compiler :^).


    Brooks wrote about the master coder as "the surgeon". A mighty target, but surely worth aspring to.
    I've never read the mythical man month, but I had a colleague who I respected greatly that use to quote brooks liberally too. So thanks for the debate and I'm off to Amazon to get myself a copy of The Mythical Man Month. I'm also going to get myself a copy of "A Guide to Zen and the Art of Motorcycle Maintenance" by L Disanto. So much to learn and so little time :^) Later, Paul.
  51. Specs[ Go to top ]

    Yes. And this is why there is no golden rules. A spec. is a low bandwith form of communication. When bandwith is short then you may need to go with a spec, but when bandwith is plentiful, then of course face-to-face communication is best.
    I don't think face-to-face is inherently high-bandwidth and specs are inherently low bandwidth. It all depends on who is meeting and who is writing. Also, both have a tendency to leave contentious points unstated in order to facilitate agreement. Or the opposite, get so bogged down in fighting over details (assuming someone will sign off on the spec and cares). Personally, I think a well-written spec conveys far more information in a much more accurate and precise manner than face to face communication, but it is impossible to write a good spec without tons of face to face communication.
  52. Re: Specs[ Go to top ]

    Yes. And this is why there is no golden rules. A spec. is a low bandwith form of communication. When bandwith is short then you may need to go with a spec, but when bandwith is plentiful, then of course face-to-face communication is best.


    I don't think face-to-face is inherently high-bandwidth and specs are inherently low bandwidth. It all depends on who is meeting and who is writing.

    Also, both have a tendency to leave contentious points unstated in order to facilitate agreement. Or the opposite, get so bogged down in fighting over details (assuming someone will sign off on the spec and cares).

    Personally, I think a well-written spec conveys far more information in a much more accurate and precise manner than face to face communication, but it is impossible to write a good spec without tons of face to face communication.
    Hi Erik, My first response to this was yes I see your point. A spec from a spec writer to an implementor is at best a monologue, and will always contain partial information, ommiting some detail and perhaps the underlying motive. So as John Brand points out, a spec immediately limits the room for creativity by the implementor. So for example, if the spec suggests a specific form of user interface and the implementor is aware of an alternative widget that may be simpler/cheaper/better,then he is somewhat limited to what he can do without encuring significant cost. Alternatively, if the implementor recieves an outline of the feature required, and has direct access with the end user, then he can have a dialogue with that user and experiment with alternative ideas at very low cost (no change request, no new version of the spec etc). I've seen this work in practice, and in some instances, through the feedback available from playing with a working system, users may decide that a feature that they thought they needed isn't needed at all. We all have experienced the situation as an implemetor when we've read something in a spec. and felt, I'm sure they don't need that, or what they should do is something else. If we were to raise a red flag every time it happended, nothing would get done, so we implement blindly. It helps customers too, because they get feedback from playing with a working system. So they can implement their most important features first, and then decide what else is really needed, side-by-side, with the developer. The developer can provide feedback, informing the user of the cost of a "would-be-nice" feature. This allows the user to decide to opt for a simpler solution, or to opt for an alternative feature all togehter that represents better business value. You really need to see this working in practice to appreciate what I mean. It really works! Paul.
  53. Widgets[ Go to top ]

    So as John Brand points out, a spec immediately limits the room for creativity by the implementor. So for example, if the spec suggests a specific form of user interface and the implementor is aware of an alternative widget that may be simpler/cheaper/better,then he is somewhat limited to what he can do without encuring significant cost.
    Which is why specs shouldn't be so detailed as to suggest a specific form of user interface. If you're dealing with an intra-net application, even saying "web based" is probably going too far. Its focus should be: 1. Defining the business problem, and why it is important 2. Putting the problem in context 3. Defining how automation is expected to contribute to the solution of the problem 4. Define the major stakeholders 5. Define any constraints. For example, that the application should work interactively, as opposed to submitting data for batch processing, because otherwise it won't help cycle time...etc... If the application is doing so sort of mathematical or complex logical analysis, this should also be thoroughly documented. It should either go towards the end of my outline above or in a separate document. Talking about fields or screens or frameworks or programming languages is in most cases a waste of time. There's too much volatility, and if you want to know what fields are on a screen or in a table you're going to look at the screen or the table.
    Alternatively, if the implementor recieves an outline of the feature required, and has direct access with the end user, then he can have a dialogue with that user and experiment with alternative ideas at very low cost (no change request, no new version of the spec etc).
    I agree completely, provided both the user and the implementor have a clear idea of the problem they are trying to solve. Otherwise they will, more likely than not, run off into the weeds on some tangential concern that maybe feels important to the user providing the feedback but is irrelevant to solving the problem.
  54. Re: Widgets[ Go to top ]

    Hi Erik, Your perspective is that of a person versed in a spec. orientated approach. There are many systems in the world where independent autonomous entities act independently following simple rules all working towards a common goal. Think of a colony of ants, using simple rules the colony survives with no single ant orchestrating from the centre. Everyone in a Scrum/XP team knows what is wanted. They all talk about it constantly. When they don't know what is wanted then they break-out and conduct a spike to find out. Someone goes off and does some investigation. This can happen in parrallel with the main work stream of known and understood stories (features). Everything happens all at once, with each implementor performing the role of analysis/Design/Implement all at the same time. The customer team acts in unison too. They speak and meet each day and have a single vision. If there are disputes then the product owner (customer team lead) has the final say. There is plenty of material available on the details of how this all works. Here is a presentation by Jeff Sutherland that provides the theoretical background: http://www.infoq.com/presentations/The-Roots-of-Scrum Ron Jefferies, has a book that goes into the practical installation of XP. Beware though each team does their thing a little differently: http://www.amazon.com/Extreme-Programming-Installed-Ron-Jeffries/dp/0201708426/sr=1-1/qid=1162372297/ref=pd_bbs_sr_1/002-2840586-3288829?ie=UTF8&s=books Lastly, like I've said several times, you really need to see it working in practice and try it yourself. Agile is very practical, it is not about theory. It is about cutting out waste, trusting and empowering teams and getting things done! Paul.
  55. Wants, Problems, and Needs[ Go to top ]

    Think of a colony of ants, using simple rules the colony survives with no single ant orchestrating from the centre.
    Ants have a common goal plugged into their genetic code. Unfortunately genetically programed goals evolved long before computers existed.
    Everyone in a Scrum/XP team knows what is wanted.
    That you for bolding "wanted." That's exactly the problem. What is wanted by the user is often out of tune with what is needed by the business.
    Prior to budget being allocated for building a non-trivial application, usually some sort of ROI analysis is required. This is because there are always far more proposals for new applications and enhancements to existing ones than there are dollars. If properly done, that ROI analysis is predicated on some very specific gains in a business process, most commonly some sort of labor savings. The business usually wants to take what it currently does and make it more efficient. Often times this isn't even because they want to reduce staff. Certain groups can become a bottleneck, causing others to sit idle. The returns on a system that alleviates idle time in multiple groups is usually much higher than one that simply saves effort in a single group. The users usually have no objection to automating their work, but they strongly object to having their workload reduced. So what I've repeatedly seen, is every hour of labor or cycle time that is automated away is replaced by an hour of effort required for increased process complexity or sophistication. The bottleneck never disappears, a level of sophistication that wasn't required is added, and the ROI is never delivered. But you have a handful of very happy users. Not only have you relieved some of their day-to-day tedium, you've also effectively enabled them to up-skill their jobs.
  56. Re: Wants, Problems, and Needs[ Go to top ]

    Hi Erik, I agree with you. The issue here though is organisational, not software development. I provided a reference to the Toyota Production System. Toyota does value stream mapping too. The difference at Toyota though is that the users (workers) own their value stream and take responsibility for improving it. When you say "the business", I think you really mean "The Managment". So Management are trying to impose change upon the workers by commissioning new software. This problem is bigger than software development. What I can say however is for a true ROI, change needs to come from the bottom. Management can install fancy new software systems, but if the users spend their time circumventing the new process because they don't like it, or because they feel it is affecting their job security, the planned ROI just won't be realised. So how does Toyota manage it? Well, change at Toyota must be Kiazen - change for the good. A change that causes people to loose their jobs isn't chnage for the good as far as the Japanese are concerned, and Japanese management would not try to impose such a change on their workers. The Toyota workers know this, which is why they are happy improving their own value streams. So what to do? Well, from what you have told me in the past, you tend to play the role of customer. So there is no reason why you can't carry on doing that with Scrum/XP. You can be the product owner, representing the management interests. Implementors can discuss stories with you. There is a big difference, between agile software development, and an Agile business. But at least making the software development Agile is a step in the right direction :^) Paul.
  57. Re: Wants, Problems, and Needs[ Go to top ]

    I agree with you. The issue here though is organisational, not software development.
    Organizational issues that impact the software development process must be addressed. Detailed specs are an attempt to make the process rigorous enough to withstand organizational issues, mostly by clearly defining who is to blame for failure.
    When you say "the business", I think you really mean "The Managment". So Management are trying to impose change upon the workers by commissioning new software.
    You would think so, but in my experience front-line management sides with their people. This is what I would expect. I would hate to have a manager who didn't side with me. At some point in the chain you get to people who are supposed to see the "big picture." Most of these people don't, and just focus on the day-to-day. Another big chunk got where they are by being opportunistic. These people are dangerous and destructive, although seldom particularly competent. The remaining minority is the people to whom I'm referring. Although it's often hard to identify them...
    This problem is bigger than software development. What I can say however is for a true ROI, change needs to come from the bottom.
    I would say change needs to be embraced by the bottom, but it doesn't need to orginate with the bottom. The Toyota system doesn't work well in the US because employers have no "social contract," either through custom or through law, with their employees. I don't really blame employers for this. It's mutual. When the labor market is tight, workers do everything they can to squeaze every last penny out of employers. When business goes slack, they get shoved out the door. It's mutual distrust instigated by both parties. Even my egotistical meddlesome self finds that issue to big to budge.
    So what to do? Well, from what you have told me in the past, you tend to play the role of customer. So there is no reason why you can't carry on doing that with Scrum/XP. You can be the product owner, representing the management interests. Implementors can discuss stories with you.
    Yes, I often play that role.
  58. Re: Wants, Problems, and Needs[ Go to top ]

    Hi Erik,
    The bottleneck never disappears, a level of sophistication that wasn't required is added, and the ROI is never delivered. But you have a handful of very happy users. Not only have you relieved some of their day-to-day tedium, you've also effectively enabled them to up-skill their jobs.
    No one ever sold more by doing less. I assume that the increased process sophistication, and the improved motivation gained through the removal of day-to-day tedium has resulted in increased productivity and/or quality. OK, no one has lost their job, but it sounds like performance has increased to me. Perhaps the workers know more about running a succesful "business" then the Managers do? LOL!!! Paul.
  59. Bottlenecks and Sophistication[ Go to top ]

    Hi Erik,

    The bottleneck never disappears, a level of sophistication that wasn't required is added, and the ROI is never delivered.

    But you have a handful of very happy users. Not only have you relieved some of their day-to-day tedium, you've also effectively enabled them to up-skill their jobs.


    No one ever sold more by doing less. I assume that the increased process sophistication, and the improved motivation gained through the removal of day-to-day tedium has resulted in increased productivity and/or quality.

    OK, no one has lost their job, but it sounds like performance has increased to me.

    Perhaps the workers know more about running a succesful "business" then the Managers do?
    LOL!!!

    Paul.
    One of the main tenets of lean is to never produce more than is required. Someone who did research in lean manufacturing once told me about an American auto parts supplier who started making parts for Toyota. They made the rails that attach the seat to the frame of the car. So Toyota contracts them to design and manufacture a part. They come up without something approximately twice as strong as it was specified, and they were quite proud of themselves. As long as they hit their cost targets, American auto makers would be perfectly happy with this. Toyota made them redesign the part to spec. The added strength gave them nothing, and the part weighed more than it would have if it just met the spec. Local optimization often doesn't contribute to the bottom line. One of the main purposes of value stream mapping is to see how a process step fits into the large value chain. In the example I cited, what was needed was faster cycle time. Timeliness of data was much more important to the value chain than increased precision.
  60. Re: Bottlenecks and Sophistication[ Go to top ]

    Hi Erik,

    The bottleneck never disappears, a level of sophistication that wasn't required is added, and the ROI is never delivered.

    But you have a handful of very happy users. Not only have you relieved some of their day-to-day tedium, you've also effectively enabled them to up-skill their jobs.


    No one ever sold more by doing less. I assume that the increased process sophistication, and the improved motivation gained through the removal of day-to-day tedium has resulted in increased productivity and/or quality.

    OK, no one has lost their job, but it sounds like performance has increased to me.

    Perhaps the workers know more about running a succesful "business" then the Managers do?
    LOL!!!

    Paul.


    One of the main tenets of lean is to never produce more than is required.

    Someone who did research in lean manufacturing once told me about an American auto parts supplier who started making parts for Toyota. They made the rails that attach the seat to the frame of the car. So Toyota contracts them to design and manufacture a part. They come up without something approximately twice as strong as it was specified, and they were quite proud of themselves. As long as they hit their cost targets, American auto makers would be perfectly happy with this. Toyota made them redesign the part to spec. The added strength gave them nothing, and the part weighed more than it would have if it just met the spec.

    Local optimization often doesn't contribute to the bottom line. One of the main purposes of value stream mapping is to see how a process step fits into the large value chain.

    In the example I cited, what was needed was faster cycle time. Timeliness of data was much more important to the value chain than increased precision.
    Agreed. I was mostly joking. But it appears to me that the software has delivered some value. I place significant weight on making people happy. A happy worker is a productive worker. The up-skilling of staff has got to be a good too. We are moving off topic, but the functional managers should be accountable for how their functional unit performs. They are indeed your customer IMO, not Senior Managers far removed from day-to-day operations. It seems to me that senior management is micro-managing, rather than setting performance goals for each functional unit. If appropriate goals are set, then it is down to the functional units how to best meet them. It may be new software, or it may not, but it should be their process improvement activity, from their budgets. Imposing such things from the top down seldom works. Paul.
  61. Re: Wants, Problems, and Needs[ Go to top ]

    Everyone in a Scrum/XP team knows what is wanted.


    That you for bolding "wanted." That's exactly the problem. What is wanted by the user is often out of tune with what is needed by the business.
    Eric, don't confuse the User and the Customer roles of an XP or Agile project. Most often they are not the same people. The Customer is someone whose primary goal is to provide the development team with the initial input an ongoing feedback about how a system is going to provide business value. In many cases, the average end user can't do this. A couple of years ago, I worked on a project that had a Customer who was able to take a broad look at the business the system we were building was going to support. This person was very good at steering the project based on the business needs, and providing frequent timely feedback. The result was a solid product that was immediately (or at least very quickly) embraced by the user community. That person was replaced by someone who, at that point, had a very narrow view of one particular facet of the business, primarily as an end user. Suddenly, decisions were being made strictly on how the UI looked, and on how the one part of the business that this person knew worked. This person also flip-flopped on many decisions, which led to a significant amount of churn during development. The result of that particular fiasco was a protracted development cycle and quite a bit of time, effort and money spent building one part of the application that very few people in the user community cared about. It also didn't provide very much overall business value for the money spent. So, the moral of this story is that the Customer for an agile project must be chosen well. If someone from the business community isn't willing to 'champion' the project, or management isn't willing to allow someone who would be champion to spend the time needed to do so, then I'd question the organization's overall dedication to project. Dave Rooney I n d u s t r i a l L o g i c I n c.
  62. Re: Widgets[ Go to top ]

    The customer team acts in unison too. They speak and meet each day and have a single vision. If there are disputes then the product owner (customer team lead) has the final say.
    This is an interesting discussion. I have a significantly different set of experiences, as I have almost always worked in, and developed software for, far smaller groups than you and Erik are talking about here, so a lot of the issues you are discussing are out of my league! However, I think it provides an interesting contrast. Any attempt, in a small company, to set up a customer team that meets each day (or even regularly) can be totally impractical, simply due to the demands on staff, and so you can potentially wait indefinitely for a single clear vision. Specifications seem to me to be the only practical way to communicate. I would be interested if you have any ideas as to how these practices 'scale down'....
  63. Customer Involvement[ Go to top ]

    The customer team acts in unison too. They speak and meet each day and have a single vision. If there are disputes then the product owner (customer team lead) has the final say.


    This is an interesting discussion. I have a significantly different set of experiences, as I have almost always worked in, and developed software for, far smaller groups than you and Erik are talking about here, so a lot of the issues you are discussing are out of my league! However, I think it provides an interesting contrast. Any attempt, in a small company, to set up a customer team that meets each day (or even regularly) can be totally impractical, simply due to the demands on staff, and so you can potentially wait indefinitely for a single clear vision. Specifications seem to me to be the only practical way to communicate.

    I would be interested if you have any ideas as to how these practices 'scale down'....
    Hi Steve, This is a common issue with Scrum/XP. Personally I believe that it is in a customers best interest to get as much involved as possible. We often have domain experts who come and camp with us a day or two a week. They bring their laptops and getting out of their office allows them to get alot of work done without interruption. We seldon need to talk to them more than a few times each day, but having them their available just saves so much time an eliminates a load of waste. In some environments though this is just not possible. The next best bet is to have some one play the role of business analyst. This person meets with the customer regularly (weekly) and does most of the story writing, if you got a question on a story you ask the BA. If the BA needs input from the customer, then the story may be put on hold (waste), or better still you contact the customer by phone or e-mail. Another novel approach that I haven't tried personally is to let one of the developers become a domain expert. This may involve the developer becomming a user for a few weeks, working with users and experiencing their day-to-day issues. This person then can act as a surrogate customer/user when back in the development team. The real issue for me is responsibility and accountability. I know Erik disagrees, but I strongly believe that it is the customer who is responsible for delivering business value and ROI. Developers are just responsible for writing software IMO. So if the resultant system doesn't meet the business need, then as long as the Developers delivered what was requested, then it is the customer who is accountable for how the money has been spent, and the ROI. As developers we often want to jump in and take the lead, but this can break accountability. The customer should never be able to say, well the developers told me to do it. We can advise, but they must decide. After all it istheir money, and their business. If the customer isn't willing to ge involved to this extent, then I would question their commitment, and whether they really want the "system to be" in the first place. BTW. We keep all our user stories on a wiki. So they are available to anyone with an internet connection. So discussing a story over the phone is relatively easy. Customers can edit stories on line assigning busness value and priority for instance. PS. The problem with specs is that most customers find them daunting. They didn't write the spec and they seldom read all of it, so they have little ownership over the spec. The "system to be" belongs to the spec writer, not the customer. With stories, the users/customers are encouraged to write them themselves. Testers, BAs and developers help refine the stories, but essentially they try to keep them in the customers words. After a while the customers start to get the hang of it and produce pretty good stories off the bat. So the level of ownership of stories by customers is much, much hgher than with a spec, which comes back to my point about accountability. Paul.
  64. Re: Customer Involvement[ Go to top ]

    This is a common issue with Scrum/XP. Personally I believe that it is in a customers best interest to get as much involved as possible. We often have domain experts who come and camp with us a day or two a week. They bring their laptops and getting out of their office allows them to get alot of work done without interruption. We seldon need to talk to them more than a few times each day, but having them their available just saves so much time an eliminates a load of waste.

    In some environments though this is just not possible.
    Absolutely; for certain sizes of company, that is simply economically impossible.
    The next best bet is to have some one play the role of business analyst. This person meets with the customer regularly (weekly) and does most of the story writing, if you got a question on a story you ask the BA. If the BA needs input from the customer, then the story may be put on hold (waste), or better still you contact the customer by phone or e-mail.
    This seems somewhat more possible, in my situation.
    Another novel approach that I haven't tried personally is to let one of the developers become a domain expert. This may involve the developer becomming a user for a few weeks, working with users and experiencing their day-to-day issues. This person then can act as a surrogate customer/user when back in the development team.
    Now this is an approach that I have used with much success. It is the way I really do prefer to work; if for no other reason that it is both interesting and very rewarding to sit with other users using software you have helped to develop and to see (hopefully) productivity increase.
    The real issue for me is responsibility and accountability. I know Erik disagrees, but I strongly believe that it is the customer who is responsible for delivering business value and ROI. Developers are just responsible for writing software IMO. So if the resultant system doesn't meet the business need, then as long as the Developers delivered what was requested, then it is the customer who is accountable for how the money has been spent, and the ROI.

    As developers we often want to jump in and take the lead, but this can break accountability. The customer should never be able to say, well the developers told me to do it. We can advise, but they must decide. After all it istheir money, and their business. If the customer isn't willing to ge involved to this extent, then I would question their commitment, and whether they really want the "system to be" in the first place.
    Well, although, as I have said, you and Erik seem to have dealt with companies that are way bigger in scale that me, in my own small way, I do come down more on Erik's side. The smaller the business, the less practical and economic it may be for customers to get involved at that level. I think having an emphasis on the developer delivering a ROI can be good for the developer, as I will discuss later. Taking the 'domain expert' approach can really help with this, especially if the developer is truly an expert, or at least has significant knowledge of, the work of the customer.
    The customer should never be able to say, well the developers told me to do it.
    Why not? In some situations the developer can be more than just a programmer - they really can be an expert in the area in which the customer does business, be it scientific, technical, financial...
    PS. The problem with specs is that most customers find them daunting. They didn't write the spec and they seldom read all of it, so they have little ownership over the spec. The "system to be" belongs to the spec writer, not the customer.

    With stories, the users/customers are encouraged to write them themselves. Testers, BAs and developers help refine the stories, but essentially they try to keep them in the customers words. After a while the customers start to get the hang of it and produce pretty good stories off the bat. So the level of ownership of stories by customers is much, much hgher than with a spec, which comes back to my point about accountability.

    Paul.
    There is a middle way - write specs in a way that the customers can understand; have discussions. It isn't black-and-white. The Wiki idea has potential. Also, I have had considerable experience of what Erik mentions: "what is wanted by the user is often out of tune with what is needed by the business.": Especially in a smaller company, with much less management structure, and where there can be considerable autonomy for staff, you can end up with a range of different, often (in hindsight) conflicting, stories depending on who you talk to. I have seen projects that have failed because they relied on user stories in that situation, without realising that the users often had little or no coherent idea of how the business works. But anyway, I am not convinced that everything fits into neat, small stories, especially not in the scientific/technical area in which I often work - a customer requirement may be able to be expressed in just a few words, but can require considerable, even long-term, research and specification to get right. Details can't always be flexible - some technical software approaches have to be fully specified before implementation. Again, this is where experience in a particular software market is valuable; a developer can recognise a pattern of requirements and have possible solutions ready, especially when the developer him/herself truly is an expert in the domain. I would also use the term accountability, but with a different emphasis - developers should not forget that they are providing a service to the customer (although sometimes it can take some effort to find out what that actually means in practice). I have seen companies that have worked hard to provide software and services of a consistently high quality , even when that is possibly more than the customer needed, and the result has been the gaining of a very high reputation and considerable extra business through repeat customers and recommendation. Of course, you can go too far with this, but at least in my experience in a limited scale of clients, leaving the client alone accountable for business value is not always good business for the developer. This may be a factor only at the scale where I work, because at this scale virtually everyone in the company can see the impact of even the smaller projects, and gets feedback from their quality. So, I really favour the 'become a domain expert' approach (nice to be able to put a name to it at last!) I just don't see (yet? - I am prepared to be educated) how other methods are practical at this scale - one can try as hard as one can to insist in more organised customer involvement, but, for a range of reasons, that is sometimes never going to happen. And yet, to question the customer's committment seems very harsh.. they are often just doing their best to cope with the heavy demands of their business; and they need you to help. I hope this is of at least some interest; as I keep saying, the scale of company I develop for is quite different from those you have been discussing, but after all, micro/small/medium businesses are in the majority.
  65. Re: Customer Involvement[ Go to top ]

    Hi Steve, Yes, I undestand the different emphasis both you and Erik take. I see how it makes sense, it is indeed a pragmatic way forward. To be honest I do it too, but I just try to get the customer to take as much ownership of the problem as I can.
    Why not? In some situations the developer can be more than just a programmer - they really can be an expert in the area in which the customer does business, be it scientific, technical, financial...
    The reason why I do this is because often software is used as a political football. Very often the people expected to use the software aren't the people who have commisioned it. Usually it is a manager far removed trying to make a name for himself we comes up with the idea for new software. Erik's situation is better than most, at least they do value stream mapping. But as we all know, there is lies, damn lies, then statitistics. Depending on what you choose to measure you will get diferent results. I just beleve that the people doing the job day-to-day know best. I come from a TQM management background, and like Lean Manufacturing, it strongly focuses on self organised teams, and people taking direct responsibility for the way they work. I often see customers looking to me to solve their problems. It takes a while, but getting them to see that they can solve their own problems with our help really does change the relationship, and the value of the software we deliver IMO. I think in this situation that pragmatism wins, I'm just taking a purest slant. Having created a Lean/Agile software team though, you quickly realise that the next challenge is getting a Lean/Agile business. I guess this is where the 37Signal guys are which is why they probably started up on their own. BTW. The story card is just a token. We sometimes have pages of supprting info on the wiki for a story. So yes like you say any mathematical algorithms need to be captured, along with possible examples of expected results (See Ward Cunningam and FIT) and test criteria. We also capture the actor invloved, what the system needs to do, and the business reason why. Each story is then assigned an estimate, business value and priority for planning. So stories can get verbose - the main thing though is that they can change. There is no version control on our wiki, although we can see who made a chnage and when. Paul.
  66. Re: Customer Involvement[ Go to top ]

    Hi Steve,

    Yes, I undestand the different emphasis both you and Erik take. I see how it makes sense, it is indeed a pragmatic way forward. To be honest I do it too, but I just try to get the customer to take as much ownership of the problem as I can.
    Well, I can't speak for Erik, but I try to do the same. I would not really call my approach pragmatic - in some situations it can be the only way forward.
    Usually it is a manager far removed trying to make a name for himself we comes up with the idea for new software. Erik's situation is better than most, at least they do value stream mapping. But as we all know, there is lies, damn lies, then statitistics. Depending on what you choose to measure you will get diferent results. I just beleve that the people doing the job day-to-day know best.
    Not always; in one situation I worked in many, many years ago a company needed to move forward to a more automated IT-based approach (part of which, for this company, required customised software from me). This was necessary to compete with other companies who were already using this approach, and to satisfy and increasing number of clients who required it. However, talking to the 'shop floor' you got a different story, with some insisting on staying with a more manual (craft-based) way of working that they had been trained in, and had used for years. These people either could not, or would not, see the overall picture, even though the new approach would mean no threat to their jobs. It would simply not have been impossible to move forward if one assumed that the people doing the day-to-day job knew best, as that you ended up with a range of wildly conflicting ideas and priorities.
    I come from a TQM management background, and like Lean Manufacturing, it strongly focuses on self organised teams, and people taking direct responsibility for the way they work.

    I often see customers looking to me to solve their problems. It takes a while, but getting them to see that they can solve their own problems with our help really does change the relationship, and the value of the software we deliver IMO.
    Well, as I said, I usually work with much smaller companies. Ideas come from everyone, and are certainly not just pushed by a manager. But what you get is plenty of ideas, but no-one with the time to discuss them in detail!
    BTW. The story card is just a token. We sometimes have pages of supprting info on the wiki for a story. So yes like you say any mathematical algorithms need to be captured, along with possible examples of expected results (See Ward Cunningam and FIT) and test criteria. We also capture the actor invloved, what the system needs to do, and the business reason why. Each story is then assigned an estimate, business value and priority for planning. So stories can get verbose - the main thing though is that they can change. There is no version control on our wiki, although we can see who made a chnage and when.

    Paul.
    All sounds pretty reasonable to me - kind of formalising they way I have usually had to work for pragmatic reasons, but nothing wrong with that. It is interesting to see that we do have some common ground, with so much difference in circumstance.
  67. Re: Customer Involvement[ Go to top ]

    Hi Steve,
    Not always; in one situation I worked in many, many years ago a company needed to move forward to a more automated IT-based approach (part of which, for this company, required customised software from me). This was necessary to compete with other companies who were already using this approach, and to satisfy and increasing number of clients who required it. However, talking to the 'shop floor' you got a different story, with some insisting on staying with a more manual (craft-based) way of working that they had been trained in, and had used for years. These people either could not, or would not, see the overall picture, even though the new approach would mean no threat to their jobs. It would simply not have been impossible to move forward if one assumed that the people doing the day-to-day job knew best, as that you ended up with a range of wildly conflicting ideas and priorities.
    True. We are using a number of labels here: Customer, Users, domain expert, champion etc. I think Dave Rooney makes the point well. You need someone with a good grasp of the business problem and the authority and clout to champion the project and instigate change. This person must be capable of seeing the big picture, and must also be sensitive to the day-to-day issues of the people who in the end will be using the software, and upon which success ultimately depends. So the Customer must be chosen well. Scrum calls this person the Product Owner. Often this person is not available for the details of how a user interface sould be laid out, or on the precise algorithm needed to calculate X or Y. So the Product Owner/Customer mostly delegates this kind of stuff to others. These others is who I refer to as the Customer Team. The Customer Team can be made up of Users and other Domain Experts, Their job is to do the bidding of the Customer and help steer the project.
    It is interesting to see that we do have some common ground, with so much difference in circumstance.
    Yep. It's the same old stuff where ever you go. The biggest problems tend to be people related, other than that, the rest is easy :^) Paul.
  68. Re: Customer Involvement[ Go to top ]

    Hi Steve,

    Not always; in one situation I worked in many, many years ago a company needed to move forward to a more automated IT-based approach (part of which, for this company, required customised software from me). This was necessary to compete with other companies who were already using this approach, and to satisfy and increasing number of clients who required it. However, talking to the 'shop floor' you got a different story, with some insisting on staying with a more manual (craft-based) way of working that they had been trained in, and had used for years. These people either could not, or would not, see the overall picture, even though the new approach would mean no threat to their jobs. It would simply not have been impossible to move forward if one assumed that the people doing the day-to-day job knew best, as that you ended up with a range of wildly conflicting ideas and priorities.


    True. We are using a number of labels here: Customer, Users, domain expert, champion etc. I think Dave Rooney makes the point well. You need someone with a good grasp of the business problem and the authority and clout to champion the project and instigate change. This person must be capable of seeing the big picture, and must also be sensitive to the day-to-day issues of the people who in the end will be using the software, and upon which success ultimately depends.

    So the Customer must be chosen well. Scrum calls this person the Product Owner. Often this person is not available for the details of how a user interface sould be laid out, or on the precise algorithm needed to calculate X or Y. So the Product Owner/Customer mostly delegates this kind of stuff to others. These others is who I refer to as the Customer Team. The Customer Team can be made up of Users and other Domain Experts, Their job is to do the bidding of the Customer and help steer the project.
    The way you describe things still seems much too large-scale for some of the situations I have worked in; with a company of 10 or 20 people there is often simply no-one to delegate to! The developer can end up like other trusted service providers to the company, such as accountants - in this case the developer(s) may be true Domain Expert, and may have to be trusted to steer things, as there is simply no spare staff to form any kind of 'Customer Team'.
  69. Re: Customer Involvement[ Go to top ]

    The way you describe things still seems much too large-scale for some of the situations I have worked in; with a company of 10 or 20 people there is often simply no-one to delegate to! The developer can end up like other trusted service providers to the company, such as accountants - in this case the developer(s) may be true Domain Expert, and may have to be trusted to steer things, as there is simply no spare staff to form any kind of 'Customer Team'.
    OK. There are no golden rules here. What we've identified are a number of roles, and a number of potentially conflicting interests. The same person can perform a number of roles if called upon. You need to decide. You have to tailor these ideas to fit your situation. if you do become the Customer though, be careful that you're not just becoming the fall guy. Organisations are political places and as developers we need to watch out for the politics , just like everyone else. Paul.
  70. Re: Customer Involvement[ Go to top ]

    Hi Steve,
    You have to tailor these ideas to fit your situation. if you do become the Customer though, be careful that you're not just becoming the fall guy
    Just some clarification. I see the Customer role as highly political. The Customer needs to overcome opposition to the project and path the way for change. The Customer also needs to be a good communicator and carry significant influence. They also need to be someone with the clout to inforce his/her will when needed. The Customer is usually also the budget holder and the person responsible for paying my invoice :^). So apart from the obvious conflict of interests :^), as a contractor, I would not want to adopt the role of Customer (besides, I dislike politics). But if you carry sufficient clout in your organisation, and like you say, in a very small company, I see no reason why a developer could not also take on a large part of the customer role. But I can't help thinking that there must be someone else who is better placed? Paul.
  71. Re: Customer Involvement[ Go to top ]

    But if you carry sufficient clout in your organisation, and like you say, in a very small company, I see no reason why a developer could not also take on a large part of the customer role. But I can't help thinking that there must be someone else who is better placed?

    Paul.
    But who? These very small companies can be intense places - staff may have neither the time or the inclination to hold detailed meetings. For example, imagine a small company with salesmen (including the managers) almost always on the road. As for ending up the fall guy and the politics; well, that goes with the territory. But, it can be very rewarding - even small projects can have dramatic impacts, and improve productivity and quality.
  72. Re: Customer Involvement[ Go to top ]

    As for ending up the fall guy and the politics; well, that goes with the territory. But, it can be very rewarding - even small projects can have dramatic impacts, and improve productivity and quality.
    Increased risk for increased award. Sometimes what an organization needs is for someone to step up to the plate and take responsibility.
  73. Re: Customer Involvement[ Go to top ]

    But if you carry sufficient clout in your organisation, and like you say, in a very small company, I see no reason why a developer could not also take on a large part of the customer role. But I can't help thinking that there must be someone else who is better placed?

    Paul.


    But who? These very small companies can be intense places - staff may have neither the time or the inclination to hold detailed meetings. For example, imagine a small company with salesmen (including the managers) almost always on the road.

    As for ending up the fall guy and the politics; well, that goes with the territory. But, it can be very rewarding - even small projects can have dramatic impacts, and improve productivity and quality.
    Well, who is ultimately responsible for the success of the business? I would say start at the CEO and work my way down. I can see that a developer may want to be a surrogate customer taking on the role of the Customer Team as I've described it, but someone with financial responsibilities should be steering the project and providing some senior management oversight in my view. But hey, what ever works for you. I know that the working culture can vary drastically in small organisations, so you need to find a balance that fits your situation. Paul.
  74. Scaling Down[ Go to top ]

    Any attempt, in a small company, to set up a customer team that meets each day (or even regularly) can be totally impractical, simply due to the demands on staff, and so you can potentially wait indefinitely for a single clear vision. Specifications seem to me to be the only practical way to communicate.
    I experience this all the time and I work in a Fortune 100 company. My preferred approach is to become a domain expert. For me personally, this gets easier every year. I tend to know more about the organization as a whole then my users for any given application because I've had the opportunity to work very closely with many more pieces of the organization. However, while I've been very successful with it, I've never quite managed to get others to be successful with it. My experience has been that typically if someone is a Java web developer (or .NET or Oracle or whatever) that's what they want to do. Honestly, with many of them it's impossible to get the interested in principles of mathematics or computer science that in my opinion are directly applicable to their jobs. At one time I would have thought having a series of differential equations describing what an application is supposed to compute would be an ideal spec. No ambiguity! Boy was I wrong... But I digress... I find it a little ironic that we're discussing "Agile" development, and then the question of "How do I make this scale down to my situation" comes up. Dave is jumping in with "don't confuse the user role with the customer role" and makes me feel like I'm reading a RUP book (I actually like RUP, but it's not a process, more like a toolkit for building processes). As for the "who is responsible for ROI" question... My name is usually on the budget, I'm the one who put together the funding proposal, and the money came out of my department - even though my internal customer is in another department. So I'm generally responsible for ensuring it will be met - or cutting bait if it won't. As for people where their customer is actually footing the bill (probably most), I would say Steve is right on this one. You want the customer to take as much ownership as possible, but actively working to ensure a good ROI will probably greatly improve your long-term relationship with the customer. It's kind of an "up-skilling" of software development. Finally, what happens when you have a problem domain where simply breaking it apart into chunks can take several months? Well, IMHO you don't even try for quick iterations in that case, at least not until you've decomposed the problem...although even for simple domains my opinion is 4-8 week icrements are much more practical, with a sweet spot around 6. That's not saying the development team should strive for a "working" build at the end of each week and a "compiling" build at the end of each day, but the customer doesn't want to be bothered by testing ever single week.
  75. Re: Scaling Down[ Go to top ]

    Hi Erik,
    As for people where their customer is actually footing the bill (probably most), I would say Steve is right on this one. You want the customer to take as much ownership as possible, but actively working to ensure a good ROI will probably greatly improve your long-term relationship with the customer. It's kind of an "up-skilling" of software development.
    I totally agree with this too. Maybe I didn't explain myself that well. Paul.
  76. Re: Scaling Down[ Go to top ]

    Hi Erik,

    As for people where their customer is actually footing the bill (probably most), I would say Steve is right on this one. You want the customer to take as much ownership as possible, but actively working to ensure a good ROI will probably greatly improve your long-term relationship with the customer. It's kind of an "up-skilling" of software development.


    I totally agree with this too. Maybe I didn't explain myself that well.

    Paul.
    I'm as eager as anyone else to see my project succeed, and if ROI is the measure of that success then, I do will do everything I can to see a good ROI. I am self employed so I know all about pleasing my customer. On one project recently, we had a customer who was very stand offish. When you spoke to him he was very much behind the project, and he was very eager to know how well we where progressing. We were using XP so we spent some time convincing him of the benefits of iterative development and regular releases and he was behind us 100%. There was just one problem... He didn't do anything. When it came close to our first release, we needed the infrastructure team to install and configure servers to run the software. We did all we could, trying to negotiate with this external body, but we needed the support of our customer. So we asked for help, and it fell on deaf hears. The issue of end user training was raised. We had been having regular meetings with small groups of trial users, but the system was going nation wide, so large numbers of end users needed to be trained. Again we spoke to our customer, and we didn't hear back. One of the engineers went half way across the country to train a bunch of users who we knew were on a training course that day. After their initial training that was sanctioned by their boss, they were told about our training. They knew very little about our project and refused the training and went home. The developers trip was a waste of time. Everything eventually came to a head. It turns out that our Customer didn't have the clout to get anything done and wasn't that interested in doing anything about it He was replaced by a superior. All of a sudden, budget was found for end-user training, and we began to get our way with the infrastruture team, having free rain over our own servers. The moral of this story is that the customer and developers succeed together as a team. There is a lot more to rolling out software then just producing code. There are people to be informed, users to train, infrastruture and resources to be bought, licenses etc. You need the support of your customer to get these types of things done. If your customer thinks that making the project a success is all your responsibility, then you can be in for a bumby ride. Paul.
  77. Re: Scaling Down[ Go to top ]

    Hi Erik,

    As for people where their customer is actually footing the bill (probably most), I would say Steve is right on this one. You want the customer to take as much ownership as possible, but actively working to ensure a good ROI will probably greatly improve your long-term relationship with the customer. It's kind of an "up-skilling" of software development.


    I totally agree with this too. Maybe I didn't explain myself that well.

    Paul.


    I'm as eager as anyone else to see my project succeed, and if ROI is the measure of that success then, I do will do everything I can to see a good ROI. I am self employed so I know all about pleasing my customer.

    On one project recently, we had a customer who was very stand offish. When you spoke to him he was very much behind the project, and he was very eager to know how well we where progressing. We were using XP so we spent some time convincing him of the benefits of iterative development and regular releases and he was behind us 100%.

    There was just one problem... He didn't do anything. When it came close to our first release, we needed the infrastructure team to install and configure servers to run the software. We did all we could, trying to negotiate with this external body, but we needed the support of our customer. So we asked for help, and it fell on deaf hears.
    I had a difficult experience when I was part of a small software company around 18 years ago. The customer was part of a much larger organisation, and the internal politics were very strange. At one point I had to reverse-engineer one of the customer's products because they were either not getting help from others in their company, or wanted to keep what they were doing secret. It put me off dealing with large customers - I much prefer the more open and exciting (and less weird) situation in smaller companies and groups. Anyway, If I may, I would like to take us briefy back to something we were discussing a short while ago - YAGNI. Erik announced something exciting on another thread here:
    XQuery support is on the roadmap of JPOX + persistence of XML data and aggregation of multiple data sources including RDBMS and XML.
    The way I personally interpret YAGNI is that developers should not work to add features that may not be needed. However, I see no harm at all in selecting technologies that already provide features, even though they may not be needed, as long as this does not impact development in other ways. For example, if I were working with Java on a project, I would have no hesitation in choosing JDO 2.0 as my persistence solution. A simple choice of JDO over other approaches gives my customer the freedom to select different implementations, can make migration between databases almost trivial (this not just wishful thinking - I make real use of these capabilities), and will allow the customer access to an increasing range of persistence methods, all with no extra effort from me. The last point is key - I have enabled this simply by the choice of a technology, not by extra coding, or the need for extra resources. If I have recommended use of JPOX to a customer, they will be able to gain these extra capabilities simply by upgrading to a future version. This relates directly to what I have been saying about producing software that is to a high quality, providing extra value to the customer, giving them a higher return for their investment. Careful choice of languages and technologies is part of the way I try and do this. It is very rewarding to have taken this approach, and to then see the benefits to customer some time after, whereas 'extreme' YAGNI could have led to short-term solutions and long-term disappointment. (Maybe I have been reacting to a 'straw man' version of YAGNI i.e. 'don't even consider what the customer might need in the future', but my impression is that these things are very hotly debated.) Another example of how careful technology choice can bring real future benefits is my experience with JSF. After a much reviewing and discussion I decided quite a while ago that I would, where possible, use JSF for web development. Now, barely a week goes by without the announcement of yet more component libraries or new features for JSF. The return on the initial investment in JSF - the time taken to learn the technology (and work with early buggy versions!) has been significant - the range of high-quality 'off-the-shelf' components is large and growing. Simply by selecting JSF I have added value for my customer, as they also get access to this important resource for any future extensions to sites we have written for them. My technology selections are not for everyone; but I hope this helps to clear up why I don't start new projects with a pure 'try and see what works' approach at least in the area of languages and APIs, as I am always bearing in mind how I can add value to a project through careful technology choices, while sticking to what I think is a reasonable degree of 'YAGNI'. I believe this approach is relevant to other themes in this thread. You can help cut the cost of change dramatically not just by development practises, but also by selecting technologies that can easily adapt to change, both as requirements change during development and in the lifetime of the application (just look at how versatile and adaptable JDO is). Particularly in circumstances that we have discussed where communication is not as good it might be, there can be significant changes in requirements as development proceeds. With my choices of technologies I know that this is not going to raise too many problems. In less-than-ideal development environments we need to be sure we aren't led up a blind alley because what seemed to fit well at one stage no longer does after a new and unexpected requirement appears out of the blue. It is not just in development languages that I take this approach; it has worked very well in terms of infrastructure and general software - I have always tried to make choices that enable the customer to deal with change, even when the customer insists that no such change is likely, as they have been so often proved wrong. Not that I want to start a long debate; it is just that reading back, I could have expressed myself more clearly (and perhaps less flippantly) about this, and I hope I have succeeded.
  78. Re: Scaling Down[ Go to top ]

    Hi Steve
    However, I see no harm at all in selecting technologies that already provide features, even though they may not be needed, as long as this does not impact development in other ways.
    I agree. The issue is does not impact development in other ways. By virtue of having features that add no value today, the danger is that the technology becomes more complex and expensive to use. So the cost of development and ongoing maintenance increases for no actual benefit. This may or may not be the case.
    believe this approach is relevant to other themes in this thread. You can help cut the cost of change dramatically not just by development practises, but also by selecting technologies that can easily adapt to change,
    I agree here too. In fact this is how agile came about. With the arrival of Object Orientation people realised that they could use techniques such as polymorphism to ensure that an idea was expressed in only one place in the code, and that there was little or no code duplication. The advantage of this was that changes became relatively cheap. If a feature needed to change, you only needed to change the code in one place, unlike the "shot gun change" scenario typical of procedural programs. This idea is often expressed as DRY (Don't Repeat Yourself) today.
    My technology selections are not for everyone; but I hope this helps to clear up why I don't start new projects with a pure 'try and see what works' approach at least in the area of languages and APIs,
    I agree here too. I would always prototype a technology (or several technologies) before embarking on their use in a critical production scenario.
    Not that I want to start a long debate; it is just that reading back, I could have expressed myself more clearly (and perhaps less flippantly) about this, and I hope I have succeeded
    Yes, you have. We are aiming for the same things, and like often in life their are many ways to skin a cat. The Agile mindset tends to focus on OO as the future proof technology of choice that will enable change at low cost. My concern is that many (most actually) Java programmers just don't get this. Instead, they happily write procedural code, hoping that elaborate tools will deal with their productivity and maintenance issues. Tools like JSF and Agile are not in conflict though. And everyones technology choices are different. I respect your choices, my goal is/was to explain a different perspective. I don't believe that there is one right way, and I would never persume to tell someone else how to develop software, especially when I have no knowledge of their context. Paul.
  79. Finally Agreement[ Go to top ]

    Wow...do you, Steve, and myself just reach some sort of agreement? Hah...I better find something to debate.
    The Agile mindset tends to focus on OO as the future proof technology of choice that will enable change at low cost. My concern is that many (most actually) Java programmers just don't get this. Instead, they happily write procedural code, hoping that elaborate tools will deal with their productivity and maintenance issues.
    I agree that most programmers of OO languages don't really grok OO. But, I don't think OO is the be-all and end-all of programming paradigms. The relational model, especially the pure relational model, offers a lot in areas where OO is lacking. Of course, the converse is true as well. Furthermore, if you look at CLOS in Common Lisp, there are interpretations of OO that really bare little, if any, resemblance to the foundations laid in Smalltalk that are very compelling. All in all, I don't think OO is future proof. I certainly hope it isn't, because if it was that would mean stagnation.
  80. Re: Finally Agreement[ Go to top ]

    Hi Erik,
    All in all, I don't think OO is future proof. I certainly hope it isn't, because if it was that would mean stagnation.
    Yes, but that is assuming that we have exploited all that Object technology has to offer. I don't believe we have. Neither does the guy who coined the term "Objects", Alan Kay: http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-October/017019.html This was posted in 1998. Object technology is not about Classes or Inheritance, yet that is what the C++ community took away from Alan Kay and his teams work with Smalltalk over two decades (from the 1960's to the early 80's). I believe that the Object Technology revolution is still to happen. We have all just spent the last 20 years down a blind ally, and it has taken this long to begin to wake up to the fact. Paul.
  81. Re: Finally Agreement[ Go to top ]

    We have all just spent the last 20 years down a blind ally, and it has taken this long to begin to wake up to the fact.

    Paul.
    That is a wild exaggeration. Mainstream development 20 years ago was unrecognisable from today (been there, felt the pain). You may call the move to managed-memory portable OOP languages a blind alley - I call it major progress - there has been a revolution in the ease of development.
  82. Re: Finally Agreement[ Go to top ]

    We have all just spent the last 20 years down a blind ally, and it has taken this long to begin to wake up to the fact.

    Paul.


    That is a wild exaggeration. Mainstream development 20 years ago was unrecognisable from today (been there, felt the pain). You may call the move to managed-memory portable OOP languages a blind alley - I call it major progress - there has been a revolution in the ease of development.
    Hi Steve, Did you read Alan Kay's post? I think you have mis-quoted me here. My point is that we may have mis-nterpreted what was important about the work into OT at Xerox-Parc. There is strong evidence, that focusing on objects themselves rather than what sits between them, the "ma" as Alan Kay describes it was the wrong emphasis. Much of the meta capbabilities available from pure message sends have been lost in todays crop of static OO languages such as Java and C#. They are only just being rediscovered again in languages like Ruby and Python. This loose coupling is akin to the web and it's REST approach as Alan points out in his post. Possibly allowing us to "script" together various components from different sources using meta-data. Lets drop this. To be honest I'm tired of people knocking concepts that they haven't even bothered to try and understand. Paul.
  83. Re: Finally Agreement[ Go to top ]

    Lets drop this. To be honest I'm tired of people knocking concepts that they haven't even bothered to try and understand.

    Paul.
    It is fair to say that a significant number of developers don't understand these issues, but I don't think it is fair to throw that accusation at those who have been using Smalltalk since the mid-80s, and following the work of Kay, Goldberg and others since that time. There are many of us. Alan Kay has one vision of how objects should be used, and it is an important one. But, it is unreasonable, and rather extreme, to claim that the past 20 years have been a blind alley in terms of OOP simply because we haven't done things his way. OOP has been a major factor in improving development, even for those who only use its capabilities in limited ways. But, to prevent 'stack overflow', I'll stop here.
  84. Now Disagreement[ Go to top ]

    Much of the meta capbabilities available from pure message sends have been lost in todays crop of static OO languages such as Java and C#. They are only just being rediscovered again in languages like Ruby and Python. This loose coupling is akin to the web and it's REST approach as Alan points out in his post. Possibly allowing us to "script" together various components from different sources using meta-data.
    Based on experience I'd say that metaprogramming, meta-data driven programming, and even plain-old data-driven programming are at least an order-of-magnitude more difficult than simply coding to requirements (or users stories or whatever you use). Don't get me wrong, I believe in those techniques. But I believe there is a growing chasm between the application developer and the framework/library/system developer. Frameworks and libraries are looking less and less like tools to abstract away "the plumbing" and more like magic. IMHO this is very, very bad. That art of metaprogramming (and the others) really needs to somehow be made more accessible to the "common developer." Otherwise it will advance its way right out of a market. I'm not saying "dumbed down." Relational theory is (arguably) more complex than OO metaprogramming, but it can be effectively taught because it has formalized. OO metaprogramming will never really take of until it has a concrete enough theory behind it.
  85. OT and a working component model[ Go to top ]

    Much of the meta capbabilities available from pure message sends have been lost in todays crop of static OO languages such as Java and C#. They are only just being rediscovered again in languages like Ruby and Python. This loose coupling is akin to the web and it's REST approach as Alan points out in his post. Possibly allowing us to "script" together various components from different sources using meta-data.


    Based on experience I'd say that metaprogramming, meta-data driven programming, and even plain-old data-driven programming are at least an order-of-magnitude more difficult than simply coding to requirements (or users stories or whatever you use).

    Don't get me wrong, I believe in those techniques. But I believe there is a growing chasm between the application developer and the framework/library/system developer. Frameworks and libraries are looking less and less like tools to abstract away "the plumbing" and more like magic.

    IMHO this is very, very bad. That art of metaprogramming (and the others) really needs to somehow be made more accessible to the "common developer." Otherwise it will advance its way right out of a market.

    I'm not saying "dumbed down." Relational theory is (arguably) more complex than OO metaprogramming, but it can be effectively taught because it has formalized.

    OO metaprogramming will never really take of until it has a concrete enough theory behind it.
    Hi Erik, I agree, meta-programing needs to be simple. DSLs as used in Rails is one attempt to simplify meta-programming. I think Alan Kay is saying as much too, when he says that there needs to be a fence when crossing over into changing the semantics of the language. I think DSL's like Rails provide such a fence. The problem is that what we need is unknown (if I knew I'd wouldn't be telling on the TSS). What we do know though is the properties of the ideal solution. Look at the mashups people are creating knowadays on the web. Using bits and pieces of existing web content, they mix them together to create something new. I like this site: http://www.netvibes.com/ So we would like to mashup ojects. The first pre-requisite is late binding. We need to link together objects after compile time. The secound is some kind of loose binding mechanism. For the web it is URLs and HTTP. For objects it is messaging and polymorphism. Lastly we do need some meta-capabilities. Using the web as an analogy. You know when you are downloading video because of the mime type. So with objects we should have some way of knowing what we are dealing with. If you remember the Self video, Dave Grisling spoke about this - objects that when you bang on them they bang back. They tell you something about themselves at runtime. So I don't know the answer, but I do know the properties - late-bound and message based, with some meta capability. I'm not sure whether it should be prototype or class based etc. We shoudn't still be building software one statement at a time in 2006 IMO. We should have discovered a viable component model by now. After all we where promised object based components back in the early 90's. So where are they? Paul.
  86. I agree, meta-programing needs to be simple. DSLs as used in Rails is one attempt to simplify meta-programming. I think Alan Kay is saying as much too, when he says that there needs to be a fence when crossing over into changing the semantics of the language. I think DSL's like Rails provide such a fence.
    You're missing my point. The Rails DSL no doubt simplifies the act of leveraging metaprogramming, but it does not simplify the metaprogramming itself. But I'm not sure simplification is what is needed. I think what's needed is a concrete theory with concrete definitions.
  87. I agree, meta-programing needs to be simple. DSLs as used in Rails is one attempt to simplify meta-programming. I think Alan Kay is saying as much too, when he says that there needs to be a fence when crossing over into changing the semantics of the language. I think DSL's like Rails provide such a fence.


    You're missing my point. The Rails DSL no doubt simplifies the act of leveraging metaprogramming, but it does not simplify the metaprogramming itself. But I'm not sure simplification is what is needed. I think what's needed is a concrete theory with concrete definitions.
    Hi Erik, I understood you the first time, and I agree. Internal DSL's do simplify the problem somewhat, but I agree that they are still complex and aren't the whole solution. Language work benches sound promising as a way of hiding programmers from what lays beneath the DSL, but this technology is still new and unproven. My point isn't that I have the answers, but that the answers are still there to be found. Alan Kay is making the same point. Smalltalk was a research vehicle, and as he says in his post that he didn't intend it to be taught as the last word in Object Technology, and he feels that the work he did at Xerox Parc is still incomplete. My point is that we have been looking in the wrong places for answers, and that only hands on experience with real applications will decide what works and what doesn't. This is how Smalltalk was created in the first place. Before Smalltalk 80, there was Smalltalk-72, Smalltalk-76 etc. I believe there was about 5 incarnations of Smalltalk before it was released to the public. I see the Self work at Sun as a continuation of the same themes that were being explored at Xerox Parc - objects as real tangible things. The ideas from xerox have been very successful, they have given us the desktop metaphor, windowing operating systems, and the idea that applications are tangible things for users to manipulate. Alan Kay applied the same idea to programming, so objects were real and alive and tangible, and could be manipulated and composed together to create new larger objects. He saw no difference between programming and the end user experience. An end user is just a programmer who manipulates larger grained objects, sending them messages and getting back useful results. So objects aren't constraned by applications as we tend to view OT today. This emphasis has been lost over the years as it was not feasible with static languages. With the rise of dynamic languages though, this whole area can be explored once more. So as I said before, I still think we are at the beginning, and Object Technology has still got a long way to go. Paul.
  88. User Stories[ Go to top ]

    Hi Erik, The devil is in the detail. One tool in the Scrum/XP toolkit that addresses many of the things you raised with regards to the usefulness of a spec. is a user story. A user story is a brief description of a feature. Normally written on a small index card. This card then becomes a token for that feature. It is similar to the Kanban system as used in Japanese Motor manufactoring: http://en.wikipedia.org/wiki/Toyota_Production_System The card gains an estimate from developers, and then a business value and priority from the customer team. It also gains a set of acceptance tests from the test team, that defines when the story is done. The card is an agreement to discuss a feature at a later date. The details of the story are only fixed after implementation. It is not uncommon for the implementor to pull together a customer, tester and business analyst to walk through a story just prior to implementation. The development team implement the stories in priority order, with the tester provisionally accepting finished stories as done. Accepted stories are recorded as "earned value" and go towards the teams "velocity" for that iteration. Earned value and velocity are used for planning. Finally stories are formally accepted at the end of iteration showcase/demo. Completed stories are demonstrated and formally accepted by the customer. At this time the system is deemed production ready and available for release. The system is deemed releaseable at the end of each iteration (every two weeks), although actual releases are usually less frequent (once every 1 to 3 months). Mike Cohen as got a book on how to write user stories, which is similar to how you would write a spec, but unlike specs stories are small (3 to 5 days worth of work to implement) and fluid. A story can change at any time. Paul.
  89. Hi John,

    But all of us are not building OS/370, and programming is not an art, but a craft, and a craftsman can grasp all but the biggest system in his mind.


    This is interesting. Art is not craft? Please explain. Also I was of the impression that our cognitvie abilities generally limited us to holding 7 +/- 2 things in our minds at once. How does a craftsman transcend this? Please expand.

    Paul.


    Hello Paul,

    Of course art *can* be craft, and craft *can* be art. But they are not synonyms, or are they?

    A Craftsman is of course human, and has the same limitations as other humans. A good craftsman is however adept at using "divide-and-conquer" to divide each of the seven things in his head into seven smaller things, which he can then ponder on, and so on.

    I didnt prohibite a craftsman from taking notes, did I?

    Anyhow, its nice that you take your time to raise objections to my thoughts, but I think that you need to take a more serious stab at it if you want to prove me wrong rather than mock me.

    Brrrr - J
    Hi John, You've been on TSS too long. No I wasn't raising objections, I was merely seeking clarifications. BTW I agree with you, I didn't want to assume that your understanding was the same as mine that's all. Paul.
  90. Hi John,
    Unless you´re actually building OS/370...
    I agree with the rest. I didn't understand this bit. Please explain. Paul.