|
Sponsored Links
Resources
Enterprise Java
Research Library
Get Java white papers, product information, case studies and webcasts
|
News
News
News
|
Messages: 36
Messages: 36
Messages: 36
Printer friendly
Printer friendly
Printer friendly
Post reply
Post reply
Post reply
XML
XML
XML
|
 |
Do We Need Unit Testing?
This poll examined how organizations perform unit testing. Is it an informal activity that is done before integration if there is some time left after programming or is it the key element of the development effort? The question was: How is unit testing performed at your location?
Answers 2008 (2006)
Unit testing is not performed: 17% (13%)
Unit testing is informal: 40% (46%)
Unit tests cases are documented: 9% (11%)
Unit tests cases and their executions are documented: 14% (16%)
We use a Test Driven Development approach: 20% (14%)
Participants: 384 (2006: 460)
Ending date: October 2008 (February 2006)
Source: Methods & Tools (http://www.methodsandtools.com/)
Despite the fact that the number of TDD adopters has grown nicely since the previous survey, you can notice that unit testing is still widely conducted in a informal manner, when it is not simply ignored by developers. This could sound weird when many people announced a general adoption of the agile approaches, but the results of our survey are similar to many other polls on the same topic.
Comparing the two surveys, it seems that people that were already doing unit testing formally have switched towards a TDD approach. People that don't do unit testing have different reasons. Some will consider simply that they don't add value to their development process, which is sometimes difficult to believe. For others, it is the lack of time, a reason more easier to understand ;o) Many complains that unit test are hard to write, but creating a good unit test is a proof that you understand what your code should do. I agree however, that it could be difficult to maintain large libraries of unit testing scripts if requirements are changing constantly. In the "good" reasons not to perform unit testing, some thinks for instance that the client side of Web application is not suited for this kind of tests. There are also some organizations that have separate testing teams. Their developers will rely entirely on the QA guys to test their application. You can also consider that when the software has a very limited life expectancy, it is not worth making unit tests.
|
|
|
Message #272091
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Do We Need Unit Testing?
Um, is that title intended to bait people (like me)? ;)
Were those the actual questions for the poll, or did the questions elaborate on what is meant by "formal" or "documented"? Does an xUnit test written after the fact count for either of those? I could see someone who doesn't practice true TDD but does write automated unit tests being somewhat confused about how to answer.
Regardless, I'm heartened to see the TDD numbers growing.
Dave Rooney
Mayford Technologies
|
|
|
Message #272094
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Provocative... but maybe not ;o)
Hello Dave,
I agree that the title is catchy. Every time you ask a question about unit testing, a vast majority will say: you cannot do without them. However if you look at these numbers (and those from other surveys listed on http://www.methodsandtools.com/dynpoll/oldpoll.php?UnitTest2), it seems that a large number of developpers give little importance to unit testing (or doesn't have time to perform them). So the question could be rephrased as: Is "we need unit testing" just a politically correct answer?
Yes, writing tests script is considered as documenting the unit testing activity.
|
|
|
Message #272097
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
my 2 cents
From my little experience (almost 10 years in java in 6 different organizations) the reason for not doing unit tests:
1. Management, most of the dev managers, project managers I worked with saw UnitTesting as an obstacle for delivery.
2. Developpers, a vast majority of developpers I worked with had never heard about unit testing and saw it as a loss of time and productivity. The vast majority never read a software engineering book ("hey man !! I don't need to I'm finally got out of school", "those are so expensive and the compagny doesn't pay for them", "What ? I should read that on my spare time?!").
After all these years I learned to unit test on my own, without telling my manager, I buy my books, read them at home, play at home with libraries, frameworks, integrate them in the projects I work and hope one day to find a mature team.
The forums/site I read and sometimes participate on are not the overall state of the art in the industry because most of developpers don't connect/read/stay tuned.
I sound bitter but I'm not I'm just more patient now because I can see that slowly, very slowly the industry is changing.
|
|
|
Message #272099
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Unit Testing
My experience: Unit Testing is always mandatory (and not important) when one creates an interface that has to stay stable and that has a long lifecycle. So typical targets for unit tests for me are things like web services, business components, ejbs and all kinds of transport objects (unit tests for copy constructors, equals operations etc.). Unit tests are a pain if they are written to test trivial stuff that always works like generated setters or that are changing their behavior quite often and in a subtle way like web application controllers and UIs.
|
|
|
Message #272107
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit Testing
Unit tests are a pain if they are written to test trivial stuff that always works like generated setters...
Agreed. Besides, for Eclipse at least, you would just be duplicating the unit tests that the developers wrote for the code that performs the code generation!
...or that are changing their behavior quite often and in a subtle way like web application controllers and UIs.
I disagree for controllers. When there is significant and/or frequent change, the unit tests provide a safety net for regressions. They can also form the specification for the changes that need to take place.
For the UI, it depends on how truly independent the UI code is from the business logic. If you really know how to to MVC/MVP/Humble Dialog/etc. well, then I think you can get away without them. Once you see an 'if' statement in your UI code, you need tests.
Dave Rooney
Mayford Technologies
|
|
|
Message #272110
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit Testing
My experience: Unit Testing is always mandatory (and not important) when one creates an interface that has to stay stable and that has a long lifecycle. So typical targets for unit tests for me are things like web services, business components, ejbs and all kinds of transport objects (unit tests for copy constructors, equals operations etc.).
For high-level components, unit tests do not make much sense to me. Regression tests are much more effective. The problem with unit tests is that they don't scale well. If you have a very complex component like a service, it's just not feasible to write unit tests that will really test the component thoroughly. It's much more efficient and effective to use automated regression testing to verify large amounts of test cases.
Unit tests are good for small parts of a system but they aren't appropriate for high-level holistic testing. Unit testing is about finding and fixing bugs early. They don't prove that a system works properly.
|
|
|
Message #272126
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Do We Need Unit Testing?
I like to use it, it gives me confidence in what I am doing and gives me a platform for replicating defects raised later at the code level and then knowing they are fixed.
However, I agree with some of the comments, testing getters and setters is pointless and high level components are hard if not impossible to test.
So the best balance is to be able to "link" your code to whatever system you're using to record business requirements. When you commit code it flags tests associated with those business requirements as needed to change so that it is easy to identify which tests need to be re-run as part of a regression test, rather than testing everything every time.
That said, I haven't found a suite of software which does this effectively yet, it is just a dream for me right now. :)
|
|
|
Message #272130
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Do We Need Unit Testing?
What's rarely mentioned with unit testing is that a TDD approach gives you better code.
This is perhaps the biggest win from unit testing / TDD.
It's generally harder to test code that wasn't built within a TDD environment, typically because its embedded in some controller or service.
I get "better objects" with TDD, which perhaps contributes as much to defect reduction as does the test harness letting me find regression errors.
|
|
|
Message #272139
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit Testing
Unit testing is about finding and fixing bugs early. They don't prove that a system works properly.
Uh, yeah... that's why it's called Unit Testing. If I have a system composed of services, I'm going to have tests that verify that I'm using the services properly. I may use an xUnit framework or something similar, but those are integration tests, not unit tests.
That said, I'd still want each service to have it's own unit tests.
Dave Rooney
Mayford Technologies
|
|
|
Message #272150
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Real reality
I agree with you that the Web could give the false idea that the majority of developers is now happily working with Agile/Rails/Functionnal Programming/Name your Trend of the Day.
The sad thing is that most of the developpers don't follow these trends because they are just too much busy trying to get their late project out of the s**t to achieve some unrealistic deadline and/or scope coverage. Software development life will not change quicker than software management practices. Dilbert anyone? ;o)
|
|
|
Message #272153
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Real reality
The sad thing is that most of the developpers don't follow these trends because they are just too much busy trying to get their late project out of the s**t to achieve some unrealistic deadline and/or scope coverage. Software development life will not change quicker than software management practices. Dilbert anyone? ;o)
Yup - this is indeed true. I'm running an agile transition at a client, and was helping interview some other contractors for a .NET development position. Out of 5 people we interviewed, only one had some experience in a Scrum shop, and none of them had any experience with unit testing tools such as NUnit or Microsoft's TeamTest. Two of them said that they wrote their own test harnesses, but hadn't even heard of NUnit.
I don't think that this is just an issue in the .NET world - back in the summer I met with several Java developers who had no idea what JUnit was.
These people did manual unit testing and they universally felt that writing automated tests would take too long. I can't disagree with them, because writing after-the-fact tests for existing code is a PITA, since the code hasn't been written to be testable, and there is a general lack of true OO skills in the development world. If the tests are written concurrently or the developers are really using TDD, then it's much easier and they actually go faster than they would with manual testing. The code is generally better as well.
Dave Rooney
Mayford Technologies
|
|
|
Message #272194
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Unit testing is good but TDD is not good
I think Unit testing is a good practice, but it becomes bad when people abuse it like the TDD practice.
|
|
|
Message #272196
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit Testing
Unit testing is about finding and fixing bugs early. They don't prove that a system works properly.
Uh, yeah... that's why it's called Unit Testing. If I have a system composed of services, I'm going to have tests that verify that I'm using the services properly. I may use an xUnit framework or something similar, but those are integration tests, not unit tests.
That said, I'd still want each service to have it's own unit tests.
It depends on what kind of service you are talking about. If it's something that aggregates data or operations across many disparate systems, databases, etc. unit testing isn't going to help much. If your service is a utility level service, then unit testing is probably a good idea. A lot of the services I write depend on 3rd party code and data that I don't have access to. Unit testing these services isn't really feasible.
The point is that there is a lot of focus on unit testing but it's not the most important testing that needs to happen. My priorities are on making sure the system works. If there are broken units but the whole system works fine, it's better than if the units work but the system doesn't. Over the long term, I think unit tests save time if they are done properly but they tend to increase costs initially.
|
|
|
Message #272216
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Do We Need Unit Testing?
I like to use it, it gives me confidence in what I am doing and gives me a platform for replicating defects raised later at the code level and then knowing they are fixed.
Sure ,the confidence is very important for me,for everyone.I agree with you ,testing getters and setters is waste your time,unless these method existed logic.
So the best balance is to be able to "link" your code to whatever system you're using to record business requirements. When you commit code it flags tests associated with those business requirements as needed to change so that it is easy to identify which tests need to be re-run as part of a regression test, rather than testing everything every time.
I like your 'best balance',haha,I think we should keep synchronize between business requirements and unit test case.
|
|
|
Message #272217
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Do We Need Unit Testing?
What's rarely mentioned with unit testing is that a TDD approach gives you better code.
This is perhaps the biggest win from unit testing / TDD.
It's generally harder to test code that wasn't built within a TDD environment, typically because its embedded in some controller or service.
I get "better objects" with TDD, which perhaps contributes as much to defect reduction as does the test harness letting me find regression errors.
Yes,better code is always best.The unit testing and refactoring is very important for every developer even though your don't develop with TDD.It's just more powerful with TDD.
|
|
|
Message #272218
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Real reality
The sad thing is that most of the developpers don't follow these trends because they are just too much busy trying to get their late project out of the s**t to achieve some unrealistic deadline and/or scope coverage. Software development life will not change quicker than software management practices. Dilbert anyone? ;o)
Yup - this is indeed true. I'm running an agile transition at a client, and was helping interview some other contractors for a .NET development position. Out of 5 people we interviewed, only one had some experience in a Scrum shop, and none of them had any experience with unit testing tools such as NUnit or Microsoft's TeamTest. Two of them said that they wrote their own test harnesses, but hadn't even heard of NUnit.
I don't think that this is just an issue in the .NET world - back in the summer I met with several Java developers who had no idea what JUnit was.
These people did manual unit testing and they universally felt that writing automated tests would take too long. I can't disagree with them, because writing after-the-fact tests for existing code is a PITA, since the code hasn't been written to be testable, and there is a general lack of true OO skills in the development world.
It's not unusual.One of my colleague with ten years work experience don't know junit,he think writibg test case is waste time.
Exactly,Most of company don't write unit test.A few days ago,when I talk about the TDD with one interviewer in my interviewing,The interviewer even thought my development is no design,he reqire the design document(class diagram,sequence diagran etc). The result is I haven't any design experience.It's very disappointment.
Besides,some developer write unit test case is just to 'write test case',they just want to tell someone 'look, I had test case.',Usually such test case is dangerous and make no sense.
IMHO,the agile developer reqired more strong design and development ability than no-agile developer.
|
|
|
Message #272229
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit testing is good but TDD is not good
I think Unit testing is a good practice, but it becomes bad when people abuse it like the TDD practice.
Could you expand on that a bit? How do you feel that TDD is an abuse of testing?
Perhaps part of the issue is that TDD is something of a misnomer. Rather than a testing exercise, TDD is a design and specification activity which uses executable tests. That's not to say you don't do some up-front design, but you do only enough to be able to move forward (which is going to be different for each person/team).
If you're uncomfortable with the 'T' in TDD, have a look at Behaviour-Driven Development (BDD).
Dave Rooney
Mayford Technologies
|
|
|
Message #272236
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit Testing
The point is that there is a lot of focus on unit testing but it's not the most important testing that needs to happen.
I agree wholeheartedly that there are different kinds of tests that are equally important - integration, system, user acceptance, exploratory, etc.
I worked with a team once that was new to automated testing with JUnit. Initially, the tests were a mix of unit and integration tests because the team hadn't learned how to isolate their units all that well. As they improved their isolation skills, defects started to creep in because there was a "hole" at the integration level. So, they wrote automated integration tests as well, and began reducing defects again.
Did it take time to do this? Absolutely. Was it worth the cost? You bet. There were parts of that system that would have been virtually impossible to maintain without automated tests owing to the complexity of the business rules.
My priorities are on making sure the system works.
Sure - so are mine.
If there are broken units but the whole system works fine, it's better than if the units work but the system doesn't.
Do you have some examples of where unit tests would be broken but the system still works? Maybe I'm seeing that statement from a different perspective - my unit tests are small enough and fast enough to execute the entire suite in 60 seconds or less. As such, it isn't a big deal to run them quite often, which in turn means that I'm immediately fixing any issues that break the tests (either in the production code or in the tests themselves).
Over the long term, I think unit tests save time if they are done properly but they tend to increase costs initially.
Sure. Testing is an investment. Unit testing in particular is a "get rich slow" scheme.
Dave Rooney
Mayford Technologies
|
|
|
Message #272237
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit testing is good but TDD is not good
Rather than a testing exercise, TDD is a design and specification activity which uses executable tests. That's not to say you don't do some up-front design, but you do only enough to be able to move forward (which is going to be different for each person/team).
Beautiful,great,sharp describing. I think the decription is very suitable with TDD.
|
|
|
Message #272238
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Do We Need Unit Testing?
What's rarely mentioned with unit testing is that a TDD approach gives you better code.
This is perhaps the biggest win from unit testing / TDD.
+1.
It's generally harder to test code that wasn't built within a TDD environment, typically because its embedded in some controller or service.
Yes. Read Michael Feathers' Working Effectively with Legacy Code (sometimes referred to as WELC).
I get "better objects" with TDD, which perhaps contributes as much to defect reduction as does the test harness letting me find regression errors.
Yes, again. After Kent Beck coined the term TDD, he and the C3 team realized that the tests were actually a side effect of what was really a specification and design activity. Behaviour-Driven Development has taken this to the next level, explicitly acknowledging that.
Dave Rooney
Mayford Technologies
|
|
|
Message #272240
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Do We Need Unit Testing?
So the best balance is to be able to "link" your code to whatever system you're using to record business requirements. When you commit code it flags tests associated with those business requirements as needed to change so that it is easy to identify which tests need to be re-run as part of a regression test, rather than testing everything every time.
That said, I haven't found a suite of software which does this effectively yet, it is just a dream for me right now. :)
Have you looked at tools such as CruiseControl? It has hooks into several different source control systems, and I've used it to automatically pull from the repo, build the whole system and execute all automated tests when a commit is detected.
The Wikipedia entry for Continuous Integration lists a number of free and commercial tools that do this.
Dave Rooney
Mayford Technologies
|
|
|
Message #272261
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
What do the data tell us?
Unit testing is not performed: 17% (13%)
Unit testing is informal: 40% (46%)
So ... almost 60% of the community polled finds that no or 'informal' unit testing is sufficient.
I think that answers the original question fairly clearly, does it not? Regardless of our views on the matter, most professionals are successful with a lower level of unit testing than unit testing enthusiasts advocate.
There's a discrepancy here that we need to understand. The data suggest that we talking about unit testing past a point where most professionals care. Why would we do this? Are our beliefs that quality is improved in this manner incorrect? If not, is the improvement in quality achieved not worth the effort for most professionals? Are we doing what critics often accuse IT people of doing - trying to improve quality for the sake of improving quality, rather than focusing on achieving acceptable quality for the sake of business goals? Or are most people achieving acceptable levels of quality using entirely different methods? Or ... what?
I'm not sure. I am sure, though, that we have long since passed the time when unit testing was an arcane concept. People not doing this now are making a decision, not operating in ignorance. It seems to me that true professionalism, at some point, requires us to accept the lessons of practice - is the lesson here that we are simply wrong to advocate unit testing past an informal level, for most projects?
|
|
|
Message #272266
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: What do the data tell us?
The data suggest that we talking about unit testing past a point where most professionals care.
Once upon a time, surgeons felt that washing their hands prior to operating was a waste of time.
Why would we do this? Are our beliefs that quality is improved in this manner incorrect? If not, is the improvement in quality achieved not worth the effort for most professionals?
Once upon a time, physicians practiced bloodletting as a therapy (George Washington died because of it). It took 150 years before the practice was completely discredited and abandoned.
Are we doing what critics often accuse IT people of doing - trying to improve quality for the sake of improving quality, rather than focusing on achieving acceptable quality for the sake of business goals?
It's entirely possible to focus on both. It's called Agile Software Development.
Or are most people achieving acceptable levels of quality using entirely different methods?
Perhaps on very small systems. Small systems have a nasty habit of growing into large systems, and manual unit testing does not scale well at all.
I am sure, though, that we have long since passed the time when unit testing was an arcane concept. People not doing this now are making a decision, not operating in ignorance. It seems to me that true professionalism, at some point, requires us to accept the lessons of practice - is the lesson here that we are simply wrong to advocate unit testing past an informal level, for most projects?
If the practice is not correct, such as bloodletting, then it's lessons are teaching us much. Bloodletting must have worked (or appeared to work) at least once, otherwise it never would have become such a widespread practice.
Manual or informal unit testing exists because many people don't have experience with unit testing frameworks, or don't know what to do when confronted with a Big Ball of Mud system. We have answers for those things now, although some people seem to feel that they don't can't invest enough time to achieve their benefits.
They do so at their peril, because there will come a time when a critical mass is reached and businesses realize the cost benefits of higher quality and shorter time to market. This is already happening at Microsoft, Adobe, Symantec and countless smaller companies.
Dave Rooney
Mayford Technologies
|
|
|
Message #272268
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Successfull ???
I think that answers the original question fairly clearly, does it not? Regardless of our views on the matter, most professionals are successful with a lower level of unit testing than unit testing enthusiasts advocate.
Who says they are successfull? I dont know where you come from but I still see a high failure rate in IT projects. And I even think there might be a connection between the results from this poll and failure rates of IT projects. But I will leave that up to the academics to investigate that.
And as stated before there is more to unit testing than quality. It is also a way of design. And often for me unit testing is the most productive way of verifying that things work fine because a junit test runs very fast and I don't have to rebuild and restart my server a thousand times a day.
|
|
|
Message #272269
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
?
All this boils down to is the idea that people are working in ignorance. If that's the case, it makes sense - but I don't think that is the case. I think that businesses are making money doing it this way.
Walking through the examples - sure, surgeons did not have the germ theory and did not wash their hands. When data showed better results achieved with sterilization, surgeons adopted the practice. This took about a generation.
Bloodletting was practiced, just like lots of other kinds of magic, not because data showed that it worked, but because nothing known at the time worked any better. Again, when data showed other techniques worked better than bloodletting, we adopted those techniques. This took a while, partly due to the poor state of information transfer, partly due to the interference of people with other interests, religious in the case of sterilization.
(It's not known why Washington died, of course. Before he summoned the doctor, he had already expressed his belief that his illness would be fatal - and he was right. Bloodletting is still practiced today in addition to newer techniques.)
In our situation, my view is that it's been a while, we don't have poor information transfer, and we don't have opposing interests. What we do have is lots of data showing that businesses make money without unit tests at the level advocated by enthusiasts - and so, low adoption of other ideas.
This reminds me a little bit of the statements I've often heard that "waterfall doesn't work" - referring to the much-maligned waterfall software development process. In fact, the waterfall process DOES work, and was used to produce many of the banking systems, for instance, upon which we depend for our daily activities.
Similarly, statements about the "need" for unit testing at level XYZ strike me as excessively categorical. My own feeling is that different projects require different strategies and that we will ultimately arrive at an acceptance of the idea that pros will choose from a bag full of alternatives rather than argue about which size fits all.
|
|
|
Message #272271
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Success ...
Who says they are successfull?
Well, Joris, if I understand the data, "they" do! Their businesses continue make money, they continue to get paid ... that's success for most people. Part of the point I am making is that people would change if they were not successful.
Maybe your question reveals the true issue though: do we so strongly want to define the meaning of acceptable quality that we lose track that our definitions may be meaningless for others?
FWIW, I personally agree with an approach that has developers designing tests before writing code. My personal view is that testing code is pretty far down the QA stream and a very expensive way of improving quality - a low-leverage activity. I think that more attention to QA at the management, requirements specification and design levels would be much more cost-effective. That does not obviate the need for unit testing, of course, but I do think it is a more fruitful area for improvement in our profession right now.
I very much agree that this approach affects and in some cases improves design. Again I think this can be carried too far: achieving testability for instance has to be balanced with other goals, such as hitting the market window.
|
|
|
Message #272275
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Waterfall
This reminds me a little bit of the statements I've often heard that "waterfall doesn't work" - referring to the much-maligned waterfall software development process. In fact, the waterfall process DOES work, and was used to produce many of the banking systems, for instance, upon which we depend for our daily activities.
Have you ever read Winston Royce's paper Managing the Development of Large Computer Systems from the 1970 IEEE Wescon conference? It's considered the 'definition' of waterfall, based mainly on the diagram at the top of page 2. In the first sentence after that diagram, Dr. Royce explains, "I believe in this in concept, but the implementation described above is risky and invites failure". He goes on to describe how, in reality, iterative feedback is required to have any success. See for yourself here.
Consider for a moment the environment in which Dr. Royce wrote this paper. He was working within the U.S. Department of Defense and IBM's Federal Systems Division. The procurement and contracting model alone put handcuffs on any development group.
Also take into consideration the technology of the time. Both Eclipse and NetBeans have a setting for the delay time between detecting a syntax error in the code and displaying the squiggly red line under the error, typically 400ms. In 1970 you were lucky if your batch submission for a compile came back in two hours. As a result, you went over the code prior to submission with a fine toothed comb. You did detailed design after similarly detailed analysis and requirements gathering. You had to because it was very expensive to change the code. As for automated unit tests, forget it!!
In 2008, computing power is ridiculously cheap. Even if you have a grid-based system, you can spark up a pile of Amazon EC2 instances and run your tests for a couple of dollars per hour for even an enormous grid.
If Winston Royce were to write that paper today, would it be the same? I seriously doubt it.
As for the banking industry, companies like Capital One, CitiGroup, BMO in Canada, DeutcheBank and others are using agile development in a large way. Yes they've shipped systems in the past using waterfall, but they realize the benefits of a different approach.
Now, back to your initial point:
All this boils down to is the idea that people are working in ignorance. If that's the case, it makes sense - but I don't think that is the case. I think that businesses are making money doing it this way.
They are, but companies are now realizing that using iterative methods with rigorous automated unit tests gives them an advantage over competitors that don't. In the end, that investment in unit testing will make them more money than they would have without them.
Dave Rooney
Mayford Technologies
|
|
|
Message #272276
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Success ...
My personal view is that testing code is pretty far down the QA stream and a very expensive way of improving quality - a low-leverage activity.
Testing code is not a QA activity, it's a development activity. Testing functionality against the business requirements is a QA activity.
Dave Rooney
Mayford Technologies
|
|
|
Message #272278
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Success ...
they continue to get paid ... that's success for most people. I don't think my clients see it this way.Part of the point I am making is that people would change if they were not successful. I see many people that are very reluctant to change even if things are not going that well I tend to disagree on this.Maybe your question reveals the true issue though: do we so strongly want to define the meaning of acceptable quality that we lose track that our definitions may be meaningless for others?
I agree with you if you mean that good is good enough but my awareness about good being good enough grew when I engaged with TDD. Before my code often suffered from what Martin Fowler calls "Speculative generality". Today I try to create a test that reflects a requirement and create production code that only meets that test nothing more.
My personal view is that testing code is pretty far down the QA stream
If you start writing unit tests well after writing your production code I agree with you that will be painfull and expensive (and probably not worth the effort to write a test set with "full" coverage). But if you write and execute unit tests before writing production code how is that far down the (QA) stream? It starts up a feedback loop even before you start coding production code.
And yes a lot can be approved in requirement specification and design. I expect that the percentage of developers doing unit tests will grow ... but time will tell.
|
|
|
Message #272324
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Waterfall
They are, but companies are now realizing that using iterative methods with rigorous automated unit tests gives them an advantage over competitors that don't.
I don't buy that.
"We have more features" is a much more compelling argument for a software product than "We crash less than our competition".
--
Cedric
|
|
|
Message #272345
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Waterfall
I don't buy that.
"We have more features" is a much more compelling argument for a software product than "We crash less than our competition".
"We get more features to you sooner", certainly is a compelling argument. Oh, and as a nice side-effect, they work too.
Dave Rooney
Mayford Technologies
|
|
|
Message #272346
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Waterfall
"We get more features to you sooner", certainly is a compelling argument.
Definitely, and that was my point: faced with the choice of writing tests *or* adding a feature, most software companies will always go for the latter.
--
Cedric
|
|
|
Message #272360
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Waterfall
"We get more features to you sooner", certainly is a compelling argument.
Definitely, and that was my point: faced with the choice of writing tests *or* adding a feature, most software companies will always go for the latter.
--
Cedric
No, I mean that having the tests makes you go faster. Maybe not in the first couple of weeks, but over time you gain back any time lost. If you practice true TDD, the break-even point comes sooner.
Dave Rooney
Mayford Technologies
|
|
|
Message #272367
Post reply
Post reply
Post reply
Go to top
Go to top
Go to top
|
|
Re: Unit Testing
Do you have some examples of where unit tests would be broken but the system still works? Maybe I'm seeing that statement from a different perspective - my unit tests are small enough and fast enough to execute the entire suite in 60 seconds or less. As such, it isn't a big deal to run them quite often, which in turn means that I'm immediately fixing any issues that break the tests (either in the production code or in the tests themselves).
I think I can explain it without examples but I'll provide an example. It's quite common for a unit or code to fail under in certain cases for which it's behavior was defined but is never used in a working system. When I start exploring old production code, I regularly find bugs / issues. I can see that given a certain valid input, the code will fall over or otherwise produce incorrect results. This same code may have been running in production for years without issue. I call this working by accident.
Another thing I've seen which I find much more interesting but is very rare is when two units that work together have flaws that cancel each other out. Another form of working by accident but a much more comical one.
One huge example of this was given by Josh Bloch in this article: http://googleresearch.blogspot.com/2006/06/extra-extra-read-all-about-it-nearly.html. Many working systems have been operating perfectly fine with this flaw.
Optimally, the units should not have any issues. Even if the conditions that create the issue have not existed yet, things change and suddenly code that was considered bug free could be failing. This will resulting is lots of flailing and chaos. But ultimately, the present has priority over possible future issues, especially since Microsoft's 'get it out the door' approach allowed it to crush so much of it's competition.
I think at a basic level, we agree about unit tests. I think that the approach you are espousing is one good approach to software development. But I think it also explains the reason why unit testing is still the exception: the people driving the deadlines are not looking for the slow-and-steady approach. They want fast and cheap. Like many things in this world, it's short-sighted and (I think) the wrong choice.
One of the things I'm trying to explain to my management is that if we accept a bit of overhead up-front, we can get our development to be much more steady and predictable. Right now they are enamoured with silver bullet tools and approaches that make development fast at first but make development more difficult and costly over time. It's a hard thing to sell. It takes a brave manager to reject what is commonly believed by their peers.
|
|
|
| New content on TheServerSide.comNew content on TheServerSide.comNew content on TheServerSide.com |
|
|
Reza Rahman explores the features of the proposed JSR 299, Contexts and Dependency Injection for Java EE (CDI). When approved, it promises to be a key feature of Java EE 6.
(November 2, Article)
SAML is an XML-based standard for exchanging authentication and authorization data between security domains. The single most important problem that SAML was created to solve is the Web browser Single Sign-On problem. Many organizations are debating whether to stay with version 1.1 or move to 2.0. This article makes observations about both options.
(September 28, Article)
Joe Ottinger takes a look at how people learn, and applies it to the practice of programming. He notes that understanding how people learn is an essential part of working in a programming team.
(September 22, Article)
Stephen Maryka gave us an article about the Asynchronous Web and posed a number of questions that get examined like an approach to delivering Asynchronous Web capabilities through extensions to existing Java EE technologies.
(July 14, Article)
JavaServer Faces Flex goal is to provide users capability in creating standard Flex components, part of flexSDK which is open sourced through MPL license, as normal JSF components. This article by Ji Hoon Kim will provide an overview of creating a simple multilingual JSF page consisting of JSF Flex tags.
(June 29, Article)
In this session Jeff explores the key characteristics of successful SOA projects. He covers some of the patterns, and anti-patterns, tool sets, and strategies that he himself learned the hard way. Last, he provides a strategy and blueprint for achieving a high likelihood of success in your SOA project.
(June 23, Tech Talk)
Ari Zilka, CTO of Terracotta, Inc., talks about the new features in Terracotta 3.1, announced during JavaOne and available now.
(June 15, Tech Talk)
In this Tech Talk, Josh Long explores an integration challenge using Spring Integration and walks through the implementation, employing and expanding on the basic patterns of Enterprise Application Integration to tie together components into a function integration solution, and then demonstrates how Spring Integration helps address the integration requirements.
(June 15, Tech Talk)
In this Tech Talk, David Geary teaches you: The basics of Google Web Toolkit; How to implement Ajax-enabled applications in Java; Internationalization; Hooking into the browser history mechanism; Remote procedure calls.
(June 4, Tech Talk)
Jon Kern discusses the best architecture/technical solutions and ensure that they are repeated by all developers. By tackling the architecture up-front in a serial manner, subsequent parallel development will be much more manageable and predictable.
(May 28, Tech Talk)
This keynote describes the frustrations of modern knowledge workers in their quest to actually get some work done, and solutions for how to guard yourself against all those distractions. Neal Ford talks about environments, coding, acceleration, automation, and avoiding repetition as ways to defeat the misguided attempts to sap your ability to produce good work.
(May 26, Tech Talk)
Gil demonstrates how new, aggressive uses of already abundant compute capacity by common applications offer competitive value for application designers.
(May 21, Tech Talk)
Chris Keene introduces WaveMaker as a new way to automate the ability to generate Hibernate classes in order to more quickly bring OR mapping into an application.
(May 19, Article)
In this session Nati Shalom demonstrates how to take a standard Java EE web application and scale it out or down dynamically without changes to the application code. Seeing as most web applications are over-provisioned to meet infrequent peak loads, this is a dramatic change because it enables growing your application as needed, when needed, without paying for unutilized resources.
(May 19, Tech Talk)
Mastering EJB was one of the original and most influential EJB books in the industry. Mastering EJB III now returns with two new expert co-authors, updated for EJB 2.1 and 30% new chapters including security, integration, best practices, open source, and more.
(Book PDF Download)
The Application Server Matrix is a detailed listing of J2EE vendors and their application server products, with information on latest version numbers, J2EE spec support and licensing, pricing, platform support, and links to product downloads and reviews.
(Application Server Comparison Matrix)
|
|
