Discussions

News: Opinion: Don't Let Yourself Get Unitized

  1. Opinion: Don't Let Yourself Get Unitized (46 messages)

    Mike Spille has provided his thoughts on the state of unit testing in the Java world today, and comments on recent articles on the topic by Martin Fowler and Cedric Beust. He thinks that too much emphasis is being placed on unit testing and advises that we refocus our energies on more important issues such as design, integration testing, performance and failure/recovery testing, etc.

    Mike Spille on the Unit Testing Hype
    "Now that I've been exposed even more heavily in recent weeks to comprehensive unit testing, my original opinions on the subject remains unchanged. Unit tests are a nice little tool in software development, but only a small one. Yeah, they do good things, and you should use 'em because they're so easy to write and do provide value. But don't fall into the obsession that the Thought leaders are pushing. The fact is that Unit tests solve a very small set of problems, and are only a tiny piece of the software development puzzle. By all means embrace the practice, but for God's sake don't make it the central practice in your projects. Focus on unit testing with the mania that a Fowler or Eckel do, and you too will end up with a toy app that breaks under any serious load. Keep your priorities straight, and allocate your precious time in an intelligent manner. By all means write those unit tests, but if you're smart you'll spend significantly more time on things that have a much larger impact - design, integration testing, performance and failure/recovery testing, writing code that solves your customers problems. Don't obsess on the easy low hanging fruit like unit tests, and let the truly difficult problems hit you over the head later in your development cycle like a fast ball whizzing from a professional baseball pitcher. And if yet another Consultant or Thought Leader gets in your face telling you that Unit Tests are the keys to success, don't waste your money on them - just politely show 'em the door."
    Read Don't Let Yourself Get Unitized

    Threaded Messages (46)

  2. Don't Let Yourself Get Unitized[ Go to top ]

    Well, I suppose you have a point. In my experience however having unit tests and placing emphasis on ensuring that code is unit tested does more than just test the code.

    Writing tests helps me organise my thoughts a little. I'm no uber programmer, I need to think hard before i jump into implementation. Writing a bunch of unit tests helps me thnk about the ways i want to use my classes, which helps me develop good interfaces. That has a added benefit of making my implementations less coupled, so i can do more at the unit testing stage when testing is relatively easy, rather than having to set up complicated contexts to run my tests in.

    Also, I'm not so scared of refactoring. I've worked on projects that were so complicated even the guy that originally wrote it was scared to change stuff. Knowing that you can run your unit tests after a change removes that fear a little.

    And, you can decorate you unit tests with things like JPerf. So you can do some of performance stuff at the unit test level.

    Finally, I use cruisecontrol to automate unit testing, and simple integration testing, as well as running unit tests through my IDE. Its great because at the end of most days, i don't have to worry that my current build doesn't work, and when things do go wrong in a team environment its pretty much immediately clear why. This helps avoid the week or two of integration hell that i've encountered on some projects.

    So, I take your point. But I think that I'd much rather see Martin Fowler et al, being strong advocates of this way of working just because the benefits of doing so make the whole development process a lot easier to manage. Integration while obviously extremely important, should be derisked, like every other part of the development process, and the more you can do up front the better....

    N.
  3. I find it a bit hard to digest that someone who has a number of years experience in the industry would go ahead and discount the utility of getting "Unitized" so easily.

    I find the comments on JUnit itself to reflect the inherent confusion of Mike - "JUnit is constraining", "JUnit is simplistic", "JUnit offers developers nothing they couldn't bang together themselves in a day"

    Wouldn't you agree that the very intention of a "framework" should be simplicity? And allowing the users flexibility to add, modify, extend it as desired?

    The endless rant in PART 2 about the "limits of user testing" contradict what Mike has written in part 1 of the article. The talk about not being able to do end-to-end tests for e.g. only reflect an lack of imagination with regards to testing.

    Anyone who has delivered software [I have intentionally used the word delivered rather than developed] would know that integration testing, user testing, performance testing et all have their respective parts to play and discounting them would only be at your own peril.

    But this in no way diminishes the role of unit testing. The amount of technical debt that a project incurs over its lifetime is definitely kept in check by incurring the overhead [if thats what you would like to term it] of unitizing from the very beginning.
  4. Loved it[ Go to top ]

    What a great blog - made me laugh, and its anti mantra.

    The great thing about senior guys is they have been here before. When he says JUnit is a poor tool he is speaking about the heard mentality of java, and he then presents a point of view - from someone else, that he thinks is better.

    How many posts here do that? Most say 'I know the best, me me me, I'm right'. What Mike said is, 'lots of this junit focus is rubbish - take a look at him over there, he has a good idea'.

    Also, his 'personal' attack is not that at all. He is saying watch out, a consultant has an agenda - bigging himself up and making money. I like Martin's posts - but I take them with a pinch of salt, and thats what Mike is saying.

    He does not go into loads of detail, defense of idiology or software theory. Why? because to him it's obvious and he is NOT looking to make money as a consultant. Unlike most of the other 'thought leader' posts here. He is just saying it as he sees it. It is also very very healthy that people do take a poke at thought leaders. Keeps em on their toes.

    So, thanks for the laugh, and the perspective.

    Jonathan
  5. In my view, unit tests do indeed have a value, however it's quite limited.

    Every Java class has certain metric - how must 'external state' does it need. Let's take a proprietary format parser class that takes a string in and populates its fields with values taken from this string. It needs zero 'external state'. It's a pleasure to write unit tests for that sort classes: just feed in some constant and add a bunch of assert() on getXXX() methods.

    However, most of the classes are not that easy. They do need an external state, and that state could be as complicated as hundreds of related objects. An example - business 'action' processing handler, working on the current 'session' of the user. The 'action' itself is just a half-screen of code, and testing it could be well three to ten lines of actual unit test code. But creating pre-defined external state, 'session', could be very complicated - hundred lines of ugly a = factory.createAAA() and a.setXXX() a.setYYY() a.setZZZ(). And still it doesn't prove much, because the 'artificial' way of creating that external state has almost nothing to do with 'normal' way.

    Therefore, I apply unit tests only when it is feasible to isolate a class or a group of classes into more or less realistic environment.

    Yet another point - to facilitate development, tests must be _automated_. Whether they are unit tests or whatever-else-tests - the automation is the key.

    just my EUR 0,02
  6. oh yeah[ Go to top ]

    yeah bro' preach it dude. Too much time gets spent on the latest "thing" and the most basic of software engineering principals go right out the window architecture, great design, system testing, user acceptance testing, proper test scripts, bounds testing, performance, concurrency testing because management has latched on to "automated unit testing".

    "so has it been unit tested?" .... "oh yeah we did Junit tests for it" .... "great lets release it next week"

    or just as bad, unit testing to death every little detail and test writing ends up taking up 2/3 development time. ive seen it

    Lets get back to just developing quality software in a timely fashion.. please..
  7. here here![ Go to top ]

    For development that happens in the corporate world, I can understand why managers love the unit test mantra. They are stuck with third rate "coders" who don't care about the success or failure of the project. Consequently, enter the unit test. Perhaps it's the only way to enforce quality output from individuals not ordinarily geared to producing quality.

    You can't argue against the value of unit testing. It's simply that the enforcement of unnatural levels of unit testing, at the expense of integration testing, restricts the flexibility of the initial design. When the unit test "cram down" comes too early in the design, it actually limits the flexibility that a developer has, because he starts measuring the value of his code by the number of unit tests he has. So great, we've got hordes of unit tests that prove that a crappy design "works". But if you spend too much time proving the crappy design works, you never move off the crappy design.

    For experienced developers, the unit test "cram down" is the same sort of b!#ch-slap that "pair programming" was. These methodologies work best for novice or mediocre programmers who don't produce successful designs without being stuffed into a manager-driven framework. Unfortunately, this is too often the reality of software development environments. So it's only natural that you find top notch guys, like Mike, scratching their heads, because uber levels of unit test just doesn't make sense to them.
  8. MS back?[ Go to top ]

    Mike asked not to be linked from TSS.

    I will be very (very-very) glad to see Mike's return, but
    unless something changed, this link is
    unethical of TSS editors...

    Alex V.
  9. MS back?[ Go to top ]

    +1
  10. MS back?[ Go to top ]

    +1
    Nitin, this blog linking may be a bad idea. People who are interested would subscribe to the blogger directly. Especially if it's true that he requested he does not to be involved with TSS.
  11. on mike[ Go to top ]

    Mike asked not to be linked from TSS.

    I will be very (very-very) glad to see Mike's return, but
    unless something changed, this link is
    unethical of TSS editors...
    The Internet is a public place. If you post something on the Internet, how can you ask that someone not link to it?

    On the other comments of Mike flame-baiting or whatever, Mike may have an overly-enunciated contrarian view at times, but he doesn't talk out of his ass. His comments are well constructed, his ideas well thought out, his specifics are (insanely) well researched. He doesn't require anyone to agree with him, he just speaks his mind. I have always found his opinions to be valuable and well researched, even when I disagree with him.

    His comments on Martin Fowler are humorous, and I believe that they were intended as such. We could all use an occasional dose of humor, and some smaller egos. ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  12. on mike[ Go to top ]

    Most people who post on here don't possess a sense of humour, they think they do but they don't get irony.

    As for all those people who defend JUnit, get a life and get a professional tool.
  13. The Heroic Age is over[ Go to top ]

    For experienced developers, the unit test "cram down" is the same sort of b!#ch-slap that "pair programming" was. These methodologies work best for novice or mediocre programmers who don't produce successful designs without being stuffed into a manager-driven framework.
    Managers are not universally short sighted. Some times they actually see a bigger picture than the developers.

    As a talented senior developer, part of the value you provide to your employer is as a mentor to the novice, mediocre programmers. While you may reasonably believe that pairing is not the best way to mentor these poor sots, for a number of documented teams it has worked well.

    With pairing at least two people will understand what a given batch of code does. It does a TEAM no good if the design is the most subtle, flexible, loosely-coupled, robust and secure design possible, if only the hero-programmer understands how it works.

    In Lake Wobegon every project is staffed with above average developers, buty everywhere else, managers often must work with what they have.
  14. The Heroic Age is over[ Go to top ]

    In Lake Wobegon every project is staffed with above average developers, buty everywhere else, managers often must work with what they have.
    Are you by chance from Minnesooota? :)
  15. The Heroic Age is over[ Go to top ]

    With pairing at least two people will understand what a given batch of code does.
    Very brave assumption, especially because you imply that mediocre programmers is the all manager has. That understanding (if any) will not last long….
    for a number of documented teams it has worked well
    Well for what? I would speculate that it worked especially well on paper and from bureaucracy point of view. I am not saying that teaching and learning are not important: I agree with Mike that “pair” programming is a very-very stupid and inefficient practice: if a person can grasp a concept then he/she can do it by way of looking at code an discussing it with author and others, if a soul is incapable, then no amount of mentoring will do any good.

    And I am in complete agreement with Mike on Unit testing: reliance on it, especially in a form of Test Driven Development is simply impractical for many projects. It looks for me more like a way to give job to max possible number of developers with questionable technical benefits (overhead is comparable to RUP).

    Testing is necessary, but good design and adequate tools sometimes nearly eliminate need for Unit tests. Few high-level acceptance tests are just what doctor recommended.
     
    It does a TEAM no good if the design is the most subtle, flexible, loosely-coupled, robust and secure design possible, if only the hero-programmer understands how it works.
    Let stop call programmers who understand the profession: hero-programmer, they are simply professionals. That will put things in the right perspective: hero-programmers are pretty often are mediocre developers and differ from rest of dummies by amount of enthusiasm.
    Workings long hours Hero-programmers and entire departments of pairing monkeys are simply not capable of producing results remotely comparable with work small professional team.
  16. Too personal?[ Go to top ]

    Maybe I'm a "glass half full kind of guy" but the comments on Martin Fowlers motivations I felt to be a bit unjustified. Looking at people like Richard Stallman who have no monetary motivations what so ever, I'd say key figures in the industry are mostly interesting in promoting best practice and a happy world (or at least some honourable ideals :). What about the author himself, has he got monetary motivations or is he keen to share something that will help others?

    I'd say Martin et al are helping move the industry forward, yeah, they may be making money but without people like this, we wouldn't be going anywhere. You don't *have* to buy into everything they say and hey, everyone's gotta eat.

    Have a think about who their audience really are; people making budget discussions and paying for their services or techies on the front line, wanting to do a better job. I don't know which it is but I certainly don't hire any consultants.
  17. The Heroic Age is over[ Go to top ]

    For experienced developers, the unit test "cram down" is the same sort of b!#ch-slap that "pair programming" was. These methodologies work best for novice or mediocre programmers who don't produce successful designs without being stuffed into a manager-driven framework.
    Managers are not universally short sighted. Some times they actually see a bigger picture than the developers.As a talented senior developer, part of the value you provide to your employer is as a mentor to the novice, mediocre programmers.
    Great.I agree.
    My thoughts:How many managers are mentors,by thier actions?
                How many knowledgeable developers want to be mentors,by thier actions?
    a dichotomy? ..solution is JUnit...
  18. managers wanting quality!!![ Go to top ]

    Third rate managers too? I have known many managers who care less about the project than the developers. Or were unable to motivate. Or who de-motaivated through crap political decisions.

    Testing has to come first or all the rest is just bollocks.
    Geoff is absolutely right that you cannot argue against testing.
    Although I know many managers who just love to cut the test time from the project plan.
    For development that happens in the corporate world, I can understand why managers love the unit test mantra. They are stuck with third rate "coders" who don't care about the success or failure of the project. Consequently, enter the unit test. Perhaps it's the only way to enforce quality output from individuals not ordinarily geared to producing quality. You can't argue against the value of unit testing.
  19. Rebuttal[ Go to top ]

    I would have to disagree with the article.

    One of the things that Mike Spille states in his article was that anyone can write a better test framework in a day. It may be true, but when you are concentrating on developing your application, you shouldn't be wasting your time building another framework and use most of your time developing your own application.

    Its basically the same concept as using the jakarta commons libraries, most of the things in there we can do ourselves, but the question would be why bother re-writing the piece of code over and over again. And if you try to justify saying that you'd want your own private library with little dependencies, you have to ensure that you maintain those small pieces of code over and over again rather than taking advantage of what is already out there.

    Using JUnit is quite simplistic I have to agree. But would you rather test your small routine with the entire application all at once or just write a simple test case. Sure you can just create a main() method or separate class to test it out, but writing a JUnit test is similiar to writing the main() method anyway only there is some IDE and tool support for the tool so you can take advantage of the reports it would generate.

    Assertions in JUnit actually prevent you from having to look at your test run once you decide what the correct output is supposed to be. I cheat and sometimes use printlns to display the result and copy and paste it into the assertions, because the result is sometimes too tedious to type.

    JUnit being lightweight is not a hinderance, its a bonus for it. Being lightweight allows it to be easily embedded into other tools such as JPerf, Eclipse, Ant and Maven. Its also easy to prove that it just works because it does so little.

    Of course I wouldn't use JUnit alone when I do integration level testing (that's just nuts at times since its going to make my test case longer than it has to be). I'd use extensions to JUnit such as HttpUnit, StrutsTestCase, JFCUnit and Cactus. Tools like HttpUnit (and a bit of refactored code to reduce the code base I maintain) allow me to translate Use Cases to test code to verify that things flow as expected (of course that would take a bit more experience to do since it is more complicated). And integration level tools allow me to regression test cheaply.

    I am not as ridiculous and run all unit tests all the time on my development PC, that's just nuts. I wait till the nightly builds execute and run the test cases for me and let the build manager assign blame appropriately.

    As another person said on the thread, unit tests also aid in refactoring. I usually some deadlines are hard and I just write whatever to get the thing working. The code I write may not be good and/or it might just be a 1000 line class, but it works. I don't write my JUnit test cases then just make sure that the demo works. Once I do get that working, I just add the unit test case afterwards, something minimal where I just prove that a boundary value works properly. Then I go on to another component. Usually, I will come back to that piece of code later, the most common reason being, I have to cut and paste come code from it, at which point I do a bit of refactoring to reduce the code base (I want to reduce the active code base as much as possible, because its harder for new people to pick up if the code base is too big).

    And to rebut the other thread where someone said that unit tests take 2/3s the time to make. I've been on application maintenence group, most developers who haven't gone through that probably just write code to get the customer happy and pay, application maintenance teams have to make sure the bloody thing runs everytime and figure out why something goes wrong. Working on application maintenance does build up sufficient skills, wisdom and maturity to find ways of avoiding code problems before it gets to them.

    Automated unit tests are a useful tool when you have teams who have to make sure that the code works because it reduces the costs for them to fix any bugs that the development teams make, by reducing regression costs. Of course like any part in application development, they have to be done properly.

    To finish this reply, here are some rules which I personally have trouble myself from doing when I write test cases. (in order of difficulty)

    1. make your tests throw exceptions (don't bother catching them unless you need to)
    2. write at least one sentence on what you are testing
    3. refactor your tests, a lot of tests can get common setup and teardown routines
    4. put unit tests reports as part of your nightly build cycle.
    5. turn use cases into test scripts
    6. build data that you are testing on, don't rely on data to be there.
    7. remove data that you are finished testing with if possible.
    8. make things testable without the container

    (I should put this in my blog)
  20. Flame: I wish the author would keep the personnal attacks out of his stuff.

    I believe that unit testing is a matter of simple economics, but this subject has been beaten to death, and I won't rehash it here.

    I'd like to instead point out what makes JUnit useful to me: the community behind it. JUnit has articles, books, IDE plug-ins, reporting, extensions (http, db, even EJB), continuous-integration build tools, and forums. That didn't grow up in a day.

    Someone will doubtlessly build a better JUnit some time. This is progress, and takes nothing away from what Fowler, Beck, and others have preached for years. (Maybe someone already has built something better...but beta was a better video format, too.)
  21. Basic standards[ Go to top ]

    It seems to me that the technical content of this article is totally diluted by the insinuations. Where has the basic editorial vetting gone ? Part one is fully a personal attack. Could we be a bit more respectful ?
  22. I am disappointed to see TSS become a place for flamebait and a hiding place for trolls. Slowly we are descending into Slashdot oblivion.

    With respect to Mike, articles like this should not get a stamp of approval. He may strongly disagree with something but that article is just flamebait; I got bored of reading it about 2/3 through not because Mike didn't have valid points (at times I feel he did) but because I really don't want to listen to someone slagging off someone else.

    Mohan put it more succintly:

    <quote>Could we be a bit more respectful ?</quote>

    With regards
    Neil
  23. Basic standards are decreasing[ Go to top ]

    This posting should have been entitled "Don't Get Fowlerized" since it really has a lot more to do with raising skepticism toward Martin Fowler than saying anything important about Unit testing. Although the author congratulates himself for exposing Martin Fowler's sinister motivations to the world, his own motivations remain murky. (What's worse he actually mis-characterizes Fowler's article in order to substantiate his accusations.) I agree with those who have raised doubts about the value of TTS drawing our attention to this sort of soapbox tirade. After all, how do you refute the "arguments" presented here. "Well, I think unit testing IS all a programmer ever needs to do!" Or more to the point, "Well, I think Martin Fowler is smarter than you are!" Isn't. Is. Isn't. Is...
  24. Missing the point.[ Go to top ]

    So Mike is pissed that ThoughtWorks didn't make him it's Chief Scientist. They should have spelled his last name Spleen.

    NOBODY ever said that Unit Tests were sufficient to ensure quality. Just that units that behave in known and correct ways are required if the wired together whole is ever to have a chance of working.
  25. Oh I love it when these thought leader/guru types take their gloves off and get down and dirty !!

    I agree with some of what Mike said but there was no need to get so personal.

    Unit testing IS important but so is the other stuff - integration testing, functional testing, performance testing, user acceptance testing etc.

    I religiously unit test my code but only so I don't waste precious time during system test and UAT tracking down trivial bugs. For example if I were to build a web based stock price checker. This simple app could use a form to prompt the user for the name of the company the price should be checked for. The form would then pass the company name to a servlet, which would then invoke a web service to fetch the price, before writing out the price. I would then write unit tests would verify that:

    1) Given a particular company the web service is able to return the correct price.
    2) Given a mock object that represents the web service the servlet is able to take the name of a particular company and get back the correct price.
    3) Given a particular price, the servlet is able write out the price correctly.

    These seem pretty comprehensive to me and let us assume that my app passes these unit tests. I now pass my app over to system test and I get back a bug report saying that my app is displaying the wrong stock price. Since I've run the unit tests I've eliminated the trivial possibilities that there could be a problem with each individual component. This now allows me to concentrate on non trivial possibilities such as the firewall between the servlet and web service not correctly handling the SOAP message. If I hadn't written and performed the unit tests then I would now be spending precious time making sure that there wasn't a problem with each individual component, rather than concentrating on the firewall. However if I had released the app straight into production on the strength of it just passing the unit tests then I'd be in trouble, as I hadn't tested how the app works with other components in the production system.

    So for my two cents worth you need to perform both unit testing and the other types of testing. To not unit test wastes time chasing down trivial bugs in system test etc, while releasing code into production on the basis of it just passing unit tests, neglects the possibility that there could be other components present in production that affect your app.
  26. I completely agree with you as far as Unit testing being important together with integration testing, functional testing, etc..

    However, when you say that you "religiously" unit test your code - I don't think that what you refer to as "unit" testing is the same thing that Mike was talking about.

    Your examples:
    1) Given a particular company the web service is able to return the correct price.
    - if the webservice is something you wrote, than this would be a system test, otherwise (if you are communicating with an outside web service over http) it would be something like an integration test

    2) Given a mock object that represents the web service the servlet is able to take the name of a particular company and get back the correct price.
    - if I understood coorectly, this would involve sending an http request to the servlet in order to test it... this would be pretty out of scope for a simple junit test..

    3) Given a particular price, the servlet is able write out the price correctly.
    - I guess I would agree that this fits a unit test scenario, but again - you would most likely need to run the servlet in a container in order to inspect the results..

    Ofcourse I may have completely missunderstood what you were saying, but it sounds to me that the comprehensive tests that you wrote about have nothing at all to do with Mike's blog - as they fall into much more "high level" tests.
    That doesn't mean that they are NOT unit tests - but they also are definitely NOT the types of tests that JUnit supports when used in the way that the "Thought Leaders" propose you use it.

    Mike was saying that it would be a HUGE waste of your development time to do the following:
    1. write testing code in your servlet to take in a Mock http request
    2. write testing code in your servlet to communicate with a Mock web service
    3. write testing code in your servlet to return a Mock http response
    all so that it can be run within the simple JUnit test harness instead of being
    run against a higher level testing framework that would handle the types of scenarios that you've described. This doesn't mean that there aren't tools in the JUnit community available for supporting this - but rather that the way that the main JUnit product tends to be hyped does not fit this bill...

    I think what Mike was mainly saying is that JUnit is too simplistic for what people have made it out to be, and the community is full of hubris - where simplistic scenarios using ridiculously simple tools are paraded as end-all-be-all solutions to your real-world problems.

    regards
    - Gary B
  27. About time[ Go to top ]

    It's about time someone spoke out on this. I don't know how many poorly designed classes I've seen as a result of this. Things like supplying constructors for utility classes that only contains static methods, using init() methods instead of constructors, etc. Basically a lot of design to fit the test, when it should be the other way around.
  28. About time[ Go to top ]

    ... Things like supplying constructors for utility classes that only contains static methods, ...

    One of the points of test driven development is to stop you from writing classes that only contain static methods.
  29. I don't understand how you get so heated about it.

    OK JUnit is not the greatest thing in the world since sliced bread. But none tools are. A tool is a tool. You have to understand its key features / its drawbacks, and use it the best way you can. So write unit tests where they deserve to be written and use other levels of tests where appropriate.

    BTW why did you spend so much time ranting about people and their agendas? How do I know you don't have one yourself when you write this little thing?

    Martin Fowler may have his constraints, and his needs, but I am pretty sure he likes to write software as much as us.

    Enough talked, I go back to code and tests (including but not limited unit tests).

    JL (coffeebreaks)
  30. My 2 cents...[ Go to top ]

    ...of the Unit Tests I have seen developed about 50% test the obvious and 25% test the compiler. Among other things they were able to discover that

    - setX(Y), getX() yields Y on a plain java bean
    - Java Serialization actually works
    - EJBs can perform remote calls
    - Java can add numbers

    and things like that really.....
  31. My 2 cents... Heh.[ Go to top ]

    Karl, you might have missed the idea of unit testing then :)
    Try to write tests that would test your business logic, not compiler.


    PS
    I just don't get the idea of unit test approach neglection because of junit simplicity. What's the connection?
    "Let's not use operational systems because M$ Win sucks." I guess it could be an alternate topic for Mike.
  32. RE: My 2 cents...[ Go to top ]

    ...of the Unit Tests I have seen developed about 50% test the obvious and 25% test the compiler. Among other things they were able to discover that- setX(Y), getX() yields Y on a plain java bean- Java Serialization actually works- EJBs can perform remote calls- Java can add numbersand things like that really.....
    Two things ...

    - Unit testing, particularly TDD, has a learning curve. It is plainly taught that what you have observed is not ideal. I suspect the people writing the unit tests you have observed are doing a much better job these days (that is, if it's been more than two weeks).

    - In an environment with automated builds, having tests for "simple" things, like adding numbers, even provides value sometimes. If a developer makes a mistake and breaks the code so that it adds the wrong numbers, or returns the wrong variable, for example, the compiler won't catch it. On the other hand, an automated build which runs the unit tests as a regression suite, will. It all depends on the confidence of the team and the importance of the units.
  33. RE: My 2 cents...[ Go to top ]

    Two things ...- Unit testing, particularly TDD, has a learning curve. It is plainly taught that what you have observed is not ideal. I suspect the people writing the unit tests you have observed are doing a much better job these days (that is, if it's been more than two weeks).-
    Generally the 25% of Unit tests I have seen are quite valuable. Unfortunately corporate development guidelines and the joy of programming can be a very ugly thing if they require you to or lead you to provide a unit test for any piece of software your create.
    In an environment with automated builds, having tests for "simple" things, like adding numbers, even provides value sometimes. If a developer makes a mistake and breaks the code so that it adds the wrong numbers, or returns the wrong variable, for example, the compiler won't catch it. On the other hand, an automated build which runs the unit tests as a regression suite, will. It all depends on the confidence of the team and the importance of the units.
    Of course it does, if the business logic defines say a particular algorithm. This is all about "testing giving you piece of mind when refactoring". But does it? How sure can I be that my testsuite is complete in relation to whatever refactoring is going on? How can I even be sure that my test is even remotely complete.

    In all but the most trivial cases - and I agree they should not break - complete testing of even a very simple business interface is a huge task that is totally underestimated by developers and "the management" alike. As a consequence all that my UnitTests can tell me is that my UnitTests either work or do not work, and given that - next caveat - my UnitTests are correct - the system behaves correctly for these particular tests. No more, no less. Unit Tests do not test a piece of software! Unit Tests test a piece of software under particular conditions. They will almost never provide closure. This difference seems subtle but it is in fact a massive.

    That in turn raises the core question: Did software quality get any better because of unit tests? And if it did it, was it because of the running of the tests or because they proved a valuable tool of determining the right interfaces, some kind of the hacker's flavor of CRC cards?
  34. joy of programming[ Go to top ]

    Generally the 25% of Unit tests I have seen are quite valuable. Unfortunately corporate development guidelines and the joy of programming can be a very ugly thing if they require you to or lead you to provide a unit test for any piece of software your create.
    Chasing down defects that unit tests easily would have nipped in the bud would subtract way more from the joy of my programming day than being "forced" to write that unit test.
    This is all about "testing giving you piece of mind when refactoring". But does it? How sure can I be that my testsuite is complete in relation to whatever refactoring is going on? How can I even be sure that my test is even remotely complete.
    Unit tests verify that what you built does what you expected. They do not validate that you built the right thing. But, if you're writing unit tests that you readily can trace back to requirements of the problem to be solved, you're in good shape. The unit tests, for me, are a great way to 1) frame the problem to be solved, and 2) check my assumptions about the problem to be solved. They help me think about design. All of these are Good Things, IMHO.
    Did software quality get any better because of unit tests? And if it did it, was it because of the running of the tests or because they proved a valuable tool of determining the right interfaces, some kind of the hacker's flavor of CRC cards?
    I think, for me, some of the benefits of unit testing and TDD aren't readily quantifiable. I guess I've drunk the Kool-Aid, but this style of development makes me more confident, leads me toward better designs and fewer defects, encourages me to experiment more and work toward how the code "wants" to be structured...
  35. RE: My 2 cents...[ Go to top ]

    This is all about "testing giving you piece of mind when refactoring". But does it? How sure can I be that my testsuite is complete in relation to whatever refactoring is going on? How can I even be sure that my test is even remotely complete.
    It helps to have understandable code.
     In all but the most trivial cases - and I agree they should not break - complete testing of even a very simple business interface is a huge task that is totally underestimated by developers and "the management" alike.
    Nothing a little reflection code won't solve.
     As a consequence all that my UnitTests can tell me is that my UnitTests either work or do not work, and given that - next caveat - my UnitTests are correct - the system behaves correctly for these particular tests. No more, no less. Unit Tests do not test a piece of software! Unit Tests test a piece of software under particular conditions. They will almost never provide closure. This difference seems subtle but it is in fact a massive.
    This is why it is so important to write the test first. Running a test before the code is "fixed" validates the test once the test fails. This is soon followed by another validation once you "fix" the code and the test magically passes --there is your closure.
  36. I believe I've seen this pattern enough many times; A good practice comes to front, but people who aren't as gung-ho as the rest on it, don't try to understand the technique, but try to put it down instead.

    I think: Unit testing can even _replace_ integration testing in some cases, for example if you replace your DB with HSQLDB, you would in essence be doing integration testing from your unit tests. HSQLDB is an in-memory DB for Java.

    IMO, unit tests are "anything that you can test from command-line, requiring no reliance on any auxillary servers, processes, and are things that from pure Java can be setup in a New York second".

    Mocking things are also a great way to isolate app logic.

    And yes, we need to integration test. But my integration testing decreased by %90 since I started unit testing, and I have much trust in the code, and better, I feel better coding this way.
  37. Unit testing can only replace integration testing if the sum of your business logic consists of one class and one method. If your integration testing has decreased by 90% since you started unit testing, you either a) are confused about what integration testing is and misspoke or b) have no regard for the overall quality of the product you work on.

    Applications spend very little time simply spinning about in one method that has no dependencies. Flow across classes, possibly even in and out of your overall system, that's the critical bit.

    Yes, unit testing is important. Yes, I think it's great that more and more people are interested in testing their code. It provides a base level of confidence for further testing. But integration testing is where the hard bugs lurk, and where it really matters for your system.
  38. Why is it everytime Mike Spille doesn't understand a tool he writes up an article about why it is bad? It's like all his articles are just flame bait to get people riled up and discussions going. Interesting but it gets tiresome when they're full of BS.

    I think Mike missed a few points:

    1 - Unit tests are the most useful when doing "test driven development" (TDD). That forces the developer to think about the interfaces and what the method will do before writing it. Mike completely missed this point.

    2 - Unit tests are absolutely mandatory for refactoring. How can I refactor code at will that doesn't have good unit test coverage?

    3 - No one said that we only do unit tests! In fact, automated functional/integration tests are just as important. Sometimes we find code that is very difficult to unit test but easy to functional test. An SMS server, to use an example on my current project. We wrote an automated functional test with xml input that simulates all the various SMS commands. Works great. Does this prove his point? No, because we still unit test the classes that are unit testable.

    4 - JUnit has been sufficient for our unit testing, with the exception that we use jMock and it extends the junit TestCase. Mock objects have really helped our testing, I love jMock.

    Finally, everytime Mike doesn't like a tool he says stupid things like "I could write a better tool in a few days". He's said this for JUnit and also for Maven. But how many open source projects has Mike started? Has he actually implemented something better? You'd think if he could do better than JUnit in 2 days that we'd see something from him, but the answer is no. So I think people should just stop responding to his flamebait. I post the JUnit points above because I hope it may help other developers. I honestly don't care what Mike thinks because I think he is full of crap.

    -Michael Mattox
  39. Bah.[ Go to top ]

    Let's see where Mike made some interesting assumptions.

    1) Junit can only be used for low-level method testing.
    FALSE: Junit can be used for testing almost tree of objects and calls, as long as you have sufficient infrastructure set up to support it. I regularly use Junit to test out the entire flow of a use case. In fact, right now I have 40 "unit tests" crossing 4000+ lines of executable code with over 50% code coverage.

    Is it perfect? No, of course not. I'll still find errors later. But the fact remains that I am finding integration errors earlier because of the tests I am writing. Errors that would otherwise go unfound until the testing phase of my iterations.

    2) Junit results in 90% unit tests code to 10% business code
    FALSE: This relies on the assumption above - that unit tests can only be used for low level testing. We have about 1000 lines of test logic to 4000 lines of business logic, and we have been producing well-layered, well-encapsulated and high-quality code because of it.

    3) Junit is a toy framework that anyone could write.
    TRUE: in fact, Test Driven Development (Beck) actually shows you how one could write your own unit testing framework.

    3a) So therefore, it is worthless
    FALSE: There are any number of quite simple applications that have made people's lives easier (Make, for example). The level of complexity of the framework is not correlated to its value. Secondly, since junit is a defacto standard, a number of third-party tools understand how to run it, and how to process its output. Try getting them to support your homegrown framework the same way.


    4) Junit only solves 10% of the problem
    TRUE: luckily it solves that 10% very, very well.

    4a) So therefore, it is worthless
    FALSE: Does Mike think that his C compiler sucks because it doesn't have a built in WYSIWYG colorizing IDE? (And before one says "Visual C++", the c compiler is distinct, but well-integrated with the IDE). Eclipse and Idea both have junit support built in, making them easier to run and easier to get the output from. Clover uses junit to help provide its code coverage analysis. Various build systems generate junit test reports as part of their build cycles. Should all of those things have been built into Junit in order to make it valuable? Mike says a lot of things about how Junit is too limited, but he doesn't make much of an effort to say what he would add to it.

    5) Martin Fowler is trying to sell consulting services
    TRUE

    5a) So therefore, anything he says can't be trusted
    FAIRLY TRUE: I would amend that to "Anything Martin Fowler says should be taken with a grain of salt."

    Try my method sometime - try to test an entire business flow (web app round trips, for example) with a few unit tests (usually I have a couple for the success path and a couple for failure paths). While certainly not the universal cure, it has done wonders for my project so far.
  40. On Martin:

    I like Fowler's bliki. Yet another made-up word like Dependency Injection. He is a good writer and makes a lot of good insightful points. Alas his job is to publish or perish which at times means he has to sell the sizzle. I think as Java matures and frameworks implement best practices and patterns, the object prophets must find new ground. The .Net community is ripe for evangelism and wouldn't you know it ThoughtWorks is there too.

    On JUnit:

    JUnit is simple. Its beauty is not its code but its ability to validate the meaningful practice of testing - trivially, but testing none-the-less. Writing a test is a mini-design iteration as well. When XP and TTD were first promoted, JUnit was the first tangible thing a developer could use to individually effect process change.

    On constructive rants:

    Rod Johnson wrote a whole book that devalued the J2EE canon or at least Entity beans. He has the Spring Framework to back up his ideas. It is good to get a second opinion on JUnit and Mock Objects but I don't see anything but a lazy rant. Thought provoking but just a rant -- like this one
  41. Rod Johnson has never ever devalued J2EE in his life. So whuts the lazy rant thing....u got to tell me why u feel so....and whats the problem with the spring approach.
    On constructive rants:Rod Johnson wrote a whole book that devalued the J2EE canon or at least Entity beans. He has the Spring Framework to back up his ideas. It is good to get a second opinion on JUnit and Mock Objects but I don't see anything but a lazy rant. Thought provoking but just a rant -- like this one
  42. Dev time is not the issue[ Go to top ]

    I don't have an opinion about TDD or refactoring. But unit tests are very useful. Sure, dev time is slower, but the debug+fix cycle is reduced. In projects I've worked in dev was 3 months, debug+fix was 9 months. When we did unit tests it took a bit longer to develop (not much) but debug+fix cycle was very straightforward. Maintanence is also better, and it helps "document" the intention of the programmer.

    Development is cheap compared to maintenance. So I think the effort is worth it (and experience has proven so).
  43. TDD is a load of...[ Go to top ]

    All I know is that TDD, fine grain unit testing, and this idea of absolutely, positively needing to test outside of the the container is a load of crap. I personally never commit any piece of code to CVS that was written for the explicit purpose of running inside a container until I run a test that tests that code running within that container.

    Sure, you need a vast, ever growing set of tests when developing a project so that you can refactor and redesign your application with less worry. But testing outside the "container" is almost completely useless as there are so many variables that come into place. Commiting code to your source control after only running Unit tests is just so irresponsible, IMO, and can seriously damage the productivity of other developers in even the smallest of projects. I'd rather spend time writing REAL tests that test the REAL application than some bogus unit test.

    Bill
  44. All I know is that TDD, fine grain unit testing, and this idea of absolutely, positively needing to test outside of the the container is a load of crap.
    Maybe for someone who is perfect. I have lots of examples where is was better to have some unit tests that could be run with out the "container".
      I personally never commit any piece of code to CVS that was written for the explicit purpose of running inside a container until I run a test that tests that code running within that container.
    Oh. Is that what you meant. Sure, if it specifically and always needs to be run in some container. And, yes, the code that was unit tested outside the container should be tested inside the container along with the container specific code.
    Sure, you need a vast, ever growing set of tests when developing a project so that you can refactor and redesign your application with less worry. But testing outside the "container" is almost completely useless as there are so many variables that come into place.
    So now it at least somewhat useful (not completely useless)? It really depends on how many variables there are.
     Commiting code to your source control after only running Unit tests is just so irresponsible, IMO, and can seriously damage the productivity of other developers in even the smallest of projects.
    True. Unless the code has no UI or container or ... .
      I'd rather spend time writing REAL tests that test the REAL application than some bogus unit test.Bill
    I guess people who write APIs or Batch style Applications aren't writing real stuff. :)
  45. Unit Testing[ Go to top ]

    I think the underlying opinion of the article is sound.

    Unit testing should be viewed as an insurance policy or proof of contract to ensure that modules behave in a deterministic fashion and are devoid of undesired/undocumented side effects. Modules that are designed well to begin with do not require significant effort to develop unit tests because the deterministic behavior of the module practically writes the unit test!

    To the author's point, only a fool would argue that unit testing is the sole mechanism by which a solid and scalable application architecture is achieved. Where the article and this discussion confuses me is that I don't think unit testing was ever advertised as such. It's like everything else in this industry: if ignorant people are the ones making decisions, there's a good chance they'll make the wrong one.

    Any energy spent trying to discount the benefits of unit testing would be better spent preaching the virtues of deterministic module design.
  46. What is a 'UNIT' after all...[ Go to top ]

    Unit testing has more scope than suggested by the article. All that has been said about what the development community must focus on 'is' attainable by unit testing. It all depends on the definition of a 'unit' at the time of testing and the type of test being conducted - ie the behavior of a defined 'unit' of software in a controlled environment - whether it is a single class or an entire subsystem. I have successfully performed functional (exhaustive scenario based testing) and non-functional (performance,scalability..etc) tests on components using unit testing tools. Whether you use JUnit and its extensions or a home grown tool is irrelevent. Unit tests also provide an insight on what the expected system output should be given a set of known inputs without extensive documentation. This information can be carried forward to those maintaining the code. Fine grained unit tests also provide faster feedback. I for one will be hesitent to change code that is not supported by a unit test. However, due to the assumptions made during unit testing, it would be ignorant to assume that a system's behavior is correct by relying on unit test results alone. I enforce both type of testing - integration/system testing and unit testing. It is an adverse practice to enforce one and not the other.
  47. it's about testing, not unit testing[ Go to top ]

    Mike, I agree with most of your comments, although that might sound strange coming from someone who is used to TDD. Actually, it isn't, IMHO it's all about throwing the different fruits into the basket, mix them up real good and THEN you are ready to make some nice jam that can't mess up your customer's sandwich ! Or how the 3 types of testing are important (unit/integration/functional) in that testing only works when combined, not forgetting perf or stress tests of course. Yet there is one thing not be underestimated. People need SOME launch pad. And if that means articles from Flower, Beck or the likes, then by all means, welcome it and bite the sweet appel as if you haven't eaten in 20 years.