JUnit 4.0 has been released. As the documentation states, "The architecture of JUnit 4.0 is a substantial departure from that of earlier releases. Instead of tagging test classes by subclassing junit.framework.TestCase and tagging test methods by starting their name with 'test', you now tag test methods with the @Test annotation."

As of the morning of February 16, it should be downloaded from the release page on Sourceforge. The web site, http://junit.org/, has not been updated yet but will surely be updated soon; when this happens, this news post will be altered to reflect the correct information.

As the use of annotations should indicate, this release of JUnit is only for JSE 5 (or later).

The basic form of a test, using the annotation framework, now looks like this (in a sample from the JUnit 4.0 cookbook, which is part of the download):

@Test public void simpleAdd() {
    Money m12CHF= new Money(12, "CHF");
    Money m14CHF= new Money(14, "CHF");
    Money expected= new Money(26, "CHF");
    Money result= m12CHF.add(m14CHF);
    assertTrue(expected.equals(result));
}

There are other capabilities, such as specifying expected exception conditions, pre- and post-conditions, and more.

Great!

This is the best message I seen recently.

Great!This is the best message I seen recently.

Me too.
I think JUnit is one of most successful OpenSource Projects. Thank you Mr.JUnit

Regards,
Ahmed Hashim
Software Engineer
Egypt
Egypt Java User Group http://www.egjug.org

That looks like a really good improvement. Can now have test methods in any class you wish. It allows you to test internal workings of classes as well as public interfaces.

Should make writing tests much easier and neater. Won't have all those annoying test classes to write!

Great News!

This is too late. I have been using TestNG for months and it is much more advanced than JUnit 4. I don't see why I should even bother with it, it seems to be just a tiny little subset of all the goodies that TestNG has to offer.

This is too late. I have been using TestNG for months and it is much more advanced than JUnit 4. I don't see why I should even bother with it, it seems to be just a tiny little subset of all the goodies that TestNG has to offer.

I couldn't agree more. I've been using TestNG for months now, and see absolutely no reason to switch back to JUnit. Cedric has shown a great deal of initiative in creating and maintaining TestNG. It is a great product and continues to evolve much faster than JUnit.

-Tim Fennell
Stripes: Because web development should just be easier.

And besided TestNG supports JUnit test cases (propably the "new" test annotation as well since TestNG already support identically named and I velieve identically functioning annotation already) in it's suite that I find way ahead JUnit's corresponding implementation.

The key feature for me isn't these annotations which I find being used way too often where they are only increasing complexity.

It's the ability to automatically skip tests when their dependencies haven't been met. With JUnit it was like needle-in-a-haystack pattern over and over again when trying to find the problem when some core functionality wasn't working as expected. Rerunning a nonworking subset is a timesaver in these kinds of situations. Exception recovery would make the problem go away, but it's just too error prone and besides I find the number of skips to be a valuable information automatically available.

I hope TestNG doesn't get bloated with adding too many case spesific solutions to the core or implementing only with annotations or xml files something that can be done at least partially without them. Grouping of test cases is a fine example of having annotations where java fails to provide a decent multiple inheritance.

--
Tuomas Kiviaho

It's the ability to automatically skip tests when their dependencies haven't been met. With JUnit it was like needle-in-a-haystack pattern over and over again when trying to find the problem when some core functionality wasn't working as expected. Rerunning a nonworking subset is a timesaver in these kinds of situations. Exception recovery would make the problem go away, but it's just too error prone and besides I find the number of skips to be a valuable information automatically available.

Very nice to hear, thanks Tuomas.

I hope TestNG doesn't get bloated with adding too many case spesific solutions to the core or implementing only with annotations or xml files something that can be done at least partially without them.

That's a legitimate concern. We've been adding a lot of features to TestNG these past months and I'm constantly asking myself "does this belong to the core"?

I tend to answer "yes" to this kind of question, though. These days, the jar size is not really a concern for anyone writing enterprise-wide code, so the main motivation should be: does this feature make things more complex?

I am a big fan of Stroustrup's principle: you shouldn't have to pay for what you don't use, so I just make sure that whenever I add a new feature, it doesn't complexify the design or doesn't impose an undue burden on users who don't care for such feature.

Overall, users seem to be happy to have an "all-in-one" solution with TestNG. They don't need to download separate packages, add-ons or ant tasks to become immediately productive.

Grouping of test cases is a fine example of having annotations where java fails to provide a decent multiple inheritance. -- Tuomas Kiviaho

Absolutely.

Groups introduce a very welcome separation between your Java structure (packages, classes) and your test structure. So far, we had to conflate them (name your packages so they vaguely represent your test intention, e.g. test.database) but groups make these points moot.

--
Cedric
TestNG

Forgot to give the JUnit people thumbs up. That's still a de facto implementation with outstanding community leverage.

I am a big fan of Stroustrup's principle: you shouldn't have to pay for what you don't use, so I just make sure that whenever I add a new feature, it doesn't complexify the design or doesn't impose an undue burden on users who don't care for such feature.

I'd have to concur, since with the current flow of information - that is the learning phase we have to keep up - Bjarne's principle is more important than ever.

It took me a while to figure out what had happened to setup and teardown and that @DataProvider had nothing to do with JCA, since nobody absorbs the documentation (which by the way is compact yet fully disclosing) right away. Before that happens people may have shifted to a different product or just be negatively impacted every time somebody mentions this product.

@Configuration compared to @Before and @After which I was looking after is fine grained in my opinion, but it's not intuitive. I still find @DataProvider now that I know how to use it as a feature along with @Factory I don't need that often. Yet it caught my eye first time I used code assistance.

Of course this is not objective feedback and can't be generalized, but I find the older JUnit being overly bloated in course of time when backwards compatibility beats "best practises". I'd still suggest that TestNG remains backwards compatible with JUnit's dialect, since there are a lot tools consuming JUnit's test cases.

But hey, good luck both JUnit and TestNG.

--
Tuomas Kiviaho

That example doesn't seem much of an improvement over 3.8.1 test code.

public void testSimpleAdd() {
    Money m12CHF= new Money(12, "CHF");
    Money m14CHF= new Money(14, "CHF");
    Money expected= new Money(26, "CHF");
    Money result= m12CHF.add(m14CHF);
    assertEquals(expected, result);
}

That example doesn't seem much of an improvement over 3.8.1 test code.public void testSimpleAdd()

Agreed. We have the same functionality as before, but now it only works on JDK 1.5 *and* we have to go and modify all of our existing test cases to use a different format. Great!

I'd love to hear the reasoning behind this decision. Using "cool" new features just for the sake of it? Surely this could have been provided as an optional way of writing test cases as well as supporting the current pattern of extending TestCase?

Andy Grove

---

FireStorm/DAO 3.0 Generates Java code from relational schemas

We have the same functionality as before, but now it only works on JDK 1.5 *and* we have to go and modify all of our existing test cases to use a different format.

Hi Andy
according to
http://www-128.ibm.com/developerworks/java/library/j-junit4.html

"Beck and Gamma have striven to maintain both forwards and backwards compatibility. The JUnit 4 test runners can run JUnit 3 tests without any changes. Just pass the fully qualified class name of each test you want to run to the test runner, just as you would for a JUnit 4 test. The runner is smart enough to figure out which test class depends on which version of JUnit and invoke it appropriately. "

Hi Andy,

according to http://www-128.ibm.com/developerworks/java/library/j-junit4.html"Beck and Gamma have striven to maintain both forwards and backwards compatibility.

Hi Edo,

Thanks for the reference. However, I don't think they have striven quite enough. From the same article:

Java-wise, however, JUnit 4 is not in the least bit backwards-compatible. JUnit 4 absolutely depends on Java 5 features. It will not compile or run with Java 1.4 or earlier.

It is reassuring that the original coding style will still work (not what the original announcement says). Still, not much use for those of us with users that are still on JDK 1.4

Cheers,

Andy Grove

It certainly is a good thing that junit 4 can run legacy tests and it shows junit's team concern for backward compatibility.
IMHO though, backward compatibility tends to prevent development of new technologies to a very large extent and must be dropped sometimes to achieve more important things. The new features that java 5 puts on the table are not to be ignored as they certainly enhance java programming. It's a shame that many leading vendors are yet to adopt it.
I hope many more valuable frameworks, toolkits and the likes to come drop compatibility with jdk 1.4 as this will accelerate the move forward. After all, Mustang is due in august/06.. that's just a 6 months from now.

The problem is that SOMEONE has to pay for it. We tend to develope our stuff in parallel with whatever project we are doing, and even then it is not always possible to upgrade.

Try telling a customer that the lion's share of version .x will be devoted to generics and annotation support rewrites. That's a tough sell. Customers barely want to pay for bugfixes and performance enhancments. They tend to go "Why wasn't it working(fast) in the first place."

java 5 can run java 1.4 code and yes, I know dropping all you have and recode it to take advantage of certain features is a no-no for any developer with a minimun of common sense. Newly added functionality however, in most cases, can leverage the new features, provided that the existing code can run in a java 5 VM; let alone start up projects.
The biggest impediment here is when you are lock to a particular vendor that does not provides support for the latest technology. Projects that do not have this problem can and should move on.

Also,
the benefit of moving on is not a benefit that pertains to clients; it pertains to developers. I single example. Using hibernate annotations ina project I saved a lot of time by not having to generate the mapping files. The mapping files just do not exist anymore with annotations in place. If you add up the time it takes to generate all this stuff I am sure it amounts to days depending on the size of the project. That's less time worked for the same amount of money. The same goes with generics and the bugs it prevents.

Also, the benefit of moving on is not a benefit that pertains to clients; it pertains to developers. I single example. Using hibernate annotations ina project I saved a lot of time by not having to generate the mapping files. The mapping files just do not exist anymore with annotations in place. If you add up the time it takes to generate all this stuff I am sure it amounts to days depending on the size of the project. That's less time worked for the same amount of money. The same goes with generics and the bugs it prevents.

Agreed, but what if you can support both. We use XDoclet to generate hibernate mappings and Spring supports common-annotations. The problem is that we've got project where the customer dictated 1.4 so we've got to support older stuff.

Since monday, Apache gump has been broken, because JUnit CVS_HEAD suddenly switched to Java1.5 only. Every single project that depends on gump (500+) suddenly breaks, because junit wont build:
http://vmgump.apache.org/gump/public/junit/junit/index.html

Hopefully this is just a transient event and someone will fix it, but otherwise we are going to have to retain junit3.8.1 as the normal junit for OSS projects.

Relax guys! Are you suggesting that we shouldn't use new Java 5 features in any of the frameworks just because legacy projects cannot use them? What's the purpose of having an evolving platform then? You still have JUnit 3.8.1, nothing has changed for you in that. You can still use it. For whoever can use Java 1.5, now there's JUnit 4.0. So what's the big deal? Relax! Take deep breaths! ;)

That example doesn't seem much of an improvement over 3.8.1 test code.public void testSimpleAdd() {    Money m12CHF= new Money(12, "CHF");     Money m14CHF= new Money(14, "CHF");     Money expected= new Money(26, "CHF");     Money result= m12CHF.add(m14CHF);     assertEquals(expected, result);}

I have the same question, is there any benefit to write a "@Test"?

Not only junit 4.0, spring 2.0 has adopt JDK 1.5 annotation.

Is this behavior sponsored by Sun behind?

I don't get it.

--
Cheng Guangnan
AutoAssist

http://NoUnit.sourceforge.net

Um, I'm a little confused. How is this:

@Test public void simpleAdd() {
    Money m12CHF= new Money(12, "CHF");
    Money m14CHF= new Money(14, "CHF");
    Money expected= new Money(26, "CHF");
    Money result= m12CHF.add(m14CHF);
    assertTrue(expected.equals(result));
}

...any different than this:

public void testSimpleAdd() {
    Money m12CHF= new Money(12, "CHF");
    Money m14CHF= new Money(14, "CHF");
    Money expected= new Money(26, "CHF");
    Money result= m12CHF.add(m14CHF);
    assertTrue(expected.equals(result));
}

Perhaps I need to download the new bundle and read the "cookbook," but if using Annotations is really the only new feature, then I must ask - why is this a full on release and not a patch or point release?

Also - I kind of enjoy the test cases being from a different object hierarchy than the implementation classes. I'm not sure if I want my developers writing test cases within implementation classes, that eventually get deployed into production. Yucky poo (my professional opinion).

Guess I should download the bundle and see what the fuss is about. Maybe there's a real reason for the major release that I'm not seeing from a theserverside.com article.

-Michael

I believe that your test classes still have to extend TestCase, so I think adding test methods into your production code won't be possible, as some people have mentioned so far.

There's a good article here that explains a lot of the new changes. A lot of syntax changes that offer things like suite-wide setUp and tearDown methods, exception testing, and multiple setUp/tearDowns. It actually follows a lot of the principles laid out by NUnit. Looks really good and finally brings JUnit closer to the level of TestNG.

Good job, guys!

Excellent work folks. Oddly enough, Junit vs TestNG has been a topic of conversation between me and a collegue today and it seems that this release of JUnit goes some way to catching up with TestNG features. A couple of points if I may...

Firstly, well done for recognising that the TestNG 'Configuration' annotation is far too coarse. I much prefer the @Before and @After style of JUnit 4.

The test grouping feature of TestNG is quite useful, is anything like this planned for JUnit? If I could make a suggestion, the hierarchy of tests in the testng.xml file, while useful is a little inflexible. It seems to me that (google-mail style) lables might be a better way of categorising tests rather than an XML-driven hierarchy. Perhaps a new attribute of @Test could allow for a 'category' (or categories) to be set on the test method which would allow for selective running of the tests. I could mark up my tests methods with categories that determine whether they are unit tests or integration tests, if they are ran on checkin to VC or before a QA release. I am sure that this kind of labeling feature would be very useful.

Also, does JUnit still instansiate a test class instance per test method? If so, is this still necessary? In our TestNG methods we have found it useful to set up some instance level collaborators before running test methods. I guess we can use statics or recreate the collaborators before every test method but it just seems unnecessary.

Finally, do we still need to mess around with ant to get it to run junit tasks correctly (by this I mean dropping junit in ant/lib or messing with the ant classpath). The TestNG guys seem to have solved this problem.

Once again, excellent work. JUnit is a project that has definatly made an impact on the developer community :)

Hi Wesley,

Firstly, well done for recognising that the TestNG 'Configuration' annotation is far too coarse. I much prefer the @Before and @After style of JUnit 4.

Can you explain what you find too coarse? Is it the syntax (you prefer @Before over @Configuration(beforeTestMethod=true)) or the fact that TestNG gives you "too much" configuration power? (around suites, around tests, around classes and around test methods)?

The test grouping feature of TestNG is quite useful, is anything like this planned for JUnit? If I could make a suggestion, the hierarchy of tests in the testng.xml file, while useful is a little inflexible. It seems to me that (google-mail style) lables might be a better way of categorising tests rather than an XML-driven hierarchy. Perhaps a new attribute of @Test could allow for a 'category' (or categories) to be set on the test method which would allow for selective running of the tests.

I'm a bit confused, not sure if you are talking about JUnit or TestNG here :-)

If TestNG, you might not be aware that not only can you specify that a @Test method can belong to one more groups, you can also define additional "groups of groups" inside testng.xml, which seems to address your concern.

For example, define various "database", "gui", "html" groups at the @Test level ("micro") and then inside your testng.xml, define coarser groups: "functional" = "database gui html", "unit" = "html", etc...

Also, does JUnit still instansiate a test class instance per test method? If so, is this still necessary? In our TestNG methods we have found it useful to set up some instance level collaborators before running test methods. I guess we can use statics or recreate the collaborators before every test method but it just seems unnecessary.

Nice to hear, and of course, I fully agree.

--
Cedric
TestNG

Hi Cedric,

the TestNG 'Configuration' annotation is far too coarse. I much prefer the @Before and @After style of JUnit 4.

Can you explain what you find too coarse?

I guess you could say it is subjective, I do prefer @Before over @Configuration(beforeTestMethod=true), yes. It seems to me that @Configuration does a little too much and that the attributes of @Configuration are not closely related. I tend to think '@AlwaysRun' would be better, for example. Also, I cannot set @Configuration a class but I can mark classes or methods with @Test and it is unclear what this achieves. Both @Configuration and @Test allow me to define groups, which do I use, are these the same type of group? does one take precedence over the other?

The grouping functionality confuses me a little, I dont really see the distinction between a 'test' and a 'group'. A test just seems to be a top level group that I cant define in annotations. This is unfortunate because, I cannot replace the xml file with annotations without accepting that all my tests will be in 'Command line test'.

I wonder whether you might be ignoring the Java based grouping systems of classes in packages and methods in classes. I think that it is this structure that should be used when displaying test results on an interface. It seems that with TestNG, if I declare two classes as part of the same test, when I view the generated reports the test methods of the various classes in this test group are all mixed up. I have opted to include certain methods in certain classes for a reason (likewise, classes in packages), I believe the test reports should reflect this without the need to duplicate this structure in an xml file (perhaps override, but should take package/class structure in absense of XML based grouping).

I find the concept of labeling test methods to include and exclude them from test runs very useful, because there probably is a finer grain level that class structure in action here. My ServerTest may have unit, integration, soak and stress tests combined, I still want to see all my 'ServerTest' methods grouped on my report but I would like to selectively run a subset of these tests. I guess I am saying, respect the current organisation concepts (packages/classes) but give us another, finer-grained level to play with if required. Test methods ARE defined according to their class or package, TestNG seems to ignore that and rely on it's own grouping strategy.

I hope none of this has been taken the wrong way. I fully accept that most of these problems might be non-issues, I just want to hit you will a perspective from a (reasonably) experienced developer who is new to TestNG.

I guess you could say it is subjective, I do prefer @Before over @Configuration(beforeTestMethod=true), yes. It seems to me that @Configuration does a little too much and that the attributes of @Configuration are not closely related.

I actually agree. In hindsight, it's one thing that I would do differently if I could.

I was already hesitating when I created this annotation, and I decided to go for @Configuration for IDE completion reasons: "if you don't know what configurations you can use, just press @Configuration(<ctrl-space> and you will get a complete list".

The problem with my current approach is that the boolean flags are pretty much mutually exclusive, so having @BeforeClass, @BeforeSuite, etc... annotations would have been a better choice.

Live and learn...

--
Cedric
TestNG

According to what I read, you don't have to rewrite your old tests but you have to run the tests with Java 5.

There is still a TestCase to extend with assertX methods to call and when you extend it all the testXxx methods get called.

It would have been nice to have other ways to specify the attributes (such as commons-attributes), so that JDK 5 was not required. Perhaps that can be added in the future.

As per the sample I am not impressed, @Test or sampleTest take same amount of time to code, more over we need JDK1.5 to work with new version. Also, I somehow, want to derive my test case class with junit TestCase class which I can effectively organize my suite.

I am also not impressed by the annotation because most of my apps and hundreds of test cases are still targeting 1.4

Furthermore, the example above show a major issue of JUnit: You have spend either a lot of source code lines on Mock Object creation or you have to go with some alternative marshalling of business data (Database, XStream, Castor, DDStep, DbUnit). We have algorithms where the test data creation has 2-3 times of the algorithms code size.
Adjusting test data starts to eat a lot of our resources in a maintenance phase of a project. The wonderful promise of Extreme Programming by changing code without being concerned of the side effects is a joke if you have to maintain 1000th of test data objects first.
I am a bit disappointed that JUnit do not address this.

Hi Johannes,

We have algorithms where the test data creation has 2-3 times of the algorithms code size. Adjusting test data starts to eat a lot of our resources in a maintenance phase of a project. The wonderful promise of Extreme Programming by changing code without being concerned of the side effects is a joke if you have to maintain 1000th of test data objects first.I am a bit disappointed that JUnit do not address this.

Indeed, and TestNG provides a convenient way to address this problem: Data Providers.

Here is a quick example, which should be self-explanatory (if the code doesn't get munged up):

    //This method will provide data to any test method that declares that its Data Provider
    //is named "test1"
    @DataProvider(name = "test1")
    public Object[][] createData1() {
     return new Object[][] {
       new Object[] { "Cedric", new Integer(36) },
       new Object[] { "Anne", new Integer(37)},
     };
    }

    //This test method declares that its data should be supplied by the Data Provider
    //named "test1"
    @Test(dataProvider = "test1")
    public void verifyData1(String n1, Integer n2) {
     System.out.println(n1 + " " + n2);
    }

will print

    Cedric 36
    Anne 37

The advantage of this approach is that the data is provided by a Java method, so the implementation is entirely up to you. You can get your data from your Java code, a properties or XML file, a database, the network, etc...

Once your test is in place, you never need to touch it again and you just change whatever feed contains that data.

--
Cedric
TestNG

Indeed, and TestNG provides a convenient way to address this problem: Data Providers.

Cedric, correct me if I am wrong, but doesn't a Data Provider solve just one end of the problem only? Usually I have a hard time to provide the data in the way you described AND have my test dependency using exactly the same data.

Cedric, correct me if I am wrong, but doesn't a Data Provider solve just one end of the problem only? Usually I have a hard time to provide the data in the way you described AND have my test dependency using exactly the same data.

Could you be more specific and explain what these two ends are?

Not sure if it's relevant, but once you have defined a @DataProvider, you can reuse it for several test methods. You can also have that same @DataProvider return slightly different data depending on which @Test method it's feeding.

--
Cedric
TestNG

Could you be more specific and explain what these two ends are?

Usually I have a client (say test case) A that executes methods of component B that depends on component C. C needs to return data that match parameters of A. So A and C are those two ends.

Of course I can go ahead and create mockups for C. However this might produce the maintenance hell Johannes was talking about: Making sure that lots of test data for A and C stay in synch, especially after touching B.

Hi Jens,

Making sure that lots of test data for A and C stay in synch, especially after touching B.

We recognized the need for supplying data to our testcases, objects (in your example A, B and C) and setting up the database and making this as easy as possible. Our choice was to create a "channel" to do the work so we can concentrate on only setting up the data, in our case a spreadsheet.
DDSteps does more, but it makes the supply-and-synch test data a lot easier.

Cheers,
Björn Granvik
Jayway

Your example has a similar structure as JUnit 4 allows to realize it. In JUnit, DataProvider becomes Parameterized (annotation) and the functionality providing the data becomes Parameters (annotation). Thus, JUnit seems to have an additional layer.

Is There any way we could pass Dynamic parameters to the JUnitee Test case?

I doubt it but can you tell javac not to include certain annotated methods?

It would be cool to be able to have test methods alongside the real code without any TestCase extension, but only if they can be stripped out.

Why? Because the closer the tests are to the real code the more likely they'll be kept up to date.

BTW: It seems to me that until you move to JDX 1.5 or above that there is no reason to upgrade your junit.jar - which isn't an issue right? Sooner or later corporate containers will be upgraded so no hurry to switch. Glad to see that JUnit is moving ahead.

TestNG - I promise to look at it if I ever get on a project that doesn't mandate JUnit. Sorry, it just has corproate mindshare in my area. (BTW: If anybody has a non-corporate project please let me know! I need a working vacation :-)

I doubt it but can you tell javac not to include certain annotated methods?

No, this would go against one of the major principles in Java: a Java program should always have the same behavior regardless of the environment in which it was compiled.

Which is one reason why the idea of ignoring annotations that cannot be found was rejected.

TestNG - I promise to look at it if I ever get on a project that doesn't mandate JUnit. :-)

Great :-)

--
Cedric
TestNG

How can I specify the methods to be run in a collection of full qualified method names with this junit core extension ?

Abby - cfds specialist.