Cedric Beust is happy to announce the availabilty of TestNG 2.0. TestNG is a testing framework that fixes most of JUnit's deficiencies and innovates in a number of ways: use of annotations, configurable dynamic invocation (no need to recompile), test method groups, dependent methods, external parameters, etc. There is only one new feature in this release, but it's quite a big one: JDK 1.4 support.

Example of using Test NG with JDK 1.4

import com.beust.testng.annotations.*;

public class SimpleTest {

  /**
   * @testng.configuration beforeTestClass = "true"
   */
  public void setUp() {
    // code that will be invoked when this test is instantiated
  }

  /**
   * @testng.test groups = "functest"
   */
  public void testItWorks() {
    // your test code
  }
}

Read more: Announcing TestNG 2.0!

Visit: Test NG home page

This is good addition. In fact it was one of the main showstoppers to use it.

Now I will re-evaluate it again.

Since it was mentioned in the blog:
-----
For the record, we are using the excellent QDox to parse annotations and Doug Lea's original concurrent utilities to implement parallel test runs.
-----
Is there any way to post a test case example that demonstrates how to write concurrency test?

Also, another question: Is there any way to customize generated HTML report? For example, to have some output customization Ant tag.

And the last question: Is there any integration with Eclipse (the way JUnit integrated)? Any plans?

Since it was mentioned in the blog:-----For the record, we are using the excellent QDox to parse annotations and Doug Lea's original concurrent utilities to implement parallel test runs.-----Is there any way to post a test case example that demonstrates how to write concurrency test?

I am not sure TestNG is doing exactly what you want. Parallel support means that by specifying a simple flag in testng.xml, TestNG will launch your tests in concurrent threads. You can customize the size of the pool and TestNG will be smart enough to run dependent methods in the same thread, but that's it.

Did you have anything specific in mind bout thread testing? (I am actually chatting with Doug Lea as we speak about this)

Also, another question: Is there any way to customize generated HTML report? For example, to have some output customization Ant tag.

Yes, there is an extensive factory and interface-based API that allows you to register listeners that will be notified whenever a test is run, stopped, succeeds, fails, etc... The default HTML reporter you are seeing in the documentation is a listener itself.

And the last question: Is there any integration with Eclipse (the way JUnit integrated)? Any plans?

Yes, I started working on this. No ETA as of now, though, sorry...

--
Cedric

Since it was mentioned in the blog:-----For the record, we are using the excellent QDox to parse annotations and Doug Lea's original concurrent utilities to implement parallel test runs.-----Is there any way to post a test case example that demonstrates how to write concurrency test?

I am not sure TestNG is doing exactly what you want. Parallel support means that by specifying a simple flag in testng.xml, TestNG will launch your tests in concurrent threads. You can customize the size of the pool and TestNG will be smart enough to run dependent methods in the same thread, but that's it.Did you have anything specific in mind bout thread testing?

I see. Actually, I was thinking that testNG could go further with test cases and define some kind of annotation tag that will tell to execute concurrently multiple instances of the same unit test.

for example (modifyed your original one):

import com.beust.testng.annotations.*;

public class SimpleTest {

  /**
   * @testng.configuration beforeTestClass = "true"
   * @testng.threadPool id = "MyThreadPool1"
   * @testng.threadPool.MyThreadPool1.threadCount = 10
   */
  public void setUp() {
    // code that will be invoked when this test is instantiated
  }

  /**
   * @testng.test groups = "functest"
   * @testng.test threadPoolId = "MyThreadPool1"
   * @testng.test fork2AllAvailableThreads = true
   */
  public void testItWorks() {
    // your test code
  }
}

I'm bad with naming, but basically the idea is to tell that testing engine should execute concurrently "testItWorks()" and thread pool would give it number of threads(test instances) to be invoked.

I think this will help developers to test their classes (DAOs for example) for concurrent access.

Currently, in JUnit I have to do it myself, (e.g. implement Runnable and put my testing logic there and then run N threads)
Would be cool, if this could be taken care of by testNG...

Concurrent testing is also what we are looking at too.

About 3 years ago, we have developed a Test Driver (In house) for JUnit Testing, The Drive takes property files allows one to test by Package(s), by Class(es) or by methods. It can also used to exclude certain package, class or methods.
The Test Driver can be called either from Ant or commandline.

For example, the following examples run package1 and 2 but exclude Myclass in package1

ant test-driver -Dtest.targets="-p com.company.product.package1 com.company.product.package2 -exp com.company.product.package1.Myclass"

For example, the following examples run package1 and 2 but exclude Myclass in package1

Another example: only runs the method in Package1.MyClass testMethod1

test-driver -Dtest.targets="-m com.company.product.package1.MyClass.testMethod1"

Another example, runs the MyClass1 in package1 and MyClass2 in package2

test-driver -Dtest.targets="-c com.company.product.package1.MyClass1 com.company.product.package2.MyClass2"

test-driver -Dtest.targets="-all"

to run it all.

You get the idea. We have found during last three years' development, it's very helpful to isolate method level unit testing (The default Junit implementation could not do, it only isolate class level unit testing).

Another area we are working on is the concurrent Junit testing. When porting our application from Weblogic to JBOSS 4.0 ( in order to expand our App server coverage), we found that there are deadlocking issue that would not be able to discover by simply run single threaded JUNIT test.
Some unit tests should be marked as concurrent and specify number of thread to run for the method.

I like the taging idea proposed here.


Thanks

Chester

Sounds like an enhancement request for testNG!

Just tried to give it a shot. Here is my experience log:
-----------
Downloaded installation ZIP file (http://beust.com/testng/testng-dist.zip). No sight of any jar inside (neither README), but it has mighty build.xml file.

Tried to run it with my ant 1.6.1 under JDK1.4 - failed. 8-|
Installed JDK1.5 and run ant 1.6.1 again – failed :o

Then I had an idea, that may be javac needs 3rd party jars it was mentioned in BLOG (http://beust.com/weblog/archives/000215.html)...

OK. downloaded qdox-1.5.jar from http://qdox.codehaus.org/
Then I had to download (http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/current/concurrent.zip) and build "concurrent.jar". It should be Doug Lea's concurrent utils right? (http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html)?

Then copied these jars into <TestNG>\3rdparty directory...

Tried to run ant 1.6.1 again - failed... :()...:(

After, looking on the compiler error messages more carefuly found that it complains about "edu.emory.mathcs.util.concurrent.ThreadFactory" but mine concurrent.jar does not have these classes.

Suggestion:
Is there any way it could be supplied a simple README file along with testNG distribution that will tell exactly from where to get required 3rd party jars from and how to compile testNG? Also, would be nice if testNG jar would be supplied as well...

Finally, gave up since it was too much for the installation...
May be I'll try some time in the future when it will have hopefully friendlier installation...

Just tried to give it a shot. Here is my experience log:-----------Downloaded installation ZIP file

Are you sure? I just downloaded it on my laptop and the two jar files are here:

testng-2.0-jdk14.jar and
testng-2.0-jdk15.jar

and I didn't upload a new zip file since the announcement.

also, doc/index.html contains the entire documentation (same as on my Web site). Feel free to email me directly if you have any more problems... (cedric at my last name.com).

--
Cedric

Just tried to give it a shot. Here is my experience log:-----------Downloaded installation ZIP file

Are you sure? I just downloaded it on my laptop and the two jar files are here:testng-2.0-jdk14.jar andtestng-2.0-jdk15.jar

Dear Cedric,

My apologies... somehow I missed the jars! Could be my fault with copiyng distribution archive content... *:(

They are present! Will give it another shot...

Thanks for your reply!