Testing concurrent programs can be extremely difficult; concurrent tests are themselves concurrent programs, which are difficult to write, and failures in concurrent programs are often rare probabilistic events rather than deterministic ones. In this article, Brian Goetz, author of the recently-released Java Concurrency in Practice, explores some of the major issues in testing concurrent classes, and offers some techniques for constructing concurrent programs that make them easier to test. What techniques have you found help overcome inevitable obstacles when testing concurrent programs?

Interesting, very well needed resource. I actually found the TestNG has nice build in support for concurrency, though allowing a test to run in specified number of threads and iterations. I didn't do a full fledged threading test, but it allowed me to consistently test a ThreadLocal implementation.

private volatile static Map objKeys = new HashMap(); @Test(threadPoolSize = 3, invocationCount = 6) public void testGetConfiguration() { ObjectManager objManager = ObjectManagerBuilder.buildObjectManager(); assertNotNull(objManager); if (objKeys.get(Thread.currentThread().getId()) == null) objKeys.put(Thread.currentThread().getId(), objManager.getId()); assertEquals(objKeys.get(Thread.currentThread().getId()).intValue(), objManager.getId()); }

I found that TestNG has pretty decent implementation of multi-threading. Easy to use, yet powerful! Thank you Cedric for listening to developers and implementing this feature in TestNG! This feature makes TestNG well ahead of the rest unit testing tools! What Ilya has shown clearly demonstrates TestNG multi-thread functionality!

I found that TestNG has pretty decent implementation of multi-threading. Easy to use, yet powerful!

Thank you Cedric for listening to developers and implementing this feature in TestNG!

This feature makes TestNG well ahead of the rest unit testing tools!

What Ilya has shown clearly demonstrates TestNG multi-thread functionality!

Yeah, it would be nice for more fine grained thread control. Right now, the threads and invocation don't really have a 1 to 1 mapping, so although there are three threads, there is no guarantee that say 2 invocations will run in the same thread, etc... It would be nice to add such a feature. @Test(threadPoolSize = 3, invocationCount = 6, threadMap = {2, 2, 2}) Basically stating that each thread has 2 invocations each. I'm sure there is a cleaner, friendlier way of doing this, with maybe more flexibility, but in general, we need to control thread/invocation combos. Ilya

Hi Ilya,

Yeah, it would be nice for more fine grained thread control.

Right now, the threads and invocation don't really have a 1 to 1 mapping, so although there are three threads, there is no guarantee that say 2 invocations will run in the same thread, etc... It would be nice to add such a feature.



@Test(threadPoolSize = 3, invocationCount = 6, threadMap =
{2, 2, 2})

Basically stating that each thread has 2 invocations each. I'm sure there is a cleaner, friendlier way of doing this, with maybe more flexibility, but in general, we need to control thread/invocation combos.

Ilya

I added a few functionalities recently to enable some of this. For example, you can specify that all the test methods inside a class should be run in the same thread, even if the current suite is being run in parallel (@Test(sequential = true)). This was necessary for users who want to run their tests multithreaded for the speed gain, but who sometimes have to test methods that are not multithread-safe. Now, they can simply put these methods in a class that has the above attribute and TestNG will run them in the same thread. As for your request and the threadMap, let me think about that (and let's take the discussion to the testng-users mailing-list). -- Cedric

This may be obvious, but newer developers may miss it (as I did long ago) . . . To detect many of the potential problems in an application with high concurrency, you have to use a multi-processor machine! Cheers, Gideon

This may be obvious, but newer developers may miss it (as I did long ago) . . . To detect many of the potential problems in an application with high concurrency, you have to use a multi-processor machine!

Cheers,

Gideon

Or today's dual core chips, which actually did a pretty good job disclosing a race condition bug I had. Also a unit test written with TestNG. Ilya

This may be obvious, but newer developers may miss it (as I did long ago) . . . To detect many of the potential problems in an application with high concurrency, you have to use a multi-processor machine!

Cheers,

Gideon

Really ? I don't think Dijkstra had such a beast approaching the dining philosophers problem. Concurrency problems must be detected (and solved) on paper. Different story are the detection of concurrency bugs. Guido.

Really ?
I don't think Dijkstra had such a beast approaching the dining philosophers problem.
Concurrency problems must be detected (and solved) on paper.
Different story are the detection of concurrency bugs.

Guido.

The world was much simpler back then :-) Nowadays, it's pretty much impossible to guarantee multithread safety other than using statistical testing... -- Cedric

Really ?
I don't think Dijkstra had such a beast approaching the dining philosophers problem.
Concurrency problems must be detected (and solved) on paper.
Different story are the detection of concurrency bugs.

Guido.

The world was much simpler back then :-)

Nowadays, it's pretty much impossible to guarantee multithread safety other than using statistical testing...

--
Cedric

AFAICR from my studies, testing is a NP-complete problem. In this respect you can only be quite confident that your implementation is sufficiently robust. You cannot guarantee anything. But, again, your are talking about the implementation and bug detection. The solution must work or fail on paper. As many of the problems' solutions. Guido. P.S. I don't think the world was really much simpler back then.