Baptiste Wicht, a student from Switzerland, has published results from a Java synchronization benchmark he created. He compared a synchronized method against a Semaphore and a ReentrantLock (with fair and unfair switching for each), and for comparison purposes, against usage of an AtomicInteger, which provides atomic get and set operations as its name might imply.
The test ran anywhere from one to 128 threads, and measured the elapsed time for the while test.
Fair switching means that the longest-waiting thread gets priority for the lock; unfair switching removes any order. The javadocs for ReentrantLock explain this pretty well, as well as pointing out that fair switching is slow (confirmed by Wicht's test) and some other notes about fairness.
The results are pretty cool: the semaphores and reentrant locks performed the same way, and AtomicInteger was fastest (and the most limited). Wicht does point out that the JVM you use will probably affect the test (he used a 64-bit Sun JDK 6 with an unspecified version on a Linux server).
AtomicInteger is the most limited because the lock is internal to the get or set operation. If all you're doing is changing an integer, then it's appropriate, but the test was using the operation just to provide an exclusion point for locks. In a "real application," the exclusion would likely be inappropriate for AtomicInteger and you'd use a lock or a semaphore.
Another thing to think about: if you're creating a test for a library or application, you'd better consider whether anyone's ever going to use what you're testing with lots of threads, because information like this can be critical to making your code scale.