IBM creates performance harness for JMS

Discussions

News: IBM creates performance harness for JMS

  1. IBM creates performance harness for JMS (10 messages)

    IBM has released a performance harness for JMS. It provides a complete set of JMS functionality as well as many other features such as throttled operation (a fixed rate and/or number of messages), multiple destinations, live performance reporting, JNDI, and multiple vendor plug-ins.

    It works from the command line. Invoking the main class with a series of parameters (defining such things as the JNDI context if JNDI is used to look up the queues, performance test type, and throttling information) will cause the performance harness to issue messages to the queue or topic specified.

    As an alphaWorks technology, it's a bit raw. What do you think of the tool? What improvements would you make? What have you seen in terms of performance of JMS providers?

    Threaded Messages (10)

  2. cool[ Go to top ]

    more testing tools for JMS is a good thing. Has anyone tried it?
  3. Great stuff[ Go to top ]

    I warmly welcome any improvement in tooling to help folks performance test enterprise systems. We need much more of this kind of thing.

    Up to now we've created our own open source JMS benchmark tooling using JMeter which works pretty well...

    http://activemq.org/JMeter+Performance+Tests

    We'll have to put IBM's performance harness through its paces and see what its like.

    James
    LogicBlaze
  4. Jmeter[ Go to top ]

    We added a custom MQ "Sampler" to JMeter that uses the MQ API and it works great and took a mornings work to add.

    You can also add regex assertions just like the HTML testing.

    It would be hard to beat.

    Graham
  5. Jmeter[ Go to top ]

    As mentioned, nice to see more attention to performance frameworks for JMS as they are sorely needed. Like many I've hastily written one or two and they've not been great.

    It will be better to have freely available source code and the ability to mix in your own behavour. I'd find it tricky to justify a framework for testing where I cannot eyeball, review the code and extend it.

    JMeter is pretty good too, I vote for some competition...

    I've checked out the distribution and documentation, its clearly early access which is fine but do tell more about licensing. Its needs to be free and easy (and controlled) to take off... Apache 2.0 is a nice license to use for such as this - surely this is no IP here?

    Regards,

    Colin.
    http://www.hermesjms.com
  6. Jmeter[ Go to top ]

    JMeter is a very useful system but did not serve the requirements we had.

    The "Performance Harness for JMS package", whilst being much more of a mouthful to say, is much smaller in size, complexity and scope. To wit, this tool:
    - has a shorter pathlength in its core messaging operations.
    - does not deal with "automation", by which I mean environment initialisation, co-ordination of multiple instances, conditional logic, graphing, etc. This is all left to other layers which perform said functions across multiple languages, not just Java.
    - is completely command-line driven, I am sure this will change over time as it does not suit all tastes.

    Marc [author of this tool]
  7. that makes sense to me[ Go to top ]

    JMeter is a very useful system but did not serve the requirements we had.

    The "Performance Harness for JMS package", whilst being much more of a mouthful to say, is much smaller in size, complexity and scope. To wit, this tool:- has a shorter pathlength in its core messaging operations.- does not deal with "automation", by which I mean environment initialisation, co-ordination of multiple instances, conditional logic, graphing, etc. This is all left to other layers which perform said functions across multiple languages, not just Java.- is completely command-line driven, I am sure this will change over time as it does not suit all tastes.

    Marc [author of this tool]

    I can definitely see the benefit of having a command line driven testing tool for JMS. JMeter definitely doesn't do it that way and requires users write a test plan. It can be quite a bit of over head for users that want to run a quick test like Apache AB. It's definitely a balancing act to choose between simplicity and features. Sometimes I still run apache AB for tests, because it is simpler and much lighter. I can see the same cases applying to testing JMS using the performance harness for JMS.

    peter lin
  8. alphaworks has a 90-day trial license. Otherwise you must purchase a license to use the alphaworks offering:

    http://www.alphaworks.ibm.com/license
  9. Comparison to SonicMQ's harness[ Go to top ]

    I based my performance evaluation on a customized version of Sonic MQ's test harness which has been available for a few years now. Anyone have an opinion on how this test harness compares to Sonic MQ's old one? When I get some time to check this out, I'll be able to comment.
  10. Does anybody know what is actually measured by this tool -- and how? I didn't find anything in the documentation. IMHO it is pretty useless to measure something if I don't have that information, since I don't know how to interpret the results, then.

    Regards,
        Dirk
  11. I have noticed this too, the documentation is considerably more "alpha" than the code :( A corresponding developerWorks article should be in the offing, this will focus practical usage scenarios and will doubtlessly feed back into the docs as well.

    Here is a brief description of "what it does"

    ---
    There are many modules implementing point-to-point and publish-subscribe modes of operation, which can be explored through the documentation. Each of these modules, when selected, sends and/or receives messages from the selected JMS provider as fast they can (unless a certain rate is specified). They share a common command-line reporting mechanism and will print their current throughput rate on a user-selected periodic basis and also output summary statistics at the end of a test. The included help and documentation provide detailed usage instructions and describe many further features and configuration parameters for investigation.
    ---

    Please see any other posts I make on this page for more info.

    Marc [author of this tool]