A new article on DeveloperWorks discusses the theory and application behind using message queuing in Java applications, especially since MQ tools are getting cheaper. When used properly, MQ technology can often improve the organization, performance, and scalability of your application. Is there room for messaging in your next app?
- Posted by: Jeff Condon
- Posted on: February 05 2002 14:19 EST
Read Should you use JMS in your next enterprise application?.
- Should you use JMS in your next enterprise application? by Adam Young on February 06 2002 14:41 EST
- Should you use JMS in your next enterprise application? by Stephan Schmidt on February 07 2002 04:16 EST
- Should you use JMS in your next enterprise application? by Peter Daly on February 06 2002 18:26 EST
- Should you use JMS in your next enterprise application? by Lori Olson on February 06 2002 19:46 EST
- Should you use JMS in your next enterprise application? by David Clarke on February 07 2002 05:20 EST
- Should you use JMS in your next enterprise application? by Raghavendra Udupa on February 07 2002 05:45 EST
Should you use JMS in your next enterprise application? by Don Stadler on February 07 2002 06:53 EST
Should you use JMS in your next enterprise application? by Paul Watson on February 07 2002 08:52 EST
Should you use JMS in your next enterprise application? by Don Stadler on February 07 2002 09:30 EST
- Should you use JMS in your next enterprise application? by Andy Yang on February 12 2002 10:04 EST
- Should you use JMS in your next enterprise application? by Don Stadler on February 07 2002 09:30 EST
- Should you use JMS in your next enterprise application? by Paul Watson on February 07 2002 08:52 EST
- Should you use JMS in your next enterprise application? by Radhakrishnan Nagamuthu on September 13 2002 04:41 EDT
- Should you use JMS in your next enterprise application? by Don Stadler on February 07 2002 06:53 EST
- Should you use JMS in your next enterprise application? by Quentin Cope on February 07 2002 06:39 EST
- Should you use JMS in your next enterprise application? by Phil Miller on February 08 2002 03:17 EST
- Should you use JMS in your next enterprise application? by Tiago Ant?o on February 07 2002 05:35 EST
- Should you use JMS in your next enterprise application? by gaetan zoritchak on February 07 2002 10:54 EST
- Should you use JMS in your next enterprise application? by imran basheer on February 07 2002 16:42 EST
Should you use JMS in your next enterprise application? by Galen Beck on February 07 2002 05:58 EST
- Should you use JMS in your next enterprise application? by Nick Minutello on February 07 2002 08:29 EST
- Should you use JMS in your next enterprise application? by Galen Beck on February 07 2002 05:58 EST
- Should you use JMS in your next enterprise application? by imran basheer on February 07 2002 16:42 EST
- JMS Performance? by Bruce Blackshaw on February 08 2002 16:11 EST
Not a bad read. My old company wanted to get into Enterprise Application INtegration (EAI) in the worst way. We kept evaluating all these Integration products like ActiveSW, MQSeries, Tibco, etc. The problem was that they seemed to have DB to DB down, and standard APPS like SAP, but you still needed to build stuff ontop of their APIs form most integration. and they cost a Bundle.
Who here has actually used a JMS connector directly, and in what realm? I think the anecdotes will be illuminating.
Hi, in my MSc thesis I evaluate JMS as a technology (besides e.g. JavaSpaces, JXTA, ...) for asynchronous indexing of knowledge. Mail gets indexed and sorted into a taxonomy. To decouple the mail server (James, Mailet) from the indexing process, I've written an API where you can plugin several providers. JMS works fine,
JMS looks to me like something I can use in many places instead of starting a thread. For instance, I've got a "log" function which performs a database query which can take several seconds. Right now the log function starts a thread, which cuts a couple seconds of off the user response time, but probably isn't the best thing for server load.
JMS sounds like it is a good solution to my problem. I just send the JMS channel the "log this" message and forget about it. On the other side, have a MDB which consumes the entries and does the actual logging. I've been hesitant to actually do this yet (I need to reimplement the process, as the requirements have changed). I'd love some more examples of proper, and improper uses of MOM, and MDB. What I really would like is example other than the generic "stock quotes" or "stock transactions" examples. What real application uses are there?
Ok, here's an example of something besides "stock quotes". We have an n-tier application which needs to export data (which lives on the server), for use in other, 3rd party applications (which live on the client side). But, depending on the type, amount, and location of the requested data, generating the export on the server can be a very long process. So the client sends an export request (MDB!) to the server. Then the server goes chugging off to generate the export. When the export is complete, the server pops another message into a (JMS!) queue on which the client is listening, to indicate that the export has completed, and the export file is available.
I'm currently designing an interface to a store & forward system which provides an intermediary service for the sale/purchase of Unit Trusts. When you send a request to buy or sell something, the counterparty can take hours or days even to acknowledge receipt of the request and more hours or days to get back to you with the price. If you don't use messaging, how else can you efficiently implement an interface of this type? This is an ideal problem for a messaging solution. You drop the request on the outbound Q and go to sleep on the reponse. Wake up when it arrives or on a timeout and you're home and dry. Wherever there is a requirement for asynchronous communication, MOM is the answer. Interestingly enough, on my system only one side of the communication is Java. JMS will obviously be used there, but the other side is some C coded legacy stuff. IBM's MQ-Series product (JMS implementation out of the box) will be used for this. MOM can therefore deal with several (though not all) system interoperability issues too.
This is another example where MoMs can be useful. I had worked for a client where the request came from a AS400 system, which had to be responded back from an ERP system which was on AIX (Unix). The problem was the ERP version the client had, did not participate in Java. As a result, the response from the ERP system could not be obtained through some standard Java API like JDBC.
And, moreover, the ERP system could be down sometimes, which could make the process dead at times, if the whole process was integrated. To avoid this tightly-coupled integration and allow the request-end (AS400) to be completely independent of the back-end, MQSeries was used for asynchronous messaging. And, this worked perfectly well for that situation. Further more, this kind of architecture allows more scalability, in the sense, the same architecture could be extended to support CRM or any other bigger application at the back-end in future.
"And, moreover, the ERP system could be down sometimes, which could make the process dead at times, if the whole process was integrated. To avoid this tightly-coupled integration and allow the request-end (AS400) to be completely independent of the back-end, MQSeries was used for asynchronous messaging. "
MOM's are perfect for loosely-coupled architectures like what you did. I had a different problem on the first MOM system I wrote. I was provisioning LDAP objects on the back end using a system utility which had an absolute hard limit of no more than 4 or 5 transactions per minute. Not threads-safe, so no way around the problem.
Obviously I could get many more messages per minute. I was using Tibco, which is a non-queueuing architecture and only guarantees keeping a message 60 seconds. I thought of using the Tibco certified messaging product, which persists the message, but in the end split the adaptor into a reader and a creator. The reader adaptor reads the message and writes config files to disk (the config files were needed anyway). Then the configuror shows up in it's own sweet time and creates the objects, sending a status message back via pub/sub.
BTW, Web Services have some of the same virtues which have been mentioned above. I see them as sort of a hybrid between ORB and MOM in some ways.
'BTW, Web Services have some of the same virtues which have been mentioned above. I see them as sort of a hybrid between ORB and MOM in some ways. '
Yeah, good point. In fact, I have always asked myself what, apart from the web level integration (using XML, SOAP blah blah), is WS offering above MOM/ORB? From my very basic understanding of WS, in some important areas it is also deficient to MOM right? (e.g. WS does not handle end-to-end distributed transaction support)
"From my very basic understanding of WS, in some important areas it is also deficient to MOM right? (e.g. WS does not handle end-to-end distributed transaction support)"
Does MOM, necessarily? I thought that was more a function of workflow engines as such.
Different tools for different purposes. I personally see Web Services as an aalternative to ORB's and Request-Reply MOM for some things. Plus, Web Services are better set up for B2B architectures than for EAI.
Another thing to consider is that SOAP is potentially agnostic as to it's transport mechanism. We currently have SOAP over HTTP and SOAP over SMNP. Why not SOAP over JMS?
Sun's JAXM provides hooks to construct SOAP messages bound to JMS. I haven't tried it myself however - would love to hear from people who have.
I read your message in the forum.
I am quite interested in how implemented this.
Actually I have to integrate my Java Application with Clarify CRM system.
I like to know how i can use jms here.
Plz explain me in detail and mail me to
nrk74 at yahoo dot com.
Thanks in advance.
"JMS looks to me like something I can use in many places instead of starting a thread. For instance, I've got a "log" function.."
Tyler Jewell discussed the use of messaging in a recent video off this site. I had an issue with the example that he chose but Markus Blumrich came up with an excellent example, a central logging function for a distributed system. To me that's a great idea. It makes sense to do it because a central logging function would really help to administer a large complex system. However I doubt the use of JMS and MDB would be less of a load than spawning a thread and doing some JDBC... I guess I am trying to say do it because it makes sense and adds value rather than considering it to be a cheap option.
Another example of message EJB's would be to help control the performance on your web server/servlet engine for complex web form processing applications.
If you have a form that posts (or gets) to a servlet, you typically
1. using a servlet, get the form elements out of a <code>HttpServletRequest</code> object,
2. process those form elements, then
3. use the <code>HttpServletResponse</code> object for that servlet to send a reply back to the user.
If step 2 is complex and expected to take a long time, the user might believe that the system has hung while they are waiting for a result web page, and quit your web application. This would leave your serlvet engine in step 2, but with no client to receive the <code>HttpServletResponse</code> in step 3 (I believe).
The user perceives the system as having poor performance. Long running processes in step 2 also hog available web server and servlet engine resources.
A solution would be some to use asychronous processing at step 2. So:
1. using a servlet, get the form elements out of a <code>HttpServletRequest</code> object,
2. conjure up a unique requestId, and send the form elements and this requestId to a message driven bean, which can asychronously handle the complex processing.
3. use the <code>HttpServletResponse</code> object for that servlet to send a reply back to the user, but add a <code><META HTTP-EQUIV=Refresh CONTENT="20; URL=/servlet/processServlet?requestId=123456"></code> tag to the web page that you send back, which will instruct the client (IE, Netscape) to call the processServlet once every 20 seconds, to check for a result. This web page will typically be a holding page, with some fancy graphic saying "please wait, we are processing your request". (These pages universally have an animated gif with grinding cogs in them, which is probably 300k in size anyway, and takes 40 seconds to load, but that's another story..)
4. In the meantime, get the message bean to write the result of its processing to a database table (maybe in XML) against the requestId.
5. processServlet has now been invoked after 20 seconds, and checks the results database table for a result, by providing the requestId. If a result is found, parse the result and display to the user. If no result is found yet, repeat step 3.
6. A Timer Task should clear up the results table of old requests every say, 5 minutes.
Hope this helps!
I have used messaging systems in my previous project and, among other things made development a much more easy task.
As there were integration between lots of components (via messaging), JMS made it very easy to find the "guilty party" on the integration phase (don't consume the message that was sent, look at the content, find which side is responsible).
Also, I am working in a windows shop (unfortunatly not doing Java at the moment - there are other ways to be burn is this economic climate, like having to do things you really dislike) and I had to make a bridge between JMS and MSMQ (via JNI), and have to say that MSMQ is very easy to administer and maintain, also easy to program with. Although the performance with transactional messages is very bad.
I would definetly recommend the checking of messaging systems for some enterprise systems, as the decoupling that messaging gives is very convenient, im my case it was a real productivity boost... like having things ready way before schedule.
The first use of JMS in our application is for integration with other applications.
But we also use it for some of the client/server communications ( most of these are RMI-IIOP calls ). This allows us to push object down to the client ( Swing).
Two examples :
- we use pessimistic locking on data. A user who wants to modify a data which is currently locked by another user can subscribe to be informed when the lock is removed,
- when a user asks for a treatment that may be long we launch it in a batch mode and the send the results to the user.
Some of you must have used JMS in developing Workflow applications. please share your experiences.
Well, we evaluated JMS for a couple different uses in our web-based business software. We decided we didn't need the complexity (and didn't have the budget) for full blown JMS.
Luckily our app server is also a transactional persistent cache (GemStone/J) that has some conflict-free multiple producer / single consumer queues.
One use for this was to develop an email service for the generation and delivery of customized email messages, reports, data exports, etc...
This email service runs in its own VM and is
"woken up" by the submission of new email requests(typically from other VMs) into a persistent queue. The service VM then fetches JSP-generated content over HTTP, connects to a mail server, and sends a customized email to the end user. (This could totally be done with JMS if we didn't have built-in persistent queues easily available.)
If a failure occurs, the message is added to another queue for administrative processing and eventual resubmission. The overall solution has proved fault tolerant and easy to administer.
Another use for messaging / messaging queues has been to perform fail-safe index updates in our OODB after initial commits.
Finally JMS is getting some publicity. I think that asynchronous messaging is under-utilised in a lot of Java/J2EE applications. A JMS forum on TSS would be good - I made a request here a while ago but I dont think there has been enough demand. Maybe now?
Asynchronous messaging has many applications. Application-to-application communication is a prime candidate for JMS - decoupling the applications from each other - already mentioned above.
Messaging is useful WITHIN an application also.
E.g., within EJB, there has been a problem that when performing several (independent) but time consuming actions, each action must be done serially (due to EJB's single-threaded model).
A good design pattern (now made much easier with the addition of MDB's) is to effectively use JMS to simulate multi-threaded processing. By sending several asynchronous JMS messages to a queue and subscribing to these messages with an MDB, you effectively get the application server to create/allocate new threads of execution for you. The payload of the message contains details of the work to be carried out (kind of like a command pattern) and the Message Driven Beans perform (or delegate) this work. The client no longer has to wait for all tasks to complete end-to-end.
The client does, however have to manage the fact that there are now a number of tasks - and that they will complete at different times and out of order - and also has to be aware of when all tasks have completed. If the client is expecting response(s), then; For a GUI client, the GUI can consume responses from a return message queue (or topic) - in the case of a web client, the servlet layer must do the JMS consumption and manage the state of the call.
The added benefit of using JMS to achieve this is that scalability can be easily achieved by adding more application servers with MDB's to consume from the queue. (obviously EJB is not a necessity here - but it often makes sense). Queue-based consumption is a one of the most natural forms of load balancing - and JMS allows this to occur remotely (or just inter-JVM).
In fact, many synchronous operations can be optimised somewhat when non-essential tasks are required to be performed. Examples are logging, profiling (you may want to send/store transaction details for statistical analysis), sending emails, transaction tracing (who did what to what, and when - accountability/security).
More interesting is the design of a system to be asynchronous by nature - responding to external events - preparing data, performing calculations so that data is available in a cache BEFORE a client requests it. Compare this to performing the SAME data retrievals, performing the same calculations EACH TIME each client requests it.
In the first case, the data integrity is maintained at low cost - the second case either implies a cost to efficiency or a risk of stale data if results are cached on a time-based strategy.
The big plus in all of this is that each message production / consumption and associated work can be done transactionally. This is especially good when you want to save something in a local database and send a JMS message (with the insert) to a central database. Both databases can be kept (approximately) in synch without the complexity/fragility/cost of database replication.
However, it isnt time to throw away synchronous RPC for JMS. There are lots of applications where synchronous RPC calls make sense and asynchronous (even synchronous) JMS calls dont make sense at all.
While it is possible to make synchronous calls using JMS (which can approximate synchronous RPC calls), there is a development and complexity cost. If using JMS in situations where RPC should be used, you wind up doing your own object activation (once the message is unpacked, you have to work out what to do), you wind up doing your own marshalling (you have to pack and unpack the message payload - which may be complex - or it may not), you lose security granularity (compared to EJB anyway) and to add insult to injury, you also have to perform the call state management, handle timeouts and network error conditions etc etc.
In addition to the added complexity, there is also a lack of transaction propagation. The message sending and the message consumption can each be transactional tasks - however it is completely impossible to propagate the transaction with the message (you cannot even hack it).
There are loads of other good applications for messaging (too many to mention here) I would recommend getting hold of this book - it used to have a microphone on the front cover till oreilly changed them all.
PS: How about that JMS forum?
Since EJB 2.0 I've been using JMS quite a lot via MDBs. These are extremely useful for batch processing - medium to long running tasks that you want to kick off using a client and then forget about.
However JMS performance doesn't matter much in batch processing - you're only sending one message.
I'm curious to see if anyone has done some performance testing with JMS, using, say, Weblogic 6.x. How many messages/sec can be sent? How many/sec can be sent to an MDB? Does anyone have any results?
Your throughput (actually the amount of messages you can receive/consume, rather than send) will depend very largely on the sort of messaging that you want to do.
The throughput will depend on if it is queue-based or if it is topic based (you cannot really cluster MDB's to consume from a topic - which reduces the scalability of your throughput). Also the type of messaging that you want - whether it is persistent messaging or not will also greatly affect throughput.
Generally, performance is less important than reliability in messaging. The decoupled nature of messaging is only usefull if the messaging provider can guarantee message delivery. In some cases losing one message can cost lots of money.