In our design, we use a JMS queue (MQSeries) to receive work. This queue is listened to by our MDB's, who forward the work to a session bean for business rule processing all deployed in Websphere.
To scale the system, we should be able to add another replicated app server on another machine which can also process work the same way.
Now, the component that creates the work will simply forward to one queue. our MDB's will all listen to that queue.
In a Websphere environment, how will that work be distributed? Is this controllable? We want the work to spread evenly if you add a new machine, not to bog down one server.
Will we need to implement some middle component that can distribute the work maybe in a round robin fashion, least used fasion, etc, or does Websphere's work flow generator handle this for us? Is there a better way of scaling this than having all MDB's listen on one queue?
As your MDBs do not process the requests, there's no need to balance their loads, do not focus on them. You should try your hand at balancing the load of those SBs. WebSphere 3.5 EdgeServer has a component named Network Dispatcher that supports load balancing. Something similar (EdgeServer 4.0 perhaps?) must be available from IBM.
You don't need to write the distribution code yourself, although WebSphere doesn't do it either. That's the fundamental adventage of MOM (message-oriented middleware) over synchronous techniques. Each App server will pull messages off the queue and hand them to its MDBs. It will simply pull them as fast as it can process them: so if you have two boxes with the same speed, they will genrally handle equal work loads. If one of the boxes becomes busy with other work, or goes down, more load will automatically be processed by the other server.
Its important that you understand that there is no software component that "handles" this smart distribution. It is inherent to MOM, and is the main reason MOM in general and MDBs in particular are so scalable.
The nature of a message que in a MOM's implementation is to deliver a message to 1 and only 1 consumer. The MDB's are the consumers. So, as quickly as your MDB's can take one and forward it to the SB and return it will get another. As long as nothing is preventing it from passing up to a session bean efficiently then all MDB's should compete equally for the que and you will automatically get your load balancing. If you are not seeing this behavior then you need to look for the reason SB's are not being made available for receiving a MDB pass off.