Cameron Purdy, in "Distributed Systems as Organisms," talks about how clustered applications resemble organisms, in that complex organisms are able to survive the loss of a single cell (or collection of them) without losing function.

Clustering is a proven approach to resolving such dilemmas by significantly reducing -- almost to the point of eliminating -- recovery time. The clustering approach was made popular first in mainframe architectures, and eventually it bubbled down to mid-range UNIX systems. Server redundancy with dedicated "clustering interconnects" allowed a second stand-by server to stay in lock-step with the primary server, allowing the second server to immediately stand in for the responsibilities of the primary server in the event that the primary server failed. Today, what we refer to as "clustering" is significantly different, although it is intended to achieve the same result. For example, in modern architectures such as Java EE application servers, clustering refers to very high level interconnects, such as TCP/IP sockets, that allow servers to stay in virtual lock-step, and for one server to detect the failure of another server. Conceptually, clustering allows failure to be handled without recovery, by having redundant copies of the state (such as a transaction log) that would otherwise have to be recovered. The two-server hardware clusters and the more modern software clustering implementations are the origin of what I refer to as "organic systems", which represent multi-cellular organisms that are designed to survive the loss of individual cells. In other words, a distributed system is no longer a collection of independent organisms, but is itself an organism composed of any number of servers that can continue processing -- without any loss of data or transactions -- even when server failure occurs.

While Cameron is talking about the advantages of Coherence (which is produced by Cameron's company, Tangosol) and he ends the essay with a sales jab ("If you're still struggling with traditional exception-based or recovery-based approaches to distributed systems (e.g. CORBA, JINI, RMI), then you have my deepest sympathy, but it's not too late to switch"), the explanation of the advantages of distributed systems is still quite useful, even if you are using one of the systems he disparages. Have you considered using distributed data or processing capabilities in your application? If so, how did you architect it and how has your implementation gone?

This is a very nice overview of the different mindsets. Very very good stuff.

While Cameron is talking about the advantages of Coherence (which is produced by Cameron's company, Tangosol) and he ends the essay with a sales jab ("If you're still struggling with traditional exception-based or recovery-based approaches to distributed systems (e.g. CORBA, JINI, RMI), then you have my deepest sympathy, but it's not too late to switch") ..

Just to provide a little background, I developed with CORBA, JINI and RMI. In a previous life, I implemented the CORBA Object Concurrency Service (OCS) and Object Transaction Service (OTS) in Java. At Tangosol, I spent a months researching and implementing with JINI. In a previous life, I co-architected a component-based design environment that would design, build and deploy RMI components and services. All three technologies mostly work "as advertised," but I experienced a great deal of pain with each. When I say "pain", I am not referring to the mechanics of each of these, but rather to the programming model and its assumption that the application developer must manage all of the hard parts related to distribution (failover, resiliency, correctness, timeouts), while the "framework" gets to handle the easy part (sockets, RPC, etc.) Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid

Instead of using JINI or RMI home made solutions how about JCS http://jakarta.apache.org/jcs/index.html or jgroups http://www.jgroups.org/javagroupsnew/docs/index.html. It seems to me these would be a good starting point.

Instead of using JINI or RMI home made solutions how about JCS ....

Man, you really should check Coherence, GigaSpaces, and GemFire. They go _way_ beyond those things you have mentioned.

Instead of using JINI or RMI home made solutions how about JCS ....

Man, you really should check Coherence, GigaSpaces, and GemFire. They go _way_ beyond those things you have mentioned.

Objectgrid also deserves a check.

Instead of using JINI or RMI home made solutions how about JCS ....

Man, you really should check Coherence, GigaSpaces, and GemFire. They go _way_ beyond those things you have mentioned.

Objectgrid also deserves a check.

There are many ways to do distributed application development. There are even many open source projects in the space. However, what I was trying to get at (in my long diatribe ;-) was that there are two different _approaches_ to building distributed systems, and that there are real implications to the fragility, availability, complexity, etc. of the resulting systems. Put another way, it is almost impossible to build continuously available distributed systems without using the organic approach. Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid

All the products you mention are commercial. My point is you should look in the OS space to start with instead of dumping money into a solution that may or may not work.

All the products you mention are commercial. My point is you should look in the OS space to start with instead of dumping money into a solution that may or may not work.

My point is that Open Source solutions you have mentioned are far from being leaders or even on-par with commercial offerings. Sorry for being pragmatic but I do not have any issues whatsoever with paying _reasonable_ money for products which _do_ the job as promised.

All the products you mention are commercial. My point is you should look in the OS space to start with instead of dumping money into a solution that may or may not work.

There are two parts to this that I would like to address: 1) You are absolutely right that one should not dump money into a solution that does not work. However, in all the examples given, the products can be downloaded and tested without paying a penny (to the best of my knowledge). Furthermore, I know that the developer license for Tangosol software has always been free. 2) However, the more important point is that choosing the wrong approach to building distributed systems has its own extraordinary cost, and could ultimately lead to failure. I am describing a mind-set and an approach to distributed systems, not a suggestion of commercial versus open source software. Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid

http://www.artima.com/forums/flat.jsp?forum=106&thread=172063