GigaSpaces releases GigaSpaces Enterprise Application Grid

GigaSpaces Enterprise Application Grid, version 4.0, has been released - adding powerful data grid, messaging and parallel processing capabilities and delivering in-memory levels of performance as well as high reliability to transactional, data intensive, business-critical enterprise applications.

New features include:
    * Enterprise Data Grid
      - Enhanced distributed caching
      - JDBC interface to in-memory JavaSpaces data-store
      - JavaSpace querying using SQL and Regular Expressions
      - Integration with O/R mapping technologies (Hibernate, JDO)
    * Enterprise Messaging Grid
      - Innovative, distributed JMS Grid
      - Enhanced JavaSpace-based messaging
      - Sequencing (FIFO)
      - Content based routing
      - Parallel distributed Workflow support
    * Core JavaSpaces Grid Engine
      - Scalable to thousands of concurrent users
      - Synchronous & asynchronous replication
      - Multicast replication option
    * Management and Provisioning
      - Java Management Extensions (JMX) interface
      - Dynamic deployment of GigaSpaces Grid components (RIO)
      - Grid management tools integration (Tivoli, Enigmatec)
    * Service-Oriented-Architecture (SOA)
      - Performance enhancements
      - Enhanced .Net, C++ and J2EE support
      - Support for Jini 2.0

The Data Grid is built for the most demanding business-critical applications, and utilizes the underlying resiliency features of the Enterprise Application Grid such as clustering, replication, fail-over and load-balancing to ensure data availability and coherence even in the most challenging environments. It offers the following features:

JDBC Interface
Object-data in the Data Grid can be queried through a standard JDBC interface using standard SQL queries. This offers powerful and finely-tuned query capabilities to developers over a distributed data-store. Developers can implement legacy business logic over a Data Grid with little or no code changes required.

Regular Expressions
In addition to the above, the Data Grid can be queried through a Regular Expression interface, for quick yet powerful access to data. This feature is especially useful for the unstructured data that exists in many large systems.

Enhanced distributed caching
GigaSpaces provides a distributed caching middleware that supports both the JavaSpaces API and the Java Map API, which is considered the foundation of the JCache API and is currently the de-facto standard cache API.

The GigaSpaces cache maintains a local view of distributed data. This view is updated on demand and can control both the amount of data that will be stored, and the time it will reside in the cache. The distributed cache takes care of the synchronization and consistency of multiple cache instances that access the same information. The local view can be accessed either through the JavaSpaces API or through the map interface, depending on application requirements.

Integration with leading O/R mapping technologies
The GigaSpaces Enterprise Application Grid offers full integration with widely-adopted O/R mapping tools such as Hibernate and JDO, allowing a broader range of options to developers wanting to harness the power of the Data Grid.

Messaging Grid
This release offers a Full-featured Messaging Grid integrating a full-fledged JMS implementation written to the full JMS specifications; native JavaSpaces-based messaging allowing a simple, powerful and flexible distributed messaging; message sequencing and ordering (FIFO mode); Dynamic content-based routing as well as the ability to manage messaging through a workflow of business logic operations.

A complete JMS Grid
The JMS Grid is very different from other JMS implementations: as an integrated component of the Enterprise Application Grid, it offers the following unique features on top of the standard JMS specifications:

in High-Availability
High-availablity for the GigaSpaces JMS Grid comes out of the box, through the highly-available dirstribued shared mamory pioneered by GigaSpaces. Tuning a JMS system for High-Availability has never been easier.

Unique performance optimizations
Through a powerful distributed shared memory, the GigaSpaces JMS Grid enjoys in-memory levels of performance as well as a array of networking optimizations that allow the processing, production and consumption of messages to happen as close as possible (or even local) to the application module that requires it.

Seamless integration between Data and messaging
Being based on an in-memory data store, the GigaSpaces JMS Grid is naively integrated with a data-store, data-base front end and data management features.
  
JavaSpaces-based messaging
The JavaSpaces model lends itself ideally to high-performance, simple and powerful messaging through the JavaSpaces API. This can serve mission-critical applications as well as simple applications.

Sequencing (FIFO)
Messages entering the Enterprise Application Grid can be read/taken according to the order they were written in. This ensures coherency and consistency in complex workflows where operation timeline is critical.

Content-based routing
To allow applications to participate in messaging in a fully adaptive and dynamic manner, the Enterprise Application Grid allows messages to be routed dynamically based on message content - parsed in real-time.

Parallel distributed workflows
Many mission-critical applications are built as workflows: passing execution at certain times according to a certain business-logic. The Enterprise Application Grid offers the ability to parallelize the processing of workflows, the data it operates on and the messaging mechanism, to maximize the useage of available resources.

Read more about the GigaSpaces Enterprise Application Grid at: http://www.gigaspaces.com

Yawn.

Yawn.

Not terribly constructive Andreas, I know you work for SwiftMQ and that it's a damn good product but "yawn" doesn't really tell us much.

SwiftMQ is a JMS implementation, it's cheap, has an excellent GUI and works extremely well, it used to even be free but it's still very affordable even now.

GigaSpaces is a Jini/JavaSpaces implementation, JavaSpaces and JMS are two different things, for marketing reasons they have chosen to provide a JMS API. As a JMS it works well but I know they do not sell this just as a stand-alone JMS. You can compare this to WebLogic, WebPhere or even JBo$$, you wouldn't buy/use any of them for their JMS but they do provide one. If I wanted just a JMS then I might go for SwiftMQ, SonicMQ or ActiveMQ, Andreas is not doing a great P.R. job so I might now be more inclined to chose Sonic or ActiveMQ but at the end of the day I'd buy GigaSpaces for JavaSpace, one of the above for pure JMS.

GigaSpaces are used in several very large banks running very high throughput both financially and volume-wise, I can't name the banks for the usual reasons but this is the way things are slowly moving, light-weight, distributed, transactional, component-based applications; It integrates well with Servlets, EJBs, JMS, WebServices and .NET. Many people have used Jini without knowing it, it's so light-weight you'd never know, it's often used to cluster J2EE applications. One of the best ways to use JMX is through Jini. Jini/JavaSpaces has proven scalability and stability, it's older than J2EE and compliments it beautifully, for clarity, they can be used together.

Take a look at the JavaSpaces side of GigaSpaces not the JMS or caching APIs, those are just a plus.

If anyone has some serious questions about this technology, I've implemented it in several banks and financial institutions and would be very happy to answer questions without a sales swing on it since I don't work for or own any shares in GigaSpaces. I must admit that they have bought me the odd beer and if I make it back to Bremen one day I'd hope for the same from the SwiftMQ guys. :-)

-John-

Thanks for the info John. I've been looking at this technology for a few years. I've not yet been able to convince anyone to use it as of yet. Came close once. I currently have an excellent candidate application.

They have their database and Web server (and maybe a app server) and that is how they build all apps. If it doesn't scale - add more dbs and web servers and app servers - or it is the apps fault. Sometimes they use JMS.

They think they have no need for a Grid (or just Jini/Javaspaces) because they aren't doing mathmatical stuff.

And they don't want to have their developers learn all that extra stuff. If a db and a web server suffice for most apps why have that one app that is different (they say)?

Hopefully I can use your info to convice them to think otherwise. I have been passing around the Computerworld article about Orbitz's usage of Jini.

(...) I currently have an excellent candidate application.They have their database and Web server (and maybe a app server) and that is how they build all apps. If it doesn't scale - add more dbs and web servers and app servers - or it is the apps fault. Sometimes they use JMS(...)

If its a regular j2ee app then you upscale and widescale your serverpark. This is a good option. No need to use grid software. This only makes thing more complicated than it needs to be.

(...) I currently have an excellent candidate application.They have their database and Web server (and maybe a app server) and that is how they build all apps. If it doesn't scale - add more dbs and web servers and app servers - or it is the apps fault. Sometimes they use JMS(...)

If its a regular j2ee app then you upscale and widescale your serverpark. This is a good option. No need to use grid software. This only makes thing more complicated than it needs to be.

Depends on how you define "regular". Depends on how you define "grid software". It doesn't necesarily make it more complicated. In fact, from what I have seen, it reduces complexity. Or at least elimates other major issues and thus is well worth any added complexity.

I'm sure hardware and app server vendors think upscaling and widescaling your serverpark is a good option.

John thanks for your comments.
Indeed I think that the main value of what we provide is layered around the space technology and the fact that it combines both messaging and data sharing capabilities.

Nortel had just presented their case study for their new generation CTI application based on GigaSpaces in the latest Jini community event in London.
One of the interesting facts was the time and effort they spent (9 months) on the evaluation of data base and messaging technologies for their next generation application. One way to view their application is as a distributed state machine. The interesting fact about state machine is that it requires a combination of both data sharing and messaging capabilities. Our unique p2p clustering approach enabled us to present the ability to process x1000 of events per second on a single CPU Intel based machine.
GigaSpaces was chosen after this evaluation process not just because of the performance but also because of the simplicity of the space as a programming model as well as our ability to provide out of the box integration with both C++ and .Net interfaces.

Nortel presentation can be viewed here .

The abstract for the presentation (look for Nortel) can viewed here

As john mentioned the idea behind the new GigaSpaces EAG is not to introduce yet another JMS, Caching or In memory data base implementation but provide a unique approach for integrating those core components under a single cluster, runtime and technology - JavaSpaces.
This approach enabled us to bring the value of grid (on demand scalability, efficiency and redundancy using low cost commodity resources) to main stream applications.

Nati S.
GigaSpaces CTO

Just to touch on the Globus reference.

Fundamentally Globus and Jini provide two different propositions. Globus seeks to define grids within the sphere of WS and thus XML as the main communication mechanism. Jini on the other hand prefers to define grids using Java and java objects as the communication mechanism. On the internet and WAN/VPN, Web Services are, IMO, a good idea because there is a boundary change either a change in network, language or platform where such a boundary change necessitates the move from a language's internal format to a string format. However on a LAN web services just seems far too much overkill. Generally, I would think that if you need any kind of grid on a local network, the constant encoding & decoding to XML would be more of a hindrance than a help, plus the fact that although you may use SOAP, actually defining an XML schema _is_ defining a protocol because each side of the wire will need to understand how to manipulate the data encoded.

With Jini, serialization, mobile code, and all the service proxy patterns you can do a hell of a lot of things such as
deploying the full service (yes that means no remote calls whilst you use the service) into the client from the LUS, without having to know about the implementation at compile time, just the service interface. Also until WS can reliably address the self-healing, network fallacies and partial failure scenarios that need to be fully understood in grid computing, it will cause a number of problems.

--Calum

looks like quite a feature list. The that interest me the most is the regexp and sql query features. man i wish I had a 50 node cluster to play with this stuff. Even if I could afford it, I would have no place to put it.

Have you guys tried deploying this on IBM's grid environment? You probably have your own cluster to play with, just curious.

Peter,
  You can get the Grid IBM is building on for $0 (http://www.globus.org/). It isn't the same as what GigaSpaces is doing or what TheMindElectric was doing with Gaia (is that Fabric now?).

Now all you need is time and hardware. :)

Peter,  You can get the Grid IBM is building on for $0 (http://www.globus.org/). It isn't the same as what GigaSpaces is doing or what TheMindElectric was doing with Gaia (is that Fabric now?). Now all you need is time and hardware. :)

yeah, I thought about signing up for some grid time on IBM's system, but man I'm already sleep deprived enough as it is :) Maybe I should call up the MIB and figure out how they go without sleep.

Maybe I should call up the MIB and figure out how they go without sleep.

I saw on Desperate Housewives (no, I don't watch it. At least not until football is over. :) ) that if you take ritalin and you don't need it, it will be almost as good as no sleep. :)

Maybe I should call up the MIB and figure out how they go without sleep.

I saw on Desperate Housewives (no, I don't watch it. At least not until football is over. :) ) that if you take ritalin and you don't need it, it will be almost as good as no sleep. :)

Desparate Housewives? Did the wife make you watch it. The things men have to do to please their wives. I'm sure there's way to tie the joke in with Gigaspaces, but my two brain cells are non-functioning at the moment.

Peter,  You can get the Grid IBM is building on for $0 (http://www.globus.org/).

Mark -

You might also want to check out JGrid - another $0 grid framework (http://jgrid.jini.org

Regards

--Calum

Just wanted to add my congratulations on the release.

You might also want to check out JGrid - another $0 grid framework

You're talking about apples and oranges here. If JavaSpaces is the API and model you've decided on, then you'll probably end up with either Blitz or Gigaspaces.

Peace,

Cameron Purdy
Tangosol, Inc.
Coherence: Shared Memories for J2EE Clusters

You might also want to check out JGrid - another $0 grid framework

You're talking about apples and oranges here. If JavaSpaces is the API and model you've decided on, then you'll probably end up with either Blitz or Gigaspaces.Peace

Probably as a result of Peter's and my side conversation. I did mention though that GigaSpaces(and thus Jini/JavaSpaces) was different then IBM's Grid (and thus Globus).

Sorry to take the spot light off of GigaSpaces. I think it is a great product based on a great but under-used technology. I can think of another such product :).

did we accidentaly hijack the thread. that was completely by accident.

Peter,  You can get the Grid IBM is building on for $0 (http://www.globus.org/).

Mark -You might also want to check out JGrid - another $0 grid framework (http://jgrid.jini.orgRegards--Calum

actually we aren't talking about software. IBM has a grid environment and developers can get free time on it. I don't have the link handy, but IBM has a setup where you can get time on their grid cluster to do testing and experiments.

peter

Have you guys tried deploying this on IBM's grid environment?

Were actually partnering with IBM on several Grid projects. One of them is for one of the big financial exchanges organization in the US.

Nati S
CTO GigaSpaces

Have you guys tried deploying this on IBM's grid environment?

Were actually partnering with IBM on several Grid projects. One of them is for one of the big financial exchanges organization in the US. Nati SCTO GigaSpaces

I like that you said "one of the big". Since there was so much consolidation in the industry this year, it would be pretty easy to guess :)

then again, it's probably easier to ask one of my colleagues.

How well can GigaSpaces handle contention for a specific entry in a space? (where the space is very active and distributed across a large cluster)?

Say for example:

- You have X & Y entries being added to the space very quickly (10000's/second).

- You have a large cluster of clients, half contending for the most recently added X entry, the other contending for Y entries (nb. To "take" those entries from the space, not "read").

Could the clients "take" and process the entries as quickly as they were being added? How big a Gigaspaces cluster could you run this on? Would you need special hardware?

Matt,

The GigaSpaces EAG provides an “optimistic-locking” implementation that allows a multitude of concurrent clients to access the same entry simultaneously. Updates will be successfully done only for the client holding the latest version. All the rest will need to resolve a version conflict. See:
http://www.gigaspaces.com/docs/doc/Concurrency_space_locking_and_blocking.htm

The EAG is tested with > 10 million entries over 10> nodes over windows/Linux/Solaris environments.

Read/Take/Write/Notify performance in embedded mode can reach 50,000 operations per second on 4 way CPU blade. In remote mode is it 5K per/sec for the same operations.

Please note that this product has been in mission-critical deployments for years under very stringent requirements.


Regards,
    Shay
GigaSpaces Product Manager

Need some more information about how GigaSpaces handle object serialization, or just workaround this issue?
Say two nodes, A and B, each has 10000 different entries. A biz logic has to aggregate 10000 entries, which unfortunately evenly reside on A and B. This logic is regular Java impl, which means it need complete on one node (either A/B), will GigaSpaces deserialize/serialize the entries from one box to another?
Or GigaSpaces is always replicating the content cross all nodes? How entries can be splitted/partitioned in GigaSpaces world?
Object serialization/deserializatino doesn't come cheap at all. I think Gemstone once achieved fast replication through their own JVM.

Need some more information about how GigaSpaces handle object serialization, or just workaround this issue?Say two nodes, A and B, each has 10000 different entries. A biz logic has to aggregate 10000 entries, which unfortunately evenly reside on A and B. This logic is regular Java impl, which means it need complete on one node (either A/B), will GigaSpaces deserialize/serialize the entries from one box to another? Or GigaSpaces is always replicating the content cross all nodes? How entries can be splitted/partitioned in GigaSpaces world?Object serialization/deserializatino doesn't come cheap at all. I think Gemstone once achieved fast replication through their own JVM.

One of the principles of Grid is to divide the work when possible. A common form is Single Process Multiple Data. If I understand the scenario correctly, it should be feasible to aggregate at node A and B, then combine the result. The trick obviously is if the aggregation can be done that way. If it's a simple sum, weight, average or standard deviation, i would think you can split the work. On the otherhand, if you're talking about calculations that need both datasets like Median, than I would think you'd have to serialize some of the data. But the gigaspaces guys would know better how it should be handled.

Sheng,

Your questions are indeed fascinating and something that we come across quite frequently.

In general, entries are not serialized when sent to the space. The data is "stripped" from the entries and placed into a GigaSpaces transport packet that then can move in the network. When working in embedded mode, no transmission is involved. In fact - embedded mode + a clustered configuration is the most efficient way to get the best performance. Our replication technology can efficiently move entries, synchronously or asynchronously, from one node.

Serialization is done only when entries utilize user defined classes. There is an option to avoid this when running in a no-serialization mode - speeding up performance. See more info in the product documentation.

Regarding partitioning - The cluster load balancing policies provide different partitioning schemes. We have hash based, round robin and other built in algorithms. You can write different types of entries in one call (writeMultiple) into the space and these will be partitioned across the different nodes based on their content.

A common parallel-compute model approach is to have different workers using these different nodes. This is how you will have a "smart" grid environment whereby data is partitioned across the relevant node where specific workers can "work" against this data based on business logic.

For more info, please send email to support at giagspaces dot com.

Regards,
   Shay
     GigaSpaces Product Manager

For more info, please send email to support at gigaspaces dot com.

Shay