Persistence Ships EdgeXtend 2.0 O/R Distributed Cache

Discussions

News: Persistence Ships EdgeXtend 2.0 O/R Distributed Cache

  1. Persistence Software this week began shipping EdgeXtend 2.0, a major revamp to its O/R mapper, real-time distributed cache product. New features include cache-only queries and improved indexing, guaranteed cache synchronization, bulk updating via a value object pattern implementation, and more. EdgeXtend supports Weblogic and Websphere via JCA.

    One of the main value propositions of EdgeXtend is that it can maintain a distributed cache across a WAN (diff. geographical locations), not just a local network (like most clustered producst).

    Check out Persistence EdgeXtend.

    For who have followed Persistence over the years, I have learned from them that the J2EE certified PowerTier Application server is no longer being sold, but is being supported for existing customers.

    Threaded Messages (20)

  2. synchronization[ Go to top ]

    what I remember about persistence was their c++ ejb server. when we evaluated distributed cache software I didn't know about their edgextend so we evaluated gemstone and coherence. gemstone supported shared memory caching. coherence had the clustering features, also for wans. both are doing jcache, also they both did the cache queries and indexing and synchronization.
  3. Persistence seems to have steadily made their caching and cache synchronization fancier over the years. Anybody have a succinct technical comparison of EdgeXtend, Gemstone, and Coherence? No marketing hype please.

    Coherence seems to be only clustered caching whereas Persistence is that + O/R mapping and other bells and whistles.
  4. Anybody have a succinct technical comparison of

    >EdgeXtend, Gemstone, and Coherence?
    >
    >Coherence seems to be only clustered caching
    >whereas Persistence is that + O/R mapping and
    >other bells and whistles.

    I don't know Gemstone well enough to comment, but Tangosol's Coherence is basically a distributed HashMap. put() on one machine, and get() on another, and both will work with the same object. If all you need is that, it's a fine product.

    EdgeXtend is an object-relational persistence mechanism with automatic caching and continuous synchronization built-in. Cached objects are transactionally synchronized with the data store, automatically, and using our own patented (lightweight) algorithms, rather than depending on heavyweight XA mechanisms. Synchronization can be done using our own messaging, or via enterprise-class messaging such as Tibco, MQSeries, and the like.

    So if you have high performance requirements AND you must always be in sync with the database, AND you want to lighten your development effort with O/R mapping, AND you'd like continuous LAN or WAN synchronization, AND you want model-to-code development tools, you should look at EdgeXtend.

    You can read more technical details in a white paper written by The Middleware Company, called The Full-Service Persistence Layer. It's not on our web site yet, so just e-mail me a request at alderete at persistence dot com.

    Michael Alderete
    Product Line Manager
    Persistence Software
  5. Michael:

    I don't know Gemstone well enough to comment

    They have a background in OODBMS, so my guess is that underlies a lot of their current technology. They have a product called Facets that (I think) uses the OODBMS inside it, and a product called GemFire that is a shared memory cache (meaning that multiple JVMs on the same machine can "cache" their objects in an area of memory that they can all access.)

    Tangosol's Coherence is basically a distributed HashMap. put() on one machine, and get() on another, and both will work with the same object.

    Yes, Coherence is as simple to use as a Java Map. Actually, it is a Java Map, which is the same API as JCache is using as its basis.

    It also supports fully cached queries (using cluster-wide processing power) and transactions (either programmatically or via J2CA.)

    EdgeXtend is an object-relational persistence mechanism with automatic caching

    Coherence does not do O/R mapping out-of-the-box; you have to implement a cache loader to do so. Coherence is transactional, and supports write coalescing, and is very flexible for persistence implementations, but we are definitely not in the O/R business.

    (A partner of ours, SolarMetric, is in that business and has the popular KODO JDO product that works hand-in-hand with Coherence to do clustered caching.)

    Synchronization can be done using our own messaging, or via enterprise-class messaging such as Tibco, MQSeries, and the like.

    How do you guarantee transactional consistency with ansynchronous messaging, since a commit can be done on one server and at the next instant another server can be using (now) stale data that is cached on that server (because the update or invalidation is still on the wire.) BTW - I'm not doubting the technology, since I know the background of the company, I'm honestly curious if this is something you address and what approach was taken. Do you use an optimistic concurrency approach, so that the thread that used the stale data rolls back its transaction?

    Also, does EdgeExtend run inside the J2EE app server, or is it a separate process (ala PowerTier)?

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  6. How do you guarantee transactional consistency with ansynchronous [sic]messaging, since a commit can be done on one server and at the next instant another server can be using (now) stale data that is cached on that server (because the update or invalidation is still on the wire.) [...] Do you use an optimistic concurrency approach, so that the thread that used the stale data rolls back its transaction?

    Good question. There's two different cases for this, one where you're using cached objects as proxies for persistent data, and one where you're working exclusively with cached objects (and therefore the sync message must get through).

    I think your question is more about the simple case, and indeed, we're using optimistic controls, which is a straightforward and well understood approach. If you attempt to write stale data, an OptimisticControlFailure exception is thrown, and the application logic needs to handle it (most commonly by refreshing the data, and attempting the transaction again, but that's up to the developer). The patented stuff (to answer another person's question) has to do with handling out-of-order sync messages, relationship knitting, isolation of in-flight changes, and so on.

    The second case is where you want to make sure that *both* the database write *and* the sync message commit or fail together (the normal situation is that the message is sent non-transactionally after the database write commits). It's less common, but useful when you want to depend entirely on the cache.

    For this we have a new feature in EdgeXtend 2.0, guaranteed/transactional messaging. We use a pretty slick integration with the durable message facility of the messaging system AND the underlying database. Essentially, we write both the data update and the sync message to the database in one transaction, in such a way that the messaging service then picks up the sync message from the DB.

    Also, does EdgeExtend [sic] run inside the J2EE app server, or is it a separate process (ala PowerTier)?

    EdgeXtend for WebLogic and WebSphere runs in-process with the J2EE application server (though it can also run stand-alone in certain contexts). EdgeXtend for .NET (in beta) and EdgeXtend for C++ (forthcoming version, renamed from PowerTier for C++) run stand-alone.

    BTW - I'm not doubting the technology, since I know the background of the company, I'm honestly curious if this is something you address and what approach was taken.

    Not taken that way at all!

    Michael Alderete
    Product Line Manager
    Persistence Software
  7. O/R Mapping and Caching[ Go to top ]

    ObjectFrontier also provides an O/R mapping + automatic caching solution at present. This is available for entity beans (FrontierSuite for J2EE), JDO (FrontierSuite for JDO) and J2SE (FrontierSuite for J2SE). This cache comes with distributed and clustering capabilities. A new Caching API will be released to coincide with JavaOne, which will provide runtime control over the caching.

    regards

    Rajesh
    (S Rajesh Babu)
    ObjectFrontier Inc
  8. Another approach for improving both performance and scalability of a J2EE
    system via data caching is to cache at the JDBC level. This has the
    advantage that it is transparent, you can use CMP, BMP, JDO, a third part
    O/R tool or raw JDBC. You don't even need to be in an application server.

    livestore provides transparent distributed caching at the JDBC level,
    supporting both local and global (XA) transactions and providing reliable
    cache 'liveness' either through messaging using XA, leveraging the local
    transaction to provide durable messaging, or clearing cached data from remote
    nodes when a node death is detected.

    Denis Howlett
    Isocra Ltd
  9. and using our own patented (lightweight) algorithms, rather than depending on heavyweight XA mechanisms

    Hmmm. Why is XA heavyweight? I mean, you need 2 phases to commit a tx on more than 1 data sources. There is no other way, IMHO.

    Can you shed a light on your patented algorithm?

    -- Andreas
  10. XA can be heavy in some implementations ... see PetStore Round I for an example ;-)
  11. XA can be heavy in some implementations

    The question is what is the alternative? Working on a half committed tx (since it's processed async) and have to abort subsequent tx (as you said), because the first one was aborted in the meantime?

    I've seen some products claiming to have clustering with replicated stores but don't do that with XA. They do the replication async in a background thread. If the primary collapses and they switch over to the secondary and the log stream wasn't fully replicated, they have an inconsistent state on the secondary and can't work with it.

    It would be very interesting to know how the Persistence algorithm solves this. I'd certainly one of the first to license that technology.
  12. I've seen some products claiming to have clustering with replicated stores but don't do that with XA. They do the replication async in a background thread. If the primary collapses and they switch over to the secondary and the log stream wasn't fully replicated, they have an inconsistent state on the secondary and can't work with it. It would be very interesting to know how the Persistence algorithm solves this. I'd certainly one of the first to license that technology.

    You have exceeded my technical competence to answer. Congratulations, you stumped the marketing choad! ;-)

    The serious answer is, it depends on the system architecture, transaction requirements, etc. We support the most common things customers want to do out-of-the-box, and do other things with some professional services. If I understand your question, our answer is something we call one-and-a-half-phase commit. I'll see if I can't get one of our prof. services consultants to chime in here with a more complete answer.

    Michael Alderete
    Product Line Manager
    Persistence Software
  13. costs?[ Go to top ]

    how much does it cost and is it per developer or per server?
  14. What if you have a trigger on a table in the source database that
    modifies a record in another table. Will your product be able to
    realize that this record has changed, and invalidate the object
    in the cache? Or do you require that all changes to the object
    go through your product? (Not cache-to-cache sync, but db-to-cache
    consistency).

    With Oracle 9i, you can register for update events to track changes.
    Orace 8i doesn't support this easily. Some third party products read
    the redo (transaction) log files to figure this out, but the only
    products I know of that do this address replication.
  15. What if you have a trigger on a table in the source database that modifies a record in another table. Will your product be able to realize that this record has changed, and invalidate the object in the cache? Or do you require that all changes to the object go through your product? (Not cache-to-cache sync, but db-to-cache consistency).

    Performance is somewhat better (and the systems diagrams are certainly prettier ;-) if all changes go through EdgeXtend, but we can handle the case where that's not possible.

    This typically requires guidance from our professional services folks. As you point out, there are differences in the capabilities of different databases, so the specific implementation varies. We have some interesting ideas of how to make this more uniform while also increasing the cleanliness and performance of the solution. That'll come in a future version of EdgeXtend.

    Michael Alderete
    Product Line Manager
    Persistence Software
  16. Distributed caching is really cool, but could lead to horrible unreproducable bugs. I'm wondering in which contexts the complexity is really justified?

    I you are mainly doing OLTP and lots of updating, then things have to get to the database anyway.

    If you have really read only lookup tables then that is easy to distribute, no need for tools.

    So you would need an application with a large amount of reading from data that is updated fairly often but not too often. Not too common I would think. (Especially given the use of warehouses.)

    And how does it compare to RDMS clustering techiques?

    In general, unless you really need the power, I'd keep things simple.

    Anthony
  17. Distributed caching is really cool, but could lead to horrible unreproducable bugs. I'm wondering in which contexts the complexity is really justified? I [sic] you are mainly doing OLTP and lots of updating, then things have to get to the database anyway. If you have really read only lookup tables then that is easy to distribute, no need for tools. So you would need an application with a large amount of reading from data that is updated fairly often but not too often. Not too common I would think. (Especially given the use of warehouses.)

    While it is not for everyone, I believe the characterization of "not too common" is wide of the mark. Many of our customers have spent multiple millions of dollars with us, because the "simple" way was going to be hidiously expensive. Read one of our white papers, the Aberdeen Group profile, for some examples and hard numbers.

    You're correct about OLTP needing to go to the database anyway, but this is actually an area where Persistence shines. Because our entire focus is on providing object-oriented applications with access to persistent data stored in a relational database, the notion that updates go back to the database is baked into the DNA of our products. We were writing stuff back to the database in our original product, far before we added caching to the core engine.

    Just because updates need to go back to the database doesn't mean there's no value in having a cache. Our single largest solutions segment is the trading of financial instruments (equities, fixed income, etc.). Our transactional caching provides performance enhancements and geographical distribution capabilities which are enabling deployments of systems globally.

    Another major segment for us is transportation and distribution. A global shipping company is tracking planes, trains, and trucks, orgins and destinations, personnel, packages, and routes. Millions of them, every day, and they are constantly changing due to weather, vehicle malfunction, personnel issues, and customer requests.

    Here it's not so much the volume of changes, but their unpredictability, and the consequences of not handling them. If a Federal Express or DHL or Sabre can improve their efficiency by 1/10% (that's one tenth of one percent), it's a multi-million dollar savings every year.

    When you start to look around, the number of applications or systems that can benefit from the distributed data management we provide (something we call Data Services) is absolutely enormous.

    It's not every system that can benefit, usually just the largest 5-10%, but those that can represent millions and millions of users, and frequently generate multiple millions in savings.
  18. Michael,

    It's not every system that can benefit, usually just the largest 5-10%, but those that can represent millions and millions of users, and frequently generate multiple millions in savings.

    First of all, even in a single-server scenario, caching makes a huge difference. There is no more cost-effective way to get more bang for the buck from your servers than to use caching effectively. That doesn't mean buying a product; you can cache using built-in features in the app server, your own frameworks, some of the Java classes (like Hashtable), the upcoming JCache standard or commercial products like those discussed here.

    When you scale up to more than one JVM (either on one or multiple servers), it becomes even more important to use caching, because each additional JVM adds that much more load to the back end systems, and thus while the JVMs scale very well, the back end systems become a huge and expensive bottleneck. Again, this doesn't always mean buying caching software, since a large amount of the benefit can be had as the result of good architecture or even just caching read-only data. OTOH, when it comes to distributed caching of data that is expensive to obtain, or data that can change (esp. transactionally), that's when these products become worth their virtual weight in gold.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Easily share live data across a cluster!
  19. It's not every system that can benefit, usually just the largest 5-10%, but those that can represent millions and millions of users, and frequently generate multiple millions in savings. blockquote>First of all, even in a single-server scenario, caching makes a huge difference. There is no more cost-effective way to get more bang for the buck from your servers than to use caching effectively. [...]

    Obviously, you're correct that caching pays large dividends even in the simple case. I was responding more to the question about the additional complexity that's introduced with a distributed shared cache, and when is it worth it.

    Basically, there's a number of different "zones" for systems, usually related to system size, or the criticality of the data/transactions. For the lower zones, built-in caching is really useful, and more sophisticated solutions are overkill. In the middle zones, you might want to build your own caching solution, or buy an add-on product that's extremely capable while still, relatively speaking, low cost.

    At the high end of systems, where there's already a great deal of complexity, a product like EdgeXtend is not going to add that much additional complexity, and is going to provide large performance gains and major cost reductions.

    Indeed, you could argue that the somewhat more complicated systems architecture is more than offset by the reduction in complexity of the physical infrastructure. We have a number of customers saving more than $1mm/year in decreased operational costs, because the overall number of servers is reduced, which means licensing, maintenance, DBAs, and other IT operating expenses go down significantly.

    Frankly, in the lowest zones, EdgeXtend is potentially overkill. The app server probably has all the capabilities you need, and you should investigate how to best use those.

    Without slighting any of the other vendors who are posting in this discussion, in the higher end zones, we certainly believe that Persistence EdgeXtend provides a compelling solution. And for certain types of systems (like equities trading), we like to believe that we stand alone.

    In the middle zones, the customer needs to do an appropriate amount of investigation, and select a product which provides the right mix of features, services, integration points, and development style which matches best with their existing systems, expertise, and processes. Sometimes EdgeXtend will be the best product, sometimes you'll choose something else.

    Diversity of solutions is good, because there certainly is diversity of systems designs and designers in the market. One size does not fit all!
  20. not promising[ Go to top ]

    you made it sound hard to use and expensive. we've got piles of software like that already sitting on shelves doing nothing.
  21. Re: not promising[ Go to top ]

    you made it sound hard to use and expensive. we've got piles of software like that already sitting on shelves doing nothing.

    Like just about everything about J2EE, EdgeXtend requires a certain amount of expertise to use. If you're most comfortable using JSP and JDBC, then EdgeXtend will indeed seem hard.

    But if you're a reasonably experienced J2EE developer with some decent architecture skills then you can learn to use EdgeXtend with a modest amount of effort -- just as it would take to learn the optimal use of any sophisticated new tool.

    I'm trying to avoid the "marketing hype" here, because this is a developer site, not a trade show booth. But in sales calls, we can often take a prospect's existing application object model, open it in our tools, and generate the persistence code for their application before the sales call is over. When you know what you're doing, EdgeXtend can be beyond easy to use!

    As for sitting around on shelves, we've a slew of customers who use it every day. One customer transacts $3 billion/day on an equities trading platform architected around Persistence. Another company depends on Persistence technology to help them manage shipment of millions of packages a day. There's a case study on our web site, a little old, but FedEx is still a major customer.

    So, if I've made it sound like it's not the best tool for a straightforward e-commerce site, then that's what I intended. But if I made it sound like no one uses our product, then that's absolutely not true. A substantial portion of our revenue comes from repeat customers -- many have been with us for 5 years and more. I think satisfied customers speak volumes. Hope you agree!

    Michael Alderete
    Product Line Manager
    Persistence Software