-
Slice: OpenJPA for distributed persistence (15 messages)
- Posted by: Pinaki Poddar
- Posted on: January 08 2008 06:24 EST
Slice is a plug-in extension for OpenJPA to interact with distributed databases within the same transaction. An early access version of Slice is available at http://people.apache.org/~ppoddar/slice/site/ Slice is being developed under the aegis of Apache Labs. The early access version executes queries in parallel across database slices, allows customized data distribution policy and detects closure of new persistent object graph automatically to honor collocation constraint. Collocation constraint refers to the core limitation of not being able to establish relationship across database slices. In O-R mapping paradigm, this restriction implies that closure of any persistent object graph must reside in the same database. Slice detects the closure when object relationships are marked for transitive persistence. However, user-defined data distribution policy must honor collocation constraint for any relationships that are not cascaded for persistence. Slice tracks the original database slice for all existing instances, so that any update gets committed to the appropriate database.Threaded Messages (15)
- Re: Slice: OpenJPA for distributed persistence by Nikita Ivanov on January 08 2008 14:46 EST
- Looks similar to Hibernate Shards by Jim Bethancourt on January 08 2008 16:01 EST
- Re: Looks similar to Hibernate Shards by Pinaki Poddar on January 08 2008 19:11 EST
-
Two-phase commit is a bottleneck by Ilya Sterin on January 09 2008 12:45 EST
-
Re: Two-phase commit is a bottleneck by Pinaki Poddar on January 09 2008 01:57 EST
-
Re: Two-phase commit is a bottleneck by Ilya Sterin on January 09 2008 03:36 EST
- Re: Two-phase commit is a bottleneck by Michael Galpin on January 11 2008 08:14 EST
-
What about HA? by Jason Carreira on January 09 2008 10:33 EST
-
Re: What about HA? by Pinaki Poddar on January 10 2008 11:44 EST
- What about HA? by Georgi Naplatanov on June 26 2008 04:39 EDT
-
Re: What about HA? by Pinaki Poddar on January 10 2008 11:44 EST
-
Re: Two-phase commit is a bottleneck by Ilya Sterin on January 09 2008 03:36 EST
-
Re: Two-phase commit is a bottleneck by Robert Greene on January 10 2008 12:30 EST
- Optimized Two-phase commit by Pinaki Poddar on January 10 2008 12:44 EST
-
Re: Two-phase commit is a bottleneck by Pinaki Poddar on January 09 2008 01:57 EST
-
Two-phase commit is a bottleneck by Ilya Sterin on January 09 2008 12:45 EST
- Re: Looks similar to Hibernate Shards by Pinaki Poddar on January 08 2008 19:11 EST
- works across WAN? by James Imber on January 09 2008 03:10 EST
- Re: works across WAN? by Pinaki Poddar on January 09 2008 13:46 EST
- Re: works across WAN? by Robert Greene on January 10 2008 12:16 EST
- Re: works across WAN? by Pinaki Poddar on January 09 2008 13:46 EST
-
Re: Slice: OpenJPA for distributed persistence[ Go to top ]
- Posted by: Nikita Ivanov
- Posted on: January 08 2008 14:46 EST
- in response to Pinaki Poddar
Very interesting... I will certainly look at it. Thanks! Nikita Ivanov. GridGain - Grid Computing Made Simple -
Looks similar to Hibernate Shards[ Go to top ]
- Posted by: Jim Bethancourt
- Posted on: January 08 2008 16:01 EST
- in response to Pinaki Poddar
At first glance, it looks very similar to the Hibernate Shards project donated to Hibernate by Google -- http://www.hibernate.org/414.html It would be interesting to see how they compare in terms of functionality and other things as well. One of the big reasons Google donated Shards was to help get it fleshed out -- I wonder if this fills any of the voids they're needing to fill? Cheers, Jim -
Re: Looks similar to Hibernate Shards[ Go to top ]
- Posted by: Pinaki Poddar
- Posted on: January 08 2008 19:11 EST
- in response to Jim Bethancourt
Slice is inspired by Hibernate Shards. I will leave comparison to users :) Few observations: i) The user API to Slice is pure JPA. ii) Slice is based on OpenJPA (http://openjpa.apache.org) as JPA runtime. Given my association with OpenJPA (am a committer), I could provide some user-friendly features. For example, data distribution policy is only to be specified for newly persistent object graph roots. The instances loaded via query or find() are tagged with the original database automatically. iii) Slice also provides two-phase commit across the slices provided the underlying database resources are XA-complaint. -
Two-phase commit is a bottleneck[ Go to top ]
- Posted by: Ilya Sterin
- Posted on: January 09 2008 12:45 EST
- in response to Pinaki Poddar
The problem with all of these replication technologies is that it's not even close to achieving linear scalability. Yes, I know there is no such thing as linear scalability, but with 2PC, the limit is quickly reached where performance really suffers. There is only so many nodes you can add until 2PC becomes a bottleneck. The more interesting replication tools these days are replication toolkits build on top of group communication tools. Besides less overhead and near linear scalability, they provide benefits of WAN replication and handling of network partitioning and joins. There is a site node limit at which performance begins to deteriorate, but the limit is far above that of 2PC. Ilya -
Re: Two-phase commit is a bottleneck[ Go to top ]
- Posted by: Pinaki Poddar
- Posted on: January 09 2008 13:57 EST
- in response to Ilya Sterin
It may be lack of clarity in my posting, but Slice is not a 'replication technology'. Slice targets the imminent use case where (non-replicated) data is partitioned across many databases -- many large-scale, enterprise apps are employing database architecture of similar nature. I concur with your concern on scalability of 2PC protocol. On performance I will mention few relevant aspects that may provide some comfort: i) Slice executes all database operations in parallel ii) Transaction management at database slice level is configurable. For example, if the application is read-only, one can configure Slice to ignore 2PC completely but still have the benefit of issuing a single query to pull data from many data sources in a single in-memory cache/persistence context. -
Re: Two-phase commit is a bottleneck[ Go to top ]
- Posted by: Ilya Sterin
- Posted on: January 09 2008 15:36 EST
- in response to Pinaki Poddar
It may be lack of clarity in my posting, but Slice is not a 'replication technology'. Slice targets the imminent use case where (non-replicated) data is partitioned across many databases -- many large-scale, enterprise apps are employing database architecture of similar nature.
Ah, agree. That's a very interesting concept. I guess one can say db scalability can be achieved through either active/active replication and/or data partitioning. Where replication is a more generic approach, partitioning scalability has to take into consideration application semantics. One could argue that same levels of scalability can be achieved if employed correctly. I'll definitely take a look at Slice in the next few weeks. Ilya
I concur with your concern on scalability of 2PC protocol. On performance I will mention few relevant aspects that may provide some comfort:
i) Slice executes all database operations in parallel
ii) Transaction management at database slice level is configurable. For example, if the application is read-only, one can configure Slice to ignore 2PC completely but still have the benefit of issuing a single query to pull data from many data sources in a single in-memory cache/persistence context. -
Re: Two-phase commit is a bottleneck[ Go to top ]
- Posted by: Michael Galpin
- Posted on: January 11 2008 20:14 EST
- in response to Ilya Sterin
One could argue that same levels of scalability can be achieved if employed correctly.
Hmm, you could argue that, but you would be wrong. In replication you have multiple databases, so you can handle lots of requests. That's great. Replication scales horizontally. However, all of your databases have the same amount of data. So as the amount of data increases, query performance and CPU usage per query grow rapidly. You can index all you want, at some point you will have to throw more hardware at the problem. And this will be especially expensive since you have many monster database servers at this point. Replication does not scale vertically. Partitioning/federation scales both horizontally and vertically. You can scale to handle more concurrent requests because you spread them across machines, similar to replication. You can scale to handle more overall data because you can manage the amount of data in each database, since no database has to contain all the data. An important flaw in Ruby on Rails is that it does not work with partitioning/federation. This was also a flaw in Hibernate that Shards addressed. -
What about HA?[ Go to top ]
- Posted by: Jason Carreira
- Posted on: January 09 2008 22:33 EST
- in response to Pinaki Poddar
How does this handle data redundancy for HA? Seems like if you're just partitioning the data across multiple databases, you're adding more single points of failure and bringing your availability way down. -
Re: What about HA?[ Go to top ]
- Posted by: Pinaki Poddar
- Posted on: January 10 2008 11:44 EST
- in response to Jason Carreira
Availability concern is addressed via 'graceful degradation'. Slice can be configured to control what happens when any of the configured database slices becomes unavailable. The property setting is The current options are pretty limited a) true: to continue working with existing slices. The queries will operate only active slices. The data distribution will fail if an inactive slice is targeted. b) false: to give up on failure But we do want to advance from this boolean simplicity:) Other options are in our roadmap i) configure slice A as backup of slice B (and can form a chain of backups that way) ii) running low-priority threads to attempt reconnection to inactive slices The idea behind posting to this forum is to gather concerns such as yours, so that we can factor these features in. You are welcome to suggest other options. Thank you for reading. -
What about HA?[ Go to top ]
- Posted by: Georgi Naplatanov
- Posted on: June 26 2008 16:39 EDT
- in response to Pinaki Poddar
Availability concern is addressed via 'graceful degradation'. Slice can be configured to control what happens when any of the configured database slices becomes unavailable. The property setting is
It's a very interesting. Let's assume the the correlation between instances of particular object and number of nodes are M:N. N = 1 - distributed database or partitioning N > 1 - replication, backup and distributed db (depends on configuration).
The current options are pretty limited
a) true: to continue working with existing slices. The queries will operate only active slices. The data distribution will fail if an inactive slice is targeted.
b) false: to give up on failure
But we do want to advance from this boolean simplicity:) Other options are in our roadmap
i) configure slice A as backup of slice B (and can form a chain of backups that way)
ii) running low-priority threads to attempt reconnection to inactive slices
The idea behind posting to this forum is to gather concerns such as yours, so that we can factor these features in. You are welcome to suggest other options.
Thank you for reading. -
Re: Two-phase commit is a bottleneck[ Go to top ]
- Posted by: Robert Greene
- Posted on: January 10 2008 12:30 EST
- in response to Ilya Sterin
....accidentally posted as response to wrong thread. Couldn't do a rollback :-) Also, as comment to some of the ongoing discussion, it is worth pointing out that data partitioning although not the same as data replication does not expressly eliminate the need for 2pc. That is an implementation limitation, i.e. not to allow cross node references. If you introduce those, then you introduce the need to 2pc. Plus if you have logical relationships that span node specific graphs, i.e. updated in same transaction, then again, you need 2pc. There are many more issues here, but there's a book on it somewhere ;-) Anyway.... 2pc can be a bottleneck, but in transactional applications the buffer flushing, disk i/o etc often have a much more damaging effect than the protocol itself. 2pc is more apt to be the bottleneck in applications where the database is configured to use asychronous I/O ... non-blocking commit, thus eliminating the disk I/O activities ( might not be a database, but a replicated memory space with a database behind it ). Plus, there are newer advances in 2pc protocols ( extended 2pc ) that allow optimizations. Example, just because you're connected to N nodes does not mean that any given transaction involves N nodes. Plus, a transaction may have certain nodes that are read only or work with data (objects) that are read-only in a particular node. All of these circumstances serve to provide areas of optimization to extend where the overhead of the protocol becomes damaging. Plus, there is also the ability to use asychronous network calls to do protocol handshaking in parallel. So, then theoretically, the overhead comes when the first node is connected and subsequent nodes are handshaked in parallel so the worst response in time comes from the slowest node .... not the summation of the nodes. The issue is whether or not your using a resource manager/ transaction coordinator which is aware of and optimized for these cases. -Robert Versant Corp -
Optimized Two-phase commit[ Go to top ]
- Posted by: Pinaki Poddar
- Posted on: January 10 2008 12:44 EST
- in response to Robert Greene
Currently, the commit does ignore database slices that have nothing to commit. The database operations run in parallel -- so time taken is *not* sum(slice time) but max(slice time) -- as you mentioned. Cross-node references is a hard problem. In Slice parlance, we referred this limitation as 'collocation constraint'. -
works across WAN?[ Go to top ]
- Posted by: James Imber
- Posted on: January 09 2008 03:10 EST
- in response to Pinaki Poddar
I wonder if it will work across WAN... -
Re: works across WAN?[ Go to top ]
- Posted by: Pinaki Poddar
- Posted on: January 09 2008 13:46 EST
- in response to James Imber
If the database slices are accessible and operable over WAN via JDBC, then Slice will work. At least Slice will not introduce any more restriction than what OpenJPA in terms of database accessibility. Used it from my laptop at Texas connected over wireless to transact against remote databases that are within corporate firewall. Please note that Slice allows you to configure each slice with its own parameters (URL, user/password, JDBC driver, connection properties etc.) -
Re: works across WAN?[ Go to top ]
- Posted by: Robert Greene
- Posted on: January 10 2008 12:16 EST
- in response to Pinaki Poddar
2pc can be a bottleneck, but in transactional applications the buffer flushing, disk i/o etc often have a much more damaging effect than the protocol itself. 2pc is more apt to be the bottleneck in applications where the database is configured to use asychronous I/O ... non-blocking commit, thus eliminating the disk I/O activities ( might not be a database, but a replicated memory space with a database behind it ). Plus, there are newer advances in 2pc protocols ( extended 2pc ) that allow optimizations. Example, just because you're connected to N nodes does not mean that any given transaction involves N nodes. Plus, a transaction may have certain nodes that are read only or work with data (objects) that are read-only in a particular node. All of these circumstances serve to provide areas of optimization to extend where the overhead of the protocol becomes damaging. Plus, there is also the ability to use asychronous network calls to do protocol handshaking in parallel. So, then theoretically, the overhead comes when the first node is connected and subsequent nodes are handshaked in parallel so the worst response in time comes from the slowest node .... not the summation of the nodes. The issue is whether or not your using a resource manager/ transaction coordinator which is aware of and optimized for these cases. -Robert Versant Corp