Discussions

News: Taking the load off: OSCache helps databases cope

  1. In this article, Andres March shows how OSCache can be used to save your processing requirement by storing data or content, to prevent continually re-reading it or re-generating it, including a quick walkthrough of what OSCache can do, through caching objects, entire HTTP responses, or fragments of HTTP via a JSP tag library.

    Read Taking the load off: OSCache helps databases cope

    Threaded Messages (26)

  2. I am not sure that OSCache programming model is good.

    Throwing exception to inform API user that entry in the cache is stale is bad design for me. Stale data is not exceptional situation, but part of normal behavior in the cache entry life-cycle.

    Not to mention that throwing exception is slow (to be fair, this is not a major flaw as data that is going to be cached is probably much slower to obtain, neglecting throwing/catching the exception).

    Returning a null when cache entry is stale is much better solution to me (although this prevents using null entries in the cache).
  3. There are definitely problems with using the exceptions to synchonize cache acess. Although changing such a basic part of the API would be difficult, I wouldn't count it out for a future release.
  4. Throwing exception to inform API user that entry in the cache is stale is bad design for me. Stale data is not exceptional situation, but part of normal behavior in the cache entry life-cycle.
    +1
  5. We have evaluated different (open-source) cache systems, for our project, about a year ago, and found that OSCache is far inferior to JBossCache (formerly known as TreeCache). Overall, Tangosol Coherence looked the best-of-the-breed (not OSS, though).

    From the article, it seems like, the situation has not changed much, during the last year.

    Among other things, OSCache seems to still have not figured-out cluster-wide replication. All it offers is - cluster-wide eviction.

    The author tries to make it look nice:
    Complete cache management in a cluster would need to include transactional replication of state. The granularity of this replication determines how much data will need to fly around your network. OSCache does not offer this degree of state management in a cluster. OSCache only permits coordinating flushing at this time. This feature allows you to invalidate an entry in the cache and have that invalidation propagated to other nodes in the cluster. Once the entry has been invalidated, the application will go to the database on the next request for that data. While clustered invalidation is more manual and lacks many of the features of a replicated cache, it does have one advantage. Clustered invalidation only necessitates the sending of the cache entry's key, rather than the entry's data, to all the nodes of the network. This means less data and ultimately less processing that the cluster needs to perform.

    Retrieving data from the database is always more expensive than retrieving from a cache, even if clustered. How can 20 nodes all hitting database with a mid-speed query be faster than only one of them hitting the DB and the rest taking ready-to-use value over the network? I don't think so.

    Tangosol Coherence, with its partitioned cache ideology, is a live proof that for average-size caches transmitting objects over the network. all the time, is not a big problem, really.

    Pre-last comment: in some applications, there is part of the data that is sooo slow, to query, that it is always pre-cached during the server startup. When you have a cluster, if some server is restarted, you do not want that server to try to fetch the data from the database, thus
    1) making the startup of the server extremely slow
    2) creating extreme load on the database, disturbing all other servers, using the same DB.

    In situations like this, it comes real handy the restarted server to fetch ready-to-use data from the running fellow servers in the cluster. Alas, for that to happen cache implementation gotta support replication (specifically, replication on the startup). Flush via eviction does not fly.

    Last comment: with a transactional, replicated cache you can enjoy the full transactional support (especially if an ORM tool like Hibernate "sits" on top of your cache) and that is so sweet - you do not want to deny yourself the pleasure.
  6. We have evaluated different (open-source) cache systems, for our project, about a year ago, and found that OSCache is far inferior to JBossCache (formerly known as TreeCache).

    Umm, even Gavin would probably say a year ago JBossCache was not ready. It is getting there but transactional replication is a hard problem. I would still not be confident enough to use it and I'm not alone. Don't think just because I contribute to OSCache I wouldn't be interested in giving it a try when its ready.
    Retrieving data from the database is always more expensive than retrieving from a cache, even if clustered. How can 20 nodes all hitting database with a mid-speed query be faster than only one of them hitting the DB and the rest taking ready-to-use value over the network? I don't think so.
    OSCache may have some issues but it works extremely well for our needs. I completely disagree that replication will suit all purposes or outperform clustered eviction. Believe it or not databases were designed for querying. If we were all just selecting by pk from the db it would likely be as fast as remote cache get. And applying transactional semantics to a distributed, clustered cache is at least as hard as with a database. Oh and I hate Oracle but RAC is pretty decent.

    There are definitely places where replication is far superior than eviction. But until there is a free product that I'm confident in, OSCache will give me what I need most.

    Cograts to Cameron and Coherence for a great product. I don't think in OSS software is much competition for them...yet ;)
    Last comment: with a transactional, replicated cache you can enjoy the full transactional support (especially if an ORM tool like Hibernate "sits" on top of your cache) and that is so sweet - you do not want to deny yourself the pleasure.
    You might want to ask Gavin if he thinks JBossCache is ready for prime time as a clustered query cache.

    btw, congrats to Bela and Ben on their work done so far. I must profess my ignorance with their latest code, so they can correct me where I'm wrong. of course, if it wasn't for jgroups, oscache would have to depend on JMS :(
  7. We looked at OSCache and JBossCache too and I'd agree that Jboss is marginally superior, but the guys at OS have included some useful entry points to the cache, like a taglib and a filter which makes it a more attractive option for q&d caching with low to medium volume.

    At the time of our trial, JBossCache 1.0 had just been released and we were having and seeing problems with the cache quality at that time. It also doesn't include the code to expire a value at a time (cache.add(value,1000); where 1000 is the expiration timeout), so we had to write additional code to manage that outside of the cache itself and then wrap that code with taglibs and filters to replicate the OSCache funtionality.

    For small sites that need a performance boost, OSCache is a simple option. Larger sites should look at JBoss cache, hopefully it will improve to the point where it is a realistic alternative to a commercial caching product. 1+ to Coherence, it can cope in production environments whilst still being insanely easy to get running and despite the number of open source alternatives available, no of them come close.

    PS. jstrachan created a CacheProvider for hibernate using ActiveSpaces, did anyone ever try that? (click here for info
  8. jstrachan created a CacheProvider for hibernate using ActiveSpaces, did anyone ever try that?

    James can you answer that please ? :-)
  9. Believe it or not databases were designed for querying. If we were all just selecting by pk from the db it would likely be as fast as remote cache get.
    In my limited experience, most of the queries to the DB are more complex. Alas, very rarely an application logic needs something as trivial as to load just a single, simple (no joint selects on the back-end) entity.

    Also, things that would typically get cached are complex. You do not want and can not cache everything so you usually end up caching limited amount of things, but those that are real slow. Which, by definition, means - getting them from the database is what is slow.

    At least, that's my experience and I dare not say it is the case for everybody else but, till now, I thought it was pretty typical.

    Regarding JBossCache stability - when we began using it, there, indeed, were some bugs and some performance problems that we ran into. But Bela and Ben were very cooperative in resolving them and I can not say we had much trouble with it, since.

    Regarding cache expiration: JBossCache does support eviction policies, including time-base one.

    To be fair, I do have a problem with JBossCache, which is - its architecture. At least for my needs, it is overarchitected and that gets into your way sometimes. This architectural overhead is inherited from the time JBossCache was TreeCache. JBossCache, instead of "usual" regions and objects into the regions has a complicated tree structure with nodes and HashMaps attached to the nodes. Also, if I am not mistaken, eviction can be done only on the node (whole HashMap, not a sinlge object), because of which Hibernate guys were forced to use JBossCache under hibernate, in a somewhat awkward manner where each node's hashmap, actually, contains only one object.

    Bottomline 1: OSCache, definitely, has its niche and I am not trying to bash it. Just mentioned an alternative view and hope - it did not create an offtopic.

    Bottomline 2: JBossCache, while preferable for projects like ours, has its own flaws, too.

    Bottomline 3: I wish Coherence was open-source :-) :joke:
  10. yeah, cameron. I mean c'mon help a brotha out.
    In my limited experience, most of the queries to the DB are more complex. Alas, very rarely an application logic needs something as trivial as to load just a single, simple (no joint selects on the back-end) entity.

    agreed. my point was that if you can only access objects stored in the cache by a key, it may be hard to locate multiple objects quickly, hence the query cache. but as I've said I think there are problems with the query cache in hibernate. but we still use it :(
  11. I would still not be confident enough to use it and I'm not alone. Don't think just because I contribute to OSCache I wouldn't be interested in giving it a try when its ready.

    I would second that assesment. Our performance tests using different caching packages (EHCache, OSCache, TreeCache) indicated that there were problems with TreeCache distributed transactional data replication.
  12. What?[ Go to top ]

    Believe it or not databases were designed for querying.

    But... I thought they were designed for storing data that was mapped into rows and columns from an object hierarchy. What's this "querying" idea you have?

    That sounds almost like you think databases store data or something, with an eye towards storage and retrieval, and like database vendors actually try to make those things work well.

    Come to think of it, it's a nifty idea. I bet we could work out some kind of common language to help with that, some kind of query language - like HQL or EJBQL or something, except removing the object semantics from it.

    What do you think?
  13. What?[ Go to top ]

    What a great idea but what would we call this new language? Hey, I bet we could implement this with AOP. Damn, Cameron beat us to it again.
  14. SQL AOP?[ Go to top ]

    What a great idea but what would we call this new language? Hey, I bet we could implement this with AOP. Damn, Cameron beat us to it again.

    Glad you enjoyed it ;-)

    BTW great article, Andres! Let me know if you're going to be at JavaOne ..

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Cluster your POJOs!
  15. Retrieving data from the database is always more expensive than retrieving from a cache, even if clustered. How can 20 nodes all hitting database with a mid-speed query be faster than only one of them hitting the DB and the rest taking ready-to-use value over the network?

    I'm not sure that you are correct. Consider this ( typical )
    scenario.

    --- Hardware (DNS based) load balancer ( failover handled by ip rerouting/heartbeat monitor )

    --- 20 application servers to which requests are directed ( each with it's own cache )

    1) User enters the system is directed to a tomcat server.
    2) Server fetches user details from the database and caches
    them locally.
    3) User modifies these details and they are sent to back to
    the server.

    So where in this ( typical e-commerce ) scenario do we need
    to have the cached data sent to every node in the cluster ?

    Doing so would create another version of the clustered entity bean problems where network saturation results from
    each and every change in the data state being replicated accross the cluster.

    So Irakli, I think that it is reasonable to say that it is not allways cheaper to replicate cache accross every
    node, sometimes it is cheaper and more scalable to
    only cache data on the node from which it is being accessed.

    --b
  16. Bryan,

    If I understand it correctly, your scenario makes several assumptions
    1) Load-balancer uses sticky sessions to make sure that user hits go to the same Tomcat server in the cluster.
    2) The information you cache is mostly user-related, and if something is shared with other users - that information (e.g. product prices?) is in "mostly read-only" mode (changes rarely).

    If those two assumptions are true, I think you are right and local cache would suffice/be preferred.

    But I was comparing: 1) cache replication to 2) eviction and retrieval from DB, not cache replication to local cache :)

    In any case, if my statement was overly general and looked like an attempt to declare The Truth, I apologize, because there is no The Truth :)

    cheers
  17. Bryan,If I understand it correctly, your scenario makes several assumptions1) Load-balancer uses sticky sessions to make sure that user hits go to the same Tomcat server in the cluster.2) The information you cache is mostly user-related, and if something is shared with other users - that information (e.g. product prices?) is in "mostly read-only" mode (changes rarely).If those two assumptions are true, I think you are right and local cache would suffice/be preferred. But I was comparing: 1) cache replication to 2) eviction and retrieval from DB, not cache replication to local cache :)In any case, if my statement was overly general and looked like an attempt to declare The Truth, I apologize, because there is no The Truth :)cheers

    Yep, I was assuming the use of sticky sessions, no I see
    now that there are a number of different scenarios. I too
    don't have a monopoly on "The Truth". No apology needed
    Irakli and hope you have a good weekend.

    Actually it is good that there is a little debate on this because caching is something that is always a little difficult to formulate a strategy for in advance of deployment, usually it is something that is added as an afterthought when the performance problems pop up.


    Bryan
  18. Eviction has at least two benefits:

    1. In several scenarios (for example if you manage own database transactions for Hibernate), you do not know in java methods which changes will be commited in database. In this scenario is cluster-wide replication problematic.

    2. Cluster-wide replication is often useless, because another aplication server in cluster often works for another users with another set of data. Sending information about evictions is very quick and has small impact on the net and CPU.

    There is still my personal experience. With Toplink we use non-evict cache. After insert objects into database, user saw these objects in application. There was serious bug in mapping (float numbers was mapped into int field), but user saw several hours float numbers. With evict-cache, this problem does not exist. This problem is often with java.util.Date objects mapped into database.
  19. Retrieving data from the database is always more expensive than retrieving from a cache, even if clustered.
    It must be something wrong in application level distributed cache implementation if it faster than mature distributed databases. It sounds too good to be true.
  20. Retrieving data from the database is always more expensive than retrieving from a cache, even if clustered.

    It must be something wrong in application level distributed cache implementation if it faster than mature distributed databases. It sounds too good to be true.

    Actually, it sounds pretty obvious to me.

    If my application is asking for an object and it has to get it from a database, that is going to require something like:

    (a) a connection (typically pre-established via a pool, meaning a single simple network hop over TCP/IP)

    (b) a transaction (relatively inexpensive due to read-only tx optimizations that defer the tx costs until the first write)

    (c) a query (typically prepared, meaning only the prepared statement handle and the actual "?" values have to get encoded and passed)

    (d) various security checks on the db side (usually pre-checked on the conn level if it's a repetitive prepared statement call, which means that there's a lookup happening on the prep'd statement cache)

    (e) handling the query (let's assume it's in cache, so it's a quick buffer copy to a TCP/IP output buffer for a single hop back)

    (f) parsing the TCP/IP response into a result set and converting the stream into JDBC types (very fast)

    (g) mapping the result set into Java object (very fast)

    So with a well-optimized database pulling from cache, we see a series of very well-optimized steps that in total might take a few milliseconds, and maybe 10ms in a production HA environment.

    Now compare that to using a local Java cache:
    myobject = myhashtable.get(mykey);

    Zero milliseconds.

    Since distributed cache implementations tend to asymptotically approach the in-memory access speed of a hash-table or similar data structure, it should be pretty obvious why a distributed database cannot approach the speed of a cache. In the case of OSCache, it uses invalidations to provide in-memory caches with async removal of stale data, which is sufficient for most caching of display data. There are also region-based approaches, replication-based approaches, transparent partitioning, etc. (Our Coherence software includes all the above.)

    Anyway, to summarize, getting an object from a well-tuned distributed cache is probably going to be faster than getting a JDBC connection from a connection pool (and that's just step "a"!)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Cluster your POJOs!
  21. Hey Cameron,

    isn't something missing from your "using a local Java cache" code? At least the code which checks if cache data is invalid or not? I say it because this cache check code wouldn't exist if there were no caching at all, so it should be accounted for when comparing query execution time with cache retrieving time, making it a bit greater than 0ms, am I correct? (I am not questioning if cache is faster or not)

    Regards,
    Henrique Steckelberg
  22. Hey Cameron,isn't something missing from your "using a local Java cache" code? At least the code which checks if cache data is invalid or not? I say it because this cache check code wouldn't exist if there were no caching at all, so it should be accounted for when comparing query execution time with cache retrieving time, making it a bit greater than 0ms, am I correct? (I am not questioning if cache is faster or not)

    The cache either maintains its own consistency (i.e. what Coherence does) or you accept the potential that the cache data is potentially stale.

    By definition, any data for which you do not hold a pessimistic lock for, and for which you did not read after obtaining that lock, that data is potentially stale. To put that data into an HTML page and to guarantee that the user sees up-to-date data, you have to hold that lock until the page has finished rendering on the user's browser. To guarantee that the user can take action based on that up-to-date data (i.e. that no one changes the data before the action against it is taken), you have to hold the lock until after the user takes the action.

    Generally speaking, HTTP-based applications don't do any of that, and they don't do it for a reason! ;-)

    There is a cost of the behind-the-scenes work to evict stale data from an async cache, and there is a cost to keep a cache in sync (for a synchronous cache). In the case of the former, the cost is in direct relationship to the frequency of updates to and evictions from the cache. In the case of the latter, the cost is in direct relationship to the frequency and potentially the size of the updates to the cache. However, in neither case does that cost present itself as a direct synchronous cost to a cache read operation, such as occurs while assembling a web page. In other words, distributed cache eviction and synchronization may use system resources which may in turn indirectly increase latency, but it does not directly introduce additional latency into the page generation.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Cluster your POJOs!
  23. Cameron,

    thanks for the caching explanation. I has basing my assumptions on the very (very!) simple caching implementations I've done in the past, which of course didn't have any background eviction process whatsoever. The moment I posted my question I realised that this would be done behind-the-scenes regarding cache reading, as you explained, and thus not impact the cache reading time directly. Anyway, taking a lecture from an expert never hurts ;). Thanks again.

    Regards,
    Henrique Steckelberg
  24. If a cached object is stale more often than not, perhaps it shouldn't be cached.
  25. JSR128 / ESI support?[ Go to top ]

    Speaking of HTML caching, does OSCache plans to support JSR128 (http://www.jcp.org/en/jsr/detail?id=128) + extend the project to support a full ESI server (http://www.esi.org/overview.html) or do you know if there are some other OSS projects which are already implementing them (JSR+ESI)?
    Stephane
  26. JSR128 / ESI support?[ Go to top ]

    Not that I don't see the benefit of standardization but I have not heard many complaints on the lack of features or flexibility of OSCache JSP tags. The ESI ones seem kinda overly verbose and complicated at first glance. But of course if it gets popular, someone might step up and make OSCache compatible.
  27. Sounds like a good idea[ Go to top ]

    I am building a website and I tiled the content very well with the intention that I can come up with some kind of fragment caching. using the tag lib to fragment caching sounds great.
    I know it is intrusive but if it works nothing like it