OSCache 2.1 released

Discussions

News: OSCache 2.1 released

  1. OSCache 2.1 released (23 messages)

    OSCache includes a JSP tag library and set of classes to perform fine grained dynamic caching of JSP content, servlet responses or arbitrary objects. It provides both in memory and persistent on disk caches, and can allow your site to have graceful error tolerance (eg if an error occurs like your db goes down, you can serve the cached content so people can still surf the site almost without knowing).

    Besides many bugs being fixed, major improvements have been made to OSCache in the way data can be persisted. Also, a new wiki has been created to expand on the existing documentation.

    Check out the site and
    Download the code

    Threaded Messages (23)

  2. OSCache 2.1 released[ Go to top ]

    This may seem a bit far-fetched, but since I've been gaining an interest recently in memcached, how would these compare, if anyone has experienced them both ?

    Yes, I do realise that memcached is a fast native {int len; void* buf} cache while OSCache is short of washing the dishes too, but still.

    Cameron, care to share your views ? Please ?
  3. OSCache 2.1 released[ Go to top ]

    Yes, I do realise that memcached is a fast native {int len; void* buf} cache while OSCache is short of washing the dishes too, but still.Cameron, care to share your views ? Please ?

    I think OSCache has helpers that are geared for Java usage, such as the tags support, etc.

    Also, OSCache can store Java objects .. it doesn't matter how fast you get the byte[] because the real cost is in deserialization, not network.

    I personally would use Coherence, though .. but I've been told I'm not very objective .. ;-)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  4. OSCache 2.1 released[ Go to top ]

    Also, OSCache can store Java objects .. it doesn't matter how fast you get the byte[] because the real cost is in deserialization, not network.

    Due to Java's (de)serialization implementation? Do you see way of improving the implementation, or is the problem reflection?
  5. OSCache 2.1 released[ Go to top ]

    Also, OSCache can store Java objects .. it doesn't matter how fast you get the byte[] because the real cost is in deserialization, not network.

    Due to Java's (de)serialization implementation? Do you see way of improving the implementation, or is the problem reflection?

    The problem is the implementation of ObjectInputStream and ObjectOutputStream; there's a lot of overhead to deserializing even a single object.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  6. OSCache 2.1 released[ Go to top ]

    The problem is the implementation of ObjectInputStream and ObjectOutputStream; there's a lot of overhead to deserializing even a single object.

    Cameron,

    I assumed that Sun provided the fastest possible way of serializing an object (given that they are the creators of Java.)

    Do you think there are faster ways of implementing ObjectInputStream/ObjectOutputStream or is this just the way things are?

    How would *you* reimplement this feature? I've looked at the source, but I'm not proficient enough to see where the speed-up potential is.

    I'm curious, because I've experienced that serialization is indeed very slow, but I always though of this as a "just how things are" kind of situation.

    Thanks for any insight.
  7. OSCache 2.1 released[ Go to top ]

    How would *you* reimplement this feature?

    1. Class versioning at the platform level, meaning that classes could optinally contain version information, and that a pluggable system for resolving version issues could be provided (allowing custom version translations of binary representations, for example)

    2. Better transitive closure support for references .. right now if I had to do it in pure Java I'd have to use the System object to get internal hash codes (i.e. pointers) and then register a list of refs per internal hash codes (just in case of collisions) to verify if I already visited an object in a persistence graph. Apparently, the data structures to provide transitive closure are relatively expensive, both in setting them up (i.e. serializing even one object with no refs is slow) and in using them (i.e. a big graph).

    Example: A points to B and C. B points to C. Serialize A. This means I have to assign transitive closure serializable IDs to A, B, and C. Let's assume A=1, B=2 and C=3. The ref from A to B gets serialized as 2, the ref from A to C gets serialized as 3, then I ask B to serialize, it goes to serialize C, how does it know that C is 3? It has to quickly "look up" C in a list of objects already serialized. For a small amount of refs (e.g. serializing an object graph containing only 5 or 10 items) I'd just use an array data structure to keep track of the refs themselves (i.e. use an Object[]) and after that I'd use a sparse array class (indexed by int, the system internal hash code) with the value either being the object or a list of objects (for collisions).

    3. I'd optimize all the "built in" types more. Only String is currently optimized. (You can see this in the sources for java.io.)

    4. I'd allow the VM (e.g. hotspot) to build externalizable implementations on the fly, as long as they matched the spec for the serialization format. In other words, if you serialize an object 10 times, maybe it's better to get code to do it. Because of other "contracts" in Java, this should probably be done without it being exposed through reflection etc. that the changes are indeed being done.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  8. For a small amount of refs (e.g. serializing an object graph containing only 5 or 10 items) I'd just use an array data structure to keep track of the refs themselves (i.e. use an Object[]) and after that I'd use a sparse array class (indexed by int, the system internal hash code) with the value either being the object or a list of objects (for collisions).

    Knowing the graph size in advance seems to be part of your optimization. Would the developer declare it?

    JSP applications are usually multi-threaded, and I wonder if serialization is slow because of a critical section in a JRE class? Or is concurrent serialization compute bound?

    Which is slower in Sun serialization: graph walking or field copying?

    Your idea about generated externalizers is awesome, but maybe better served with portable aspects rather than a proprietary JVM enhancement. An aspect has two benefits at it:

    1) It supports all JVMs.

    2) If the transport is JRMP, then the aspected bytecode can be shared dynamically. That might be useful on a WAN, where codebases aren't centrally managed.
  9. Serialization, hmmm....[ Go to top ]

    For a small amount of refs (e.g. serializing an object graph containing only 5 or 10 items) I'd just use an array data structure to keep track of the refs themselves (i.e. use an Object[]) and after that I'd use a sparse array class (indexed by int, the system internal hash code) with the value either being the object or a list of objects (for collisions).

    Knowing the graph size in advance seems to be part of your optimization. Would the developer declare it?

    Nope. You use an array until it's full, then switch to a more expensive data structure. In a lot of cases, you don't even need an array (single object serialization).
    JSP applications are usually multi-threaded, and I wonder if serialization is slow because of a critical section in a JRE class? Or is concurrent serialization compute bound? Which is slower in Sun serialization: graph walking or field copying?

    Bill Burke had some good comments on the various costs of the Sun implementation. Since he's probably going to see this comment, I'll hope he can dig them up and post them (or a link).

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  10. Serialization, hmmm....[ Go to top ]

    Some thought about serialisation, although i am not a guru on this domain... :o)

    Regarding the whole graph serialisation, and cyclic references. The Java serialisation process is ignorant about the business objects relationships (of course i am not talking about the techical relationship), which is not the case of the developer. I believe in some if not most cases the developer should be able to do some optimisations, such as the cyclic dependancy lookup can be reduced to a very minimum overhead, even allowing duplicates that would be eliminated by the deserialisation process for instance.

    Also, if i understood it well (but i did not do a deep study), at runtime the java serialisation mechanism would dynamically ask or an object to serialise itself, which is when the object would be start from zero its serialisation process, field by field. Why not have it another time the other way around ? Have the object partially or completely serialise itself at each method invocation, in an attached byte buffer for instance ? Of course to the extent of the RAM usage, but RAM is soooooo cheap nowadays ! Most of the bytes for the objectstream would be already ready at time of the graph serialisation invocation, so the overall time to serialise would be spread on the object lifecycle, enabling very fast snapshot. If there is no flaw here that i would have missed (really first thoughs), AOP could greatly help there i suppose (yes, i am very lazy...).

    To sumup, in cases where performance is absolutely crucial, the business objects should be designed around the serialisation mechanism, and not the way around.

    Peace ! :o)

    Christian

    PS : Cameron, when you get to Paris give me a call i have a demo for you ;o)
  11. OSCache 2.1 released[ Go to top ]

    has anyone moved from oscache to jbosscache, or viceversa?

    We've done some load testing pitting OSCacahe against TreeCache, and found that TreeCache had some synchronization problems when working in a two node cluster. I can share the load testing code, it's basically randomly selected Hibernate load, query and save actions executed by a Servlet. Performancewise, OSCache performed well, so it's looking like we will be using OSCache for production. Glad to head 2.1 is out, we will swith to that immediately.

    One project in our company also reported TreeCache failure in single node usage, which are sort of surprising knowing the product comes from JBoss.
  12. OSCache is just great[ Go to top ]

    The tag library is so powerful yet so simple. I've seen a 5000% increase in the number of pages I can serve from a single tomcat instance just by adding these tags to the relevant pages in our CMS.

    A really great piece of code.
  13. OSCache is just great[ Go to top ]

    is anyone here in a position to (objectively) compare the *caching* of oscache v. jboss cache (i know jboss cache doesn't have the taglib and filter support)? i was kicked from #java on efnet by fate last time i questioned the quality of the caching implementation, but the company where i was working at the time had to drop it because it's cache often failed and we would see the same object appear several times on a page :(

    has anyone moved from oscache to jbosscache, or viceversa? i'd be particularly interested to hear from people who are using either product in a distributed environment (even tho they both seem to use jgroups underneath).
  14. OSCache is just great[ Go to top ]

    is anyone here in a position to (objectively) compare the *caching* of oscache v. jboss cache (i know jboss cache doesn't have the taglib and filter support)? i was kicked from #java on efnet by fate last time i questioned the quality of the caching implementation, but the company where i was working at the time had to drop it because it's cache often failed and we would see the same object appear several times on a page :(

    OSCache used to have some pretty serious issues .. it was responsible for some of the major downtime on jRoller for example. However, I think that most of those issues got addressed a while back.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  15. More on tags[ Go to top ]

    Check out also Cache taglib in Coldtags suite:
    http://www.servletsuite.com/jsp.htm

    Eugene
  16. OSCache 2.1 released[ Go to top ]

    I could only recommend OSCache! It is so simple yet so powerful piece of software.

    I use it in a very loaded database driven application as a servlet filter. Configuration and integration were plain simple and I had no problems of any kind with it. The mentioned application serves about 1.000.000 cached items (2.5GB) per day for 8 months now.

    You won't find many products/libs, both open- and closed-source, which fit so seamlessly into your application.

    Although the very source is a little bit cumbersome, if you don't need to customize it, you don't have to worry.

    Again, great piece of software!
  17. OSCache 2.1 released[ Go to top ]

    How would *you* reimplement this feature?
    If we stick with replicable caches (those that use some kind of object data replication on updates to maintain cache consistency), then the options are pretty slim since serialization/deserialization is not really pluggable and cannot be fixed without significant change in java.io.* itself.

    There is another type of caches that don’t use replication – non-replicable caches. This type of cache simply uses short invalidation message sent over non-instrumented UDP/IP-multicast on updates and never requires object serialization/deserialization. So, the performance associated with general marshalling is eliminated here by design.

    Both types of caches (and that’s not the only classification, of course) have their own sweet spots and anti-patterns. Non-replicable caches clearly perform better with higher rate of updates (and when I say higher it still means much lower than rate of reads; otherwise cache of any kind is useless).

    Hope it helps,
    Nikita.
  18. OSCache 2.1 released[ Go to top ]

    If we stick with replicable caches (those that use some kind of object data replication on updates to maintain cache consistency), then the options are pretty slim since serialization/deserialization is not really pluggable and cannot be fixed without significant change in java.io.* itself. There is another type of caches that don't use replication - non-replicable caches. This type of cache simply uses short invalidation message sent over non-instrumented UDP/IP-multicast on updates and never requires object serialization/deserialization.

    Hi Nikita, I do have some disagreements:

    1. Moving data requires serialization. Invalidation has to move a key. A key is data. Invalidation therefore requires serialization.

    2. There are plenty of ways to optimize serialization. You're right about one thing though -- replacing it "seamlessly" is not very easy.

    3. Use of "invalidation message sent over non-instrumented UDP/IP-multicast on updates" means "non-reliable invalidation" .. under load, multicast can easily have a 99% packet loss rate (see the INRIA study on JGroups).

    BTW - I think OSCache supports the local model and the invalidation model -- not a replicated model. (Tangosol Coherence supports local, invalidation, replicated, tiered, and partitioned models, plus directed agent invocation.)
    Non-replicable caches clearly perform better with higher rate of updates (and when I say higher it still means much lower than rate of reads; otherwise cache of any kind is useless).

    For a pull model with a tolerance for dirty data, that can be true. In honesty, it depends what you are "pulling" from, and how expensive the "pull" is, and how big the data set is that you are "pulling". One of the benefits of combining an invalidation model with a cluster-partitioned model (e.g. the Coherence "near" cache) is that you can cache gigabytes (and in one case that I can think of, terabytes) of information with almost no impact on the application servers' heap sizes, with an average latency to access an object from cache that is very close to 0ms. And that's a coherent cache (and optionally transactional), meaning that the application can avoid dirty reads altogether.

    At any rate, you got me off the topic of serialization and deserialization improvements, which was off the topic of OSCache, which is what this thread is [was] about.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  19. OSCache 2.1 released[ Go to top ]

    Ok, I’ll stick to serialization aspect of it only for the sake of this thread.
    Moving data requires serialization. Invalidation has to move a key. A key is data. Invalidation therefore requires serialization.
    I can only vouch for one implementation of non-replicable cache that I am very familiar with cache (these things can very from impl to impl) and sending “keys” never requires object serialization (full transitive closure invoking java.io serialization routines). A key, in our case a hash of it, is just a series of bytes that gets converted into Java’s ‘int’ in a less than a two dozens of bytecode ops (we have test it internally to be x100s times faster than comparable object serialization). This approach reduces marshalling load quite dramatically.
    Use of "invalidation message sent over non-instrumented UDP/IP-multicast on updates" means "non-reliable invalidation" .. under load, multicast can easily have a 99% packet loss rate (see the INRIA study on JGroups).
    Agree. Certain instrumentation is required and my ill-written point was to compare it with TCP or IP-multicast-based protocols that have to deal with variable-length messages and which instrumentations are heavier. In our case, we reduced this communication to fully-reliable, fixed-length messaging over UDP/IP-multicast. Much simpler and faster, in my opinion. Obviously, in expense of not having a data replication as a feature.
    For a pull model with a tolerance for dirty data.
    As any other caching product we don’t allow dirty data so there is some misunderstanding here. As I said there are sweet spots for all types of caches and none of them are universal. As far as pulling I agree: it’s a debatable question whether to pull from another node, or pull from data source. I guess the answer varies greatly based on a specific application…

    I, by the way, believe the original serialization question was very good and right on the topic of this thread: nobody cares about a news flash, but good and relevant technical discussions aid greatly in forums like TSS.

    Best,
    Nikita.
  20. OSCache 2.1 released[ Go to top ]

    If I read this correctly,
    Tangosol Coherence supports local, invalidation, replicated, tiered, and partitioned models, plus directed agent invocation.
    I guess memcached uses partitioned models with a partition size of 1.
    That's to say it partitions cached objects among available servers, and no two servers have the same objects; fetching an object determines the appropriate "partition" by the object's hash.
    Is this correct ? Care to detail on tiered and directed agent ?
  21. OSCache 2.1 released[ Go to top ]

    I guess memcached uses partitioned models with a partition size of 1. That's to say it partitions cached objects among available servers, and no two servers have the same objects; fetching an object determines the appropriate "partition" by the object's hash. Is this correct ?

    From a brief look at memcached, I think that's correct. I have no idea if memcached is based on Coherence partitioned caches or not.

    Coherence partitioned caches provide for dynamic repartitioning (failover and failback) without data loss, and being built in Java, the complete back end for the caches is customizable, allowing for read-through caching, write-through and write-behind (i.e. coalesced asynchronous write-backs to a database). All the caches can be size limited and/or set for expiry, and support on-heap, off-heap and disk storage. They're also searchable, with the partitioned model doing the searches in parallel (parallel query).
    Care to detail on tiered and directed agent ?

    Tiered refers to models in which there is a core set of servers that own the master responsibility of managing a cache, and you can hang servers (or typically clusters of servers) off of the first, and hang servers (or clients) off of those, and so on. It is hierarchical, but each stage in the hierarchy can be a cluster, and the leaves can either be servers or even fat clients (e.g. via JMS).

    Directed agent refers to the ability to send processing to be executed on cache entries where they are being managed within a cluster. For example, imagine implementing a primary key counter in a cluster using a simple cache API supporting concurrency control .. something like:
    if (cache.lock(key)) {
      int value = cache.get(key);
      cache.set(key, value + 1);
      cache.unlock(key);
    }

    The same could be accomplished with an agent:
    public class KeyIncAgent extends AbstractAgent {
        public void run() {
            // some factory call to get the actual cache storage
            Cache cache = ...;

            Entry entry = cache.getEntry(key);
            int value;

            // since we're on the owning server, local sync works!
            synchronized (entry) {
                value = entry.getValue();
                entry.setValue(value + 1);
            }

            // agent returns the key value to use
            setResult(value);
        }

        Object key;
    }

    So now instead of several potential network round trips, we only need one:
    // what cluster member owns the key? ask the cache
    Member member = cache...;

    // send the agent there
    int value = service.query(new KeyIncAgent(key), member);

    Note that all this code is pseudo-code that I just typed in here, so don't expect it to compile and run ;-). For example, the query call returns a Map keyed by Member, since you can ask multiple members the same question in parallel (it's a clustered request/response model).

    Anyway, to make a long story short, the purpose for directed invocation is to handle situations where it's more efficient to put the processing where the data are than to move the data to where the processing is. This is incredibly useful when the data are large, or when you want to process different pieces of data in parallel (since you can send the agent to multiple servers at once, each will process just the data owned there).

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  22. OSCache 2.1 released[ Go to top ]

    Thank you Cameron, that was enlightening.
    I think I need to check the Tangosol docs, if they're available on-line and to non-customers.
    Regards,
    Radu
  23. OSCache 2.1 released[ Go to top ]

    Heh, so I guess good old stored procedures return with the directed agent ! >:)
  24. Javapedia: Cache[ Go to top ]

    I compiled a list of cache libraries in Sun's Javapedia:

    http://wiki.java.net/bin/view/Javapedia/Cache