distributed static cache for readonly data from legacy system

Discussions

EJB design: distributed static cache for readonly data from legacy system

  1. Hi,

    I have a design question. In my EJB application I have to use from all clients special readonly data, which are coming from a legacy system. All other data in my system can be made persistant using CMP 2.0 in a relational database. The data from the legacy system are very costly to retrieve and should retrieved only one time and then shared between the clients. The construction of the data (complex graph structure) is also very costly. The question is where to put this data, in an Entity Bean it is not possible, in a statefull session bean the legacy data would be constructed for each client which I want to avoid and a stateless session bean can exist in a pool and I can not prevent that here the data will also constructed multiple times when many instances will be constructed. Is the only way to stick this data in the JNDI tree? I found no other solution.
    Can somebody please help out?

    Thank's in advance,
    René Meyer

     
  2. I would suggest you to use collection classes ,such as HashTable ,retrieve the data from database (or legacy applications)once and place it in HashTable.
    Then whenever you need you can just get it from HashTable.
  3. The problem here is not the data structure used to store the data from the legacy system, it can be a hashtable or Tree or any other container. The problem is which construct controls the lifecycle of this container? Where to put the Hashtable so that it will be created only once and can be shared bewteen the clients? I can't put in an Entity Bean, makes no sence and I can't put in a session bean. Which concept enables the usage of global readonly data that can be shared between different clients?
  4. Why couldn't you store the data in read only entity beans?
  5. OK when using Readonly Entity Bean, the Entity Bean in my case has no relationship to any database. I would implement it as BMP bean, but inside the Bean I will have no calls to a RDBMS with SQL statements. The data which should be shared among the clients can be implemented as value objects. The data access to the legacy data will be implemented in some kind of DAO class or with an Adapter. The Entity Bean controls the creation of the data with the DAO and manages the lifecycle and access of the created value objects (object graphs created from the legacy data).

    Some ideas are so simple, thank you very much.

    Best Regards,
    René Meyer
  6. I think it is a very interesting question René has raised.
    In fact I am also struguling to find a solution on those lines. My problem is similar. I have a very big chunk of read-only data which i want my ejb(s) to SHARE.

    One solution which came in my mind was to have read-only static field in an EJB, but according to the EJB-specs, the static field on EJB SHOULD be final. I am thinking it is possible for an ejb to declare a static final hashtable or so and load data....Am I on the right track ?

    Followup on the above topic...
    My undertanding on the read-only bean is that that ejbStore will be be called by the container...but that does not necessarily mean that the instance variable in the bean are not replicated, hence if one does have a very big chunk of read-only data, read-only bean may not be the solution of the above problem...
  7. Hi René

    your need was very well covered by the Corba SHARED activation policy. But it seems that this concept is actually missing in EJB as a builtin construct. I see basically 4 variants in EJB for computation expensive shared objects:
    1. JNDI. The JNDI entry has to be serializable, which is not always the case. Moreover the deserialization may be costly.
    2. Entity Bean. The readonly Entity Bean should not activate before a business method call and not passivate after it. Otherwise the whole state is lost between business calls. I don't know how to disable the Entity-Bean-to-Data-Source-synchronization. Beware that the EJB container is free to call the passivation and activation anytime. So ejbLoad should get the value from your leagacy system again.
    3. Singleton. A Java class with static instance variable. If the setters are called at startup time of the container it shouldn't be a problem that the instance variable is not "really" final. The question is: is it a problem if there is many singletons? This could happen when the container uses many class loaders. Which container do you use?
    4. Corba Server. Call a Corba Server with shared activation policy. JDK1.4 has (almost) everything you need and it's free. The only problem is that the ORB within the EJB container should also be a "singleton" and we have to solve this problem "in EJB". The ORB should be a singleton because it is costly to instanciate and takes a lot of memory.

    As I understand you are trying solution 2. Could you please tell me how do you configure your EB so that it doesn't synchronize (call ejbStore and unsetEntityContext) for each business method call? I tried with an EB behind a SB with several different transaction modes. But with Silverstream each business method call implies a passivation of the EB so that my first setting has been reset.

    Regards
    Alain
  8. Hi Alan,

    thank you for your detailed answer.
    I think when we have a cluster of EJB containers the complexity of a solution for the described problem would be much higher.

    1. JNDI tree serialization
    Does it works in a cluster?

    2. readonly Entity Bean?

    3. I think singleton classes approach will not work when multiple classloaders are present, for each node we have at least one classloader

    4. dedicated CORBA or RMI Server process seems for me the only possible choice in this scenario. The process should be independend from the EJB container processes. If I have an ORB or RMI Server running as independend process, it should be no problem to retrieve data from for example session beans via the ORB.

    Regards,
    René Meyer




     
  9. Hi René

    1. JDNI tree serialization. Yes it works in a cluster like Weblogic because it has a "global" JNDI service which is always synchronized with the "local" JNDI services of each node. But it doesn't work in simple local JNDI services like Silverstream which has no notion of global JNDI for the whole cluster.

    2. Readonly Entity Bean. Yes it works in a cluster like Weblogic. WLS load balances the retrieval of the home as well as the method calls. The EB is replicated on each node.

    3. Singleton. It's correct there is a singleton per classloader. There is never a singleton in a cluster.

    4. Corba server. This is a clean "singleton" in an EJB cluster. But if there is only 1 server it's a single point of failure. You can have a corba cluster to avoid this. But dont's forget to cluster the Corba naming service if you want to avoid the single point of failure of the naming service itself (like weblogic global JNDI case).

    Regards
    Alain
  10. Have you considered using an object database to store the cached information? Systems like GemStone Facets, Versant enJin, Poet, etc. would work well and should scale to handle any amount of data.

    I guess this would be similar to the JNDI option, but the static data you extracted would survive a VM failure. A good oodb will cache information in shared memory so you don't have to read it from disk every time you want to access it.

    I didn't notice what app server you're using, but I know that BEA recommends against putting information in JNDI if you're using clustered servers. The cost to synchronize across a cluster is high.


  11. This is a common problem. Until there is an in-memory-only kind of EJB, we will continue to have this problem. Perhaps the JCache JSR may deliver what we need... until then:

    1) JNDI tree:
    Firstly, whether or not your caching object (HashTable or whatever) needs to be serialised entirely depends on your appserver and your JNDI implementation (whether the JNDI implementation is co-located or not).

    If you are using something like weblogic (where the JNDI service runs in the same JVM), your cache object does not have to be serialisable. However, if your server is in a cluster, then your object WILL need to be serialisable - but only if you want it replicated across the cluster (which you dont really want).

    In a clustered environment, create an empty object and bind it to the JNDI tree - do this from a startup class (since weblogic's JNDI is not persistent). If your cache object is serialisable, you can do it from only one server in the cluster and the empty object will be replicated around the cluster (note, though, that if that server crashes you will lose your object). If your object is not serialisable - or if you want a more robust solution, turn off the JNDI replicate-bindings before you bind your object (you will obviously have to deploy the startup class to each server in the cluster).

    (Note that your object would only be replicated in the cluster on a bind() or rebind(). You can add data to your cache object but it wont be replicated until you rebind your object)

    Your EJB's in each node of the cluster can now access their local copy of the cache object - and if your cache object is truly read-only, use a HashMap instead (there is no internal synchronization)

    Read-Only Entity Bean:
    Again, the absence of a shared in-memory EJB forces us to use an entity bean when it is not what we want...
    In your BMP entity bean, you can use the following pattern:
    1) implement an empty ejbStore() method
    2) in your ejbLoad, only "load" if there is no state (check for null) - or if a cache validity time is exceded.
    3) in all callback methods that lead to an invalidation of state - set the state holding variables to null.
    4) check how often your bean is invalidated - just in case the container is loading/unloading your bean a lot.

    The only problem with this is that you have to write a finder method.

    Connector Architecture:
    The connector architecture is a very good place to step outside of the EJB limitations. The connector can manage the connection to your legacy system and manage the cache - in a way it is a good abstraction of the problem away from your beans.
    To implement the SPI's for the connector, you will have to implement some essentially empty methods for security and transaction management - however dont let this put you off.

    There are a number of long-lived objects in the connector implementation where you can hold the reference to your cache object - so you dont have to worry about the activation/passivation issues of your EJB's etc.

    It is really quite neat and scalable - your connector can decide when it is time to flush and re-load the cache.

    Regular Singleton:
    Depending on your appserver a singleton object (with factory method etc) may be useless. Some appserver implement a seperate classloader for each EJB - which would mean this singleton pattern would not scale at all. Most appservers, however, at the very least implement a different classloader for each EAR (or JAR if not inside an ear)
    If you are using Weblogic, the singleton pattern will work fine.
    True, your singleton will exist exactly once per server (ie it is not a network singleton)- but I suspect this is not going to matter in the slightest.

  12. Hi

    nice problem, I'd never thought about this before. You have a data source which cannot be made EJB because of the life cycles of EJB's, and because of the class loaders of the containers.

    Are you worried about the server holding the data dieing? Do you want to replicate the read only source or not? When you say the data must be made available to the client, does this mean its needed in the EJB's or just in the web tier?

    Hopefully, just in the web tier. In which case, why not have a servlet that on initialisation loads all the data into a singleton in the web tier. This will be very quick to use within the web tier, but impossible to use from EJB's. It's also robust - every web server has it's own copy. But it does create overhead when starting up the web site.

    Jonathan
  13. Thank you all for your very good answers,
    I will try the connector architecture and the readonly Entity bean approach.

    Best Regards,
    René Meyer