EJB programming & troubleshooting: WAS5 classloader and EJBs on different EARs

  1. WAS5 classloader and EJBs on different EARs (6 messages)

    Now, this looks like a tough one:
    I am talking about EJB 2.0, specifically about WebSphere 5.
    When a web application (WAR) resides in the same EAR of an EJB (i.e. EAR contains both WAR and ejb-JAR), I understand that the classloader hierarchy is as follows:
    EAR classloader --> EJB classloader --> WAR classloader
    This means that the web application can "see" the EJB interfaces (and all the other classes as well), and there is no need to do anything with the WAR manifest file at all. Since these interfaces exist on a single classloader, there will be no ClassCastException when the web application typecasts (or narrows) the JNDI lookup result.
    EJBs with local AND remote interfaces can have only one JNDI name on WebSphere, the possible conflict is resolved by the explicit EJB reference stated on web.xml (ejb-ref or ejb-local-ref). So the web application in the same EAR can use any of the two interfaces. The web app *must* use the indirect JNDI name (java:comp/env/...) - using the real JNDI name returns always the remote stub.

    This is somewhat expected behaviour, and it's ok for me.
    Now, let us imagine another web application in another EAR (but on the same server). The questions are: How can this WAR see the EJB interfaces (let us pretend the interfaces themselves use only JRE commom classes, so there are no other dependencies)? How can another application (in another EAR) use this EJB? Can local interfaces be used, or are only the remote available?

    I have built a web app (WAR2) referencing both the EJB interfaces of an EJB inside another EAR. So I have:
    EAR1 -> (WAR1, EJB-JAR1) -> this EJB is used in WAR1 and WAR2
    EAR2 -> (WAR2)

    Let us assume these filenames (remember the EARs turn into folder names in "installedApps" directory):

    EAR1 = EAR1App.ear
    EAR2 = EAR2App.ear
    WAR1 = webapp1.war
    WAR2 = webapp2.war
    EJB-JAR1 = ejb-jar1.jar


    My first attempt was to use a dirty trick: editing EAR2 manifest file and inserting the Class-Path entry below:

      Class-Path: ../EAR1App.ear/ejb-jar1.jar

    Well, it partially works. This way WAR2 can do a successful lookup for the EJB remote home (both the "java:comp/env/..." and real JNDI name work), and everything works fine. But I cannot use the EJB local interface (even though we are at the same JVM), because typecasting the lookup result raises a ClassCastException.

    It makes some sense, because "identical" classes in different classloaders are considered different, and the ClassCastException is expected. You see, it doesn't matter if the manifest Class-Path points to the very same JAR file that holds the EJB, it is still beeing loaded by a different classloader (the EAR2 classloader).


    My second idea was to package in a simple "ejb-client.jar" *only* the EJB interfaces, and I mean ONLY the interfaces. This "ejb-client-JAR" can can be deployed in WEB-INF/lib of WAR2 (or anywhere else, but referenced in the WAR2 manifest file). This approach works with WebLogic (WebLogic can even generate this client JAR for you).

    Now *nothing* works in WAR2. I get a ClassCastException after a local home lookup (this is expected, see the "first attempt". Strangely, I get ClassCastException also after a remote home lookup. Apparently WebSphere complains about not beeing able to load the remote stub class - shouldn't this stub be loaded transparently to the web app (the EJB client)? Is this a side effect of the "remote-interface-but-same-JVM-so-let's-make-it-local" optimization? Very weird.


    Just to be sure I understood what was happening, I moved the minimal "ejb-client.jar" up in the classloader hierarchy. This can be done by moving the "ejb-client.jar" file to the WebSphere "lib/ext" folder, or by declaring a Shared Library in the admin console and associating it to the server instance. This way the JAR classes (i.e. the EJB interfaces) is loaded by a higher classloader - and this will be the *only* classloader that loads these classes (remember the PARENT delegation rule, the standard classloader behaviour).

    The problem here is that any changes in the EJB interfaces will require a server restart, but this is only for testing purposes anyway.

    Guess what? EVERYTHING worked. WAR2 can do both the remote and local lookups (and typecasting) successfully. As expected in WebSphere, local lookups only work with the indirect name ("java:comp/env/..."), and remote lookups work with both the indirect or real JNDI name. There was no complaint about missing the stub class (like in the "second attempt").

    One might argue that a change in a highly reused EJB interfaces would also require the redeployment (or refactoring) of its client apps, so a server restart in this case might be acceptable. I don't know, it smells bad.


    Opening the ejb-jar file in WinZip it is easy to pick wich classes are the EJB stubs: their names always are "_NameOfEJBInterface_Stub.class" (ex: for an EJB with "MyEJB.class" and "MyEJBHome.class" remote/home interfaces there will be the "_MyEJB_Stub.class" and "_MyEJBHome_Stub.class" stubs). So I included in the minimal "ejb-client.jar" these few stub classes, producing a "ejb-client-stub.jar" with both the interfaces and the chosen stubs.

    My plan was to retry the "second attempt" with the "ejb-client-stub.jar" in place of the simpler "ejb-client.jar".

    Now the remote lookups (and typecasts) work fine, but the local ones don't (as expected, because of the same-class-different-classloader issue). The "ejb-client-stub.jar" file can be distibuted to all web applications the use this EJB (WEB-INF/lib folder), or the manifest file of all these applications can point to a single path (ex: "C:/libs/ejb-client-stub.jar", yes, WebSphere accepts it).


    It seems that the only way to explicitly use EJB local interfaces in different EARs is to create a minimal "ejb-client.jar" (with the interfaces and its dependencies, like VOs) and "deploy" it at a higher classloader (like "%WAS_HOME%/lib/ext").

    Another option is to assume that remote interfaces in the same JVM perform as well as local interfaces (I heard it is true with WebLogic and JBoss, but I haven't done any stress test at all on any server to prove it): the "fourth" and "first" attempt solutions are good enough then. Note that if this is true, it is a *VERY* bad idea to use local interfaces anywhere but in CMR entity beans (where they are required by the spec). I am one of those who believe that local interfaces were a silly idea from the very beginning, since every serious app server at that time already optimized the use of "remote" EJBs within the same JVM.


    Are these conclusions correct?

    I am not bashing WebSphere (at least not this time). I am just trying to figure out what is the _best_ way to make an EJB reusable across different EARs in the same server.
  2. I am sorry for posting it twice; my proxy went nuts.
  3. A very good analysis of problem and description of various solution.
    Anotehr solution is to change the application classloader policy to SINGLE. Please bear in mind in this case applications are not isolated and I am not sure whether you application need that
  4. Thanks.

    I forgot to mention this solution (changing the classloader policy), but in my case giving up the isolation between applications isn't possible.
  5. That sounds good. Now based on your various approaches, I would recommend choice number 4 as opposed to having ejb-client.jar %WAS_HOME%/lib/ext as lib/ext folder is meant for sharing utility libraries, third party libraries etc. I am not sure whether Websphere internally optimizes remote look-ups when they are in the same JVM.

    Please note that I am giving my opinions based on my experiecne with classloaders on different application servers and I am not an expert in websphere in general.
  6. I have just received an answer from IBM support (I just happen to work on a BIG contractor).

    It seems that all these assumptions are 100% correct. The conclusions are:

    1. There is no way you can use an EJB's local interfaces if the EJB is deployed in another EAR in the same WebSphere server (unless you create an ejb-client.jar package containing the EJB's interfaces and put it in a higher classloader, like copying it into "lib-ext"). I haven't found ANY documentation stating this fact (and I searched a lot) - in fact, local interfaces are praised everywhere like if there was no drawback. Changing the classloader policy to a flat model solves this issue, but brings tons of new problems (you just can't use struts, for example, for it MUST be isolated in each application classloader);

    2. You are responsible for the packaging of an appropriate ejb-client.jar (apparently there is no tool to help you here). I didn't find any doc about it too: every example seems to find normal and acceptable to copy the full EJB-jar onto everywhere.

    I will not bash WebSphere this time, since these classloader issues are quite commom. They could at least be better documented.

    Let's hope for smarter implementations (like redeployable shared *classloaders*, instead of just shared libraries).
  7. Possible Alternatives[ Go to top ]

    hi Andre,
       Excellent stuff. I am almost solving the same problem. I am looking at different ways of establishing the communication between two ear files with EJBs and Struts components in individual EAR files.

    Here are other solutions that I could think of could you offer your feedback on them. I would personally like to discuss with you (and anyone on this board) for quicker feedback. If you would like please email me at "Programming 95 at Yahoo dot com", No underscores, no Spaces.

    -You Create a client adapter that contains a single method as below

    Object execute(Object)

    - The client adapter is designed as a simple Java class. This class is jarred up and along with EJBs in EAR1.
    - when components in WAR2/JAR2 in EAR2 needs to communicate with EAR1 components, they need to create the client adapter object in EAR1 and formulate their request in the form of an XML message and send to the execute method as an argument. The execute method hands off the request to appropriate components within EAR1 and returns the result/exceptions in the form of an XML string.

    This approach decouples both EAR1 and EAR2.
    EAR1 and EAR2 Teams each can work on their own even if the other team is making changes to their domain model...as long as the client adapter with the execute(Object) signature is available.
    - One of the concerns of this approach is the use of XML. I am wondering if this could become a performance bottleneck. Thinking about using JAXB to free programmers from having to parse xml, but again what about the performance part???
    What are your thoughts in this regard.... will it be an issue in terms of performance???
    App Server is :: Websphere.

    Do you see an issue with transactions in this approach. For instance, a Business delegate in EAR2 initiates a business transaction that requires EJBs from both EAR1 and EAR2. The business method in the businessDelegate ( a plain java class) can create a UserTransaction and invoke the EJB component in EAR2. For invoking the EJB in EAR1, it create EAR1 Client Adapter and sends an XML request. EAR1 client adapter invokes appropriate EJBs in EAR1. Can a transaction span multiple EAR files like this?

    I got this idea from the book "Software Architecture Design Patterns in Java" , great book at Amazon....

    -In this approach NO cilent jars and NO client adapters are created.
    -Both EAR1 and EAR2 contains business delegates (simple java classes) in Jar files. When there is a need for EAR1 component (say EAR1 business delegate) to communicate with EJBs in EAR2, it invokes a method on EAR2 business delegate using reflection. EAR2 business delegate in turn talks to the EJB components in EAR2. Since reflection does not require the class to be available at the compile time, all teams can go on with their own implementation.

    - My concern is, is it a good practice and a viable solution to use reflection in a business solution like this...for communication between teams.

    I have not tried these approaches out. Just wanted to collect some ideas.

    I welcome any questions, suggestions.