Discussions

News: Andy Oliver: Versioning and Classloading in Java

  1. Andy Oliver, in "Versioning and Classloading in Java," says that he hates Java classloading. "You may think you hate JBoss classloading, but JBoss classloading is just massive configuration for Java classloading." After explaining how classes are tied to their classloaders, he outlines some possibilities for fixing Java's classloader structure.

    It should be noted in the interests of fairness to all sides that while JBoss has been maligned for its classloading structure, every application server suffers from some degree of "classloader hell," partially because the J2EE specification itself only recommends a classloader structure, and there are many situations in which the recommendation isn't optimal.

    Andy's solution:
    ...fix Java. Forget versioning by classloading and lets go versioning by compiler and by runtime profile definition.
    • When classes are compiled they get an auto "version" of the epoch time and a hash of the hostname. The form is FQN.epoch.machinehash. We will call this classversion.
    • When a class references another class the FQN is replaced by FQN.classversion
    • When the classloader looks for a class, it looks for one matching classversion. If it does not find one, it looks at the manifest of the requesting deployment for a matching pattern.
    • Matching patterns may be FQN.(>12324567).*. That would mean any later epoch from any host.
    • The whole declaration is "FQNOfRequestor.*.* : FQNOfRequested.*.*" For example: "org.jboss.example.Requestor.*.* : org.apache.commons.logging.CrashJava.(>1234567).*" - that would mean that if org.jboss.example.Requestor tries to load some specific version of CrashJava and it doesn't find it...load a version built on any host with an epoch time later than 1234567.
    • Timezones don't matter because this isn't needing versions that are quite that fine grained
    • Classes are defined solely by FQN.classversion. They are NEVER defined by Classloader instance.
    • For backward compatibility this is implemented via compiler and a new classloader.
    • For backward compatibility classes without version information are versioned at runtime with their classloader id (basically the same behavior as present). Linkages with no version are treated as FQN.*.*.
    What do you think? How do you think the classloader structures in J2EE should work - if any differently than it does now? Should the specification mandate a classloader structure?

    Threaded Messages (42)

  2. Java class versioning is a nightmare, and we've all known it for a long time. (Where is the JCP when you need it.) The problem is not new, and it has been solved before (think Smalltalk).

    Just another item in the long list of java inadequacies.
  3. Java class versioning is a nightmare, and we've all known it for a long time. (Where is the JCP when you need it.) The problem is not new, and it has been solved before (think Smalltalk). Just another item in the long list of java inadequacies.

    ... yet people manage to deploy solid, complex, workable, even maintainable solutions on it despite it being so inadequate.

    I wonder how that works?
  4. Through a lot of blood, sweat, and tears. So many things are really painful in java. As I am sure you will agree.

    By inadequacies, I didn't mean that things can't be done, just that they are not done in an optimal way. You can do all of that in assembly and yet we don't see anyone blindly defending that strategy.

    My criticism of java is not a dismissive one... I am just frustrated that in the last 8 years I have been running circles around some problems that have been solved, in other programming languages, a long time ago.

    I hope, that the next great language that comes along will improve on the past successes and mistakes. So, when the marketing kicks in again we don't go back another 10 years.

    P.
    Java class versioning is a nightmare, and we've all known it for a long time. (Where is the JCP when you need it.) The problem is not new, and it has been solved before (think Smalltalk). Just another item in the long list of java inadequacies.
    ... yet people manage to deploy solid, complex, workable, even maintainable solutions on it despite it being so inadequate.I wonder how that works?
  5. I am just frustrated that in the last 8 years I have been running circles around some problems that have been solved, in other programming languages, a long time ago.
    +1
    This has been addressed in C# at the creation of the language. Versioning is something completly missing in Java. Versioning at the class level would be great, but if we could already have it at the lib level, it'd already be better than the current situation.
  6. I am just frustrated that in the last 8 years I have been running circles around some problems that have been solved, in other programming languages, a long time ago.

    +1This has been addressed in C# at the creation of the language. Versioning is something completly missing in Java. Versioning at the class level would be great, but if we could already have it at the lib level, it'd already be better than the current situation.

    In C# 2.0, the situaion is better, but I wouldn't say it's completely fixed. Comparing classloaders to AppDomains, I'll take classloaders any day. In order to dynamically load/unload classes in .NET 1.0, 1.1 and 2.0, one has to load it in an appDomain. that means an extra serialization cost, which for many cases is not desirable.

    I agree Sun should address the versioning issue and there isn't a good excuse for not addressing it properly. Sun's had plenty of time to address the problem.

    peter
  7. Having specifications mandate explicit class-loader policies will help, however, it will not address the whole problem.

    I agree If the same class version is loaded by N different class-loaders they should resolve to the same class across the JVM.

    This may mean having one central registry for classes in the JVM. The individual class-loaders (for various modules) merely represent loading implementations and urls only - with a cache back into the central registry for classes it was personally responsible for loading.

    Child-First class loading policy should be mandated as well ie. if a child class-loader hasn't seen the class yet - it gets a chance to load a "different" "version" of the same class already present in the central registry if it hasn't done so already (and placed into registry as well). This allows modules to be isolated to use older/newer versions of a class as well as share a class of the same version across all modules in the JVM.
  8. Having specifications mandate explicit class-loader policies will help, however, it will not address the whole problem.I agree If the same class version is loaded by N different class-loaders they should resolve to the same class across the JVM. This may mean having one central registry for classes in the JVM. The individual class-loaders (for various modules) merely represent loading implementations and urls only - with a cache back into the central registry for classes it was personally responsible for loading. Child-First class loading policy should be mandated as well ie. if a child class-loader hasn't seen the class yet - it gets a chance to load a "different" "version" of the same class already present in the central registry if it hasn't done so already (and placed into registry as well). This allows modules to be isolated to use older/newer versions of a class as well as share a class of the same version across all modules in the JVM.

    The problem with this is static members... Would you share them across class loaders as well?
  9. Yes, ofcourse. For a given class-version you'd be sharing the same 'Class' object across all of the JVM and so its 'static' members as well.

    I don't see why this would be an issue? Infact, to have anything less is 'invalid' in my opinion.
  10. Yes, ofcourse. For a given class-version you'd be sharing the same 'Class' object across all of the JVM and so its 'static' members as well.I don't see why this would be an issue? Infact, to have anything less is 'invalid' in my opinion.

    Because it will break LOTS of things. In addition, it will make hosting of Java applications at hosting providers even more troublesome, since different web applications in a container are protected from each other via using separate classloaders.
  11. Unfortunately, if you do not use the classloaders to identify class instances, but technically use only one classloader, then you are not able to isolate static fields of the same class in different deployment units.

    So some part of the current classloader hierarchy isolating deployment units from one-another must be retained, to keep the expected behaviour.
  12. Jim Waldo's team have been looking at this stuff from another angle (mobile code). Check out Mike Warres paper on classloading:

    http://research.sun.com/techrep/2006/abstract-149.html

    They've got a number of projects that are in the works one of which rumour has it will provide some interesting mechanisms for addressing some of these issues.

    http://archives.java.sun.com/cgi-bin/wa?A2=ind0604&L=jini-users&P=R3242&I=3&m=18407
  13. For shame. Andy doesn't understand class loading; I hope readers don't take this seriously. You can abuse any API, but that doesn't mean JBoss should make it easy for users to do so. Class loaders do suffer from mixing of too many concerns but not those Andy mentioned.

    In response to Joseph's question, "should the specification mandate a classloader structure?" Yes! With the utmost respect, JBoss and WebSphere are perfect examples of why.
  14. Usually the problem is that many products come with their own versions of other products, which inevitably means the classloaders in your jvm will load different versions of the same code. This can be fixed at the product level by allowing the user to deploy certain specific versions of libraries.

    The jdk had this problem with Xerces, for example, and they solved it via the well-known endorsed extension mechanism. Other products can just use similar mechanisms. But usually people don't bother and leave themselves open to this problem.

    Adding a version number also works, but I don't know if it's worth doing. It has a huge cost in complexity. Right now you don't have to assign release numbers to code which makes it cheap to develop applications in the best case scenario, which the vast, vast majority of code out there.

    Whatever you do, you need everyone's cooperation, which makes me think that we shouldn't do anything.

    But definitely, having an evolvable code model would be interesting if well thought-out. It would also have applications to application availability.

    Guglielmo

    Enjoy the Fastest Known Reliable Multicast Protocol with Total Ordering

    .. or the World's First Pure-Java Terminal Driver
  15. Right now you don't have to assign release numbers to code which makes it cheap to develop applications in the best case scenario, which the vast, vast majority of code out there.Whatever you do, you need everyone's cooperation, which makes me think that we shouldn't do anything.But definitely, having an evolvable code model would be interesting if well thought-out.

    Actually the idea of this is that it requires no cooperation. It is an "opt in/out system". Meaning if you compile from source you get a unique version at the class level (of everything seperately, I might be persuaded that an assembly level system ought to be able to override the lower class level). At deployment time I can decide to just [*.*.*:*.*.*] and thus disable version linking entirely. The static version and linkage meta data would happen via the compiler automatically. They could be overridden via configuration. OR you might say javac --noversions or something to assure present behavior (As specificed). Does that make sense?
  16. Actually the idea of this is that it requires no cooperation. It is an "opt in/out system". Meaning if you compile from source you get a unique version at the class level (of everything seperately, I might be persuaded that an assembly level system ought to be able to override the lower class level). At deployment time I can decide to just [*.*.*:*.*.*] and thus disable version linking entirely. The static version and linkage meta data would happen via the compiler automatically. They could be overridden via configuration. OR you might say javac --noversions or something to assure present behavior (As specificed). Does that make sense?

    What I meant to say is that it requires everyone's cooperation in order for the system to work effectively. I understand that it won't cause needless incompatibilities, but for it to achieve the effect that you want people do have to opt in.

    Why can't you just allow users to override libraries as needed?

    Guglielmo

    Enjoy the Fastest Known Reliable Multicast Protocol with Total Ordering

    .. or the World's First Pure-Java Terminal Driver
  17. To be clear:

    How appservers should isolate threads/classloaders from seeing what each other load is othogonal to class identity when they can see classes from each other.

    For those commenting on "why doesn't Java work better" and "I hope the next great langauge". I'd encourage you to look at .NET and (http://www.theserverside.net/articles/showarticle.tss?id=AssemblyVersioning).

    The system I mention would allow for instance:

    MyApp.ear
      myApp.war
        WEB-INF/lib
         mylib1.war
         mylib2.war
         commons-logging1.jar
         commons-logging2.jar

    Where MyApp.ear contains all of the children.
    mylib1.war indeed is linked to commons-logging1.jar
    mylbi2.war is indeed linked to commons-logging2.jar
    MyApp.ear defines equivilence of the classloader boundry.

    In JBoss (default configuration of all classloaders):

    this will fail in a few places, first if the version of commons-logging (as a limitation of tomcat) that Tomcat uses conflicts in some incompatible way -- then the war may fail to deploy properly. If it doesn't fail there then if commons-logging1 and 2 are incompatible then mylib1 or mylib2 is going to get the short end of the stick and fail to load properly (when the class dependencies are resolved).

    In WebLogic:

    This will fail more or less the same way as JBoss minus the Tomcat stuff.

    In WebSphere:

    More or less same as weblogic.

    Today:

    Presently the "best" solution is to wrap the jars in some kind of wrapper and do something bizzare like pass by value (which is never optimal in local calls).

    Otherwise you can fork commons logging under org.foobar or something and resolve the linkage by hand. (Which will earn you great scorn as well) Consider doing that for every possible library conflict? What if you do need to share compatible references?

    Thus an example of the same thing that doesn't involve shared classloaders :-).

    Part of the problem is indeed how Joe (and I) framed this. Take J2EE out of the picture. This is a Java problem in general and a computing problem beyond that frankly. I've had similar problems in Ruby when installing the various libraries that various apis depend on.

    What I'm proposing is (compiler produced) static but overridable version linkages and removing runtime class identity (clinstance+FQN) as a fixture with backward compatibility with the present system when the static linkages are not present. I'm proposing overrides for those linkages as an aspect of classloader configuration.

    I am very interested in constructive feedback (including criticism) on this. However, only technical in nature (not bob lee's apparent style).
  18. The system I mention would allow for instance:

    MyApp.ear
      myApp.war
        WEB-INF/lib
        mylib1.war
        mylib2.war
        commons-logging1.jar
        commons-logging2.jar

    Where MyApp.ear contains all of the children.
    mylib1.war indeed is linked to commons-logging1.
    jarmylbi2.war is indeed linked to commons-logging2.jar
    MyApp.ear defines equivilence of the classloader boundry.

    Either the two clogging jars are identical, in which case the Jboss flat model could work, or they are not identical, in which case there is no explicit indication of linkage.
    What I'm proposing is (compiler produced) static but overridable version linkages ..

    Yes, perhaps something like that. We could build Maven in as well and get automatic JAR downloading ;-)

    Classloaders do handle hierarchical scoping easily. For example:

    MyApp.ear
      myApp.war
        WEB-INF/lib
        mylib1.war
          commons-logging1.jar
        mylib2.war
          commons-logging2.jar

    That would enable scoping, but hierarchical scoping breaks sharing, so if mylib3 is introduced and wants to use clogging v1, it has to:

    MyApp.ear
      myApp.war
        WEB-INF/lib
        mylib1.war
          commons-logging1.jar
        mylib2.war
          commons-logging2.jar
        mylib3.war
          commons-logging1.jar (another copy)

    So, it could be done in a manner that allows explicit version linking.

    MyApp.ear
      myApp.war
        WEB-INF/lib
        mylib1.war (i use clogging v1)
        commons-logging1.jar (my name is clogging, version v1)
        mylib2.war (i use clogging v2)
        commons-logging2.jar (my name is clogging, version v2)
        mylib3.war (i use clogging v1)

    And perhaps even:

    MyApp.ear
      myApp.war
        WEB-INF/lib
        mylib1.war (i use clogging v1)
        commons-logging1.jar (my name is clogging, version v1)
        mylib2.war (i use clogging v2)
        commons-logging2.jar (my name is clogging, version v2)
        mylib3.war (i use clogging v1)
        mylib4.war (i use clogging)
        mylib3.war (i use clogging v2 or later)
    I am very interested in constructive feedback (including criticism) on this. However, only technical in nature (not bob lee's apparent style).

    If you were truly confident in your knowledge, you wouldn't care what Bob said or how he said it. Frankly, I thought your original post was crap, but your follow-ups are almost starting to make me wonder if you do possibly have some previous experiences with classloading.

    Peace,

    Cameron Purdy
    Tangosol Coherence: Clustered Shared Memory for Java
  19. I know the maven bit was a joke, but dear god forget maven ever existed. the horror, the horror.

    on a less silly note, alot of people do complain about shared libs with Tomcat. For example, if someone uses the latest version of JDT for a compiler, chances are it won't work with tomcat. Since tomcat uses JDT to compile the JSP's, one either has to use the same version of JDT, or fork another process using ANT. I've written custom classloaders in the past and I can understand people struggling with it. The first time I tried to extend URLClassLoader to dynamically compile and load schemas, it was rather challenging.

    peter
  20. Cameron, let me make sure I understand correctly. The above shows one peculiar classloading structure where you have a classloader at each level with straight parent-child delegation. That would be one big fat package with a whole lot of duplication. Yuck.

    So your extension proposes package assembly versions. I kind of like that as an over-ride rather than default behavior. An advantage of this is that my class versioning scheme will break if mylib1 tries to pass something from commons to mylib2 (though as opposed to the classloading is versioning scheme it does not break if they pass anything else) even if it was compiled from the same .java file (this is deliberate because I don't know what bugs or whatever were in your compiler and if I have compatible bugs, etc -- it is a paranoid system). However your system requires more by default from the packager. And people don't like trying real hard :-) Presumably if from the same maven repository they'd be properly linked against dependencies from that repository. So I like it but what do you think of the static compilation versioning as a default and "my name is" package versioning as an override?

    No I don't really care what Bob thinks but was hoping for constructive feedback and thus am asking that respondants take this up a level above slashdot rather than our usual flamewar fare (part of why you'll notice that I have very rarely commented here).
  21. Andy -

    I've been playing with ideas on library versioning for a while. I haven't had much time to work with .NET assemlies, but I think it's a little bit similar. Basically, as an application developer, you need the ability to dictate what versions of what libraries you use (aka support). Yet, many times you don't even care, as long as the API isn't broken (i.e. as long as there is binary compat at the API level.) So when I deploy my app, I'd like to say "it uses library X and I've tested v2 and v3 of that library" or even "it must only use v2.3.5.1.32.1.18" (apologies to Oracle).

    Linking is a conceptual stage at which the person (or process) driving the process can say "pull it all in right now" or "dynamically link to those libs" or anything in between. It would be nice if the app could be deployed without X (maybe X is optional), but with enough information to find X. For example, there could be an X.JAR included that only has a manifest, and that manifest says:

    library-name: X
    required-version: 2.3.5.1.32.1.18
    site: archive.X.org
    alt-site: archive.lostandfoundjars.org

    Now the app sends off an http-get to archive.X.org asking for http://archive.X.org/X/2.3.5.1.32.1.18 (or be more creative with the hierarchies and/or names).

    The result could be a JAR or a redirect to a mirror or ..

    Regardless of the ability to locate JARs as necessary, it is inevitable that different parts of an app will either embed or use/require different versions of the "same" JAR (i.e. X). It is possible to load those classes "in the scope of" each of the "using" application components, but I think that it is also possible to share compatible class instances across multiple classloader contexts within the same hierarchy. Going back to the previous example, it would appear to be a "diamond problem" with the shared lib being used by two different libs within the same application, but conceptually, if the shared lib is loaded by a classloader that is a contextual peer of the other libs within the application, and if the loadClass on the app lib knew when to delegate to the classloader of the shared lib, then it should be possible to use (share) classes loaded by the peer classloader without having to actually nest it.

    The limitation is that the references cannot go both ways, i.e. when you have a shared class loader (what Sun refers to as a delegation model), it cannot (or it is provably ugly to) load classes from the libraries that are in turn sharing that library.

    Unfortunately, the classloader implementation has grown progressively more complex (and ultimately more rigid) as time has gone on. Maybe we should roll back to the JDK 1.0.2 version ;-)

    Peace,

    Cameron Purdy
    Tangosol Coherence: Clustered Shared Memory for Java
  22. Isn't the idea of using a flat classloader model along with dependency definitions kind of what the OSGi classloading model does?

    I haven't gotten around to try it out in practice, but from the looks of it OSGi should solve all of these problems fairly neatly. Why reinvent the wheel?... (he said rhethorically, indifferently, and not entirely without a certain sense of deja vu)
  23. Wow, now that makes sense![ Go to top ]

    If you were truly confident in your knowledge, you wouldn't care what Bob said or how he said it. Frankly, I thought your original post was crap, but your follow-ups are almost starting to make me wonder if you do possibly have some previous experiences with classloading.

    I for one haven't got a clue about classloaders, other then they are complicated and don't/didn't work that well for me. I remember a particularly bad problem we once had with Weblogic, where we wanted two "components" packaged as seperate ears to talk to each other.

    Well we spent weeks and couldn't get it to work. We ended up having to unpackage an "off the shelf" ear component and repackage it's ejbs into our own application ear just to get the thing to work - so much for a component model and a market of off-the-shelf components!

    I've had problems "late binding" with RMI too and I once tried to use Applets and found I had to download a tonne load of jars to the browser each time so gave up.

    So my question is does Java have an intrinsic problem with late-binding and mobile code? and why hasn't Sun got round to fixing it already?

    Dynamic languages seem able to do all this stuff quite readilly without all the fuss. Java has always felt broken to me in this regard.

    Paul.
  24. Wow, now that makes sense![ Go to top ]

    I remember a particularly bad problem we once had with Weblogic, where we wanted two "components" packaged as seperate ears to talk to each other.Well we spent weeks and couldn't get it to work. We ended up having to unpackage an "off the shelf" ear component and repackage it's ejbs into our own application ear just to get the thing to work - so much for a component model and a market of off-the-shelf components!

    Did you try invoking the "off the shelf" EJBs through their remote interfaces? If so, did you remember to package the client jars for EJBs from the "off the shelf" ear in your ear?

    If you don't like how this works, you have a problem with the J2EE spec. or the vendor's implementation of the spec., not class loaders in general.
    I've had problems "late binding" with RMI too and I once tried to use Applets and found I had to download a tonne load of jars to the browser each time so gave up.So my question is does Java have an intrinsic problem with late-binding and mobile code? and why hasn't Sun got round to fixing it already?Dynamic languages seem able to do all this stuff quite readilly without all the fuss. Java has always felt broken to me in this regard.Paul.

    When you serialize an object, you serialize an entire object graph, i.e. every object directly and indirectly referenced from the root object. When the client deserializes that data, it must have the classes for the objects. Otherwise, how would it know what to do with the data?

    If you don't want the client to need so many jars, the solution is to not pass objects of those types to the client. The DTO pattern addresses this problem.
  25. In JBoss (default configuration of all classloaders):this will fail in a few places, first if the version of commons-logging (as a limitation of tomcat) that Tomcat uses conflicts in some incompatible way -- then the war may fail to deploy properly.

    I don't use Tomcat, so I don't know for sure, but it sounds line Tomcat is putting commons logging on the system classpath when it shouldn't.
    I am very interested in constructive feedback (including criticism) on this. However, only technical in nature (not bob lee's apparent style).

    I think you're trying to provide a solution before you've identified a concrete problem. You've indirectly touched on many different and possibly exclusive problems, and I think trying to fix them all at once would be fruitless. Perhaps JBoss's class loader infrastructure suffers the same problem? In any case, enabling users to treat two independently loaded classes (even if they have the exact same bytecode) will probably do more harm than good.

    I'm also surprised you haven't mentioned JSR 277: http://jcp.org/en/jsr/detail?id=277
  26. I'm also surprised you haven't mentioned JSR 277: http://jcp.org/en/jsr/detail?id=277

    Yeah, I was wondering when that would be mentioned. It seems to be solving exactly this issue, if it gets through...
  27. I believe Andy clearly outlined the problem:
    Instances of the same class definition loaded by different class-loaders (due to tier/module separation) are incompatible when passed between the tiers/modules. I believe this is a common problem.
  28. I believe Andy clearly outlined the problem:Instances of the same class definition loaded by different class-loaders (due to tier/module separation) are incompatible when passed between the tiers/modules. I believe this is a common problem.

    Why are you loading the same class twice? That sounds like the real problem.

    Keep in mind that using a hierarchical model for class loaders is recommended but not mandatory (not that I would ever use a non-hierarchical model because I understand the tradeoffs).
  29. ...even under jboss. Just get the Manifest Class-Path entries correct.

    But maybe I'm just puzzled because I never nested WARs inside WARs
  30. I agree!

     Moreover your example shows the problems which do derive from badly designed Library evolution.
     consider this example:
    logging1.jar:
      com.sample.logger.Logger

    logging2.jar:
      com.sample.logger2.Logger

    Here logger1.jar defines a contract with users and requires/exposes certain methods. As long as the interface is met logger1.jar may continue to evolve without pain.

    Once you decide to create a completly incompatible interface you should be aware of that fact and place the interface into another package + jar: this way both interfaces can coexist in ther same application (classloader space, if you like to this it this way...)

    apache commons logging has violated this rule and for a certain period we all ended up mad because our projects included an httpclient which requires logging v2 and pdf generator which relies on v1 ... :-/

    APIs and Contracts are well-known-concepts of software engineering
    Classloader hacks _might_ work as surrogates to unfortunate design choices but this is _NOT_ a good practice!
  31. The system I mention would allow for instance:MyApp.ear  myApp.war    WEB-INF/lib     mylib1.war     mylib2.war     commons-logging1.jar     commons-logging2.jarWhere MyApp.ear contains all of the children.mylib1.war indeed is linked to commons-logging1.jarmylbi2.war is indeed linked to commons-logging2.jarMyApp.ear defines equivilence of the classloader boundry...

    To be honest, I haven't tried your example, but I know that setting the Class-Path Manifest Entry in EJB jars contained in ears allows you to link a ejb jar (contained in an ear) to arbitrary other jar (also contained in the ear). Works like specified. That worked at least for weblogic 7...

    However, I think it doesn't make real sence to mix different versions of libs like you proposed. First it might look like a convenient feature, but my opinion is that it would introduce more complexity than it solves.

    Look at OSGI Classloading. Basic Features are wonderfull: Bundles have their classloader and define exports to and imports from other bundles on a package level. OSGI spec evolved to have complex, version-related exports and imports. From easy to grasp to overly complex, I would say...

    And tell me: Why would you want to do it? Is your proposal not like a cure for "technical project managment that needs improvement"? If your team members deliver such a lib-mixture that clarifies one thing: Get them out of their cubicles to talk to each other...

    Cheers - Peter
  32. It is an interesting idea that the JVM could have an option to allow for automatic type compatibility among objects constructed from binary-identical classes that were defined within different classloader contexts.

    However, the technical explanation is not correct:
    .. if I have the same (down to the inode of the filesystem) class file loaded by two different classloaders and I (for instance) assign the two classloaders to two different threads (Thread.currentThread().setContextClassloader()) then pass values of the type defined by that class between the threads -- I will get an "instanceof" failure which may manifest itself as a ClassCastException.

    Simply put, you can share objects from binary-identical (or binary-different) class definitions that are loaded by any number of different classloaders among any number of different threads without any problems, unless you assume that you can cast some arbitrary object (e.g. one that you blindly assume is of type com.myco.Foo) to a particular type (e.g. com.myco.Foo). You can cast it to any super or interface type that is common to both contexts, such as java.lang.Object etc. You just can't down-cast it to a type that is separately-loaded by both of the classloaders.

    The Java "instanceof" and cast operations work correctly. If instanceof returns true, the cast succeeds. If instanceof for a non-null value returns false, the cast fails.

    Part of the problem is that some programmers assume that some object passed in a message somewhere or tucked into a session or jammed into a request is of a particular type, regardless of context.

    (As an interesting aside, try to write a classloader that loads / defines its own java.lang.Object.)
    It should be noted in the interests of fairness to all sides that while JBoss has been maligned for its classloading structure, every application server suffers from some degree of "classloader hell"..

    Joe, unless you were planning to pummel Jboss for its classloading approach, I'm not sure why you'd introduce this argument ;-)

    Peace,

    Cameron Purdy
    Tangosol Coherence: Clustered Shared Memory for Java
  33. Cameron, yeah my explanation sucked....attempted to be terse...instead achieved inaccuracy.
  34. Classloading seems limiting and terrible. But as you dig down into it, and see what's really going on, you keep finding that each annoyance and limitation is there for a reason.

    ClassCastException just because of a difference in class loader is perfectly valid. The name is the same, the bytecode on the disk may or may not be the same, the class in memory may be very different (I've been working on code that uses AOP techniques to transform classes as they are loaded).

    Many of the other limitations seem that way when you think of a class loader as something that loads .class files from a .jar file on a disk. When you realize all the very necessary variations, including loading over the network, or supporting multiple class versions simultaneously, the limitations make sense.

    Not that Sun couldn't make things easier for a lot of typical cases.
  35. Changing the class defintion at runtime would mean assigning a new class-version to it as well distinguishing it from the compiled class-version on disk in the event it gets loaded by another class-loader.
  36. Classloading seems limiting and terrible. But as you dig down into it, and see what's really going on, you keep finding that each annoyance and limitation is there for a reason.

    Yes, but is this just a natural consequence of late-binding, or due to intrinsinc limitations in the Java Language?

    Dynamic languages just don't seem to suffer from these "annoyances and limitations" or do they?

    Paul.
  37. Classloading seems limiting and terrible. But as you dig down into it, and see what's really going on, you keep finding that each annoyance and limitation is there for a reason.
    Yes, but is this just a natural consequence of late-binding, or due to intrinsinc limitations in the Java Language?Dynamic languages just don't seem to suffer from these "annoyances and limitations" or do they?Paul.

    They do. For example, there are issues in an image-based language like Smalltalk as to how you handle changes - do you allow instances of the class as it used to be to still exist, or do you have some migration method to convert all instances, or do you allow only new instances to have the new 'shape'? There are many different approaches: the Smalltalk application server Gemstone has a neat class versioning system.
  38. Classloading seems limiting and terrible. But as you dig down into it, and see what's really going on, you keep finding that each annoyance and limitation is there for a reason.
    Yes, but is this just a natural consequence of late-binding, or due to intrinsinc limitations in the Java Language?Dynamic languages just don't seem to suffer from these "annoyances and limitations" or do they?Paul.
    They do. For example, there are issues in an image-based language like Smalltalk as to how you handle changes - do you allow instances of the class as it used to be to still exist, or do you have some migration method to convert all instances, or do you allow only new instances to have the new 'shape'? There are many different approaches: the Smalltalk application server Gemstone has a neat class versioning system.
    Hi Steve,

    Yes, I believe this is the next layer of the onion. How do you bind objects whose classes have changed dynamically. Gemstone sounds interesting, I've heard good things about it too.

    Paul.
  39. >Hi Steve,Yes, I believe this is the next layer of the onion. How do you bind objects whose classes have changed dynamically. Gemstone sounds interesting, I've heard good things about it too.Paul.

    It is good - I had a chance to evaluate Gemstone/Smalltalk in the late 90s - very impressive.
  40. The link is broken. Can somebody please check the link or
    inform Andy Oliver.

    Cheers
    Alan Mehio
    London, The City
  41. Manifest versions[ Go to top ]

    Java actually had a nice start in that you could specify specification and implementation versions for "packages" within manifests. When i first saw this, I thought I had stumbled upon a way to have multiple implementations of the same interface and determine compatibility.

    However, it seems its there purely as optional information. Not all manifests (indeed, hardly any) supply this information and nothing at all seems to leverage it, much less enforce it.

    I agree that Java needs a formal versionsins sol'n, and I htink what theys tarted in the manifest is the right idea. They just need to finish it, or start anew but with the same spirit.
  42. Manifest versions[ Go to top ]

    I agree that Java needs a formal versionsins sol'n, and I htink what theys tarted in the manifest is the right idea. They just need to finish it, or start anew but with the same spirit.
    Or perhaps just use the existing OSGi dependency, versioning, and classloading model? Why reinvent the wheel?... (he said rhethorically, indifferently, and not entirely without a certain sense of deja vu)
  43. Really naive solution[ Go to top ]

    For many projects I've worked on it seemed to make sense to have one application per container instance. Class loading issues for the most part go away, the container can be tuned more closely to the apps needs, container modules not needed by the app can be removed (modular container design like jboss' makes this easy).

    The versioning idea seems to fix one problem but may add others. For instance, what if I don't know the version of the class I'm loading but just want to use it if it implements an interface? Is versioning optional? I'm thinking of dynamic class loading where you replace a module (jar) and the application reads it in without having to shutdown and restart.

    Maybe there are solutions that would partition a JVM so that what gets shared/versioned is all configurable (not that additional configuration makes things easier).

    Instead of adding versioning, would making the classloader spec more specific on how to handle common situations make more sense?