Rod Johnson Clarifies His Thoughts on OSGi

Discussions

News: Rod Johnson Clarifies His Thoughts on OSGi

  1. In an interview TheServerSide did with Rod Johnson, after discussing the core Spring framework and the evolution of the Spring TC Server, Rod was asked about his take on the topic of OSGi. TheServerSide transcribed his comments on the topic while also providing the audio of what was said, all under the headline of “OSGi: Not Easy to Use. Not as Productive as it Should Be, says Rod Johnson.”

    Perhaps not surprisingly, the article garnered a great deal of attention, a barrage of comments from those in the OSGi community in defense of their technology, a few articles discussing OSGi's ease of use, and it even got Rod Johnson to log into TheServerSide for the first time in over two years to clarify his position, while providing a proper context for his comments.

    In fairness to both Rod and the OSGi community, we’re republishing Rod Johnson’s comment on the original article, allowing the General Manager of the SpringSource Division of VMWare to more properly frame his stance on the discussion.


    "I’d like to add some clarification here, and say a little more about our experience with OSGi and my conclusions from it—note the emphasis on my, as my opinions aren’t always those of SpringSource or VMware.

    "We made a major investment in OSGi in creating our dm Server product. We subsequently took that technology to Eclipse, where it continues to move forward, with our continued input. We have an ongoing revenue stream from our OSGi-related technologies,with customers including some household name companies. We plan to maintain our current level of commitment to OSGi.

    "The lessons I personally drew from the experience were that:

    (a) OSGi is a great solution for complex applications with stringent modularity requirements;
    (b) typical business applications (from which we make the bulk of our revenue) don’t have such requirements;
    (c) our efforts to reduce the complexity of writing server-side OSGi applications were promising, but the road to simplification was longer and less certain than we’d hoped.Thus continuing down that road at the Eclipse Foundation, in partnership with other companies and individuals, was a natural move.

    "I don’t think OSGi is a bad technology—merely one that isn’t applicable to every application out there and one which therefore didn't belong at the core of our business. If you don’t have modularity requirements, the simplest thing that can possibly work won't have a module system. I applaud the fact that the Virgo project continues to make progress in simplifying the OSGi experience, and am proud that our engineers play a major role."

    Rod


    Related Links
    Rod Johnson Speaks about OSGi
    Comments and discussions on the original Rod Johnson Article
    Peter Kriens on OSGi and Ease of Use
    SpringSource and vFabric
  2. So Rod confirms what he previous said about OSGi.

     I can agree with him: I studied and used spring dm-server and OSGi in deep (some year agoo);

    I appreciated SpringSource's efforts in order to simplify OSGi, but they could not do more: the technology is too complicated and modularity is not the target for the 80 % of java/j2ee apps.

    I don't know why the technology is too complicated: is it the technology itself or the problem is not solvable with a simple solution?

  3. Simple vs. Complicated[ Go to top ]

    My take on the discussion is that with simple problems and projects, needlessly using OSGi would be an added complication that just isn't justified. But on more complicated projecs, OSGi is a technology that can be used to greatly simplify them. 

    Glassfish and JDeveloper? They swear by OSGi, and the performance impact on their software is palpable.

  4. Simple vs. Complicated[ Go to top ]

    I actually think business applications that use osgi is akin to code smell for the architecture. Most likely there are other dysfunctional issues that don't involve technology.

  5. good technology solves the right problem but too complicated and not much traction. ?

    also can j2ee spec be changed around class loading out of box ?

    all this round about way of going into manifest for dependency etc isnt easy. why cant SUN/ORACLE change the class loading of jvm and provide facility to load different versions.  

  6. This is a helpful clarification, though Rod still has it backwards in my opinion. Think for a moment about what a "complex application" means. Is the application YOU are developing not complex? How are you supposed to decide whether you have "stringent" modularity requirements, or just ordinary ones?

    What Rod calls "typical business applications" are amongst the largest and most complex software projects! To suggest that they don't need modularity, while products like WebSphere, Glassfish, JDeveloper etc do, stretches credulity. Worse, the typical business application is usually developed by programmers of much more variable quality than those assigned to work on vendor products.

    Now as I said in my comment on the original thread, I agree with Rod that productivity is not yet as high with modular development as it should be. Also, there are some projects -- small, simple ones -- where modularity is unnecessary. However it's clear to me that modular development techniques are already more productive than the typical "big ball of mud" architecture, into which non-modular projects inevitably degenerate. That's why vendors like IBM and Oracle have adopted OSGi. With the tools getting better all the time, it is only a matter of time before modular development with OSGi is considered more productive and more manageable than old-style flat classpath development.

  7. This is a helpful clarification, though Rod still has it backwards in my opinion. Think for a moment about what a "complex application" means. Is the application YOU are developing not complex? How are you supposed to decide whether you have "stringent" modularity requirements, or just ordinary ones?

    I've written plenty of applications that are both modular and not complex. There are a number of tools to achieve modularity: A modular build, webapps and shared web services solve 99.9% of people's modularity needs. It's not complex and it really shouldn't require too much thought or worrying about.

    What Rod calls "typical business applications" are amongst the largest and most complex software projects! To suggest that they don't need modularity, while products like WebSphere, Glassfish, JDeveloper etc do, stretches credulity. Worse, the typical business application is usually developed by programmers of much more variable quality than those assigned to work on vendor products.

    I don't think that anyone is suggesting that applications not be modular. It's just that we don't need OSGi to do that and it gets in the way. Having a shared artifact in a maven repository that other builds can reuse is what most people end up needing. They don't need to replace a single jar at runtime on the fly.

    Now as I said in my comment on the original thread, I agree with Rod that productivity is not yet as high with modular development as it should be.

    I don't think it's ever really possible for it to be, it just places too much burden on developers.

    Also, there are some projects -- small, simple ones -- where modularity is unnecessary.

    modularity != OSGi.

    However it's clear to me that modular development techniques are already more productive than the typical "big ball of mud" architecture, into which non-modular projects inevitably degenerate.

    Somehow the customers I visit manage not to have a big ball of mud. Maybe it's that I'm in the middleware space, but I tend to see a natural progression. Project starts of small, project grows, project gets refactored into different build artifacts or different services that different teams use.

    That's why vendors like IBM and Oracle have adopted OSGi. With the tools getting better all the time, it is only a matter of time before modular development with OSGi is considered more productive and more manageable than old-style flat classpath development.

    I'll wager you a beer that in 3 years OSGi development will still be more unproductive than regular flat classpath development. :-)

    Dan Diephouse

    MuleSoft

  8. Well said Dan. I think there is a difference between modularity and separation/segregation using standard language constructs (interfaces, layers, SOLID) and a modularity platform. You can go a long way without OSGI unless of course, you are building an app server, or ide, or platform. Most business applications that I have seen, don't need osgi. In some cases, I've seen pretzel twists with Java and Spring DM that could easily be done with 1/100th the amount of code in Drupal.

  9. This is a helpful clarification, though Rod still has it backwards in my opinion. Think for a moment about what a "complex application" means. Is the application YOU are developing not complex? How are you supposed to decide whether you have "stringent" modularity requirements, or just ordinary ones?

    I've written plenty of applications that are both modular and not complex. There are a number of tools to achieve modularity: A modular build, webapps and shared web services solve 99.9% of people's modularity needs. It's not complex and it really shouldn't require too much thought or worrying about.

    What Rod calls "typical business applications" are amongst the largest and most complex software projects! To suggest that they don't need modularity, while products like WebSphere, Glassfish, JDeveloper etc do, stretches credulity. Worse, the typical business application is usually developed by programmers of much more variable quality than those assigned to work on vendor products.

    I don't think that anyone is suggesting that applications not be modular. It's just that we don't need OSGi to do that and it gets in the way. Having a shared artifact in a maven repository that other builds can reuse is what most people end up needing. They don't need to replace a single jar at runtime on the fly.

    Now as I said in my comment on the original thread, I agree with Rod that productivity is not yet as high with modular development as it should be.

    I don't think it's ever really possible for it to be, it just places too much burden on developers.

    Also, there are some projects -- small, simple ones -- where modularity is unnecessary.

    modularity != OSGi.

    However it's clear to me that modular development techniques are already more productive than the typical "big ball of mud" architecture, into which non-modular projects inevitably degenerate.

    Somehow the customers I visit manage not to have a big ball of mud. Maybe it's that I'm in the middleware space, but I tend to see a natural progression. Project starts of small, project grows, project gets refactored into different build artifacts or different services that different teams use.

    That's why vendors like IBM and Oracle have adopted OSGi. With the tools getting better all the time, it is only a matter of time before modular development with OSGi is considered more productive and more manageable than old-style flat classpath development.

    I'll wager you a beer that in 3 years OSGi development will still be more unproductive than regular flat classpath development. :-)

    Dan Diephouse

    MuleSoft

     

    I agree with most of what you said but shared classloader are still a pain in the ass. I can't recall how many times I've lost a day of work because of a bizarre ClassNotFoundException and Maven transitive dependencies are sure not helping there. 

    As an enterprise developer, my main need is a solution allowing my different modules to use their own isolated classloaders. That's all I care really. I want to be able to upgrade the libs used by my application modules at a different pace. Right now the only solution is to deploy each module in his own application, something that is really cumbersome when you don't need your module to be accessed remotely.

  10. John Wells, who at the time was a senior architect at BEA (pre-acquisition), talked about why they adopted OSGi across their entire product stack. The quote I remember was this:

    We thought we were doing modular development until we started using OSGi. 

    It's true that you can avoid OSGi for a time by using careful development techniques, but ultimately your application will not be modular until you try it in an environment that enforces the boundaries between modules. How do you know that your "modules" are really decoupled from other modules? Hint: if you use Spring to hide dependencies in a big ball of XML and Reflection, they are not. How do you know that your modules are reusable? Unti you actually try to reuse them, they are probably not.

    I'll take that bet or a richer one if you prefer. The problem will be finding somebody objective to adjudicate. Some people refuse to accept that you have made their lives easier even when presented with incontrovertible evidence of that fact.

  11. Neil,

    As you might already know, both Java SE and Java EE will bring modularity to the table in the not-so-distant future. I think it is best to see if OSGi interoperability can be achieved through that vehicle. If in the process OSGi gets sanitized/abstracted a bit, that would be good overall, correct?

    Cheers,

    Reza

  12. Hi Reza. It's still very unclear when and how Java SE and EE will offer modularity, especially since the SE effort (aka Jigsaw) has gone back to the drawing board in the form of a Requirements Document -- and incidentally those requirements are very closely aligned with what OSGi offers.

    Then if SE defines anything new it will take another few years to have any effect on EE. On the other hand, OSGi is here now and has already been adopted and exposed by all the major EE vendors.

    For any company that regards modularity as a business benefit or even a competetive advantage -- and plenty do -- it would be stupid to wait years for EE to rubberstamp what is already the gold standard for modularity in Java.

  13. Neil,

    I'm not one to kick anyone when I think they are already down, but the point I was trying to make is that standardization via Java SE and Java EE might be one useful vehicle for working on the OSGi complexity issues folks are concerned about. In the end, OSGi containers can become the de-facto implementations for Java SE and Java EE modularity, albeit exposed through a different API/abstraction level. Oracle has significant supporters of OSGi, so this should be entirely doable.

    Cheers,

    Reza

  14. Dan, very well said. I totally agree with you. modularity != OSGi. Mule is a very well designed modular product itself which I have been using happily in the last couple of years. It's not OSGi, maven has done an amazing job to help me use Mule's modular design.

    Almost all of my projects, 50+ of them, big or small, are somehow modular, but none of them requires OSGi to do the job. Modular design with clear interfaces, modular build system like Maven and a well planned class loader is most what I need. I guess because I am working for projects of system integration, not a framework or toolkit.

    I never required the OSGi way of hot swap of a newer modular at runtime. I have been achiving this by versioning web services and versioning workflows for some long runing processes. For typical OLTP applications, I always have to redeploy the entire application.

     

  15. This is a Mule Soft error[ Go to top ]

    Mule Soft failed to get OSGi working.

    Instead of acknowledging that failure, they came out and bashed OSGi.

    This is a Mule Soft error.

  16. OSGi is sound[ Go to top ]

    Responding late. Sorry.

    The nature of this issue is so clear that I am surprised people cannot see it.

    First, to Dan Diephouse who correctly states "modularity != OSGi", I say "This doesn't mean that the two are inconsistent. Thanks for all the FUD." Mule Soft publicly failed to modularize its architecture with OSGi. You have a vested interest in discrediting OSGi lest your company be found guilty of creating a product which is incapable of being modularized.

    Second, special thanks are in order to Tao Zhang for rubbing noses with Dan. That was nice.

    The essence of this debate is to be found in Neil's comment "ultimately your application will not be modular until you try it in an environment that enforces the boundaries between modules". Those people who say that "OSGi is too hard" fall into two camps: those who understand dependencies, layering, and contracts and those who don't. Nonetheless, these two camps have one thing in common: they do not appreciate enforcement.

    Resentment of enforcement is a common thread. Many developers chose NoSQL solutions over constraint enforced relational databases. Similarly, front-end developers chose JSON over schema validated XML. Finally, many developers chose lighter weight processes over heavier ones. Those who resist the enforcement of modularity are simply part of a larger phenomenon.

    Back to those people who think that OSGi is too hard: While some of the members in this group do truly understand modularity, they fail to appreciate the cumulative effects of many developers and software maintainers and they fail to appreciate the real rot that bodies of software experience over their lifetimes.

    I believe in the need for modularity enforcement just as I believe in the need for constraint checks and message validity checks. I blame agile deviants for the "OSGi is too hard" movement. While I applaud the agile camp's ability to produce prototypes quickly and produce better teams, I find agile deviants guilty of moving far too many of those prototypes into production, leading to unmaintainable codebases. While I respect the abilities of those who understand modularity without OSGi, I nonetheless blame them for failing to apply modularity enforcement in order to combat software decay. Finally, I blame Tao Zhang and those who don't understand modularity for taking a dump on OSGi in order to make yourselves feel less inadequate. Downloading the internet (maven) is NOT modularity.

  17. OSGi is sound[ Go to top ]

    Responding late. Sorry.

    The nature of this issue is so clear that I am surprised people cannot see it.

    First, to Dan Diephouse who correctly states "modularity != OSGi", I say "This doesn't mean that the two are inconsistent. Thanks for all the FUD." Mule Soft publicly failed to modularize its architecture with OSGi. You have a vested interest in discrediting OSGi lest your company be found guilty of creating a product which is incapable of being modularized.

    Second, special thanks are in order to Tao Zhang for rubbing noses with Dan. That was nice.

    The essence of this debate is to be found in Neil's comment "ultimately your application will not be modular until you try it in an environment that enforces the boundaries between modules". Those people who say that "OSGi is too hard" fall into two camps: those who understand dependencies, layering, and contracts and those who don't. Nonetheless, these two camps have one thing in common: they do not appreciate enforcement.

    Resentment of enforcement is a common thread. Many developers chose NoSQL solutions over constraint enforced relational databases. Similarly, front-end developers chose JSON over schema validated XML. Finally, many developers chose lighter weight processes over heavier ones. Those who resist the enforcement of modularity are simply part of a larger phenomenon.

    Back to those people who think that OSGi is too hard: While some of the members in this group do truly understand modularity, they fail to appreciate the cumulative effects of many developers and software maintainers and they fail to appreciate the real rot that bodies of software experience over their lifetimes.

    I believe in the need for modularity enforcement just as I believe in the need for constraint checks and message validity checks. I blame agile deviants for the "OSGi is too hard" movement. While I applaud the agile camp's ability to produce prototypes quickly and produce better teams, I find agile deviants guilty of moving far too many of those prototypes into production, leading to unmaintainable codebases. While I respect the abilities of those who understand modularity without OSGi, I nonetheless blame them for failing to apply modularity enforcement in order to combat software decay. Finally, I blame Tao Zhang and those who don't understand modularity for taking a dump on OSGi in order to make yourselves feel less inadequate. Downloading the internet (maven) is NOT modularity.

     

    This is 100% correct.  MuleSoft failed at and so did Spring.  Rod Johnson fails to mention it was issues with the way that the Spring IoC container was written that caused them so many probablems.  A lot of bad usuage of Classloaders, etc.  

    Most people don't write well designed modular code and do a lot of anti-patterns such as throwing all classes in the same package or using the same package accross different jars.  If you have code written this way, it will be OSGI hell, but that isn't OSGI's fault that is poorly written/designed code.  If you start with OSGI from the beginning it is a a breeze becasue you code and design your bundles to modular from the onset.

    A lot of people talking about not needing hot swap.  Well, once you have the ability to do it sure does come in handy and make devops that much easier in pushing out small or large bugs without full regressions. 

    Also, it isn't hot swap that is so valuable.  It the fact that I can run two libraries of different versions at the same time.  For example, my code might depend on a 3rd party library which depends on a very old version of an Apache Commons library.  My code also uses the same Apache Common library but the latest version.  The two versions are not compatable.  Without OSGI, I am forced to either used the older version so we can all play nicely or do some classloading hacking to get around it.  Anyone who has expereinced knows what a nightmare this can be.  With OSGI, I can deploy both versions and my code can use the lates version while the 3rd party code that my code uses will use the older version.   It is like heaven, if you have ever had to deal with this without OSGI.