So, what's the deal with OSGi? Some of the most respected members of the industry just love it and swear by it. Others are as cold as a cut.

In an interview with Rod Johnson, I had the opportunity to ask him about his take on the technology, and was surprised by his comments, especially given the past time and effort SpringSource had put into OSGi. "We have changed our views on OSGi over the years, and one of the reasons for that is that OSGi simply cannot be made as easy to use and as productive as we feel is consistent with Spring values."

Yet at the same time, others in the industry whom I respect greatly seem to have a diametrically opposed attitude on the technology. The Oracle JDeveloper team swears by it, and credits it for many of the latest advancements and efficiencies they've achieved in their latest product release. Others share the same hands-on experience.

Where is the middleground?

Rod Johnson on When and Where OSGi is Useful

A poor workman blames his tools...? 
Posted By: Richard Nicholson On Jun 21, 2011 3:24 AM  

*Reposted by Cameron McKenzie on behalf of Richard 

To see just how simple and productive an OSGi runtime can be - see the following Paremus Service Fabric / OSGi presentation: this demonstrating a high performance Financial Services Cloud enabled application.

http://skillsmatter.com/podcast/scala/lightning-talks-talk-3-at-scalax

http://www.dzone.com/links/rss/videopresentation_how_to_build_a_high_performance.html

Its simple when you know how.

Paremus believe that OSGi directly addresses the most important Enterprise issue we face today. 'IT Debt' - the issue of System maintenance and environmental complexity. Maintainable Systems must be Modular Systems and OSGi is the industry standard for Java modularity.

A number of companies - including Paremus - are working hard to address these fundamental issues: these issue endemic in both traditional enterprise runtimes; and issue that will be endemic in all Cloud solutions that fail to embrace modularity.

What is he smoking and how can I get some?

He's out of his mind, and he's forgetting his own history. J2EE was easy to use, but lacked documentation, so he wrote up some excellent books that documented an alternative - and that alternative took root and sprung forth very successfully and bought him a boat when he sold it to VMWare,

Now he bet the farm on OSGi because they thought it'd be a nice shiny technology to attract peoples' attention, and because it let them piggy back on eclipse like a cheap frog.

Oops, it's not easy to use! It's just like J2EE back in the day, easy enough but poorly documented and the tools around it aren't easy. But now, instead of fixing the problem, he says to cut and run ... to java ee, with tc server.

I'm sorry, I call foul. Spring is trying to lead people around by the nose, and it's sad that people might go along with it.

The problem with OSGi isn't OSGi, it's the tooling around it. When the OSGi supporters like Spring and Eclipse figure out how to automate manifest generation and remove "you have to understand OSGi" from the equation, OSGi will be accessible to the rank and file and all these "OSGi sucks" proclamations will go away as they should.

Rod, YOU are the problem, just like Peter Kriens and the other people are. When you guys finally realize that OSGi is still a cathedral, and you fix it, your pet technology will be a clothed empror like you want it to be.

Of course, by then rod will hae moved on to something else and he'll just wish he'd stuck with his original idea.

"Oops, it's not easy to use! It's just like J2EE back in the day, easy enough but poorly documented and the tools around it aren't easy. But now, instead of fixing the problem, he says to cut and run ... to java ee, with tc server."

Which isn't EE...

"The problem with OSGi isn't OSGi, it's the tooling around it. When the OSGi supporters like Spring and Eclipse figure out how to automate manifest generation and remove "you have to understand OSGi" from the equation, OSGi will be accessible to the rank and file and all these "OSGi sucks" proclamations will go away as they should."

We autogenerate manifests in all the GlassFish related project under oracle using hk2.  Pretty painless most of the time.  Of course, when it's not it's full on OSGi hell but even that's pretty short lived.

 

it's close enough to ee to pass the common man's evaluation. Ask a normal coder if tomcat is java ee and they'll say yes, even if they're not thinking of limiting it to the web profile. Look at how many people thought they were doing full j2ee with pure tomcat. Same thing here.

tc isn't java ee, but it might as well be for the ordinary coders out there.

hk2 may be a solution but it's not obvious. The tooling *I* think of first is BND, and BND isn't good enough yet. And when I looked up hk2 the tooling nature of it wasn't obvious - and it should be.

I think there is some misunderstanding. GlassFish actually uses maven-bundle-plugin, which uses bnd underneath, to generate OSGi metadata. 

See what I mean?

I think there is some misunderstanding. GlassFish actually uses maven-bundle-plugin, which uses bnd underneath, to generate OSGi metadata. 

Whoops!  My bad, Sahoo.  I got that confused a bit.  But that's actually even better for The Common Man as that's a fairly mainstream options.  hk2 is nice but it brings extra baggage that not everyone might want or need.

(Obligatory disclaimer - I work for SpringSource, and I led much of the effort on OSGi development there back in the day)

Now he bet the farm on OSGi because they thought it'd be a nice shiny technology to attract peoples' attention, and because it let them piggy back on eclipse like a cheap frog.

Since SpringSource is still highly-successful *and* we aren't massive supporters of OSGi, you can clearly see that we did not 'bet the farm on OSGi'. That would have been a daft thing to do no matter the technology.

Oops, it's not easy to use! It's just like J2EE back in the day, easy enough but poorly documented and the tools around it aren't easy. But now, instead of fixing the problem, he says to cut and run ... to java ee, with tc server.

OSGi is not poorly documented - the documentation is excellent. The tools are pretty poor, but getting better. The underlying model is not as easy to understand as J2EE. The OSGi resolver model is an NP-complete problem - not ideal for metadata that developer's are expected to reason about.

I'm sorry, I call foul. Spring is trying to lead people around by the nose, and it's sad that people might go along with it.

Call foul all you like. We put over three years of significant effort into OSGi, but we just couldn't get it to be as easy as we wanted. All the while, we had a user community watching what we were doing. We have a responsibility to all those to developers to give the best advice we can. The fact remans that plain Java EE is still miles easier for most developers than OSGi.

The problem with OSGi isn't OSGi, it's the tooling around it. When the OSGi supporters like Spring and Eclipse figure out how to automate manifest generation and remove "you have to understand OSGi" from the equation, OSGi will be accessible to the rank and file and all these "OSGi sucks" proclamations will go away as they should.

I partially agree. Tooling isn't great, but the underlying OSGi model isn't without its own complexity. I don't see how your argument that OSGi isn't the problem can hold water if you think the problem is that developers have to understand OSGi. Developers need to understand the tools they are working with.

Rod, YOU are the problem, just like Peter Kriens and the other people are. When you guys finally realize that OSGi is still a cathedral, and you fix it, your pet technology will be a clothed empror like you want it to be.

I can't speak for Peter, but I can definitely speak up for Rod here. Rod is NOT the problem. My team spent two years working on trying to move OSGi away from being the cathedral that you mention, mostly under guidance from Rod (and others like Adrian Colyer). Rod was a big proponent of manifest-generation, tooling support and anything that could be done to make OSGi more accessible. I'm guessing you don't know Rod very well, if he's anything, he's a pragmatist.

 

I love any response that includes a reference to NP-complete. 

Great to hear you piping in on the topic Rob. I'd love to hear more from you on your experience with the topic if you'd like to do either an interview or op-ed piece to TSS. 

cmckenzie aht techtarget dawt calm

 

If you are interested in OSGi & NP completeness read Robert Dunne's analysis. 

http://stackoverflow.com/questions/2085106/is-the-resolution-problem-in-osgi-np-complete

But this really is 'By-the-By'. Good OSGi based solutions Developer Tooling and Runtime will leverage the power of OSGi without dragging everyone into these sorts of academic discussion. Just as PC users out there don't care about the simulated annealing techniques used by Intel or AMD to optimise silicon design?

IMO

Every time when I hear people say that "just wait for the tooling to be completed" (in a year or so). that tells me it's too complex. If development completely depends on Netbean/Eclipse/Oracle/xyz, because the underlying framework is too complex to use without the tools then I'm getting the heck out.

Autogenerating xonfiugraiton/XML/whatever...shudder. I remember XDoclet for J2EE, thank god we don't need that with JEE.

 

Building Systems that can simply adapt and change over time is a challenge. The problem is not simple. But without addressing this the IT industry is heading to 1Trillion dollars of 'IT Debt' in 5 years time. If you like supporting legacy systems running on Tomcat - then that's great news.

The problems are complex. The solution - modularity - and so OSGi - is both extremely powerful and sophisticated. The tooling and the runtime must provide a simple abstraction to shield this from the normal user. 

The Paremus Service Fabric does this in the runtime. Tooling like BNDTools achieves this for the developer.

 

For the record, please note that SpringSource is continuing to work to improve the usability of the build and development tooling associated with OSGi by leading the Eclipse Virgo and Gemini Web projects and by participating in the Eclipse Libra Enterprise OSGi tooling project. These projects are now proper community efforts with committers from SpringSource, SAP, and others.

Hi Rob, I do agree that Rod is a pragmatic person. However the quote from the article is as follows: "OSGi cannot be made ... productive" (emphasis added). What he's essentially saying is that modularity is too hard. That's not pragmatism, it's defeatism.

The thing is, others are now proving that modularity -- through OSGi -- can actually be made productive and easy. In fact I firmly believe that OSGi development can be made more productive than "traditional" Java development.

So when one person says that something cannot be done, and then somebody else actually does it, what does that say about the first person?

Hi Rob, I do agree that Rod is a pragmatic person. However the quote from the article is as follows: "OSGi cannot be made ... productive" (emphasis added). What he's essentially saying is that modularity is too hard. That's not pragmatism, it's defeatism.

To be precise, he isn't saying that modularity is too hard, just that OSGi can't be made as productive as he'd like.

The thing is, others are now proving that modularity -- through OSGi -- can actually be made productive and easy. In fact I firmly believe that OSGi development can be made more productive than "traditional" Java development.

I think that productively is totally subjective. You believe it is productive, I don't. You believe that OSGi will be more productive than "traditional" Java, I don't. If I'm wrong, I'll hold my hands up and say so.

So when one person says that something cannot be done, and then somebody else actually does it, what does that say about the first person?

I'll answer that when OSGi is more productive than not OSGi :)

(Obligatory disclaimer - I work for SpringSource, and I led much of the effort on OSGi development there back in the day)

Disclaimer, I don't work for SpringSource nor draw any benefit from them. Rob's assement is spot on. I know that Rod was very hot on OSGi a few years ago and I know that he gave it an honest try. But just as Spring shed EE complexity.....

Rod, YOU are the problem, just like Peter Kriens and the other people are.

Keep focus on the idea, don't attack the people.

 

The problem with OSGi isn't OSGi, it's the tooling around it. When the OSGi supporters like Spring and Eclipse figure out how to automate manifest generation and remove "you have to understand OSGi" from the equation, OSGi will be accessible to the rank and file and all these "OSGi sucks" proclamations will go away as they should.

Rod, YOU are the problem, just like Peter Kriens and the other people are. When you guys finally realize that OSGi is still a cathedral, and you fix it, your pet technology will be a clothed empror like you want it to be.

In other words, the problem isn't that OGSi it too complex, the problem is that Rod Johnson hasn't created the tools to hide the complexity, which is apparently something that he's obliged to do for reasons left unstated.

 

Yes, but the maven bundle plugin doesn't provide all of the options necessary to generate consistent MANIFEST.MF from a pom.xml. Version ranges are sadly lacking and until they are supported people will have to maintain such information in multiple places.

Hi Andy,

Actually the maven bundle plugin does very well with versions now, based on improvements in bnd. First, you maintain the version of a package using a "packageinfo" file (or an annotated package-info.java if you prefer) that is located *in* the package... this means it is always close at hand when you make a change to a package. Then for imports, bnd generates a range based on the version of an API you have built against, using the standard Semantic Versioning rules.

Also bndtools supports analysing the bytecode of your APIs and comparing them with previously released versions to calculate whether the changes you have made are backwards compatible for clients, implementers or neither. Based on this it tells you pretty accurately how to do your versions. My ultimate goal is to make versions something that developers just don't have to worry about most of the time. (NB, full disclosure: I'm the main author of bndtools, but the version analysis functionality was written by another committer).

Neil

Hi Neil,

I raised https://issues.apache.org/jira/browse/FELIX-2972 a while ago. At that point it certainly wouldn't do what I wanted. I want to build with Maven and have MANIFEST.MF generated, replicating the version ranges in pom.xml.

I never understood the OSGi hype.  The fine-grain classloading isn't really needed in a majority of apps (majority being somewhere between 50.01% and 100%), specifically ones that don't run multiple applications within them.  Applications got by with more coarse grain classloading for years until OSGi became popular so even that feature is a nice-to-have rather than a must-have.  As for the OSGi component model, I think people are better servered writing application components in their favorite framework (like Spring) than write OSGi services.  If you think about it, all this OSGi classloading is a glorified replacement for the Linker that we had in C/C++ applications.

Furthermore, I always thought OSGi should be completely hidden from developers by a maven plugin and only framework developers should be writing to the OSGi API.  I was originally excited about OSGi as a kernel technology so that EE vendors could have an SPI to plug in their own subsystem implementations cross-app-server, but that never came to fruition.  So, OSGi == yawn in my book. 

http://williamlouth.wordpress.com/2009/07/03/jazoon-‘09-the-conference/

Lets not be fooled in to believing that to achieve modularity we just put a class chastity belt around our private parts. In the enterprise space state is just as important as behavior and it is not so easy to tackle in or outside of a container which is why most systems go occasionally offline (at least internally) to sync changes across different data processing domains (messaging, databases, …). Even if you can isolate classes from each other you cannot do the same for the external side effects (of multiple & different versions). This last item seems foreign to many OSGi disciples who apparently do not understand one of the main reasons why operations stop and restart processes in applying a managed change – to be sure the process can indeed be restarted when something goes terribly amiss in production outside of the container (hardware). It is much easier rolling back at the time the change is applied.

OSGI was just a round about solution for a problem that could have been resolved by a simple packaging standard like WAR/EAR. All you needed was a way to specify which jar a jar is refering to and have a class loader look up classes in that order. Just like ear but standardardize it.

The entire idea of trying to specify which package to refer to in a jar is a waste of time and serves no purpose.. if you donot want some one to use a class donot include it in the package.

I have invloved in a project like this for a year. And the only thing that was running through my mind was WHY WHY WHY.

I’d like to add some clarification here, and say a little more about our experience with OSGi and my conclusions from it—note the emphasis on my, as my opinions aren’t always those of SpringSource or VMware.

We made a major investment in OSGi in creating our dm Server product. We subsequently took that technology to Eclipse, where it continues to move forward, with our continued input. We have an ongoing revenue stream from our OSGi-related technologies,with customers including some household name companies. We plan to maintain our current level of commitment to OSGi.

The lessons I personally drew from the experience were that (a) OSGi is a great solution for complex applications with stringent modularity requirements; (b) typical business applications (from which we make the bulk of our revenue) don’t have such requirements; (c) our efforts to reduce the complexity of writing server-side OSGi applications were promising, but the road to simplification was longer and less certain than we’d hoped. Thus continuing down that road at the Eclipse Foundation, in partnership with other companies and individuals, was a natural move.

I don’t think OSGi is a bad technology—merely one that isn’t applicable to every application out there and one which therefore didn't belong at the core of our business. If you don’t have modularity requirements, the simplest thing that can possibly work won't have a module system. I applaud the fact that the Virgo project continues to make progress in simplifying the OSGi experience, and am proud that our engineers play a major role.

HTH,

Rod

JDeveloper? Really?

Things should be as simple as possible, but not simpler.

As true as this is, the boundary between simple and simplistic is unfortunately highly subjective. Never change things that work for you. It is as simple as that. However, many development organizations are not in that blissful state because they have complex problems they need to solved in an increasingly complex environment.

The value that OSGi brings to the table is when you have to maintain a large code base over long periods of time, you have many external dependencies, you need to provide middleware, if you work with multiple developers, or if your development is spread over multiple locations. For those situations OSGi provides a type safe module extension to Java that can significantly reduce your development complexity.

So what is the usability issue? You can build powerful applications with very little complexity with OSGi?

Unfortunately developing with OSGi can be really hard because both the modularity and type safety of OSGi are at odds with prevailing dynamic class loading patterns in Java. OSGi is often the messenger telling you that your baby is neither modular nor really type safe. For example, Dependency Injection in Spring requires access to implementation classes from the configuration module into the providing module, exactly the classes you should really keep private in a modular world. Worse, dynamic loading is only type safe if there is a single namespace. A single namespace is the cause of DLL hell aka JAR hell that OSGi addresses by managing multiple name spaces. The resulting mess is what causes the usability problem.

These are not OSGi problems, if Jigsaw one day will be modular and type safe it will have exactly the same issues!

To get the many benefits of modularity and type safety you must therefore change the way you think about Java ... If something is inside your module everything works as before but if you want to work with other modules you need to consider type safety and module boundaries. Object Oriented Design forced you to consider inheritance, encapsulation, and object identity, things that did not exist in Structured Design. Java modularity requires just such a paradigm shift. What we need is an inter module communication mechanism that truly hides implementation classes and enforces the type safety across modules. OSGi services are such a mechanism.

As Spring heavily relies on dynamic class loading it ran head on into the module boundaries and multiple name spaces. Though Spring DM included OSGi services the problem was that they were more supported than embraced, resulting in many Spring DM/Blueprint modules to use the existing dynamic class loading model. This results in the import/export of large numbers of implementation classes, increasing the public space, the opposite of modularity. The history would be very different if the Spring libraries had embraced services, setting an example for true modularity.

There are only two solutions to this problem: Change OSGi or change the world. As I do believe there is no value in a little bit of modularity I can only see false advertising value in punching convenient holes in the modularity story of OSGi.  I do believe it is worth keep strong modularity and its enforcement. Very large applications like Eclipse, Websphere, JDeveloper, and others demonstrate that at certain scales OSGi becomes a necessity. And as more and more open source projects start to embrace OSGi it will become easier and easier to build OSGi based systems.

That said, as Rod justly points out, there are lots of people that are happy with a Tomcat based solution. Moving to OSGi when have dependencies on libraries that are hard core dynamic class loaders is then often not worth it. We're therefore thinking about a version of OSGi that inherits the class loading architecture of whatever environment it runs in; a JAR on the class path is then treated as a bundle. For example, in a WAR all entries in the WEB-INF/lib directory would be available as bundles. We tested this and it is a testament to modularity that many existing bundles actually worked out of the box. Though this solution obviously does not help when you are in JAR hell, it does provide many of the benefits of OSGi in your own familiar environment. And one day, when JAR hell finally arrives for you, the OSGi Service Platform is there to serve.

  Peter Kriens

Somehow my reply never made it so I wrote it on my blog: http://www.osgi.org/blog/2011/06/ease-of-use-and-spring.html

Ha ha, to me it's really funny with the answer from Rob...

Can we use Spring for powerful embedded system? No of course! Spring is too heavy... and OSGi can be used from embedded system to middleware to... unlimitted purpose.

Why they have Spring DM? because they think one day OSGi will replace them...

Why Above has turned to use OSGi as core framework for their SW suites? JDeveloper has supported OSGi bundles? Because OSGi is going to be widely used than ever.

We have used OSGi for several years in Public Transportation domain.. with hundreds of SW modules we see it's really simple to manage and  maintain...

WE LOVE MODULARITY!

Why use OSGI when there's PaaS? Why restart modules inside of a jvm among many servers when you can launch/terminate servers with one api call? PaaS seems be the preferable unit of operational modularity these days, not osgi, and Spring/VMWare is in the PaaS game.

Ara.

I wrote a response here: http://osgithoughts.blogspot.com/2011/06/need-for-modularity.html