I believe that code sharing is inevitable and this belief has driven my work since the eighties. Software should be built from components that collaborate on a peer-to-peer basis through explicit interfaces. This is the only way to solve the big ball of mud problem where the maintenance and modification of a code base becomes harder and harder due to its entanglement. If your software dependency diagrams look like a painting by Jackson Pollock you have what they call a big ball of mud.
One way to shape up a muddy infrastructure is to move toward modularity and a more component-based model. In a component model, we separate the roles of the developer (who makes components) and the assembler (who groups components into an application). Component developers should preferably be blind to other components (even collaborating components). This blindness can prevent unwanted dependencies within your implementation. The less a component can assume, the fewer bugs it will have. Only the contract should be visible to a component during development.
Much work in our industry is pointing in the direction of greater autonomy of different software subsystems. SOA is clearly based on a peer-to-peer model, but large software libraries will also have their own threads and models of control that are independent of the main application. Open source projects are really close to components, but unfortunately we have not settled on a module system that would make these components so much easier to use.
The logical consequence of this higher autonomy is the need for shared interfaces and allowing components to be linked together as late in the cycle as possible in order to maximize flexibility. A set of components can then be assembled together to form a unified application.
In 1998, with this collaborative model in mind, we started to develop OSGi. We wanted a model where components from different vendors would collaborate in a peer-to-peer fashion without prior awareness of other components. We wanted components that could be brought together with other components and work together instead of living in a constrained silo. To achieve this goal we knew we had to address the sharing problem, a.k.a. DLL Hell.
So what did we learn about modularity in those 13 years?
If modularity is not enforced, it does not exist. Unless someone slaps you on the wrist when you violate a module boundary you will not be working modularly. This is the greatest disappointment of people moving to OSGi: finding out that their code base is not nearly as modular as the architecture and dependency diagrams had made people believe. Unfortunately, many people tend to get upset and blame the messenger.
Sometimes, you have to keep secrets. The best solution to the problem of sharing is sometimes simply not sharing everything. Any shared thing has a hidden cost in the later stages of maintaining a component. Any shared aspect must be carefully versioned over time and is restricted in its evolution. Sharing can be expensive!
For this reason, OSGi gives you a private Java name space in your bundle to hide as much code as possible from the rest of the world. Whatever you do there is guaranteed not to affect anybody outside your bundle. Different bundles can actually contain the same classes without causing naming problems.
Cost of dependencies
Every dependency has a cost but you might be surprised by how many people do not know exactly where their dependencies come from. Once I found more than 30 MB of dependencies were dragged in by a single (unnecessary) reference to Apache commons collections. Many WARs and Maven projects have a very fuzzy dependency chain because it is just too hard to figure out what the real dependencies are in today's Java.
We found that for Java, the package is the right granularity to share. Classes are too fine grained to share. They are too connected to their neighboring classes in the same package. Exporting classes also overlaps awkwardly with packages in Java. JARs are cumbersome to share because they are an aggregation of the packages that are actually shared in your code. This aggregation has nasty side effects on versioning and dependency management.
Unexpected consequences of API-based programming
API-based programming, that is, an API that can be implemented by different parties, requires one to revisit versioning as we know it. In Unix package management systems - the grandfathers of software dependency models - versions are based on a consumer that requires a provider. There is a tendency in Java to blindly follow this model, but this is wrong.
It is usually a good strategy to not reinvent the wheel but with an API-based programming model we actually get a third party: the API/interface/contract. It turns out that this triad has profound implications for the compatibility rules and thus the dependency model.
To allow the components to be properly connected requires resolving their requirements against the capabilities of other components. In such a wiring model it is paramount that packages are versioned correctly.
In OSGi, an exporter declares its version and an importer can declare a range of versions with which it is compatible. The OSGi specification outlines the rules for semantic versioning[pdf], a model where the parts of a version have a well defined compatibility meaning. Many other (fuzzy) schemes only enforce always backward compatibility and do not recognize the consumer/interface/provider triad. The OSGi version scheme is strong enough to be calculated by automated tools, a necessity as human are awfully bad with versions.
In OSGi, you can clearly and unequivocally declare that you're not compatible with a prior version. Being able to express that you don't support backward compatibility with older versions is essential for a component system.
Dependency on Multiple Versions
Relying on multiple versions of the same package is not something to which one should aspire, but when enough external dependencies are used in an application it becomes inevitable. In the Java community there is a tendency to ignore those pesky multiple versions as if they cause no problems.
For example, in Maven the first version found in the dependencies is used even if a later dependency has a higher version. It is not uncommon to find multiple versions of the same library on a large class path, which is odd because we are willing to expend quite a lot of time on type safety in the language but then throw much of the advantage away at runtime by being sloppy with our versions.
In the OSGi, dependencies are carefully managed and it is an exact science, no fuzzyness or heuristics. OSGi frameworks handle the diamond problem by linking bundles to their correct version but only allowing collaboration between bundles that have no conflicts.
For example, there is a general LOG DLL that is quite popular. A WIDGET DLL uses this library as well as a COMM DLL, see the following figure.
In this DLL example, OSGi could easily handle the collaboration between App and COMM and App and WIDGET as long as they did not exchange LOG objects. The fact that OSGi supports collaboration and sharing but also isolation when needed is one of the key reasons Application Server vendors heavily use OSGi.
The majority of the OSGi Core specification is in the module layer where we define the rules for sharing. These rules are often complex, hard to understand, and require a lot of knowledge regarding name spaces, type systems, and Java’s inner bowels. However, these rules only have to be understood by a handful of OSGi framework developers. In return, OSGi gives modular developers a solid sharing model that is powerful and surprisingly easy to use.
OSGi in Action By Richard Hall
OSGi in Depth By Alexandre de Castro Alves
Enterprise OSGi in Action By Holly Cummins
Liferay Portal Systems Development By Jonas X. Yua
Liferay in Action By Richard Sezov
The Well-Grounded Java Developer By Martijn Verburg