Discussions

General J2EE: Reuse of Components and Services - Objects Considered Harmful

  1. Looking around TheServerSide, I'm surprised that there is no forum for discussions about reuse issues: What makes a component or service reusable, how to design good components and services, methodologies that encourage reuse of components, markets for components, etc.

    At my company, we spend quite a bit of time worrying about reuse issues. We've also been reading most of the current literature out there, and find it confusing and often misleading. One significant problem we've seen is that object-oriented programming can often get in the way of building reusable components and services. I think this is one reason why some other countries are ahead of the US when it comes to software reuse.

    We've written a couple of papers about these issues, but I'd like to hear from others who have experience in this area. You can access the two papers at http://www.digislice.com/docs/guide.html and http://www.digislice.com/docs/components.pdf

    Wm Leler
    DigiSlice Corporation
    wm@digislice.com

    Threaded Messages (17)

  2. I read over both your papers, and while I agree with most of your facts, I disagree with your conclusions. I think that the component market for Enterprise components will probably never be successful, primarily due to the nature of problems Enterprise applications attempt to solve.

    First off, I agree that in general, OOP does not make it easier to write new software. The reuse benefits you get from OOP during the initial development are modest. OOP does make it much easier to maintain existing software, however. Most business software projects spend considerably more on software maintenance than the original project costs. Maintenance costs are critical for Enterprise applications because business rules change so frequently.

    Compare this to the GUI component market. If you buy a GUI widget to draw a pie chart on the screen, its logic will probably be good more or less forever (or at least until your next language upgrade). Since the logic is stable, self-contained software components are practical. Enterprise component are much more difficult to encapsulate because the logic requires too high a degree of customization.

    There does seem to be a market for business software in the arena of business systems like SAP or (on the small scale) QuickBooks. Selling software systems rather than individual components allows vendors to manage complexity by offering a self-consistent package. Furthermore, they increase their market share by targeting end users directly. There are a lot more accountants than developers who write accounting software.

    There was a myth that arose when OOP was first introduced that there would eventually be a complete object model for everything that would be the foundation for all software development. Similar myths were put forward for component systems like CORBA and more recently for Web Services. Until the basic laws of business change, I don't see any reason why anything like this will succeed.

    Having said all this, I think there is a great future for software frameworks like J2EE. These frameworks will be successful not by providing a market place for generic accounting widgets, but by making it easier for developers to create custom accounting widgets to deal with the (often bizarre) business rules for their organization.
  3. More interested in How, not If[ Go to top ]

    Paul, I appreciate your comments, but I was more interested in other people's experiences with reuse and how they make it happen, rather than a discussion about whether or not reuse is even possible. At our company (DigiSlice) we use and reuse components all the time, so we don't often stop to worry that we might be doing something that people think is impossible. I would be very saddened to think that J2EE is not amenable to reuse, since reuse is far more common on the VB/.NET side, and this would be a huge disadvantage for Java (if it were true, which we think not since we do it all the time). (Side note: we use AppComposer to assemble components, even J2EE ones -- you can get a free copy of it at appcomposer.org)

    Also, I should be clearer that I don't mean to start a debate on the virtues of OOP -- we use OOP all the time to implement components, and as you say it is extremely important for reducing maintenance (and increasing reliability) -- I say that several times in the paper on components. But object-oriented design doesn't seem appropriate for designing components (especially their interfaces) nor for building glue code.

    As you say, there are lots of reusable GUI components, but there are also lots of reusable enterprise components. What about components to process credit cards? To access databases? To generate HTML? To track contacts or contracts?

    You say that reusable components won't succeed because the needs are not stable and require customization, but that makes components more desirable, not less. In any application area (whether it be accounting, e-commerce, sales automation, ERP, SCM, etc.) where each company needs slightly different (customized) software, then reuse is especially important. Even if you assume that 10% of an application needs to be customized, then that means that as much as 90% of it can be reused. This is especially important now, since typical enterprise applications cost over $1million to customize and over half of them fail.

    What is needed is to find those things that can be reused, and figure out the best way to design and implement them so they can be reused. Software frameworks (like J2EE) are one (small) way to encourage reuse, since the developer doesn't have to implement transactions and stuff like that, but we need way more than that. But I for one don't want to limit my software reuse to what Sun puts into J2EE.

    So again, if people want to talk about ways to increase reuse, please share your thoughts and experiences.
  4. More interested in How, not If[ Go to top ]

    \Wm Leler\
     In any application area (whether it be accounting, e-commerce, sales automation, ERP, SCM, etc.) where each company needs slightly different (customized) software, then reuse is especially important
    \Wm Leler\

    What's become painfully apparent is that, for many applications, each company doesn't need slightly different software. They need highly customized software - customized enough so that you might not recognize your original once it's completely customized :-)

    For some things which are highly regulated, like payroll, a component or a service makes sense, because everyone has to do things the same way. But once you step out of that model, businesses vary widely in how they do things (even in the same industry), and they want their software to match their style, not vice-versa. They also want to differentiate themselves from the competition, and again that's not an environment where everyone can use the same components.

    There's a grain of truth in what you say, but I think it's a small one. For highly regular, well-known areas, you can get re-use. For true business areas, you'll find that the size of the customization code starts to outgrow the size of the "common" code. This has been the case for me on many projects - using some framework/product/system was chosen by a team, and when all was said and done only a small part of it was applicable. The rest of the work was coding around the so-called universal framework/product/system.

    Where I think your assumptions fall apart is the "10%" customization number. It's typically much, much higher. Companies just aren't as regular as software developers would like :-)

        -Mike
  5. How, not If?[ Go to top ]

    Oy! It looks like I'm getting a discussion about if reuse is possible, whether I like it or not!

    \Mike Spille\
    What's become painfully apparent is that, for many applications, each company doesn't need slightly different software. They need highly customized software - customized enough so that you might not recognize your original once it's completely customized :-)
    \Mike Spille\

    The number of companies that can afford to spend $5million on a customized enterprise application is getting smaller and smaller. And, they are already saturated. A company I once worked for spent that much on a well-known CRM system, and it never gave ROI. It didn't even meet customer needs (style) -- in particular the customer support people hated it. So, one of the customer support people threw together a web app in two weeks (shameless plug -- from components using AppComposer) to meet his needs. Other support people saw it and wanted a copy, so he spent another two weeks customizing and generalizing it, and the company standardized on it, replacing one of the modules of the application they had spent so much money on. The resulting app not only cost far less, it did exactly what they needed (as you say, it matched their style, not vice-versa).

    The point is that I'm not arguing against customization, I'm promoting it! We need to make software much easier to customize (customization is just one form of reuse).

    \Mike Spille\
    using some framework/product/system was chosen by a team, and when all was said and done only a small part of it was applicable. The rest of the work was coding around the so-called universal framework/product/system.
    \Mike Spille\
    Aha! Please share your experiences. What made this framework difficult to reuse? How could it have been done better? This is the conversation I want to have!

    You are not alone -- we run into companies all the time that say they tried reuse and it didn't work. And yet other companies (including us, and many others in Korea and Europe) are doing reuse and love it. We need more information on how to make this happen.

    The Open Source Conference is this week in Portland (where I live) and last night I had an interesting conversation with a major figure in software engineering (I want to ask his permission before I attribute this quote to him in public, but you would recognize his name). He does quite a but of consulting, and he claimed that in his experience most software engineers are not at all interested in improving productivity, they are interested in keeping/looking busy (so they can keep their jobs). I'm not sure I agree, but it is an interesting perspective. It may explain the current fascination with process, process, process.

    In case his observation is true, then I'll rephrase my request -- I'm looking for software engineers who believe it is possible to (dramatically) increase productivity to share their ideas and experiences on how to do that. Most software engineers I know don't believe that current practice is as good as it is going to get, and we just need more of the same. So let's have a positive discussion, not just reasons why software is difficult and the situation isn't going to get much better.
  6. How, not If?[ Go to top ]

    \Wb Leler\
    The number of companies that can afford to spend $5million on a customized enterprise application is getting smaller and smaller. And, they are already saturated. A company I once worked for spent that much on a well-known CRM system, and it never gave ROI. It didn't even meet customer needs (style) -- in particular the customer support people hated it. So, one of the customer support people threw together a web app in two weeks (shameless plug -- from components using AppComposer) to meet his needs.
    \Wm Leler\

    Something like a CRM system is re-use writ large. In the case you cited, truly general reuse wasn't possible, and a small custom application did the trick far more cheaply and better to boot. If this is supposed to be about re-use, I think you just argued against your own point :-)

    \Wm Leler\
    The point is that I'm not arguing against customization, I'm promoting it! We need to make software much easier to customize (customization is just one form of reuse).
    \Wm Leler\

    See my comments on how much re-use vs. customization. Customization is good, but at some point you may have more customization than the size of your original framework.

     \Wm Leler\
    Aha! Please share your experiences. What made this framework difficult to reuse? How could it have been done better? This is the conversation I want to have!
    \Wm Leler\

    Ah, this is simple (and I'm sure many people will recognize it). For just one example, <mumble> product was billed as a auction/trading engine. You should be able to just plug in your rules and look & feel, and away you go. Unfortunately, it had many bugs, supported only a very small subset of auction types, and only allowed customization in limited areas.

    You could argue that the answer would be to make it more general. However - the ultimate auction/trading engine which covered everything we needed, and everything every other customer wanted, would be so complex that it would be very difficult to understand and use.

    Put another way - what I had to put up with made the easy things easy, and the difficult impossible. A "perfect" engine would, most likely, make the easy things difficult and the difficult possible.

    We effectively did our own engine, tuned to our own needs, and used the commercial product for a teensy bit of functionality. It worked for us because we limited it only to the universe of our own needs. I'm sure another group trying to do their own auction/trading thing would've thought it sucked for their needs, and built their own :-)

    \Wm Leler\
    You are not alone -- we run into companies all the time that say they tried reuse and it didn't work. And yet other companies (including us, and many others in Korea and Europe) are doing reuse and love it. We need more information on how to make this happen.
    \Wm Leler\

    You can get some re-use, particularly in "regular" and consistent application areas, and certain technical areas. It's much harder in your standard business environment.

    As for those many others elsewhere - um, sure. I'm sure the non-Americans in the world have a magic technique or product which enables their re-use and which us Americans can't achieve. Or perhaps those companies don't, um, report their reuse statistics quite the same way we do.

    \Wm Leler\
    [...]
    So let's have a positive discussion, not just reasons why software is difficult and the situation isn't going to get much better.
    \Wm Leler\

    Positive discussions do need to keep reality firmly in mind, even when they're not pleasant or what you originally had in mind. It may not be what you want to hear, but it just may be that software is, indeed, difficult.

        -Mike
  7. reuse vs customization[ Go to top ]

    /Mike Spille/
    Something like a CRM system is re-use writ large. In the case you cited, truly general reuse wasn't possible, and a small custom application did the trick far more cheaply and better to boot.
    /Mike Spille/

    What you call a "small custom application" was assembled entirely out of reusable components.

    Customization is just another form of reuse. If you make something easier to customize, you make it easier to reuse it. And I'm not talking about adding customization by throwing every bell and whistle into it. We do customization by replacing components and modifying glue code.

    /Mike Spille/
    You could argue that the answer would be to make it more general.
    /Mike Spille/

    In the two papers, I specifically argue against over-generality. Over-general components are typically unusable. It is better to have several different components with different features, and make it easy to replace one component with another.

    /Mike Spille/
    ... but it just may be that software is, indeed, difficult.
    /Mike Spille/

    I'm sorry you think so. We have already (informally) seen 3X to 10X speedups compared to similar projects done without components, and the numbers get better as we amass more components.

    And I'm not sure what you meant by those comments about "non-Americans" but I'm not talking about reuse statistics. I'm talking about what I've seen myself, especially in Korea. Korea has only started going after software recently, but it looks like they have every intention (and the capability) of doing to software what the Japanese did to automobiles and consumer electronics. Casting aspersions on Japanese (or European) automakers didn't save American automakers. The fact that the Japanese had trouble with software has lulled American software developers into a false sense of security. The Koreans have a language based on an alphabet, like us, and they are already selling software in the US.
  8. How, not If?[ Go to top ]

    \Wm Leler\
    Oy! It looks like I'm getting a discussion about if reuse is possible, whether I like it or not!
    \Wm Leler\

    My apologies, I didn't mean to push your thread in a direction other than the one you wanted.

    I do think the question of "if" is very important with regards to reuse, though. Reusable components are all about economies of scale. You need to make sure that enough people will use your component to make the effort of creating the component worthwhile. That's why you see lots of generic software components that address encryption and hardly any generic components that represent a Purchase Order.

    One of the basic tensions in software development is maintaining existing functions vs. adding new ones. Basically, you need to separate the stable and unstable parts of your system and turn the stable ones into components if you can. Unfortunately, even Nostradomus was not 100% accurate in his predictions, so you are going to be at least partly wrong (at least I am).

    I have seen organizations that have both succeeded and failed going both ways. I have seen companies put together elaborate component frameworks that never got used because they delivered functionality too slowly or not at all. I have also seen companies take the stove-pipe approach and had their systems devolve into an unmaintainable mess.

    The art of software engineering is all about finding the middle ground between the two extremes, providing a stable enough foundation to keep your application architecture coherent while being flexible enough to meet the needs of changing business requirements. The reality is that this is very hard to get right.

    With regards to the "How", I think the papers on your site hit the major points:

    1. Seperate interface from implementation

    2. Provide some sort of Service Broker or Factory to access components

    3. Build a coherent Service Framework to address common component needs (security, persistence, transactions, whatever).

    To these I would add:

    1. Build your foundations on software standards whenever you can.

    2. Use proprietary solutions when you can't afford to do the development and when the solution requires little or no customization. As other people have written, that extra "10%" of functionality you need to add may be so hard to implement that it is easier to produce everything from scratch.

    3. Open Source is good, because unlike proprietary software, you always have option of hacking the software to make it do whatever you need. In practice, this option is more theoretically than real, but it is nice to have.
  9. How, not If?[ Go to top ]

    Thank you, this is helpful. Do you mind if I use some of your ideas in the guide.html document? That document is meant to be a compilation of ideas from many different people -- a collection of guidelines on building component-based software.

    --wm
  10. How, not If?[ Go to top ]

    Feel free. :)
  11. Single Reuse is Still Good[ Go to top ]

    \Paul Strack\
    I do think the question of "if" is very important with regards to reuse, though. Reusable components are all about economies of scale. You need to make sure that enough people will use your component to make the effort of creating the component worthwhile. That's why you see lots of generic software components that address encryption and hardly any generic components that represent a Purchase Order.
    \Paul Strack\

    I was just perusing through an old book on JavaBeans (The JavaBeans Developer's Reference by Dan Brookshier) and came across this quote that can be applied to any sort of component model:

    "Even though a component might only be used once, it is still better to code it as a Bean ... [One] reason for creating JavaBeans with only one apparent use is that they can still be used in a rewrite of the same application ... the probability that the Bean is reusable depends on how well it was initially written."

    On that same page he lists another valid reason for creating a "single use" component, and that is for future re-customization: "The flexibility of being a Bean is that they can be modified without recoding and recompliation. This could greatly reduce the amount of work that would normally be required over the life of a project."

    One third reason why one would want to make your program a collection of reusable components (even with no thought for reuse) is ... and this is just my own opinion ... the act of creating well-understood and thoughout component interfaces (like adhering to the JavaBeans spec) tends to make code more robust. Heh, maybe it is just my code that turns out that way ...
  12. most excellent code[ Go to top ]

    I agree -- creating good components is worthwhile even if you have no idea if anyone will ever reuse them. After all, how many of us spend time polishing code that nobody else will ever see? Good code is its own reward.

    And besides, if anyone ever has to maintain an application, and they modify any part of the application, doesn't that count as reusing the rest of the application? How easy it is to reuse the rest of the application (i.e., how easy it is to maintain a program) is directly influenced by following good reuse guidelines.
  13. Levels of Reuse[ Go to top ]

    Wm,

    I would like to add a thought on reuse.

    I think it would help classifying component reuse into the following three levels:

    1. reusable across subsystem of a given project
    2. reusable across projects within a given enterprise
    3. reusable across enterprises

    I see that the proposed guidelines assume Level-3 reuse.

    Nalla
  14. Levels of Reuse[ Go to top ]

    Yes, there are definitely levels of reuse, and I'm somewhat assuming level three (reuse across enterprises). However, we are finding that if you assume #3, you provide benefits to #1 and #2 as well.

    --wm
  15. reusability Example[ Go to top ]

    i want reusability example and hirarchi of project
  16. How, not If?[ Go to top ]

    Glad that somebody wants to discuss how to improve productivity and achieve reuse.
    The reuse problem have three aspects.

    1. How to represent reusable assets that we have
    2. How to match current requirements against the reusable asset that we have, and identify the reuse candidates and gaps between requirement and what is supported.
    3. Produce the glue that is required to bridge the gaps.

    The solution to item 1 is key to the problem. We cannot reuse because we cannot represent accurately what we have. Only when you know accurately what you have got, you can think of reuse.
    Modeling (UML or otherwise) will provide a part of solution . It will at least help us define the interfaces accurately. But the side effects of invocation of methods (of either components or objects) cannot be described accurately as of now. When we have a universal description language (requirement language) we may be able to take a shot at reuse.

    Till then , solutions packaged by an organization for a particular problem is closest we will come to reuse. This approach says you change your problem to fit the available solution, in order to reuse the solution.

    Another pragmatic approach, within constraints that we have is to use modeling heavily. Use natural language descriptions to describe the side effects, and use natural language search / human expertise to make a guess at reusability of an interface. If you have interfaces available in model, you can very well generate the glue necessary. The human intervention is not, however completely removed. And one also has to depend on verification techniques (testing) to make sure the whole excercise was succesful. All this makes sense when dealing with very large systems, for smaller apps the initial efforts to required will be more than fresh development. [However you would make the saving on maintenance].

    -Vilas.
  17. I think that something that is being overlooked, yet underlying in the discussion, is that component based programming is a different way of thinking than oop, but similar in the profound effects that it can have on the way we develop software.

    I started researching components in graduate school in 1996 and really loved the idea, but found it to be wishful thinking for the most part, especially after trying out the component models of the day, like CORBA. In 1998, I began developing server side Java software and have gone through the ups and downs of the specs and app server changes. I have worked as a consultant for an application server company on several different projects, all of which had similar, yet not so similar requirements. I ended up as an engineer on AppComposer in 2000 after seeing a very compelling presentation by Wm and another of the founders at the application server company. The product wasn't where it is today - they were still Zat at the time and were in startup mode, but it was still a great product in my mind that had boundless applications in my mind, based on my experience with customers.

    As an engineer on the product, I am forced to think about ways to make programming easier on programmers and non-programmers alike. My first week working on the product was a little frustrating because I had to break down some paradigms I had developed in my mind in order to get my head around developing for developers trying to think of the types of things I wish I had working those long hours on integration and customization on previous projects.

    The end result does not look like OOP, though the internals of the component may look be OOP. Actually, the interface to a component generally looks really nasty to someone firmly entrenched in OOP. You don't think about components like "customer", what you think of are tasks that you have repeated over and over and how they could be applicable to other programmers - these tend to be algorithmic components rather than domain components.

    Think for a moment of the Java programming language. While not components, there are many constructs and utilities that all programmers find useful. These tend to be simpler constructs. The idea here is to step forward from this, where you develop components that are useful to *most* programmers (and they may all be in the same vertical), but do not go so far as to make any assumptions about how their business works or how their software environment is configured.

    Consulting gets you into this mindset better than anything else. Going from client to client and solving the same problems over and over got me to the point where I had a "utility belt" of Java classes that I used to solve the same problems over and over. While I had not developed these into components, they were reuseable.

    Component based development is a natural follow on for programmers/architects/whatever that are looking to make their own lives easier and want to finish projects more quickly. They may not call it "component based development" or even build components, but they tend to end up with a set of reuseable utilities that they use over and over.
  18. Here are my thought's on reuse. I consider a component to be managed by a
    container. Gui components have had good re-use a la beanbox type palletes.

    For business components the re-use is somewhat newer. Before components,
    in OO we exclusively used inheritance which provided some in-application
    reuse. For components in a J2EE business component container I think the largest
    measured re-use is at the entity bean level. Different Data transfer objects and
    session front ends can re-use the same domain model, across applications, if
    you are careful to keep application/workflow logic out of the entity beans and
    provide only domain logic for business logic in the entity beans.

    For the services (session) layer, I think web service endpoints will allow the session
    beans to be re-used, provided that the original use case of that particular service
    is needed by the outside application. Since this layer is use case specific, this
    can be more difficult to find a match.