Discussions

News: Should SOAs be developed top down or bottom up?

  1. Should SOAs be developed top down or bottom up? (23 messages)

    There has been much discussion in the blogsphere on whether an SOA should be driven by business requirements or be more adaptive the realities of IT. While John Crupi suggests that a "top-down" approach is the one that works best, Stefan Tilkov and Bill de hora discuss the dangers of centralized SOA governance and the need to take a more agile approach to SOA development.

    The BU should own the business drivers, use-cases and processes, according to Crupi. It is then IT's job to implement the BU requirements and own the service definitions.

    Crupi advised against using a "bottom-up" approach to SOA development, in which existing systems are simply wrapped using Web services to create a service layer.

    ***

    "When the business people and the architects are thrashing out the documents and processes, you cannot have the programmers sitting on their hands," de hóra said. "They need to build something and show it to the business, so the business can ratify and refine what it's asking for."

    "A culture of continuous deployment and tending to front line services will help the organization infrastructure become robust to continuous requirements changes," de hóra said.

    Read SOA - Top down or bottom up approach?

    Threaded Messages (23)

  2. Meet in the middle[ Go to top ]

    My experience is that top down, and bottom-up are both short sided. Meet in the middle balances both.
  3. Depends ...[ Go to top ]

    Top Down Planning & Bottom up execution. Most architects should have an intution of the first set of services that is needed by a business.As IT and Business managers figure out the services stack the development team should develop the first set of services and periodically present to the leadership.
  4. Business driven[ Go to top ]

    Talking about Top Downdown or Bottom Up misses a key point.

    SOAs are about business services - and the granularity of the business services is at the level that is understood by customers, partners, and suppliers.




    PJ Murray
    CodeFutures Software
    Java Code Generation for Data Persistence
    http://www.codefutures.com
  5. If it doesn't come from the CIO and work its way down, the type of organizational changes and governance requried to do SOA does not happen. When it goes the other way it tends to be a bunch of siloed apps wired together with web services with an XML API at the wrong granularity.
  6. Bottoms up?
  7. Middle Out -[ Go to top ]

    Hi All,

    I read this article with great interest as I am currently working two jobs, one in which there is a top-down driven integration of Web Services (WS), and another where there is a bottom-up push for WS. The top-down authority has a 2 year investment in a BPEL solution, while the bottom up is more of an in house development effort to supply WS integration to other enterprises altogether. Actually, they are both going extremely well, but I recently read a book about performance management which noted that the most effectivee budget planning is usually a function of a "Middle-Out" planning strategy.

    The idea here being the it does not matter what kind of push you have, if the two sides are disconnected - you are going to inevitably run into trouble. For budgeting the middle layer has the best knowledge of both ends. Sometimes in a smaller organization (my experience) this is not such an issue, (no middle layer?) but in a larger organization where web service architecture is most needed, there can be QUITE a bit of disconnectivity between the management authority and the detail-level implementers.

    For this case, I think the article kept mentioning things such as "culture" and other behavioral patterns (fear of losing jobs) that affect the project.

    In combination with this Middle-Out strategy - I put up to flame that the ultimate solution is not one of a top-down or bottom-up strategy - rather - ANY means of moderating the two groups to be as cohesive as possible. The middle-out strategy effectively demanding an intermediary (moderator) between the two groups, and eliminating the social problems that arise by keeping the interests of both groups in mind.

    In addition to the idea of having a moderator to connect the business case as well as the need to be aware of IT realities, I think the idea of a middle-out architectural view is very appropriate for WS architecture, due to the fact that I find for many enterprise level organizations - many of use find ourselves in a more project management oriented role. This is due to things such as outsourcing and how IT is changing, as well as for the current context: If you are charged with planning a WS architecture, then you need to have enterprise breadth knowledge, and therefore some amount of responsibility and technical authority to feasibly plan accordingly, just to begin with! This makes the idea of a middle-out architecture appropriate for many of us. I guess the comprimise of the top-down and bottom-up ideas would ideally be an individual that is business user that is very technically savvy, or a IT person that can converse with business users in each area of the business.

    In any case, I hope to generate a lot of feedback from this, because with the face of IT changing, I see many of us as having to have to engender ourselves into these roles - to be most successful and still stay in the technical field.

    Thanks!
    ~tim
  8. In your bottom-up experience..[ Go to top ]

    how did the IT group get the business to pay for their SOA desires? In my experience, BU's are extremely averse to letting developers do anything other than work specific projects with specific timelines. Unless IT management makes push for it, SOA has no chance at the companies I help...and sometimes the idea is still rejected.
  9. I certianly agree with your assertion that BU's are extremely adverse to writing a check for anything other than immediate, and tangible results.

    I have observed that in both situations, (enterprise company with silos of functionality to sew together, and small company with enterprise software to integrate with other companies) that it was an amazing leap of faith for both sides to help the other - ESPECIALLY with the check writing.

    Two years ago - I was far less aware of how badly enterprise level architectures could fail ( was not at that budgeting level yet ;) ), but now I see that it was the binding of the two sides by an extremely effective middle out project management individual. The culture is extremely good to begin with, and the enterprise level company is in the business of technology (manufacturing), and sometimes even the BU that are furthest from the technology will put out their neck to take a chance.

    So I guess (to get to the question) the HOW of the success was initially signing our own bill - to devote a technical person to be a BU partner. As this person developed a repore with the business, they were able to have sway and constuctive criticism that the BU considered to be more than the techies trying to get more money out of them. This person is the intermediary I was picturing as a key component of the middle-out strategy.

    ~tim
  10. I'm new to SOA and our company is looking at adding some services to integrate our enterprise app with other enterprise systems. We have not determined yet if that would include non J2EE systems. In any event, our architecture is already set in stone so wouldn't we by default go with the bottom up approach - wrapping WS interfaces on top of existing business logic or implementing a JCA interface?? What other approach would you take instead of re-architecting the whole system?
  11. - wrapping WS interfaces on top of existing business logic or implementing a JCA interface?? What other approach would you take instead of re-architecting the whole system?

    What about remote session EJB? It uses the RMI/IIOP so you can use any CORBA clients also ...

    DF
  12. Stone eh?[ Go to top ]

    Hi Jeff,

    Based on the context of the thread, I would say that if the development effort is based on the support of the technical community, (which of course is a single, unified cohesive unit of intelligence, trust and dedicated hardwork), than bottom-up is definitely most appropriate.

    The idea here is that the business requirement for this integration is actually within the technical community already! The middle-out person I would look for will probably be the technical project leader who has enterprise wide understanding of the architecture, and would therefore know where high-level integration would be most helpful to reduce redundancy, and still maintain low-coupling within the architecture.

    One thing from your post to note is that (in my experience) I do not know why you would NOT include non J2EE systems for WS wrapping, b/c if you are not going to include them, then you should (probably) really consider whether the added performance overhead of WS integration is necessary for systems that could natively speak to one another in the first place? I understand that it is a VERY good policy to make clear distinctions of application boundaries, especially for cross-network calls, and if that is indeed the push, than that makes sense to me as a scalable architecture.

    Some things I have run into include: making extra provisions for security and authentication for WS API calls, as well as SERIOUS considerations for the possibility of paging on any large result-set methods.
    ~tim
  13. I hope just BU2BU and B2B[ Go to top ]

    I hope the SOA advocates are just advocating using SOA for BU2BU and B2B. What I'm worried about is that the marchitects will push this architecture down into individual projects and will end up with the same kind of mess we had in the mid-90s with CORBA implemented applications. Let's hope that the proponents of SOA educate their users in this regard.

    Bill
  14. I hope just BU2BU and B2B[ Go to top ]

    I hope that if people just want to connect siloed apps or BU's that they don't do it with SOA...that's a recipe for disaster. Either do it right or don't do it.

    The only reason to move to SOA is because you have complexity that is best managed by it, like partner federation (not just businesses, BTW), or an overarching need for app security which is much better addressed via WS security policy enforcement as opposed to the current JAAS/I&AM debacle we now have.
  15. I hope just BU2BU and B2B[ Go to top ]

    The only reason to move to SOA is because you have complexity that is best managed by it, like partner federation (not just businesses, BTW), or an overarching need for app security which is much better addressed via WS security policy enforcement as opposed to the current JAAS/I&AM debacle we now have.

    You seem to be saying that extranet tunneling and/or J2EE-.NET integration are insufficient reasons to embrace WS. But my experience has found WS unbeatable for these.
  16. Nope...[ Go to top ]

    not limiting the reasons...just listing the most compelling ones that I have been involved with...
  17. I hope just BU2BU and B2B[ Go to top ]

    extranet tunneling

    I will never understand why stupid firewall administrators and byrocratic corporate policys are forcing us to tunnel all protocols through HTTP

    After 10 eyers we can throw away all firewalls, because all protocols will be tunneled through HTTP anyway ...
  18. I hope just BU2BU and B2B[ Go to top ]

    What I'm worried about is that the marchitects will push this architecture down into individual projects and will end up with the same kind of mess we had in the mid-90s with CORBA implemented applications. Let's hope that the proponents of SOA educate their users in this regard.Bill

    Well, it's the biggest demon and threat ot this industry isn't it? People who try to sound so very smart by talking complete and confusing bollocks, while most others are to afraid to look stupid so they dare not say "the emperor has no clothes".

    Oh, and by the way, I do think SOA can be very useful, I am a big believer in it. But I am just getting tired of marketing people blowing out of proportion and "thought leaders" going on about "....and this is the Oompa-loompa Schmervice-finder-receiver pattern invoking the [insert buzzword of the week here]-component".
  19. Evolutionary approach[ Go to top ]

    I would have to agree that simply "slapping on a service interface on existing systems" is not a good way to build an SOA.
    However I do feel that the discussion of bottom-up or top-down is slightly misplaced, when "evolutionary approach" is probably a better way to describe what would be best practices.
    For instance Krafzig, Banke and Slama describe the steps from "Fundamental", "Networked" through to "process-enabled" SOA which I feel is quite a logical path: evolve both the SOA infrastructure AND the services contained in it as you go, rather than building either the infrastructure or the services first. I guess this could probably be construed "meet-in-the-middle" as someone mentioned, but I do feel that these labels can be somewhat confusing from time to time and draw attention from what really needs to be done.

    David Chappell in his book "Enteprise Service Bus" gives a couple of very good examples of how to create a Service architecture in a very evolutionary fashion. And in my opinion the ESB pattern/architecture style is an integral part in any SOA effort except for the very simplest ones (which would beg the question, what do you need an SOA for if the environment is very simple?).
  20. Agile top down ?[ Go to top ]

    http://www.onjava.com/lpt/a/5742
  21. not just for business clients[ Go to top ]

    Business clients are not the only consumers of services in the sense of SOA. Consumption can also be internal to an application, e.g., a connection to a resource, or application-to-application, e.g., low-level inquiry or update on another physical system within the same organization.

    Services can be at the interface between packages, application components, applications, systems, and external business clients. They can be initiated by the party on either side of the (application) interface, but their realization, I feel, needs balanced negotiation between both parties for ultimate success.
  22. The biggest mistake is to think that any specific technology or implementation technique can solve all the world's problems. SOA assists in the construction of decentralized systems, however this should not be seen as a justification to abandon top down design, and to simply concentrate on wrapping a service layer around legacy systems.

    Firstly, good decentralized architecture still exhibits significant structure. Compare it to the web. From a distance it looks chaotic, however as you get closer, you expect a meaningful structure at some point, i.e. when you have hit a home page of an organization. Web sites that lack structure are not very effective. The lesson for implementing SOA: think of your enterprise architecture as a web site, and not as a chaotic, unstructured web. Use a top down approach to define how you interact with external parties. This leads to a design where services are explicitly traceable to core business processes. The service implementations can then pragmatically consist of a combination of new components and wrapped up legacy systems - the important point is not to compromise the external design of the services. It requires a good understanding of the business domain, and someone experienced in the skill of API lifting.

    Secondly, only invest in wrapping a legacy system if it is clear that the system is still economically maintainable. If the system has already become a liability, then building on top of it only compounds the maintainability issue, and a better approach is to use an SOA initiative to re-implement the functionality that is still required going forward in a componentized approach. The idea is not to reinvent the wheel, but to have the courage to incrementally refactor parts of the code base that lack appropriate separation of concerns, and to get rid of a lot of dead wood in the process.

    One of the biggest problems that I come across again and again in enterprise systems is lack of dependency management. See the SoftMetaWare white paper Complexity and Dependency Managementfor further background on this topic. SOA can be an opportunity to reduce spurious complexity, but if implemented using a naive approach, it can have the opposite effect.
  23. This maybe a bit off the main topic but I’d like to make 2 points:

    - SOA is NOT Webservices. SOA is just a paradigm of thought around how we build systems.
    - WS* standards are a convoluted bunch of standards that will burn in flames because they fail to manifest what SOA proposes (Simplicity).

    I’m not totally against WS* but those people that sit on these standards boards have to realize that flooding the world with stacks of standards is not the way to go. People want simple ways to build systems that can communicate efficiently and securely.

    WS Reliable Messaging, WS Security, etc are just news ways of solving the same old problems in a new and “Sophisticated” way that sounds good but falls short in presenting adaptable solutions. I understand that people are already building systems with these “work in progress” standards and probably getting some results, but nothing beats simplicity when it comes to community endorsement, and WS* is NOT simple, neither coherent.

    To read more about these points you can refer to Adam Bosworth’s blog where he discusses these points in detail: http://www.adambosworth.net/archives/000031.html

    Flame me J,
    AT
  24. Both top-down and bottom-up can succeed, but the cultural fit is a critical success factor. Big (top-down) SOA initiatives focus on enterprise governance processes, enterprise-wide re-use, and portfolio consolidation. Achieving these big goals requires structurally changing design-time processes and overcoming significant cultural bias that lead to inhibiting design, accounting, control, and operational ramifications. Traditional Big SOA initiatives can succeed, but only if top-level organizational support (e.g. C-level) overcomes organizational inertia and project teams bridge silos and operate as one team. A Small (bottom-up) SOA approach implements SOA principles on a project-by-project basis. This approach incurs less risk, but produces a smaller return. Typically there’s limited coordination across projects, and Small SOA doesn’t require as much cultural and political unrest. Using this process, the organization can slowly build a portfolio of services, but with limited coordination, the services are less likely to be reusable. More insight on the top-down and bottoms-up dynamics can be viewed at: http://blog.cobia.net/cobiacomm/2014/01/27/defining-a-service-oriented-architecture-soa-mindset-big-soa-or-small-soa/