Tech Talk: Ted Neward on Web Services and Security

Discussions

News: Tech Talk: Ted Neward on Web Services and Security

  1. The term "Web services" has been in place for seven years now, but global interoperability still hasn't been achieved. According to Ted Neward, an independent consultant specializing in high-scale enterprise systems, this is due in part to two perils: incompatible data type platforms and the object XML mismatch. In this tech talk, Ted talks about Web Services in the industry, focusing on security and choreography issues.

    Among topics he discusses are:

    - How we should think about Web services
    - Intermediaries
    - Messaging approach vs. RPC approach
    - Web services five years from now
    - SOAP vs. REST
    - How Ted would fix Web services

    Watch Ted Neward on Web Services and Security.

    Note: This tech talk was recorded in March of 2005.

    Threaded Messages (24)

  2. What a coincidence... yesterday I also released an interview with Ted Neward on the JavaPolis site... I wonder if it has the same questions ?! :o)
  3. Where's the text transcript?[ Go to top ]

    Hi,

    I hate RealPlayer and I don't mind at all reading the text transcripts of these interviews. I think I can read them faster than I can watch and listen too. Are there no longer transcripts? I could not find one for this or the last couple.

    Thank you,
    Scott S.
  4. Where's the text transcript?[ Go to top ]

    Hi,I hate RealPlayer and I don't mind at all reading the text transcripts of these interviews. I think I can read them faster than I can watch and listen too. Are there no longer transcripts? I could not find one for this or the last couple.Thank you,Scott S.
    i agree. we need transcripts. its also sometimes not easy to watch a video at work. instead its good to have a text article. please someone in TSS can help. ?
  5. Where's the text transcript?[ Go to top ]

    i agree. we need transcripts. its also sometimes not easy to watch a video at work. instead its good to have a text article. please someone in TSS can help. ?

    You must start the presentation _then_ you get the transcript (really, no joke).
  6. Thank you[ Go to top ]

    You're right, thanks! I don't even have RealPlayer and I was able to get to the transcript as you suggested.
  7. Really relevant[ Go to top ]

    If you are struggling with web service design decisions like RPC vs. Document Oriented or SOAP vs. REST this is a good interview to get you acclimated to the various issues and concerns you'll need to take into consideration.

    I thought it was very interesting when he talked about how a REST architecture may in fact use SOAP wrappers to leverage the more fleshed out services available in the standard web service stack.

    Also the bit on how RPC doesn't give you much more than what was provided by CORBA. This is often said but going forward and talking about how RPC and CORBA are both the wrong solution made a lot of sense.
    Thanks for posting this interview, was worth a listen for me.
  8. Message == protocol[ Go to top ]

    Having worked with EDIFACT and ISO messages in the past I wonder why one needs to reinvent the wheel. Message protocols have been successful in many domains for years without Java, OO, RPC and XML. Just use the existing standardized message definitions. It's not about RPC vs. XML, it's all about using standard protocols.
  9. Message == protocol[ Go to top ]

    I kind of agree ;-). Actually, my personal beef is with XML in general. Not that I haven't worked with it, I work with it a lot, from the first POS to Onstar integration using xml4j pre-tomcat Jserve days to custom trading schemas. I think the complexit (yes complexity) of xml has dragged this space down quite a bit. By complexity I don't mean to imply that xml is complex, just that there are other forms for describing data that are just as easy to decipher for humans, but more importantly, more easy for computers in terms of space and time, e.g. JSON, Yaml, etc. Personally, I think that tight adherence to xml schema between web service providers and clients is kind of a joke, in that in all likelyhood, you know the message you have to send/receive, and by the time you get past the initial front door goofiness of simple schema validation, its up to your back end to validate the data anyway. In some ways, I think it has been good for the industry, but mostly in that it has been an opportunity for capital investment and product cycles ;-), but in others, the complexity has pushed toolkit vendors to provide automagic generators that end up stinking ;0).
  10. Message == protocol[ Go to top ]

    ...By complexity I don't mean to imply that xml is complex, just that there are other forms for describing data that are just as easy to decipher for humans, but more importantly, more easy for computers in terms of space and time, e.g. JSON, Yaml, etc...

    Yeah, only reason to use XML is that XML is being used, you know? Its the popular choice therefore the de-facto standard. In a single shop any alternative can be used. Integration with vendor middle-ware usually means XML these days though.

    I'm not willing to jump on the boat and say that there is nothing new with the web service stack and how it is being applied. I'm more inline with saying that it is still relatively new and that the first stab at using tit was mostly copying what was done before: Corba RPC and messaging.

    I think Ted's presentation is correct in that there are opportunities for the current stack to push further into inter-operation and the service-oriented arena than any previous technology; if only on momentum alone. The funny part is that so much time was spent copying Corba when it failed to capture mind share. If at first you don't succeed, change up your protocols and try again. When I say failure here I mean that it isn't popular not that it isn't technologically sound. Maybe Corba failed because RPCs across the network are just a bad idea for most cases.

    Here is another totally different idea. Would it be useful to define a language (maybe just a scripting language) that used the XML Schema types as the built-in types? If it was a language that was primarily focused at working with xml, web services, schema, etc. Would it have an advantage? Obviously, it would have all the standard base operations that could build up any libraries but if it was focused toward easily providing different kinds of web-service endpoints would that be interesting?
  11. Message == protocol[ Go to top ]

    The funny part is that so much time was spent copying Corba when it failed to capture mind share. If at first you don't succeed, change up your protocols and try again. When I say failure here I mean that it isn't popular not that it isn't technologically sound. Maybe Corba failed because RPCs across the network are just a bad idea for most cases.

    CORBA's adoption difficulties are caused by:
    - vendors desire to lock in customers, therefore CORBA 1 implementations were not interoperable; (They are interoperable since CORBA 2)
    - lack of education among crowd of so called 'developers' after “Program in C++ in 24 hours” - they simply cannot grok why CORBA is convenient and make sense;
    - General desire of vendors to come up with things to sell – tools, trainings, etc. It is very difficult to make money out of technology that simply works;

    In my opinion it is time to take honest look at CORBA and start using it!
  12. Message == protocol[ Go to top ]

    http://www.ecma-international.org/publications/standards/Ecma-357.htm

    I believe this is implemented in ActionScript 3.0 (Flex 2) as well.
  13. Message == protocol[ Go to top ]

    Here is another totally different idea. Would it be useful to define a language (maybe just a scripting language) that used the XML Schema types as the built-in types? If it was a language that was primarily focused at working with xml, web services, schema, etc. Would it have an advantage? Obviously, it would have all the standard base operations that could build up any libraries but if it was focused toward easily providing different kinds of web-service endpoints would that be interesting?

    ECMA-357 is pretty darn close and has been a joy to work with in my experience not to mention that you can easily punch-out to standard Java APIs if needed.

    http://www.mozilla.org/rhino/overview.html
  14. Message == protocol[ Go to top ]

    Here is another totally different idea. Would it be useful to define a language (maybe just a scripting language) that used the XML Schema types as the built-in types? If it was a language that was primarily focused at working with xml, web services, schema, etc. Would it have an advantage? Obviously, it would have all the standard base operations that could build up any libraries but if it was focused toward easily providing different kinds of web-service endpoints would that be interesting?
    ECMA-357 is pretty darn close and has been a joy to work with in my experience not to mention that you can easily punch-out to standard Java APIs if needed.http://www.mozilla.org/rhino/overview.html

    Interesting. My only exposure to ECMAScript is using ActionScript 2.0 with Flash. Would be interesting to see the XML Schema/Web Service specific features that you are talking about.
  15. Message == protocol[ Go to top ]

    check out Flexbuilder beta 2, ActionScript 3.0 implements the e4x as well as all(most? can't remember) of the latest ecma std.
  16. Message == protocol[ Go to top ]

    Having worked with EDIFACT and ISO messages in the past I wonder why one needs to reinvent the wheel. Message protocols have been successful in many domains for years without Java, OO, RPC and XML. Just use the existing standardized message definitions. It's not about RPC vs. XML, it's all about using standard protocols.

    couldn't agree more. Besides that, I am sick and tired of this debate regarding RPC vs document centric Web Services and SOA, and some body telling us which one is better for us. Grow up guys, neither of them are good or bad, at the end it all depends, what are you trying to accomplish.

    BTW if we all agreed to adopt IIOP (you all know who had an itch not to adopt it and why)we would be much better off by now. My biggest frustation/concern against current Web Service standards is that, it is the only distributive computing technology that exposes all the low level details of the protocl stack (XML stack and Datatypes) to the developers with all the complexity, after that they dare to call it Simple Object Acess Protocl.
  17. Ted's REST Comments[ Go to top ]

    So Ted says REST is not good enough when:
    • you need to secure the message (he apparently forgot HTTPS for the moment, though he mentions it in the previous topic). Frankly, the idea that you'd be sending highly sensitive, but encrypted, messages over HTTP is unlikely. IT administrators and CIOs will lock down the transport protocol and ignore fancy message encryption schemes that allow snooping of headers or anything else.
    • when you want to *not* use HTTP at all because of performance reasons, in which case you'd definitely not be using SOAP over HTTP either (and I don't think most commercial development tools have a solution for Web services that doesn't assume HTTP as the transport protocol for SOAP, do they?). Actually, in reference to the previous point, I think performance is a valid reason not to use HTTPS everywhere if you can secure the message instead of the transport, though, usually, HTTPS is easier to turn on and maintain in a network than is coordinating public and private keys and encryption decisions between decoupled development teams.
    • I'm twisting his final comments on the topic a bit, but basically he talks about MOM and RPC-like functionality that WS-Reliable messaging purportedly provides but REST does not -- then in the next topic stresses that Web Services are not RPC!
  18. Re: Ted's REST Comments[ Go to top ]

    He damns REST because REST requires HTTP. SOAP does not. SOAP can use anything.

    He damns HTTPS for security because it only secures the transport of the message, not the message itself. This is fine if you're running the same protocol all the way from the client to the servicing end point, but if you jump protocols in the middle (such as his example with HTTPS to the load balancer, and HTTP back to the servers), then suddenly your message is no longer necessarily secure because you've changed transports.

    He also mentins that there is really no particular reason you can not send SOAP messages to a RESTful end point (after all, SOAP is XML, and REST delivers XML payloads).

    He also points out that in scenarios that are more involved than a simple point-to-point connection, SOAP offers more flexibility than pure REST.

    The gist I got out of this talk is that the most basic of SOAP messages is not particularly more involved than any other trivial XML based protocol, however, it can be (and has been) augmented to enhnance the characteristics of the payload in a standard manner. So, if you need to pick a trivial XML document protocol, SOAP is a reasonable solution with no more dramatic overhead than any other, but also extensible in a more industry standard way. You can start out simple, and then later make it as complicated as you want.

    I thought it was a very interesting talk.
  19. I hear ya[ Go to top ]

    He "damns" REST seems a bit harsh.

    Show me a real world example with the HTTP/S jumping you're talking about -- does this jumping take place on the Wild Wild Web or behind firewalls? Come on, as I said, locking down the network is something that IT departments do, not Java developers.

    And no kidding SOAP doesn't require HTTP, in theory. And you and Ted can come up with some edge cases (oh, grid computing? Everyone's doing that...), but the fact remains that the tools and frameworks that most of us use are supporting SOAP over HTTP.

    My main beef is that REST is a lot simpler and more practical than Web services, in general. I think Ted clouded that point with some edge cases that aren't realistic for the majority of developers and businesses working in Java middleware, at least not in my experience. Of course there will be the outliers to pipe up about how they use SOAP over some non-TCP/IP protocol everyday . . .

    Scott
  20. Re: I hear ya[ Go to top ]

    Show me a real world example with the HTTP/S jumping you're talking about -- does this jumping take place on the Wild Wild Web or behind firewalls? Come on, as I said, locking down the network is something that IT departments do, not Java developers.

    We do exactly this, run HTTPS to the load balancer and HTTP back to the servers. And we did it for the exact same reasons he mentioned. We couldn't use the load balancer effectively if we pushed HTTPS back to our servers, and it certainly didn't make any kind of sense to decrypt it at the load balancer only to re-encrypt it before pushing it back.

    I do take pause with his notion that this scenario is not HIPAA compliant, because we are a HIPAA shop. As you said all of this is behind firewalls, so as far as we can see and understand (and according to our 3rd party auditors), this is a safe and compliant scenario.

    But that doesn't mean that per message encryption is necessarily a bad thing just because you have transport encryption. If your application can leverage the vast internet infrastructure designed to move HTTP web packets around (notably proxies and caches, etc.), then message based security makes a lot more sense. HTTPS pretty much burns all of those bridges by requiring a "mini" VPN to the host, whereas packet signing and encryption leverages all of that infrsastructure.

    I agree nothing that WE do would necessarily fit this mold, and nothing that YOU may do can perhaps obviously leverage this. But some applications may.
    And no kidding SOAP doesn't require HTTP, in theory. And you and Ted can come up with some edge cases (oh, grid computing? Everyone's doing that...), but the fact remains that the tools and frameworks that most of us use are supporting SOAP over HTTP.

    Everyone can come up with contrived reasons to use some technology. They can also come up with contrived reasons not to :-).
    My main beef is that REST is a lot simpler and more practical than Web services, in general.

    Yea, but see, this is the part I don't understand. If you're saying the REST is easier then, say, SOAP-RPC, then "Hell Ya", "Hallelujah" I can hear that song. The vernacular is that "SOAP" took the "Simple" and "Object" out of an "Access Protocol" or "SOAP stands for 'SOAP is Not an Acronym".

    But, he's not advocating SOAP-RPC. He's just talking simple SOAP envelopes. A SOAP envelope is trivial. A simple Envelope element, a simple Body element, and your payload. That's it! Nothing more to it. Just because there are a gazillion WS-* proposed standards ON TOP OF the generic SOAP packet, doesn't mean you need to actually USE those standards.

    See, here's the deal. Package up your "proprietary" (*snort*, I know, it's stupid, nonetheless) XML in to an HTTP request and then PUT, POST, DELETE, GET to a REST endpoint is NO DIFFERENT from packaging it up in to a "standard" SOAP envelope and then PUT, POST, DELETE, GET it to a REST endpoint.

    I mean, seriously, the two are identical at a functional level here.

    However, if and when you wish to perhaps interact with some other SOAP system, or toss a WS-* thing (say, WS-Security) in to the mix, well, then you're already part way up the ladder.

    I understand your concern. There's an example WS-Security SOAP message where 28 of the 36 lines of the XML message are security tags and data, and that's just for a signature! (DOH!) So, how can any sane person look at that and say that it's "better" than just pushing the original 6 line packet of HTTPS. But for some application it may be better.

    As much as vendors and what not like to cook this crap up in order to get customers to throw out and rewrite perfectly functional software every 5 years, a lot of this junk does get inspired by real customer demands. Someone, somewhere, is bouncing messages across disparate systems and, apparently, even on clear channels.

    The only real difference between REST and SOAP at this level is that REST doesn't define the payload in any way, shape or form, while SOAP does.
    I think Ted clouded that point with some edge cases that aren't realistic for the majority of developers and businesses working in Java middleware, at least not in my experience. Of course there will be the outliers to pipe up about how they use SOAP over some non-TCP/IP protocol everyday.

    Eh, perhaps. I think the real message here from Ted was basically a "WS Messages > WS RPC". And if you're going to use messaging, you may as well use at least the most base SOAP infrastructure. Simple SOAP over REST easily handles the needs of a vast majority of developers, and it's cheap. If the edge cases should ever bump in to your domain, then you don't necessarily have to even throw anything away to facilitate them. Suck the envelope wholesale from your REST servlet, and stuff it directly in to a JMS queue to integrate with that AS/400 that Accounting just bought.

    Who knows.
  21. I thought the talk was informative. What does this line of thinking say about design methodologies? It seems that not only is there a XML/Object (hierarchical/inheritance) impedance mismatch, but also a mismatch from a methodology perspective.

    Many would argue that OOAD has been a de-facto standard for the last 10 years at least for modeling your domain problems and processes. If we follow Ted's argument that we should be thinking more about *messaging* than *methods/RPC*, then what is the equivalent XML design methodology to OOAD? Instead of coming up with object/domain models, activity diagrams, etc, what do we design, just the XML Schema? How does that task fit into say, an Agile approach to developing messaging-centric SOA systems? Perhaps there is some room for a new messaging/SOA based development methodology.

    In a component based system, where "objects" and components are being shared, OOAD is a great design strategy and methodology. But if we're not going to simply annotate methods in our design as web services (RPC approach), what's the alternative? Is he advocating working more with XML using XPath, XQuery, etc, and never getting into the object domain?
  22. 1. Tell us about yourself and what are you working on?

    My name is Ted Neward. I am an independent consultant and mentor, architect, speaker, author, and whole bunch of other stuff. General prognosticator of rabid opinions in both the Java and .NET space. I am just at this point consulting with a couple of companies and going off to various shows like TheServerSide Java Symposium and in general just making a nuisance of myself to various vendors.

    2. What are the perils of web services?

    There is a couple, fundamentally what it really centers around, is a fact, if you look at the history of web services, we’ve been doing this now, the term has been in place now for seven years. The initial SOAP tool kit started to ship around 1998 and it is now 2005 and we still haven’t really achieved this web service ubiquity that everybody was promising that, just be able to take this SOAP tool kit and be able to talk against that particular middleware server implementation. You can fill more of vendor blanks you want here. No one vendor is more compatible than any of the others with any of the others. Really the origins of that discussion was really centered around the fact that along the way, about three years into this, the web service “thought leaders,” to use Hani’s term, they sat back and looked at what was going on and the way we were building web services and said it’s not working. Why is it not working? Why are we not achieving that global interoperability, that ubiquitous interoperability. We’re are using XML, which is portable data, we are using HTTP, which is a ubiquitous protocol, why are we not succeeding? Why is this not working? They came to a number of conclusions, which took web services into an entirely different direction. This is really where we started hearing about SOA, service-oriented architecture and all of that new buzz kind of stuff. Part of the thing that we run into is the fact that a lot of developers want to look at web services as just CORBA with angle brackets. This is what it really boils down to, RMI with angle brackets. We want to take some of the old style OORPC programming approaches and we want to just apply them to the web service space. Unfortunately when we do that we run into all kinds of problems. For example, one of the things that Simon Guest from Microsoft has identified and he has been really trying to nail down some of the concrete differences between the various web services tool kits in the Java space, for example when we want to create a date we say, new Java.util.Date. It’s what in the .NET space we would refer to as a reference type as an object that’s out on the garbage collector. What that means is that I can pass a null date reference around. In the .NET space on the other hand, we have system.datetime and daytime is what we call a value type. In other words, it behaves just like a primitive type in Java. When I say datetime D, D lives on the stack and occupies X amount of space and so forth. As we know in the Java space I can’t have a null int. In the .NET space, for example I can’t have a null datetime. What these web service tool kits do on the .NET side is they will take a null datetime reference and turn it into a datetime of zero, which is in some place back there in the ancient Roman times. That is different from having a null Date. How do we manage those? That is a very very simple difference between the two platforms. How do we adjust for that? How do we deal with that? And then on top of that, we have the schema type systems that says a datetime, schema has got eight different ways to represent dates and times. We have date, we have time, we have time span, we have month day, day year, month year. I mean all these different ways to represent some point in time on the chronology. How do we wrestle with that? What would a date look like in the Java space? Well, typically it is a Java.util.Date with just the time portion set to midnight. That is a little bit different than just a day. That is the daytime with just the time part, we are just going to agree to ignore. That is one of the basic problems. The other one that we run into is simply that objects in XML, people are starting to pick up on this. This is what I referred to in effective enterprise Java as the object hierarchical impedance mismatch. People are starting to pick up on the fact that objects in XML do not necessarily go together like chocolate and peanut butter. We had this perception that objects could turn into XML and back again with no work required, it would be a lossless conversion, but what we are finding is that XML being an intrinsically hierarchical storage facility does not necessarily mesh well with objects that aren’t. The example that I use for example is you take a person object that has a field spouse, you take another person object that has a field spouse and they point to each other, in this two-object hierarchy which one is the well-formed root? This is the problem that we run into. It is the fact that..

    The man.

    What’s that?

    The man, of course.

    Of course. I am sure if you ever get married and your wife ever gets hold of this tape, you're done. Seriously, this is the problem that we face and so try to solve this problem with what they call SOAP encoding, in that we will create just this arbitrary element, the SOAP body element, and that will be the well-formed root and we can just lay person object here with hrefs that point to the appropriate object. There are two problems there. Number one, I don’t necessarily know that the href is good until I parse the entire body of the soap packet. Number two, there is no typing information associated with that. If we have two people and a cat for ever reason there is a bug in the encoding implementation, we could easily end up with the situation where Floyd points to Floyd’s wife who in turn has a spouse of Fluffy, which is probably not what was intended, but because the ID href mechanism got off guard, it just doesn’t work right. The second problem is even if we make that scheme work, assume that it’s completely bug free. What we end up with is an XML document that doesn’t really look like XML, it doesn’t obey the XPath rules all that well, which means I can't XSLT against and so forth. That in turn means that platforms that aren’t objects will have a hard time consuming and producing this XML. There are a bunch of other things that we could go into, but I don’t want to make this one question to be the entirety of the talk. I could do that, but I do not want to. The point is that for a variety of reasons, we’ve come to recognize that web services can't, shouldn’t, should never have been about taking RPC calls and turning them into things with angle brackets. Now there is this shift in focus, this change in idea where instead of looking at it as an RPC approach, what we want to do is see it as a messaging based approach.

    3. How should we think about web services?

    We start by saying “messages.” We start by saying I want to instead, of here as a Floyd web service and I want to make method calls on that service, we shift it around. This is now getting into at the beginning levels. The difference between RPC encoded and doc literal. I want to shift it around to say look “I am going to just send you a message.” “Floyd, here is the information regarding TheServerSide Java symposium, do what you do with this message.” I am not going to tell you how to do it. I am just going to say here is the message, you go off and take care of what needs to be taken care of. What that does when we take it out of the RPC mindset and we put it into a more messaging mindset. A, it allows to be more coarse grained, which of course as we’ve learned the hard way in the J2EE space is the better approach to take because of the implicit penalties of pushing something across the network. B, it helps preserve encapsulation to a greater degree. I don’t necessarily have to have intrinsic knowledge of the various APIs that are available on this service. I just say look, do this. There are still some open questions about the granularity of a service. I just say “Do this.” What is a service? It is just an end point that I can send a variety of different messages to and if so what is the granularity of those messages? Don’t get me wrong, we do not have all these answers figured out yet. People who say they do may have figured it out for a particular service that they have worked on, but at a larger level it is still very much an open question. I posed this on my web log at one point “I said look let’s work out, let’s describe what is the granularity of service. Is a database a service?” You could think of it that way in a lot of respects. It’s a service that accepts SQL as the message. Is a filesystem a service? To put it in J2EE terms, is a stateless session bean method call a service? There is a huge degree of granularity there. We still have to work out some of that stuff. But once we start to take this into a messaging based approach, we allow for a lot of things to happen that could not happen in a traditional RPC scenario such as intermediaries.
    What are intermediaries?

    An intermediary is effectively a node, you can think of it as a hardware node or maybe a software node, but let’s just call it a node, nice abstract term that sits between you and me as the principals in terms of this communication. In the networking world, the intermediary might be the router, it might be a firewall, it might be a caching server, a proxy server, any of a bunch of these things that provide what in the J2EE space we would call a crosscutting service. In the soft world, if we start talking about building web service intermediaries, one intermediary might be a load balancing sprayer. So that a request would come in and then that messaged based on perhaps the content of the message, based on perhaps load across the various servers in my farm, whatever, that message gets routed to one of those for further processing. One of the things we’ve run into in the web services space is let’s assume we want to secure that message. There is some sensitive information, medical, legal, financial or whatever data that nobody who is anywhere between the two endpoints should be able to see. How do we secure it? Today people are saying just use HTTPS. Okay, that is simple and it works for point-to-point as long as I am talking directly to you but if there is an intermediary between us, now the trust relationships get a little screwy because in HTTPS, I get hold of your digital certificate and I say, “Yes, I trust that I am talking to Floyd.” because I have verified the certificate, the certificate is signed by whomever, Verisign usually, which validates that you are who you say you are, etc., good, but if there is an intermediary in between us, HTTPS says I have to get hold of your certificate if I am going to trust you. The intermediary wants to sit here and in particular if that intermediary is a security intermediary, one of two things has to happen, either he has to be entirely unaware of the conversation, so he just passes the HTTPS request along in which case we can't do any of those useful services that we might want to do like examine the message and figure out which of four different servers to spread that message to. Gregor Hohpe and Bobby Woolf called this the content-based routing pattern in Enterprise Integration Patterns. It just has to pass that across in an opaque fashion and another useful service would be, for example, a load balancer may want to peg requests from me to always go to that particular machine so it can maintain in memory session state for example. You can't do that if it sees that it is just an opaque set of bits. What we do today often is we put the digital certificate right here on the load balancer and then say okay, that's really sort of the front end to my service. That means everything from the load balancer to the actual machine that carries out the request is unencrypted and suddenly you are in violation of HIPAA requirements. Let's assume we want to do HTTPS to both levels. Now, we have to get into this interesting, “I have to trust that certificate and he has to trust that certificate” and if we ever put a third party into this picture, which a lot of large scale enterprises are starting to think about, for example, voice over IP and some of that stuff, we might want to send, I might want to have an intermediary be the physical infrastructure that a message travels across and these intermediaries may want to tag it with entry point and exit point, so we can figure out what to charge you for passing this message along the network, just like the United States Postal Service requires a stamp on any letter you send out. Same idea, we want to put that into the networking infrastructure. HTTPS isn’t going to work there, it’s just not going to be able to convey what we need to convey. Thus was born WS-Security for elevating the security bits, the encryption bits, into the message itself rather than relying on the underlying transport.
  23. 4. You mentioned that messaging approach as opposed to RPC approach opens up web services for many new use cases and scenarios. Can you talk about some of those?

    Well, one of the things that the messaging approach opens up, for example, and this is sort of again another one of the new ways of thinking in the web services space what we call transport agnosticity. It shouldn't matter how we transport this message and if you think about it, we have that today with the US Postal Service. I write a letter, I put it in an envelope, I drop the envelope in the box. How it gets to somebody else's mailbox really doesn't matter to me. I don't care if they Pony Express it, I don't care if some guy drives from here to your house. I don't care if they drive to a local post office sorting station where from there it goes to another post office sorting station. Both in those cases over the ground from there it hops on a plane and then from there it goes to a sorting office, I don't care about any of that plumbing and I shouldn't care about the plumbing is the point that the new web services mantra is trying to make. We can imagine scenarios where transport agnosticity can really, really work for us. For example, let's assume you have an interesting service that I want to consume but I don't necessarily need a response. What I will do because you are a 24 x 7 guaranteed-to-be-up server, I will send you the initial message over HTTP. For what I am going to say is that in the event of a fault and this is the WS-Addressing part of the web services spec, in the event of a fault I want you to send me an e-mail back. You don't need to reply unless something screwed up, in which case you send me an e-mail back. Again, because we are not tied to the underlying transport because of these specs care only about the message itself, I can do this and once the message gets into your network cloud, you may decide to use a much faster transport than HTTP, for example, named pipes on Windows box or you may choose to use shared memory on a UNIX System V IPC box. The message doesn't care where it’s being transported and as we start introducing this notion of intermediaries we can imagine that I do an HTTP request to here and from there it becomes an FTP drop into some directory, where from there it goes across shared memory to maybe some high speed gigabit LAN network to do the actual processing and if there is a fault, we send an e-mail, an SMTP message back to me indicating the fault. All of that is possible if we don't make any assumptions about the underlying transport and again this was one of the flaws in the SOAP 1.1 spec that it relied on a HTTP header, what we call the SOAP action header to be there. That basically was the indicator of, “Hey, when this message arrives at the server, this is the method I want to invoke” and if I can't know the transport, if I can't assume HTTP, then we need to change that assumption and SOAP 1.2 again is transport agnostic, which is why WS-Addressing was born, which is why WS-Reliable messaging was born, because if I need to make sure that message arrives I can't rely on the transport. All of these things this is the real shift in thinking that wasn't there seven years ago.

    5. What do you see five years from now? Once we have transport agnostic web services, how do you see typical projects integration being done?

    If I could answer that reliably, I could be a very, very rich man. A lot of it depends on how things go and a lot of it doesn't even have to do with the technology. A lot of it has to do with how developers warm up to web services. If we continue to see web services as RPC with angle brackets, web services aren’t going to make that big of an impact on our industry. They’re really not. Because in many respects if you look at web service as RPC and then compare it against CORBA, what's really different here? What's new and different between web services and CORBA? The short answer is “not much.” CORBA guys sat down and defined a platform agnostic data representation. They sat down and wrote a whole bunch of services. There were like 18 or 20 different CORBA services at last count. There is just no difference between web services and CORBA at that level. If we sit down and really sort of re-examine our thinking and if the developer community as a whole is willing to say, “You are right, RPC works maybe for these scenarios but for real integration I don't want to go to the RPC route, SOA, messaging, this is all the right way of doing things,” if you are really will to admit that and accept that and take it into our hearts, then we have some real opportunities for doing integration across platforms and it’s possible that we could really reach a state where .NET and J2EE and any other platform you care to name here could work, not necessarily seamlessly but very well with one another and these platform wars that we get into would just sort of fall away. Today, for example, across the Internet does it really matter what operating system your web server is running. Not really. I could run anything, LINUX, I could run Solaris, I could run Windows, I could run DOS, doesn’t matter. We could really see that if we are willing to accept the necessary prerequisites to platform agnostic web services.

    6. You mentioned earlier the two perils of web services - incompatible data type platforms and also the object XML mismatch. How would you take a message, when you say message I essentially mean the document oriented approach to solve those two perils?

    What we are doing at that point is we’re saying it’s not about your type system, it’s not about my type system. It’s all about the XML. Justin Gatlin, one of the guys I speak with on the No Fluff Just Stuff tour and a former developMentor instructor, buddy of mine and so forth, he really nails it because he says the message is the application. The message is really the centerpiece, the focus and what it boils down to is you and I sit down and we work out what that message wants to look like at least between the two of us. We will say, “All right, we’re going to agree that this is going to be an XML datetime.” We are just going to make that assumption and by the way, we are going to make sure that this datetime is never null. There is no null capability here. If we want it to be null, we can explicitly say that because we can set an attribute on this schema element that says nullable is equal to true, which then tells the .NET guy, by the way, it is possible for this thing to be null so I need to check to see if that element is present or not. We just sort of agree that it is not about my type system, it is not about yours, it is about this message and we have to agree on what this message will contain and we have to agree on what it won't contain. There is a danger here though and that is that we have to be willing to lay down this contract, but write this contract in such a way that if ever you want to send more data than was present in this contract, it is not going to screw up my end and this gets into what we start to call relative schema discussions. Noah Mendelson, according to Don Box and some of the other web services folks, was the first guy to say, “Schemas are not type systems, schemas are relative.” There is this real perception, there is this real sort of attempt right now on the web services space to say, “Look, Schema and WSDL are your IDL, we are back to CORBA terminology again” and you and I sit down and we work out the one true WSDL, the one true schema for how we are going to exchange information. Problem with this is if you and I work for different companies, particularly for part of a consortium, say, four, five, six or ten different companies and we sit down and we hammer out that V1, there is never gonna be a V2. Once you publish an interface like that, you can never change it because getting all of those players to agree on a change is just flat out impossible. You lose health, sanity and hair when you do this. Instead what we’d like to do is recognize, look it’s not about we’re gonna to build schema types and we’re going to make this the type system for XML. They are relative integrity constraints like which you see in a database. If you and I have two different databases and I take a table and dump it to a comma separated value file and hand it to you, the integrity constraints on my end don’t go with that file. As long as the file matches the integrity constraints on your end, that’s fine even if our two integrity constraints are different. This is really what we want to do in the web services space to some degree. It requires a bit more work because we have to make sure that the subset that we agree on is the same, but if you want to ignore particular elements of the data that I send you, that’s fine, that’s up to you, and we have to write schemas in such a way that we in fact accept that. For example, schema has a wild card element type called xsd:any or wild card attribute-type called xsd:any attribute. When we design a schema we have to allow for that evolution, we have to allow for that flexibility by saying, “All right, this is the baseline what you and I agreed we’re going to send back and forth to one another” and by the way, our wild card or void star or object reference in Java is going to be an xsd:any right here. If there is anything else, it’s all right. The other thing we have to be willing to do in some cases is be willing to infer data even if it’s not present in the packet. It’s an old maxim by John Postel, one of the guys who was part of the initial growing Internet, way, way, way, way back in the mid 80s and said, “Be strict in what you send, be liberal in what you accept.” This is really a key if you're going to make interoperability work. What I’m basically saying is, you send me an XML that doesn’t quite fit the schema, but I can figure out what you meant, I’ll accept it, I’ll take it. Programmers don’t like that. They really don’t. They like having people who obey the rules, but if you look across the history of the Internet, for example, every browser, every version of every browser had a different interpretation of HTML and it supported different tags in HTML and it supported extensions to HTML and let’s assume the browser got a particular extension that it didn’t understand, it just ignored it. Page might not render exactly the same way as it would in another browser, but it just ignored it. If we had a strict interpretation of HTML rather than the very loosey goosey implementation that most of the browsers supported, the world wide web would nowhere be as it is now, we just wouldn’t even be close, we wouldn’t even be talking about building browser-based applications because “Oh, I’ve got to make sure I’m exactly correct to HTML” and well, what if my interpretation of HTML is different from yours, that means that my definitions of exactly correct is different from yours and so half the pages you would go to on the Internet would come back with a 412 error, invalid HTML, which doesn’t exist, but we would have said code and I can’t see anything on the page, that sucks man. I’m not going to do this browser-based stuff. Be strict in what you send, be liberal in what you accept has to be the driving mantra for web services and we can see it. We have been strict up until this point and name for me a web service that is a stock tick or a weather report, they're not there, they're just not there.

    7. Recently there has been a lot of debate in the blog space about whether SOAP is comatose and REST is the way to go. How do you feel about that?

    Again, we have to be a little precise with the terminology. A lot of people when they say SOAP, they're referring to the mound of stuff that comes with the web services spec and when they say REST, they want to say, “Oh, let’s just XML in, XML out over HTTP” and I think that these two perspectives serve two very different purposes. The RESTful approach has as a natural advantage, it’s simple. RESTful design basically says “Look you modeled your resource as just an HTTP URL. If we want to talk about users on a system, when I want to add a new user, I do a put, that’s part of the HTTP 1.1 spec, I do a put containing the XML that describes the user to HTTP://floyd.com/users and you have a servlet on them and bound to that URL that sees the put, sees the XML and says, “Okay, this is cool.” If I want to retrieve information about a particular user, I do get http://floyd.com/users/name or ID or some other identifying characteristic and then, you’ll send me back the XML for that given user. I do a delete if I want to remove that user, I do a post if I want to modify that given user, etc., etc. It’s a very natural way of thinking about doing middleware-based system that’s very entity based. We run across a couple of problems when we start talking about the RESTful world. Security is one of them. Again, the RESTful guys say, “Security just rely on HTTPS, we’re done.” Okay, but we start getting into that intermediaries model and so forth and that notion of “just do HTTPS” breaks down. Interestingly enough, the SOAP 1.2 spec is just framing an extensibility. It just says SOAP envelope contains SOAP header, which is optional, in which all these different headers, none of which SOAP mandates, can be present. Things like WS-Security, WS-Addressing etc., and then SOAP body, which is the actual payload of the message. So you can actually envision a world where the RESTful guys take their XML and wrap it up in a SOAP packet and now all of a sudden, we’re not talking about two approaches that are incompatible with one another. We’re talking about sort of incrementally getting into the web services space. We start by saying, “Look, we’re going to have that users endpoint off of your web servers from your site and I’m going to send you a SOAP packet.” Now, okay, I need to actually take that message and spray it out to various servers behind the scenes because I’ve started to scale higher. Well, okay. The client who sends me the message WS-Security encrypted before you send that message to that resource and then that message can flow through intermediaries until it gets to the actual recipient who then unencrypts it and does the usual RESTful thing. These two models, these two approaches are not necessarily antithetical to one another. They are, if we continue to see web services as a RPC thing, but the RESTful model is really just model these resources on the web and then send me a SOAP message and there is actually, I wish I could remember the name of it right now, but there is actually a WS spec that talks about doing this exact approach, the name escapes me, it’s really bugging me, if I find it I will send you an e-mail and we can just stick that into the text of the transcript. (Editor’s note: Ted was referring to WS-Transfer: see http://www.systinet.com/ws_specs for more detail.) Basically what this does is it’s essentially REST for the SOAP, for the WS spec, which allows for a lot of possibilities, allows for a lot of incrementally getting started with this. Again the biggest advantage that REST has is it’s simple and because we describe this XML typically with schema or Relax-NG or a Word doc, DTD, doesn’t matter how I describe the XML document just so long as you and I can understand it. We start by saying, “Here is your RESTful endpoint,” I know how to call you if we start to get more complicated, if we start to get out of what I want to take on myself as a programmer because REST really sort of assumes you're not going to have a whole lot of toolkit help, then we can start leveraging bits and pieces of the WS spec. We want to go transport agnostic, I can start indicating what I want to have happen, who this message is targeted to with WS-Addressing headers and then, this way if something goes wrong, you know how to send it back. That just sits very naturally in the SOAP header section and here is your RESTful XML to begin with. These two really start to work well together. I don’t think it’s a debate, I think it’s just two different perspectives than can in fact mesh if we look at it that way.

    8. In what scenarios is REST not good enough?

    Like I said, the WS-Security model where I need to secure the message because it’s going to move across multiple intermediaries, REST by definition relies on the underlying transport. It wants to make use of HTTP headers and wants to provide HTTP headers and for example, the endpoint of a RESTful service is described by an HTTP URL. What if I want to do something faster, if I want to do this over named pipes because I’m going to actually follow the IBM grid model and I want these web services these RESTful XML in, XML out to be part of a large network of machines and I want communication to be just really, really fast. I don’t want to go through the overhead of HTTP and TCP/IP and so forth. REST has no answer here quite frankly. Interestingly enough, there has been sort of a new thought pattern, new sort of thought leadership analytical exercise, I’m not sure what else to call it, called MEST, Messaging State Transfer. Basically let’s take the RESTful approach and just put more of a messaging orientation to it and do it that way, which I think is really at least based on what I’ve been reading so far is really what we’ve described what I talked about earlier, where we start with a RESTful model but look at it as messages going back and forth and we will build from there. Again, I don’t think these two are contrary to one another, but again, the RESTful approach’s strength is in its simplicity but unfortunately that’s also its weakness because of its simplicity, because it’s based around HTTP and XML and if we need to do anything beyond what HTTP and XML give us, I need to make sure that the thing got there. I want you to acknowledge the fact that it arrived and if you haven't acknowledged within a certain period of time, I need to resend the message, once and only once kind of quality of service, and text, I’ve got to build all that myself as do you, whereas if I have like a WS-Reliable messaging implementation that’s baked into the plumbing, so that’s the strength and the weakness of REST is its simplicity.

    9. If you are God, not that you are, assuming you're like all powerful, how would you fix web services?

    I think probably if I had the capacity to do whatever, if I’m all powerful as you say, I think probably what we need to do is we need to get developers who are building web services to accept that this isn't RPC. If we can get that far, I think a lot of other good things will result. Unfortunately that takes a two-fold action. 1. I have to go to each of the vendors and I have to beat them over the head with a two-by-four to stop making web services look like RPC calls because unfortunately, because these guys make money off of tools, they need to build stuff that is tool oriented. For example, Microsoft, the web methods API that they're doing, which we are starting to see some in the Java community where I just take my class, I annotate it with a special annotation, what we call a custom attribute and lo and behold it’s a web service. EJB 2.1 does the same thing with web services. JSR181 wants to do the same thing. Start from your code, just sprinkle some web service magic pixie dust on it and lo and behold you have a web service, bad things, bad, bad, bad, bad. I need to beat the vendors over the head to stop doing that, to instead make tools, build tools that want to look at and see and recognize that (A) this is a message and (B) it’s in XML because wanting to take your code and seamlessly turning it into objects and back again, we’ve already talked about that. That’s just not going to work. It’s not going to happen. Once we beat the vendors into line, then it’s going out to the development community and saying, “Okay, here we have tools that support the idea of looking at this from a message oriented perspective, don’t fight that, don’t try to build RPC scenarios out of this messaging oriented approach” and what I would do is, as soon as the developer starts to think about get onto the slippery slope that leads to RPC, have their machine pop up a message box and say, “Are you really sure you want to do this?” and if they click “Yes,” send a really strong electric shock through the keyboard to remind them that this is a bad idea and it’s not going to interoperate well. Practically speaking I think what this means to developers is, if you're going to be building web services, you cannot take it from a language or platform oriented perspective. If you're out there today building a web service and you're not downloading the other platform and working with that as part of your test leads, if you're building a Java web service, you have to download the .NET framework, you have to download NUnit and you have to write your unit tests as NUnit .NET unit tests. If you don’t want to learn C# that’s fine. Visual J# is actually very approachable for many Java programmers, so you can just write your tests in NUnit J# and you don’t have to learn any of the .NET plumbing because all you're trying to do is test it to make sure it works, but if you're not doing that you're fooling yourself. You're just checking off the marketing check box that says, “Oh, we’re a web service,” can’t do that. You have to go into this with this mindset that says, “I care about interoperability.” If you just want to do CORBA with angle brackets, fine, you can do it, but don’t be surprised when partners come to you and say it doesn’t work and when you tell them, “Well, that’s because the problem is on your end” don’t be surprised if they walk away and sever the partnership. That’s really what we have to do from a practical perspective to make this web services stuff come to fruition. Beat the vendors into line, beat the developers into line, and we can be successful.

    All right. Thanks.

    Thank you.
  24. Ted Neward says 2005.

    Look like Floyd Marinescu interviewed Ted Neward.

    2005 interview is only 1-year old in 2006. Therefore, readers: you can still read and watch this interview. It is ok. Go ahead.
  25. great interview[ Go to top ]

    Unfortunately, one important thing was overlooked in the interview: the current stack of WS standards is hideously complicated and bloated beyond belief. You practically have to put a gun to programmers heads to make them do WS stuff. Most of the OASIS and W3C stuff is great bed time reading if you suffer from insomnia.

    I've been involved with several web service projects. In each of these projects I had to deal with the fact that most of the technical partners + all of the management had a very poor understanding of SOAP technology (forget about WS*). Basically that sucks if they are exposing a .Net service (read, a C# programmer clicked his way through some wizard) that you need to consume. Forget about stuff like UDDI, some guy will mail you a URL.

    The key problem with the WS* standards and SOAP is that they require tools that have not been invented yet. WSDL and XML schema are nice machine readable formats. They are however highly inappropriate for any kind of non machine processing. Basically to the untrained eye it looks like tag soup. It doesn't get much better if you do know what you are looking at. I've spent entire afternoons debugging WSDL files only to find that some typo that I finally spotted the 50th time looking at the same xml was the cause of my grief. Basically the toolsets from all the big vendors are mostly about vendor lock in. Bea, IBM, JBoss & Microsoft love SOAP + WS* because they know that once developers are on their tool sets there is no way in hell they can get off cheaply.

    All the simple tools are RPC oriented. When doing Java you only have two realistic options: spend lots of money on one of the vertical stacks of tools or use Apache Axis. Good luck if you are trying to do anything more complicated than RPC with the latter. You'll quickly find yourself skimming through the axis developer mailinglists for bits and pieces of information.

    Basically all the usecases for SOAP and WS* I've seen don't apply outside the really big enterprise world. In fact 99.9999% of all usecases I've seen in real life, actually are easier to implement and support not using SOAP and WS* but using REST or even good old RMI. Often the only reason SOAP enters the picture is that some dork puts it in the requirements for no obvious reason and imagined benefits that never materialize. The whole REST vs WS* debate is about light weight vs heavy weight. In the vast majority of cases, light weight is cheaper, faster and more flexible.