Discussions

News: RabbitMQ - Open source AMQP implementation released

  1. LShift and CohesiveFT have launched RabbitMQ, a complete open source implementation of Advanced Message Queuing Protocol (AMQP), an emerging standard for high performance enterprise messaging. This isn't the first implementation of AMQP; that was done by iMatix. It isn't the first open source implementation, that was release as "Qpid" under Apache but it is the first implementation written in Erlang, an open source DSL designed specifically for the telecommunication industry. The interesting thing about this and in fact ANY AMQP implementation is that they should all inter-operate with each other: you should be able to send a message in Java over an Erlang broker and read it in C++. AMQP was specifically designed to help commoditize the messaging market place, something until now dominated by IBM's MQ Series and Tibco RV (all other JMS's occupy the remaining <5%). This IS NOT a replacement to JMS; it operates at a lower level. You can run JMS over AMQP just as you can with MQ Series. AMQP is a universal, open, wire-level messaging protocol for high performance point to point and pub/sub. Message was edited by: joeo@enigmastation.com

    Threaded Messages (50)

  2. Interop verification[ Go to top ]

    The interesting thing about this and in fact ANY AMQP implementation is that they *should* all inter-operate with each other
    Are there any plans for any kind of independent verification testing to validate this? I have evaluated AS2 adapters before now, and the Drummond Group maintain a matrix of AS2 adapter that have been *proven* to both comply with standards and actually interoperate. This makes a big difference for end-users because a) there is obviously a list of compliant vendors, and b) it puts pressure on the vendors to *ensure* their offerings interoperate. Regards Kit
  3. proving interoperability[ Go to top ]

    Kit Good question. My understanding from tracking the AMQP WG is that a TCK is in the works. There may be professional services and product offerings to certify interop in the future. Given how eg. the FIX protocol evolved, I would expect this to happen in line with adoption. For now we are offering the following 1. a test server: http://www.rabbitmq.com/examples.html 2. evidence of interoperability and conformance: http://www.rabbitmq.com/documentation.html Please *do* let us know if you disagree with any of the conclusions we have drawn. We are keen to see a TCK or move towards one ourselves as this is good for the market. If you would live to get involved, please let us know. Thank-you alexis
  4. ... the messaging market place, something until now dominated by IBM's MQ Series and Tibco RV (all other JMS's occupy the remaining <5%).</blockquote> This might be true for one bank, may be. But otherwise this 5% is a complete misleading bullshit. JMS is the standard in Java messaging and covers nearly 100%. -- Andreas
  5. ... the messaging market place, something until now dominated by IBM's MQ Series and Tibco RV (all other JMS's occupy the remaining <5%).</blockquote>

    This might be true for one bank, may be. But otherwise this 5% is a complete misleading bullshit. JMS is the standard in Java messaging and covers nearly 100%.

    -- Andreas
    Andreas, you're a bit confused. JMS takes nearly 100% of the market for messaging in Java, it's true. But JMS is only an API. You use JMS to talk to a broker like MQ Series, Tibco etc. The 5% figure that John refers to is for the other messaging vendors besides IBM and Tibco... eg SpiritSoft, Sonic, SwiftMQ, etc. JMS does nothing to encourage interoperability between messaging products, because it doesn't say anything about the wire protocol that they have to speak. However JMS certainly will continue to be the standard API for Java clients connecting to messaging brokers, including AMQP brokers.
  6. JMS does nothing to encourage interoperability between messaging products, because it doesn't say anything about the wire protocol that they have to speak.
    Neither does AMQP either yet as you can't use AMQP to talk to MQSeries, TibCo, BEA, Sonic, SwiftMQ, Sun etc. So neither the 95% nor the 5% of current commercial messaging middleware products. Sure one day IBM, TibCo, Sonic, BEA, Sun et al may decide to support AMQP. Until then, the best way to bridge different MOM providers is to actually use the JMS API :). James LogicBlaze Open Source SOA
  7. JMS does nothing to encourage interoperability between messaging products, because it doesn't say anything about the wire protocol that they have to speak.


    Neither does AMQP either yet as you can't use AMQP to talk to MQSeries, TibCo, BEA, Sonic, SwiftMQ, Sun etc. So neither the 95% nor the 5% of current commercial messaging middleware products.

    Sure one day IBM, TibCo, Sonic, BEA, Sun et al may decide to support AMQP. Until then, the best way to bridge different MOM providers is to actually use the JMS API :).

    James
    LogicBlaze
    Open Source SOA
    Of course the other way to do it is to use the JMS API's provided by the current AMQP implementations and use that as the basis to integrate with those that do not interoperate. In so doing you isolate those that do not want to talk together from those that do and exert pressure on the market to change it's proprietary practices.
  8. JMS does nothing to encourage interoperability between messaging products, because it doesn't say anything about the wire protocol that they have to speak.


    Neither does AMQP either yet as you can't use AMQP to talk to MQSeries, TibCo, BEA, Sonic, SwiftMQ, Sun etc. So neither the 95% nor the 5% of current commercial messaging middleware products.

    Sure one day IBM, TibCo, Sonic, BEA, Sun et al may decide to support AMQP. Until then, the best way to bridge different MOM providers is to actually use the JMS API :).

    James
    LogicBlaze
    Open Source SOA

    Of course the other way to do it is to use the JMS API's provided by the current AMQP implementations and use that as the basis to integrate with those that do not interoperate. In so doing you isolate those that do not want to talk together from those that do and exert pressure on the market to change it's proprietary practices.
    I should have added that the other way to go, which seems to be espoused, has a logical end game of everyone gives up now and just uses MQSeries. Then we don't have any interop problems at all.
  9. JMS does nothing to encourage interoperability between messaging products, because it doesn't say anything about the wire protocol that they have to speak.


    Neither does AMQP either yet as you can't use AMQP to talk to MQSeries, TibCo, BEA, Sonic, SwiftMQ, Sun etc. So neither the 95% nor the 5% of current commercial messaging middleware products.

    Sure one day IBM, TibCo, Sonic, BEA, Sun et al may decide to support AMQP. Until then, the best way to bridge different MOM providers is to actually use the JMS API :).

    James
    LogicBlaze
    Open Source SOA

    Of course the other way to do it is to use the JMS API's provided by the current AMQP implementations and use that as the basis to integrate with those that do not interoperate. In so doing you isolate those that do not want to talk together from those that do and exert pressure on the market to change it's proprietary practices.

    I should have added that the other way to go, which seems to be espoused, has a logical end game of everyone gives up now and just uses MQSeries. Then we don't have any interop problems at all.
    I think congratulations are in order because at least some folks are trying to solve the interop issues. Better to congratulate than to say it will never work or even to state the obvious that it is early days on the adoption curve. I suspect Alexander Graham Bell had the same problem with the post office when we unveiled the telephone. I look fwd to the day when other open source JMS providers also provide AMQP. Any idea how long this might be?
  10. Better to congratulate than to say it will never work or even to state the obvious that it is early days on the adoption curve. I suspect Alexander Graham Bell had the same problem with the post office when we unveiled the telephone. I look fwd to the day when other open source JMS providers also provide AMQP. Any idea how long this might be?
    AMQP is more akin to changing the size of everybody's letter box to accommodate a standard postage size ;). The problem is the barrier to entry is too high, if it was easier to implement without a re-write, vendors might be more inclined to do it. And yes, ofcourse ActiveMQ has a had a go at shoe horning it in to the existing broker, it ain't pretty, but if there's demand for it, I'm sure we'll try and make it work.
  11. Better to congratulate than to say it will never work or even to state the obvious that it is early days on the adoption curve. I suspect Alexander Graham Bell had the same problem with the post office when we unveiled the telephone. I look fwd to the day when other open source JMS providers also provide AMQP. Any idea how long this might be?


    AMQP is more akin to changing the size of everybody's letter box to accommodate a standard postage size ;). The problem is the barrier to entry is too high, if it was easier to implement without a re-write, vendors might be more inclined to do it. And yes, ofcourse ActiveMQ has a had a go at shoe horning it in to the existing broker, it ain't pretty, but if there's demand for it, I'm sure we'll try and make it work.
    Hmmm. Interesting observation. I thought it would be easy to put a broker between the AMQP world and the non-AMQP world. Bit like the way that SpiritWAVE pioneered JMS with it's layered open architecture. Maybe it is more difficult. But curious that Mule and RabbitMQ seem to have done it rather quickly though. Maybe we should all try it. After all people never really did believe that SpiritWAVE was as open as it was and it took a long time for the doubters to realise it was quite good - think it was your work too Rob ;-)
  12. Better to congratulate than to say it will never work or even to state the obvious that it is early days on the adoption curve. I suspect Alexander Graham Bell had the same problem with the post office when we unveiled the telephone. I look fwd to the day when other open source JMS providers also provide AMQP. Any idea how long this might be?


    AMQP is more akin to changing the size of everybody's letter box to accommodate a standard postage size ;). The problem is the barrier to entry is too high, if it was easier to implement without a re-write, vendors might be more inclined to do it. And yes, ofcourse ActiveMQ has a had a go at shoe horning it in to the existing broker, it ain't pretty, but if there's demand for it, I'm sure we'll try and make it work.

    Hmmm. Interesting observation. I thought it would be easy to put a broker between the AMQP world and the non-AMQP world. Bit like the way that SpiritWAVE pioneered JMS with it's layered open architecture. Maybe it is more difficult.
    The devils always in the details - damn those pesky leaky abstractions
    But curious that Mule and RabbitMQ seem to have done it rather quickly though.
    ESBs like Mule and ServiceMix interoperate between messaging vendors using the JMS API - not by going from (say) MQSeries to AMQP to (say) TibCo. James LogicBlaze Open Source SOA
  13. James old chap, I really think you're missing the point of AMQP, in fact I think you've been missing it for some time. Yes it's very true that if the whole world implemented one of the many many wonderful ActiveMQ clients all our problems would be solved and we could start to direct our collective brain power to something more worthy like global warming. "But", say the customers, what if the nice people at LogicBlaze decide to do a Mule on us and change the licensing, what if they do what SwiftMQ did and provide an excellent free implementation but then suddenly start charging for it? With an open protocol, NOT the API but the protocol then we're no longer at the mercy of anyone, vendor or open source implementation. If someone starts using RabbitMQ and a better version comes along we can start to use it and everything will continue to work together, we don't need to replace everything to get it all working again. It's very like IIOP, does anyone remember the early days of J2EE, we all downloaded various attempts from IBM and wrote little example clients in Java. The problem was that Web'phere used IBM's JVM and most of our demo clients were using Sun's JVM, nothing worked. We were still using the same APIs, the code was identical it just didn't work! This was all solved by the mandatory use of IIOP in J2EE 1.3, all of a sudden everything could inter-operate regardless of the JVM or J2EE implementation being used. Another good example of open protocols is HTTP, as long as you stick to HTTP 1.0 or 1.1 etc. you never hear of one browser not talking to a different make of web server. HTTP is a pretty limiting (I usually use the word "crap"), protocol but it's formed the core or a lot of classic web services because of its interoperability. With luck, AMQP might just become a new core component of web services, it could do with a good revamp. AMQP has the same goals (as IIOP and HTTP), the bright JMS vendors will start to implement AMQP underneath as the standard protocol, the JMS API will stay the same but anyone using the new JMS implementation will be able to inter-operate with other JMSs that also use AMPQ. Finally we're not locked into specific products. What I and it seems a lot of potential AMQP users would like to see next is tooling. Being open, in theory, one tool should fit all. If anyone's interested or working on such things please let me know, I'd love to hear form you. Don't take it personally James, you're not the only one. Long live ActiveMQ, LogicBlaze, RabbitMQ, Qpid, Celtix, Mule and all these other wonderful technologies. -John Davies- CTO, C24 London, UK
  14. James old chap,
    I really think you're missing the point of AMQP, in fact I think you've been missing it for some time
    I really do get the point of protocols John. I think you've been misunderstanding me for some time.


    Yes it's very true that if the whole world implemented one of the many many wonderful ActiveMQ clients all our problems would be solved and we could start to direct our collective brain power to something more worthy like global warming.

    "But", say the customers, what if the nice people at LogicBlaze decide to do a Mule on us and change the licensing, what if they do what SwiftMQ did and provide an excellent free implementation but then suddenly start charging for it?
    There's zero chance of the licensing of Apache ActiveMQ changing as the code is all Apache 2.0 licensed, the trademark has been donated to the ASF (a non profit) and the community is all at Apache. So it would be against the trademark to call something Apache ActiveMQ (or just ActiveMQ) unless its at Apache under the Apache license. Though I can't speak for other vendors/products.


    With an open protocol, NOT the API but the protocol then we're no longer at the mercy of anyone, vendor or open source implementation.
    With an open API - namely JMS - you are not at the mercy of any JMS vendor either which has been the case for the last 9 or 10 years. The only real difference with AMQP is if you are not using Java. Like I said - the big win for AMQP is a set of reusable non-Java clients that work across the 2 or 3 AMQP providers availble today and whatever AMQP providers come along in the future.
    If someone starts using RabbitMQ and a better version comes along we can start to use it and everything will continue to work together, we don't need to replace everything to get it all working again.
    Assuming things really do interoperate. YMMV - see the lovely WS-* stuff :). Who here is old enough to remember how bad IIOP was at interop, even from different implementations from the same vendor? :)
    Long live ActiveMQ, LogicBlaze, RabbitMQ, Qpid, Celtix, Mule and all these other wonderful technologies.
    Agreed! Especially the first two! :) James LogicBlaze Open Source SOA
  15. Well, what I don't understand either is this thing about breaking market power and cutting prices of IBM and Tibco while none of these smart banking VPs has the balls to buy middleware from anybody else, except these two companies.
  16. Well, what I don't understand either is this thing about breaking market power and cutting prices of IBM and Tibco while none of these smart banking VPs has the balls to buy middleware from anybody else, except these two companies.
    Because very few, if any, of the smaller vendors (open source or otherwise) can provide the necessary financial guarantees in the event of a lost message. Supposing you loose just one message at the wrong time - perhaps a notification of change in price on an interest rate. If this causes a buy/sell of an embedded euro bond on a complex derivative and the message goes missing for 24 hours you could be down 10's of millions of dollars. Andreas, do you have the ability to offer the necessary guarantees? Are they believable enough for a smart banker to trust? One thing that marks out AMQP is that it has been backed by a large investment bank from day one. Banks are not in the habit of doing this unless something is badly wrong with the existing offerings. In part this may be due to commoditisation of messaging (which JMS heralded) and in part due to a lack of functionality that matches the requirements. From my understanding of AMQP both apply. If other vendors provided what was needed then why would a large bank - not a software company - back such an effort and part fund/part encourage implementations? When customers start to create technology standards you really do have to ask what went wrong with the market place. And it is a good idea to engage with customers that do so because at the end of the day they are the consumers of what we do as technologists. Cheers Steve T
  17. "balls"[ Go to top ]

    Andreas, What you have to realise is that these banks turn over billions of dollars every day, they buy tens of millions of pounds of messaging middleware every year, their IT spend is in the billions per year. They CAN NOT depend on some little tiny company to run their messaging middleware, even sonic (a la Progress) is considered a small company and they've got hundreds of millions of dollars in the bank. This is pretty much the ONLY reason banks buy IBM, it's certainly not for the technology, they're not stupid, it's because they know that IBM will still be there in 10 years time, they know that if something goes wrong with a version that was released in 1997 IBM will still support it (in practice it's probably the latest version). It's not balls they lack, it's a problem of scale. If your IT spend is $X billion, your turnover is $X trillion, are you really going to run it all on some tiny little company who's turn over is just a few $million? I used to have problems justifying Borland (in the late 90s), the salesman came in and proudly told us how much money they had in the bank, it was a few hundred million $s, the guy in the bank said he'd already traded that this morning. SwiftMQ is a great product but other than a few niche projects you're not going to make it in banks until you've got over 10,000 employees in 20 countries and turn over a good $billion, Sonic can't do it and SpiritSoft didn't do it. Open source or open protocols is as close as you're ever going to get. JMS is commodity, even the best vendors are too small and that's the sum of it, it's survival of the fittest, you've got a good change. Good luck, -John-
  18. Re: "balls"[ Go to top ]

    This is pretty much the ONLY reason banks buy IBM, it's certainly not for the technology, they're not stupid, it's because they know that IBM will still be there in 10 years time, they know that if something goes wrong with a version that was released in 1997 IBM will still support it (in practice it's probably the latest version).
    Not sure I agree with this statement. If this were true, there would be no BEA, Redhat, and hell Microsoft and Oracle if we go back far enough. These companies were incredibly small while being adopted. Let's not forget that these financials have significant interest in companies like IBM(ie. HUGE market caps). It's not IT evangelism, it's dollars and cents. Holding millions of shares in pensions and corporate coffers trumps any technology consideration in addition to managing large chunks of 401k and the like goes a long way in the decision process for product adoption. JMS failed -- at least from a spec perspective. As with other JCP specs, in the final analysis, the specs get whittled down to the lowest functionality and leave it up to the vendor to value add. This is the main problem with what you've stated in other posts here as to JMS incompatibilities and leaves the client in the awkward position of having to accept less than from a product from a spec basis. It seems a logical step to normalize the wire protocol and not the edge contract API. Plus, if you read the FAQ http://www.amqp.org/tikiwiki/tiki-view_faq.php?faqId=1#q6, there is a definite goal to move the MOM architectures out of just the java space. Java is unfortunately losing it's prestige as an infrastructure play. The disappointment of the "economy of scale" argument by going with a J2EE middleware architecture has bitten many a manager.
  19. Re: "balls"[ Go to top ]

    This is pretty much the ONLY reason banks buy IBM, it's certainly not for the technology, they're not stupid, it's because they know that IBM will still be there in 10 years time, they know that if something goes wrong with a version that was released in 1997 IBM will still support it (in practice it's probably the latest version).


    Not sure I agree with this statement. If this were true, there would be no BEA, Redhat, and hell Microsoft and Oracle if we go back far enough. These companies were incredibly small while being adopted. Let's not forget that these financials have significant interest in companies like IBM(ie. HUGE market caps). It's not IT evangelism, it's dollars and cents. Holding millions of shares in pensions and corporate coffers trumps any technology consideration in addition to managing large chunks of 401k and the like goes a long way in the decision process for product adoption.

    JMS failed -- at least from a spec perspective. As with other JCP specs, in the final analysis, the specs get whittled down to the lowest functionality and leave it up to the vendor to value add. This is the main problem with what you've stated in other posts here as to JMS incompatibilities and leaves the client in the awkward position of having to accept less than from a product from a spec basis. It seems a logical step to normalize the wire protocol and not the edge contract API.

    Plus, if you read the FAQ http://www.amqp.org/tikiwiki/tiki-view_faq.php?faqId=1#q6, there is a definite goal to move the MOM architectures out of just the java space. Java is unfortunately losing it's prestige as an infrastructure play. The disappointment of the "economy of scale" argument by going with a J2EE middleware architecture has bitten many a manager.
    It is one thing to bet the bank on some new technology and another to use some new technology within a niche in the bank. Banks (investment banks) have a long history of trying stuff out. This is how SpiritSoft came into being as the very first JMS vendor. Indeed it was an architect from a major bank that suggested JMS to SpiritSoft (when it was still Push Technologies). I seem to remember Instinet pioneering J2EE early on. My experience of banks is thus the same as John's. They do try stuff but at the fringe. They do not bet the bank on new stuff. The fact that it is banks that came up with AMQP does suggest - as I mentiioned before - that MOM vendors simply have not provided them with what they have needed over the years. Did JMS really fail? I think that is a harsh judgement. I would rather see it as one point on a continuum. It is clearly not the end point. What specs can anyone point to that are not some form of lowest common denominator. All of the standards work that I have been involved in has been similar because standards are about consensus amongst many competing parties. Even WS-CDL is a compromise. And WS-BPEL clearly is. So JMS was bound to be. The lack of any real MOM based standard outside the Java world has not helped at all. But then again Java has lost its way. If only we had just used serialised Java as the on the wire format then ..... utopia ;-) cheers Steve T
  20. Re: "balls"[ Go to top ]

    Did JMS really fail? I think that is a harsh judgement. I would rather see it as one point on a continuum. It is clearly not the end point.
    Ok, fair enough. Failure may have been too harsh. I guess JMS suffers from "diminshing expectations". I had a recent meeting with a Wall Street client working on a trade processing enhancement. When I brought up JMS , their inhouse messaging architect nearly had a stroke. 'Why would I limit myself to JMS when I can use low level constructs to optimize?'. And you know, he had a point. He purchased their MOM product for it's feature set not a compromised spec compliance.
    What specs can anyone point to that are not some form of lowest common denominator. All of the standards work that I have been involved in has been similar because standards are about consensus amongst many competing parties. Even WS-CDL is a compromise. And WS-BPEL clearly is. So JMS was bound to be.
    But this begs the question as to whether standards bring any value if they are a lowest common denominator(LCD) -- especially for the client. If you are going to go the LCD route, then I think the AMQP group has it right. Do it at the wire level, not at the edge.
    The lack of any real MOM based standard outside the Java world has not helped at all. But then again Java has lost its way. If only we had just used serialised Java as the on the wire format then ..... utopia ;-)
    Depends on which serialization spec you use ;)
  21. Re: "balls"[ Go to top ]

    Andreas,

    What you have to realise is that these banks turn over billions of dollars every day, they buy tens of millions of pounds of messaging middleware every year, their IT spend is in the billions per year. They CAN NOT depend on some little tiny company to run their messaging middleware, even sonic (a la Progress) is considered a small company and they've got hundreds of millions of dollars in the bank.

    This is pretty much the ONLY reason banks buy IBM, it's certainly not for the technology, they're not stupid, it's because they know that IBM will still be there in 10 years time, they know that if something goes wrong with a version that was released in 1997 IBM will still support it (in practice it's probably the latest version).
    John, I know these arguments and I understand them. But - the reason why AMQP was created is to get out of the chokes of these 2 companies, right? At least that is what I read as a reason. AMQP should allow competition (like JMS on wire level) but if you (banks; AMQP is banking only) don't buy from competitors implementing AMQP, this makes not much sense to talk any further with prospective vendors whose name is different from IBM and Tibco. Actually I agree with James and do not see any technical advantage in AMQP, except language interop. There is a good reason in JMS to have a border between admin and client and to prohibit JMS clients to create resources like queues at their will. No need to define broker semantics in the wire protocol. -- Andreas
  22. Balls![ Go to top ]

    Andreas, you state that AMQP is "banking only". That's certainly nonsense - it's a technical standard for messaging which is applicable in any industry. Sure, it's a good fit for finance, but it's also a good fit for telecoms, retailers, pharma... pretty much anybody who needs to send messages. Now, lets play a little game. Imagine if you will that HTTP does not exist. Instead, there's an API (called, say, Java Hypertext Service or JHS) which allows browsers written in Java to request HTML documents from hypertext servers. JHS is a standard API, but the JHS drivers must be provided by the vendors of the "hypertext servers", because they talk to the servers using a proprietary protocol. JHS drivers from one vendor don't work with other vendor's servers, and of course you have to buy all your proxies, firewalls etc from the same vendor as well. Last but not least, browsers written in non-Java languages can't use JHS at all, they have to use a proprietary API, if one is even provided for their language. Given this environment, do you think the Web would exist today? Or would it be an expensive luxury, dominated by a few large vendors, and only affordable by the richest investment banks? My apologies for going down the route of argument-by-analogy. It's just extraordinary to me that there is this one major area of enterprise architecture where people are quite happy for the market to be balkanised by vendors seeking lock-in. It simply wouldn't be tolerated at other layers of the stack. AMQP is very far from "banking only". If successful, then through commoditization it will make messaging an affordable proposition to businesses that don't have the wide profit margins of a global bank.
  23. Re: Balls![ Go to top ]

    It's just extraordinary to me that there is this one major area of enterprise architecture where people are quite happy for the market to be balkanised by vendors seeking lock-in. It simply wouldn't be tolerated at other layers of the stack.
    Fundamentally, AMQP is a great idea but the frustrating thing for me is that it will only be adopted by new messaging implementations because it has taken the route of defining Broker semantics as well as on-the-wire format. I'd be happy to help work on a lighter version of the spec that would be easier to adopt by existing messaging implementations, including ActiveMQ ;)

    Rob Davies
    LogicBlaze
    Open Source SOA
  24. Re: Balls![ Go to top ]

    I'd be happy to help work on a lighter version of the spec that would be easier to adopt by existing messaging implementations, including ActiveMQ ;)
    Hmm that's an interesting thought. If ActiveMQ were to use the same encoding, framing and flow control protocol, but leave most of the command-style classes and methods unimplemented, then it could probably still interoperate to a high degree with a "full" AMQP server. The question would be whether you could legally claim to have that interoperability.
  25. Re: Balls![ Go to top ]

    Neil, if you would concentrate AMQP on a more basic level of message exchange without defining unnecessary broker semantics, file transfer etc in the protocol, without your aggressive marketing attitudes against existing vendors or standards, with a clear direction, a TCK etc... that would be a way to go (and would fit with your analogy above). It would be a good interop wire format approach and I'm sure you'd see many established vendors supporting it. Your approach is wrong, your attitude is wrong, your protocol is wrong, your expectations are wrong. -- Andreas
  26. Re: Balls![ Go to top ]

    Your approach is wrong, your attitude is wrong, your protocol is wrong, your expectations are wrong.
    Actually I have nothing to do with AMQP or RabbitMQ, I'm just an enthusiastic observer. As for whether it's all wrong or at least partially right, lets see what the market does.
  27. calm down[ Go to top ]

    Guys Please don't tilt at windmills or each other, the stakes just aren't high enough. AMQP will succeed if and only if it is not finance only. That's why the role of people like Cisco and Red Hat in the WG is important. Speaking as a vendor, it's why we teamed up with some guys who know their way around telco architectures really well. As for defining the semantics of the broker. The whole point is to define behavioural semantics at protocol level, otherwise it would be a format and not a protocol. Consider e.g. TCP, or SCTP. More important is to do this tightly enough to play a consistent role in the architecture (eg for the web as Neil hinted with his analogy), and yet loosely enough to allow vendors to make money by competing on different propositions. Feel free to disagree with the idea that semantics have a role to play at wire level. We can have an interesting debate about that, e.g. what Guaranteed Delivery means. @Andreas, you have made yourself very clear, thanks. alexis
  28. congrats to the RabbitMQ team[ Go to top ]

    I think it is great to see another implementation of AMQP - congratulations to the RabbitMQ team. Regarding the many questions about AMQP, the official AMQP site http://www.amqp.org has an FAQ section that answers many common questions and also a user section for anyone who would like to provide feedback on the specification. AMQP was not created just for banking, although the Chairperson of the AMQP Working Group is John O'Hara,a Distinguished Engineer at JPMorganChase Bank. You can hear John speak about AMQP on a recorded webcast here http://www.ebizq.net/webinars/7440.html. Regarding John Davies comments about IBM and Tibco having a large percentage of the market, market share estimates are typically based on percentage of overall revenue not percentage of overall number of implementations or users - so while these two companies are taking in the dominant share of the revenue in the message oriented middleware market segment it doesn't mean that other vendors products are not being used extensively by users. Open source software in particular is going to force a change in the way we look at market share etc. since the market models often focus on license only and not ongoing support/maintenance charges. Debbie
  29. AMQP as a necessary utility..[ Go to top ]

    Hi, allow me to introduce myself as the chap who started AMQP (but it's now the work of dozens of talented people across several firms). John D, Steve R-T, and Neil B all appear to have a good understanding of why AMQP exists and what it is; I encourage you to read their posts above if you've not already done so. For a specification which was only made public in June last year, I think AMQP has come a long way quickly. The firms involved are serious and committed about this, there is a steady momentum, strong backing and a very real need for this in the user community. AMQP was created because middleware is a "necessary utility" (like roads and bridges) that no one company should control or monopolise. It seems obvious in that sense, it's a proven pattern we see every day in society. AMQP was also a reaction to the islands of automation problem, the lack of required functionality in many well known products, and the vendor risk implied by either too many incompatible small vendors or too few incompatible large ones. If AMQP fulfills its vision, every endpoint on every network will be equipped with AMQP middleware (either client or broker). Imagine what new kinds of applications can be built on that assumption? Perhaps one will help solve Global Warming (JD). It sounds unlikely, but I know someone is actually looking at applying AMQP in that field. I think it is a significant milestone that RabbitMQ is the first implementation written just from the published specification with no interaction with the AMQP Working Group. I congratulate them on that. Finally, the exciting thing about this vision is not building the infrastructure, though it is vital and given the scale of the vision it can be lucrative. The exciting part is the applications that come next and ride on top of that infrastructure. John JPMorgan Chair of the AMQP Working Group See the AMQP FAQ on http://www.amqp.org/tikiwiki/tiki-view_faq.php?faqId=1 ; it addresses many common misconceptions.
  30. Re: AMQP as a necessary utility..[ Go to top ]

    AMQP was created because middleware is a "necessary utility" (like roads and bridges) that no one company should control or monopolise.
    Yeah - great. We all get it John - I just don't know why you insist we have rip up all the existing roads and bridges to make this 'vision thing' happen ?
  31. Re: AMQP as a necessary utility..[ Go to top ]

    Q: Which wire level protocol should we have started from? Q: Which common semantic model can we assume is presented by 'most' middleware? I would be interested in the answer. Thanks John
  32. Re: AMQP as a necessary utility..[ Go to top ]

    Q: Which wire level protocol should we have started from?
    Creating one is fine, I certainly won't argue against that :-)
    Q: Which common semantic model can we assume is presented by 'most' middleware?
    JMS -- they have already fought the semantic interoperability battle and done a good job of getting folks on the same page. -Brian
  33. Re: AMQP as a necessary utility..[ Go to top ]

    Q: Which common semantic model can we assume is presented by 'most' middleware?

    JMS -- they have already fought the semantic interoperability battle and done a good job of getting folks on the same page
    JMS has certainly gone a long way to get messaging providers on the same page but defines only a core set of semantics, there are lots of optimisation options and additional semantics that providers add in order to make their product both better and to differentiate them from competitors. Look at all the additional stuff in ActiveMQ, its way way more than the specification alone and no one can deny that that is a good and necessary thing. I think its time to deconstruct and specify the MOM stack down to the wire, imagine what can be done when we have network hardware and services that understand the wire level protocol. Now its gets interesting! The protocol is just the start, the really interesting stuff is what we can build on it - us - the users. As an aside, JMS was always lacking in any specified management and monitoring. When working with HermesJMS I've had to deal with 10 different ways of getting monitoring statistics (even just the basic queue depth) not to mention any management functionality such as administering queues and so on. JMX, PCF and custom APIs that make creating cross MOM tooling a headache. This is basic stuff IMO. I can create a table in a relational database and query it with largely the same syntax anywhere. If we can at least agree on the same way to do similar admin work with messaging then we're moving forward in a big way. In the world of a standard protocol but many products a standard way to administer and monitor the platform becomes increasingly important. AMQP looks to me like the only candidate I've seen so far that can go there. I think the AMQP route is the logical progression from just an API and although its true that the semantic gap with existing providers is certainly there, I doubt its as insurmountable as some may think. Finally, as a consumer of messaging, having it user driven via a working group with the requirement of a market that creates mutliple implementation that must conform to a TCK is only a good thing. Of course, a TCK is an essential deliverable of 2007 from the AMQP folk as they themselves I am sure would agree. Lets hope AMQP does not go the way of BEEP for all our sakes. I'm had enough of middleware pain and am optimistic that AMQP is a stepping stone out of it. Regards, Colin. http://hermesjms.com
  34. Re: AMQP as a necessary utility..[ Go to top ]

    Colin, Excellent post, it's great to hear from a vendor/consumer of JMS with vision, you're spot on with your observations. It's the comments about AMQP in hardware and being able to "understand" the protocol in hardware that are really interesting for the future. It doesn't need much imagination to guess why Cicso are seriously into AMQP and they're not the only hardware/CPU manufacturer working on it. It would be great to see something in HermesJMS, if there's anything the AMQP community can do to help I'm sure they will, please contact John O'Hara or myself for further introductions. Early adoption of AMQP will drive early adoption of your product, you will be seen as future safe. -John-
  35. Re: AMQP as a necessary utility..[ Go to top ]

    AMQP was created because middleware is a "necessary utility" (like roads and bridges) that no one company should control or monopolise.


    Yeah - great. We all get it John - I just don't know why you insist we have rip up all the existing roads and bridges to make this 'vision thing' happen ?
    I think this idea that is postulated of having to rip up the existing roads and bridges is very much a FUD issue. JMS for my money was a key step along the way. I wish we had done AMQP at that stage (so back in 1997/1998). But we live in a world in which we evolve things. I do not buy for one minute that AMQP requires all bridges and roads to be ripped up. It might require a new lane to be added. And yes it is painful but lets get real. This is what happens and it is part of the innovation curve that drives new technology. As I pointed out this is coming from users not from vendors. There are many other initiatives that many of you are involved in that have come from vendors with no user having ever asked for it. So when we talk about customer-lead that is at best no more than political subterfuge and at worst a poor understanding of how markets and innovation work. So once more congratulations onthe RabbitMQ team for showing the spec is implementable. If they can do it I suspect more can do it too. I would be interested to know how long it took to write it. Perhaps then we shall undertsand how long our extra lane on the highway needs to be be ;-) Cheers Steve T
  36. Time to develop RabbitMQ[ Go to top ]

    So once more congratulations onthe RabbitMQ team for showing the spec is implementable. If they can do it I suspect more can do it too. I would be interested to know how long it took to write it. Perhaps then we shall undertsand how long our extra lane on the highway needs to be be ;-)
    I believe it was on the order of around 6 to 8 man-months. Alexis may be able to give a more accurate number. Admittedly it was developed by the enormous brains at LShift, and one of their man-months is worth three of anybody else's. But still I think it shows the productivity that they achieved through using a language (Erlang) which was designed for exactly this kind of task. The team developing Apache QPid in Java is much bigger. I'm surprised that most of the discussion on this thread hasn't been about that language choice. This is a Java forum after all, shouldn't it be at least a little controversial to see a front-page post about a product that has shunned Java as an implementation language? Java is over ten years old now, why is it still not the optimal choice for developing high performance messaging servers? Perhaps it's something to do with Java's approach to concurrency, based on locking over shared memory. This barely works today, and will not work at all as CPUs get ever more cores.
  37. Re: Time to develop RabbitMQ[ Go to top ]

    I would be interested to know how long it took to write it.
    I believe it was on the order of around 6 to 8 man-months.
    We started development last summer, with a small team of 1-3 people. Our first prototype implementing the core AMQP functionality was ready in less than 8 weeks. Since then we have gone through two architectural revisions, implemented many more of the AMQP features (see RabbitMQ Compatibility for the current status), carried out lots of testing, tuning and QA, implemented a Java client API and some example apps, worked on packaging, wrote documentation, etc. All in, this took about 8 person months.
    Admittedly it was developed by the enormous brains at LShift, and one of their man-months is worth three of anybody else's. But still I think it shows the productivity that they achieved through using a language (Erlang) which was designed for exactly this kind of task.
    Basing RabbitMQ on Erlang and OTP certainly was a key factor in speeding up development. Erlang/OTP were designed for implementing highly scalable and reliable systems with messaging at their core, and thus forms an ideal foundation for an AMQP implementation. As a result the entire RabbitMQ server code base weighs in at less than 7,000 lines, and we can add new features and experiment with different implementation strategies with ease.
  38. So 8 L-shift man months. Given the suggestion that it is costly to change in this thread already it suggests that it is not so very very hard and costly to do. SpiritWAVE took a huge amount of time and a huge amount of money before it was anywhere near for production use. I'd suggest, assuming wide spread adoption is to follow, pretty well a slam dunk. And about time too. Erlang makes it even more technically interesting. I'd like to hear more and who cares if it is on the TheServerSide. It's just good stuff and highly educational. Cheers Steve T
  39. SpiritSoft[ Go to top ]

    SpiritWAVE took a huge amount of time and a huge amount of money before it was anywhere near for production use.

    Cheers

    Steve T
    Yes but you had "SpiritSoft man months", worth only a normal many week. :-) :-) <-- Just in case you missed the first one! -John-
  40. Re: SpiritSoft[ Go to top ]

    Actually most of the cost was Steve's bar bill :)
  41. With an open API - namely JMS - you are not at the mercy of any JMS vendor either which has been the case for the last 9 or 10 years. The only real difference with AMQP is if you are not using Java.
    So James, I've got a line of business with several applications using SonicMQ (one of the best JMSs), I have a Sonic hub and several Java applications using JMS 1.1. I am asked to integrate two new applications from another department, one using the wonderful ActiveMQ and the other using SwiftMQ. Of course they're using Java too but use a different JMS clients and different hubs. They are all pure JMS 1.1, nothing outside of the spec. Will the ActiveMQ or SwiftMQ hubs talk to my SonicMQ hub? Will they heck, you have to either start using JMS bridges which adds latency, risk and another point of failure or change the factory lookup for the JMS and port everything from one JMS to another. You have to do this EVERY time you move from one JMS to another, surely it's better to find a common standard? If ActiveMQ implemented AMQP under the JMS API then you'd be fully interoperable and no doubt promoting it, you started off with a view to implementing it (here but seem to have given up since, what happened? -John-
  42. With an open API - namely JMS - you are not at the mercy of any JMS vendor either which has been the case for the last 9 or 10 years. The only real difference with AMQP is if you are not using Java.

    So James, I've got a line of business with several applications using SonicMQ (one of the best JMSs), I have a Sonic hub and several Java applications using JMS 1.1. I am asked to integrate two new applications from another department, one using the wonderful ActiveMQ and the other using SwiftMQ. Of course they're using Java too but use a different JMS clients and different hubs. They are all pure JMS 1.1, nothing outside of the spec. Will the ActiveMQ or SwiftMQ hubs talk to my SonicMQ hub?
    Sure!. Where've you been John? Pretty much all JMS providers, J2EE containers and ESBs come with bridges to connect JMS brokers together.
    Will they heck, you have to either start using JMS bridges which adds latency, risk and another point of failure
    Erm. Lets start with latency first. The latency of bridging from one message broker to another is more an implementation detail than something intrinsic in the particular API or protocol being used. Unless both brokers reside in the same process, you'll typically be communicating between the two using some kind of socket irrespective of what solution you pick - AMQP or JMS API. Sure if you use a remote bridge of any kind (such that its a network hop to both brokers) then sure, you'll have extra latency. You'll also have extra latency depending on your OS/JVM and threading model too. Though if you are using pure Java message brokers you can host the bridge inside one of the brokers to reduce latency (to one network hop only). Or you could host the brokers on the same box, or even the same JVM to further reduce latency of bridging them. Either way, its got absolutely nothing to do with what protocol is used for the communication or whether you use the JMS API or not. Now lets look at risk and another point of failure. For a good JMS provider the bridge can be hosted inside the broker, so there's no additional point of failure or extra network hop & latency. The risk of things working comes down to vendors complying properly with the JMS spec & TCK - which is pretty rigorous and has been around for a while. The risk of the AMQP approach is providers not complying with the AMQP spec and (whenever it arrives) its TCK. So about the same - albeit AMQP is much much newer so I'd say there's way more risk today with that approach.
    or change the factory lookup for the JMS and port everything from one JMS to another.
    Most decent JMS providers just use a URL to connect to - which you'd need anyway with an AMQP bridge. So really there's not that much difference is there?
    You have to do this EVERY time you move from one JMS to another, surely it's better to find a common standard?
    Hey I'm not dissing AMQP - please re-read everything I've said in this thread. If AMQP gets completed and, most importantly, gets adopted and implemented properly by all the messaging vendors then yes it will be useful. I'm just saying that most of the supposed benefits of AMQP listed so far in this thread (interop and avoiding vendor lockin) can be realised right now today, with all the current messaging providers (and AMQP providers) using the JMS API on the Java platform. The hole that AMQP fills is dealing with vendor lockin on non-Java platforms as I've already said in this thread.


    If ActiveMQ implemented AMQP under the JMS API then you'd be fully interoperable and no doubt promoting it, you started off with a view to implementing it (here but seem to have given up since, what happened?

    -John-
    We are driven by customer demand. So far we've yet to have a single customer or user who've expressed any interest in AMQP support. Who knows, that could change one day but right now its bottom of our TODO list I'm afraid. Don't get me wrong - I wish AMQP well & hope it doesn't end up being another BEEP. Making a good protocol is hard enough, but getting it to a critical mass and widespread adoption is really hard - so fingers crossed it all works out. (Widespread protocol adoption seems to be more about luck than design it seems). But when folks are promoting AMQP, lets go easy on the bogus marketing fluff and complete JMS FUD shall we? If AMQP is to succeed, you probably want to try get the various messaging vendors on your side to adopt it, so I'd recommend you go easy on the silly JMS bashing. James LogicBlaze Open Source SOA
  43. JMS does nothing to encourage interoperability between messaging products, because it doesn't say anything about the wire protocol that they have to speak.


    Neither does AMQP either yet as you can't use AMQP to talk to MQSeries, TibCo, BEA, Sonic, SwiftMQ, Sun etc. So neither the 95% nor the 5% of current commercial messaging middleware products.

    Sure one day IBM, TibCo, Sonic, BEA, Sun et al may decide to support AMQP. Until then, the best way to bridge different MOM providers is to actually use the JMS API :).

    James
    LogicBlaze
    Open Source SOA

    Of course the other way to do it is to use the JMS API's provided by the current AMQP implementations and use that as the basis to integrate with those that do not interoperate. In so doing you isolate those that do not want to talk together from those that do and exert pressure on the market to change it's proprietary practices.

    I should have added that the other way to go, which seems to be espoused, has a logical end game of everyone gives up now and just uses MQSeries. Then we don't have any interop problems at all.

    I think congratulations are in order because at least some folks are trying to solve the interop issues.
    Hey I'm all for innovation; let a thousand flowers bloom and all that. I'm just sticking up for the JMS API a little as some AMQP folks tend to knock it. (e.g. the silly 5% figure used in this thread to try make it sound like JMS is not relevant). BTW the proprietary vendors have bowed to market pressure & they all pretty much support the JMS API standard (even TibCo!).
    Better to congratulate than to say it will never work or even to state the obvious that it is early days on the adoption curve.
    Erm, I didn't say it would never work. I just said that if you want to interoperate, today, among the messaging providers people are actually using right now - the best way to interop is via the JMS API. This doesn't look like changing any time soon. But who knows, in a few years time anything might happen. The current sweet spot of AMQP doesn't seem to be about interop between existing messaging providers (at least until IBM, TibCo, BEA, Sun, Sonic et al get on the bus). Afterall would a customer really notice if a JMS bridge between MOM providers was swapped with an AMQP bridge? Its more that AMQP is a way of providing reusable client libraries in different languages to Java for those newly written messaging providers which do support AMQP. James LogicBlaze Open Source SOA
  44. I think something is being missed here...and that is, the power of a standard. Not to say that AMQP is anywhere near as widely accepted as, say, HTTP, but it's at least a step in that direction. Web services standards like WS-RM, etc. are being developed at the message level (the wrong place, IMHO) simply because of the lack of standardization in messaging protocols.
  45. hear hear..[ Go to top ]

    Back a bit to the fundamentals... I would like to be able to pick an open, high performance, secure and production-proven queue infrastructure... Can I do this now? Well, yes if I have a lot of money! Will these interoperate? Well, no unless you write some bridges... So the problems highlighted in interoperability at protocol level already exist, AMQP does not solve it but offers an open standard. So, when I see a standard proposed that fills a gap in the array of protocols and when I see quite a lot of activity and already several implementation by people who are not new to write successful software (Apache anyone?), then I think that we're onto something worth keeping an eye on... In my view a very robust TCK and a control on the use of the AMQP name (only if your implementation passes the TCK) are absolutely critical. Good luck to AMPQ and congratulations to RabbitMQ team. Benoit
  46. Andreas, you're a bit confused. JMS takes nearly 100% of the market for messaging in Java, it's true. But JMS is only an API. You use JMS to talk to a broker like MQ Series, Tibco etc.
    Ahh, Neil, thanks for putting me back on track! ;-)
    The 5% figure that John refers to is for the other messaging vendors besides IBM and Tibco... eg SpiritSoft, Sonic, SwiftMQ, etc.
    The 5% is misleading and wrong! It's a number from 1998! ;-) -- Andreas
  47. rabbitmq and jms: client and server[ Go to top ]

    i'm quite excited by AMQP in general and RabbitMQ in particular, but there are actually quite a few questions around interoperability between AMQP (and, in particular, RabbitMQ) and JMS. firstly, it seems to me that mapping AMQP to JMS is actually not so straightforward. in particular, the AMQP concept of an "exchange" (which is the primary "destination" where AMQP clients send messages) does not exist in JMS. further questions are: - does RabbitMQ offer a JMS client implementation? - how should a JMS client send messages to an exchange? should the "exchange" be the third sub-class of the JMS Destination (besides Queue and Topic)? - does the RabbitMQ server integrate as a JMS provider into appservers? the standard way to do this would be via the Connector Architecture... this would allow MDBs to listen on AMQP queus, for instance. - related is the question if RabbitMQ is a XA-resource. - AMQP explicitly allows clients to create exchanges, queues and bindings (which connect a queue to an exchange). surely this does not map to the JMS API... kind regards, gerald http://www.gerald-loeffler.net
  48. AMQP explicitly allows clients to create exchanges, queues and bindings (which connect a queue to an exchange). surely this does not map to the JMS API...
    From my perspective, this is the big problem with AMQP - its gone beyond specifying a wire-level interoperability format like STOMP - hence making it difficult for existing messaging vendors to adopt without re-writing their existing brokers anyway. So you have to ask - what's the point on an interoperability specification which will probably never work with 99.9% of message brokers ?
  49. other messaging standards[ Go to top ]

    Rob You are right to identify format and standards proliferation as an issue. On the one hand there seem to be many new standards and formats, some but not all driven by needs. On the other hand some standards get picked up by major players and backed, because of some business driver such as enabling hardware based solutions with a view to cost reduction, or there are attempts to unify via governance-type standards, such as Unifi (ISO 20022) driven by SEPA. I think a really interesting question here is what the FIX guys do, since FIX is very widely used and its use is growing. Cheers alexis
  50. RabbitMQ and JMS and XA[ Go to top ]

    Gerald, Thank-you for your questions. Currently RabbitMQ has a Java client: http://www.rabbitmq.com/api-guide.html You are right that the AMQP - JMS mapping is needed. This is being addressed right now by the AMQP Working Group. They have done some very good work which will become available in due course. The RabbitMQ release is tracking the spec; we are using various JMS adaptors internally and plan to include them in the future. The same is true for XA. We also anticipate that Java users may wish to use RabbitMQ through Mule, e.g. for some integration scenarios. Please do take a look at our FAQ: http://www.rabbitmq.com/faq.html And at our roadmap: http://www.rabbitmq.com/roadmap.html And ... please do try RabbitMQ out and let us know how you get on! alexis
  51. Hi Gerald, You're right there is a gap between what AMQP provides and what JMS gives you access to. For example using the JMS API there is no way to programmatically create a new Topic or Queue (except a temporary one). Therefore in order to perform administration-type tasks like this you have to use the vendor's proprietary tools. I believe that the AMQP group regarded this as a barrier to commoditization, so they built a standard set of commands to control the broker from the client. So there is some functionality of AMQP which can never be accessed from JMS, but I think it's valid just to leave it out. After all, that functionality isn't accessible to clients today without dropping down to a product-specific API provided by the vendor. I think therefore that we will see, in addition to the JMS "wrapper", an AMQP-specific library for Java that can be used to access the full range of commands when needed. By the way, there is a working group going on within AMQP to figure out the best way of mapping the JMS API onto the facilities of AMQP. Also there is a JMS provider under development at QPid, so you should be able to use that with RabbitMQ or any other AMQP broker implementation. RabbitMQ does provide a Java client but it is non-JMS, AMQP-specific. Neil